Debian Bug report logs - #666772
apt cross-build-dep handling should be liberal with Arch: all packages

Reply or subscribe to this bug.

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Steve Langasek <vorlon@debian.org>

To: submit@bugs.debian.org

Subject: apt cross-build-dep handling should be liberal with Arch: all packages

Date: Sun, 1 Apr 2012 09:48:23 -0700

[Message part 1 (text/plain, inline)]

Package: apt
Version: 0.8.16~exp12
Severity: normal
Tags: patch
User: multiarch-devel@lists.alioth.debian.org
Usertags: multiarch

Hey guys,

While working on getting the Ubuntu base system's cross-build-dependencies
satisfiable using only apt-get build-dep -a $arch, I came to the realization
that the only Architecture: all package I'd seen being used as a
build-dependency that *couldn't* be marked Multi-Arch: foreign was
python-dev - and we had separately determined that python-dev needs to be
made an Architecture: any package precisely *because* other packages needed
to build-depend on the host version of python2.7-dev for cross-building.

And of course, unlike in the case of binary deps, there's no way that the
new build-dep behavior would accidentally pull in the wrong dependencies and
break a user's system on upgrade.  In both cases, an Architecture: all
package that masks a transitive arch-dependent dependency is *wrong*; but
whereas in the binary dep case we have to be very conservative in treating
that package the same way previous package managers would have (i.e.,
treating it like a native package), cross-build-dep handling is an entirely
new feature with no upgrade considerations - so it should be entirely
reasonable to treat all Arch: all packages as implicitly M-A: foreign when
satisfying cross-build-deps.


In support of this argument I'm attaching two bits of evidence:

 - arch-indep-build-deps.sh, that recurses through all the build-deps in the
   specified distro (here, Ubuntu) to generate a list of Arch: all packages
   that are used as build-dependencies.
 - arch-all-build-dep-list.txt, the sorted output of the above script.

This shows the scope of the problem; up to 1000 Arch: all packages that
might need to be marked M-A: foreign without this change, which may not need
any metadata updates with this change.  (Of course, this script doesn't
filter out build-deps of source packages that only build Architecture: all
binaries, so the actual number of packages affected may be somewhat lower.)


Finally, I'm attaching a patch that I believe implements the desired
behavior.  I've tested this locally, and it does the right thing in my
tests.  Example:

Before:

$ sudo apt-get build-dep -a armhf htmldoc
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'libjpeg-turbo8-dev:armhf' instead of 'libjpeg-dev:armhf'
Note, selecting 'libpng12-dev' instead of 'libpng-dev'
E: Build-Depends dependency for htmldoc cannot be satisfied because the package autoconf2.59 cannot be found
$

After:

$ sudo apt-get build-dep -a armhf htmldoc
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'libpng12-dev' instead of 'libpng-dev'
The following NEW packages will be installed:
  autoconf2.59 libdrm2:armhf libexpat1:armhf libfltk1.1:armhf
  libfltk1.1-dev:armhf libfontconfig1:armhf libfreetype6:armhf
  libgl1-mesa-glx:armhf libglapi-mesa:armhf libjpeg-dev libjpeg-turbo8:armhf
  libjpeg-turbo8-dev libjpeg8:armhf libjpeg8-dev libpng12-0:armhf libpng12-dev
  libssl-dev:armhf libssl1.0.0:armhf libx11-xcb1:armhf libxcb-glx0:armhf
  libxdamage1:armhf libxfixes3:armhf libxft2:armhf libxinerama1:armhf
  libxrender1:armhf libxxf86vm1:armhf zlib1g:armhf zlib1g-dev
  zlib1g-dev:armhf
The following packages will be upgraded:
  libssl1.0.0
1 upgraded, 29 newly installed, 0 to remove and 33 not upgraded.
Need to get 2134 kB/6199 kB of archives.
After this operation, 15.6 MB of additional disk space will be used.
Do you want to continue [Y/n]? ^C
$

Of note here is that libjpeg-dev is an Arch: all transitional package
depending on an Arch: any libjpeg8-dev, so apt doesn't get the answer
completely right - but as mentioned, the only way to fix that case is by
making libjpeg-dev not be Arch: all.

Thanks,
-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek@ubuntu.com                                     vorlon@debian.org

[arch-indep-build-deps.sh (application/x-sh, attachment)]

[arch-all-build-dep-list.txt (text/plain, attachment)]

[build-deps-all-like-multiarch-foreign (text/plain, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #10 received at 666772@bugs.debian.org (full text, mbox, reply):

From: Steve Langasek <vorlon@debian.org>

To: 666772@bugs.debian.org

Subject: Re: apt cross-build-dep handling should be liberal with Arch: all packages

Date: Thu, 5 Apr 2012 21:26:40 -0700

[Message part 1 (text/plain, inline)]

Refined patch, now with test cases included.

This patch has been uploaded to Ubuntu for precise.

Thanks,
-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek@ubuntu.com                                     vorlon@debian.org

[build-deps-all-like-multiarch-foreign (text/plain, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #15 received at 666772@bugs.debian.org (full text, mbox, reply):

From: Julian Andres Klode <jak@debian.org>

To: Steve Langasek <vorlon@debian.org>, 666772@bugs.debian.org

Subject: Re: Bug#666772: apt cross-build-dep handling should be liberal with Arch: all packages

Date: Sat, 7 Apr 2012 11:38:19 +0200

[Message part 1 (text/plain, inline)]

On Thu, Apr 05, 2012 at 09:26:40PM -0700, Steve Langasek wrote:
> Refined patch, now with test cases included.
> 
> This patch has been uploaded to Ubuntu for precise.

Have you talked to the dpkg developers about this? There is at
least dpkg-checkbuilddeps which checks build dependencies, and
I think dpkg also does build-time dependency handling in some
other tools. And as dpkg is one level lower than APT, I think
they should be asked first before doing something like this.

-- 
Julian Andres Klode  - Debian Developer, Ubuntu Member

See http://wiki.debian.org/JulianAndresKlode and http://jak-linux.org/.

[Message part 2 (application/pgp-signature, inline)]

Message #20 received at 666772@bugs.debian.org (full text, mbox, reply):

From: Nick Leverton <nick@leverton.org>

To: Julian Andres Klode <jak@debian.org>, 666772@bugs.debian.org

Subject: Re: Bug#666772: apt cross-build-dep handling should be liberal with Arch: all packages

Date: Fri, 21 Nov 2014 00:50:54 +0000

[Message part 1 (text/plain, inline)]

Hi,

TL;DR  Dpkg does the useful thing, apt-get build-dep doesn't.

I've just been getting a setup working for Debian cross-building.
At work I use OpenEmbedded which is a time consuming pain in
the neck.  I hope to get to a work flow that can replace crappy
embedded vendor toolchains but for the moment I'm following Wookie's
recipe at https://wiki.debian.org/CrossToolchains which complements
his talk at Cambridge mini-debconf.  To the extent that there is any
maintainer-supported user-level cross building support in Debian, this
seems to be it from my reading.

However this bug is a significant blocker to that work process.
Several of the basic Priority Essential packages won't cross-build
with target != host because of the current bug.

dpkg-checkbuilddeps already allows Arch:all Multi-Arch:none packages to
satisfy build-depends.  I've built apt with the patch here and this allows
"apt-get build-dep" to do the useful thing also.  Both the failures in
the attached log go away with this patch to apt.

Please therefore can this patch be applied and released ?

Many thanks

Nick Leverton

[apt-build-dep.log (text/plain, attachment)]

Message #25 received at 666772@bugs.debian.org (full text, mbox, reply):

From: Nick Leverton <nick@leverton.org>

To: Julian Andres Klode <jak@debian.org>, 666772@bugs.debian.org

Subject: Re: Bug#666772: apt cross-build-dep handling should be liberal with Arch: all packages

Date: Fri, 21 Nov 2014 01:05:48 +0000

On Fri, Nov 21, 2014 at 12:50:54AM +0000, Nick Leverton wrote:
> embedded vendor toolchains but for the moment I'm following Wookie's

Err s/Wookie/Wookey/ *ahem* sorry.

Nick

Message #30 received at 666772@bugs.debian.org (full text, mbox, reply):

From: Johannes Schauer <j.schauer@email.de>

To: Nick Leverton <nick@leverton.org>, 666772@bugs.debian.org, Nick Leverton <nick@leverton.org>, "Julian Andres Klode" <jak@debian.org>, 666772@bugs.debian.org

Subject: Re: Bug#666772: apt cross-build-dep handling should be liberal with Arch: all packages

Date: Fri, 21 Nov 2014 06:58:36 +0100

Hi,

Quoting Nick Leverton (2014-11-21 01:50:54)
> TL;DR  Dpkg does the useful thing, apt-get build-dep doesn't.
> 
> I've just been getting a setup working for Debian cross-building.
> At work I use OpenEmbedded which is a time consuming pain in
> the neck.  I hope to get to a work flow that can replace crappy
> embedded vendor toolchains but for the moment I'm following Wookie's
> recipe at https://wiki.debian.org/CrossToolchains which complements
> his talk at Cambridge mini-debconf.  To the extent that there is any
> maintainer-supported user-level cross building support in Debian, this
> seems to be it from my reading.
> 
> However this bug is a significant blocker to that work process.
> Several of the basic Priority Essential packages won't cross-build
> with target != host because of the current bug.
> 
> dpkg-checkbuilddeps already allows Arch:all Multi-Arch:none packages to
> satisfy build-depends.

I don't think dpkg behaves different than apt here. Can you give the example?

Maybe you are running into dpkg bug #710769?

> I've built apt with the patch here and this allows "apt-get build-dep" to do
> the useful thing also.  Both the failures in the attached log go away with
> this patch to apt.
> 
> Please therefore can this patch be applied and released ?

Treating Architecture:all packages implicitly as Multi-Arch:foreign would be
wrong for many Architecture:all packages. Here two examples of Architecture:all
packages that can NOT be marked Multi-Arch:foreign (without further work):

 - haskell-devscripts #769377
 - dh-ocaml #769379

If apt would treat all Architecture:all packages as Multi-Arch:foreign then
these two packages would have a wrong Multi-Arch annotation. This shows that
marking packages as Multi-Arch:foreign must be done with care and not in an
automated fashion.

cheers, josch

Message #37 received at 666772@bugs.debian.org (full text, mbox, reply):

From: Vasyl Gello <vasek.gello@gmail.com>

To: 666772@bugs.debian.org

Cc: Steve Langasek <vorlon@debian.org>, Nick Leverton <nick@leverton.org>, Julian Andres Klode <jak@debian.org>, Mattia Rizzolo <mattia@debian.org>, Jessica Clarke <jrtc27@jrtc27.com>, Wookey <wookey@debian.org>

Subject: Re: Bug 666772

Date: Thu, 28 Oct 2021 14:10:38 +0000

[Message part 1 (text/plain, inline)]

Dear colleagues,

Nine years after this bug had last activity upon I hit it :)

I was trying to cross-compile Kodi for armhf on amd64 and noticed that apt-get treats arch:all build-dependencies as arch:native and fails to find them.
Furthermore, packages providing the implementation of virtual packages from Build-Depends list (aka "Reverse Provides") are treated the same way.

Since Wookey vetted Steve's apt-get patch I decided to work around this bug in pbuilder-satisfydepends-apt and filed the MR#20 on Salsa:
https://salsa.debian.org/pbuilder-team/pbuilder/-/merge_requests/20

I tested that one building Kodi, its addons and several other random packages.
-- 
Vasyl Gello
==================================================
Certified SolidWorks Expert

Mob.:+380 (98) 465 66 77

E-Mail: vasek.gello@gmail.com

Skype: vasek.gello
==================================================
호랑이는 죽어서 가죽을 남기고 사람은 죽어서 이름을 남긴다

[Message part 2 (text/html, inline)]

Message #40 received at 666772@bugs.debian.org (full text, mbox, reply):

From: Johannes Schauer Marin Rodrigues <josch@debian.org>

To: 666772@bugs.debian.org, Vasyl Gello <vasek.gello@gmail.com>

Cc: Steve Langasek <vorlon@debian.org>, Nick Leverton <nick@leverton.org>, Julian Andres Klode <jak@debian.org>, Mattia Rizzolo <mattia@debian.org>, Jessica Clarke <jrtc27@jrtc27.com>, Wookey <wookey@debian.org>

Subject: Re: Bug#666772: Bug 666772

Date: Thu, 28 Oct 2021 17:22:27 +0200

[Message part 1 (text/plain, inline)]

Quoting Vasyl Gello (2021-10-28 16:10:38)
> I was trying to cross-compile Kodi for armhf on amd64 and noticed that
> apt-get treats arch:all build-dependencies as arch:native and fails to find
> them.

Good. That behaviour is by design and not a bug.

> Furthermore, packages providing the implementation of virtual packages from
> Build-Depends list (aka "Reverse Provides") are treated the same way.

Good.

> Since Wookey vetted Steve's apt-get patch I decided to work around this bug
> in pbuilder-satisfydepends-apt and filed the MR#20 on Salsa:
> https://salsa.debian.org/pbuilder-team/pbuilder/-/merge_requests/20

This is not a bug. Architecture:all packages are *not* implicitly M-A:foreign.
See my other mail to this bug. If an Architecture:all package should be made
M-A:foreign, then file a bug against that package. The maintainer can then
decide whether that marking is correct or not. If in doubt, CC
debian-cross@lists.debian.org to get advice about whether marking a package as
M-A:foreign is correct or not.

Thanks!

cheers, josch

[signature.asc (application/pgp-signature, inline)]

Message #45 received at 666772@bugs.debian.org (full text, mbox, reply):

From: Vasyl Gello <vasek.gello@gmail.com>

To: Johannes Schauer Marin Rodrigues <josch@debian.org>, 666772@bugs.debian.org

Cc: Helmut Grohne <helmut@subdivi.de>, debian-cross@lists.debian.org

Subject: Re: Bug#666772: Bug 666772

Date: Thu, 28 Oct 2021 16:12:36 +0000

[Message part 1 (text/plain, inline)]

Hi Johannes!

Thanks for clarification! So I have to ask maintainers of groovy:all and libcommons-lang-java:all whether they are OK marking it M-A: foreign?

Also, why does apt-get install all arch:host build-deps but not libdevel:host duplicated with libdevel:build ?

Is it possible at all to have "pure" cross-compilation without need to install qemu-binfmt-support on host?
-- 
Vasyl Gello
==================================================
Certified SolidWorks Expert

Mob.:+380 (98) 465 66 77

E-Mail: vasek.gello@gmail.com

Skype: vasek.gello
==================================================
호랑이는 죽어서 가죽을 남기고 사람은 죽어서 이름을 남긴다

[Message part 2 (text/html, inline)]

Send a report that this bug log contains spam.