Debian Bug report logs - #610689
sbuild: cross support

version graph

Package: sbuild; Maintainer for sbuild is Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>; Source for sbuild is src:sbuild.

Reported by: Hector Oron <hector.oron@gmail.com>

Date: Fri, 21 Jan 2011 12:15:02 UTC

Severity: wishlist

Tags: patch

Found in version sbuild/0.60.8-1

Fixed in version sbuild/0.63.0-1

Done: Roger Leigh <rleigh@debian.org>

Bug is archived. No further changes may be made.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#610689; Package sbuild. (Fri, 21 Jan 2011 12:15:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to Hector Oron <hector.oron@gmail.com>:
New Bug report received and forwarded. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Fri, 21 Jan 2011 12:15:05 GMT) Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: Hector Oron <hector.oron@gmail.com>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: sbuild: cross support
Date: Fri, 21 Jan 2011 12:09:55 +0000
Package: sbuild
Version: 0.60.8-1
Severity: wishlist

Hello,

  I would love sbuild to gain the ability to handle cross builds. What is a cross build? Well, it is a way to generate a binary which it is meant to run on a foreign architecture. For cross builds you need cross tools, which are not yet in Debian archive, but those are available via emdebian.org repositories. So basically, you need to add
  deb http://emdebian.org squeeze main
into your sources.list, update and install, for example, linux-libc-dev-armel-cross, libc6-dev-armel-cross, gcc-4.3-arm-linux-gnueabi, ..
Once you got the cross toolchain in place, you can try to cross compile hello application passing -a$arch option to dpkg-buildpackage and the package must support cross building, usually by adding a three liner [0] to the package build system file.

(sid_amd64)zumbi@cat:/tmp/hello-2.6$ dpkg-buildpackage -us -uc -rfakeroot -aarmel
[...]
chown -R root:root debian/tmp
chmod -R u+w,go=rX debian/tmp
dpkg --build debian/tmp ..
dpkg-deb: building package `hello' in `../hello_2.6-1_armel.deb'.
 dpkg-genchanges  >../hello_2.6-1_armel.changes
dpkg-genchanges: including full source code in upload
 dpkg-source --after-build hello-2.6
dpkg-buildpackage: full upload (original source is included)

When trying sbuild to do it for me,

$ sudo sbuild -d sid_amd64 --debbuildopt="-aarmel" hello_2.6-1
[...]
chmod -R u+w,go=rX debian/tmp
dpkg --build debian/tmp ..
dpkg-deb: building package `hello' in `../hello_2.6-1_armel.deb'.
 dpkg-genchanges -B >../hello_2.6-1_armel.changes
dpkg-genchanges: arch-specific upload - not including arch-independent packages
dpkg-genchanges: binary-only upload - not including any source code
 dpkg-source --after-build hello-2.6
dpkg-buildpackage: binary only upload (no source included)
Build finished at 20110121-1105
Can't find hello_2.6-1_amd64.changes -- can't dump info
  Package contents
  Finished
Built successfully
Purging /srv/chroot/sid_amd64/build/root-hello_2.6-1-amd64-D9WaOO
Finished at 20110121-1105
Build needed 00:00:12, 4812k disc space

Sbuild builds the package but it is unable to take it out of the chroot, because it is unable to find the right changes file for the right architecture.
"Can't find hello_2.6-1_amd64.changes -- can't dump info"

Doing a $nasty hack on Build.pm we are able to follow *_armel.changes file.

$ sudo vi /usr/share/perl5/Sbuild/Build.pm +1944
# Figure out chroot architecture
sub chroot_arch {
    my $self = shift;
    my $nasty = "armel";
^^^^^
    my $pipe = $self->get('Session')->pipe_command(
        { COMMAND => [$self->get_conf('DPKG'),
                      '--print-architecture'],
          USER => $self->get_conf('USERNAME'),
          CHROOT => 1,
          PRIORITY => 0,
          DIR => '/' }) || return undef;

    chomp(my $chroot_arch = <$pipe>);
    close($pipe);

    die "Can't determine architecture of chroot: $!\n"
        if ($? || !defined($chroot_arch));

    return $nasty;
^^^^^
 #   return $chroot_arch;
}

It seems that sbuild triggers dpkg --print-architecture to find out chroot architecture, but when doing cross it should try to follow dpkg-architecture's DEB_HOST_ARCH.

Would it be possible to modify Build.pm in some way, either calling dpkg-architecture or autodetecting -a$arch has been passed and it should follow other architecture changes file?

Have a very nice day!

[0] http://wiki.debian.org/EmdebianGuide#Addingcross-builddetection

Best regards,
-- 
 Héctor Orón

-- System Information:
Debian Release: 6.0
  APT prefers unstable
  APT policy: (700, 'unstable'), (600, 'testing'), (500, 'stable'), (1, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.32-5-amd64 (SMP w/8 CPU cores)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash

Versions of packages sbuild depends on:
ii  adduser                       3.112+nmu2 add and remove users and groups
ii  libsbuild-perl                0.60.8-1   Tool for building Debian binary pa
ii  perl                          5.10.1-17  Larry Wall's Practical Extraction 
ii  perl-modules                  5.10.1-17  Core Perl modules

Versions of packages sbuild recommends:
ii  debootstrap                   1.0.26     Bootstrap a basic Debian system
ii  fakeroot                      1.14.5-1   Gives a fake root environment

Versions of packages sbuild suggests:
pn  deborphan                     <none>     (no description available)
ii  wget                          1.12-2.1   retrieves files from the web

-- no debconf information




Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#610689; Package sbuild. (Sat, 22 Jan 2011 19:06:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Roger Leigh <rleigh@codelibre.net>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Sat, 22 Jan 2011 19:06:03 GMT) Full text and rfc822 format available.

Message #10 received at 610689@bugs.debian.org (full text, mbox):

From: Roger Leigh <rleigh@codelibre.net>
To: Hector Oron <hector.oron@gmail.com>, 610689@bugs.debian.org, debian-embedded@lists.debian.org
Subject: Re: [buildd-tools-devel] Bug#610689: sbuild: cross support
Date: Sat, 22 Jan 2011 19:03:48 +0000
[Message part 1 (text/plain, inline)]
On Fri, Jan 21, 2011 at 12:09:55PM +0000, Hector Oron wrote:
>   I would love sbuild to gain the ability to handle cross builds.
[…]

Thanks for the informative explanation, it has helped to make the
problem a bit clearer for me.

> It seems that sbuild triggers dpkg --print-architecture to find out chroot architecture, but when doing cross it should try to follow dpkg-architecture's DEB_HOST_ARCH.
> 
> Would it be possible to modify Build.pm in some way, either calling dpkg-architecture or autodetecting -a$arch has been passed and it should follow other architecture changes file?

We can definitely do this.

I think that the several uses of the --arch option are going to need
separating into separate options.  Does the following make sense, or
am I missing out anything major?

1) Selection of the build chroot

   We look for a chroot named $distribution-$arch-sbuild or
   $distribution-$arch

   When we are genuinely building for a different arch, then this works.
   When we're cross-compiling, we want to build on a chroot using the
   host arch (most likely, though you /could/ use a different arch with
   qemu and do an emulated cross-build once we have such support!)

2) Selection of the host architecture

   Linked to (1).  We are assuming that the build and host architectures
   are the same, so --host is implicitly setting both.

I think we should add two additional options:
  --host   select host arch
  --build  select build arch
(or whatever are most commonly used in existing Debian tools; these are
from autoconf configure) and have "--arch=foo" be equivalent to
"--host=foo --build=foo".

The chroot selection will use the host arch only.  So this will
typically be the "real" host arch, but it will also allow "native"
(non-cross) building using qemu as well as kernel personalities (i386
on amd64 etc, which is already supported).  When we come to run
dpkg-buildpackage, we will then pass the build arch which will result
in a cross-build if this differs from the host arch.

Hopefully this will implement support fairly cleanly and flexibly.

Following on from that there's the issue of resolver support and
use of xapt.  Some examples of how xapt would be used would help
here; I'm a bit unsure of if it's used directly as an apt-get
wrapper, or if it's a multi-step process.  Understanding the
workflow would be most useful.


Regards,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
   `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#610689; Package sbuild. (Sat, 22 Jan 2011 21:54:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to Roger Leigh <rleigh@codelibre.net>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Sat, 22 Jan 2011 21:54:05 GMT) Full text and rfc822 format available.

Message #15 received at 610689@bugs.debian.org (full text, mbox):

From: Roger Leigh <rleigh@codelibre.net>
To: Hector Oron <hector.oron@gmail.com>, 610689@bugs.debian.org, debian-embedded@lists.debian.org
Subject: Re: [buildd-tools-devel] Bug#610689: Bug#610689: sbuild: cross support
Date: Sat, 22 Jan 2011 21:51:53 +0000
[Message part 1 (text/plain, inline)]
On Sat, Jan 22, 2011 at 07:03:48PM +0000, Roger Leigh wrote:
> On Fri, Jan 21, 2011 at 12:09:55PM +0000, Hector Oron wrote:
> >   I would love sbuild to gain the ability to handle cross builds.
> […]
> 
> Thanks for the informative explanation, it has helped to make the
> problem a bit clearer for me.
> 
> > It seems that sbuild triggers dpkg --print-architecture to find out chroot architecture, but when doing cross it should try to follow dpkg-architecture's DEB_HOST_ARCH.
> > 
> > Would it be possible to modify Build.pm in some way, either calling dpkg-architecture or autodetecting -a$arch has been passed and it should follow other architecture changes file?
> 
> I think we should add two additional options:
>   --host   select host arch
>   --build  select build arch
> (or whatever are most commonly used in existing Debian tools; these are
> from autoconf configure) and have "--arch=foo" be equivalent to
> "--host=foo --build=foo".

I've attached a patch (against sbuild git master, also on the
cross-build branch of git://git.debian.org/users/rleigh/sbuild.git)
which adds --host and --build options and then uses the correct
host or build arch in the appropriate places.  While I've tested
everything still works fine for normal non-cross builds, this will
need testing for cross builds to make sure I've not got the
host/build arch mixed up in any places (since I will not have
picked any error up when they are both the same).


Regards,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
   `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.
[signature.asc (application/pgp-signature, inline)]

Added tag(s) patch. Request was from Roger Leigh <rleigh@codelibre.net> to control@bugs.debian.org. (Sat, 22 Jan 2011 21:54:07 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#610689; Package sbuild. (Sat, 22 Jan 2011 22:00:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Hector Oron <hector.oron@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Sat, 22 Jan 2011 22:00:03 GMT) Full text and rfc822 format available.

Message #22 received at 610689@bugs.debian.org (full text, mbox):

From: Hector Oron <hector.oron@gmail.com>
To: Roger Leigh <rleigh@codelibre.net>
Cc: 610689@bugs.debian.org, debian-embedded@lists.debian.org
Subject: Re: [buildd-tools-devel] Bug#610689: sbuild: cross support
Date: Sat, 22 Jan 2011 21:56:26 +0000
Hello,

2011/1/22 Roger Leigh <rleigh@codelibre.net>:
> On Fri, Jan 21, 2011 at 12:09:55PM +0000, Hector Oron wrote:

> I think that the several uses of the --arch option are going to need
> separating into separate options.  Does the following make sense, or
> am I missing out anything major?

> 1) Selection of the build chroot
>
>   We look for a chroot named $distribution-$arch-sbuild or
>   $distribution-$arch
>
>   When we are genuinely building for a different arch, then this works.
>   When we're cross-compiling, we want to build on a chroot using the
>   host arch (most likely, though you /could/ use a different arch with
>   qemu and do an emulated cross-build once we have such support!)

$distribution, as in Debian or derivatives could be guessed using
dpkg-vendor variables.

$distribution as in Debian suites probably needs to be passed as argument,
as it is probably now.

$arch, it is splitted by dpkg-architecture into DEB_BUILD_* and
DEB_HOST_*.

Let's use m68k as an example, in case we do native build:
  DEB_BUILD_ARCH (m68k) = DEB_HOST_ARCH (m68k)

if we do a cross-build
  DEB_BUILD_ARCH (amd64|i386) != DEB_HOST_ARCH (m68k)
we need to use DEB_BUILD_ARCH chroot and follow DEB_HOST_ARCH
*.changes file.

if we do an emulated build
  DEB_BUILD_ARCH (amd64|i386) = DEB_HOST_ARCH (amd64|i386) -- out of the chroot
  DEB_BUILD_ARCH (m68k) = DEB_HOST_ARCH (m68k) -- within the chroot
I do not think we need a new variable on dpkg-architecture, as this is same
as doing a native build but picking a foreign architecture, which we could use
current --arch switch on sbuild. If --arch is used, dpkg
--print-architecture checks
should probably need to be bypassed.

if we do an emulated build faking emulated compilers by cross
compilers (croco approach),
it can be done transparently, just depending on the chroot
configuration (i.e. croco is
installed ) or maybe we need to explicitly tell the build to make use
of cross compilers or not
make use of them, in any case I do not think sbuild/schroot need to
know about it.

What do others think?

> 2) Selection of the host architecture
>
>   Linked to (1).  We are assuming that the build and host architectures
>   are the same, so --host is implicitly setting both.
>
> I think we should add two additional options:
>  --host   select host arch
>  --build  select build arch
> (or whatever are most commonly used in existing Debian tools; these are
> from autoconf configure) and have "--arch=foo" be equivalent to
> "--host=foo --build=foo".
>
> The chroot selection will use the host arch only.  So this will
> typically be the "real" host arch, but it will also allow "native"
> (non-cross) building using qemu as well as kernel personalities (i386
> on amd64 etc, which is already supported).  When we come to run
> dpkg-buildpackage, we will then pass the build arch which will result
> in a cross-build if this differs from the host arch.
>
> Hopefully this will implement support fairly cleanly and flexibly.

On the previous options described above, we might want to develop some
syntax in the control files which can mark packages to be able to be cross
built or built emulated. Sbuild should decode that information from control
file information, then set accordingly --host, --build or whatever we call them,
but I am afraid this naming might help confusing things much more with build,
host and target.

(( Info: I was going to suggest the use of  '--cross $arch --arch $_arch'
which implies building on a chroot of architecture $_arch and passing
dpkg-buildpackage '-a$arch' in the build. Then, in the case of emulated build,
we could explicitly tell sbuild by other option '--emulate' (or else).
But maybe your suggestion is cleaner ))

> Following on from that there's the issue of resolver support and
> use of xapt.  Some examples of how xapt would be used would help
> here; I'm a bit unsure of if it's used directly as an apt-get
> wrapper, or if it's a multi-step process.  Understanding the
> workflow would be most useful.

xapt reads build dependencies of package and then it downloads those
for the cross architecture, passing the packages to dpkg-cross which
puts headers and libraries ready to be used by the cross tools.
It can be called directly from command-line (before running the build).
All this trickery should go away after multiarch bits are in place which
should be able to provide libraries and headers cleanly and transparently
for foreign architectures.

Best regards,
-- 
 Héctor Orón

"Our Sun unleashes tremendous flares expelling hot gas into the Solar
System, which one day will disconnect us."

-- Day DVB-T stop working nicely
Video flare: http://antwrp.gsfc.nasa.gov/apod/ap100510.html




Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#610689; Package sbuild. (Sat, 22 Jan 2011 22:39:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Roger Leigh <rleigh@codelibre.net>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Sat, 22 Jan 2011 22:39:03 GMT) Full text and rfc822 format available.

Message #27 received at 610689@bugs.debian.org (full text, mbox):

From: Roger Leigh <rleigh@codelibre.net>
To: Hector Oron <hector.oron@gmail.com>
Cc: 610689@bugs.debian.org, debian-embedded@lists.debian.org
Subject: Re: [buildd-tools-devel] Bug#610689: sbuild: cross support
Date: Sat, 22 Jan 2011 22:37:50 +0000
[Message part 1 (text/plain, inline)]
On Sat, Jan 22, 2011 at 09:56:26PM +0000, Hector Oron wrote:
> Hello,
> 
> 2011/1/22 Roger Leigh <rleigh@codelibre.net>:
> > On Fri, Jan 21, 2011 at 12:09:55PM +0000, Hector Oron wrote:
> 
> > I think that the several uses of the --arch option are going to need
> > separating into separate options.  Does the following make sense, or
> > am I missing out anything major?
> 
> > 1) Selection of the build chroot
> >
> >   We look for a chroot named $distribution-$arch-sbuild or
> >   $distribution-$arch
> >
> >   When we are genuinely building for a different arch, then this works.
> >   When we're cross-compiling, we want to build on a chroot using the
> >   host arch (most likely, though you /could/ use a different arch with
> >   qemu and do an emulated cross-build once we have such support!)
> 
> $distribution, as in Debian or derivatives could be guessed using
> dpkg-vendor variables.
> 
> $distribution as in Debian suites probably needs to be passed as argument,
> as it is probably now.

$distribution means the latter (the suite name), as used in
debian/changelog.  If we could read it directly from debian/changelog,
that would be great, but that's not usually possible.  There's a
chicken-and-egg problem with this because we need to know the
distribution in order to download and/or unpack the sources most of
the time, and we can't know that until we have the sources, so it
must be specified.

$arch is referring to the build architecture only (this is what the
patch I just sent uses, though I realise I have confused host and build
in my patch, I'll swap them around).

> $arch, it is splitted by dpkg-architecture into DEB_BUILD_* and
> DEB_HOST_*.
> 
> Let's use m68k as an example, in case we do native build:
>   DEB_BUILD_ARCH (m68k) = DEB_HOST_ARCH (m68k)
> 
> if we do a cross-build
>   DEB_BUILD_ARCH (amd64|i386) != DEB_HOST_ARCH (m68k)
> we need to use DEB_BUILD_ARCH chroot and follow DEB_HOST_ARCH
> *.changes file.

Hmm, I think I've confused host and build in my patch, I'll swap them
around.

> if we do an emulated build
>   DEB_BUILD_ARCH (amd64|i386) = DEB_HOST_ARCH (amd64|i386) -- out of the chroot
>   DEB_BUILD_ARCH (m68k) = DEB_HOST_ARCH (m68k) -- within the chroot

Yes.  From the point of view of sbuild, it shouldn't need to worry about
emulated builds; it should be entirely transparent and treated no
differently than normal native builds.

> I do not think we need a new variable on dpkg-architecture, as this is same
> as doing a native build but picking a foreign architecture, which we could use
> current --arch switch on sbuild. If --arch is used, dpkg
> --print-architecture checks
> should probably need to be bypassed.

It should work with sbuild today (once schroot does this).  The check
should still succeed when --arch is used (after all, it's a sanity
check to ensure we have the correct build arch in the chroot).  If
"dpkg --print-architecture" returns the emulated arch in the chroot
(and since it's the native m68k package I would assume it should return
the correct value), then that should be sufficient.  Is this how qemu
works?  Would it return m68k on amd64 in an m68k chroot with qemu?

> > 2) Selection of the host architecture
> >
> >   Linked to (1).  We are assuming that the build and host architectures
> >   are the same, so --host is implicitly setting both.
> >
> > I think we should add two additional options:
> >  --host   select host arch
> >  --build  select build arch
> > (or whatever are most commonly used in existing Debian tools; these are
> > from autoconf configure) and have "--arch=foo" be equivalent to
> > "--host=foo --build=foo".
> >
> > The chroot selection will use the host arch only.  So this will
> > typically be the "real" host arch, but it will also allow "native"
> > (non-cross) building using qemu as well as kernel personalities (i386
> > on amd64 etc, which is already supported).  When we come to run
> > dpkg-buildpackage, we will then pass the build arch which will result
> > in a cross-build if this differs from the host arch.
> >
> > Hopefully this will implement support fairly cleanly and flexibly.
> 
> On the previous options described above, we might want to develop some
> syntax in the control files which can mark packages to be able to be cross
> built or built emulated. Sbuild should decode that information from control
> file information, then set accordingly --host, --build or whatever we call them,
> but I am afraid this naming might help confusing things much more with build,
> host and target.

What's the use case for such information?

sbuild can certainly extract such information from the control file.  It
couldn't set --host/--build though surely?; (like the --distribution, this
would need specifying by the user/buildd running sbuild since they would
be build-specific depending upon the arch one wants to build for.)

Information telling us that cross-compilation works (or not) would allow
for aborting builds that are pointless.  OTOH, such information might
better belong in databases like Packages-arch-specific.

> (( Info: I was going to suggest the use of  '--cross $arch --arch $_arch'
> which implies building on a chroot of architecture $_arch and passing
> dpkg-buildpackage '-a$arch' in the build. Then, in the case of emulated build,
> we could explicitly tell sbuild by other option '--emulate' (or else).
> But maybe your suggestion is cleaner ))

I think --build/--host match the existing usage in dpkg-architecture
and autoconf etc, so may possibly be clearer, but I'm by no means an
expert!

--emulate should never be necessary should it?  If the emulation is
completely transparent, why would we need to be informed about it, and
what could we do with the information?

> > Following on from that there's the issue of resolver support and
> > use of xapt.  Some examples of how xapt would be used would help
> > here; I'm a bit unsure of if it's used directly as an apt-get
> > wrapper, or if it's a multi-step process.  Understanding the
> > workflow would be most useful.
> 
> xapt reads build dependencies of package and then it downloads those
> for the cross architecture, passing the packages to dpkg-cross which
> puts headers and libraries ready to be used by the cross tools.
> It can be called directly from command-line (before running the build).
> All this trickery should go away after multiarch bits are in place which
> should be able to provide libraries and headers cleanly and transparently
> for foreign architectures.

Could I possibly see an example of its command-line usage and output?
One thing we would need to do it know what it installed/removed so we
can reverse it post-build.


Regards,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
   `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#610689; Package sbuild. (Sat, 22 Jan 2011 23:00:06 GMT) Full text and rfc822 format available.

Acknowledgement sent to Loïc Minier <lool@dooz.org>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Sat, 22 Jan 2011 23:00:07 GMT) Full text and rfc822 format available.

Message #32 received at 610689@bugs.debian.org (full text, mbox):

From: Loïc Minier <lool@dooz.org>
To: Roger Leigh <rleigh@codelibre.net>
Cc: Hector Oron <hector.oron@gmail.com>, 610689@bugs.debian.org, debian-embedded@lists.debian.org
Subject: Re: [buildd-tools-devel] Bug#610689: sbuild: cross support
Date: Sat, 22 Jan 2011 23:57:35 +0100
On Sat, Jan 22, 2011, Roger Leigh wrote:
>                                                      Is this how qemu
> works?  Would it return m68k on amd64 in an m68k chroot with qemu?

 Yup; dpkg --print-architecture seems to return the architecture dpkg
 was built for, so an armel dpkg would return armel (it would happen to
 run thanks to qemu).  dpkg-architecture runs "gcc -dumpmachine", so
 would run the chroot's gcc to guess the architecture so same story:
 gcc_armel would return an ARM triplet, but if you pass -a<arch> it
 shouldn't matter.  uname() is intercepted by qemu at least on ARM
 (don't know about other arches), so will return armv7l or something
 similar depending on the CPU you're actually emulating.

-- 
Loïc Minier




Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#610689; Package sbuild. (Sat, 22 Jan 2011 23:12:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Loïc Minier <lool@dooz.org>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Sat, 22 Jan 2011 23:12:03 GMT) Full text and rfc822 format available.

Message #37 received at 610689@bugs.debian.org (full text, mbox):

From: Loïc Minier <lool@dooz.org>
To: Roger Leigh <rleigh@codelibre.net>
Cc: Hector Oron <hector.oron@gmail.com>, 610689@bugs.debian.org, debian-embedded@lists.debian.org
Subject: Re: [buildd-tools-devel] Bug#610689: Bug#610689: sbuild: cross support
Date: Sun, 23 Jan 2011 00:08:59 +0100
[Message part 1 (text/plain, inline)]
On Sat, Jan 22, 2011, Roger Leigh wrote:
> I've attached a patch (against sbuild git master, also on the
> cross-build branch of git://git.debian.org/users/rleigh/sbuild.git)
> which adds --host and --build options and then uses the correct
> host or build arch in the appropriate places.  While I've tested
> everything still works fine for normal non-cross builds, this will
> need testing for cross builds to make sure I've not got the
> host/build arch mixed up in any places (since I will not have
> picked any error up when they are both the same).

 I think the patches got lost somewhere; I've extracted them from git
 and am attaching them here for others; thanks!

 I confirm that the first patch seems to mix build and host arch: you
 want to search for a chroot named after the build architecture rather
 than the host architecture (at least that's how I name mine  ;-)  this
 affects bin/sbuild-createchroot and the man page changes at least.  I
 also saw you're testing for build_arch in the .dsc, but the build arch
 is always irrelevant when looking at debian/* ATM -- they only speak of
 host architectures:
        for my $a (split(/\s+/, $dscarchs)) {
-           if (Dpkg::Arch::debarch_is($arch, $a)) {
+           if (Dpkg::Arch::debarch_is($build_arch, $a)) {

 The second patch has a typo, it tests BUILD_ARCH != BUILD_ARCH.


 If you're looking for test cases, hello/hello-debhelper should be
 fairly easy to cross-compile, or u-boot (only latest version is fixed).

-- 
Loïc Minier
[0001-Add-initial-host-build-arch-separation-for-cross-bui.patch (text/x-diff, attachment)]
[0002-Sbuild-Build-Add-dpkg-buildpackage-a-option-when-cro.patch (text/x-diff, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#610689; Package sbuild. (Sat, 22 Jan 2011 23:21:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Hector Oron <hector.oron@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Sat, 22 Jan 2011 23:21:03 GMT) Full text and rfc822 format available.

Message #42 received at 610689@bugs.debian.org (full text, mbox):

From: Hector Oron <hector.oron@gmail.com>
To: Roger Leigh <rleigh@codelibre.net>
Cc: 610689@bugs.debian.org, debian-embedded@lists.debian.org
Subject: Re: [buildd-tools-devel] Bug#610689: Bug#610689: sbuild: cross support
Date: Sat, 22 Jan 2011 23:16:44 +0000
Hi,

2011/1/22 Roger Leigh <rleigh@codelibre.net>:

> I've attached a patch (against sbuild git master, also on the
> cross-build branch of git://git.debian.org/users/rleigh/sbuild.git)
> which adds --host and --build options and then uses the correct
> host or build arch in the appropriate places.  While I've tested
> everything still works fine for normal non-cross builds, this will
> need testing for cross builds to make sure I've not got the
> host/build arch mixed up in any places (since I will not have
> picked any error up when they are both the same).

Thanks, that was quick...

From a quick glance, the package is missing automake (and autotools-dev)
build dependency. It also fails to run bootstrap code...

make: *** No rule to make target `configure', needed by
`debian/build/config.status'.  Stop.
dpkg-buildpackage: error: debian/rules build gave error exit status 2

Maybe a
configure:
   ./bootstrap
was forgotten on the debian/rules file?

Once it is built and installed,

zumbi@cat:~$ sudo sbuild -d sid_amd64 hello_2.6-1
Argument "amd64" isn't numeric in numeric ne (!=) at
/usr/share/perl5/Sbuild/Build.pm line 1809.
sbuild (Debian sbuild) 0.60.9 (22 Jan 2011) on cat.emdebian.org
Argument "amd64" isn't numeric in numeric ne (!=) at
/usr/share/perl5/Sbuild/Build.pm line 1809.
sbuild (Debian sbuild) 0.60.9 (22 Jan 2011) on cat.emdebian.org

ââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ
â hello 2.6-1 (amd64)                                        22 Jan 2011 23:11 â
ââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ

Package: hello
Version: 2.6-1
Source Version: 2.6-1
Architecture: amd64
Host Architecture: amd64
Build Architecture: amd64
[...]
Checking correctness of dependencies...

Cannot open /srv/chroot/sid_amd64/etc/lsb-release: No such file or directory
ââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ
â Install hello build dependencies (internal resolver)                         â
ââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ

Cannot open /srv/chroot/sid_amd64/etc/lsb-release: No such file or directory
Build-Depends: base-files, base-passwd, bash, coreutils, dash,
debianutils, diffutils, dpkg, e2fsprogs, findutils, grep, gzip,
hostname, ncurses-base, ncurses-bin, perl-base, sed, login,
sysvinit-utils, sysvinit, tar, bsdutils, mount, util-linux,
libc6.1-dev | libc-dev, libc6-dev-sparc64, gcc (>= 4:4.4.3), g++ (>=
4:4.4.3), make, dpkg-dev (>= 1.13.5)
Checking for already installed dependencies...
Use of uninitialized value $ver in hash element at
/usr/share/perl5/Sbuild/InternalResolver.pm line 558, <$pipe> line 7.
Use of uninitialized value $ver in hash element at
/usr/share/perl5/Sbuild/InternalResolver.pm line 560, <$pipe> line 7.
Use of uninitialized value $ver in hash element at
/usr/share/perl5/Sbuild/InternalResolver.pm line 558, <$pipe> line 8.
Use of uninitialized value $ver in hash element at
/usr/share/perl5/Sbuild/InternalResolver.pm line 558, <$pipe> line 8.
Can't use an undefined value as an ARRAY reference at
/usr/share/perl5/Sbuild/InternalResolver.pm line 571.

When trying --host and --build options I get:
zumbi@cat:~$ sudo sbuild -d sid_amd64 --host=armel --build=amd64 hello_2.6-1
Argument "amd64" isn't numeric in numeric ne (!=) at
/usr/share/perl5/Sbuild/Build.pm line 1809.
Argument "amd64" isn't numeric in numeric ne (!=) at
/usr/share/perl5/Sbuild/Build.pm line 1809.
sbuild (Debian sbuild) 0.60.9 (22 Jan 2011) on cat.emdebian.org

ââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ
â hello 2.6-1 (amd64)                                        22 Jan 2011 23:12 â
ââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ

age: hello
Version: 2.6-1
Source Version: 2.6-1
Architecture: amd64
Host Architecture: armel
Build Architecture: amd64
Requested host architecture (armel) and chroot architecture (amd64) do
not match.  Skipping build.

---------------------------------

zumbi@cat:~$ sudo sbuild -d sid_amd64 --host=amd64 --build=armel hello_2.6-1
Argument "armel" isn't numeric in numeric ne (!=) at
/usr/share/perl5/Sbuild/Build.pm line 1809.
Argument "armel" isn't numeric in numeric ne (!=) at
/usr/share/perl5/Sbuild/Build.pm line 1809.
sbuild (Debian sbuild) 0.60.9 (22 Jan 2011) on cat.emdebian.org

ââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ
â hello 2.6-1 (armel)                                        22 Jan 2011 23:13 â
ââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ

Package: hello
Version: 2.6-1
Source Version: 2.6-1
Architecture: armel
Host Architecture: amd64
Build Architecture: armel
[...]
Checking correctness of dependencies...
Cannot open /srv/chroot/sid_amd64/etc/lsb-release: No such file or directory

ââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ
â Install hello build dependencies (internal resolver)                         â
ââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ

Build-Depends: base-files, base-passwd, bash, coreutils, dash,
debianutils, diffutils, dpkg, e2fsprogs, findutils, grep, gzip,
hostname, ncurses-base, ncurses-bin, perl-base, sed, login,
sysvinit-utils, sysvinit, tar, bsdutils, mount, util-linux,
libc6.1-dev | libc-dev, libc6-dev-sparc64, gcc (>= 4:4.4.3), g++ (>=
4:4.4.3), make, dpkg-dev (>= 1.13.5)
Checking for already installed dependencies...
Use of uninitialized value $ver in hash element at
/usr/share/perl5/Sbuild/InternalResolver.pm line 558, <$pipe> line 7.
Use of uninitialized value $ver in hash element at
/usr/share/perl5/Sbuild/InternalResolver.pm line 560, <$pipe> line 7.
Use of uninitialized value $ver in hash element at
/usr/share/perl5/Sbuild/InternalResolver.pm line 558, <$pipe> line 8.
Use of uninitialized value $ver in hash element at
/usr/share/perl5/Sbuild/InternalResolver.pm line 558, <$pipe> line 8.
Can't use an undefined value as an ARRAY reference at
/usr/share/perl5/Sbuild/InternalResolver.pm line 571.

Notice I am using an already existing chroot handmade created, not by
using sbuild-create.

Best regards,
-- 
 Héctor Orón

"Our Sun unleashes tremendous flares expelling hot gas into the Solar
System, which one day will disconnect us."

-- Day DVB-T stop working nicely
Video flare: http://antwrp.gsfc.nasa.gov/apod/ap100510.html

Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#610689; Package sbuild. (Sat, 22 Jan 2011 23:45:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Hector Oron <hector.oron@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Sat, 22 Jan 2011 23:45:03 GMT) Full text and rfc822 format available.

Message #47 received at 610689@bugs.debian.org (full text, mbox):

From: Hector Oron <hector.oron@gmail.com>
To: Roger Leigh <rleigh@codelibre.net>
Cc: 610689@bugs.debian.org, debian-embedded@lists.debian.org
Subject: Re: [buildd-tools-devel] Bug#610689: sbuild: cross support
Date: Sat, 22 Jan 2011 23:41:41 +0000
Hello,

2011/1/22 Roger Leigh <rleigh@codelibre.net>:

> It should work with sbuild today (once schroot does this).  The check
> should still succeed when --arch is used (after all, it's a sanity
> check to ensure we have the correct build arch in the chroot).  If
> "dpkg --print-architecture" returns the emulated arch in the chroot
> (and since it's the native m68k package I would assume it should return
> the correct value), then that should be sufficient.  Is this how qemu
> works?  Would it return m68k on amd64 in an m68k chroot with qemu?

Yes.

zumbi@enorme:~$ file test/bin/ls
test/bin/ls: ELF 32-bit MSB executable, PowerPC or cisco 4500, version
1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.8,
with unknown capability 0x41000000 = 0xf676e75, with unknown
capability 0x10000 = 0x70401, stripped
zumbi@enorme:~$ sudo chroot test
enorme:/# dpkg --print-architecture
powerpc
enorme:/# dpkg-architecture
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LANG = "en_GB.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
/bin/sh: gcc: command not found
dpkg-architecture: warning: Couldn't determine gcc system type,
falling back to default (native compilation)
DEB_BUILD_ARCH=powerpc
DEB_BUILD_ARCH_OS=linux
DEB_BUILD_ARCH_CPU=powerpc
DEB_BUILD_GNU_CPU=powerpc
DEB_BUILD_GNU_SYSTEM=linux-gnu
DEB_BUILD_GNU_TYPE=powerpc-linux-gnu
DEB_HOST_ARCH=powerpc
DEB_HOST_ARCH_OS=linux
DEB_HOST_ARCH_CPU=powerpc
DEB_HOST_GNU_CPU=powerpc
DEB_HOST_GNU_SYSTEM=linux-gnu
DEB_HOST_GNU_TYPE=powerpc-linux-gnu
enorme:/# exit
zumbi@enorme:~$ dpkg --print-architecture
amd64

>> On the previous options described above, we might want to develop some
>> syntax in the control files which can mark packages to be able to be cross
>> built or built emulated. Sbuild should decode that information from control
>> file information, then set accordingly --host, --build or whatever we call them,
>> but I am afraid this naming might help confusing things much more with build,
>> host and target.
>
> What's the use case for such information?

I was just thinking we might want to tag a local built package (or a
kernel package)
to be cross-built, but yes, it seems to be a chicken-egg problem.

> --emulate should never be necessary should it?  If the emulation is
> completely transparent, why would we need to be informed about it, and
> what could we do with the information?

It should not be needed, I was just thinking on adding extra information. We
get a source package uploaded to amd64 host, how do you know if you want
to build native, cross build or build in an emulated environment? for which
architecture do we want it to be built?

> Could I possibly see an example of its command-line usage and output?
> One thing we would need to do it know what it installed/removed so we
> can reverse it post-build.

I do not think xapt has an option to remove packages. A brute-force option
would be to remove *-$arch-cross packages from chroot.

(experimental)zumbi@enorme:~$ sudo xapt -a armel -M
http://ftp.uk.debian.org/debian -S unstable -k hello
apt-get  -o Apt::Get::Download-Only=true -y -o Apt::Architecture=armel
-o Apt::Install-Recommends=false -o Dir::Etc=/var/lib/xapt/etc/xapt/
-o Dir::Etc::TrustedParts=/etc/apt/trusted.gpg.d -o
Dir::Etc::Trusted=/etc/apt/trusted.gpg -o
Dir::Etc::SourceList=/var/lib/xapt/etc/xapt/sources.list -o
Dir::Etc::SourceParts=/var/lib/xapt/etc/xapt/sources.list.d/ -o
Dir::State=/var/lib/xapt/ -o
Dir::State::Status=/var/lib/xapt//armel/dpkg/status -o
Dir::Cache=/var/lib/xapt/ update
Get:1 http://ftp.uk.debian.org sid Release.gpg [835B]
Ign http://ftp.uk.debian.org/debian/ sid/contrib Translation-en
Ign http://ftp.uk.debian.org/debian/ sid/contrib Translation-en_GB
Ign http://ftp.uk.debian.org/debian/ sid/main Translation-en
Ign http://ftp.uk.debian.org/debian/ sid/main Translation-en_GB
Ign http://ftp.uk.debian.org/debian/ sid/non-free Translation-en
Ign http://ftp.uk.debian.org/debian/ sid/non-free Translation-en_GB
Get:2 http://ftp.uk.debian.org experimental Release.gpg [835B]
Ign http://ftp.uk.debian.org/debian/ experimental/contrib Translation-en
Ign http://ftp.uk.debian.org/debian/ experimental/contrib Translation-en_GB
Ign http://ftp.uk.debian.org/debian/ experimental/main Translation-en
Ign http://ftp.uk.debian.org/debian/ experimental/main Translation-en_GB
Ign http://ftp.uk.debian.org/debian/ experimental/non-free Translation-en
Ign http://ftp.uk.debian.org/debian/ experimental/non-free Translation-en_GB
Get:3 http://ftp.uk.debian.org unstable Release.gpg [835B]
Ign http://ftp.uk.debian.org/debian/ unstable/contrib Translation-en
Ign http://ftp.uk.debian.org/debian/ unstable/contrib Translation-en_GB
Ign http://ftp.uk.debian.org/debian/ unstable/main Translation-en
Ign http://ftp.uk.debian.org/debian/ unstable/main Translation-en_GB
Ign http://ftp.uk.debian.org/debian/ unstable/non-free Translation-en
Ign http://ftp.uk.debian.org/debian/ unstable/non-free Translation-en_GB
Get:4 http://ftp.uk.debian.org sid Release [104kB]
Get:5 http://ftp.uk.debian.org experimental Release [102kB]
Get:6 http://ftp.uk.debian.org unstable Release [104kB]
Get:7 http://ftp.uk.debian.org sid/main Sources [4,074kB]
Get:8 http://ftp.uk.debian.org sid/contrib Sources [38.6kB]
Get:9 http://ftp.uk.debian.org sid/non-free Sources [83.6kB]
Get:10 http://ftp.uk.debian.org sid/main armel Packages [6,785kB]
Get:11 http://ftp.uk.debian.org sid/contrib armel Packages [48.8kB]
Get:12 http://ftp.uk.debian.org sid/non-free armel Packages [92.3kB]
Get:13 http://ftp.uk.debian.org experimental/main Sources [396kB]
Get:14 http://ftp.uk.debian.org experimental/contrib Sources [4,450B]
Get:15 http://ftp.uk.debian.org experimental/non-free Sources [6,418B]
Get:16 http://ftp.uk.debian.org experimental/main armel Packages [794kB]
Get:17 http://ftp.uk.debian.org experimental/contrib armel Packages [5,246B]
Get:18 http://ftp.uk.debian.org experimental/non-free armel Packages [6,108B]
Get:19 http://ftp.uk.debian.org unstable/main armel Packages [6,785kB]
Get:20 http://ftp.uk.debian.org unstable/contrib armel Packages [48.8kB]
Get:21 http://ftp.uk.debian.org unstable/non-free armel Packages [92.3kB]
Fetched 19.6MB in 22s (884kB/s)
Reading package lists... Done
apt-get  -o Apt::Get::Download-Only=true -y -o Apt::Architecture=armel
-o Apt::Install-Recommends=false -o Dir::Etc=/var/lib/xapt/etc/xapt/
-o Dir::Etc::TrustedParts=/etc/apt/trusted.gpg.d -o
Dir::Etc::Trusted=/etc/apt/trusted.gpg -o
Dir::Etc::SourceList=/var/lib/xapt/etc/xapt/sources.list -o
Dir::Etc::SourceParts=/var/lib/xapt/etc/xapt/sources.list.d/ -o
Dir::State=/var/lib/xapt/ -o
Dir::State::Status=/var/lib/xapt//armel/dpkg/status -o
Dir::Cache=/var/lib/xapt/ install hello
Reading package lists... Done
Building dependency tree... Done
The following extra packages will be installed:
  coreutils dpkg gcc-4.4-base libacl1 libattr1 libbz2-1.0 libc-bin
libc6 libgcc1 liblzma2 libselinux1 xz-utils zlib1g
Suggested packages:
  apt glibc-doc debconf debconf-2.0 locales xz-lzma
The following NEW packages will be installed:
  coreutils dpkg gcc-4.4-base hello libacl1 libattr1 libbz2-1.0
libc-bin libc6 libgcc1 liblzma2 libselinux1 xz-utils zlib1g
0 upgraded, 14 newly installed, 0 to remove and 0 not upgraded.
Need to get 12.7MB of archives.
After this operation, 34.0MB of additional disk space will be used.
Get:1 http://ftp.uk.debian.org/debian/ sid/main libc-bin armel 2.11.2-9 [702kB]
Get:2 http://ftp.uk.debian.org/debian/ sid/main gcc-4.4-base armel
4.4.5-10 [122kB]
Get:3 http://ftp.uk.debian.org/debian/ sid/main libc6 armel 2.11.2-9 [4,182kB]
Get:4 http://ftp.uk.debian.org/debian/ sid/main libgcc1 armel
1:4.4.5-10 [25.1kB]
Get:5 http://ftp.uk.debian.org/debian/ sid/main libattr1 armel
1:2.4.44-2 [11.7kB]
Get:6 http://ftp.uk.debian.org/debian/ sid/main libacl1 armel 2.2.49-4 [26.6kB]
Get:7 http://ftp.uk.debian.org/debian/ sid/main libselinux1 armel
2.0.96-1 [82.0kB]
Get:8 http://ftp.uk.debian.org/debian/ sid/main coreutils armel 8.5-1 [4,624kB]
Get:9 http://ftp.uk.debian.org/debian/ sid/main libbz2-1.0 armel
1.0.5-6 [49.7kB]
Get:10 http://ftp.uk.debian.org/debian/ sid/main zlib1g armel
1:1.2.3.4.dfsg-3 [77.5kB]
Get:11 http://ftp.uk.debian.org/debian/ sid/main liblzma2 armel 5.0.0-2 [184kB]
Get:12 http://ftp.uk.debian.org/debian/ sid/main xz-utils armel 5.0.0-2 [207kB]
Get:13 http://ftp.uk.debian.org/debian/ sid/main dpkg armel 1.15.8.8 [2,332kB]
Get:14 http://ftp.uk.debian.org/debian/ sid/main hello armel 2.6-1 [58.4kB]
Fetched 12.7MB in 14s (861kB/s)
Download complete and in download only mode
Building libacl1-armel-cross_2.2.49-4_all.deb
dpkg-deb: building package `libacl1-armel-cross' in
`./libacl1-armel-cross_2.2.49-4_all.deb'.
Building libgcc1-armel-cross_4.4.5-10_all.deb
dpkg-deb: building package `libgcc1-armel-cross' in
`./libgcc1-armel-cross_4.4.5-10_all.deb'.
Building libc6-armel-cross_2.11.2-9_all.deb
dpkg-deb: building package `libc6-armel-cross' in
`./libc6-armel-cross_2.11.2-9_all.deb'.
Building libbz2-1.0-armel-cross_1.0.5-6_all.deb
dpkg-deb: building package `libbz2-1.0-armel-cross' in
`./libbz2-1.0-armel-cross_1.0.5-6_all.deb'.
Building libselinux1-armel-cross_2.0.96-1_all.deb
dpkg-deb: building package `libselinux1-armel-cross' in
`./libselinux1-armel-cross_2.0.96-1_all.deb'.
Building libattr1-armel-cross_2.4.44-2_all.deb
dpkg-deb: building package `libattr1-armel-cross' in
`./libattr1-armel-cross_2.4.44-2_all.deb'.
dpkg-cross: package xz-utils doesn't provide any useful files, but
processing it anyway as requested
Building xz-utils-armel-cross_5.0.0-2_all.deb
dpkg-deb: building package `xz-utils-armel-cross' in
`./xz-utils-armel-cross_5.0.0-2_all.deb'.
dpkg-cross: package coreutils doesn't provide any useful files, but
processing it anyway as requested
Building coreutils-armel-cross_8.5-1_all.deb
dpkg-deb: building package `coreutils-armel-cross' in
`./coreutils-armel-cross_8.5-1_all.deb'.
dpkg-cross: package hello doesn't provide any useful files, but processing
it anyway as requested
Building hello-armel-cross_2.6-1_all.deb
dpkg-deb: building package `hello-armel-cross' in
`./hello-armel-cross_2.6-1_all.deb'.
Building zlib1g-armel-cross_1.2.3.4.dfsg-3_all.deb
dpkg-deb: building package `zlib1g-armel-cross' in
`./zlib1g-armel-cross_1.2.3.4.dfsg-3_all.deb'.
dpkg-cross: package libc-bin doesn't provide any useful files, but
processing it anyway as requested
Building libc-bin-armel-cross_2.11.2-9_all.deb
dpkg-deb: building package `libc-bin-armel-cross' in
`./libc-bin-armel-cross_2.11.2-9_all.deb'.
dpkg-cross: package dpkg doesn't provide any useful files, but processing
it anyway as requested
Building dpkg-armel-cross_1.15.8.8_all.deb
dpkg-deb: building package `dpkg-armel-cross' in
`./dpkg-armel-cross_1.15.8.8_all.deb'.
Building liblzma2-armel-cross_5.0.0-2_all.deb
dpkg-deb: building package `liblzma2-armel-cross' in
`./liblzma2-armel-cross_5.0.0-2_all.deb'.
dpkg-cross: package gcc-4.4-base doesn't provide any useful files, but
processing it anyway as requested
Building gcc-4.4-base-armel-cross_4.4.5-10_all.deb
dpkg-deb: building package `gcc-4.4-base-armel-cross' in
`./gcc-4.4-base-armel-cross_4.4.5-10_all.deb'.
Selecting previously deselected package coreutils-armel-cross.
(Reading database ... 32202 files and directories currently installed.)
Unpacking coreutils-armel-cross (from
.../coreutils-armel-cross_8.5-1_all.deb) ...
Selecting previously deselected package dpkg-armel-cross.
Unpacking dpkg-armel-cross (from .../dpkg-armel-cross_1.15.8.8_all.deb) ...
Selecting previously deselected package gcc-4.4-base-armel-cross.
Unpacking gcc-4.4-base-armel-cross (from
.../gcc-4.4-base-armel-cross_4.4.5-10_all.deb) ...
Selecting previously deselected package hello-armel-cross.
Unpacking hello-armel-cross (from .../hello-armel-cross_2.6-1_all.deb) ...
Selecting previously deselected package libacl1-armel-cross.
Unpacking libacl1-armel-cross (from
.../libacl1-armel-cross_2.2.49-4_all.deb) ...
Selecting previously deselected package libattr1-armel-cross.
Unpacking coreutils-armel-cross (from
.../coreutils-armel-cross_8.5-1_all.deb) ...
Selecting previously deselected package dpkg-armel-cross.
Unpacking dpkg-armel-cross (from .../dpkg-armel-cross_1.15.8.8_all.deb) ...
Selecting previously deselected package gcc-4.4-base-armel-cross.
Unpacking gcc-4.4-base-armel-cross (from
.../gcc-4.4-base-armel-cross_4.4.5-10_all.deb) ...
Selecting previously deselected package hello-armel-cross.
Unpacking hello-armel-cross (from .../hello-armel-cross_2.6-1_all.deb) ...
Selecting previously deselected package libacl1-armel-cross.
Unpacking libacl1-armel-cross (from
.../libacl1-armel-cross_2.2.49-4_all.deb) ...
Selecting previously deselected package libattr1-armel-cross.
Unpacking libattr1-armel-cross (from
.../libattr1-armel-cross_2.4.44-2_all.deb) ...
Selecting previously deselected package libbz2-1.0-armel-cross.
Unpacking libbz2-1.0-armel-cross (from
.../libbz2-1.0-armel-cross_1.0.5-6_all.deb) ...
Selecting previously deselected package libc-bin-armel-cross.
Unpacking libc-bin-armel-cross (from
.../libc-bin-armel-cross_2.11.2-9_all.deb) ...
Selecting previously deselected package libc6-armel-cross.
Unpacking libc6-armel-cross (from .../libc6-armel-cross_2.11.2-9_all.deb) ...
Selecting previously deselected package libgcc1-armel-cross.
Unpacking libgcc1-armel-cross (from
.../libgcc1-armel-cross_4.4.5-10_all.deb) ...
Selecting previously deselected package liblzma2-armel-cross.
Unpacking liblzma2-armel-cross (from
.../liblzma2-armel-cross_5.0.0-2_all.deb) ...
Selecting previously deselected package libselinux1-armel-cross.
Unpacking libselinux1-armel-cross (from
.../libselinux1-armel-cross_2.0.96-1_all.deb) ...
Selecting previously deselected package xz-utils-armel-cross.
Unpacking xz-utils-armel-cross (from
.../xz-utils-armel-cross_5.0.0-2_all.deb) ...
Selecting previously deselected package zlib1g-armel-cross.
Unpacking zlib1g-armel-cross (from
.../zlib1g-armel-cross_1.2.3.4.dfsg-3_all.deb) ...
Setting up gcc-4.4-base-armel-cross (4.4.5-10) ...
Setting up libc-bin-armel-cross (2.11.2-9) ...
Setting up libgcc1-armel-cross (1:4.4.5-10) ...
Setting up libc6-armel-cross (2.11.2-9) ...
Setting up liblzma2-armel-cross (5.0.0-2) ...
Setting up libselinux1-armel-cross (2.0.96-1) ...
Setting up xz-utils-armel-cross (5.0.0-2) ...
Setting up zlib1g-armel-cross (1:1.2.3.4.dfsg-3) ...
Setting up libattr1-armel-cross (1:2.4.44-2) ...
Setting up libbz2-1.0-armel-cross (1.0.5-6) ...
Setting up libacl1-armel-cross (2.2.49-4) ...
Setting up coreutils-armel-cross (8.5-1) ...
Setting up dpkg-armel-cross (1.15.8.8) ...
Setting up hello-armel-cross (2.6-1) ...


-- 
 Héctor Orón

"Our Sun unleashes tremendous flares expelling hot gas into the Solar
System, which one day will disconnect us."

-- Day DVB-T stop working nicely
Video flare: http://antwrp.gsfc.nasa.gov/apod/ap100510.html




Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#610689; Package sbuild. (Sat, 22 Jan 2011 23:51:06 GMT) Full text and rfc822 format available.

Acknowledgement sent to Roger Leigh <rleigh@codelibre.net>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Sat, 22 Jan 2011 23:51:06 GMT) Full text and rfc822 format available.

Message #52 received at 610689@bugs.debian.org (full text, mbox):

From: Roger Leigh <rleigh@codelibre.net>
To: Hector Oron <hector.oron@gmail.com>
Cc: 610689@bugs.debian.org, debian-embedded@lists.debian.org
Subject: Re: [buildd-tools-devel] Bug#610689: Bug#610689: sbuild: cross support
Date: Sat, 22 Jan 2011 23:49:08 +0000
[Message part 1 (text/plain, inline)]
On Sat, Jan 22, 2011 at 11:16:44PM +0000, Hector Oron wrote:
> Hi,
> 
> 2011/1/22 Roger Leigh <rleigh@codelibre.net>:
> 
> > I've attached a patch (against sbuild git master, also on the
> > cross-build branch of git://git.debian.org/users/rleigh/sbuild.git)
> > which adds --host and --build options and then uses the correct
> > host or build arch in the appropriate places.  While I've tested
> > everything still works fine for normal non-cross builds, this will
> > need testing for cross builds to make sure I've not got the
> > host/build arch mixed up in any places (since I will not have
> > picked any error up when they are both the same).
> 
> Thanks, that was quick...
> 
> From a quick glance, the package is missing automake (and autotools-dev)
> build dependency. It also fails to run bootstrap code...
> 
> make: *** No rule to make target `configure', needed by
> `debian/build/config.status'.  Stop.
> dpkg-buildpackage: error: debian/rules build gave error exit status 2
> 
> Maybe a
> configure:
>    ./bootstrap
> was forgotten on the debian/rules file?

No, this is intended.  The debian packaging builds from the release
tarball, not the plain git repo, which needs "make dist" running to
generate the release tarball (as for most upstream projects--it's not
Debian native).
If you run the top-level ./bootstrap from the git repository, it will
set things up and then autoreconf (it gets the version from NEWS and
the release metadata out of git).

> zumbi@cat:~$ sudo sbuild -d sid_amd64 hello_2.6-1
> Argument "amd64" isn't numeric in numeric ne (!=) at
> /usr/share/perl5/Sbuild/Build.pm line 1809.

Fixed in git on cross-build.

> Use of uninitialized value $ver in hash element at
> /usr/share/perl5/Sbuild/InternalResolver.pm line 558, <$pipe> line 7.

This one I'm not sure about yet; I'll take a better look tomorrow.
I didn't test with "internal" because it most likely can't be made
to work to install cross-build dependencies.

> When trying --host and --build options I get:
> zumbi@cat:~$ sudo sbuild -d sid_amd64 --host=armel --build=amd64 hello_2.6-1

As I mentioned to Loïc, I messed up and confused the host/build.
I'll fix up that and the other issues and then hopefully have a
better place to start testing!

The internal resolver errors look the same as without cross-building.


Many thanks for testing!  I'll get back to you with something better
as soon as I can.


Regards,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
   `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#610689; Package sbuild. (Thu, 10 Mar 2011 16:54:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Hector Oron <hector.oron@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Thu, 10 Mar 2011 16:54:03 GMT) Full text and rfc822 format available.

Message #57 received at 610689@bugs.debian.org (full text, mbox):

From: Hector Oron <hector.oron@gmail.com>
To: 610689@bugs.debian.org, Roger Leigh <rleigh@codelibre.net>
Cc: debian embedded <debian-embedded@lists.debian.org>
Subject: sbuild: cross build support
Date: Thu, 10 Mar 2011 16:52:01 +0000
Hello,

  I have a set of patches at:
    < http://git.debian.org/?p=users/zumbi/sbuild.git;a=shortlog;h=refs/heads/zumbi/cross-build
>

  We still do not know how to properly hook cross dependency
resolvers, but there are a couple of options (embuilddeps and xapt)
provided by `xapt' package.

  Basically I am doing the following:

  Use git to clone the zumbi/cross-build branch
    git clone git://git.debian.org/git/users/zumbi/sbuild.git -b
zumbi/cross-build
    cd sbuild
    ./bootstrap
    debuild
    sudo dpkg -i ../*.deb

  Create a chroot
    sudo sbuild-createchroot sid /srv/chroot/sid-amd64-sbuild
http://ftp.uk.debian.org/debian

  Add new user
    sudo sbuild-adduser zumbi

  Copy sbuildrc configuration file
    cp /usr/share/doc/sbuild/examples/example.sbuildrc /home/zumbi/.sbuildrc

  Follow the instructions to update and upgrade
    cd /path/to/source
    sbuild-update <distribution>
    sbuild-upgrade <distribution> apt-get upgrade
    (or "sbuild-apt <distribution> apt-get -f install"
        first if the chroot is broken)
    sbuild -d <distribution> <package>_<version>

  Install cross toolchains in the chroot
    schroot -u root -c /srv/chroot/sid-amd64-sbuild
    echo "deb http://emdebian.org/debian squeeze main" >>
/etc/apt/sources.list.d/emdebian.list
    apt-get update
    apt-get install linux-libc-dev-armel-cross libc6-dev-armel-cross \
        gcc-4.3-arm-linux-gnueabi g++-4.3-arm-linux-gnueabi

  Having made a chroot called sid-amd64-sbuild build a package with
    sbuild -d sid-amd64-sbuild hello_2.6-1

  Cross build that same package with
    sbuild -d sid-amd64-sbuild hello_2.6-1 --host=armel --build=i386

  Resolve cross dependencies calling
    sbuild -d sid-amd64-sbuild --host=armel --build=amd64
--build-dep-resolver=xapt mc_4.7.0.9-1.dsc

 Issues I am having...
  1. schroot sessions do not end, so I end up having a bunch of sessions opened
  2. xapt resolver is not doing its job, alternatively embuilddeps
could be used.

  Brief explanation on cross resolvers:
  1. xapt
     `xapt' should work if a list of packages is passed in its command line:
       xapt -a $HOST_ARCH $PACKAGE_LIST
  2. embuilddeps
     `embuilddeps' parses control file information to extract the
packages list, then passes this list to xapt
     Usually `embuilddeps' need to be run under unpacked package
directory: embuilddeps -a $HOST_ARCH
     Neil Williams, `embuilddeps' maintainer has updated this package
to allow -d option, so one can tell directly to embuilddeps where is
the unpacked package to resolve dependencies.
     I.e.,
       `embuilddeps -a $HOST_ARCH -d /path/to/unpacked/directory'

  It would be really nice if we enhance sbuild with cross support.

Best regards and thanks for your support,
-- 
 Héctor Orón

"Our Sun unleashes tremendous flares expelling hot gas into the Solar
System, which one day will disconnect us."

-- Day DVB-T stop working nicely
Video flare: http://antwrp.gsfc.nasa.gov/apod/ap100510.html




Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#610689; Package sbuild. (Thu, 10 Mar 2011 21:45:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Roger Leigh <rleigh@codelibre.net>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Thu, 10 Mar 2011 21:45:03 GMT) Full text and rfc822 format available.

Message #62 received at 610689@bugs.debian.org (full text, mbox):

From: Roger Leigh <rleigh@codelibre.net>
To: Hector Oron <hector.oron@gmail.com>
Cc: 610689@bugs.debian.org, debian embedded <debian-embedded@lists.debian.org>
Subject: Re: sbuild: cross build support
Date: Thu, 10 Mar 2011 21:39:58 +0000
[Message part 1 (text/plain, inline)]
On Thu, Mar 10, 2011 at 04:52:01PM +0000, Hector Oron wrote:
> Hello,
> 
>   I have a set of patches at:
>     < http://git.debian.org/?p=users/zumbi/sbuild.git;a=shortlog;h=refs/heads/zumbi/cross-build
> >
> 
>   We still do not know how to properly hook cross dependency
> resolvers, but there are a couple of options (embuilddeps and xapt)
> provided by `xapt' package.
> 
>   Basically I am doing the following:

[...]
The first part all looks good.

>   Install cross toolchains in the chroot
>     schroot -u root -c /srv/chroot/sid-amd64-sbuild
>     echo "deb http://emdebian.org/debian squeeze main" >>
> /etc/apt/sources.list.d/emdebian.list
>     apt-get update
>     apt-get install linux-libc-dev-armel-cross libc6-dev-armel-cross \
>         gcc-4.3-arm-linux-gnueabi g++-4.3-arm-linux-gnueabi

This part can be done by the xapt resolver directly, without any
special user setup required.

If you look at the apt/aptitude resolvers, you'll see they add a
local apt source by creating a file in /etc/apt/sources.list.d/
too.  Providing you add a configuration variable for the line so
it's not hardcoded, this would be fine.  It will then update at
the normal update/distupgrade stage, so long as you set up the
apt source in the resolver setup() function (and delete it in the
cleanup()).  You'll then want to force the APT_UPDATE configuration
variable to 1 to make it update.

For the cross-packages, you can add these to the core build
dependencies list, so they will be installed along with build-essential
etc.  Or (better) you can add a separate XAPT_DEPENDS variable which can
be appended to CORE_DEPENDS in Build.pm prior to installing the core
dependencies.  Does xapt not bring in these packages directly, or do
they always need installing by hand?

>   Having made a chroot called sid-amd64-sbuild build a package with
>     sbuild -d sid-amd64-sbuild hello_2.6-1
> 
>   Cross build that same package with
>     sbuild -d sid-amd64-sbuild hello_2.6-1 --host=armel --build=i386
> 
>   Resolve cross dependencies calling
>     sbuild -d sid-amd64-sbuild --host=armel --build=amd64
> --build-dep-resolver=xapt mc_4.7.0.9-1.dsc
> 
>  Issues I am having...
>   1. schroot sessions do not end, so I end up having a bunch of sessions opened

Does this always happen, or just with --build-dep-resolver=xapt?
If you add --verbose to the schroot options ($schroot_options=['--verbose'])
does this tell you anything about why it's not deleting them?
Note: $end_session=0 will deliberately make sbuild not end sessions;
is this set?

>   2. xapt resolver is not doing its job, alternatively embuilddeps
> could be used.
> 
>   Brief explanation on cross resolvers:
>   1. xapt
>      `xapt' should work if a list of packages is passed in its command line:
>        xapt -a $HOST_ARCH $PACKAGE_LIST
>   2. embuilddeps
>      `embuilddeps' parses control file information to extract the
> packages list, then passes this list to xapt
>      Usually `embuilddeps' need to be run under unpacked package
> directory: embuilddeps -a $HOST_ARCH
>      Neil Williams, `embuilddeps' maintainer has updated this package
> to allow -d option, so one can tell directly to embuilddeps where is
> the unpacked package to resolve dependencies.
>      I.e.,
>        `embuilddeps -a $HOST_ARCH -d /path/to/unpacked/directory'

Well, when we install the build dependencies the package isn't yet
unpacked.  The unpacking is done in the build() function.  But there's
no reason we can't reorder it if we need to.  The unpacking could
be split out of build() into a separate unpack_source() function,
which could be run earlier if xapt/embuilddeps is in use.

What information does embuilddeps require from the unpacked source
package?  Is this information also available in the .dsc? (We do
have the .dsc information directly to hand).

xapt is doing two jobs it seems:
1) from the package list, make a list containing all the cross packages
   needed
2) install them (but it doesn't list them in the list of packages to
   install, which makes removal harder potentially)

From the resolver POV, what would be really good if it could just do
(1).  This way, we can just give the expanded list back to the
resolver for it to install.  Because we need to clean up after
ourselves, the apt and aptitude resolvers build a dummy
"dependency package" from the dependency list, and then get apt/
aptitude to install it.  We then reverse the process to clean up
after the build.  So ideally, we don't really want xapt to do any
installation: we just want it to give us the package list.  If
this is possible, it might not even be require to have a separate
XaptResolver--we just expand the cross-dependencies and give them
to the regular resolver.

If xapt can't do this directly, how complex is the code that
implements (1)?  Would it be possible to split it out as a separate
tool?

>   It would be really nice if we enhance sbuild with cross support.

Absolutely.

I've rebased your patches against current sbuild.git master, and
they may be found here: git://git.debian.org/users/rleigh/sbuild.git



Regards,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
   `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#610689; Package sbuild. (Fri, 25 Mar 2011 17:09:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Hector Oron <hector.oron@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Fri, 25 Mar 2011 17:09:03 GMT) Full text and rfc822 format available.

Message #67 received at 610689@bugs.debian.org (full text, mbox):

From: Hector Oron <hector.oron@gmail.com>
To: Roger Leigh <rleigh@codelibre.net>, 610689@bugs.debian.org
Cc: debian embedded <debian-embedded@lists.debian.org>
Subject: Re: Bug#610689: sbuild: cross build support
Date: Fri, 25 Mar 2011 17:06:02 +0000
Hi Roger,

A bit late, but I have not found the time before to look into this.

2011/3/10 Roger Leigh <rleigh@codelibre.net>:

> [...]
> The first part all looks good.

Great!

>>   Install cross toolchains in the chroot
>>     schroot -u root -c /srv/chroot/sid-amd64-sbuild
>>     echo "deb http://emdebian.org/debian squeeze main" >>
>> /etc/apt/sources.list.d/emdebian.list
>>     apt-get update
>>     apt-get install linux-libc-dev-armel-cross libc6-dev-armel-cross \
>>         gcc-4.3-arm-linux-gnueabi g++-4.3-arm-linux-gnueabi
>
> This part can be done by the xapt resolver directly, without any
> special user setup required.

True.

> For the cross-packages, you can add these to the core build
> dependencies list, so they will be installed along with build-essential
> etc.  Or (better) you can add a separate XAPT_DEPENDS variable which can
> be appended to CORE_DEPENDS in Build.pm prior to installing the core
> dependencies.  Does xapt not bring in these packages directly, or do
> they always need installing by hand?

xapt downloads the foreign architecture packages using APT with -o
APT::Architecture=$HOST_ARCH into a directory controlled by xapt
(/var/lib/xapt/, irrc), then throws dpkg-cross into that directory to
convert those packages and installs them.

> Well, when we install the build dependencies the package isn't yet
> unpacked.  The unpacking is done in the build() function.  But there's
> no reason we can't reorder it if we need to.  The unpacking could
> be split out of build() into a separate unpack_source() function,
> which could be run earlier if xapt/embuilddeps is in use.
>
> What information does embuilddeps require from the unpacked source
> package?  Is this information also available in the .dsc? (We do
> have the .dsc information directly to hand).

It is possible to use .dsc information, but embuilddeps does not know
how (yet). :-)

> xapt is doing two jobs it seems:
> 1) from the package list, make a list containing all the cross packages
>   needed

That is given with simulation option (-n).

> 2) install them (but it doesn't list them in the list of packages to
>   install, which makes removal harder potentially)
>
> From the resolver POV, what would be really good if it could just do
> (1).  This way, we can just give the expanded list back to the
> resolver for it to install.  Because we need to clean up after
> ourselves, the apt and aptitude resolvers build a dummy
> "dependency package" from the dependency list, and then get apt/
> aptitude to install it.

This is how old pbuilder used to do it, but it does not work with
cross packages as adding dependencies on
$foreign_package-$HOST_ARCH-cross won't work, as those packages are
output from dpkg-cross and not available from repositories.

> We then reverse the process to clean up
> after the build.  So ideally, we don't really want xapt to do any
> installation: we just want it to give us the package list.  If
> this is possible, it might not even be require to have a separate
> XaptResolver--we just expand the cross-dependencies and give them
> to the regular resolver.
>
> If xapt can't do this directly, how complex is the code that
> implements (1)?  Would it be possible to split it out as a separate
> tool?

embuilddeps -a armel -d /path/to/source -n will give you that
information, and /path/to/dsc might be in the way.
We really need to think more about how we are going to do it to be
able to fit multiarch in the cooking.

> I've rebased your patches against current sbuild.git master, and
> they may be found here: git://git.debian.org/users/rleigh/sbuild.git

I tried to work on top of that, but it did not build for me.

I'll be looking into how to fit pieces together for cross and
multiarch support. Let you know once I get something.

Best regards,
-- 
 Héctor Orón

"Our Sun unleashes tremendous flares expelling hot gas into the Solar
System, which one day will disconnect us."

-- Day DVB-T stop working nicely
Video flare: http://antwrp.gsfc.nasa.gov/apod/ap100510.html




Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#610689; Package sbuild. (Sat, 26 Mar 2011 20:09:06 GMT) Full text and rfc822 format available.

Acknowledgement sent to Roger Leigh <rleigh@codelibre.net>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Sat, 26 Mar 2011 20:09:06 GMT) Full text and rfc822 format available.

Message #72 received at 610689@bugs.debian.org (full text, mbox):

From: Roger Leigh <rleigh@codelibre.net>
To: Hector Oron <hector.oron@gmail.com>, 610689@bugs.debian.org
Cc: debian embedded <debian-embedded@lists.debian.org>
Subject: Re: [buildd-tools-devel] Bug#610689: sbuild: cross build support
Date: Sat, 26 Mar 2011 20:06:21 +0000
[Message part 1 (text/plain, inline)]
On Fri, Mar 25, 2011 at 05:06:02PM +0000, Hector Oron wrote:
> 2011/3/10 Roger Leigh <rleigh@codelibre.net>:
> 
> > This part can be done by the xapt resolver directly, without any
> > special user setup required.
> 
> True.
> 
> > For the cross-packages, you can add these to the core build
> > dependencies list, so they will be installed along with build-essential
> > etc.  Or (better) you can add a separate XAPT_DEPENDS variable which can
> > be appended to CORE_DEPENDS in Build.pm prior to installing the core
> > dependencies.  Does xapt not bring in these packages directly, or do
> > they always need installing by hand?
> 
> xapt downloads the foreign architecture packages using APT with -o
> APT::Architecture=$HOST_ARCH into a directory controlled by xapt
> (/var/lib/xapt/, irrc), then throws dpkg-cross into that directory to
> convert those packages and installs them.

This is the part that I think can be significantly improved.

sbuild creates a local APT archive to install dependency packages from.
We can use the same mechanism to allow the dpkg-cross packages to be
installed.  It could even be used to cache generated packages between
builds.  We can quite easily refactor the code to allow multiple
archives (if required).

> > Well, when we install the build dependencies the package isn't yet
> > unpacked.  The unpacking is done in the build() function.  But there's
> > no reason we can't reorder it if we need to.  The unpacking could
> > be split out of build() into a separate unpack_source() function,
> > which could be run earlier if xapt/embuilddeps is in use.
> >
> > What information does embuilddeps require from the unpacked source
> > package?  Is this information also available in the .dsc? (We do
> > have the .dsc information directly to hand).
> 
> It is possible to use .dsc information, but embuilddeps does not know
> how (yet). :-)

OK.  For the purpose of actually implementing this, I would like to
have a description of exactly which fields we need for cross building
from which files in the source or DSC, and what they do and why.  Is
this all documented somewhere?

> > xapt is doing two jobs it seems:
> > 1) from the package list, make a list containing all the cross packages
> >   needed
> 
> That is given with simulation option (-n).

This is with embuilddeps, not xapt?

This option appears to give a package list, prefixed with
"/usr/bin/apt-get -y".  This just isn't suitable.
- it's omitted packages which were already installed; this is not
  necessary and involves poking around under /var/lib/dpkg.
- we need to strip off the "/usr/bin/apt-get -y" which might be subject
  to change and then cause breakage, at some point in the future.
All we want is the /complete/ package list, without any removal of
packages or addition of other bits.

> > 2) install them (but it doesn't list them in the list of packages to
> >   install, which makes removal harder potentially)
> >
> > From the resolver POV, what would be really good if it could just do
> > (1).  This way, we can just give the expanded list back to the
> > resolver for it to install.  Because we need to clean up after
> > ourselves, the apt and aptitude resolvers build a dummy
> > "dependency package" from the dependency list, and then get apt/
> > aptitude to install it.
> 
> This is how old pbuilder used to do it, but it does not work with
> cross packages as adding dependencies on
> $foreign_package-$HOST_ARCH-cross won't work, as those packages are
> output from dpkg-cross and not available from repositories.

Right.  But with sbuild, as I mentioned above, we can install the
dpkg-cross-generated packages into a local archive, which *is*
accessible to apt-get/aptitude, and hence you will then be able to
use cross packages in the dependencies /directly/.  This will permit
proper cleanup after a build using the normal mechanisms sbuild uses
for all builds.

This is to say the resolver can do the following:
- get the cross dependencies from the dsc or unpacked source
- download the debs
- run dpkg-cross
- install the dpkg-cross-generated debs into the local archive
- add the cross-dependencies to the package build dependencies
...
- run dpkg-buildpackage with the necessary options to cross build
...

We already allow various things to add to the package build deps,
and so adding cross-deps will be trivial.  All we really need is the
necessary information for what to parse from the dsc or source package,
and then how to use dpkg-cross.

Just to be clear: none of the above uses xapt/embuilddeps, because
sbuild can, I think, do a better job with its local archive and which
will integrate much better with our existing cleanup code when it comes
to removing build deps.

> > We then reverse the process to clean up
> > after the build.  So ideally, we don't really want xapt to do any
> > installation: we just want it to give us the package list.  If
> > this is possible, it might not even be require to have a separate
> > XaptResolver--we just expand the cross-dependencies and give them
> > to the regular resolver.
> >
> > If xapt can't do this directly, how complex is the code that
> > implements (1)?  Would it be possible to split it out as a separate
> > tool?
> 
> embuilddeps -a armel -d /path/to/source -n will give you that
> information, and /path/to/dsc might be in the way.
> We really need to think more about how we are going to do it to be
> able to fit multiarch in the cooking.

As I mentioned above, I really don't like this.  It's not giving us the
unfiltered information we need.  And it's telling us nothing about what
xapt is actually going to install (in terms of letting us reverse the
procedure).  And, it's also in a format that requires parsing to strip
out all the unwanted bits in order to give us what we want.

embuilddeps is giving us command lines to do the job.  What we need is
the information to do that job ourselves.

> > I've rebased your patches against current sbuild.git master, and
> > they may be found here: git://git.debian.org/users/rleigh/sbuild.git
> 
> I tried to work on top of that, but it did not build for me.

Details?

> I'll be looking into how to fit pieces together for cross and
> multiarch support. Let you know once I get something.

What I'll do next is refactor the needed bits so that we can unpack the
source tree and get at the cross build deps.  Right now this isn't
possible because we get that from the dsc.

For getting things merged onto the master branch, having discrete
changes that don't break non-cross building will make it easier to merge;
rebasing it against git master will also make it easier to review.


Thanks,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
   `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#610689; Package sbuild. (Thu, 14 Apr 2011 09:57:14 GMT) Full text and rfc822 format available.

Acknowledgement sent to Neil Williams <codehelp@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Thu, 14 Apr 2011 09:57:19 GMT) Full text and rfc822 format available.

Message #77 received at 610689@bugs.debian.org (full text, mbox):

From: Neil Williams <codehelp@debian.org>
To: Debian Embedded <debian-embedded@lists.debian.org>
Cc: 610689@bugs.debian.org, Roger Leigh <rleigh@codelibre.net>
Subject: Re: [buildd-tools-devel] Bug#610689: sbuild: cross build support
Date: Thu, 14 Apr 2011 10:52:57 +0100
[Message part 1 (text/plain, inline)]
On Sat, 26 Mar 2011 20:06:21 +0000
Roger Leigh <rleigh@codelibre.net> wrote:

> On Fri, Mar 25, 2011 at 05:06:02PM +0000, Hector Oron wrote:
> > 2011/3/10 Roger Leigh <rleigh@codelibre.net>:
> > > For the cross-packages, you can add these to the core build
> > > dependencies list, so they will be installed along with build-essential
> > > etc.  Or (better) you can add a separate XAPT_DEPENDS variable which can
> > > be appended to CORE_DEPENDS in Build.pm prior to installing the core
> > > dependencies.  Does xapt not bring in these packages directly, or do
> > > they always need installing by hand?
> > 
> > xapt downloads the foreign architecture packages using APT with -o
> > APT::Architecture=$HOST_ARCH into a directory controlled by xapt
> > (/var/lib/xapt/, irrc), then throws dpkg-cross into that directory to
> > convert those packages and installs them.
> 
> This is the part that I think can be significantly improved.

Not until we have Multi-Arch-Cross. Indeed, there's no point making
special changes to sbuild ahead of Multi-Arch as the entire
cross-dependency resolution mechanism will be completely thrown away at
that stage.

What you're trying to create in this bug report is a Multi-Arch-Cross
resolver for sbuild and then hacking it to handle dpkg-cross. That is
the wrong way around. Use the hacks which already work with the
dpkg-cross hacks and then implement a *clean* Multi-Arch-Cross resolver
when that becomes possible.

dpkg-cross is fundamentally broken and is completely alien to
Multi-Arch. Anything based on dpkg-cross will need to be completely
re-written for Multi-Arch-Cross, so use the tools which exist (and
which will disappear alongside dpkg-cross) to retain as much isolation
as possible between the sbuild code and the broken dpkg-cross methods.

> sbuild creates a local APT archive to install dependency packages from.
> We can use the same mechanism to allow the dpkg-cross packages to be
> installed.  It could even be used to cache generated packages between
> builds.  We can quite easily refactor the code to allow multiple
> archives (if required).

The problems in this area are:

0: The setup_apt_archive code uses the Dpkg bindings to resolve the
dependencies but these are inherently native (despite taking an arch,
it is difficult to get this right when there are alternatives and
architecture-specific dependencies). These have been re-written in
embuilddeps but all that code will be completely pointless once
Multi-Arch-Cross arrives. There is no point implementing yet another
broken version of it in an XaptResolver for sbuild.

1: embuilddeps has tried to solve these problems but has a
fundamentally different approach to actually installing the packages.
Sbuild::AptResolver and Sbuild::AptitudeResolver use the dummy-package
method to actually do the work of resolving the dependencies within the
specified constraints. embuilddeps resolves the constraints directly.
e.g. constraints of foo (>= 1.2.3) [armel] | bar [!linux-any] | baz
[i386| amd64] are particularly hard to resolve correctly when running
on i386 to cross-build for armel.

2: Cross-dependency resolution requires re-doing the dpkg deps_parse
calculation with an empty dpkg status file and then chasing the
dependencies all the way down because -cross packages do not exist in
the archives. This is why I'm looking at Multi-Arch being the preferred
solution here.

3: dummy-package methods won't work for cross because we must resolve
all the constraints ahead of passing a list of packages to apt so that
these can be downloaded *ahead* of being installed, then converted
using dpkg-cross and installed as packagename-$arch-cross packages.
libfoo-dev_1.2.3_armel.deb becomes libfoo-dev-armel-cross_1.2.3_all.deb

4: The converted packages do not (indeed cannot) exist in any
network-accessible archive - these are all generated on-the-fly until
Multi-Arch makes the whole conversion avoidable.

5: apt and aptitude consistently treat -$arch-cross packages as "local
or obsolete" and consistently get removal parsing wrong when upgrading
existing packages. This means that the list of -$arch-cross packages to
be removed needs to be carefully managed, especially if the cross
toolchain is not to be removed upon every build.

Much easier is for xapt|embuilddeps to put the downloaded and the
converted packages into a directory which sbuild can manage, or for
sbuild to become aware of the current defaults which
are /var/lib/xapt/output for -cross packages and /var/lib/xapt/archives
for the foreign architecture originals.

This will also solve problems of package removal after the build,
albeit that some extra processing is required to skip certain
-cross packages which would otherwise cause the removal of the
cross-building toolchain. (libc6-dev-$arch-cross is a frequent problem
here.) This list of packages does need to be kept updated to cope with
changes in gcc and in eglibc - e.g. the Squeeze chroots will have a
libc-bin-$ARCH-cross package which doesn't exist in Lenny chroots.

Whilst it is certainly a good idea to try and re-use the existing
sbuild support for keeping the downloaded packages available, there are
significant problems with doing this the way that sbuild currently does
it. It would need to be a post-process update, reading in the list from
the xapt directory. (xapt can be told to keep the archives around, it
usually deletes them once all the -cross packages are installed and
this option has also been added to embuilddeps, so the new -k option
passes that down to xapt which keeps the /var/lib/xapt/ tree in place
for subsequent handling.)

I did think about putting that list into a file but the list is simply
the list of .debs which were downloaded into a clean directory - one
directory for the (typically) armel packages and one for the
-armel-cross packages. A simple opendir(); readdir and pattern match
on /^(.*)_.*_($arch|all)\.deb$/ or similar would get the list of package
names and if /var/lib/xapt/* is removed by whatever does the
subsequent package removal handling, then this list will be guaranteed
to be the complete and exact list of all dependencies downloaded as
$host architecture and installed as -$host-cross packages for that
specific build.

Overall, sbuild cross-build support ahead of Multi-Arch-Cross is best
done by leaving as much of the work as possible to tools outside sbuild
- tools which themselves will disappear once Multi-Arch-Cross is in
place. Only at that stage is it worth looking at how the internals of
sbuild can enfold the needs of the Multi-Arch-Cross world which,
currently, is largely opaque.

> > > Well, when we install the build dependencies the package isn't yet
> > > unpacked.  The unpacking is done in the build() function.  But there's
> > > no reason we can't reorder it if we need to.  The unpacking could
> > > be split out of build() into a separate unpack_source() function,
> > > which could be run earlier if xapt/embuilddeps is in use.
> > >
> > > What information does embuilddeps require from the unpacked source
> > > package?  Is this information also available in the .dsc? (We do
> > > have the .dsc information directly to hand).
> > 
> > It is possible to use .dsc information, but embuilddeps does not know
> > how (yet). :-)

It will - in the next release. It's as easy to parse debian/control as
the .dsc. I've got a change in SVN which adds a --dsc option to
embuilddeps.

So sbuild would actually use:
embuilddeps -a $host_arch -k --dsc $dsc_file

No use of -n, no parsing of the output - let embuilddeps call xapt to
download the packages and dpkg-cross to convert and install them, then
parse the list of downloaded and converted packages directly from the
directory listing, handle the removal of the $host-cross converted
packages when appropriate and then tidy up /var/lib/xapt/*

Once we have Multi-Arch-Cross, the entire process folds back into a
standard sbuild/apt|aptitude problem because M-A-C will *have to* allow
dpkg to fix the problems of foreign architecture dependency resolution.
(And it will do a better job of it than either sbuild or embuilddeps
could do themselves.)

> OK.  For the purpose of actually implementing this, I would like to
> have a description of exactly which fields we need for cross building
> from which files in the source or DSC, and what they do and why.  Is
> this all documented somewhere?

There was an idea to separate the "necessary" build depends from the
"ancillary" build depends - the tools like debhelper and automake which
need to be installed but also need to be executable, so need to be the
build arch, not host. This was the xcontrol debacle and it's dead.

Instead, embuilddeps parses the entirety of Build-Depends and
Build-Depends-Indep as well as Build-Depends-Conflicts and resolves
them all for the host architecture. This results in tools like
debhelper being converted by dpkg-cross into empty
debhelper-armel-cross packages but that can't be helped. (We tried to
do that with apt-cross and it resulted in an unfixable RC bug and the
removal of apt-cross.)

It's another reason why this needs to be done properly via
Multi-Arch-Cross, eventually.

So the fields are exactly as per Policy - albeit that the parsing of
those fields for a $host architecture is not currently supported
properly by dpkg or apt.

> > > xapt is doing two jobs it seems:
> > > 1) from the package list, make a list containing all the cross packages
> > >   needed
> > 
> > That is given with simulation option (-n).
> 
> This is with embuilddeps, not xapt?

embuilddeps parses the debian/control file or the .dsc to get a list of
packages. That list is then passed to xapt which downloads the host
architecture binaries, passes those to dpkg-cross which converts and
installs the -cross packages.
 
> This option appears to give a package list, prefixed with
> "/usr/bin/apt-get -y".  This just isn't suitable.
> - it's omitted packages which were already installed; this is not
>   necessary and involves poking around under /var/lib/dpkg.
> - we need to strip off the "/usr/bin/apt-get -y" which might be subject
>   to change and then cause breakage, at some point in the future.
> All we want is the /complete/ package list, without any removal of
> packages or addition of other bits.

The list output by the -n|--simulate option does NOT include any of the
dependencies of the packages listed. The complete package list needs to
be parsed from the listing of /var/lib/xapt/archives (for the
downloaded host architecture packages) and /var/lib/xapt/output/ for
the -$arch-cross packages converted by dpkg-cross.

> > > 2) install them (but it doesn't list them in the list of packages to
> > >   install, which makes removal harder potentially)

See the directory listing.

> > > From the resolver POV, what would be really good if it could just do
> > > (1).  This way, we can just give the expanded list back to the
> > > resolver for it to install. 

The resolver can't handle the conversion of the package name from foo
to foo-armel-cross and the resulting changes in the dependencies from
Depends: bar (>= 1.2.3) to Depends: bar-armel-cross (>= 1.2.3)

No resolver *can* handle that (current, past or future) - this is why we
need Multi-Arch-Cross. xapt doesn't try to handle it. xapt simply gets
the *unconverted* build-dependencies and downloads them, dpkg-cross
converts them and installs. The entire process is based on simply
converting and installing everything that appears to be a .deb
in /var/lib/xapt/archives/ and putting the converted stuff
into /var/lib/xapt/output prior to using dpkg -i *.

> Because we need to clean up after
> > > ourselves, the apt and aptitude resolvers build a dummy
> > > "dependency package" from the dependency list, and then get apt/
> > > aptitude to install it.
> > 
> > This is how old pbuilder used to do it, but it does not work with
> > cross packages as adding dependencies on
> > $foreign_package-$HOST_ARCH-cross won't work, as those packages are
> > output from dpkg-cross and not available from repositories.
> 
> Right.  But with sbuild, as I mentioned above, we can install the
> dpkg-cross-generated packages into a local archive, which *is*
> accessible to apt-get/aptitude, and hence you will then be able to
> use cross packages in the dependencies /directly/.  This will permit
> proper cleanup after a build using the normal mechanisms sbuild uses
> for all builds.

You can copy the converted packages in /var/lib/xapt/output into an
archive, yes. Or you could just use the packages which are already
installed and parse the directory listing to get the list to remove -
always remembering to skip $host-cross packages which would otherwise
cause the removal of the toolchain. This hack too will disappear with
M-A-C.

> This is to say the resolver can do the following:
> - get the cross dependencies from the dsc or unpacked source
> - download the debs
> - run dpkg-cross
> - install the dpkg-cross-generated debs into the local archive
> - add the cross-dependencies to the package build dependencies
> ...
> - run dpkg-buildpackage with the necessary options to cross build
> ...
> 
> We already allow various things to add to the package build deps,
> and so adding cross-deps will be trivial.  All we really need is the
> necessary information for what to parse from the dsc or source package,
> and then how to use dpkg-cross.
> 
> Just to be clear: none of the above uses xapt/embuilddeps, because
> sbuild can, I think, do a better job with its local archive and which
> will integrate much better with our existing cleanup code when it comes
> to removing build deps.

It will have to use embuilddeps because Dpkg::Deps.pm is *not able* to
resolve the build-dependencies for a different $host architecture.

It will need to use either xapt or something almost identical to xapt
(which is not much more than a wrapper to apt-get -o
Apt::get::Architecture=armel -d install $packages which sbuild would
have to do anyway). All that's being changed is the final path. It's
embuilddeps which is the attempt at being clever and resolving
dependencies for a foreign architecture using modifications to
Dpkg::Deb.pm.

Doing this without xapt and embuilddeps is just going to cause more
work because dpkg-cross IS going to be removed and the entire process
will have to be re-written.

> > > We then reverse the process to clean up
> > > after the build.  So ideally, we don't really want xapt to do any
> > > installation: we just want it to give us the package list.  If
> > > this is possible, it might not even be require to have a separate
> > > XaptResolver--we just expand the cross-dependencies and give them
> > > to the regular resolver.

No. There is no point re-doing all the work encoded within xapt and
embuilddeps and then doing it all again when Multi-Arch-Cross arrives.

Do it once - when M-A-C is usable.

> > > If xapt can't do this directly, how complex is the code that
> > > implements (1)?  Would it be possible to split it out as a separate
> > > tool?

By the time you've got the list of converted packages, there is nothing
for the resolver to do except dpkg -i *

> > embuilddeps -a armel -d /path/to/source -n will give you that
> > information, and /path/to/dsc might be in the way.

That's meant to be human-readable, not machine-parseable.

> > We really need to think more about how we are going to do it to be
> > able to fit multiarch in the cooking.

Exactly - use xapt/embuilddeps and then handle M-A-C later.
 
> As I mentioned above, I really don't like this.  It's not giving us the
> unfiltered information we need. 

The unfiltered information you need is simply the directory listing
of /var/lib/xapt/archives and /var/lib/xapt/output

> And it's telling us nothing about what
> xapt is actually going to install (in terms of letting us reverse the
> procedure). 

Those directory listing specify *exactly* what xapt has downloaded and
what dpkg-cross has converted and installed.

> embuilddeps is giving us command lines to do the job.  What we need is
> the information to do that job ourselves.

No. The information needs to come from the results of xapt and
embuilddeps.

-- 


Neil Williams
=============
http://www.linux.codehelp.co.uk/

[Message part 2 (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#610689; Package sbuild. (Wed, 07 Dec 2011 20:21:08 GMT) Full text and rfc822 format available.

Acknowledgement sent to Wookey <wookey@wookware.org>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Wed, 07 Dec 2011 20:21:09 GMT) Full text and rfc822 format available.

Message #82 received at 610689@bugs.debian.org (full text, mbox):

From: Wookey <wookey@wookware.org>
To: 610689@bugs.debian.org
Subject: sbuild: cross support
Date: Wed, 7 Dec 2011 20:17:27 +0000
I'm looking at adding multiarch-style cross-building support to sbuild.

I've taken the work done in this bugrep so far (thanks to hector and
Roger for that), which essentially makes sbuild BUILD/HOST aware andd
the xapt resolver, and moved that forward to the current 0.62.5 sbuild
release.

However, the procedure for multiarch crossbuilding is not the same as
for xapt+dpkg-cross, so we need some slightly different changes.

Essentially what needs to happen is:

0) get a chroot containing the usual build-essential stuff

1) Install cross-tools. (A build-essential-$arch package would be
useful for this, to gloss over variations between distros and changes
over time, but in the meantime we know what want and can add the
packages to the config in $core-depends)
this means: g++-arm-linux-gnueabi libc6-dev-armel-cross

1b) ensure that arm-linux-gnueabi-pkg-config wrapper link exists (will
come with toolchain if toolchain is new enough)

1c) Add qemu-static support to chroot. Best done by schroot?

2) Configure dpkg for multiarch:
add 
foreign-architecture armel
to /etc/dpkg/dpkg.cfg.d/multiarch

3) configure a $arch source for apt: in /etc/apt/sources.list.d/foo
deb [arch=armel] http://ports.ubuntu.com/ubuntu-ports oneiric main universe

4) to resolve the dependencies we need to run 
apt-get build-dep -aarmel $package 
instead of 
apt-get build-dep $package

5) Do the build by setting some environment variables:
'CONFIG_SITE' => '/etc/dpkg-cross/cross-config.$host_arch'
'DEB_BUILD_OPTIONS' => 'nocheck'
'DH_VERBOSE' => '1'
(easily done by setting $build_environment in config)
and run dpkg-buildpackage -aarmel 


So, all of this is straightforward in principle, but I'm not familiar
with sbuild internals so it's not clear to me what the best method
might be in each case.

1) seems easy enough. Add needed packages to add-depends. To write a
generic config needs some way to set $DEB_HOST_GNU_TYPE in the config
file. Does such functionaility exist already? We have $host_arc, which
could be set to 'armel', but then somewhere we need to do
DEB_HOST_GNU_TYPE  ?= $(shell dpkg-architecture -qDEB_HOST_GNU_TYPE -a$host_arch
to get arm-linux-gnueabi.

Does sbuild do any of this already? If not where might be a good place?

1b) needs a little script to be run. Along the lines of
if [ ! -L /usr/bin/${host_triplet}-pkg-config ]; then
 ln -s /usr/share/pkg-config-crosswrapper ${host_triplet}pkg-config
fi
We have chroot-setup-commands
which can presumably run any command which already exists in the
chroot, but doesn't look like it lends itself to little scripts like
that, so the chroot would have to be pre-populated with a suitable
script?

Same considerations apply to 2 and 3. chroot-setup-commands and
pre-build-commands don't look like they support commands of the form
echo foo > file

Can these scripts be supplied by sbuild (in the same way that pbuilder
hook-scripts are), or do they need to come from somewhere else? If the
latter can they be in a package specified by add-depends or do those
get pulled in too late?

4) I thought this would be a simple matter of adding -a$host_arch in
lib/Sbuild/AptResolver.pm but it doesn't run apt-get build-dep
directly, it just installs sbuild-build-depends-foo-dummy which has
been prepared elsewhere. ResolverBase seems to generate this but
nowhere does apt-get build-dep seem to get run so I'm not quite sure
where to go poking. 

Clues welcome. 

5) I think this can be done by just adding 
$debbuildopt = '-a$host_arch' to the config
should if there is a better way.

I'm happy to go prodding about but if there are obvious 'right ways'
to do these things then direction is very welcome.

Wookey
-- 
Principal hats:  Linaro, Emdebian, Wookware, Balloonboard, ARM
http://wookware.org/




Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#610689; Package sbuild. (Mon, 12 Dec 2011 16:00:07 GMT) Full text and rfc822 format available.

Acknowledgement sent to Roger Leigh <rleigh@codelibre.net>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Mon, 12 Dec 2011 16:00:14 GMT) Full text and rfc822 format available.

Message #87 received at 610689@bugs.debian.org (full text, mbox):

From: Roger Leigh <rleigh@codelibre.net>
To: Wookey <wookey@wookware.org>
Cc: 610689@bugs.debian.org
Subject: Re: sbuild cross support
Date: Mon, 12 Dec 2011 15:57:46 +0000
On Mon, Dec 12, 2011 at 02:19:20PM +0000, Wookey wrote:
> +++ Roger Leigh [2011-12-12 11:51 +0000]:
> > On Sat, Dec 10, 2011 at 12:25:15AM +0000, Wookey wrote:
> > > +++ Roger Leigh [2011-12-09 16:48 +0000]:
> > > > On Fri, Dec 09, 2011 at 04:30:20PM +0000, Wookey wrote:
> > > > 
> > > > WRT building, it should build directly from git with plain
> > > > dpkg-buildpackage.  I normally configure, do a "make dist" and then
> > > > build the Debian packages (for upload) from the resulting tarball.
> > > > This shouldn't be necessary just for working on sbuild though--
> > > > that's just to get a pristine build from a clean tree.
> > > > 
> > > > I think the error is likely a result of the changes made to
> > > > configuration file parsing around 27th of February
> > > > (commit 3fa12596 and later).  It's likely the cross-build
> > > > changes to this file need reworking to fit in with the new
> > > > parsing scheme.  If you don't get anywhere with it, I'll try
> > > > to take a look on Monday as time permits.
> > > 
> > > Ah, spot-on; that was enough clue to get that fixed. 
> > > 
> > > Now I can build it but installation failed because both sbuild and
> > > schroot try to install /etc/schroot/buildd/nssdatabases
> > > 
> > > Not clear to me which package should really be in charge of that? I
> > > just let it force-overwrite for now.
> > 
> > I don't know if you saw my comments on IRC last night.  
> 
> No, My IRC-client-running server was rebooted this morning so all
> logged IRC chat has gone.
> 
> > I think this
> > is possibly due to using the buildd git branch, that is
> > buildd-0.62.5 or similar.  The actual release "release/sbuild-0.62.5"
> > and the master branch do not contain these files.
> 
> Aha. git branch -a doesn't show 'release/sbuild-0.62.5'. I picked
> 'origin/buildd-0.62.5' as a plausible-looking branch to use.
> 
> I should probably just move forward to 'master', but I wonder where
> the branch you mention has got to?

Ah, it's a (GPG-signed) release tag, rather than a branch.  You can
run "git tag -l" to view them.  They behave almost like branches,
in that you can do "git checkout $tag", and you can branch from them,
but you can't commit to them.  In this case, it's a tag on the master
branch.  The latest is release/sbuild-0.62.6.

> > I should be able to merge some of the more trivial parts such as
> > the command-line options and associated configuration parameters
> > into master later this week.
> 
> Cool. I was hoping to have something actually working by now so I
> could provide some tested patches, but I'm stuck on the queries I put
> in http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=610689#82
> 
> mostly 'what's the best way to get 'apt-get build-dep -aarmel
> $package' run and what mechanism to use to get dpkg foreign-arch
> config, pkg-config triplet config and apt foreign-arch sources set up.

I'm not sure about the latter bits off the top of my head, but the
"apt-get build-dep" part is fairly simple.  We don't use "apt-get
build-dep" directly--it's not sufficient due to the way it resolves
alternative (and virtual) dependencies.  Instead, we create a
"dummy" dependency package, and set up a tiny apt archive inside the
build chroot, which gets added to /etc/apt/sources.list.d.  We then
"apt-get install" the dummy package, which contains the filtered
build-dependencies.  This is the mechanism used the "apt" and
"aptitude" resolvers; the older "internal" resolver does this a
rather harder and more fragile way, but hopefully internal will
go away soon.  I don't think it's worth adding multi-arch support
to it, should multi-arch require resolver-specific work.

This mechanism also permits the addition or removal of arbitrary
deps, e.g. addition of gcc-snapshot, build-essential and other
things.  We also allow multiple dependency packages to be installed,
e.g. a core depends package containing build-essential, and
package build-depends containing the package-specific deps.  We
could also add an additional one for multiarch deps (or just add
them to the core or package deps as needed).

This system is implemented in lib/Sbuild/*Resolver*.pm.  You can
see it used in lib/Sbuild/Build.pm in run_fetch_install_package().
If you need specific packages installing for cross-builds, here
would likely be the place for it (if it's generic), or in the
resolver base (if it's package-specific).

Given that the full power of apt/aptitude's dependency resolving
is available, you can just use standard multi-arch dependency
syntax to get arch-specific packages installed.  You would probably
need to also have dpkg/apt configured to download that arch's
Packages list; this could be done either by the admin, or by
sbuild each time it's run.  If it's just an occasional cross-build,
it would be OK to fetch each time, but if it's to be used much
more frequently then manual setup might make more sense.  I think
to start with, it would help to know exactly what needs configuring
inside the chroot, then we can work out what should be responsible
for doing it.

I'm not sure what needs doing WRT pkg-config; I assumed that
setting --host and --build when running configure would suffice.
Is anything additional needed?


Regards,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
   `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.




Reply sent to Roger Leigh <rleigh@debian.org>:
You have taken responsibility. (Wed, 30 May 2012 23:09:03 GMT) Full text and rfc822 format available.

Notification sent to Hector Oron <hector.oron@gmail.com>:
Bug acknowledged by developer. (Wed, 30 May 2012 23:09:03 GMT) Full text and rfc822 format available.

Message #92 received at 610689-close@bugs.debian.org (full text, mbox):

From: Roger Leigh <rleigh@debian.org>
To: 610689-close@bugs.debian.org
Subject: Bug#610689: fixed in sbuild 0.63.0-1
Date: Wed, 30 May 2012 23:05:20 +0000
Source: sbuild
Source-Version: 0.63.0-1

We believe that the bug you reported is fixed in the latest version of
sbuild, which is due to be installed in the Debian FTP archive:

buildd_0.63.0-1_all.deb
  to main/s/sbuild/buildd_0.63.0-1_all.deb
libsbuild-perl_0.63.0-1_all.deb
  to main/s/sbuild/libsbuild-perl_0.63.0-1_all.deb
sbuild_0.63.0-1.diff.gz
  to main/s/sbuild/sbuild_0.63.0-1.diff.gz
sbuild_0.63.0-1.dsc
  to main/s/sbuild/sbuild_0.63.0-1.dsc
sbuild_0.63.0-1_all.deb
  to main/s/sbuild/sbuild_0.63.0-1_all.deb
sbuild_0.63.0.orig.tar.gz
  to main/s/sbuild/sbuild_0.63.0.orig.tar.gz



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 610689@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Roger Leigh <rleigh@debian.org> (supplier of updated sbuild package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.8
Date: Wed, 30 May 2012 22:49:18 +0100
Source: sbuild
Binary: libsbuild-perl sbuild buildd
Architecture: source all
Version: 0.63.0-1
Distribution: unstable
Urgency: low
Maintainer: Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>
Changed-By: Roger Leigh <rleigh@debian.org>
Description: 
 buildd     - Daemon for automatically building Debian binary packages from Deb
 libsbuild-perl - Tool for building Debian binary packages from Debian sources
 sbuild     - Tool for building Debian binary packages from Debian sources
Closes: 610689 622788
Changes: 
 sbuild (0.63.0-1) unstable; urgency=low
 .
   [ Wookey ]
   * Support for cross-compiling has been added.  This includes the
     addition of $host and $build configuration variables, with
     corresponding --host and --build command-line options.  This
     includes the addition of a new 'xapt' dependency resolver.
     - Merge cross-build support (thanks to Hector Oron,
       Closes: #610689).
     - Add multiarch cross-build support.
 .
   [ Roger Leigh ]
   * The deprecated 'internal' dependency resolver has been removed,
     along with the configuration variables $apt_policy,
     $check_depends_algorithm and $resolve_virtual, and the
     command-line option --check-depends-algorithm.  The 'apt'
     resolver is the default replacement for 'internal'.
     (Closes: #622788)
   * Support for watches has been removed.  The configuration
     variables $check_watches, $ignore_watches_no_build_deps and
     $watches (and obsolete variables @ignore_watches_no_build_deps
     and %watches) have also been removed.
   * sbuild-stats and support for build time and space statistics
     recording has been removed.  These statistics are recorded in
     both the build log and are available as build metadata
     internally.  The statistics recorded in the database were not
     particularly informative; storing the statistics in a proper
     relational database is recommended.  The configuration variables
     $avg_time_db and $avg_space_db have been removed.
   * Drop 25nssdatabases schroot setup script used on compatibility
     mode (on buildds).  This has been replaced by the schroot
     20nssdatabases for many years.
Checksums-Sha1: 
 d841b2b8f4fe5b0305ec37a036c719b6cd199e1c 2114 sbuild_0.63.0-1.dsc
 c3c8e8acb53a83218160cf45de60e21fc3d3624f 540501 sbuild_0.63.0.orig.tar.gz
 a0fddd5458378c1bf3c10dd2f5c060d1347741ed 20 sbuild_0.63.0-1.diff.gz
 2bfe154d79ab0b77ed7903faa0dce68af12f9c44 286956 libsbuild-perl_0.63.0-1_all.deb
 e015b6fcd914a9f12a0dff760ba26e232460c3ec 301792 sbuild_0.63.0-1_all.deb
 5447f18872b8880348b9179441141ee614f0f964 285918 buildd_0.63.0-1_all.deb
Checksums-Sha256: 
 d7c780814310c3f5528969a8967aa77dad9e2a7c5ba47df45aca496ec2340408 2114 sbuild_0.63.0-1.dsc
 b2c6fe02368a4f422a825a7200eab3ae2e4500708c7aa4264cbcf308a72e3282 540501 sbuild_0.63.0.orig.tar.gz
 f61f27bd17de546264aa58f40f3aafaac7021e0ef69c17f6b1b4cd7664a037ec 20 sbuild_0.63.0-1.diff.gz
 29f463dabed9d307849fea1b04f2c44d10ab3d54919fbed2041f430f0680fc05 286956 libsbuild-perl_0.63.0-1_all.deb
 648fbe7aef6bad751f775d6df83a96c10905b39610fee15f476502b979dad466 301792 sbuild_0.63.0-1_all.deb
 d87fdee85abb8d7800905a12323a2566689bef174e736a51aa914a618ea844ff 285918 buildd_0.63.0-1_all.deb
Files: 
 80fb0f73cded3fc680b3639598efe5d7 2114 devel extra sbuild_0.63.0-1.dsc
 6964c9922807747f54f55e0f0c29e406 540501 devel extra sbuild_0.63.0.orig.tar.gz
 4a4dd3598707603b3f76a2378a4504aa 20 devel extra sbuild_0.63.0-1.diff.gz
 7f7ea3d0735812bbecaff93ebc993bea 286956 perl extra libsbuild-perl_0.63.0-1_all.deb
 22db0f293ef830cf851503a1f70d7abf 301792 devel extra sbuild_0.63.0-1_all.deb
 0b7c42da0688fde86fd062fc32ebb9f4 285918 devel extra buildd_0.63.0-1_all.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iQIcBAEBCgAGBQJPxqP3AAoJEOJSSsUKn1xZl6AQAMJWH5tBstMOSXngQuC1JSIP
yvjpvv4FHu5/R2UH9ZMh5g8vnyLITsLRab8KIC0Q6cKjM/VRSDEPi/qzmXxieyCP
nwQgFmfrcsfYM4QubqM23oYJB9FZabPC8X944vdx99vgk8UH3ijyB9s06MDs8DHX
mooxXvGBVL4YJg5g9jUWtjNUGmY4biGdghxh9DhSIMMLermkaopPDQf11MdQW2D0
QtrB+qVzIx+gcIm9zMF0BeNRqTBzjHpmcHlBwiDzmkA0i+fY+TCtLoV4+MGwHIc3
YjAzaEQq/vn3huVV/syvFYtuI52nBhzDQVO+K5c+WV7VDnhKbm9FiIjerJxmplSI
Ybur6YfAxs34hDbzAmLJhfVd9JN8RO40MjCOrg9gYcrgqNhlXUNaAWmc/lJ6sl6l
LAzuaIQex8BqDmF4K01xF4g76psp7Qroc51/5RwU55jha6eYRN4XuSdrPx8E/pW8
IUixi0qEYv6LsAT3HpC/5BPZd4WqfLNpfLsaxRHF2rVxRAf56eVlELI2wRE7588p
hvIF50bUCu5eOEcukbRps62PeysCaq7RSHuants/d94MH3OrPCqWDT+GbkHOayJi
1GmSxGezdaRV8SvG/8YIowsvwySCOpi+B52PjPNwcWcG3ZiXZh+oWNwjZCThy+k0
a1WfIMNzuAnuieqct/Ta
=3/Dn
-----END PGP SIGNATURE-----





Bug archived. Request was from Debbugs Internal Request <owner@bugs.debian.org> to internal_control@bugs.debian.org. (Thu, 28 Jun 2012 07:52:17 GMT) Full text and rfc822 format available.

Added indication that bug 610689 blocks 671437 Request was from Samuel Bronson <naesten@gmail.com> to control@bugs.debian.org. (Mon, 03 Dec 2012 03:39:05 GMT) Full text and rfc822 format available.

Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Wed Apr 23 21:45:09 2014; Machine Name: beach.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.