Debian Bug report logs - #604268
QEMU linux-user support

version graph

Package: schroot; Maintainer for schroot is Christoph Biedl <debian.axhn@manchmal.in-ulm.de>; Source for schroot is src:schroot (PTS, buildd, popcon).

Reported by: Loïc Minier <lool@dooz.org>

Date: Sun, 21 Nov 2010 17:57:04 UTC

Severity: wishlist

Tags: patch

Found in version schroot/1.4.15-1

Fixed in version schroot/1.5.3-1

Done: Roger Leigh <rleigh@debian.org>

Bug is archived. No further changes may be made.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#604268; Package schroot. (Sun, 21 Nov 2010 17:57:07 GMT) (full text, mbox, link).


Acknowledgement sent to Loïc Minier <lool@dooz.org>:
New Bug report received and forwarded. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Sun, 21 Nov 2010 17:57:07 GMT) (full text, mbox, link).


Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Loïc Minier <lool@dooz.org>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: QEMU linux-user support
Date: Sun, 21 Nov 2010 18:55:23 +0100
[Message part 1 (text/plain, inline)]
Package: schroot
Version: 1.4.15-1
Severity: wishlist
Tags: patch

        Hi there

 I wrote a setup.d snippet which once dropped into
 /etc/schroot/setup.d/11qemu will arrange for a static qemu linux-user
 to be copied into the chroot on setup.  These are provided for many
 architectures in the qemu-user-static (Debian) or qemu-kvm-extras
 (Ubuntu) packages.

 Basically this works thanks to binfmt-misc which will tell the host
 kernel to run a helper program whenever it tries running programs which
 match a specific binary pattern.  The above packages set this up as
 well, so it "just works" upon installation.

 There are limitations to syscall emulation: some syscalls aren't
 implemented, and some things are plainly not simulated (for instance if
 you access files like /proc/cpuinfo, it will return the host's cpuinfo,
 not some simulated one).

 Using QEMU syscall emulation is slow.  Nevertheless, this happens to
 just work for many packages, and it's sometimes handier to use this
 than a porter machine.  It's faster than QEMU machine emulation
 (qemu-system-*).

 I'm attaching the setup.d wrapper; I've used it successfully to start
 sessions in an armel chroot and with sbuild:
    sbuild -d natty --arch armel hello_2.6-1.dsc

 I just downloaded my chroot from Launchpad as I explained in
 Debian #602571, but "qemu-debootstrap" is a debootstrap wrapper doing
 pretty much the same thing which allows creating chroots for schroot
 just as if you'd use debootstrap.  I named my schroot config entry
 "natty-armel" for the above to be picked up.

   Cheers,
-- 
Loïc Minier
[11qemu (text/plain, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#604268; Package schroot. (Sun, 21 Nov 2010 19:09:05 GMT) (full text, mbox, link).


Acknowledgement sent to Roger Leigh <rleigh@codelibre.net>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Sun, 21 Nov 2010 19:09:05 GMT) (full text, mbox, link).


Message #10 received at 604268@bugs.debian.org (full text, mbox, reply):

From: Roger Leigh <rleigh@codelibre.net>
To: Loïc Minier <lool@dooz.org>, 604268@bugs.debian.org
Subject: Re: [buildd-tools-devel] Bug#604268: QEMU linux-user support
Date: Sun, 21 Nov 2010 19:06:13 +0000
[Message part 1 (text/plain, inline)]
On Sun, Nov 21, 2010 at 06:55:23PM +0100, Loïc Minier wrote:
>  I wrote a setup.d snippet which once dropped into
>  /etc/schroot/setup.d/11qemu will arrange for a static qemu linux-user
>  to be copied into the chroot on setup.  These are provided for many
>  architectures in the qemu-user-static (Debian) or qemu-kvm-extras
>  (Ubuntu) packages.
[...]
>  I'm attaching the setup.d wrapper; I've used it successfully to start
>  sessions in an armel chroot and with sbuild:
>     sbuild -d natty --arch armel hello_2.6-1.dsc
> 
>  I just downloaded my chroot from Launchpad as I explained in
>  Debian #602571, but "qemu-debootstrap" is a debootstrap wrapper doing
>  pretty much the same thing which allows creating chroots for schroot
>  just as if you'd use debootstrap.  I named my schroot config entry
>  "natty-armel" for the above to be picked up.

This is really cool stuff.  I'll be happy to include it in schroot.
Just a couple of issues with it:

1) It's not checking if $system_arch == $chroot_arch for all
   arches, just a limited hardcoded set; it should probably
   exit 0 if the above condition is true before looking for
   specific pairings.

2) The licence is not the same as the schroot packages, which is
   GPLv3+.  Would it be OK to have this licenced at GPLv2+ or 3+
   for compatibility with the rest of the package?

3) It would be nice to have a configuration in the script-config
   file to disable support completely, enable it unconditionally,
   or enable for selected architectures as at present.

Since we're frozen at the moment, I can't really include it just
now in a release, but it can certainly go into git for the next
release.

> # TODO
> # - allow setting the qemu interpreter in the script-config file
> # - allow forcing use of qemu even when not particularly needed
> # - test support for all architectures
> # - add support for more architectures

Setting in script-config will require writing a chroot facet object,
which is something I'd really like to see support for.  I'm
especially interested in support for kvm and qemu-kvm and qemu, so
these would all fit under the same umbrella.

Currently, we do require that the guest is a chrooted filesystem,
but I'd like to abstract that to allow running commands in entirely
isolated virtual systems such as kvm.


Regards,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
   `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#604268; Package schroot. (Sun, 21 Nov 2010 19:45:03 GMT) (full text, mbox, link).


Acknowledgement sent to Loïc Minier <lool@dooz.org>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Sun, 21 Nov 2010 19:45:03 GMT) (full text, mbox, link).


Message #15 received at 604268@bugs.debian.org (full text, mbox, reply):

From: Loïc Minier <lool@dooz.org>
To: Roger Leigh <rleigh@codelibre.net>
Cc: 604268@bugs.debian.org
Subject: Re: [buildd-tools-devel] Bug#604268: QEMU linux-user support
Date: Sun, 21 Nov 2010 20:41:09 +0100
[Message part 1 (text/plain, inline)]
On Sun, Nov 21, 2010, Roger Leigh wrote:
> 1) It's not checking if $system_arch == $chroot_arch for all
>    arches, just a limited hardcoded set; it should probably
>    exit 0 if the above condition is true before looking for
>    specific pairings.

 Ah right, I initially wanted to do this as I thought I'd add support
 for more arches, then I added support for more arches and forgot to
 update it, exactly as expected  :)

 I'm attaching an updated version which does just that, with a note that
 more biarch cases should be added

> 2) The licence is not the same as the schroot packages, which is
>    GPLv3+.  Would it be OK to have this licenced at GPLv2+ or 3+
>    for compatibility with the rest of the package?

 Yeah; it's just my boilerplate "public domain"-ish snippet, which
 includes permission to sublicense.  You're welcome to change the
 license to GPLv3+ when merging into schroot; you could also keep it as
 is and claim that schroot is distributed under GPLv3+.  (I'm keeping my
 local scripts under this permissive licensing because I'm copy-pasting
 shell snippets I write across scripts and projects.)

> 3) It would be nice to have a configuration in the script-config
>    file to disable support completely, enable it unconditionally,
>    or enable for selected architectures as at present.

 Ok; I wanted to add this support at some point anyway

 I'll look at adding this

> > # - allow setting the qemu interpreter in the script-config file
> > # - allow forcing use of qemu even when not particularly needed
> > # - test support for all architectures
> > # - add support for more architectures
> 
> Setting in script-config will require writing a chroot facet object,
> which is something I'd really like to see support for.  I'm
> especially interested in support for kvm and qemu-kvm and qemu, so
> these would all fit under the same umbrella.
> 
> Currently, we do require that the guest is a chrooted filesystem,
> but I'd like to abstract that to allow running commands in entirely
> isolated virtual systems such as kvm.

 Awesome!  I guess remote machines, or cloud/EC2 vms come to mind as
 well

 It might make sense to look into libvirt for this

    Cheers
-- 
Loïc Minier
[11qemu (text/plain, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#604268; Package schroot. (Sun, 12 Dec 2010 12:15:03 GMT) (full text, mbox, link).


Acknowledgement sent to Roger Leigh <rleigh@codelibre.net>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Sun, 12 Dec 2010 12:15:03 GMT) (full text, mbox, link).


Message #20 received at 604268@bugs.debian.org (full text, mbox, reply):

From: Roger Leigh <rleigh@codelibre.net>
To: Loïc Minier <lool@dooz.org>, 604268@bugs.debian.org
Subject: Re: [buildd-tools-devel] Bug#604268: Bug#604268: QEMU linux-user support
Date: Sun, 12 Dec 2010 12:11:29 +0000
[Message part 1 (text/plain, inline)]
On Sun, Nov 21, 2010 at 08:41:09PM +0100, Loïc Minier wrote:
> On Sun, Nov 21, 2010, Roger Leigh wrote:
> > 1) It's not checking if $system_arch == $chroot_arch for all
> >    arches, just a limited hardcoded set; it should probably
> >    exit 0 if the above condition is true before looking for
> >    specific pairings.
> 
>  Ah right, I initially wanted to do this as I thought I'd add support
>  for more arches, then I added support for more arches and forgot to
>  update it, exactly as expected  :)
> 
>  I'm attaching an updated version which does just that, with a note that
>  more biarch cases should be added

Thanks.

> > 2) The licence is not the same as the schroot packages, which is
> >    GPLv3+.  Would it be OK to have this licenced at GPLv2+ or 3+
> >    for compatibility with the rest of the package?
> 
>  Yeah; it's just my boilerplate "public domain"-ish snippet, which
>  includes permission to sublicense.  You're welcome to change the
>  license to GPLv3+ when merging into schroot; you could also keep it as
>  is and claim that schroot is distributed under GPLv3+.  (I'm keeping my
>  local scripts under this permissive licensing because I'm copy-pasting
>  shell snippets I write across scripts and projects.)

Thanks.

> > 3) It would be nice to have a configuration in the script-config
> >    file to disable support completely, enable it unconditionally,
> >    or enable for selected architectures as at present.
> 
>  Ok; I wanted to add this support at some point anyway
> 
>  I'll look at adding this
> 
> > > # - allow setting the qemu interpreter in the script-config file
> > > # - allow forcing use of qemu even when not particularly needed
> > > # - test support for all architectures
> > > # - add support for more architectures
> > 
> > Setting in script-config will require writing a chroot facet object,
> > which is something I'd really like to see support for.  I'm
> > especially interested in support for kvm and qemu-kvm and qemu, so
> > these would all fit under the same umbrella.
> > 
> > Currently, we do require that the guest is a chrooted filesystem,
> > but I'd like to abstract that to allow running commands in entirely
> > isolated virtual systems such as kvm.
> 
>  Awesome!  I guess remote machines, or cloud/EC2 vms come to mind as
>  well
> 
>  It might make sense to look into libvirt for this

Yes, it's one of the things I should really investigate more, since
I don't want to duplicate existing functionality unless it's really
necessary.

Thanks for the updated patch.  I'm afraid that it can't go into
schroot 1.4 just at the moment due to the squeeze freeze.  But I'll
put in onto the master branch when I have some free time, and it
can go into schroot 1.5/1.6 for wheezy/unstable once the freeze is
over.


Regards,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
   `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#604268; Package schroot. (Sat, 22 Jan 2011 17:00:03 GMT) (full text, mbox, link).


Acknowledgement sent to Roger Leigh <rleigh@codelibre.net>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Sat, 22 Jan 2011 17:00:03 GMT) (full text, mbox, link).


Message #25 received at 604268@bugs.debian.org (full text, mbox, reply):

From: Roger Leigh <rleigh@codelibre.net>
To: Loïc Minier <lool@dooz.org>, 604268@bugs.debian.org
Subject: Re: [buildd-tools-devel] Bug#604268: Bug#604268: Bug#604268: QEMU linux-user support
Date: Sat, 22 Jan 2011 16:56:26 +0000
[Message part 1 (text/plain, inline)]
On Sun, Dec 12, 2010 at 12:11:29PM +0000, Roger Leigh wrote:
> Thanks for the updated patch.  I'm afraid that it can't go into
> schroot 1.4 just at the moment due to the squeeze freeze.  But I'll
> put in onto the master branch when I have some free time, and it
> can go into schroot 1.5/1.6 for wheezy/unstable once the freeze is
> over.

OK, I've worked on this a bit today, and I've attached a patch for
the current work in progress.  Basically, I've added support for
machine emulation as a "chroot facet" inside the actual C++
sources, rather than the setup scripts.  This means the qemu
configuration etc. can be done directly in the chroot configuration
in /etc/schroot.  A patch is attached, as is the documentation
(searching for "machine emulation" will find the relevant bits).

Essentially, you can now set
emulation=qemu|none
emulation-arch=auto|$arch

and when the setup scripts are run, those are then set as
$CHROOT_EMULATION and
$CHROOT_EMULATION_ARCH

Your 11qemu script has been added as 50emulation (does it need to
be sooner?)  I renamed it to allow other emulation programs to be
used potentially; the same applies to the option names.

I'm not sure if emulation-arch is that useful; is there any reason
not to always autodetect?  Maybe if autodetection fails?  Also, we
can use emulation=qemu to force qemu usage--since we enable it in the
configuration explicitly, do we then need to check the host arch to see
if we should run qemu?  This would make the setup script simpler and
behave the same under most conditions.

If any additional options need adding (I think you mentioned supporting
other executables?) then this is now simple to do.  They can simply be
added as additional members of the class in
sbuild/sbuild-chroot-facet-emulation.(cc|h), and the configuration file
and environment settings are also defined here.

If you're unhappy with any of the names used, that's also trivial to
change if you like.  (I'm fairly rubbish at naming things!)

BTW, does kfreebsd have the same binfmt-misc and qemu support as Linux?
I'm just wondering if this should be enabled by default on BSD as well.

Anyway, that's it so far.  I'd appreciate any thoughts you have!


Regards,
Roger


% ./bin/schroot/schroot -i -c bsnaptest
  ─── Chroot ───
  Name                      bsnaptest
  Description               Debian sid btrfs snapshot
  Type                      btrfs-snapshot
…
  Btrfs Source Subvolume    /srv/chroot/sid-snap
  Btrfs Snapshot Directory  /srv/chroot/snapshots
  Personality               undefined
  Machine Emulation         qemu
  Emulation Architecture    powerpc
…

% ./bin/schroot/schroot --config -c bsnaptest
# schroot configuration generated by schroot 1.5.0 on 22 Jan 2011

[bsnaptest]
…
emulation=qemu
emulation-arch=powerpc
…

% ./bin/schroot/schroot -V                   
schroot (Debian sbuild) 1.5.0 (16 Jan 2011)
…
Configured features:
  DEVLOCK      Device locking
  PAM          Pluggable Authentication Modules
  PERSONALITY  Linux kernel Application Binary Interface switching
  UNION        Support for filesystem unioning
  EMULATION    Machine emulation
…


 bin/schroot-base/schroot-base-main.cc   |    5 
 configure.ac                            |   29 +++++
 debian/control                          |    2 
 etc/setup.d/50emulation                 |  141 +++++++++++++++++++++++++++
 etc/setup.d/Makefile.am                 |   17 ++-
 man/schroot-setup.5.in                  |    7 +
 man/schroot.conf.5.in                   |   30 +++++
 sbuild/Makefile.am                      |   17 ++-
 sbuild/sbuild-chroot-facet-emulation.cc |  166 ++++++++++++++++++++++++++++++++
 sbuild/sbuild-chroot-facet-emulation.h  |  137 ++++++++++++++++++++++++++
 sbuild/sbuild-chroot.cc                 |    6 +
 sbuild/sbuild-config.h.in               |    3 
 test/sbuild-chroot-block-device.cc      |    5 
 test/sbuild-chroot-btrfs-snapshot.cc    |    5 
 test/sbuild-chroot-directory.cc         |    5 
 test/sbuild-chroot-file.cc              |    5 
 test/sbuild-chroot-loopback.cc          |    5 
 test/sbuild-chroot-lvm-snapshot.cc      |    5 
 test/sbuild-chroot-plain.cc             |    5 
 test/sbuild-chroot.cc                   |    5 
 test/test-sbuild-chroot.h               |   17 +++
 21 files changed, 608 insertions(+), 9 deletions(-)

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
   `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.
[schroot-qemu-1.patch (text/x-diff, attachment)]
[schroot-qemu.pdf (application/pdf, attachment)]
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#604268; Package schroot. (Sat, 22 Jan 2011 22:30:03 GMT) (full text, mbox, link).


Acknowledgement sent to Loïc Minier <lool@dooz.org>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Sat, 22 Jan 2011 22:30:03 GMT) (full text, mbox, link).


Message #30 received at 604268@bugs.debian.org (full text, mbox, reply):

From: Loïc Minier <lool@dooz.org>
To: Roger Leigh <rleigh@codelibre.net>
Cc: 604268@bugs.debian.org
Subject: Re: [buildd-tools-devel] Bug#604268: Bug#604268: Bug#604268: QEMU linux-user support
Date: Sat, 22 Jan 2011 23:26:41 +0100
On Sat, Jan 22, 2011, Roger Leigh wrote:
> OK, I've worked on this a bit today, and I've attached a patch for
> the current work in progress.  Basically, I've added support for
> machine emulation as a "chroot facet" inside the actual C++
> sources, rather than the setup scripts.  This means the qemu
> configuration etc. can be done directly in the chroot configuration
> in /etc/schroot.  A patch is attached, as is the documentation
> (searching for "machine emulation" will find the relevant bits).

 Sounds good!

 I think I'm happier with more things being outside of /etc; I am a bit
 unconfortable with the amount of important logic which one is able to
 modify under /etc/schroot  ;-)

> Your 11qemu script has been added as 50emulation (does it need to
> be sooner?)  I renamed it to allow other emulation programs to be
> used potentially; the same applies to the option names.

 I might have forgotten the exact reason I picked 11qemu, but it should
 relatively early as to allow subsequent scripts to run commands in the
 chroot; I seem to recall that some setup.d script where chrooting +
 running stuff.  I think I placed it as soon after the chroot becomes
 usable to chroot into (after the filesystems are mounted).

> I'm not sure if emulation-arch is that useful; is there any reason
> not to always autodetect?  Maybe if autodetection fails?

 One thing I pondered recently is if it would be best to ask the host
 environment's binfmt-support database about the interpreter to copy,
 based on an executable from the chroot.  IOW, match
 /var/lib/binfmts/qemu-* against some executable from the chroot, and
 then find out the qemu binary to use from that file.  Or perhaps binfmt
 could tell us with a new interface, or we could find out by running a
 static binary and looking at its interpreter:
    CHROOT=/var/lib/schroot/mount/natty-armel-f3e92a7f-f1a5-472e-8c0b-3e2c6f8f70cf/chroot-autobuild
    $CHROOT/lib/ld-linux.* --library-path $CHROOT/lib $CHROOT/bin/cat /proc/self/cmdline

 this outputs:
    /usr/bin/qemu-arm-static\0<chroot>/lib/ld-linux.so.3\0 ...
 (the above form might not be terribly portable, but note how we avoid
 the whole dance of running "file", or the name of qemu architectures;
 heck, we don't even need to know it's qemu!!)

>                            Maybe if autodetection fails?  Also, we
> can use emulation=qemu to force qemu usage--since we enable it in the
> configuration explicitly, do we then need to check the host arch to see
> if we should run qemu?  This would make the setup script simpler and
> behave the same under most conditions.

 It's fine to always run qemu even if not needed; just a waste -- I'm
 not sure it's actually honored though, the kernel's matching rule might
 pick up 32-bits binaries before binfmt binaries.

>                                       (I think you mentioned supporting
> other executables?)

 maybe, I don't recall; it would sound wrong to hardcode qemu or the
 path to qemu in any case

> If you're unhappy with any of the names used, that's also trivial to
> change if you like.  (I'm fairly rubbish at naming things!)

 That's probably as good as what I would think of; qemu syscall
 emulation is usually named "CPU transparency" because it's basically a
 mapping of a flow of CPU instruction to another one, with syscall
 translation.  This is different from qemu machine emulation which
 emulates hardware; this is sometimes called simulation.  Upstream, the
 syscall emulation stuff is called "qemu-user emulation".

> BTW, does kfreebsd have the same binfmt-misc and qemu support as Linux?
> I'm just wondering if this should be enabled by default on BSD as well.

 I have no idea; in the past, I thought about a couple of ways qemu
 could be modified to provide the same functionality, but without kernel
 support.  I could think of at least two ways, but didn't pursue any of
 them in the end as binfmt is too convenient; one way would be via
 LD_PRELOAD, and the other way would be to teach qemu that when it
 translates exec() and similar it should actually prepend itself.  I
 think the -L option is also supposedly useful, but broken since ages.

> --- a/configure.ac
> +++ b/configure.ac
> @@ -246,6 +246,18 @@ AC_ARG_ENABLE([union], [AS_HELP_STRING([--enable-union], [Enable support for uni
>                [enable_union="auto"])
>  AC_MSG_RESULT([$enable_union])
>  
> +AC_MSG_CHECKING([whether to enable support for machine emulation])
> +AC_ARG_ENABLE([emulation], [AS_HELP_STRING([--enable-emulation], [Enable suppoer for machine emulation (qemu)])],

 "support" not "suppoer"

> --- a/debian/control
> +++ b/debian/control
> @@ -95,7 +95,7 @@ Architecture: any
>  Depends: ${shlibs:Depends}, ${misc:Depends}, schroot-common (=${source:Version})
>  Conflicts: dchroot (<< 0.13)
>  Enhances: sbuild
> -Suggests: debootstrap, lvm2, btrfs-tools, aufs-modules | unionfs-modules
> +Suggests: debootstrap, lvm2, btrfs-tools, aufs-modules | unionfs-modules, file, qemu-user-static | qemu-kvm-extras

 Ah it's actually qemu-kvm-extras-static in Ubuntu, not qemu-kvm-extras;
 I can see I'm at the origin of this bug

> +# Depends: file, qemu-user-static | qemu-kvm-extras

 Should be | qemu-kvm-extras-static

> +# - allow setting the qemu interpreter in the script-config file

 The idea was allowing to set a qemu interpreter of one's choice,
 without relying on autodetection or to override it; I'm not sure it's
 very important

> +# - allow forcing use of qemu even when not particularly needed

 a bit of the same

> +# - add more biarch cases

 would be nice if we had a way to test for these cases instead of
 hardcoding a list; in fact, I wish I had done so instead of using file.
 (Matching chroot binaries against /var/lib/binfmts/qemu-* to find the
 interpreter.)

> --- a/man/schroot.conf.5.in
> +++ b/man/schroot.conf.5.in
> +If configured to do so, schroot will allow the use of a machine emulator such
> +as
> +.BR qemu (1)
> +inside the chroot environment.  This may be used to build and use software
> +using an entirely different CPU architecture, for example to build programs for
> +the ARM CPU on an amd64 system.

 You might want to stick to GNU names there, e.g. "for the ARM CPU on an
 x86-64 system"

> +Currently, qemu is the sole method for supporting machine emulation.  If one
> +wishes for example to emulate an ARM machine, schroot will copy the static
> +\fIqemu-arm-static\fP binary from the host system to the chroot in order to
> +allow the emulation to work transparently.

 This reminds me that there's an inherent bug in the current approach:
 if you install qemu-user-static in the chroot, you're screwed
 (the x86 qemu-arm-static which you copied into the chroot gets replaced
 by an ARM one).  A quick and dirty way around this would be
 dpkg-divert (run in the chroot just after copying qemu-arm-static), a
 long-term elegant way might be multiarch.


 I've read through the diff, it seemed fine; thanks for the efforts
 you've put into this C++ version!  I would suggest that you proceed
 with this version and we keep the other ideas from above for a second
 version; there is no need to delay this (IMO quite useful) support for
 the new issues I bring up above.

-- 
Loïc Minier




Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#604268; Package schroot. (Sun, 23 Jan 2011 19:06:03 GMT) (full text, mbox, link).


Acknowledgement sent to Roger Leigh <rleigh@codelibre.net>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Sun, 23 Jan 2011 19:06:03 GMT) (full text, mbox, link).


Message #35 received at 604268@bugs.debian.org (full text, mbox, reply):

From: Roger Leigh <rleigh@codelibre.net>
To: Loïc Minier <lool@dooz.org>, 604268@bugs.debian.org
Subject: Re: [buildd-tools-devel] Bug#604268: Bug#604268: Bug#604268: Bug#604268: QEMU linux-user support
Date: Sun, 23 Jan 2011 19:03:55 +0000
[Message part 1 (text/plain, inline)]
On Sat, Jan 22, 2011 at 11:26:41PM +0100, Loïc Minier wrote:
> On Sat, Jan 22, 2011, Roger Leigh wrote:

>  One thing I pondered recently is if it would be best to ask the host
>  environment's binfmt-support database about the interpreter to copy,
>  based on an executable from the chroot.  IOW, match
>  /var/lib/binfmts/qemu-* against some executable from the chroot, and
>  then find out the qemu binary to use from that file.  Or perhaps binfmt
>  could tell us with a new interface, or we could find out by running a
>  static binary and looking at its interpreter:
>     CHROOT=/var/lib/schroot/mount/natty-armel-f3e92a7f-f1a5-472e-8c0b-3e2c6f8f70cf/chroot-autobuild
>     $CHROOT/lib/ld-linux.* --library-path $CHROOT/lib $CHROOT/bin/cat /proc/self/cmdline
> 
>  this outputs:
>     /usr/bin/qemu-arm-static\0<chroot>/lib/ld-linux.so.3\0 ...
>  (the above form might not be terribly portable, but note how we avoid
>  the whole dance of running "file", or the name of qemu architectures;
>  heck, we don't even need to know it's qemu!!)

This should be sufficiently portable for Linux usage.  I am a little
concerned that it might be fragile though.  How does the binfmt-misc
code know which file to pick?  Can't we use the same mechanism and
avoid this?

Note that we don't need to use file or this mechanism.  We are a
compiled binary, so we can simply use libbfd or libelf directly.  We
can then get the architecture identifier /directly/.  We would need to
map numbers to names for qemu (and additional maps for other
interpreters with different naming schemes, but that's not important
for now).  OTOH, your method guarantees that we pick the correct
interpreter.

> > If you're unhappy with any of the names used, that's also trivial to
> > change if you like.  (I'm fairly rubbish at naming things!)
> 
>  That's probably as good as what I would think of; qemu syscall
>  emulation is usually named "CPU transparency" because it's basically a
>  mapping of a flow of CPU instruction to another one, with syscall
>  translation.  This is different from qemu machine emulation which
>  emulates hardware; this is sometimes called simulation.  Upstream, the
>  syscall emulation stuff is called "qemu-user emulation".

Would it be better to use "emulation=qemu-user" rather than just
"qemu"?  It would allow addition of "qemu-system" at a later point,
and also makes the distinction between the two.

[Typos and corrections]
Thanks, I've fixed those.

>  Ah it's actually qemu-kvm-extras-static in Ubuntu, not qemu-kvm-extras;
>  I can see I'm at the origin of this bug
> 
> > +# Depends: file, qemu-user-static | qemu-kvm-extras
> 
>  Should be | qemu-kvm-extras-static

If the package is not in Debian, maybe this one should be patched
in when put in Ubuntu?

> > +Currently, qemu is the sole method for supporting machine emulation.  If one
> > +wishes for example to emulate an ARM machine, schroot will copy the static
> > +\fIqemu-arm-static\fP binary from the host system to the chroot in order to
> > +allow the emulation to work transparently.
> 
>  This reminds me that there's an inherent bug in the current approach:
>  if you install qemu-user-static in the chroot, you're screwed
>  (the x86 qemu-arm-static which you copied into the chroot gets replaced
>  by an ARM one).  A quick and dirty way around this would be
>  dpkg-divert (run in the chroot just after copying qemu-arm-static), a
>  long-term elegant way might be multiarch.

Yes, this is a definite problem.  Even if we divert the binary, we
would be breaking things for the user who wanted that package
installing.  Diverting means we can delete the copied binary and
undivert on session end, so this would be good to do in any case.
I guess this is what update-binfmts creates:

[/var/lib/binfmts/qemu-ppc]
qemu-user-static
magic
0
\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x14
\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff
/usr/bin/qemu-ppc-static

Which is hardcoding the path.  If the path wasn't hardcoded, we could
have put it into /usr/local or even some non-standard name or location
if we could adjust the interpreter.  It's a shame it's looking inside
the chroot, rather than the main root, but that's probably the only
sane thing to do now a system can have multiple namespaces.

I think that clearly documenting this limitation is the most
pragmatic approach here.  I think the chance of installing and
using the same qemu-$arch-static binary inside the chroot is small,
but using a diversion will be a good improvement.


Regards,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
   `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#604268; Package schroot. (Mon, 24 Jan 2011 00:24:08 GMT) (full text, mbox, link).


Acknowledgement sent to Loïc Minier <lool@dooz.org>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Mon, 24 Jan 2011 00:24:08 GMT) (full text, mbox, link).


Message #40 received at 604268@bugs.debian.org (full text, mbox, reply):

From: Loïc Minier <lool@dooz.org>
To: Roger Leigh <rleigh@codelibre.net>
Cc: 604268@bugs.debian.org
Subject: Re: [buildd-tools-devel] Bug#604268: Bug#604268: Bug#604268: Bug#604268: QEMU linux-user support
Date: Mon, 24 Jan 2011 01:20:34 +0100
On Sun, Jan 23, 2011, Roger Leigh wrote:
> This should be sufficiently portable for Linux usage.  I am a little
> concerned that it might be fragile though.  How does the binfmt-misc
> code know which file to pick?  Can't we use the same mechanism and
> avoid this?

 So binfmt-support has an update-binfmts which registers
 /usr/share/binfmt-support/* stuff into /var/lib/binfmts/*; these end up
 enabled in the running kernel (see /proc/sys/fs/binfmt_misc) via
 update-binfmts --enable which is run by /etc/init.d/binfmt-support.

 While we could try parsing binfmt-support's format or binfmt_misc's
 format, I think the best thing here would be to check with the
 binfmt-support maintainer.  I wonder whether binfmt-support is
 Debian/Ubuntu specific.  If it is, then poking the kernel format or
 trying to run a binary might be best.  If it's also used on other
 distros, perhaps we can get some command which tells us what the
 interpreter is for a specific binary.  If that makes sense to you, I
 can poke Colin about it, and perhaps open a bug report for the
 binfmt-support changes.

 (ptrace() might also allow finding out which interpreter is run, but
 that seems fragile too.)

> > > If you're unhappy with any of the names used, that's also trivial to
> > > change if you like.  (I'm fairly rubbish at naming things!)
> > 
> >  That's probably as good as what I would think of; qemu syscall
> >  emulation is usually named "CPU transparency" because it's basically a
> >  mapping of a flow of CPU instruction to another one, with syscall
> >  translation.  This is different from qemu machine emulation which
> >  emulates hardware; this is sometimes called simulation.  Upstream, the
> >  syscall emulation stuff is called "qemu-user emulation".
> 
> Would it be better to use "emulation=qemu-user" rather than just
> "qemu"?  It would allow addition of "qemu-system" at a later point,
> and also makes the distinction between the two.

 I'm happy with either; I don't know of any other implementation of this
 feature than qemu, and I can't think of any other name that we would
 use than "emulation".  So emulation=qemu is fine!  You could also name
 it emulation=binfmt-misc if we manage to get the information without
 special casing qemu  ;-)


 qemu-system-* > ah, I hope you don't mind if I share some thoughts
 here:
 * qemu-system-* can be managed much like a remote system over TCP/IP,
   with some commands to setup and teardown; I've seen software like
   Hudson deal with by having two ways of controlling slaves: a) install
   a piece of software on the slave which connects to the master and
   gets orders, intrusive but clean b) have the master connect to the
   slave over SSH and send command / receive responses over that link.
     I think having simple pre-/post-commands would be fine to deal with
   such use.

 * There are some specificities that you could exploit to tune:
   - qemu-system-* often allows you to interact with consoles of the vm;
     I think that's what qemubuilder and "rootstock" use to interact
     with the guest.  That's nifty, but very specific to this use case.
   - if your guest has virtio support, you have more efficient network
     and block device, can share memory, but you can also share
     filesystems!  This might be very efficient (more than NFS or scp),
     but again, very specific.

 * The complexity is really in getting a good qemu-system-foo
   environment working and up-to-date, including kernel, rootfs, cmdline
   opts, serial line setup etc.  This is a problem which can be solved
   separately, much like creating a chroot is a separate problem from
   using it.  In fact, even for qemu-system-arm, each machine might have
   a different boot mechanism, this board only supports booting from a
   SD image, this other only from flash, that one does not have
   networking etc.

 I found libvirt to be a really nice abstraction to control vms of
 various types; it worked fine for kvm, virtualbox and qemu based vms
 for me, and has a quite complete stack with some language bindings,
 higher level software, UIs etc.  libvirt allows defining additional
 types of vms, and I'm convinced we could define new types of vms to
 start qemu-system-arm with this arm kernel and these command-line flags
 etc.

> >  Ah it's actually qemu-kvm-extras-static in Ubuntu, not qemu-kvm-extras;
> >  I can see I'm at the origin of this bug
> > > +# Depends: file, qemu-user-static | qemu-kvm-extras
> >  Should be | qemu-kvm-extras-static
> If the package is not in Debian, maybe this one should be patched
> in when put in Ubuntu?

 It's not too good though because it means manual merge every time
 schroot is uploaded, which adds work and delays.  I could propose a
 dpkg-vendor based test to generate a ${qemu-user-suggests} or similar
 to use in control, but I think you'll find that uglier in your package
 than the above control snippet  ;-)

 (NB: ideally, Ubuntu would use the same package layout as Debian; it's
 quite complex here and involves many packages and a long story; I'm
 happy to share the details)

>                                If the path wasn't hardcoded, we could
> have put it into /usr/local or even some non-standard name or location
> if we could adjust the interpreter.  It's a shame it's looking inside
> the chroot, rather than the main root, but that's probably the only
> sane thing to do now a system can have multiple namespaces.

 Yeah, it's a complex security problem: it's a kernel service, so
 looking up things in the PATH already sounds scary, the interpreter
 also ends up in userspace memory, so I would be scared if it was
 allowed to load stuff outside of the chroot (albeit it seems useful).

> I think that clearly documenting this limitation is the most
> pragmatic approach here.  I think the chance of installing and
> using the same qemu-$arch-static binary inside the chroot is small,
> but using a diversion will be a good improvement.

 Makes complete sense.

-- 
Loïc Minier




Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#604268; Package schroot. (Tue, 25 Jan 2011 14:48:03 GMT) (full text, mbox, link).


Acknowledgement sent to Loïc Minier <lool@dooz.org>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Tue, 25 Jan 2011 14:48:03 GMT) (full text, mbox, link).


Message #45 received at 604268@bugs.debian.org (full text, mbox, reply):

From: Loïc Minier <lool@dooz.org>
To: Roger Leigh <rleigh@codelibre.net>
Cc: 604268@bugs.debian.org
Subject: Re: [buildd-tools-devel] Bug#604268: Bug#604268: Bug#604268: Bug#604268: QEMU linux-user support
Date: Tue, 25 Jan 2011 15:44:29 +0100
On Mon, Jan 24, 2011, Loïc Minier wrote:
>  While we could try parsing binfmt-support's format or binfmt_misc's
>  format, I think the best thing here would be to check with the
>  binfmt-support maintainer.  I wonder whether binfmt-support is
>  Debian/Ubuntu specific.  If it is, then poking the kernel format or
>  trying to run a binary might be best.  If it's also used on other
>  distros, perhaps we can get some command which tells us what the
>  interpreter is for a specific binary.  If that makes sense to you, I
>  can poke Colin about it, and perhaps open a bug report for the
>  binfmt-support changes.

 I've asked Colin about this, and he implemented support for that in
 trunk!  :-)   (I tested it, there was an issue in the original version
 but r109 fixed it)

 (bzr+ssh://bzr.debian.org/bzr/binfmt-support/trunk or
 http://bzr.debian.org/binfmt-support/trunk)

    % sudo apt-get build-dep binfmt-support
    % ./configure && make
    % src/update-binfmts --find $chroot/bin/true
    /usr/bin/qemu-arm-static

 Note that the --find might return multiple interpreters; in this case,
 the kernel tries them one by one.

 Perhaps we should do something like find the first static interpreter,
 or always copy the first interpreter and warn if there is more than
 one?

   Cheers,
-- 
Loïc Minier




Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#604268; Package schroot. (Tue, 25 Jan 2011 14:57:03 GMT) (full text, mbox, link).


Acknowledgement sent to Loïc Minier <lool@dooz.org>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Tue, 25 Jan 2011 14:57:03 GMT) (full text, mbox, link).


Message #50 received at 604268@bugs.debian.org (full text, mbox, reply):

From: Loïc Minier <lool@dooz.org>
To: Roger Leigh <rleigh@codelibre.net>
Cc: 604268@bugs.debian.org
Subject: Re: [buildd-tools-devel] Bug#604268: Bug#604268: Bug#604268: Bug#604268: QEMU linux-user support
Date: Tue, 25 Jan 2011 15:54:03 +0100
 Colin uploaded this to experimental already; he also told me it's not
 used in non-Debian distros that he knows of, but he would be happy to
 help with this.  I think binfmt-support needs to be ./configure-able in
 schroot for now

-- 
Loïc Minier




Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#604268; Package schroot. (Tue, 14 Jun 2011 11:27:17 GMT) (full text, mbox, link).


Acknowledgement sent to Julian Andres Klode <jak@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Tue, 14 Jun 2011 11:27:24 GMT) (full text, mbox, link).


Message #55 received at 604268@bugs.debian.org (full text, mbox, reply):

From: Julian Andres Klode <jak@debian.org>
To: Loïc Minier <lool@dooz.org>, 604268@bugs.debian.org
Subject: Re: Bug#604268: [buildd-tools-devel] Bug#604268: Bug#604268: Bug#604268: Bug#604268: QEMU linux-user support
Date: Tue, 14 Jun 2011 13:25:42 +0200
[Message part 1 (text/plain, inline)]
On Tue, Jan 25, 2011 at 03:44:29PM +0100, Loïc Minier wrote:
> On Mon, Jan 24, 2011, Loïc Minier wrote:
> >  While we could try parsing binfmt-support's format or binfmt_misc's
> >  format, I think the best thing here would be to check with the
> >  binfmt-support maintainer.  I wonder whether binfmt-support is
> >  Debian/Ubuntu specific.  If it is, then poking the kernel format or
> >  trying to run a binary might be best.  If it's also used on other
> >  distros, perhaps we can get some command which tells us what the
> >  interpreter is for a specific binary.  If that makes sense to you, I
> >  can poke Colin about it, and perhaps open a bug report for the
> >  binfmt-support changes.
> 
>  I've asked Colin about this, and he implemented support for that in
>  trunk!  :-)   (I tested it, there was an issue in the original version
>  but r109 fixed it)
> 
>  (bzr+ssh://bzr.debian.org/bzr/binfmt-support/trunk or
>  http://bzr.debian.org/binfmt-support/trunk)
> 
>     % sudo apt-get build-dep binfmt-support
>     % ./configure && make
>     % src/update-binfmts --find $chroot/bin/true
>     /usr/bin/qemu-arm-static
> 
>  Note that the --find might return multiple interpreters; in this case,
>  the kernel tries them one by one.
> 
>  Perhaps we should do something like find the first static interpreter,
>  or always copy the first interpreter and warn if there is more than
>  one?
> 
Attaching my solution for this.

-- 
Julian Andres Klode  - Debian Developer, Ubuntu Member

See http://wiki.debian.org/JulianAndresKlode and http://jak-linux.org/.
[99binfmt (text/plain, attachment)]
[Message part 3 (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#604268; Package schroot. (Wed, 27 Jul 2011 18:03:10 GMT) (full text, mbox, link).


Acknowledgement sent to Julian Andres Klode <jak@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Wed, 27 Jul 2011 18:03:11 GMT) (full text, mbox, link).


Message #60 received at 604268@bugs.debian.org (full text, mbox, reply):

From: Julian Andres Klode <jak@debian.org>
To: Loïc Minier <lool@dooz.org>, 604268@bugs.debian.org
Subject: Re: Bug#604268: [buildd-tools-devel] Bug#604268: Bug#604268: Bug#604268: Bug#604268: QEMU linux-user support
Date: Wed, 27 Jul 2011 20:01:13 +0200
[Message part 1 (text/plain, inline)]
On Tue, Jun 14, 2011 at 01:25:42PM +0200, Julian Andres Klode wrote:
> On Tue, Jan 25, 2011 at 03:44:29PM +0100, Loïc Minier wrote:
> > On Mon, Jan 24, 2011, Loïc Minier wrote:
> > >  While we could try parsing binfmt-support's format or binfmt_misc's
> > >  format, I think the best thing here would be to check with the
> > >  binfmt-support maintainer.  I wonder whether binfmt-support is
> > >  Debian/Ubuntu specific.  If it is, then poking the kernel format or
> > >  trying to run a binary might be best.  If it's also used on other
> > >  distros, perhaps we can get some command which tells us what the
> > >  interpreter is for a specific binary.  If that makes sense to you, I
> > >  can poke Colin about it, and perhaps open a bug report for the
> > >  binfmt-support changes.
> > 
> >  I've asked Colin about this, and he implemented support for that in
> >  trunk!  :-)   (I tested it, there was an issue in the original version
> >  but r109 fixed it)
> > 
> >  (bzr+ssh://bzr.debian.org/bzr/binfmt-support/trunk or
> >  http://bzr.debian.org/binfmt-support/trunk)
> > 
> >     % sudo apt-get build-dep binfmt-support
> >     % ./configure && make
> >     % src/update-binfmts --find $chroot/bin/true
> >     /usr/bin/qemu-arm-static
> > 
> >  Note that the --find might return multiple interpreters; in this case,
> >  the kernel tries them one by one.
> > 
> >  Perhaps we should do something like find the first static interpreter,
> >  or always copy the first interpreter and warn if there is more than
> >  one?
> > 
> Attaching my solution for this.
> 
Updated as per IRC comments. Changes since v1:

    * Run in setup-recover as well
    * Reindent in accordance to other schroot scripts
    * Simply touch the emulator if it does not exist yet instead of
      trying to write a script there.


-- 
Julian Andres Klode  - Debian Developer, Ubuntu Member

See http://wiki.debian.org/JulianAndresKlode and http://jak-linux.org/.
[99binfmt (text/plain, attachment)]
[Message part 3 (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#604268; Package schroot. (Wed, 27 Jul 2011 19:21:06 GMT) (full text, mbox, link).


Acknowledgement sent to Roger Leigh <rleigh@codelibre.net>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Wed, 27 Jul 2011 19:21:06 GMT) (full text, mbox, link).


Message #65 received at 604268@bugs.debian.org (full text, mbox, reply):

From: Roger Leigh <rleigh@codelibre.net>
To: Julian Andres Klode <jak@debian.org>, Loïc Minier <lool@dooz.org>, 604268@bugs.debian.org
Subject: Re: [buildd-tools-devel] Bug#604268: Bug#604268: Bug#604268: Bug#604268: Bug#604268: QEMU linux-user support
Date: Wed, 27 Jul 2011 20:18:05 +0100
[Message part 1 (text/plain, inline)]
On Wed, Jul 27, 2011 at 08:01:13PM +0200, Julian Andres Klode wrote:
> On Tue, Jun 14, 2011 at 01:25:42PM +0200, Julian Andres Klode wrote:
> > On Tue, Jan 25, 2011 at 03:44:29PM +0100, Loïc Minier wrote:
> > > On Mon, Jan 24, 2011, Loïc Minier wrote:
> > > >  While we could try parsing binfmt-support's format or binfmt_misc's
> > > >  format, I think the best thing here would be to check with the
> > > >  binfmt-support maintainer.  I wonder whether binfmt-support is
> > > >  Debian/Ubuntu specific.  If it is, then poking the kernel format or
> > > >  trying to run a binary might be best.  If it's also used on other
> > > >  distros, perhaps we can get some command which tells us what the
> > > >  interpreter is for a specific binary.  If that makes sense to you, I
> > > >  can poke Colin about it, and perhaps open a bug report for the
> > > >  binfmt-support changes.
> > > 
> > >  I've asked Colin about this, and he implemented support for that in
> > >  trunk!  :-)   (I tested it, there was an issue in the original version
> > >  but r109 fixed it)
> > > 
> > >  (bzr+ssh://bzr.debian.org/bzr/binfmt-support/trunk or
> > >  http://bzr.debian.org/binfmt-support/trunk)
> > > 
> > >     % sudo apt-get build-dep binfmt-support
> > >     % ./configure && make
> > >     % src/update-binfmts --find $chroot/bin/true
> > >     /usr/bin/qemu-arm-static
> > > 
> > >  Note that the --find might return multiple interpreters; in this case,
> > >  the kernel tries them one by one.
> > > 
> > >  Perhaps we should do something like find the first static interpreter,
> > >  or always copy the first interpreter and warn if there is more than
> > >  one?
> > > 
> > Attaching my solution for this.
> > 
> Updated as per IRC comments. Changes since v1:
> 
>     * Run in setup-recover as well
>     * Reindent in accordance to other schroot scripts
>     * Simply touch the emulator if it does not exist yet instead of
>       trying to write a script there.

Many thanks.  I've added this to git on the master and schroot-1.4
branches in the contrib/setup.d directory for additional setup
scripts (it won't be used by default at the moment, I'm afraid).
Once we have a multiarchified qemu and we can properly update it
in the chroot, as well as only enabling it when required, we can
have it run by default.


Thanks again,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
   `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>:
Bug#604268; Package schroot. (Mon, 28 May 2012 23:24:02 GMT) (full text, mbox, link).


Acknowledgement sent to Roger Leigh <rleigh@codelibre.net>:
Extra info received and forwarded to list. Copy sent to Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>. (Mon, 28 May 2012 23:24:02 GMT) (full text, mbox, link).


Message #70 received at 604268@bugs.debian.org (full text, mbox, reply):

From: Roger Leigh <rleigh@codelibre.net>
To: Julian Andres Klode <jak@debian.org>, Loïc Minier <lool@dooz.org>, 604268@bugs.debian.org
Subject: Re: [buildd-tools-devel] Bug#604268: Bug#604268: Bug#604268: Bug#604268: Bug#604268: QEMU linux-user support
Date: Tue, 29 May 2012 00:20:42 +0100
On Wed, Jul 27, 2011 at 08:01:13PM +0200, Julian Andres Klode wrote:
> On Tue, Jun 14, 2011 at 01:25:42PM +0200, Julian Andres Klode wrote:
> > On Tue, Jan 25, 2011 at 03:44:29PM +0100, Loïc Minier wrote:
> > > On Mon, Jan 24, 2011, Loïc Minier wrote:
> > > >  While we could try parsing binfmt-support's format or binfmt_misc's
> > > >  format, I think the best thing here would be to check with the
> > > >  binfmt-support maintainer.  I wonder whether binfmt-support is
> > > >  Debian/Ubuntu specific.  If it is, then poking the kernel format or
> > > >  trying to run a binary might be best.  If it's also used on other
> > > >  distros, perhaps we can get some command which tells us what the
> > > >  interpreter is for a specific binary.  If that makes sense to you, I
> > > >  can poke Colin about it, and perhaps open a bug report for the
> > > >  binfmt-support changes.
> > > 
> > >  I've asked Colin about this, and he implemented support for that in
> > >  trunk!  :-)   (I tested it, there was an issue in the original version
> > >  but r109 fixed it)
> > > 
> > >  (bzr+ssh://bzr.debian.org/bzr/binfmt-support/trunk or
> > >  http://bzr.debian.org/binfmt-support/trunk)
> > > 
> > >     % sudo apt-get build-dep binfmt-support
> > >     % ./configure && make
> > >     % src/update-binfmts --find $chroot/bin/true
> > >     /usr/bin/qemu-arm-static
> > > 
> > >  Note that the --find might return multiple interpreters; in this case,
> > >  the kernel tries them one by one.
> > > 
> > >  Perhaps we should do something like find the first static interpreter,
> > >  or always copy the first interpreter and warn if there is more than
> > >  one?
> > > 
> > Attaching my solution for this.
> > 
> Updated as per IRC comments. Changes since v1:
> 
>     * Run in setup-recover as well
>     * Reindent in accordance to other schroot scripts
>     * Simply touch the emulator if it does not exist yet instead of
>       trying to write a script there.

Hello,

I've now moved this script from the "contrib" directory to
/etc/schroot/setup.d/15binfmt (reordered so later scripts can make
use of this to e.g. start services and run programs inside).

I'd appreciate it if you could possibly give this a check over
to make sure it works the way you want.  I've put a snapshot at
http://people.debian.org/~rleigh/schroot/
which is just a packaged version of the current git repo.  It
would be good to know if the script is working correctly, and
if you're happy with the breaks/suggests for schroot.

I've left out all the C++ code previously mentioned in the report,
since this script makes it all completely self-configuring, and
just gets enabled transparently when needed.  We can easily add
support for tweakable options if needed; in fact, for this
version of schroot, it no longer even requires writing any C++.


Many thanks,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux    http://people.debian.org/~rleigh/
 `. `'   schroot and sbuild  http://alioth.debian.org/projects/buildd-tools
   `-    GPG Public Key      F33D 281D 470A B443 6756 147C 07B3 C8BC 4083 E800




Reply sent to Roger Leigh <rleigh@debian.org>:
You have taken responsibility. (Tue, 29 May 2012 22:09:06 GMT) (full text, mbox, link).


Notification sent to Loïc Minier <lool@dooz.org>:
Bug acknowledged by developer. (Tue, 29 May 2012 22:09:07 GMT) (full text, mbox, link).


Message #75 received at 604268-close@bugs.debian.org (full text, mbox, reply):

From: Roger Leigh <rleigh@debian.org>
To: 604268-close@bugs.debian.org
Subject: Bug#604268: fixed in schroot 1.5.3-1
Date: Tue, 29 May 2012 22:06:42 +0000
Source: schroot
Source-Version: 1.5.3-1

We believe that the bug you reported is fixed in the latest version of
schroot, which is due to be installed in the Debian FTP archive:

dchroot-dsa_1.5.3-1_amd64.deb
  to main/s/schroot/dchroot-dsa_1.5.3-1_amd64.deb
dchroot_1.5.3-1_amd64.deb
  to main/s/schroot/dchroot_1.5.3-1_amd64.deb
libsbuild-dev_1.5.3-1_amd64.deb
  to main/s/schroot/libsbuild-dev_1.5.3-1_amd64.deb
libsbuild-doc_1.5.3-1_all.deb
  to main/s/schroot/libsbuild-doc_1.5.3-1_all.deb
schroot-common_1.5.3-1_all.deb
  to main/s/schroot/schroot-common_1.5.3-1_all.deb
schroot-dbg_1.5.3-1_amd64.deb
  to main/s/schroot/schroot-dbg_1.5.3-1_amd64.deb
schroot_1.5.3-1.debian.tar.gz
  to main/s/schroot/schroot_1.5.3-1.debian.tar.gz
schroot_1.5.3-1.dsc
  to main/s/schroot/schroot_1.5.3-1.dsc
schroot_1.5.3-1_amd64.deb
  to main/s/schroot/schroot_1.5.3-1_amd64.deb
schroot_1.5.3.orig.tar.xz
  to main/s/schroot/schroot_1.5.3.orig.tar.xz



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 604268@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Roger Leigh <rleigh@debian.org> (supplier of updated schroot package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.8
Date: Tue, 29 May 2012 21:26:47 +0100
Source: schroot
Binary: schroot-common libsbuild-dev schroot-dbg libsbuild-doc schroot dchroot dchroot-dsa
Architecture: source all amd64
Version: 1.5.3-1
Distribution: unstable
Urgency: low
Maintainer: Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>
Changed-By: Roger Leigh <rleigh@debian.org>
Description: 
 dchroot    - Execute commands in a chroot environment
 dchroot-dsa - Execute commands in a chroot environment
 libsbuild-dev - development files for the Debian source builder
 libsbuild-doc - development documentation for the Debian source builder
 schroot    - Execute commands in a chroot environment
 schroot-common - common files for schroot
 schroot-dbg - schroot, dchroot and dchroot-dsa debugging symbols
Closes: 604268 674040 674041
Changes: 
 schroot (1.5.3-1) unstable; urgency=low
 .
   * New upstream development release.
   * dchroot:
     - Always use "/bin/sh -c" to run the specified command, rather than
       the user's shell, in order to ensure consistent behaviour.
   * schroot:
     - Add shell fallbacks (Closes: #674040).  When running a login
       shell, try $SHELL (if preserving the environment), or else passwd
       pw_shell, then /bin/bash and finally /bin/sh.  This may be
       overidden using the shell configuration key, which may in turn be
       overidden by the --shell option.
     - Don't warn the user about groups which do not exist
       (Closes: #674041).  This is now debug log info only.
     - Add support for running programs in non-native architecture
       chroots using binfmt support for qemu user binaries
       (Closes: #604268).  Thanks to Loïc Minier, Julian Andres Klode
       and Colin Watson.
Checksums-Sha1: 
 cf1d0e442d4a4c6fb3184a99a3178e10d3a8eb46 2424 schroot_1.5.3-1.dsc
 1037078bfb3dc9563a37f51f573112c2c706aebf 721204 schroot_1.5.3.orig.tar.xz
 0070ea91f10320a7bd48d58eedcc6629f2d627d7 27885 schroot_1.5.3-1.debian.tar.gz
 6e0cbc2ff3d6d09be19ac735fabf651b6e30b013 255274 schroot-common_1.5.3-1_all.deb
 3c93492dfffb03bd95c2368880b06ab4065a6bf8 2293348 libsbuild-dev_1.5.3-1_amd64.deb
 8cf95553d18e527bfaf15562390a6068fcc8dd31 29722960 schroot-dbg_1.5.3-1_amd64.deb
 63a05d8ca4710ce4119e70baaaf6710ab85a5b42 8257012 libsbuild-doc_1.5.3-1_all.deb
 9e0bead60e12cdfa7ce1be597ca9c572bcb7fe25 958126 schroot_1.5.3-1_amd64.deb
 a4c1acce81ae091cf400f4f19308a6e19efe77cd 372800 dchroot_1.5.3-1_amd64.deb
 1b84a9ec0b0c8f819d0e8d9e7a0f1e4ea3a41135 372236 dchroot-dsa_1.5.3-1_amd64.deb
Checksums-Sha256: 
 4f2b9fd9ef53e22e2564165cdc47e844cd05cea8d78a16b5b93239a6dab79fdc 2424 schroot_1.5.3-1.dsc
 bdbc66c357d2a3ac14a1a2bfad1176b1ccfb3628c2b4607354ae3e26c03aafb1 721204 schroot_1.5.3.orig.tar.xz
 84d41377e26efb468be74591619bed329d3cd0b6e0fb25ca383124e432158c99 27885 schroot_1.5.3-1.debian.tar.gz
 d6d4af344e790d9c6d5001a7413ad9f605130965a22b08a4a02317c3cb1efd6b 255274 schroot-common_1.5.3-1_all.deb
 732035fdac3632fd246b2454be4731c14e7f720084edaddb4b0f6c6bbfa992d9 2293348 libsbuild-dev_1.5.3-1_amd64.deb
 df828605b618886a1b068f31112e58439bb3c8e7d39824b6d188044f711e0063 29722960 schroot-dbg_1.5.3-1_amd64.deb
 48ea7339481244ff29633aa65a9febf548204b459cbdbb5acc811087df0fd370 8257012 libsbuild-doc_1.5.3-1_all.deb
 8811200220414b765b1ed304037cc49dad101d30884509efef6f5a152ede4f25 958126 schroot_1.5.3-1_amd64.deb
 d5dae2107cf3d46fa5eff696ae409ad26a19d9ded33c1f478422d5fc4198adee 372800 dchroot_1.5.3-1_amd64.deb
 0d677523baa5ede32d8bcfa65dfa6d9d5bb21fbbc5ff1d2f8a8cc1cb30da8a98 372236 dchroot-dsa_1.5.3-1_amd64.deb
Files: 
 750d0afa6d08608146283a9739c07e2a 2424 admin optional schroot_1.5.3-1.dsc
 ef9ffb7c4fe25c074c9f3544ec14085a 721204 admin optional schroot_1.5.3.orig.tar.xz
 7ce62eb64d478f04e8ddecea9c60b18c 27885 admin optional schroot_1.5.3-1.debian.tar.gz
 d479a9ec4db4e3230c4e99624b775f87 255274 admin optional schroot-common_1.5.3-1_all.deb
 66ea444ae35d7ad5491f094f0db80174 2293348 libdevel optional libsbuild-dev_1.5.3-1_amd64.deb
 23c813cf19fa552ad10c3755ac4b2f39 29722960 debug extra schroot-dbg_1.5.3-1_amd64.deb
 c12b4731ab392d460ece1e179ecbafc1 8257012 doc optional libsbuild-doc_1.5.3-1_all.deb
 ea12bd508948c58e4db32c6799b33254 958126 admin optional schroot_1.5.3-1_amd64.deb
 8815d13362a861dd8259721368177e1d 372800 admin optional dchroot_1.5.3-1_amd64.deb
 961271b58a3998ac832c4d09396d53f2 372236 admin optional dchroot-dsa_1.5.3-1_amd64.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iQIcBAEBCgAGBQJPxUNxAAoJEOJSSsUKn1xZ5PUQAKyQo3gSbu2iW2eLC+8zjuoe
IyeUFfzm+P2hKD6Od+6kCTi2fCgNIjOeeSbo0wiIQFTGVLq7l/JMjQlZky0KhnB4
84cGoHRflvLKsZrwR6LdFVmv87aO0LzHjwxOj4PVT8jNlNFkoPgahWIzca1MQIRg
0G3dVnG5sUW6yVkmptYi23SKtUaPPlaonpDHijWJAxpwhd3y/ijA7uxNKMvbNOod
V6p571eL0Z+A9rEVdYH9I7dNtphyg4AJonIkqQDU9hzzOHGEOMGMJz+gUCxXBd7x
D5vhxmpJwZ16bjLj48r7i0H84geyC9kN1C4zAIoZGA9ioUZpJRmxYbzbX2lx8z9X
QWDV0NhskvvjydBlIrNy+h9yi4xFVzvR+zGUenKLLDwXLxGE+ZfKChlYUfnWy3K4
08ZfRsWdQuztok8dkGVIuk3Yu55+E+He6cOBIhA+lQV3sU0jPSZbIiLxtFE6PJBf
VvNWY55L+WCtyazjYCouUeseDprWMlcT7nI3JR0m+B6ALpO2X81Ldi0wWKlznSqO
omFddMkVy/7SQgZBbiPpQsBGGUJnlWyuExMgB7cTjWZyxJs/BF+iAWDthXTvVAhW
Z84wPbPUOJ+noUX8VcHiUffk6IcjHCxd8meC29lg4RJo8G+ZhgtnZsNVIqH86h+m
Kma3AOISAbi17ERDXkDi
=7ddP
-----END PGP SIGNATURE-----





Bug archived. Request was from Debbugs Internal Request <owner@bugs.debian.org> to internal_control@bugs.debian.org. (Thu, 28 Jun 2012 07:48:25 GMT) (full text, mbox, link).


Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Tue Jan 30 06:52:48 2024; Machine Name: bembo

Debian Bug tracking system

Debbugs is free software and licensed under the terms of the GNU Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.

Copyright © 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson, 2005-2017 Don Armstrong, and many other contributors.