Debian Bug report logs - #701936
btrfs can't fsck /run/rootdev on boot with sysvinit

version graph

Package: btrfs-tools; Maintainer for btrfs-tools is Dimitri John Ledkov <xnox@debian.org>; Source for btrfs-tools is src:btrfs-tools.

Reported by: Axel Beckert <abe@debian.org>

Date: Thu, 28 Feb 2013 23:18:02 UTC

Severity: critical

Tags: sid

Found in versions btrfs-tools/0.19+20130131-2, btrfs-tools/0.19+20130315-1

Done: Dmitrijs Ledkovs <xnox@debian.org>

Bug is archived. No further changes may be made.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, 701776@bugs.debian.org, Klaus@Ethgen.de, yann.soubeyrand@gmx.fr, ratcheer@gmail.com, christian.stalp@gmx.de, bzed@debian.org, Daniel Baumann <daniel.baumann@progress-technologies.net>:
Bug#701936; Package btrfs-tools. (Thu, 28 Feb 2013 23:18:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to Axel Beckert <abe@debian.org>:
New Bug report received and forwarded. Copy sent to 701776@bugs.debian.org, Klaus@Ethgen.de, yann.soubeyrand@gmx.fr, ratcheer@gmail.com, christian.stalp@gmx.de, bzed@debian.org, Daniel Baumann <daniel.baumann@progress-technologies.net>. (Thu, 28 Feb 2013 23:18:05 GMT) Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: Axel Beckert <abe@debian.org>
To: submit@bugs.debian.org
Subject: btrfs-tools: renders systems with btrfs root file systems unbootable: "check_mounted(): Could not open /run/rootdev" + "Could not check mount status: No such device or address"
Date: Fri, 1 Mar 2013 00:14:39 +0100
Package: btrfs-tools
Version: 0.19+20130131-2
Severity: critical
Justification: renders systems with btrfs as root file-system unbootable

Hi,

Axel Beckert wrote on Wed, 27 Feb 2013 20:01:30 +0100:
> >  btrfs-tools (0.19+20130131-2) unstable; urgency=low
> >  .
> >    * Replacing fsck.btrfs with wrapper arround 'btrfs check' to avoid
> >      different behaviour based on the filename btrfs is copied to (Closes:
> >      #701776).
> 
> Thanks for the quick fix, but unfortunately the problem persists, just
> with a different error message and a different exit-code, but with the
> same symptom: The system fails to start.
> 
> check_mounted(): Could not open /run/rootdev
> Could not check mount status: No such device or address
> fsck died with exit status 250

There's one more confirmation that there's still an issue which breaks
systems, even with the newest version, at
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=701776#65

Additionally, several people contacted me on Jabber/IRC about how to
fix this issue, too, so I'm posting a recipe at the end of this mail
and also Cc this mail to #701776.

> Do you have any preference if #701776 should be reopend or if a new
> bug should be filed for that?

Since I didn't get a reply from Daniel on this question within a day,
I'm herewith filing a new bug report as the issue may have a different
cause despite very similar symptoms and the affected systems still
being broken if you don't downgrade btrfs-tools to 0.19+20121004-1 or
earlier.

As I already wrote in #701776, I consider such breakage of severity
critical since http://www.debian.org/Bugs/Developer.en.html#severities
defines "critical" as "makes unrelated software on the system (or the
whole system) break". I see this condition as given.


For all those who are also suffering from this issue, too, here's the
recipe how I downgraded btrfs-tools to the version from testing again
on my system:

At the maintenance mode password prompt, enter the root password.

You may need to add /sbin and /usr/sbin to the path depending on your
login shell setup. Thanks Christian Stalp!
# export PATH="/sbin:/usr/sbin:$PATH"

With the default setup, sourcing /etc/profile should suffice.
# . /etc/profile

Mount the root file system read-write:
# mount -o remount,rw /

Mount the /boot partition, if existing:
# mount /boot

Either get network, ...
# dhclient eth0

(Since Julien Cristau NMUed btrfs-tools in Wheezy yesterday[1], you
may want to update your package lists, too.)
# apt-get update

..., fetch the version from testing (assuming you have a deb-line with
testing in your sources.list, too) and downgrade to it, ...
# apt-get install -t testing btrfs-tools

... or, if you haven't done "apt-get clean" for a while, you may be able to
just do
# dpkg -i /var/cache/apt/archives/btrfs-tools_0.19+20121004-1_*.deb

Then call sync to make sure the new version is on disk:

# sync

and reboot, e.g. by pressing Ctrl-D.

That way, both, Christian and me got working systems back again.


[1] http://packages.qa.debian.org/b/btrfs-tools/news/20130228T204729Z.html

		Regards, Axel
-- 
 ,''`.  |  Axel Beckert <abe@debian.org>, http://people.debian.org/~abe/
: :' :  |  Debian Developer, ftp.ch.debian.org Admin
`. `'   |  1024D: F067 EA27 26B9 C3FC 1486  202E C09E 1D89 9593 0EDE
  `-    |  4096R: 2517 B724 C5F6 CA99 5329  6E61 2FF9 CD59 6126 16B5



Information forwarded to debian-bugs-dist@lists.debian.org, Daniel Baumann <daniel.baumann@progress-technologies.net>:
Bug#701936; Package btrfs-tools. (Fri, 01 Mar 2013 02:18:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to Axel Beckert <abe@debian.org>:
Extra info received and forwarded to list. Copy sent to Daniel Baumann <daniel.baumann@progress-technologies.net>. (Fri, 01 Mar 2013 02:18:05 GMT) Full text and rfc822 format available.

Message #10 received at 701936@bugs.debian.org (full text, mbox):

From: Axel Beckert <abe@debian.org>
To: 701936@bugs.debian.org
Subject: Re: Bug#701936: btrfs-tools: renders systems with btrfs root file systems unbootable: "check_mounted(): Could not open /run/rootdev" + "Could not check mount status: No such device or address"
Date: Fri, 1 Mar 2013 03:15:15 +0100
Hi,

some more information about the issue:

Happens on Sid i386 for me. /boot is a separate partition and / is a
btrfs upon a dm_crypt partition.

Axel Beckert wrote:
> > check_mounted(): Could not open /run/rootdev
> > Could not check mount status: No such device or address
> > fsck died with exit status 250

I provoked this situation again and also tried Dmitrijs's patch from
http://bugs.debian.org/cgi-bin/bugreport.cgi?msg=45;filename=fix-up-fsck.patch;att=1;bug=701776
in combination with Daniel's new wrapper, but that doesn't change
anything.

/run/rootdev exists in that situation and looks like this:

brw------- 1 root root 0, 16 Mar  1 00:56 /run/rootdev

It's not mounted at that point. (The root partition is mounted "ro" at
/dev/mapper/sda2_crypt in my case.)

From utils.c:

    812         fd = open(file, O_RDONLY);
    813         if (fd < 0) {
    814                 fprintf (stderr, "check_mounted(): Could not open %s\n", file);
    815                 return -errno;
    816         }

From open(3):

    ENXIO  The named file is a character special or block special file,
    	   and the device associated with this special file does not exist.

So "btrfs check" tries to read /run/rootdev but can't find the
according device to that block special file.

Potential explanations for this behaviour:

* /run/rootdev has the wrong major and minor numbers. (But it
  worked before, e.g. with btrfs-tools from testing. Nevertheless
  other tools like "less -f" provoke the same error.)

* "btrfs check" accesses the blockdevice in a different way than
  btrfsck did. (But I couldn't find any relevant difference in the
  source code compared to the code of the working version on a first a
  glance.)

So both explanations seem unrealistic for some reasons... Which
suggests that the culprit is probably somewhere else. :-/

		Regards, Axel
-- 
 ,''`.  |  Axel Beckert <abe@debian.org>, http://people.debian.org/~abe/
: :' :  |  Debian Developer, ftp.ch.debian.org Admin
`. `'   |  1024D: F067 EA27 26B9 C3FC 1486  202E C09E 1D89 9593 0EDE
  `-    |  4096R: 2517 B724 C5F6 CA99 5329  6E61 2FF9 CD59 6126 16B5



Changed Bug title to 'btrfs can't fsck /run/rootdev on boot' from 'btrfs-tools: renders systems with btrfs root file systems unbootable: "check_mounted(): Could not open /run/rootdev" + "Could not check mount status: No such device or address"' Request was from Daniel Baumann <daniel.baumann@progress-technologies.net> to control@bugs.debian.org. (Fri, 01 Mar 2013 06:33:03 GMT) Full text and rfc822 format available.

Severity set to 'serious' from 'critical' Request was from Daniel Baumann <daniel.baumann@progress-technologies.net> to control@bugs.debian.org. (Fri, 01 Mar 2013 06:33:04 GMT) Full text and rfc822 format available.

Added tag(s) sid. Request was from Daniel Baumann <daniel.baumann@progress-technologies.net> to control@bugs.debian.org. (Fri, 01 Mar 2013 06:33:04 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Daniel Baumann <daniel.baumann@progress-technologies.net>:
Bug#701936; Package btrfs-tools. (Fri, 01 Mar 2013 06:54:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to daniel.baumann@progress-technologies.net:
Extra info received and forwarded to list. Copy sent to Daniel Baumann <daniel.baumann@progress-technologies.net>. (Fri, 01 Mar 2013 06:54:03 GMT) Full text and rfc822 format available.

Message #21 received at 701936@bugs.debian.org (full text, mbox):

From: Daniel Baumann <daniel.baumann@progress-technologies.net>
To: 701936@bugs.debian.org
Subject: Re: btrfs can't fsck /run/rootdev on boot
Date: Fri, 01 Mar 2013 07:50:37 +0100
retitle 701936 btrfs can't fsck /run/rootdev on boot with sysvinit
severity 701936 important
clone 701936 -1
reassign -1 sysvinit
thanks

works with systemd, it's sysvinit specific.

-- 
Address:        Daniel Baumann, Donnerbuehlweg 3, CH-3012 Bern
Email:          daniel.baumann@progress-technologies.net
Internet:       http://people.progress-technologies.net/~daniel.baumann/



Changed Bug title to 'btrfs can't fsck /run/rootdev on boot with sysvinit' from 'btrfs can't fsck /run/rootdev on boot' Request was from Daniel Baumann <daniel.baumann@progress-technologies.net> to control@bugs.debian.org. (Fri, 01 Mar 2013 06:54:05 GMT) Full text and rfc822 format available.

Severity set to 'important' from 'serious' Request was from Daniel Baumann <daniel.baumann@progress-technologies.net> to control@bugs.debian.org. (Fri, 01 Mar 2013 06:54:05 GMT) Full text and rfc822 format available.

Bug 701936 cloned as bug 701956 Request was from Daniel Baumann <daniel.baumann@progress-technologies.net> to control@bugs.debian.org. (Fri, 01 Mar 2013 06:54:06 GMT) Full text and rfc822 format available.

Removed tag(s) sid. Request was from Julien Cristau <jcristau@debian.org> to control@bugs.debian.org. (Fri, 01 Mar 2013 08:15:10 GMT) Full text and rfc822 format available.

Added tag(s) sid. Request was from Daniel Baumann <daniel.baumann@progress-technologies.net> to control@bugs.debian.org. (Fri, 01 Mar 2013 08:33:03 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Daniel Baumann <daniel.baumann@progress-technologies.net>:
Bug#701936; Package btrfs-tools. (Fri, 01 Mar 2013 10:09:11 GMT) Full text and rfc822 format available.

Acknowledgement sent to Klaus Ethgen <Klaus@Ethgen.de>:
Extra info received and forwarded to list. Copy sent to Daniel Baumann <daniel.baumann@progress-technologies.net>. (Fri, 01 Mar 2013 10:09:11 GMT) Full text and rfc822 format available.

Message #36 received at 701936@bugs.debian.org (full text, mbox):

From: Klaus Ethgen <Klaus@Ethgen.de>
To: Axel Beckert <abe@debian.org>, 701936@bugs.debian.org
Subject: Re: Bug#701936: btrfs-tools: renders systems with btrfs root file systems unbootable: "check_mounted(): Could not open /run/rootdev" + "Could not check mount status: No such device or address"
Date: Fri, 1 Mar 2013 10:10:56 +0100
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Am Fr den  1. Mär 2013 um  0:14 schrieb Axel Beckert:
> For all those who are also suffering from this issue, too, here's the
> recipe how I downgraded btrfs-tools to the version from testing again
> on my system:
> 
> At the maintenance mode password prompt, enter the root password.
> 
> You may need to add /sbin and /usr/sbin to the path depending on your
> login shell setup. Thanks Christian Stalp!
> # export PATH="/sbin:/usr/sbin:$PATH"
> 
> With the default setup, sourcing /etc/profile should suffice.
> # . /etc/profile
> 
> Mount the root file system read-write:
> # mount -o remount,rw /
> 
> Mount the /boot partition, if existing:
> # mount /boot
> 
> Either get network, ...
> # dhclient eth0
> 
> (Since Julien Cristau NMUed btrfs-tools in Wheezy yesterday[1], you
> may want to update your package lists, too.)
> # apt-get update
> 
> ..., fetch the version from testing (assuming you have a deb-line with
> testing in your sources.list, too) and downgrade to it, ...
> # apt-get install -t testing btrfs-tools
> 
> ... or, if you haven't done "apt-get clean" for a while, you may be able to
> just do
> # dpkg -i /var/cache/apt/archives/btrfs-tools_0.19+20121004-1_*.deb
> 
> Then call sync to make sure the new version is on disk:
> 
> # sync
> 
> and reboot, e.g. by pressing Ctrl-D.

It is even easier to fix. Even more, that method above will not work if
you have a proper partition layout keeping /usr and /var on different
partitions.

Just mount your / rw on the prompt, then replace the broken fsck by
/bin/true:
   rm /sbin/fsck.btrfs
   ln -s /bin/true !$

Sync and reboot. No need for network, no need to fiddle with installing
stuff on this very limited prompt (in some cases) ... More fail-safe.

After the reboot it is easy to just let it this way until a fixed
package comes out or install a older version like Axel mentioned above.
But there is no need for it as the fsck of old version is even just a
dummy.

Regards
   Klaus
- -- 
Klaus Ethgen                              http://www.ethgen.ch/
pub  4096R/4E20AF1C 2011-05-16   Klaus Ethgen <Klaus@Ethgen.de>
Fingerprint: 85D4 CA42 952C 949B 1753  62B3 79D0 B06F 4E20 AF1C
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iQGcBAEBCgAGBQJRMHCfAAoJEKZ8CrGAGfasGhsMALeEhS6b+lh4w7K2Ahb++oWE
d0TYa0ZMUdsNbMulA6UcsmBk2NFuvUS1BI7nq2mjiscLIjYFRgN2u1zgrCpIgFNd
erpCnYGaY8ujH+6qzHguIBN6tSuO5OZcKndnVfnEYo4T43HfgK+jhxN0Qk6JTAgR
Rm+jurDHCigox6T/AZGhMQP1zkS9Elv1Pj4VwTQCjvY9ryefGh8dgYV7DnnpHlef
i4PAem51PuMAxIjB8VmMwOllEWg0rJXmyi0E+CC3GRzNbppCtsyVEh2J9DdZKBp8
pQDNYbYNiPkQ++d39OgoegxanmC5niKbKUhX9qP005l9ugigQe9VJFgPlS7KcM+r
lNpKjpSAsmKwSEkcQrXfAsmjAXP096Fv+9EOX8BniMAPbZyHjnVDOAyEN6BA9xjQ
aFcqaqgDTg63H6RnnEwG925DcwsMZ4CEIVcWDoZcQevuXcG3wqrJaz3Hm2jGw/Kh
bitBdWtsecWQ6MiTHXkdcv0FYTFR+W5Ot4AXD0+KZQ==
=q2Dd
-----END PGP SIGNATURE-----



Information forwarded to debian-bugs-dist@lists.debian.org, Daniel Baumann <daniel.baumann@progress-technologies.net>:
Bug#701936; Package btrfs-tools. (Wed, 06 Mar 2013 20:09:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to Roger Leigh <rleigh@codelibre.net>:
Extra info received and forwarded to list. Copy sent to Daniel Baumann <daniel.baumann@progress-technologies.net>. (Wed, 06 Mar 2013 20:09:05 GMT) Full text and rfc822 format available.

Message #41 received at 701936@bugs.debian.org (full text, mbox):

From: Roger Leigh <rleigh@codelibre.net>
To: daniel.baumann@progress-technologies.net, 701936@bugs.debian.org
Cc: debian-release@lists.debian.org
Subject: Re: Bug#701936: btrfs can't fsck /run/rootdev on boot
Date: Wed, 6 Mar 2013 20:05:02 +0000
reassign 701936 initscripts
severity 701936 serious
thanks

On Fri, Mar 01, 2013 at 07:50:37AM +0100, Daniel Baumann wrote:
> retitle 701936 btrfs can't fsck /run/rootdev on boot with sysvinit
> severity 701936 important
> clone 701936 -1
> reassign -1 sysvinit
> thanks
> 
> works with systemd, it's sysvinit specific.

This is largely init-system agnostic.  There's only a single issue
with initscripts, and even that's arguably a Btrfs issue.  Please do
not use "works with systemd" as an excuse for gratuitous brokenness.
It's broken everywhere.

There are a number of serious problems here.  I'll go through each
in turn.


1) checkroot.sh creates an invalid /run/rootdev

Btrfs, unlike every other filesystem I'm aware of, reports an
invalid device with stat(2).

    % findmnt / 
    TARGET SOURCE    FSTYPE OPTIONS
    /      /dev/sda4 btrfs  rw,nodev,relatime,ssd,discard,space_cache

    % stat / | grep ^Device
    Device: 10h/16d	Inode: 256         Links: 1

    % mountpoint -qx /dev/sda4
    8:4

    % mountpoint -d /         
    0:16

The mountpoint discrepancy triggers the creation of /run/rootdev, but
since the block device is actually *invalid*, this therefore causes
fsck to fail.  This is the root cause of this bug.  checkroot.sh makes
the assumption that the filesystem device is valid; this is not the
case with Btrfs.  Up until now, this assumption has been valid in all
the circumstances triggering this codepath.

We can teach checkroot to prefer the mount device, but this is not
always a good choice (there is a reason why we have this particular
fallback).  I have tried this, and it does allow fsck ro run.  But for
non-Btrfs rootfses, this is the wrong thing to do.


2) fsck.btrfs fails to fsck a mounted filesystem

fsck.btrfs won't check a mounted filesystem, even if mounted
read-only.  We need to be able to do this, since we are running
fsck from the rootfs.  We do this for all other filesystem types.


3) fsck.btrfs does not support the standard fsck options

fsck.btrfs does not include support for most of the options in
fsck(8), even to ignore them.  Since we make use of these options,
fsck.btrfs breaks.


4) fsck.btrfs error codes

I haven't tested this due to point (2) above, but if you look at
checkroot.sh, you'll notice that the exit codes are quite
important for doing the right thing for fsck failures and in
some cases success.  fsck.btrfs *must* use the same codes.


So, to summarise the current situation:

• systems with a btrfs root filesystem are currently *unbootable*
  without using "fastboot" to skip fsck
• even if the checkroot script is "fixed", fsck.btrfs remains
  broken and all the unbootable systems remain unbootable
• at this late stage in the freeze, btrfs-tools should never
  have been uploaded to unstable
  · it's fundamentally broken
  · it's broken countless systems (including my own)
  · it's obviously not been tested properly; these tools are
    fundamental to basic system functioning, and I expect
    better quality work from a Debian developer
  · this is obviously unsuitable for wheezy
• I'm loath to make any changes to initscripts to work around this
  breakage, not only because it won't fix the root cause of the
  problems, but because to change the core scripts at this point
  would be to compromise the months of testing they have had, and
  that's simply unacceptable

I would recommend that this be immediately reverted in unstable.
If you want to put it into experimental, feel free, but please
add a big disclaimer to avoid further breakage.  Given the
brokenness, probably best not uploaded at all until it will
not break booting.

If Btrfs needs special handling in initscripts due to unique
requirements, then I'm happy to look at adding such support.
However... you should have communicated this to me /before/
uploading this and breaking lots of systems, so that the
support would have been in place ahead of time.  And you should
have also done some testing to avoid breaking so many people's
computers; this is just not acceptable.


Regards,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux    http://people.debian.org/~rleigh/
 `. `'   schroot and sbuild  http://alioth.debian.org/projects/buildd-tools
   `-    GPG Public Key      F33D 281D 470A B443 6756 147C 07B3 C8BC 4083 E800



Bug reassigned from package 'btrfs-tools' to 'initscripts'. Request was from Roger Leigh <rleigh@codelibre.net> to control@bugs.debian.org. (Wed, 06 Mar 2013 20:09:07 GMT) Full text and rfc822 format available.

No longer marked as found in versions btrfs-tools/0.19+20130131-2. Request was from Roger Leigh <rleigh@codelibre.net> to control@bugs.debian.org. (Wed, 06 Mar 2013 20:09:08 GMT) Full text and rfc822 format available.

Severity set to 'serious' from 'important' Request was from Roger Leigh <rleigh@codelibre.net> to control@bugs.debian.org. (Wed, 06 Mar 2013 20:09:08 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Debian sysvinit maintainers <pkg-sysvinit-devel@lists.alioth.debian.org>:
Bug#701936; Package initscripts. (Wed, 06 Mar 2013 20:18:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to daniel.baumann@progress-technologies.net:
Extra info received and forwarded to list. Copy sent to Debian sysvinit maintainers <pkg-sysvinit-devel@lists.alioth.debian.org>. (Wed, 06 Mar 2013 20:18:03 GMT) Full text and rfc822 format available.

Message #52 received at 701936@bugs.debian.org (full text, mbox):

From: Daniel Baumann <daniel.baumann@progress-technologies.net>
To: Roger Leigh <rleigh@codelibre.net>
Cc: 701936@bugs.debian.org, debian-release@lists.debian.org
Subject: Re: Bug#701936: btrfs can't fsck /run/rootdev on boot
Date: Wed, 06 Mar 2013 21:16:01 +0100
On 03/06/2013 09:05 PM, Roger Leigh wrote:
> 1) checkroot.sh creates an invalid /run/rootdev

afaic, this is the only "real" problem.

> 2) fsck.btrfs fails to fsck a mounted filesystem
> 
> fsck.btrfs won't check a mounted filesystem, even if mounted
> read-only.  We need to be able to do this, since we are running
> fsck from the rootfs.  We do this for all other filesystem types.

the common workaround that e.g. other distributions do is to do this in
initramfs. probably debian should do that for all filesystems in future too.

> 3) fsck.btrfs does not support the standard fsck options

easy fixable, but hasn't much to do with the current remaining problem
in sid.

> 4) fsck.btrfs error codes
> 
> I haven't tested this due to point (2) above

me neither, but that's easy fixable too and hasn't much to do with the
current remaining problem in sid.

> • systems with a btrfs root filesystem are currently *unbootable*
>   without using "fastboot" to skip fsck

*iff* sysvinit is used.

> • even if the checkroot script is "fixed", fsck.btrfs remains
>   broken and all the unbootable systems remain unbootable

the current package works fine on systems with systemd (not yet fixed
points 3 and 4 from above are not breaking it).

> I would recommend that this be immediately reverted in unstable.

i disagree.

-- 
Address:        Daniel Baumann, Donnerbuehlweg 3, CH-3012 Bern
Email:          daniel.baumann@progress-technologies.net
Internet:       http://people.progress-technologies.net/~daniel.baumann/



Added indication that 701936 affects btrfs-tools Request was from Axel Beckert <abe@debian.org> to control@bugs.debian.org. (Wed, 06 Mar 2013 20:42:07 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Debian sysvinit maintainers <pkg-sysvinit-devel@lists.alioth.debian.org>:
Bug#701936; Package initscripts. (Wed, 06 Mar 2013 20:54:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Roger Leigh <rleigh@codelibre.net>:
Extra info received and forwarded to list. Copy sent to Debian sysvinit maintainers <pkg-sysvinit-devel@lists.alioth.debian.org>. (Wed, 06 Mar 2013 20:54:03 GMT) Full text and rfc822 format available.

Message #59 received at 701936@bugs.debian.org (full text, mbox):

From: Roger Leigh <rleigh@codelibre.net>
To: Daniel Baumann <daniel.baumann@progress-technologies.net>
Cc: 701936@bugs.debian.org, debian-release@lists.debian.org
Subject: Re: Bug#701936: btrfs can't fsck /run/rootdev on boot
Date: Wed, 6 Mar 2013 20:51:25 +0000
On Wed, Mar 06, 2013 at 09:16:01PM +0100, Daniel Baumann wrote:

> > 2) fsck.btrfs fails to fsck a mounted filesystem
> > 
> > fsck.btrfs won't check a mounted filesystem, even if mounted
> > read-only.  We need to be able to do this, since we are running
> > fsck from the rootfs.  We do this for all other filesystem types.
> 
> the common workaround that e.g. other distributions do is to do this in
> initramfs. probably debian should do that for all filesystems in future too.

What about *right now*, today.  What we "should" do is irrelevant
to the problem right now.  And it's not like doing it in the
initramfs is the only "correct" way to do this.  We've done it
the existing way since forever.  And you broke that.  That's
unacceptable.

> > 3) fsck.btrfs does not support the standard fsck options
> 
> easy fixable, but hasn't much to do with the current remaining problem
> in sid.

Yes, it does.  We use those options.  And that breaks things.

> > 4) fsck.btrfs error codes
> > 
> > I haven't tested this due to point (2) above
> 
> me neither, but that's easy fixable too and hasn't much to do with the
> current remaining problem in sid.

Yes, it does.  If they don't work, then it will again break the
checkroot script.  Please do check this.

> > • systems with a btrfs root filesystem are currently *unbootable*
> >   without using "fastboot" to skip fsck
> 
> *iff* sysvinit is used.

It's the default init system for crying out loud.  You broke
booting with the default init system for all people using a
btrfs rootfs.  And even if it was not the default, it's still
massively broken.  Would breaking booting with systemd or
upstart be any more acceptable?  No, it would not.  End this
ridiculous line of reasoning right now.

> > • even if the checkroot script is "fixed", fsck.btrfs remains
> >   broken and all the unbootable systems remain unbootable
> 
> the current package works fine on systems with systemd (not yet fixed
> points 3 and 4 from above are not breaking it).

That does not solve the problem for the vast majority who are not
using systemd.  Think for a minute about how much breakage you've
just caused.

> > I would recommend that this be immediately reverted in unstable.
> 
> i disagree.

If you are not going to revert this, then please tell me what
you are going to do to fix this, today.  Leaving people's
systems in a broken state is not on.  It must be fixed, and
fixed very quickly.  There are lots of people who can't boot
their computers thanks to this recklessness.

Please, revert it today.


Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux    http://people.debian.org/~rleigh/
 `. `'   schroot and sbuild  http://alioth.debian.org/projects/buildd-tools
   `-    GPG Public Key      F33D 281D 470A B443 6756 147C 07B3 C8BC 4083 E800



Bug reassigned from package 'initscripts' to 'btrfs-tools'. Request was from Roger Leigh <rleigh@codelibre.net> to control@bugs.debian.org. (Wed, 06 Mar 2013 21:09:03 GMT) Full text and rfc822 format available.

Severity set to 'critical' from 'serious' Request was from Roger Leigh <rleigh@codelibre.net> to control@bugs.debian.org. (Wed, 06 Mar 2013 21:09:04 GMT) Full text and rfc822 format available.

Severity set to 'important' from 'critical' Request was from Daniel Baumann <daniel.baumann@progress-technologies.net> to control@bugs.debian.org. (Wed, 06 Mar 2013 21:12:16 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Daniel Baumann <daniel.baumann@progress-technologies.net>:
Bug#701936; Package btrfs-tools. (Wed, 06 Mar 2013 21:45:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to daniel.baumann@progress-technologies.net:
Extra info received and forwarded to list. Copy sent to Daniel Baumann <daniel.baumann@progress-technologies.net>. (Wed, 06 Mar 2013 21:45:05 GMT) Full text and rfc822 format available.

Message #70 received at 701936@bugs.debian.org (full text, mbox):

From: Daniel Baumann <daniel.baumann@progress-technologies.net>
To: Roger Leigh <rleigh@codelibre.net>
Cc: 701936@bugs.debian.org, debian-release@lists.debian.org
Subject: Re: Bug#701936: btrfs can't fsck /run/rootdev on boot
Date: Wed, 06 Mar 2013 22:41:05 +0100
On 03/06/2013 09:51 PM, Roger Leigh wrote:
>>> 3) fsck.btrfs does not support the standard fsck options
>>
>> easy fixable, but hasn't much to do with the current remaining problem
>> in sid.
> 
> Yes, it does.

no, it does not.

even if fsck.btrfs ignores all required options, it still fails because
of sysvinits /run/rootdev.

>>> 4) fsck.btrfs error codes
>>>
>>> I haven't tested this due to point (2) above
>>
>> me neither, but that's easy fixable too and hasn't much to do with the
>> current remaining problem in sid.
> 
> Yes, it does.

no, it does not.

even if fsck.btrfs returns equal values, it still fails because of
sysvinits /run/rootdev.

> You broke booting with the default init system for all people using a
> btrfs rootfs.

if people are using unstable, yes.

keep in mind that eventhough it is not desirable nor intended, bugs do
happen. also in unstable.

unrelated to that, i would appreciate if you could calm down your tone a
bit, it is more productive that way, thank you.

-- 
Address:        Daniel Baumann, Donnerbuehlweg 3, CH-3012 Bern
Email:          daniel.baumann@progress-technologies.net
Internet:       http://people.progress-technologies.net/~daniel.baumann/



Information forwarded to debian-bugs-dist@lists.debian.org, Daniel Baumann <daniel.baumann@progress-technologies.net>:
Bug#701936; Package btrfs-tools. (Wed, 06 Mar 2013 22:03:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Roger Leigh <rleigh@codelibre.net>:
Extra info received and forwarded to list. Copy sent to Daniel Baumann <daniel.baumann@progress-technologies.net>. (Wed, 06 Mar 2013 22:03:04 GMT) Full text and rfc822 format available.

Message #75 received at 701936@bugs.debian.org (full text, mbox):

From: Roger Leigh <rleigh@codelibre.net>
To: Daniel Baumann <daniel.baumann@progress-technologies.net>
Cc: 701936@bugs.debian.org, debian-release@lists.debian.org
Subject: Re: Bug#701936: btrfs can't fsck /run/rootdev on boot
Date: Wed, 6 Mar 2013 21:59:28 +0000
On Wed, Mar 06, 2013 at 10:41:05PM +0100, Daniel Baumann wrote:
> On 03/06/2013 09:51 PM, Roger Leigh wrote:
> >>> 3) fsck.btrfs does not support the standard fsck options
> >>
> >> easy fixable, but hasn't much to do with the current remaining problem
> >> in sid.
> > 
> > Yes, it does.
> 
> no, it does not.
> 
> even if fsck.btrfs ignores all required options, it still fails because
> of sysvinits /run/rootdev.
> 
> >>> 4) fsck.btrfs error codes
> >>>
> >>> I haven't tested this due to point (2) above
> >>
> >> me neither, but that's easy fixable too and hasn't much to do with the
> >> current remaining problem in sid.
> > 
> > Yes, it does.
> 
> no, it does not.
> 
> even if fsck.btrfs returns equal values, it still fails because of
> sysvinits /run/rootdev.

When the /run/rootdev issue is fixed, all of the above are
points of failure.  Both point (2) and (3) in my original mail
*are* failure points *right now*, while (4) is potential.  I've
personally tested (2) and (3), and they do fail.  Thus fixing
the /run/rootdev issue (which is really also a Btrfs bug), is
only fixing the first problem.  The others are just as serious.

> > You broke booting with the default init system for all people using a
> > btrfs rootfs.
> 
> keep in mind that eventhough it is not desirable nor intended, bugs do
> happen. also in unstable.

This one was totally avoidable.  It happened because you didn't
do your job, and you didn't respect the code freeze.  I have
wasted three nights working on this now, time which I wanted to
spend on fixing issues for the release, and which has instead
been spent on this.  And during all this time, lots of other
people have been suffering the consequences of this.

> unrelated to that, i would appreciate if you could calm down your tone a
> bit, it is more productive that way, thank you.

Err, we do have a very serious problem here.  Systems will no
longer boot.  If I appear frustrated about the situation, it's
because I am frustrated.  You don't appear to be very concerned
about the very real breakage caused here.

Please could you answer the question in my previous email which
you appear to not have addressed:

What are you going to do to rectify this?
Are you going to revert to the previous version?


Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux    http://people.debian.org/~rleigh/
 `. `'   schroot and sbuild  http://alioth.debian.org/projects/buildd-tools
   `-    GPG Public Key      F33D 281D 470A B443 6756 147C 07B3 C8BC 4083 E800



Information forwarded to debian-bugs-dist@lists.debian.org, Daniel Baumann <daniel.baumann@progress-technologies.net>:
Bug#701936; Package btrfs-tools. (Wed, 06 Mar 2013 22:09:09 GMT) Full text and rfc822 format available.

Acknowledgement sent to Roger Leigh <rleigh@codelibre.net>:
Extra info received and forwarded to list. Copy sent to Daniel Baumann <daniel.baumann@progress-technologies.net>. (Wed, 06 Mar 2013 22:09:09 GMT) Full text and rfc822 format available.

Message #80 received at 701936@bugs.debian.org (full text, mbox):

From: Roger Leigh <rleigh@codelibre.net>
To: Daniel Baumann <daniel.baumann@progress-technologies.net>
Cc: 701936@bugs.debian.org, debian-release@lists.debian.org
Subject: Re: Bug#701936: btrfs can't fsck /run/rootdev on boot
Date: Wed, 6 Mar 2013 22:08:04 +0000
severity 701936 critical
thanks

This is a critical bug.  Please do not lower the severity again.


-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux    http://people.debian.org/~rleigh/
 `. `'   schroot and sbuild  http://alioth.debian.org/projects/buildd-tools
   `-    GPG Public Key      F33D 281D 470A B443 6756 147C 07B3 C8BC 4083 E800



Severity set to 'critical' from 'important' Request was from Roger Leigh <rleigh@codelibre.net> to control@bugs.debian.org. (Wed, 06 Mar 2013 22:09:11 GMT) Full text and rfc822 format available.

Marked as found in versions btrfs-tools/0.19+20130131-2. Request was from Axel Beckert <abe@debian.org> to control@bugs.debian.org. (Wed, 06 Mar 2013 23:03:03 GMT) Full text and rfc822 format available.

Reply sent to Luk Claes <luk@debian.org>:
You have taken responsibility. (Sun, 10 Mar 2013 22:21:05 GMT) Full text and rfc822 format available.

Notification sent to Axel Beckert <abe@debian.org>:
Bug acknowledged by developer. (Sun, 10 Mar 2013 22:21:05 GMT) Full text and rfc822 format available.

Message #89 received at 701936-close@bugs.debian.org (full text, mbox):

From: Luk Claes <luk@debian.org>
To: 701936-close@bugs.debian.org
Subject: Bug#701936: fixed in btrfs-tools 0.19+20130131-3+really20121004-1
Date: Sun, 10 Mar 2013 22:17:35 +0000
Source: btrfs-tools
Source-Version: 0.19+20130131-3+really20121004-1

We believe that the bug you reported is fixed in the latest version of
btrfs-tools, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 701936@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Luk Claes <luk@debian.org> (supplier of updated btrfs-tools package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.8
Date: Sun, 10 Mar 2013 22:28:39 +0100
Source: btrfs-tools
Binary: btrfs-tools btrfs-tools-udeb btrfs-tools-dbg
Architecture: source amd64
Version: 0.19+20130131-3+really20121004-1
Distribution: unstable
Urgency: low
Maintainer: Luk Claes <luk@debian.org>
Changed-By: Luk Claes <luk@debian.org>
Description: 
 btrfs-tools - Checksumming Copy on Write Filesystem utilities
 btrfs-tools-dbg - Checksumming Copy on Write Filesystem utilities (debug)
 btrfs-tools-udeb - Checksumming Copy on Write Filesystem utilities (udeb) (udeb)
Closes: 701936
Changes: 
 btrfs-tools (0.19+20130131-3+really20121004-1) unstable; urgency=low
 .
   * Taking over maintenance with maintainer's consent.
   * Reverting to previous upstream (Closes: #701936).
Checksums-Sha1: 
 a51fe2d39b3b301cd353a6933fda4a3ecef776a8 2068 btrfs-tools_0.19+20130131-3+really20121004-1.dsc
 6b8587744f6c7ea221ec4b6eaf2ad20a0cf3d1f7 188784 btrfs-tools_0.19+20130131-3+really20121004.orig.tar.xz
 20727b50ac35922311af4e397876da1410f18416 10488 btrfs-tools_0.19+20130131-3+really20121004-1.debian.tar.xz
 0f713c1c23a4e4a035fe25e79057b0047b2a417b 277780 btrfs-tools_0.19+20130131-3+really20121004-1_amd64.deb
 410875e94f568ed033ea49f6edcb26d87773c948 120506 btrfs-tools-udeb_0.19+20130131-3+really20121004-1_amd64.udeb
 49e6da101e78428bee160a349ac6b30869eae47f 4509996 btrfs-tools-dbg_0.19+20130131-3+really20121004-1_amd64.deb
Checksums-Sha256: 
 086e092962bd1e9a9534a14ec05d2840e00e0ad6b718c26dc4cac15d7e2e8cb4 2068 btrfs-tools_0.19+20130131-3+really20121004-1.dsc
 3e504ccc322b3a19b4e69f5ca92a54e31dbdcf442f1427898feadfb71f391c68 188784 btrfs-tools_0.19+20130131-3+really20121004.orig.tar.xz
 45d3f2da04a6cb26ded4063b5ddec99867b5db4d98dde1f765bce634ab13b4f7 10488 btrfs-tools_0.19+20130131-3+really20121004-1.debian.tar.xz
 473af4dc3dba77bdf854eeff20302c12d6059ba9d0eeebe094e40c5f26684dbe 277780 btrfs-tools_0.19+20130131-3+really20121004-1_amd64.deb
 dc842701b8c37d63d92a002a06a81ca3a6ebdc813137259720c06070dac86832 120506 btrfs-tools-udeb_0.19+20130131-3+really20121004-1_amd64.udeb
 cafce1670bbe7a8aea6adfddcd9af4b7cdf5d023b12ac0645976d7356d5395e0 4509996 btrfs-tools-dbg_0.19+20130131-3+really20121004-1_amd64.deb
Files: 
 3f4f0c95ae5b92bd1fde3866c25c64ad 2068 admin optional btrfs-tools_0.19+20130131-3+really20121004-1.dsc
 e248c6429a38c0828a0b6c1b06e1ce61 188784 admin optional btrfs-tools_0.19+20130131-3+really20121004.orig.tar.xz
 da0921467d84de40bc1cbb00abc31aaa 10488 admin optional btrfs-tools_0.19+20130131-3+really20121004-1.debian.tar.xz
 27fcec34639739037dda4951640f473e 277780 admin optional btrfs-tools_0.19+20130131-3+really20121004-1_amd64.deb
 62ada4275bdeb5d002de2ef6e193191c 120506 debian-installer optional btrfs-tools-udeb_0.19+20130131-3+really20121004-1_amd64.udeb
 f2d792de98be80b2b6231625c4b77046 4509996 debug extra btrfs-tools-dbg_0.19+20130131-3+really20121004-1_amd64.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iQIbBAEBAgAGBQJRPQRAAAoJECEnNxubsjBiUCAP+LVUtvzIXUCLuiZLKsOqaxoT
FAkQqnXnHukZIFozlhdgJCCfqKXgQhtUi4LFXQYLZb6lniSZDItglA1500N7S8bv
+pDrtNH5ePeMdrTTxjo2y0EvzySOUpTfU0J0iw0hM4z/X/bwgSt/+4hTnO9RIOdi
HdtqT4UpsE+nhtNE1Yt9YTAta2U7rlZzzut6pNR4ZLHzf6fKoUgMx+sgMX32f9sJ
8SOoY4J0Uh/PBj7ZVZSbom9BYSUy1OLKvVablJWyDqPy86TgLkD1RIpe5NNSSqO9
K9DyxeHfbcbBDRqIG5IolNEAWG9fbpILfzaXl5Hqhn31uN+RLEiiFqwHQ1W5/tFT
sRx8WzIjZ4MJtgoO6dkAx0N0EFea8rq3XBtv/t4VMMVs2AwEL3UKwzLIXfcyEmgz
JdoMie6ILKUPB/Rs6bU0qA20gmAhjIxecgHXAtR1dtqpxow1/+cq15dDRazwqwun
HY2ULHawXdJKRibMPhw8ZYynwUarRJiAwSGa7SYc6+RYY+oXu/05S+iZt1HaJFV7
omFM7+3ALfTkNWBo7OhfOZb/cFBI8UZVXdrEliSu9Sg72/AKJQdLflzZFyjr+wzF
fuPftQeBb42lXfVM9NGbSdd36JCNF85YkoCHDygf4W8+BHCtHs4YKqq0Ck/G2aMb
zzq3vMqLntX565bKW/c=
=GHnm
-----END PGP SIGNATURE-----




Information forwarded to debian-bugs-dist@lists.debian.org, Luk Claes <luk@debian.org>:
Bug#701936; Package btrfs-tools. (Sun, 17 Mar 2013 13:00:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Nis Martensen <nis.martensen@web.de>:
Extra info received and forwarded to list. Copy sent to Luk Claes <luk@debian.org>. (Sun, 17 Mar 2013 13:00:04 GMT) Full text and rfc822 format available.

Message #94 received at 701936@bugs.debian.org (full text, mbox):

From: Nis Martensen <nis.martensen@web.de>
To: 701936@bugs.debian.org
Cc: Daniel Baumann <daniel.baumann@progress-technologies.net>
Subject: Re: btrfs can't fsck /run/rootdev on boot
Date: Sun, 17 Mar 2013 13:56:22 +0100
[Message part 1 (text/plain, inline)]
On Fri, Mar 01, 2013 at 07:50:37AM +0100, Daniel Baumann wrote:
> works with systemd, it's sysvinit specific.

Systemd skips the root file system check if the major device number of
the root file system is 0 (which is what btrfs reports):
http://cgit.freedesktop.org/systemd/systemd/tree/src/fsck/fsck.c#n297

The same could be done in sysvinit. Possible (untested) patch attached.
[0001-Skip-root-filesystem-check-for-btrfs-file-systems.patch (text/x-diff, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, Luk Claes <luk@debian.org>:
Bug#701936; Package btrfs-tools. (Tue, 19 Mar 2013 22:57:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Roger Leigh <rleigh@codelibre.net>:
Extra info received and forwarded to list. Copy sent to Luk Claes <luk@debian.org>. (Tue, 19 Mar 2013 22:57:04 GMT) Full text and rfc822 format available.

Message #99 received at 701936@bugs.debian.org (full text, mbox):

From: Roger Leigh <rleigh@codelibre.net>
To: Nis Martensen <nis.martensen@web.de>, 701956@bugs.debian.org
Cc: 701936@bugs.debian.org, Daniel Baumann <daniel.baumann@progress-technologies.net>
Subject: Re: Bug#701956: btrfs can't fsck /run/rootdev on boot
Date: Tue, 19 Mar 2013 22:55:32 +0000
On Sun, Mar 17, 2013 at 01:56:22PM +0100, Nis Martensen wrote:
> On Fri, Mar 01, 2013 at 07:50:37AM +0100, Daniel Baumann wrote:
> > works with systemd, it's sysvinit specific.
> 
> Systemd skips the root file system check if the major device number of
> the root file system is 0 (which is what btrfs reports):
> http://cgit.freedesktop.org/systemd/systemd/tree/src/fsck/fsck.c#n297
> 
> The same could be done in sysvinit. Possible (untested) patch attached.

Thanks for the patch.  This will have to be deferred until after
wheezy is released, but if we have to work around the btrfs fsck
issues in this way, it looks like a reasonable approach.  I would
hope however that this would not be necessary, and that the btrfs
fsck will become compatible with the standard /sbin/fsck options
and behaviour, including being able to fsck a read-only mount, or
else some users of btrfs won't be able to check their filesystems.

Daniel, if you haven't done do already, it would be great if these
issues could be brought to the attention of the upstream developers.
As it stands, fsck.btrfs is not a compatible fsck, and this will
break any tool expecting it to work like /sbin/fsck, not just
sysvinit.


Regards,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux    http://people.debian.org/~rleigh/
 `. `'   schroot and sbuild  http://alioth.debian.org/projects/buildd-tools
   `-    GPG Public Key      F33D 281D 470A B443 6756 147C 07B3 C8BC 4083 E800



Bug archived. Request was from Debbugs Internal Request <owner@bugs.debian.org> to internal_control@bugs.debian.org. (Wed, 17 Apr 2013 07:28:01 GMT) Full text and rfc822 format available.

Bug unarchived. Request was from Roger Leigh <rleigh@codelibre.net> to control@bugs.debian.org. (Sat, 13 Jul 2013 16:39:12 GMT) Full text and rfc822 format available.

Bug reopened Request was from Roger Leigh <rleigh@codelibre.net> to control@bugs.debian.org. (Sat, 13 Jul 2013 16:39:13 GMT) Full text and rfc822 format available.

No longer marked as fixed in versions btrfs-tools/0.19+20130131-3+really20121004-1. Request was from Roger Leigh <rleigh@codelibre.net> to control@bugs.debian.org. (Sat, 13 Jul 2013 16:39:13 GMT) Full text and rfc822 format available.

Marked as found in versions btrfs-tools/0.19+20130315-1. Request was from Roger Leigh <rleigh@codelibre.net> to control@bugs.debian.org. (Sat, 13 Jul 2013 16:39:14 GMT) Full text and rfc822 format available.

Reply sent to Dmitrijs Ledkovs <xnox@debian.org>:
You have taken responsibility. (Mon, 29 Jul 2013 14:57:16 GMT) Full text and rfc822 format available.

Notification sent to Axel Beckert <abe@debian.org>:
Bug acknowledged by developer. (Mon, 29 Jul 2013 14:57:16 GMT) Full text and rfc822 format available.

Message #114 received at 701936-done@bugs.debian.org (full text, mbox):

From: Dmitrijs Ledkovs <xnox@debian.org>
To: 701936-done@bugs.debian.org
Subject: Btrfs & SysV init
Date: Mon, 29 Jul 2013 15:55:11 +0100
I believe since sysvinit -42 upload, it is possible to correctly boot
btrfs systems with sysvinit, since this particular problem has been
worked around.

I do understand a general desire for a proper and better behaved
fsck.btrfs. And that's something I will continue to work towards.

But this bug per se (can't boot), is resolved for now.

Regards,

Dmitrijs.



Bug archived. Request was from Debbugs Internal Request <owner@bugs.debian.org> to internal_control@bugs.debian.org. (Tue, 27 Aug 2013 07:30:15 GMT) Full text and rfc822 format available.

Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Thu Apr 17 22:44:25 2014; Machine Name: buxtehude.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.