Debian Bug report logs - #652946
"partman-target should not create /var/run"

version graph

Package: partman-target; Maintainer for partman-target is Debian Install System Team <debian-boot@lists.debian.org>; Source for partman-target is src:partman-target.

Reported by: "Rui Miguel P. Bernardo" <rui.bernardo.pt@gmail.com>

Date: Thu, 22 Dec 2011 02:03:02 UTC

Severity: important

Tags: patch

Fixed in version partman-target/87

Done: Christian Perrier <bubulle@debian.org>

Bug is archived. No further changes may be made.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, Debian Install System Team <debian-boot@lists.debian.org>:
Bug#652946; Package live-installer. (Thu, 22 Dec 2011 02:03:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to "Rui Miguel P. Bernardo" <rui.bernardo.pt@gmail.com>:
New Bug report received and forwarded. Copy sent to Debian Install System Team <debian-boot@lists.debian.org>. (Thu, 22 Dec 2011 02:03:05 GMT) Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: "Rui Miguel P. Bernardo" <rui.bernardo.pt@gmail.com>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: live-installer does not install /var/spool and breaks crontab
Date: Thu, 22 Dec 2011 02:00:07 +0000
Package: live-installer
Version: 34
Severity: normal

Hello,

after building a live image with live-build using the "daily" wheezy/sid 
installer and installed the image to disk crontab and postfix failed to 
start (this is a debian wheezy/testing image). The problem was a missing 
/var/spool directory in the installed system, that live-installer did 
not "copy" to disk.

While the installer "copies" the files in /live/filesystem.squashfs I've 
opened tty2 and confirmed that /var/spool is in the image

	ls /mnt/var/spool

and is polulated. But after copying the files to disk, still in the 
installer, /var/spool does not exist.

	ls /target/var/spool

After install /var/spool does not exist. This breaks crontab and 
probably all other packages that expect a /var/spool directory.

To fix (only) the crontab error, recreate the /var/spool directory in 
the installed system after 1st boot:

	sudo mkdir /var/spool

I also had postfix in the image, but this one requires one to recreate 
its directory in /var/spool, not just /var/spool:

	sudo mkdir -p /var/spool/postfix

Samba is another package that might break (not confirmed).

-- System Information:
Debian Release: wheezy/sid
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: i386 (i686)

Kernel: Linux 3.1.0-1-686-pae (SMP w/1 CPU core)
Locale: LANG=pt_PT.UTF-8, LC_CTYPE=pt_PT.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Install System Team <debian-boot@lists.debian.org>:
Bug#652946; Package live-installer. (Fri, 23 Dec 2011 08:00:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Rui Miguel <rui.bernardo.pt@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian Install System Team <debian-boot@lists.debian.org>. (Fri, 23 Dec 2011 08:00:05 GMT) Full text and rfc822 format available.

Message #10 received at 652946@bugs.debian.org (full text, mbox):

From: Rui Miguel <rui.bernardo.pt@gmail.com>
To: 652946@bugs.debian.org
Subject: live-installer does not install /var/spool and breaks crontab
Date: Fri, 23 Dec 2011 07:53:51 +0000
[Message part 1 (text/plain, inline)]
# 652946 - live-installer does not install /var/spool and breaks crontab

Hi again,

Sorry for the details and this long email. Yesterday reportbug wouldn't 
send the mail through postfix so I've sent a bug report without logs by 
lack of patience. This bug is very pertinent for me because I can't see 
why it happens and also the consequences of a missing /var/spool. Here 
follows attached several logs about the bug and a way to recreate the bug. 
This is very time consuming, so here is how I've recreated the bug, my 
findings about it and also a possible workaround patch for 
live-installer. I still don't know why it happens.


# manual fix in installed system (mkdir)

As in my bug report, to fix (only) the crontab error in an installed 
system, recreate the /var/spool directory in the installed system after 
the 1st boot (can be done in tty2 still in installer, or in 
late_command, after changing paths):

	sudo mkdir /var/spool

I also had postfix in the image, but this one requires one to recreate 
its directory in /var/spool, not just /var/spool:

	sudo mkdir -p /var/spool/postfix

Samba is another package that might break (not confirmed).


# recreate bug

## live-build configuration

	lb config -d wheezy --debian-installer live \
		--debian-installer-distribution daily \
		--archive-areas "main contrib non-free" \
		--security true --apt-recommends false --apt-secure true \
		--linux-packages "linux-image linux-headers" \
		-a i386 --linux-flavours 486 \
		--tasks "standard laptop"

To add an install option to live image boot options I use the following 
hook in config/hooks/add-install-boot-option.binary

	#!/bin/sh
	
	# remove "fb=false" to have all languages
	# see http://www.debian.org/releases/stable/i386/ch05s02.html.en
	cat >> binary/isolinux/live.cfg << EOF
	
	label install
	        menu label ^Install
	        menu default
	        kernel /install/vmlinuz
	        append initrd=/install/initrd.gz fb=false quiet 
	EOF

Then build image with "sudo lb build".


## what is where

After build, boot the image, select "Install" and proceed with install. 
After the partitioning and after files were copied to disk, open tty2 
and check /target/var/spool. I saw this:

	ls /target/var -la
	drwxr-xr-x    8 root     root          4096 Dec 23 04:17 .
	drwxr-xr-x   22 root     root          4096 Dec 23 04:39 ..
	drwxr-xr-x    2 root     root          4096 Jul 29 19:30 backups
	drwxr-xr-x    6 root     root          4096 Dec 23 04:17 cache
	drwxr-xr-x   18 root     root          4096 Dec 23 04:17 lib
	drwxrwsr-x    2 root     50            4096 Jul 29 19:30 local
	drwxr-xr-x    2 root     root          4096 Dec 23 04:17 lock
	drwxr-xr-x    2 root     root          4096 Dec 23 04:17 run
	ls /target/var/spool -la
	ls: /target/var/spool: No such file or directory

No /var/spool in /target. Files that are in the live image filesystem in 
/mnt:

	mount /cdrom/live/filesystem.squashfs /mnt
	ls /mnt/var/spool -la
	drwxr-xr-x    3 root     root            39 Dec 22 04:54 .
	drwxr-xr-x   11 root     root           160 Dec 23 01:29 ..
	drwxr-xr-x    3 root     root            31 Dec 22 04:54 cron
	lrwxrwxrwx    1 root     root             7 Dec 22 04:53 mail -> ../mail

The content of /var/spool in the installer is:

	ls /var/spool -la
	drwxr-xr-x    3 root     root             0 Dec 23 04:17 .
	drwxr-xr-x    7 root     root             0 Dec 23 04:17 .
	drwxr-xr-x    2 root     root             0 Jul  8 15:03 kickseed

What is mounted:

	mount
	rootfs on / type rootfs (rw)
	none on /run type tmpfs (rw,nosuid,relatime,size=51488k,mode=755)
	none on /proc type proc (rw,relatime)
	none on /sys type sysfs (rw,relatime)
	tmpfs on /dev type tmpfs (rw,relatime,mode=755)
	none on /dev/pts type devpts (rw,relatime,gid=5,mode=620,ptmxmode=000)
	/dev/sda1 on /target type ext4 (rw,relatime,errors=remount-ro,user_xattr,barrier=1,data=ordered)
	tmpfs on /target/dev type tmpfs (rw,relatime,mode=755)
	none on /target/proc type proc (rw,relatime)
	/dev/sr0 on /cdrom type iso9660 (ro,relatime)
	/dev/loop0 on /mnt type squashfs (ro,relatime)

To proceed with install, unmount /mnt, return to tty1 and proceed with 
install.

	umount /mnt

In the end of the install /var/spool was not copied from live image 
filesystem and will not be created, being absent in the installed 
system.


## fix while still in installer (copy /var/spool)

To make the install sucessfull, before grub is installed to disk, return 
to tty2 and manually mount the filesystem of the live image to /mnt and 
manually copy to /target:

	mount /cdrom/live/filesystem.squashfs /mnt
	cp -ra /mnt/var/spool /target/var/
	umount /mnt

Just creating /var/spool is not enough because cron conplains on boot 
that /var/spool/cron does not exist while the service starts, then it 
recreates the directory and starts correctly. For postfix 
/var/spool/postfix **must** exist before the service starts or the 
postfix service fails on start complaining it can't "cd to 
/var/spool/postfix". If /var/spool/postfix exists then postfix service 
populates /var/spool/postfix and starts correctly.

I don't know what is debian policy about non existing /var/spool, but 
both cases could be bugs in the packages if policy states that a service 
must check if $HOME exists (in the case of postfix), or packages in 
general that use /var/spool must check if it exists before trying start 
a service that use /var/spool. Both packages created their directories 
in the live image upon postinst, so their directories in /var/spool are 
expected to be there.


## missing files and dirs

Anyway, the fact that /var/spool was not copied made me check for all 
files that were in the live image filesystem and the ones that 
live-installer did install. To do this I've edited 
live-installer.postinst and added the following line.

	 		(chdir /target && tar xv) | \
	 		(
	 			while read line; do
	+				echo "$line" >> /target/installed-files.txt
	 				COUNT=$(($COUNT + 1))
	 				CURRENT=$(($COUNT * 100 / $STEPS))

Then rebuilt the udeb package and installed it to live-build 
config/packages/ directory to include it in the image and rebuilt the 
image with "sudo lb clean && sudo lb build".

This gave me a list of the installed files done by tar upon install. To 
compare with the list of the files present in the live image filesystem 
I've mounted the live image filesystem after the live image was copies 
to disk (couldn't mount it before the copy was made, mount conplained 
about invalid argument) and ran

	cd /mnt
	find . > /target/files-in-image.txt

After striping trailing slashes from the file installed-files.txt and 
diff both files I had a list of files and dirs that were not installed 
by live-installer:

	./var/log
	./var/log/alternatives.log
	./var/log/apt
	./var/log/apt/history.log
	./var/log/boot
	./var/log/bootstrap.log
	./var/log/btmp
	./var/log/dmesg
	./var/log/dpkg.log
	./var/log/faillog
	./var/log/fsck
	./var/log/fsck/checkfs
	./var/log/fsck/checkroot
	./var/log/lastlog
	./var/log/wtmp
	./var/mail
	./var/opt
	./var/run
	./var/spool
	./var/spool/cron
	./var/spool/cron/crontabs
	./var/spool/mail
	./var/tmp
	./vmlinuz

I can not find any reason why those files are missing in the installed 
system, not in live-installer.postinst or in any other place I've 
searched. I've also checked mount after the copy was made and it does 
not mount bind /var/spool or any files and dirs that were missing in the 
installed system at that time. My mount checks before the copy didn't 
show anything, but was not exaustive about it. During the copy it was 
not checked but no code I've found shows it. All this testing took time, 
that's why I'm posting them.

I've searched in debian-installer also, but there are so many udebs and 
I don't know debian-installer internals, so maybe someone with 
debian-installer knowledge could say if debian-installer does anything. 
I could not find anything, but maybe I don't know where to search.

The only place I've found where all those paths are common (except for 
/var/opt) was in live-build, in lb_chroot_hacks script 
(COW_DIRECTORIES), but the files are in the live image filesystem, so 
it's not it, I suppose. I've also searched live-boot and live-config but 
nothing came up.


# proposed workaround patch

To workaround the problem I've changed live-installer.postinst to really 
install /mnt/var/spool in /target/var/ right after the tar copy ended, 
while the live image filesystem is mounted in /mnt, before "eval 
${SUPPORT}_teardown".

	cp -ra /mnt/var/spool /target/var/

Of course there might be other better ways to do it, maybe with tar 
also. See attached patch

Still, other directories are not created. They are

	/var/mail
	/var/opt
	/var/tmp

I still did not see if they are relevant later in a working installed 
system. The patch just for /var/spool. /var/mail is a link.


# rebuild image with udeb

For live-build, copy the resulting udeb package to config/packages/ so 
the fixed live-installer udeb is used by the live image.

[boot (text/plain, attachment)]
[files-in-image.txt (text/plain, attachment)]
[stripped-installed-files.txt (text/plain, attachment)]
[0001-Make-sure-mnt-var-spool-is-also-installed-in-target.patch (text/x-diff, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Install System Team <debian-boot@lists.debian.org>:
Bug#652946; Package live-installer. (Sun, 25 Dec 2011 10:00:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to "Rui Miguel P. Bernardo" <rui.bernardo.pt@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian Install System Team <debian-boot@lists.debian.org>. (Sun, 25 Dec 2011 10:00:06 GMT) Full text and rfc822 format available.

Message #15 received at 652946@bugs.debian.org (full text, mbox):

From: "Rui Miguel P. Bernardo" <rui.bernardo.pt@gmail.com>
To: Debian Bug Tracking System <652946@bugs.debian.org>
Subject: Re: live-installer does not install /var/spool and breaks crontab
Date: Sun, 25 Dec 2011 09:58:53 +0000
[Message part 1 (text/plain, inline)]
Package: live-installer
Version: 34
Followup-For: Bug #652946


Hi again.

Because tar process doesn't return ant error, apparently, I didn't try 
to extract it's error output in my previous tests. When I did, by adding 
someting like "2> /target/tar-errors.txt" in the "tar xv" line, the file 
showed this

	tar: can't remove old file ./var/lock: Is a directory

So the extraction is not completed and stops when extracting /var/lock 
**and** does not return an error to live-installer. Maybe the exit code 
of tar extraction process should be also be redirected to track future 
errors while extracting the filesystem.

After trying to get busybox tar to exclude var/lock, without success 
because tar in debian busybox does not handle exclude option, I've tried 
the lenny approach I've seen in live-installer.postinst history in git 
repo, and use live system installed tar to extract the filesystem 
excluding var/lock.

	chroot . tar c . --exclude=var/lock | \
	(chdir /target && tar xv 2> /target/tar-errors.txt) | \

This gave me another error later upon install about /var/run:

	tar: can't remove old file ./var/run: Is a directory

Then I added var/run to excludes, rebuilt udeb, rebuild live image and 
the install worked without missing dirs and files, even the excluded 
dirs were in /target/var. After boot no errors.

Patch attached.

Merry Christmas.


-- System Information:
Debian Release: wheezy/sid
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: i386 (i686)

Kernel: Linux 3.1.0-1-686-pae (SMP w/1 CPU core)
Locale: LANG=pt_PT.UTF-8, LC_CTYPE=pt_PT.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
[0001-use-live-system-tar-to-use-exclude.patch (text/x-diff, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Install System Team <debian-boot@lists.debian.org>:
Bug#652946; Package live-installer. (Sat, 04 Feb 2012 17:03:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to daniel.baumann@progress-technologies.net:
Extra info received and forwarded to list. Copy sent to Debian Install System Team <debian-boot@lists.debian.org>. (Sat, 04 Feb 2012 17:03:03 GMT) Full text and rfc822 format available.

Message #20 received at 652946@bugs.debian.org (full text, mbox):

From: Daniel Baumann <daniel.baumann@progress-technologies.net>
To: 652946-submitter@bugs.debian.org
Cc: 652946@bugs.debian.org
Subject: Re: live-installer does not install /var/spool and breaks crontab
Date: Sat, 04 Feb 2012 18:00:37 +0100
retitle 652946 busybox tar fails to copy filesystem.squashfs
thanks

i cannot reproduce it with plain squeeze, and not with custom squeeze
images (that use the same live-installer/d-i as wheezy, mostly).

while the error clearly is in busybox, and using the tar from the target
system is not acceptable solution, i assume this was a temporary issue
with busybox in wheezy.

i'll try to reproduce it later with a wheezy and sid/daily based image.

-- 
Address:        Daniel Baumann, Donnerbuehlweg 3, CH-3012 Bern
Email:          daniel.baumann@progress-technologies.net
Internet:       http://people.progress-technologies.net/~daniel.baumann/




Changed Bug title to 'busybox tar fails to copy filesystem.squashfs' from 'live-installer does not install /var/spool and breaks crontab' Request was from Daniel Baumann <daniel.baumann@progress-technologies.net> to control@bugs.debian.org. (Sat, 04 Feb 2012 17:03:05 GMT) Full text and rfc822 format available.

Message sent on to "Rui Miguel P. Bernardo" <rui.bernardo.pt@gmail.com>:
Bug#652946. (Sat, 04 Feb 2012 17:03:16 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Install System Team <debian-boot@lists.debian.org>:
Bug#652946; Package live-installer. (Sat, 26 May 2012 16:09:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to "Rui Miguel P. Bernardo" <rui.bernardo.pt@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian Install System Team <debian-boot@lists.debian.org>. (Sat, 26 May 2012 16:09:03 GMT) Full text and rfc822 format available.

Message #30 received at 652946@bugs.debian.org (full text, mbox):

From: "Rui Miguel P. Bernardo" <rui.bernardo.pt@gmail.com>
To: daniel.baumann@progress-technologies.net, 652946@bugs.debian.org
Cc: 652946-submitter@bugs.debian.org
Subject: Re: Bug#652946: live-installer does not install /var/spool and breaks crontab
Date: Sat, 26 May 2012 17:04:58 +0100
> i cannot reproduce it with plain squeeze, and not with custom squeeze
> images (that use the same live-installer/d-i as wheezy, mostly).
>
> while the error clearly is in busybox, and using the tar from the target
> system is not acceptable solution, i assume this was a temporary issue
> with busybox in wheezy.
>
> i'll try to reproduce it later with a wheezy and sid/daily based image.

Hi Daniel,

sorry for the long delay. Only now I can reproduce live-installer bugs
in squeeze and wheezy with confidence.

I can also confirm that this bug does __not__ happen in squeeze for sure.

I suspect that bug #673328 is the same as this one because /var/log is
not restored in disk installation.

To confirm it I've build a wheezy standard live image with daily
installer udeb, and with a patched live-installer udeb. In the patched
live-installer /var/log/ directory is in /target/ after install,
without it /var/log does not exist in /target/.

wheezy with patched udeb http://www.adrive.com/public/q4BeZW.html
wheeze-standard-may2012-patched-udeb.iso

normal wheezy http://www.adrive.com/public/HPRHfB.html
wheeze-standard-may2012-without-patch.iso

The patched udebs can be found in github
https://github.com/rbern/bravance/blob/master/live-installer_35~2.gbpe79f3c~debian6460%2B0_i386.udeb

Should this bug be re-assigned to busybox/tar?




Message sent on to "Rui Miguel P. Bernardo" <rui.bernardo.pt@gmail.com>:
Bug#652946. (Sat, 26 May 2012 16:09:04 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Install System Team <debian-boot@lists.debian.org>:
Bug#652946; Package live-installer. (Mon, 09 Jul 2012 18:30:07 GMT) Full text and rfc822 format available.

Acknowledgement sent to Rui Bernardo <rui.bernardo.pt@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian Install System Team <debian-boot@lists.debian.org>. (Mon, 09 Jul 2012 18:30:07 GMT) Full text and rfc822 format available.

Message #38 received at 652946@bugs.debian.org (full text, mbox):

From: Rui Bernardo <rui.bernardo.pt@gmail.com>
To: 652946@bugs.debian.org
Subject: busybox tar fails to copy filesystem.squashfs
Date: Mon, 9 Jul 2012 19:28:28 +0100
Today I've gave another try and this is still happening with today's sid 
build.

Following your hint that this is busybox related I've tried to find how 
far back this issue was happening using a patched live-build to use 
snapshots.debian.org. From the available daily installers this bug is 
present in all installers in http://d-i.debian.org/daily-images/i386. 
The only date I didn't confirm this bug was 20110509 because the 
installer stopped in the keyboard step. So I can assume this bug is 
present at least since 20110930.

I've tried to enable/disable config options in busybox udeb that were 
enabled since 20110930 (the last build I could reproduce this).

Looking at 
http://anonscm.debian.org/gitweb/?p=d-i/busybox.git;a=history;f=debian/config/pkg;hb=HEAD 
I've tried to disable in busybox udeb CONFIG_MKTEMP and 
CONFIG_SWITCH_ROOT, just because their names are suspicious. Didn't 
work. The installer stopped in the keyboard step without any of them.

I can't check all the busybox udeb changes to trace this, it takes too 
much time to do it without knowing why the changes were made.

Another factor is busybox upstream version change since squeeze. It went 
from 1.17 to actual 1.20. So, this can also be an upstream problem or 
something that changed upstream. I don't know busybox enough to trace 
this in upstream versions in debian-installer.

But, while I was messing with busybox udeb, I've tried to enable 
CONFIG_FEATURE_TAR_FROM and tried to exclude var/lock and var/run in 
busybox tar (not the target's tar as in my previous patches) and all 
worked ok.

In busybox:

@@ -152,7 +152,7 @@ CONFIG_GZIP_FAST=0
 CONFIG_TAR=y
 CONFIG_FEATURE_TAR_CREATE=y
 # CONFIG_FEATURE_TAR_AUTODETECT is not set
-# CONFIG_FEATURE_TAR_FROM is not set
+CONFIG_FEATURE_TAR_FROM=y
 # CONFIG_FEATURE_TAR_OLDGNU_COMPATIBILITY is not set
 # CONFIG_FEATURE_TAR_OLDSUN_COMPATIBILITY is not set
 CONFIG_FEATURE_TAR_GNU_EXTENSIONS=y

In live-installer:

@@ -42,9 +42,13 @@ install_live_system () {
 
 		COUNT=0
 		OLD_IFS=$IFS
+
+		echo 'var/lock' > /tmp/excludes
+		echo 'var/run' >> /tmp/excludes
+
 		mkdir -p /target
 		exec 4>&0
-		tar c . | \
+		tar c . -X /tmp/excludes | \
 		(chdir /target && tar xv) | \
 		(
 			while read line; do
@@ -60,6 +64,8 @@ install_live_system () {
 		exec 0>&4
 		IFS=$OLD_IFS
 
+		rm /tmp/excludes
+
 		chdir /
 		eval ${SUPPORT}_teardown
 	done

It worked, without using target tar and without chroot.

busybox udeb must be changed or the tar command will fail during the 
install.




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Install System Team <debian-boot@lists.debian.org>:
Bug#652946; Package live-installer. (Mon, 09 Jul 2012 19:15:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Michael Tokarev <mjt@tls.msk.ru>:
Extra info received and forwarded to list. Copy sent to Debian Install System Team <debian-boot@lists.debian.org>. (Mon, 09 Jul 2012 19:15:04 GMT) Full text and rfc822 format available.

Message #43 received at 652946@bugs.debian.org (full text, mbox):

From: Michael Tokarev <mjt@tls.msk.ru>
To: Rui Bernardo <rui.bernardo.pt@gmail.com>, 652946@bugs.debian.org
Subject: Re: Bug#652946: busybox tar fails to copy filesystem.squashfs
Date: Mon, 09 Jul 2012 23:11:24 +0400
Um.  This is just insane.  The bug is filed 22-Dec 2011,
today is 09-Jul 2012, you have the actual cause, and it
is still open, and the maintainers of suspected package
(busybox) knows nothing about it... :(

Okay.

Rui, you found the root cause of this issue, it is there:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=652946#15

The problem is that during extraction, target directory
var/lock exists, and the installer tries to replace it
with something else (I guess a symlink).

Busybox tar does not try to remove directories in this
case, to be replaced with files.  So it stops there,
treating it as fatal error, and does not extract remaining
files, creating issues for the system running later.

Whenever it is a bug in busybox tar or not is an open
question.  POSIX does not specify whenever it should do
that or not.  GNU tar tries to rmdir() the existing
file if unlink()ing it returned EISDIR, and will fail
only if the directory is non-empty.  Ofcourse this rmdir
can be added to busybox tar too, it is rather trivial.

But this is not the point.

The point is the presence of var/lock directory in the
target system.  It should not be there to start with.
We're extracting a whole set of directories, and any
existing directory in the target which clashes with
a file in the archive we're extracting may break the
extraction with any tar implementation, not just
busybox one.  Ie, always extract to empty target
directory.

Now, I've no idea how this live system works, so I can't
say more.

Besides, I verified - busybox tar will correctly return
failure (and write accutate reason to stderr) in this
case.  So something else in the live installer should be
broken too -- the part which loses error messages and
non-zero status codes.

So, to sum it all: it is a bug in the live installer,
it a) calls tar incorrectly (losing exit code and stderr)
and b) does not verify that the target directory is empty
or at least has sane content.

Thanks,

/mjt




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Install System Team <debian-boot@lists.debian.org>:
Bug#652946; Package live-installer. (Tue, 10 Jul 2012 00:18:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Rui Bernardo <rui.bernardo.pt@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian Install System Team <debian-boot@lists.debian.org>. (Tue, 10 Jul 2012 00:18:04 GMT) Full text and rfc822 format available.

Message #48 received at 652946@bugs.debian.org (full text, mbox):

From: Rui Bernardo <rui.bernardo.pt@gmail.com>
To: Michael Tokarev <mjt@tls.msk.ru>, 652946@bugs.debian.org
Subject: Re: Bug#652946: busybox tar fails to copy filesystem.squashfs
Date: Tue, 10 Jul 2012 01:15:20 +0100
[Message part 1 (text/plain, inline)]
Hi Michael,

On Mon, Jul 09, 2012 at 11:11:24PM +0400, Michael Tokarev wrote:
> Um.  This is just insane.  The bug is filed 22-Dec 2011,
> today is 09-Jul 2012, you have the actual cause, and it
> is still open, and the maintainers of suspected package
> (busybox) knows nothing about it... :(

I think we didn't know for sure that busybox was the to blame. Daniel 
said the error is in busybox, not a busybox error.

Because this bug is hard to trace (got to build and rebuild several live 
images), lack of time, etc, time passed... Anyway, as you pointed out, 
it's not busybox. Thank you.

> 
> Okay.
> 
> Rui, you found the root cause of this issue, it is there:
> 
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=652946#15
> 
> The problem is that during extraction, target directory
> var/lock exists, and the installer tries to replace it
> with something else (I guess a symlink).
> 
> Busybox tar does not try to remove directories in this
> case, to be replaced with files.  So it stops there,
> treating it as fatal error, and does not extract remaining
> files, creating issues for the system running later.
> 
> Whenever it is a bug in busybox tar or not is an open
> question.  POSIX does not specify whenever it should do
> that or not.  GNU tar tries to rmdir() the existing
> file if unlink()ing it returned EISDIR, and will fail
> only if the directory is non-empty.  Ofcourse this rmdir
> can be added to busybox tar too, it is rather trivial.
> 
> But this is not the point.
> 
> The point is the presence of var/lock directory in the
> target system.  It should not be there to start with.
> We're extracting a whole set of directories, and any
> existing directory in the target which clashes with
> a file in the archive we're extracting may break the
> extraction with any tar implementation, not just
> busybox one.  Ie, always extract to empty target
> directory.

Thanks again for clearing the fog here :)

Following your pointers I've put a "sleep 2000" in 
live-installer.postinst just _before_ the line "mkdir -p /target" and 
then I've open tty2 console in debian-installer to check what was in 
/target (see below).

> 
> Now, I've no idea how this live system works, so I can't
> say more.
> 
> Besides, I verified - busybox tar will correctly return
> failure (and write accutate reason to stderr) in this
> case.  So something else in the live installer should be
> broken too -- the part which loses error messages and
> non-zero status codes.

I think the problem about redirecting the error messages is because 
live-installer.postint extracts the filesystem in background with exec 
4. The relevant part of postinst:

		eval ${SUPPORT}_prepare
		STEPS=$(eval ${SUPPORT}_count)

		db_progress INFO live-installer/progress/copying

		COUNT=0
		OLD_IFS=$IFS
		mkdir -p /target
		exec 4>&0
		tar c . | \
		(chdir /target && tar xv) | \
		(
			while read line; do
				COUNT=$(($COUNT + 1))
				CURRENT=$(($COUNT * 100 / $STEPS))

				[ x$CURRENT = x$LAST_UPDATE ] && continue

				LAST_UPDATE=$CURRENT
				db_progress STEP 1 <&4
			done
		)
		exec 0>&4

I don't know if adding a "2>" or a "|tee -a" would work. It's a question 
of trying it. It would really help in the future.

> 
> So, to sum it all: it is a bug in the live installer,
> it a) calls tar incorrectly (losing exit code and stderr)
> and b) does not verify that the target directory is empty
> or at least has sane content.

Thanks for this, I really thought it was something about links or 
hardlinks in the squash filesystem all the time. Turns out that is 
/target that is not empty when the extraction happens!

As I've said above, I've put a "sleep 2000" before "mkdir -p /target" 
and SURPRISE (for me at least): /target, which I thought should be empty 
because it is apparently created after partman has formated the 
partitions (so I assumed completely empty), is _not_ empty:

	root@quartor:~# tar tf target.tar.gz 
	./
	./etc/
	./etc/fstab
	./lost+found/
	./media/
	./media/cdrom0/
	./media/cdrom
	./var/
	./var/lock/
	./var/run/

I've attached the tar file with what I've found in /target/ before the 
extraction began.

I presume this must be something that line "eval ${SUPPORT}_prepare" 
does.

> 
> Thanks,
> 
> /mjt

Thank you very much!! Debian is just great :)
[target.tar.gz (application/octet-stream, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Install System Team <debian-boot@lists.debian.org>:
Bug#652946; Package live-installer. (Tue, 10 Jul 2012 05:39:06 GMT) Full text and rfc822 format available.

Acknowledgement sent to Rui Bernardo <rui.bernardo.pt@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian Install System Team <debian-boot@lists.debian.org>. (Tue, 10 Jul 2012 05:39:06 GMT) Full text and rfc822 format available.

Message #53 received at 652946@bugs.debian.org (full text, mbox):

From: Rui Bernardo <rui.bernardo.pt@gmail.com>
To: 652946@bugs.debian.org
Subject: Re: Bug#652946: busybox tar fails to copy filesystem.squashfs
Date: Tue, 10 Jul 2012 06:38:58 +0100
I was going to prepare another patch but I hold myself this time...

I've searched what was creating those directories in the /target/. Turns 
out to be /lib/partman/finish.d/20mount_partitions from partman-target [1].

That code isn't touched since 2007 and creates the var/run and var/lock 
directories in /target.

I'm not sure if /var/run and /var/lock still need to be created in 
/target since debian policy changed for those directories. As Michael 
pointed out, they are links now.

If the directories still need to exist for another package other than 
live-installer or for anything else, then we should patch live-installer 
to "rmdir /target/var/run /target/var/lock" before extracting the squashfs.

If not, maybe it should be partman-target that should stop creating
those directories.

What do you guys think?


[1] http://anonscm.debian.org/gitweb/?p=d-i/partman-target.git




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Install System Team <debian-boot@lists.debian.org>:
Bug#652946; Package live-installer. (Wed, 11 Jul 2012 06:06:07 GMT) Full text and rfc822 format available.

Acknowledgement sent to Rui Bernardo <rui.bernardo.pt@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian Install System Team <debian-boot@lists.debian.org>. (Wed, 11 Jul 2012 06:06:07 GMT) Full text and rfc822 format available.

Message #58 received at 652946@bugs.debian.org (full text, mbox):

From: Rui Bernardo <rui.bernardo.pt@gmail.com>
To: 652946@bugs.debian.org
Subject: Re: Bug#652946: busybox tar fails to copy filesystem.squashfs
Date: Wed, 11 Jul 2012 07:04:13 +0100
On Tue, Jul 10, 2012 at 06:38:57AM +0100, Rui Bernardo wrote:
> I was going to prepare another patch but I hold myself this time...
> 
> I've searched what was creating those directories in the /target/. Turns 
> out to be /lib/partman/finish.d/20mount_partitions from partman-target [1].
> 
> That code isn't touched since 2007 and creates the var/run and var/lock 
> directories in /target.

I meant that the relevant file was not changed since 2007.
 
> I'm not sure if /var/run and /var/lock still need to be created in 
> /target since debian policy changed for those directories. As Michael 
> pointed out, they are links now.

Reading the debian wiki about /var/run transition [2] and debian policy 
9.1.1.7 [3] I think debian-installer partman-target could only create 
/run now. It shouldn't be a problem if initscripts and initramfs-tools 
are ready to use /run instead of /var/run.


[1] http://anonscm.debian.org/gitweb/?p=d-i/partman-target.git
[2] http://wiki.debian.org/ReleaseGoals/RunDirectory
[3] http://www.debian.org/doc/debian-policy/ch-opersys.html




Changed Bug title to '"partman-target should not create /var/run"' from 'busybox tar fails to copy filesystem.squashfs' Request was from Rui Bernardo <rui.bernardo.pt@gmail.com> to control@bugs.debian.org. (Wed, 11 Jul 2012 18:27:04 GMT) Full text and rfc822 format available.

Severity set to 'important' from 'normal' Request was from Rui Bernardo <rui.bernardo.pt@gmail.com> to control@bugs.debian.org. (Wed, 11 Jul 2012 18:27:05 GMT) Full text and rfc822 format available.

Bug reassigned from package 'live-installer' to 'partman-target'. Request was from Rui Bernardo <rui.bernardo.pt@gmail.com> to control@bugs.debian.org. (Wed, 11 Jul 2012 18:27:05 GMT) Full text and rfc822 format available.

No longer marked as found in versions live-installer/34. Request was from Rui Bernardo <rui.bernardo.pt@gmail.com> to control@bugs.debian.org. (Wed, 11 Jul 2012 18:27:06 GMT) Full text and rfc822 format available.

Added tag(s) patch. Request was from Rui Bernardo <rui.bernardo.pt@gmail.com> to control@bugs.debian.org. (Wed, 11 Jul 2012 18:27:06 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Install System Team <debian-boot@lists.debian.org>:
Bug#652946; Package partman-target. (Wed, 11 Jul 2012 18:36:06 GMT) Full text and rfc822 format available.

Acknowledgement sent to Rui Bernardo <rui.bernardo.pt@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian Install System Team <debian-boot@lists.debian.org>. (Wed, 11 Jul 2012 18:36:06 GMT) Full text and rfc822 format available.

Message #73 received at 652946@bugs.debian.org (full text, mbox):

From: Rui Bernardo <rui.bernardo.pt@gmail.com>
To: 652946@bugs.debian.org
Subject: Bug#652946: partman-target should not create /var/run
Date: Wed, 11 Jul 2012 19:34:10 +0100
[Message part 1 (text/plain, inline)]
Attached is an eventual patch that could fix the issue.

[0001-create-run-instead-of-var-run-in-target.-Closes-6529.patch (text/x-diff, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Install System Team <debian-boot@lists.debian.org>:
Bug#652946; Package partman-target. (Thu, 12 Jul 2012 00:09:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Rui Bernardo <rui.bernardo.pt@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian Install System Team <debian-boot@lists.debian.org>. (Thu, 12 Jul 2012 00:09:04 GMT) Full text and rfc822 format available.

Message #78 received at 652946@bugs.debian.org (full text, mbox):

From: Rui Bernardo <rui.bernardo.pt@gmail.com>
To: 652946@bugs.debian.org
Subject: Re: Bug#652946: partman-target should not create /var/run
Date: Thu, 12 Jul 2012 01:05:15 +0100
[Message part 1 (text/plain, inline)]
On Wed, Jul 11, 2012 at 07:34:10PM +0100, Rui Bernardo wrote:
> Attached is an eventual patch that could fix the issue.

Sorry, the previous patch didn't create /target/run/lock. Better create 
it to prevent other possible failures about missing /run/lock.

New patch attached.

[0001-create-run-instead-of-var-run-in-target.-Closes-6529.patch (text/x-diff, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Install System Team <debian-boot@lists.debian.org>:
Bug#652946; Package partman-target. (Thu, 12 Jul 2012 05:15:06 GMT) Full text and rfc822 format available.

Acknowledgement sent to Michael Tokarev <mjt@tls.msk.ru>:
Extra info received and forwarded to list. Copy sent to Debian Install System Team <debian-boot@lists.debian.org>. (Thu, 12 Jul 2012 05:15:06 GMT) Full text and rfc822 format available.

Message #83 received at 652946@bugs.debian.org (full text, mbox):

From: Michael Tokarev <mjt@tls.msk.ru>
To: Rui Bernardo <rui.bernardo.pt@gmail.com>, 652946@bugs.debian.org
Subject: Re: Bug#652946: partman-target should not create /var/run
Date: Thu, 12 Jul 2012 09:13:26 +0400
On 11.07.2012 22:34, Rui Bernardo wrote:
> Attached is an eventual patch that could fix the issue.

I think partman should not create /var/run & /var/lock
at all at this point.  Note the comment right before
the mkdir:


 				# Create these before /var is mounted,
 				# so that they can be mounted as tmpfses
-				mkdir -p /target/var/lock
-				mkdir -p /target/var/run
+				mkdir -p /target/run

In previous life, there was a hack to mount /var/run &
/var/lock as tmpfs, and it was done BEFORE /var is
mounted!  Ie, once real /var gets mounted, it hides
run & lock dirs below itself.  The code in initscripts
tried to move /var/run and /var/lock elsewhere before
mounting /var, and to move them back after mounting /var.

Hence, /var/lock and /var/run should exist in root
filesystem even if separate /var is being used, --
these directories wont be created later during install
after we mount /target/var since it will hide the root
filesystem.

This whole hack is not needed anymore, and /run will
be created in a usual way during install process.

So this whole trick should be removed.

Thanks,

/mjt




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Install System Team <debian-boot@lists.debian.org>:
Bug#652946; Package partman-target. (Thu, 26 Jul 2012 09:45:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Andrei Purdea <andrei@purdea.ro>:
Extra info received and forwarded to list. Copy sent to Debian Install System Team <debian-boot@lists.debian.org>. (Thu, 26 Jul 2012 09:45:04 GMT) Full text and rfc822 format available.

Message #88 received at 652946@bugs.debian.org (full text, mbox):

From: Andrei Purdea <andrei@purdea.ro>
To: 652946@bugs.debian.org
Subject: error reporting
Date: Thu, 26 Jul 2012 12:41:20 +0300
Also I think mjt is right. The installer should be able to report if
tar failed for any reason.
Maybe open a new bug about this issue?

-- 
Purdea Andrei



Information forwarded to debian-bugs-dist@lists.debian.org, Debian Install System Team <debian-boot@lists.debian.org>:
Bug#652946; Package partman-target. (Thu, 06 Dec 2012 15:57:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Raphael Hertzog <hertzog@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Install System Team <debian-boot@lists.debian.org>. (Thu, 06 Dec 2012 15:57:03 GMT) Full text and rfc822 format available.

Message #93 received at 652946@bugs.debian.org (full text, mbox):

From: Raphael Hertzog <hertzog@debian.org>
To: Michael Tokarev <mjt@tls.msk.ru>
Cc: Rui Bernardo <rui.bernardo.pt@gmail.com>, 652946@bugs.debian.org
Subject: Re: Bug#652946: partman-target should not create /var/run
Date: Thu, 6 Dec 2012 16:55:15 +0100
[Message part 1 (text/plain, inline)]
On Thu, 12 Jul 2012, Michael Tokarev wrote:
> This whole hack is not needed anymore, and /run will
> be created in a usual way during install process.
> 
> So this whole trick should be removed.

I agree. Here's the corresponding patch. It would be nice if someone
could commit it so that it gets included in the next d-i release.

It's needed to have a proper live image for wheezy.

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Get the Debian Administrator's Handbook:
→ http://debian-handbook.info/get/
[0001-Do-not-create-var-run-and-var-lock-directories-in-ta.patch (text/x-diff, attachment)]

Added indication that 652946 affects live-installer Request was from Raphaël Hertzog <hertzog@debian.org> to control@bugs.debian.org. (Thu, 06 Dec 2012 16:18:02 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Install System Team <debian-boot@lists.debian.org>:
Bug#652946; Package partman-target. (Mon, 17 Dec 2012 09:54:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Cyril Brulebois <kibi@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Install System Team <debian-boot@lists.debian.org>. (Mon, 17 Dec 2012 09:54:03 GMT) Full text and rfc822 format available.

Message #100 received at 652946@bugs.debian.org (full text, mbox):

From: Cyril Brulebois <kibi@debian.org>
To: Raphael Hertzog <hertzog@debian.org>, 652946@bugs.debian.org
Cc: Michael Tokarev <mjt@tls.msk.ru>, Rui Bernardo <rui.bernardo.pt@gmail.com>
Subject: Re: Bug#652946: partman-target should not create /var/run
Date: Mon, 17 Dec 2012 10:51:05 +0100
[Message part 1 (text/plain, inline)]
Hi,

Raphael Hertzog <hertzog@debian.org> (06/12/2012):
> I agree. Here's the corresponding patch. It would be nice if someone
> could commit it so that it gets included in the next d-i release.
> 
> It's needed to have a proper live image for wheezy.

thanks everyone involved, but rc1 doesn't look like an appropriate
target to change /var/* creation; especially since (AFAICT) that only
affects the live installer.

Please get a fix/workaround added there, and push your partman-target
patch into a 'jessie' branch for later inclusion.

Mraw,
KiBi.
[signature.asc (application/pgp-signature, inline)]

Reply sent to Christian Perrier <bubulle@debian.org>:
You have taken responsibility. (Wed, 09 Oct 2013 17:06:05 GMT) Full text and rfc822 format available.

Notification sent to "Rui Miguel P. Bernardo" <rui.bernardo.pt@gmail.com>:
Bug acknowledged by developer. (Wed, 09 Oct 2013 17:06:05 GMT) Full text and rfc822 format available.

Message #105 received at 652946-close@bugs.debian.org (full text, mbox):

From: Christian Perrier <bubulle@debian.org>
To: 652946-close@bugs.debian.org
Subject: Bug#652946: fixed in partman-target 87
Date: Wed, 09 Oct 2013 17:03:26 +0000
Source: partman-target
Source-Version: 87

We believe that the bug you reported is fixed in the latest version of
partman-target, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 652946@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Christian Perrier <bubulle@debian.org> (supplier of updated partman-target package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.8
Date: Wed, 09 Oct 2013 07:02:24 +0200
Source: partman-target
Binary: partman-target
Architecture: source all
Version: 87
Distribution: unstable
Urgency: low
Maintainer: Debian Install System Team <debian-boot@lists.debian.org>
Changed-By: Christian Perrier <bubulle@debian.org>
Description: 
 partman-target - Provides partman with ability to prepare /target (udeb)
Closes: 652946
Changes: 
 partman-target (87) unstable; urgency=low
 .
   [ Raphaël Hertzog ]
   * Do not create /var/run and /var/lock directories in /target. Nowadays
     those are supposed to be symlinks and their existence hurts more than
     helps. Closes: #652946
     Thanks to Rui Bernardo and Michael Tokarev for the investigations.
 .
   [ Updated translations ]
   * Polish (pl.po) by Michał Kułach
Checksums-Sha1: 
 5a71bf5a13b2e81648f48ed3ccd835e407eeae99 1696 partman-target_87.dsc
 a4f913798cab3236b5693a58ac449ee97d71bac2 149132 partman-target_87.tar.gz
 21c43ed3f9185f5fb92cf95db0ac852d11fa88ef 121676 partman-target_87_all.udeb
Checksums-Sha256: 
 171d0fa5633277e62d3e8691cf91df8ea93570ad93abb1c75626189c05f34da6 1696 partman-target_87.dsc
 3d2fc67ff98b94a6b2ccf4aa3995396d115221f357beb86b9f2d889103a1b539 149132 partman-target_87.tar.gz
 fb7af0acc12bf0f25de44a8587d2d1d86c881cda7a498b56157b3370a0447e17 121676 partman-target_87_all.udeb
Files: 
 b0fac732e61cc2ea22c9037dc3c46ec7 1696 debian-installer standard partman-target_87.dsc
 7b0f6151a517f052c6ad57de515d7561 149132 debian-installer standard partman-target_87.tar.gz
 5c676504c70d6946f5f01221c4b39384 121676 debian-installer standard partman-target_87_all.udeb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iQIVAwUBUlWHD4cvcCxNbiWoAQKToA//UFZQ/3bog/u8JvKdp2Pi+XSEgx2hGEvw
5g5rCYzqBZ7gmq3B5uVkCdiRQXKR4OUqA7Mkjh6s4lZ/E5Sf+uwlMOLrcY2Sy8xW
RbN7l//V3YmRTfFUoBcmm3GN9eZ6ObvEmyTkBEzmXnwcyroNdZFPr1F4c5yvB4hZ
G54PWRLblNHwPp1eVHaR/A/aBG4NGiVvme0udc2bA/MXGNNh0BEtbjNjkpkasSs2
ZTtAQA3mnZxM03aKgU4vurCi2oTfhcdpQKygcGfFTB2VYs4Yrabv7cyomTY72Z8A
HQaNSAVhGKkJfebxM+9T83U5fVkrmzgPf/MW1O/rOSJU3MRlP45J88E9I0TjV489
PHVSSdaAdza258vdURpGmMmeETLxEomeo48bX5qAbzxzcRh3TYzUUTiYdCTzZgwZ
DNyQOWHkTY1hVxa0K+jtS+UT8ex30JiDYGPVKVy+JuvOtxcEb+jBtEtSROKJS5Uu
wN02gd/ZCHU102VkzdH03wsz0/SI0bOPafsdGy8tMoRwJNuJmsJhAU1FIMAYXP7w
zDY3vXo6adVMwhdzHMFMVzhxvwoSg4wFaoVChhpuwocJ+VcPIb7gSiz2rM3NVFwc
QJvRJkLjYEq9gDFX8Z8zq0wOhS3X8m3hjuqrVKa/bepr7L1Oh7ovXXLF2mK/oMzY
7YAkPmf4CyU=
=REm2
-----END PGP SIGNATURE-----




Bug archived. Request was from Debbugs Internal Request <owner@bugs.debian.org> to internal_control@bugs.debian.org. (Sun, 17 Nov 2013 07:30:04 GMT) Full text and rfc822 format available.

Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Sat Apr 19 02:17:42 2014; Machine Name: beach.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.