Debian Bug report logs -
#575891
dpkg makes wrong assumption about readdir() and lose metadata files with btrfs
Reported by: "Trent W. Buck" <trentbuck@gmail.com>
Date: Tue, 30 Mar 2010 07:45:05 UTC
Severity: important
Tags: confirmed, patch
Merged with 567135,
568908
Found in version dpkg/1.15.5.6
Fixed in version dpkg/1.15.7
Done: Guillem Jover <guillem@debian.org>
Bug is archived. No further changes may be made.
Toggle useless messages
Report forwarded
to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#575891; Package dpkg.
(Tue, 30 Mar 2010 07:45:08 GMT) (full text, mbox, link).
Acknowledgement sent
to "Trent W. Buck" <trentbuck@gmail.com>:
New Bug report received and forwarded. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>.
(Tue, 30 Mar 2010 07:45:08 GMT) (full text, mbox, link).
Message #5 received at submit@bugs.debian.org (full text, mbox, reply):
[Message part 1 (text/plain, inline)]
Package: dpkg
Version: 1.15.5.6
Severity: important
Sometimes files in /var/lib/dpkg/info aren't created when
(re)installing packages with aptitude.
I became concerned that postinstall scripts weren't being run: first
/usr/bin/crontab wasn't setgid, and then #575797 for graphviz. On
investigating the latter, I noticed that both were missing .postinst
scripts in /var/lib/dpkg/info/ -- but the source and binary packaging
included postinst scripts.
The graphviz issue occurred while I was running dpkg from experimental
(per the recent dda request), but IIRC the cron issue arose long
before then. So I
- downgraded source:dpkg packages back to unstable
- took a tarball of /var/lib/dpkg/info
- ran aptitude reinstall '~i !~E !~prequired !x11-common'
- took a tarball of /var/lib/dpkg/info
Attached is a transcript of the aptitude run, and the result of
diffing the (sorted) output of tar -tf on the two tarballs. You can
see that there's a staggering number of metadata added and removed,
despite "aptitude reinstall" installing the same binary package
versions.
The aptitude pattern matches all packages that are
- installed;
- not Essential: yes
- not Priority: required; and
- not x11-common.
I basically did this to reinstall as much as possible while avoiding
dependency loops (since they cause aptitude to abort).
Note that three packages in the run fail to reinstall because one of
their dependencies (zope) fails to run its postinst, and thus isn't
visible to the python calls in THEIR postinsts.
I'm initially assigning this to dpkg because I don't know what other
tests will isolate the problem. I *am* running btrfs, but I can't
imagine how that could cause this symptom. My kernel comes from
experimental, and is not hand-rolled.
$ cat /proc/mounts
rootfs / rootfs rw 0 0
none /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
none /proc proc rw,nosuid,nodev,noexec,relatime 0 0
udev /dev tmpfs rw,relatime,size=10240k,mode=755 0 0
/dev/disk/by-label/dali / btrfs rw,relatime,compress,ssd_spread 0 0
tmpfs /lib/init/rw tmpfs rw,nosuid,relatime,mode=755 0 0
varrun /var/run tmpfs rw,nosuid,relatime,size=8192k,mode=755 0 0
varlock /var/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=8192k 0 0
tmpfs /dev/shm tmpfs rw,nosuid,nodev,relatime,size=131072k 0 0
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
/dev/sda1 /boot ext2 rw,relatime,errors=continue 0 0
tmpfs /tmp tmpfs rw,relatime 0 0
fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0
-- System Information:
Debian Release: squeeze/sid
APT prefers unstable
APT policy: (500, 'unstable'), (1, 'experimental')
Architecture: amd64 (x86_64)
Kernel: Linux 2.6.33-2-amd64 (SMP w/2 CPU cores)
Locale: LANG=en_AU.utf8, LC_CTYPE=en_AU.utf8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Versions of packages dpkg depends on:
ii coreutils 8.4-2 GNU core utilities
ii libc6 2.10.2-6 Embedded GNU C Library: Shared lib
ii lzma 4.43-14 Compression method of 7z format in
dpkg recommends no packages.
Versions of packages dpkg suggests:
ii apt 0.7.25.3 Advanced front-end for dpkg
-- no debconf information
[listing.diff (text/plain, attachment)]
[typescript (text/plain, attachment)]
Information forwarded
to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#575891; Package dpkg.
(Tue, 30 Mar 2010 08:18:03 GMT) (full text, mbox, link).
Acknowledgement sent
to Sven Joachim <svenjoac@gmx.de>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>.
(Tue, 30 Mar 2010 08:18:03 GMT) (full text, mbox, link).
Message #10 received at 575891@bugs.debian.org (full text, mbox, reply):
On 2010-03-30 09:31 +0200, Trent W. Buck wrote:
> Package: dpkg
> Version: 1.15.5.6
> Severity: important
>
> Sometimes files in /var/lib/dpkg/info aren't created when
> (re)installing packages with aptitude.
>
> I became concerned that postinstall scripts weren't being run: first
> /usr/bin/crontab wasn't setgid, and then #575797 for graphviz. On
> investigating the latter, I noticed that both were missing .postinst
> scripts in /var/lib/dpkg/info/ -- but the source and binary packaging
> included postinst scripts.
> [...]
> I'm initially assigning this to dpkg because I don't know what other
> tests will isolate the problem. I *am* running btrfs, but I can't
> imagine how that could cause this symptom. My kernel comes from
> experimental, and is not hand-rolled.
See also http://kitenet.net/~joey/blog/entry/aloha_btrfs/ for the
misfortunes somebody else had with the btrfs filesystem.
Sven
Information forwarded
to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#575891; Package dpkg.
(Tue, 30 Mar 2010 09:18:06 GMT) (full text, mbox, link).
Acknowledgement sent
to Raphael Hertzog <hertzog@debian.org>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>.
(Tue, 30 Mar 2010 09:18:07 GMT) (full text, mbox, link).
Message #15 received at 575891@bugs.debian.org (full text, mbox, reply):
Hi Trent,
On Tue, 30 Mar 2010, Trent W. Buck wrote:
> I'm initially assigning this to dpkg because I don't know what other
> tests will isolate the problem. I *am* running btrfs, but I can't
> imagine how that could cause this symptom. My kernel comes from
> experimental, and is not hand-rolled.
You can't imagine how an experimental filesystem could loose your data?
I'm sorry, there's no way that we as dpkg maintainers are going to
investigate problems that are likely to come from the filesystem itself at
this point.
It's possible that dpkg triggers some special cases that btrfs doesn't
deal well with. I suggest you report your problems to the upstream btrfs
maintainers, they are likely interested in a recipe to reproduce the
data loss.
But this bug should not be assigned to dpkg. You might want to reassign to
the kernel but I doubt debian kernel maintainers are interested in dealing
with it. IMO bugs against experimental filesystems should go upstream
directly.
Cheers,
--
Raphaël Hertzog
Like what I do? Sponsor me: http://ouaza.com/wp/2010/01/05/5-years-of-freexian/
My Debian goals: http://ouaza.com/wp/2010/01/09/debian-related-goals-for-2010/
Information forwarded
to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#575891; Package dpkg.
(Tue, 06 Apr 2010 16:12:08 GMT) (full text, mbox, link).
Acknowledgement sent
to Chris Mason <chris.mason@oracle.com>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>.
(Tue, 06 Apr 2010 16:12:08 GMT) (full text, mbox, link).
Message #20 received at 575891@bugs.debian.org (full text, mbox, reply):
We've gone through the strace logs and thanks to some pointers to the dpkg
code, I think I can explain this. Dpkg does something like this:
while( filename = readdir(/var/dpkg/info)) {
err = rename(/var/lib/dpkg/tmp.ci/filename, /var/lib/dpkg/info);
if (err) {
unlink /var/lib/dpkg/info/filename
}
}
The basic idea is that for each file in /var/lib/dpkg/info we overwrite it
with a file from /var/lib/dpkg/tmp.ci. Btrfs implements readdir with a
per-directory sequence number. Each time you make a new file in the
directory it gets a new sequence number, which is returned as the offset
for seekdir/telldir use.
The rename operation is creating a new file, so it can show up again in
the next readdir call. The strace shows readdir finding a file name,
then renaming over it into /var/lib/dpkg/info and then finding the file
name again.
This could actually happen with any of the filesystems, but btrfs is
especially consistent at reproducing it. A while ago, git hit a similar
problem:
http://www.mail-archive.com/btrfs-devel@oss.oracle.com/msg00370.html
Which we solved in btrfs by forcing a very high readdir offset when the
entire directory had been read, but this doesn't quite work for dpkg
because it doesn't read all the directory entries before it starts
processing them. Git later added a commit to avoid the problem
internally as well.
I would suggest changing the loop to look like this:
while (filename = readdir(/var/dpkg/info)) {
add filename to list of things that need processing
}
for each filename in list {
rename (/var/lib/dpkg/tmp.ci/filename, /var/lib/dpkg/info)
}
In other words, buffer the readdir results. Just let me know if you
have other questions about this, thanks for bringing us into the
debugging.
-chris
Information forwarded
to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#575891; Package dpkg.
(Tue, 06 Apr 2010 16:48:05 GMT) (full text, mbox, link).
Acknowledgement sent
to Raphael Hertzog <hertzog@debian.org>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>.
(Tue, 06 Apr 2010 16:48:05 GMT) (full text, mbox, link).
Message #25 received at 575891@bugs.debian.org (full text, mbox, reply):
retitle 575891 dpkg makes wrong assumption about readdir() and lose metadata files with btrfs
severity 575891 important
tag 575891 confirmed
thanks
On Tue, 30 Mar 2010, Raphael Hertzog wrote:
> I'm sorry, there's no way that we as dpkg maintainers are going to
> investigate problems that are likely to come from the filesystem itself at
> this point.
Fortunately other people stepped in (cwillu aka Carey Underwood) and asked
the btrfs authors.
The problem appears to be in dpkg itself, it is assuming that readdir()
will not return the same filename twice but with btrfs it does when one
of the files in the directory is replaced while readdir() is still
happening.
The problematic code is in src/processarc.c after line 828:
What it does in pseudo-code:
opendir("/var/lib/dpkg/info");
while(readdir() != NULL) {
if (rename(newer_version_in_tmp, file_in_info_dir) == 0) {
// file updated with newer version
} else if (errno == ENOENT) {
// file not found in newer version, assume it's removed
unlink(file_in_info_dir);
}
}
Now considers that after a successfull rename(), btrfs will return the
newly replaced file again because it's not the same file (different inode
even though it has a filename that has already been returned)
and it's treated like an added file (and POSIX doesn't specify what
implementations are supposed to do:
http://www.opengroup.org/onlinepubs/9699919799/functions/readdir.html)
The second time we see the file, the original temporary file is gone and
we decide to unlink the just installed file...
Thus the rename() needs to happen after the whole readdir() loop is finished.
Cheers,
--
Raphaël Hertzog
Like what I do? Sponsor me: http://ouaza.com/wp/2010/01/05/5-years-of-freexian/
My Debian goals: http://ouaza.com/wp/2010/01/09/debian-related-goals-for-2010/
Changed Bug title to 'dpkg makes wrong assumption about readdir() and lose metadata files with btrfs' from '/var/lib/dpkg/info metadata being "eaten".'
Request was from Raphael Hertzog <hertzog@debian.org>
to control@bugs.debian.org.
(Tue, 06 Apr 2010 16:48:07 GMT) (full text, mbox, link).
Added tag(s) confirmed.
Request was from Raphael Hertzog <hertzog@debian.org>
to control@bugs.debian.org.
(Tue, 06 Apr 2010 16:48:07 GMT) (full text, mbox, link).
Information forwarded
to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#575891; Package dpkg.
(Tue, 06 Apr 2010 18:48:06 GMT) (full text, mbox, link).
Acknowledgement sent
to Joey Hess <joey@kitenet.net>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>.
(Tue, 06 Apr 2010 18:48:06 GMT) (full text, mbox, link).
Message #34 received at 575891@bugs.debian.org (full text, mbox, reply):
[Message part 1 (text/plain, inline)]
cwillu wrote:
> New information on http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=575891
>
> Turns out to have been an unsafe assumption on dpkg's part with
> apparently astronomic odds of being triggered on most filesystems.
>
> Putting /var/lib/dpkg on an ext3 mount (I used a bind mount from my
> /boot) works around it until the problem can be fixed in dpkg.
Cwillu, thanks for following up.
That does seem to explain #569058, closing that bug.
At the moment, my other two problems, #567135 and #568908, do not
seem likely to be caused by the dpkg bug.
--
see shy jo
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#575891; Package dpkg.
(Wed, 07 Apr 2010 07:27:10 GMT) (full text, mbox, link).
Acknowledgement sent
to trentbuck@gmail.com:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>.
(Wed, 07 Apr 2010 07:27:10 GMT) (full text, mbox, link).
Message #39 received at 575891@bugs.debian.org (full text, mbox, reply):
cwillu wrote:
> Putting /var/lib/dpkg on an ext3 mount (I used a bind mount from my
> /boot) works around it until the problem can be fixed in dpkg.
After that, don't forget to reinstall packages to get their missing
files back. I couldn't think of an easy way to work out which
packages were busted so I reinstalled everything. From memory:
# Move /var/lib/dpkg into a loopback ext2 filesystem.
cp -a /var/lib/dpkg /var/lib/dpkg.~1~
dd bs=1 count=0 seek=128M of=/var/lib/dpkg.ext
mke2fs /var/lib/dpkg.ext
echo >>/etc/fstab "# Workaround for http://bugs.debian.org/575891"
echo >>/etc/fstab "/var/lib/dpkg.ext /var/lib/dpkg ext2 loop,sync 0 2"
mount -a
cp -a /var/lib/dpkg.~1~/* /var/lib/dpkg/
# Reinstall (almost) everything at once.
aptitude reinstall '~i !~E !~prequired !x11-common'
# These have dependency loops, so must be reinstalled singly.
for i in `aptitude search -F %p '~i(~E|~prequired|x11-common)'`
do aptitude reinstall $i
done
Information forwarded
to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#575891; Package dpkg.
(Fri, 09 Apr 2010 07:06:03 GMT) (full text, mbox, link).
Acknowledgement sent
to Raphael Hertzog <hertzog@debian.org>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>.
(Fri, 09 Apr 2010 07:06:03 GMT) (full text, mbox, link).
Message #46 received at 575891@bugs.debian.org (full text, mbox, reply):
[Message part 1 (text/plain, inline)]
tag 575891 patch
thanks
On Tue, 06 Apr 2010, Joey Hess wrote:
> cwillu wrote:
> > New information on http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=575891
> >
> > Turns out to have been an unsafe assumption on dpkg's part with
> > apparently astronomic odds of being triggered on most filesystems.
>
> Cwillu, thanks for following up.
Hi all,
I have prepared a patch for this issue, see attachment. I would be glad
if some of you could test it.
Guillem, a quick ok from you would be nice (feel free to point stylistic
issues if you have some).
I want to push this patch in the next update (1.15.6.2).
Cheers,
--
Raphaël Hertzog
Like what I do? Sponsor me: http://ouaza.com/wp/2010/01/05/5-years-of-freexian/
My Debian goals: http://ouaza.com/wp/2010/01/09/debian-related-goals-for-2010/
[patch (text/plain, attachment)]
Added tag(s) patch.
Request was from Raphael Hertzog <hertzog@debian.org>
to control@bugs.debian.org.
(Fri, 09 Apr 2010 07:06:05 GMT) (full text, mbox, link).
Information forwarded
to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#575891; Package dpkg.
(Fri, 09 Apr 2010 13:57:11 GMT) (full text, mbox, link).
Acknowledgement sent
to cwillu <cwillu@cwillu.com>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>.
(Fri, 09 Apr 2010 13:57:11 GMT) (full text, mbox, link).
Message #53 received at 575891@bugs.debian.org (full text, mbox, reply):
Just finished testing the patch; looks good here.
Information forwarded
to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#575891; Package dpkg.
(Fri, 09 Apr 2010 14:27:08 GMT) (full text, mbox, link).
Acknowledgement sent
to Guillem Jover <guillem@debian.org>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>.
(Fri, 09 Apr 2010 14:27:08 GMT) (full text, mbox, link).
Message #58 received at 575891@bugs.debian.org (full text, mbox, reply):
Hi!
On Fri, 2010-04-09 at 09:02:22 +0200, Raphael Hertzog wrote:
> I want to push this patch in the next update (1.15.6.2).
Ah perfect! I wanted to fix this too for the next release, glad you
got to it while I was away.
> Guillem, a quick ok from you would be nice (feel free to point stylistic
> issues if you have some).
Overall the patch looks good, I had in mind doing it in just two
passes by storing enough information, but this is fine for now, the
rest can be dealt with in future refactoring work. But see below:
> diff --git a/src/archives.h b/src/archives.h
> index 2390bb8..0ee8057 100644
> --- a/src/archives.h
> +++ b/src/archives.h
> @@ -35,6 +35,12 @@ struct pkg_deconf_list {
> struct pkginfo *pkg_removal;
> };
>
> +struct rename_list {
> + struct rename_list *next;
> + char *src;
> + char *dest;
I'd name this dst for consistency with src (the rest of the new code
uses this too).
> +};
> +
And ‘struct rename_list’ does not need to be exposed, it can just placed
in processarc.c.
> extern int fnameidlu;
> extern struct varbuf fnamevb;
> extern struct varbuf fnametmpvb;
> diff --git a/src/processarc.c b/src/processarc.c
> index fe5ca1f..b596c79 100644
> --- a/src/processarc.c
> +++ b/src/processarc.c
> @@ -118,6 +118,7 @@ void process_archive(const char *filename) {
> struct dirent *de;
> struct stat stab, oldfs;
> struct pkg_deconf_list *deconpil, *deconpiltemp;
> + struct rename_list *rename_head = NULL, *rename_temp = NULL;
I'd name rename_temp as rename_node instead.
>
> cleanup_pkg_failed= cleanup_conflictor_failed= 0;
> admindirlen= strlen(admindir);
> @@ -850,19 +851,35 @@ void process_archive(const char *filename) {
> varbufaddstr(&infofnvb,de->d_name);
> varbufaddc(&infofnvb,0);
> strcpy(cidirrest,p);
> - if (!rename(cidir,infofnvb.buf)) {
> + /* We keep files to rename in a list as doing the rename immediately
> + * might influence the current readdir(), the just renamed file might
> + * be returned a second time as it's actually a new file from the
> + * point of view of the filesystem. */
> + rename_temp = m_malloc(sizeof(struct rename_list));
Here use sizeof(*rename_node) so it's resilient against type change.
> + rename_temp->next = rename_head;
> + rename_temp->src = m_strdup(cidir);
> + rename_temp->dest = m_strdup(infofnvb.buf);
> + rename_head = rename_temp;
> + }
> + pop_cleanup(ehflag_normaltidy); /* closedir */
> +
> + for(rename_temp = rename_head; rename_temp; rename_temp = rename_head) {
Missing space after ‘for’, but to avoid the redundancy I'd use instead:
while ((rename_node = rename_head)) {
> + if (!rename(rename_temp->src, rename_temp->dest)) {
> debug(dbg_scripts, "process_archive info installed %s as %s",
> - cidir, infofnvb.buf);
> + rename_temp->src, rename_temp->dest);
> } else if (errno == ENOENT) {
> /* Right, no new version. */
> - if (unlink(infofnvb.buf))
> - ohshite(_("unable to remove obsolete info file `%.250s'"),infofnvb.buf);
> - debug(dbg_scripts, "process_archive info unlinked %s",infofnvb.buf);
> + if (unlink(rename_temp->dest))
> + ohshite(_("unable to remove obsolete info file `%.250s'"), rename_temp->dest);
> + debug(dbg_scripts, "process_archive info unlinked %s", rename_temp->dest);
> } else {
> - ohshite(_("unable to install (supposed) new info file `%.250s'"),cidir);
> + ohshite(_("unable to install (supposed) new info file `%.250s'"), rename_temp->src);
Both ohshite node arguments could get wrapped into the next line to
avoid exceeding line length.
> }
> + rename_head = rename_temp->next;
> + free(rename_temp->src);
> + free(rename_temp->dest);
> + free(rename_temp);
> }
> - pop_cleanup(ehflag_normaltidy); /* closedir */
>
> *cidirrest = '\0'; /* the directory itself */
> dsd= opendir(cidir);
With those:
Acked-by: Guillem Jover <guillem@debian.org>
thanks,
guillem
Information forwarded
to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#575891; Package dpkg.
(Fri, 09 Apr 2010 16:06:12 GMT) (full text, mbox, link).
Acknowledgement sent
to Raphael Hertzog <hertzog@debian.org>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>.
(Fri, 09 Apr 2010 16:06:12 GMT) (full text, mbox, link).
Message #63 received at 575891@bugs.debian.org (full text, mbox, reply):
Hi,
On Fri, 09 Apr 2010, Guillem Jover wrote:
> > Guillem, a quick ok from you would be nice (feel free to point stylistic
> > issues if you have some).
>
> Overall the patch looks good, I had in mind doing it in just two
> passes by storing enough information, but this is fine for now, the
> rest can be dealt with in future refactoring work. But see below:
Thanks for the quick review. Fixed them all and pushed.
Cheers,
--
Raphaël Hertzog
Like what I do? Sponsor me: http://ouaza.com/wp/2010/01/05/5-years-of-freexian/
My Debian goals: http://ouaza.com/wp/2010/01/09/debian-related-goals-for-2010/
Added tag(s) pending.
Request was from Raphaël Hertzog <hertzog@debian.org>
to control@bugs.debian.org.
(Fri, 09 Apr 2010 16:06:14 GMT) (full text, mbox, link).
Message sent on
to "Trent W. Buck" <trentbuck@gmail.com>:
Bug#575891.
(Fri, 09 Apr 2010 16:06:20 GMT) (full text, mbox, link).
Message #68 received at 575891-submitter@bugs.debian.org (full text, mbox, reply):
tag 575891 pending
thanks
Hello,
Bug #575891 reported by you has been fixed in the Git repository. You can
see the changelog below, and you can check the diff of the fix at:
http://git.debian.org/?p=dpkg/dpkg.git;a=commitdiff;h=18b1208
---
commit 18b12083b5fee4e7e26e1382e50321e7956fcdb9
Author: Raphaël Hertzog <hertzog@debian.org>
Date: Fri Apr 9 08:35:47 2010 +0200
dpkg: fix metadata installation by not mixing rename() in a readdir() loop
dpkg's process_archive() was doing the improper assumption that a
readdir() loop would not return the same filename twice even when the
scanned directory has files renamed into it (coming from tmp.ci).
The net result of having the same filename returned twice is that the
the second time the updated file to install is no longer there and
thus dpkg removed the current metadata file believing that it was
obsolete. btrfs triggers this bug consistently.
All other readdir() occurrences have been reviewed as well for similar
problems. But they are all safe, they mainly unlink() files rather
than adding new files into the scanned directory.
Thanks to Carey Underwood and Chris Mason for their help in diagnosing
this problem.
Acked-by: Guillem Jover <guillem@debian.org>
diff --git a/debian/changelog b/debian/changelog
index adaa95f..f699fea 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -14,6 +14,9 @@ dpkg (1.15.6.2) UNRELEASED; urgency=low
do not include that file in the generated source package.
* Improve explanation of --all option in dpkg-parsechangelog(1). Thanks to
Jari Aalto. Closes: #575706
+ * Fix dpkg to not lose package metadata on filesystems where readdir()
+ returns new files added after the opendir() call, btrfs in particular
+ triggered the problematic behaviour. Closes: #575891
[ Updated dpkg translations ]
* German (Sven Joachim).
Reply sent
to Guillem Jover <guillem@debian.org>:
You have taken responsibility.
(Wed, 21 Apr 2010 03:36:26 GMT) (full text, mbox, link).
Notification sent
to "Trent W. Buck" <trentbuck@gmail.com>:
Bug acknowledged by developer.
(Wed, 21 Apr 2010 03:36:26 GMT) (full text, mbox, link).
Message #73 received at 575891-close@bugs.debian.org (full text, mbox, reply):
Source: dpkg
Source-Version: 1.15.7
We believe that the bug you reported is fixed in the latest version of
dpkg, which is due to be installed in the Debian FTP archive:
dpkg-dev_1.15.7_all.deb
to main/d/dpkg/dpkg-dev_1.15.7_all.deb
dpkg_1.15.7.dsc
to main/d/dpkg/dpkg_1.15.7.dsc
dpkg_1.15.7.tar.bz2
to main/d/dpkg/dpkg_1.15.7.tar.bz2
dpkg_1.15.7_amd64.deb
to main/d/dpkg/dpkg_1.15.7_amd64.deb
dselect_1.15.7_amd64.deb
to main/d/dpkg/dselect_1.15.7_amd64.deb
libdpkg-dev_1.15.7_amd64.deb
to main/d/dpkg/libdpkg-dev_1.15.7_amd64.deb
libdpkg-perl_1.15.7_all.deb
to main/d/dpkg/libdpkg-perl_1.15.7_all.deb
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to 575891@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Guillem Jover <guillem@debian.org> (supplier of updated dpkg package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Format: 1.8
Date: Wed, 21 Apr 2010 04:05:55 +0200
Source: dpkg
Binary: libdpkg-dev dpkg dpkg-dev libdpkg-perl dselect
Architecture: source amd64 all
Version: 1.15.7
Distribution: unstable
Urgency: low
Maintainer: Dpkg Developers <debian-dpkg@lists.debian.org>
Changed-By: Guillem Jover <guillem@debian.org>
Description:
dpkg - Debian package management system
dpkg-dev - Debian package development tools
dselect - Debian package management front-end
libdpkg-dev - Debian package management static library
libdpkg-perl - Dpkg perl modules
Closes: 514316 546577 553928 556889 559519 560070 568566 574599 575706 575891 577756 578162
Changes:
dpkg (1.15.7) unstable; urgency=low
.
[ Raphaël Hertzog ]
* Clarify the plan concerning dpkg-source, debian/source/format and
the default source format in dpkg-source(1). Add a warning
in dpkg-source to invite people to always create debian/source/format.
We deprecate the fallback to "1.0" (it's there for backwards compatibility
only) and debian/source/format is going to be mandatory at some point in
the future. Closes: #553928
* Add .gitattributes to list of files ignored by dpkg-source.
* Document most common warnings and errors of dpkg-source in its manual
page.
* Let dpkg-source read options from debian/source/local-options as well but
do not include that file in the generated source package.
* Improve explanation of --all option in dpkg-parsechangelog(1). Thanks to
Jari Aalto. Closes: #575706
* Fix dpkg to not lose package metadata on filesystems where readdir()
returns new files added after the opendir() call, btrfs in particular
triggered the problematic behaviour. Closes: #575891
* Tigthen the regex used by dpkg-source to match the component name of
supplementary tarballs so that undercore (_) are not allowed as it was
supposed to be.
* Introduce a new script called dpkg-buildflags: its purpose is to retrieve
compilation flags and it should be used within debian/rules to pass
the right compilation flags to the build process. dpkg-builpackage still
exports them to not break packages currently relying on them but packages
should now start using dpkg-buildflags instead. Closes: #560070
* For Ubuntu set default value of LDFLAGS to -Wl,-Bsymbolic-functions.
* Cleanup some old Conflicts/Replaces, thanks to Helge Kreutzmann.
* Modify dselect to treat all unknown package as known and marked for purge.
This is a temporary work-around so that dselect doesn't try to reinstall
packages of priority > standard that were removed or not installed. Thanks
to Robert Luderda for the patch. Closes: #559519, #556889
* dpkg now exports DPKG_MAINTSCRIPT_NAME to maintainer scripts with the
type of maintainer script currently running (preinst, postinst, prerm,
postrm). Closes: #546577
* dpkg now exports DPKG_LIBDIR to maintainer scripts pointing to the
private directory containing internal programs like the upcoming
maintscript-helper.
* Add $DPKG_LIBDIR/maintscript-helper program that can be used in
maintainer scripts to perform common operations working around
current dpkg limitations: first version supports removing obsolete
conffiles and renaming conffiles. Closes: #514316
* Fix "dpkg-scansources -e", it was calling a non-existing function.
Closes: #578162
* Add new script dpkg-mergechangelogs to do 3-way merges of Debian
changelogs. Add libalgorithm-merge-perl to Recommends for the
benefit of this script.
.
[ Colin Watson ]
* Modern tar files typically use NormalFile1 rather than NormalFile0 for
file objects. A typo meant that the former never triggered rename
deferral. Closes: #577756
* Use the new list of files on rename deferral instead of old one, so that
newly added files get installed.
.
[ Guillem Jover ]
* Report deferred trigger errors on status-fd. Closes: #574599
Thanks to Michael Vogt <michael.vogt@ubuntu.com>.
* When creating hard links to normal files on extraction use the .dpkg-new
filename for source as the file is not yet in place due to the rename
deferral. Thanks to Colin Watson for the initial patch.
* Do not output the Package-Type field on udeb.
* Fix versioned Replaces to not produce file overwrite errors on downgrades.
Closes: #568566
* Fix installation of replaced and replacing packages in reverse order
(first the replacing then the replaced) for which the replaced package
is supposed to get disappeared, to disappear the correct package and not
lose track of the ownership of the replaced files.
.
[ Updated dpkg translations ]
* German (Sven Joachim).
.
[ Updated dselect translations ]
* German (Sven Joachim).
.
[ Updated man page translations ]
* German (Helge Kreutzmann).
.
[ Updated scripts translations ]
* German (Helge Kreutzmann).
Checksums-Sha1:
2f096f1027105eb1a9950692cda75d338c4b52c7 1213 dpkg_1.15.7.dsc
fdf14a74370fb9daac4a67d703f6e60c20eaec51 4872332 dpkg_1.15.7.tar.bz2
ccc84900a91c120d260153f3095a43445013548e 371632 libdpkg-dev_1.15.7_amd64.deb
41ab7052b7d606749f434dc519388b6d1821ae22 2062348 dpkg_1.15.7_amd64.deb
4a5fbbf048151f1468a00155cf13daa0ce6aea86 828512 dselect_1.15.7_amd64.deb
6760d76f7b085b8e2c2d7f6dfec4d69984413823 648164 dpkg-dev_1.15.7_all.deb
6a28af1efa9b249a4536b4cc10d8e61d9dfb9925 578192 libdpkg-perl_1.15.7_all.deb
Checksums-Sha256:
d8d261e40d6d0f48dd39dee867cfcb0eb2385423fc9ee5948657a6785406bddd 1213 dpkg_1.15.7.dsc
f8d0a779361f3fd7f89a138cd632a4ab47f62ca1ccd0f393f2e81be899605e4f 4872332 dpkg_1.15.7.tar.bz2
e8470ce90649f17a05f1e9d47e440ae2568feb0946e514431ba579130afe9378 371632 libdpkg-dev_1.15.7_amd64.deb
077c270b0f31181081751b8a66a98a9ca598d1b64a689218e882d2841f8b6bbe 2062348 dpkg_1.15.7_amd64.deb
0e0d45a5001dcbafbfb36b789b3c6316ddd7cb4d9d4d4cf847912a7273932aa7 828512 dselect_1.15.7_amd64.deb
ca757de11c7a65791e67d807068ef5d592605dbce7632ed4b83d94715fb9acd0 648164 dpkg-dev_1.15.7_all.deb
05fc8150c8b8ea29315d8a5f1a48d8cb7453d1ce1a95a3af492f3e8acec7461d 578192 libdpkg-perl_1.15.7_all.deb
Files:
2df9bfed2c1c2342da2196925ab597fb 1213 admin required dpkg_1.15.7.dsc
f4638fd8a623aa48efd62b48644b1ab3 4872332 admin required dpkg_1.15.7.tar.bz2
508f4f97ac08d1e23f98fad3801535fe 371632 libdevel optional libdpkg-dev_1.15.7_amd64.deb
c9d13d664114569a3884721f52a2b8b5 2062348 admin required dpkg_1.15.7_amd64.deb
ca86a0334edbbaef23354ada093473a6 828512 admin optional dselect_1.15.7_amd64.deb
e850c25f7e57b1b570181b94522259de 648164 utils optional dpkg-dev_1.15.7_all.deb
2ff2c47b5e043f0704b90b1245fb3614 578192 perl optional libdpkg-perl_1.15.7_all.deb
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
iEYEARECAAYFAkvOZUkACgkQuW9ciZ2SjJt4TgCeNn227r3zWFjOyy/12jo6WH9z
XLwAoNjTBZAyEb30antcMbFdZvgtP1h8
=egWK
-----END PGP SIGNATURE-----
Reply sent
to Guillem Jover <guillem@debian.org>:
You have taken responsibility.
(Wed, 21 Apr 2010 03:36:27 GMT) (full text, mbox, link).
Notification sent
to Joey Hess <joeyh@debian.org>:
Bug acknowledged by developer.
(Wed, 21 Apr 2010 03:36:27 GMT) (full text, mbox, link).
Reply sent
to Guillem Jover <guillem@debian.org>:
You have taken responsibility.
(Wed, 21 Apr 2010 03:36:28 GMT) (full text, mbox, link).
Notification sent
to Joey Hess <joeyh@debian.org>:
Bug acknowledged by developer.
(Wed, 21 Apr 2010 03:36:28 GMT) (full text, mbox, link).
Bug archived.
Request was from Debbugs Internal Request <owner@bugs.debian.org>
to internal_control@bugs.debian.org.
(Thu, 20 May 2010 07:31:31 GMT) (full text, mbox, link).
Send a report that this bug log contains spam.
Debian bug tracking system administrator <owner@bugs.debian.org>.
Last modified:
Tue Aug 14 21:55:17 2018;
Machine Name:
buxtehude
Debian Bug tracking system
Debbugs is free software and licensed under the terms of the GNU
Public License version 2. The current version can be obtained
from https://bugs.debian.org/debbugs-source/.
Copyright © 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson,
2005-2017 Don Armstrong, and many other contributors.