Debian Bug report logs - #598438
coreutils: new du corrupted filesystem warning breaks scripts

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Graham Cobb <g+debian@cobb.uk.net>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: coreutils: new du corrupted filesystem warning breaks scripts

Date: Tue, 28 Sep 2010 23:16:16 +0100

Package: coreutils
Version: 8.5-1
Severity: important


Like many systems, my filesystem contains loops due to bind mounts (in my case
it is for scratchbox, the reporter of bug 563254 has a different reason, I
have had other reasons in the past as well).

I use a backup script called dt-s3-backup.sh, which has been working for some
time but has suddenly broken due to the change in du's handling of filesystem
loops. 

Although I exclude the looped files from my backup, the script uses a du 
command which crosses the filesystem loop. Previously, du silently ignored the 
files below the loop -- the script depends on this behaviour and breaks
completely now that du returns an error.

I know about the filesystem loop.  It in no way represents any filesystem
corruption and the change in behaviour should not have been made without some
way to disable it.  I do not object to the new behaviour being the default but
there needs to be an option to disable the error and silently ignore the
loop.

A simple test case to reproduce this problem:

mkdir -p /tmp/du-test/loop
echo hello >/tmp/du-test/a
mount -o bind /tmp /tmp/du-test/loop
du -s /tmp
umount /tmp/du-test/loop

Note: there is no way to avoid this problem using the --exclude option as
excluding the directory does not stop du reporting the error.  This can be seen
by replacing the du command above with:

du --exclude /tmp/du-test/loop -s /tmp

It is possible to use "--exclude=/tmp/du-test" but that excludes too much --
the du command needs to count files such as /tmp/du-test/a.

Note: in this case the script does not specify -x (and it cannot do so).  That
is why I did not add this as additional information to bug #563254.

I believe the change should be backed out for Squeeze as it has the potential 
to break important scripts on any system with filesystem loops.  For the 
subsequent release, a new option can be added to du to ignore loops and the
default behaviour can be changed to report the problem.

-- System Information:
Debian Release: squeeze/sid
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.32-5-amd64 (SMP w/2 CPU cores)
Locale: LANG=en_IE@euro, LC_CTYPE=en_IE@euro (charmap=ISO-8859-15) (ignored: LC_ALL set to en_IE@euro)
Shell: /bin/sh linked to /bin/bash

Versions of packages coreutils depends on:
ii  libacl1                       2.2.49-3   Access control list shared library
ii  libattr1                      1:2.4.44-2 Extended attribute shared library
ii  libc6                         2.11.2-5   Embedded GNU C Library: Shared lib
ii  libselinux1                   2.0.96-1   SELinux runtime shared libraries

coreutils recommends no packages.

coreutils suggests no packages.

-- no debconf information

Message #10 received at 598438@bugs.debian.org (full text, mbox, reply):

From: Jim Meyering <jim@meyering.net>

To: Graham Cobb <g+debian@cobb.uk.net>

Cc: 598438@bugs.debian.org

Subject: Re: Bug#598438: coreutils: new du corrupted filesystem warning breaks scripts

Date: Thu, 30 Sep 2010 14:40:09 +0200

Graham Cobb wrote:
> Package: coreutils
> Version: 8.5-1
> Severity: important
>
> Like many systems, my filesystem contains loops due to bind mounts (in my case
> it is for scratchbox, the reporter of bug 563254 has a different reason, I
> have had other reasons in the past as well).
>
> I use a backup script called dt-s3-backup.sh, which has been working for some
> time but has suddenly broken due to the change in du's handling of filesystem
> loops.
>
> Although I exclude the looped files from my backup, the script uses a du
> command which crosses the filesystem loop. Previously, du silently ignored the
> files below the loop -- the script depends on this behaviour and breaks
> completely now that du returns an error.
>
> I know about the filesystem loop.  It in no way represents any filesystem
> corruption and the change in behaviour should not have been made without some
> way to disable it.  I do not object to the new behaviour being the default but
> there needs to be an option to disable the error and silently ignore the
> loop.
>
> A simple test case to reproduce this problem:
>
> mkdir -p /tmp/du-test/loop
> echo hello >/tmp/du-test/a
> mount -o bind /tmp /tmp/du-test/loop
> du -s /tmp
> umount /tmp/du-test/loop
>
> Note: there is no way to avoid this problem using the --exclude option as
> excluding the directory does not stop du reporting the error.  This can be seen
> by replacing the du command above with:
>
> du --exclude /tmp/du-test/loop -s /tmp
>
> It is possible to use "--exclude=/tmp/du-test" but that excludes too much --
> the du command needs to count files such as /tmp/du-test/a.

Thank you for the report and reproducer.
That is a bug in how du's --exclude works in coreutils-8.1 through 8.5.
It was fixed for the next release via this change by Paul Eggert:

    du: tune, and fix some -L bugs with dangling or cyclic symlinks
    http://git.savannah.gnu.org/cgit/coreutils.git/commit/?id=77428214f13e1

However, this particular fix was not noted in NEWS,
so I'm adding an entry for it:

From 65b50c6cdd4a140a2974907423a1df1692b6d3ae Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Thu, 30 Sep 2010 14:24:42 +0200
Subject: [PATCH] maint: mention the du-exclude--vs--cycle-dir fix

* NEWS (Bug fixes): Mention the du-exclude--vs--cycle-dir fix.
Reported by Graham Cobb in http://bugs.debian.org/598438,
that bug was fixed by the 2010-07-24 commit, 77428214f,
"du: tune, and fix some -L bugs with dangling or cyclic symlinks"
---
 NEWS |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/NEWS b/NEWS
index 898ee09..11a8b74 100644
--- a/NEWS
+++ b/NEWS
@@ -11,6 +11,10 @@ GNU coreutils NEWS                                    -*- outline -*-
   du -H and -L now consistently count pointed-to files instead of
   symbolic links, and correctly diagnose dangling symlinks.

+  du --ignore=D now ignores directory D even when that directory is
+  found to be part of a directory cycle.  Before, du would issue a
+  "NOTIFY YOUR SYSTEM MANAGER" diagnostic and fail.
+
   tac would perform a double-free when given an input line longer than 16KiB.
   [bug introduced in coreutils-8.3]

--
1.7.3.293.gca9a76

Message #17 received at 598438-close@bugs.debian.org (full text, mbox, reply):

From: Michael Stone <mstone@debian.org>

To: 598438-close@bugs.debian.org

Subject: Bug#598438: fixed in coreutils 8.13-1

Date: Mon, 12 Sep 2011 22:17:18 +0000

Source: coreutils
Source-Version: 8.13-1

We believe that the bug you reported is fixed in the latest version of
coreutils, which is due to be installed in the Debian FTP archive:

coreutils_8.13-1.diff.gz
  to main/c/coreutils/coreutils_8.13-1.diff.gz
coreutils_8.13-1.dsc
  to main/c/coreutils/coreutils_8.13-1.dsc
coreutils_8.13-1_amd64.deb
  to main/c/coreutils/coreutils_8.13-1_amd64.deb
coreutils_8.13.orig.tar.gz
  to main/c/coreutils/coreutils_8.13.orig.tar.gz
mktemp_8.13-1_all.deb
  to main/c/coreutils/mktemp_8.13-1_all.deb



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 598438@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Michael Stone <mstone@debian.org> (supplier of updated coreutils package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Mon, 12 Sep 2011 17:21:19 -0400
Source: coreutils
Binary: coreutils mktemp
Architecture: source all amd64
Version: 8.13-1
Distribution: unstable
Urgency: low
Maintainer: Michael Stone <mstone@debian.org>
Changed-By: Michael Stone <mstone@debian.org>
Description: 
 coreutils  - GNU core utilities
 mktemp     - coreutils mktemp transitional package
Closes: 115833 395430 545347 584837 585509 595019 596262 598438 622182 630735
Changes: 
 coreutils (8.13-1) unstable; urgency=low
 .
   * New upstream version
     - no diagnostic when tail -f from pipe (Closes: #622182)
     - du ignores specified dir when part of cycle (Closes: #598438)
     - corrects translation errors (Closes: #595019, #545347)
     - fixes missing prototype which caused FTBFS in some cases (Closes: #585509)
     - documentation updates (Closes: #395430, #115833, #545347)
   * preserve upstream man pages in clean (Closes: #630735)
   * respect CFLAGS in build (Closes: #596262)
   * remove LC_TIME symlinks (Closes: #584837)
Checksums-Sha1: 
 899b8ab3bd9e5ea164ee440064f5599871af018e 1791 coreutils_8.13-1.dsc
 83b7e25661c439ecac55e99ff0dd816b9ff478a5 11660808 coreutils_8.13.orig.tar.gz
 6c609b750f463404b371ae6ae62418f20535ce0c 21955 coreutils_8.13-1.diff.gz
 b876538681247a0c9714e4de100a589f67ee0746 15018 mktemp_8.13-1_all.deb
 ee3b28559c1aa1e5d4147f15d7b2c3f3d1bb2bf8 5523430 coreutils_8.13-1_amd64.deb
Checksums-Sha256: 
 b95037b922265b83aba5df3a93d1e15feca1d34f5e22f26ac05bf9c0a035a3c5 1791 coreutils_8.13-1.dsc
 06180e0116b8106b2e75d8eb43d11cd0f365418a61d946bce109944902de63b3 11660808 coreutils_8.13.orig.tar.gz
 27f162d3fc52241764fef2caa87c92819a0bb34f960abc730eba46c163b92520 21955 coreutils_8.13-1.diff.gz
 f35956d5375e156b18cee47872b0df6c210e4418d094f21aa60b1086f8cfa6b8 15018 mktemp_8.13-1_all.deb
 79ce7cc49c8a881fb02d0353032d04ee16909f64dbf6689dbefbfbafe731f9fe 5523430 coreutils_8.13-1_amd64.deb
Files: 
 37dbf54b9d8dae410c6e271e9d210835 1791 utils required coreutils_8.13-1.dsc
 f5e8bb4752ee2e876ddd99bda7471f35 11660808 utils required coreutils_8.13.orig.tar.gz
 81c70fccb4fb841b25e2e61982853adb 21955 utils required coreutils_8.13-1.diff.gz
 4664342a9df3abc446c59f9e3392982d 15018 oldlibs extra mktemp_8.13-1_all.deb
 27e3b3fa6d22495c443c90a68cdfbe81 5523430 utils required coreutils_8.13-1_amd64.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQIVAwUBTm6CVfYarNz6Ef/eAQhHFQ//dJa0KJGUfLPOebtuoVsE8E8LROFWGp28
5MQqnQ1OSF9bfjH5l+L+WmSc43/jsrRoGWflAblMPWhIGa4Fixfu+4sTleDopUk7
+hM0W5pRMplKTumtGhgcbMFSsySDCL4M9dUlpYIHEeqwN7AhLGTSfOscgnU2rS6o
qTl91zcQOAiG22UssUNqnhTzJxI8EN91zFKGdPUqVmw84ZhnqP/KYGfR4OJBldT9
wnbl4F69kaJPtpBu5WsPseMMazY3lka6H53soZSiZdvFaexqYr8/E4UBGNBXhzv4
8jnNIdlfrrnHzy5PwaFYkt+gXYAa+L+kTHAeWVTrg3bzqbSB6+2UoGVN9/o5UmoX
0e0c5NfO0unuoyThxTGiITSeaM5GVqaQBydDW8i6g4SGg+u3/qiG46J/B0DS4Naz
piXaCQAmvVEF4w0ANx3DIPS/2CFvbwlR6N02OkoQpeDfxfcTfnegG/8uIBFQbQxO
KdPczV0wsZU6V46fO87CDamQbjXC90vW5quiLK7bA8mH6pBz2u2zzdSixu4kPnUQ
LNugPvZaTiwZxZKqf03gG6UZETHt0s7LY0kzU1YlsrVNxDbBwnwB0lTKCjc5kzcL
b5sdAR1m8kiQ/c5QPs9oSq8yXukHY3nEoh4DUMqlcM0rg83Y0VElzGN4nHC9nYcD
EemT31vcDOs=
=rfS6
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.