Debian Bug report logs - #525818
tar: closes file stream before real EOF

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: tar: closes file stream before real EOF

Date: Mon, 27 Apr 2009 12:02:04 +0200

[Message part 1 (text/plain, inline)]

Package: tar
Version: 1.22-1
Severity: normal
Tags: patch

Hello,

when listing the contents of module-init-tools-3.5.tar.bz2[1] as follows
(redirecting to /dev/null to keep the report nice and small):

	$ bzcat module-init-tools-3.5.tar.bz2 | tar t > /dev/null
	tar: Record size = 8 blocks

tar's stdin is closed before the stream ends:

	$ echo ${PIPESTATUS[@]}
	141 0

(I think PIPESTATUS is a bashism.)  141 = 128 + SIGPIPE

This upsets our build system, which is a bit picky about such errors.

Some debugging details:

	- module-init-tools-3.5.tar has a length of 0x16d000
	- the last 0x2b43 bytes are zeros only
	- bzcat is killed by SIGPIPE at offset 0x16b000
	- stracing bzcat ends with:

		write(1, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = -1 EPIPE (Broken pipe)
		--- SIGPIPE (Broken pipe) @ 0 (0) ---
		+++ killed by SIGPIPE +++
	- adding -i to the options of tar makes the problem go away

This is a regression from fixing http://bugs.debian.org/235820 plus (I
think) the upstream change described in http://bugs.debian.org/522858.
In tar-1.20 this doesn't show up.

Reverting the diff for src/list.c makes this problem go away.  It
doesn't fix #522858, though.

A suggested patch is attached.

Best regards
Uwe

PS: I wonder about the output to stderr above
("tar: Record size = 8 blocks").  Is this intended?

[1] available at
http://www.kernel.org/pub/linux/utils/kernel/module-init-tools/module-init-tools-3.5.tar.bz2

-- System Information:
Debian Release: 5.0.1
  APT prefers proposed-updates
  APT policy: (900, 'proposed-updates'), (900, 'stable'), (600, 'testing-proposed-updates'), (600, 'testing'), (500, 'oldstable-proposed-updates'), (500, 'oldstable'), (200, 'unstable'), (2, 'experimental')
Architecture: i386 (i686)

Kernel: Linux 2.6.29-1-686 (SMP w/1 CPU core)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash

Versions of packages tar depends on:
ii  libc6                         2.9-7      GNU C Library: Shared libraries

tar recommends no packages.

Versions of packages tar suggests:
ii  bzip2                         1.0.5-1    high-quality block-sorting file co
ii  ncompress                     4.2.4.2-1  Original Lempel-Ziv compress/uncom

-- no debconf information

[tar.diff (text/x-diff, attachment)]

Message #10 received at 525818@bugs.debian.org (full text, mbox, reply):

From: Will Dyson <will.dyson@gmail.com>

To: Debian Bug Tracking System <525818@bugs.debian.org>

Subject: tar: Also causes trouble for openwrt buildroot

Date: Sat, 09 May 2009 16:24:33 -0400

[Message part 1 (text/plain, inline)]

Package: tar
Version: 1.22-1
Severity: normal


I'm also seeing the effects of this bug when using the OpenWrt build system.

This is my test-case (tst.tar.gz attached):

ROUNDS=100
TESTFILE=./tst.tar.gz
TESTDIR=TestDir

for i in $(seq $ROUNDS); do
	mkdir -p $TESTDIR
	tar -xzOf $TESTFILE ./data.tar.gz | tar -C $TESTDIR -xzf -
	if [ $? -ne 0 ]; then
		echo FAIL!!! $i
	fi
	rm -r $TESTDIR
	echo $i
done


It fails about 5% of the time, on my system.

-- System Information:
Debian Release: squeeze/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (1, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.29-2-amd64 (SMP w/2 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8) (ignored: LC_ALL set to en_US.UTF-8)
Shell: /bin/sh linked to /bin/bash

Versions of packages tar depends on:
ii  libc6                         2.9-12     GNU C Library: Shared libraries

tar recommends no packages.

Versions of packages tar suggests:
ii  bzip2                         1.0.5-1    high-quality block-sorting file co
ii  ncompress                     4.2.4.2-2  Original Lempel-Ziv compress/uncom

-- no debconf information

[tst.tar.gz (application/x-gzip, attachment)]

Message #15 received at 525818@bugs.debian.org (full text, mbox, reply):

From: Carl Worth <cworth@cworth.org>

To: 525818@bugs.debian.org

Subject: Re: tar: closes file stream before real EOF

Date: Wed, 29 Jul 2009 16:24:56 -0700

[Message part 1 (text/plain, inline)]

On Mon, 27 Apr 2009 12:02:04 +0200 Uwe Kleine-König <u.kleine-koenig@pengutronix.de> wrote:
> This is a regression from fixing http://bugs.debian.org/235820 plus (I
> think) the upstream change described in http://bugs.debian.org/522858.
> In tar-1.20 this doesn't show up.
> 
> Reverting the diff for src/list.c makes this problem go away.  It
> doesn't fix #522858, though.

Fortunately, #522858 has since been fixed independently by a change in
dpkg, so we shouldn't have to worry about that at all. Meanwhile, the
proposal to revert the diff for src/list.c would undo the fix for
#235820 (which was to simply avoid printing a confusing warning when tar
was actually successful).

I think we can resolve the current bug without undoing the fix for
#235820 by reading the header, but remaining silent about any lone zero
block.

Please examine the patch below and let me know if you find it
acceptable. (The patch moves the #if 0 down to exclude only the printing
of the warning, without otherwise changing tar's behavior. The diff
doesn't make that as obvious as one might like.)

-Carl

commit 37efd16782afb35738fcd67c85f134fef2cc409f
Author: Carl Worth <cworth@cworth.org>
Date:   Wed Jul 29 16:10:01 2009 -0700

    Don't close file stream before EOF, closes #525818
    
    To fix bug #235820 src/list.c was modified to avoid printing
    a confusing warning message about a lone zero block. However,
    the change also caused the input stream to be closed before
    reading EOF which could cause a SIGPIPE to be sent to the
    writing process.
    
    In this change, the previous fix is made more narrow to exclude
    only the printing of the warning message, but without the side
    effect that was causing the SIGPIPE.

diff --git a/src/list.c b/src/list.c
index c650172..126a3c1 100644
--- a/src/list.c
+++ b/src/list.c
@@ -136,7 +136,11 @@ read_and (void (*do_something) (void))
 
          if (!ignore_zeros_option)
            {
+             char buf[UINTMAX_STRSIZE_BOUND];
 
+             status = read_header (false);
+             if (status == HEADER_ZERO_BLOCK)
+               break;
              /* 
               * According to POSIX tar specs, this is wrong, but on the web
               * there are some tar specs that can trigger this, and some tar
@@ -144,11 +148,6 @@ read_and (void (*do_something) (void))
               * let's not be pedantic about issuing the warning.
               */
 #if 0         
-             char buf[UINTMAX_STRSIZE_BOUND];
-
-             status = read_header (false);
-             if (status == HEADER_ZERO_BLOCK)
-               break;
              WARN ((0, 0, _("A lone zero block at %s"),
                     STRINGIFY_BIGINT (current_block_ordinal (), buf)));
 #endif

[signature.asc (application/pgp-signature, inline)]

Message #24 received at 525818-close@bugs.debian.org (full text, mbox, reply):

From: Carl Worth <cworth@cworth.org>

To: 525818-close@bugs.debian.org

Subject: Bug#525818: fixed in tar 1.22-2

Date: Tue, 04 Aug 2009 22:08:17 +0000

Source: tar
Source-Version: 1.22-2

We believe that the bug you reported is fixed in the latest version of
tar, which is due to be installed in the Debian FTP archive:

tar_1.22-2.diff.gz
  to pool/main/t/tar/tar_1.22-2.diff.gz
tar_1.22-2.dsc
  to pool/main/t/tar/tar_1.22-2.dsc
tar_1.22-2_i386.deb
  to pool/main/t/tar/tar_1.22-2_i386.deb



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 525818@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Carl Worth <cworth@cworth.org> (supplier of updated tar package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.8
Date: Tue, 04 Aug 2009 12:07:06 -0700
Source: tar
Binary: tar
Architecture: source i386
Version: 1.22-2
Distribution: unstable
Urgency: low
Maintainer: Bdale Garbee <bdale@gag.com>
Changed-By: Carl Worth <cworth@cworth.org>
Description: 
 tar        - GNU version of the tar archiving utility
Closes: 188663 525818 535319
Changes: 
 tar (1.22-2) unstable; urgency=low
 .
   * Add Carl Worth as an uploader.
   * Fix to allow parallel build (-j2), closes: #535319
   * Don't close file stream before EOF, closes: #525818
   * Preserve hard links with --remove-files, closes: #188663
     Thanks to Ted T'so for the idea and Sergey Poznyakoff for
     cleaning up my original implementation.
   * Respect DEB_BUILD_OPTIONS=nocheck to conform with Policy 3.8.2
Checksums-Sha1: 
 24163c1e052e310c51232fe1f202b40cb3a318eb 971 tar_1.22-2.dsc
 08f40955a77a640cd5bba57a823a29dc5e8a1562 18584 tar_1.22-2.diff.gz
 642d961d9c98aaa859d697511dccbc8cae5ee292 830362 tar_1.22-2_i386.deb
Checksums-Sha256: 
 13e010e1dc39a3a614a1e4a63ee3621e28b110667271edead4f2b4d5944efddc 971 tar_1.22-2.dsc
 ab26300424c280edf1d5445c97672cc5fd7242a3175b6973967e72bf7d422c04 18584 tar_1.22-2.diff.gz
 07f0552d6152d5523ba523f2394972d8c37046404552c1839319c1b36e2edf30 830362 tar_1.22-2_i386.deb
Files: 
 11ef8d4959601dfeb53dce2c7a25a075 971 utils required tar_1.22-2.dsc
 b0a4a1bd3a4f4ce6005f374e27ea2cd9 18584 utils required tar_1.22-2.diff.gz
 97e9420c3d36ad7f811d88f931df49ec 830362 utils required tar_1.22-2_i386.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iD8DBQFKeJY0ZKfAp/LPAagRApbGAJ4rCdh5yctfRdumBa3W6ZhUliIaZQCdF6W2
rcohHLToC7o4HWTMY4Rocow=
=q2QU
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.