Debian Bug report logs - #184057
gzip: crash on certain file on Alpha

version graph

Package: gzip; Maintainer for gzip is Bdale Garbee <bdale@gag.com>; Source for gzip is src:gzip.

Reported by: Jochen Friedrich <jochen@scram.de>

Date: Sun, 9 Mar 2003 10:33:04 UTC

Severity: serious

Merged with 187417

Found in version 1.3.5-4

Fixed in version gzip/1.3.5-5

Done: Bdale Garbee <bdale@gag.com>

Bug is archived. No further changes may be made.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, Bdale Garbee <bdale@gag.com>, gzip@packages.qa.debian.org:
Bug#184057; Package gzip. Full text and rfc822 format available.

Acknowledgement sent to Jochen Friedrich <jochen@scram.de>:
New Bug report received and forwarded. Copy sent to Bdale Garbee <bdale@gag.com>, gzip@packages.qa.debian.org. Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: Jochen Friedrich <jochen@scram.de>
To: submit@bugs.debian.org
Subject: gzip: crash on certain file on Alpha
Date: Sun, 9 Mar 2003 11:20:43 +0100 (CET)
Subject: gzip: crash on certain file on Alpha
Package: gzip
Version: 1.3.5-4
Severity: normal

File X can be found on http://jochen.scram.de/X

Starting program: /home/pbuilder/gzip-1.3.5/gzip X

Program received signal SIGSEGV, Segmentation fault.
0x00000001200022dc in longest_match (cur_match=65485) at deflate.c:468
468             do {
(gdb) bt
#0  0x00000001200022dc in longest_match (cur_match=65485) at deflate.c:468
#1  0x000000012000325c in deflate () at deflate.c:745
#2  0x000000012000fd0c in zip (in=65485, out=8) at zip.c:76
#3  0x00000001200047f0 in treat_file (iname=0x120023b60 "\a") at gzip.c:870
#4  0x0000000120003bac in main (argc=8, argv=0x120023af8) at gzip.c:632

-- System Information:
Debian Release: testing/unstable
Architecture: alpha
Kernel: Linux ayse.bocc.de 2.5.56 #19 Sun Jan 12 00:07:39 CET 2003 alpha
Locale: LANG=C, LC_CTYPE=C

Versions of packages gzip depends on:
ii  debianutils                   2.4        Miscellaneous utilities specific t
ii  libc6.1                       2.3.1-14   GNU C Library: Shared libraries an

-- no debconf information




Information forwarded to debian-bugs-dist@lists.debian.org, Bdale Garbee <bdale@gag.com>, gzip@packages.qa.debian.org:
Bug#184057; Package gzip. Full text and rfc822 format available.

Acknowledgement sent to Falk Hueffner <falk.hueffner@student.uni-tuebingen.de>:
Extra info received and forwarded to list. Copy sent to Bdale Garbee <bdale@gag.com>, gzip@packages.qa.debian.org. Full text and rfc822 format available.

Message #10 received at 184057@bugs.debian.org (full text, mbox):

From: Falk Hueffner <falk.hueffner@student.uni-tuebingen.de>
To: 184057@bugs.debian.org
Subject: Re: gzip: crash on certain file on Alpha
Date: 11 Mar 2003 11:42:09 +0100
This is cause by insufficient lookahead, as can be seen when compiling
with -DDEBUG.

last_lit 4096, last_dist 2393, in 28529, out ~6845(77%) 
last_lit 8192, last_dist 3668, in 59294, out ~11995(80%) 
opt 9946(79561) stat 11906(95238) stored 59294 lit 8192 dist 3668 
gzip: stdin: insufficient lookahead

Please forward this upstream. This bug is pretty nasty IMHO, since it
could lead to silent data corruption, and also it keeps X from
building.

BTW, it seems like a good idea to enable the lookahead assertion
unconditionally, since it is pretty cheap.

-- 
	Falk



Information forwarded to debian-bugs-dist@lists.debian.org, Bdale Garbee <bdale@gag.com>, gzip@packages.qa.debian.org:
Bug#184057; Package gzip. Full text and rfc822 format available.

Acknowledgement sent to Falk Hueffner <falk.hueffner@student.uni-tuebingen.de>:
Extra info received and forwarded to list. Copy sent to Bdale Garbee <bdale@gag.com>, gzip@packages.qa.debian.org. Full text and rfc822 format available.

Message #15 received at 184057@bugs.debian.org (full text, mbox):

From: Falk Hueffner <falk.hueffner@student.uni-tuebingen.de>
To: bug-gzip@gnu.org
Cc: 184057@bugs.debian.org
Subject: gzip: crash on certain file on Alpha
Date: 12 Mar 2003 11:03:30 +0100
Hi,

please have a look at http://bugs.debian.org/184057. A certain file
makes gzip crash on Alpha because of insufficient lookahead. This does
not occur with 1.2.4, but with 1.3.5.

Please keep 184057@bugs.debian.org in the Cc when replying.

-- 
	Falk



Information forwarded to debian-bugs-dist@lists.debian.org, Bdale Garbee <bdale@gag.com>, gzip@packages.qa.debian.org:
Bug#184057; Package gzip. Full text and rfc822 format available.

Acknowledgement sent to Anthony Towns <aj@azure.humbug.org.au>:
Extra info received and forwarded to list. Copy sent to Bdale Garbee <bdale@gag.com>, gzip@packages.qa.debian.org. Full text and rfc822 format available.

Message #20 received at 184057@bugs.debian.org (full text, mbox):

From: Anthony Towns <aj@azure.humbug.org.au>
To: control@bugs.debian.org, 187417@bugs.debian.org, 184057@bugs.debian.org
Cc: Falk Hueffner <falk.hueffner@student.uni-tuebingen.de>, Daniel Stone <dstone@trinity.unimelb.edu.au>, debian-alpha@lists.debian.org, debian-x@lists.debian.org
Subject: Re: xfree86 4.2.1-6 build on alpha
Date: Thu, 3 Apr 2003 22:18:26 +1000
severity 184057 serious
severity 187417 serious
merge 187417 184057
thanks

On Thu, Apr 03, 2003 at 11:07:16AM +0200, Falk Hueffner wrote:
> > Ah, the gzip killer bug. Works with -3, fails with -4 - where the
> > --rsyncable patch was introduced. However, StevenK claimed he couldn't
> > reproduce it, so I didn't file a bug about it.
> > Bug needs to be filed on gzip about the 'gzip killer' .bdf.
> Already reported 3 weeks ago as #184057.

So, here's the deal. On alpha, this bug is reproducible when compiled
with gcc-3.2 at any optimisation, but not reproducible with gcc-2.95
at -O2. When -DDEBUG is enabled, the assertion is triggered on alpha
with gcc-2.95 and gcc-3.2. With -DDEBUG enabled, the assertion is also
triggered on powerpc.

The problem appears to be that the checks in deflate_fast() and deflate(),
namely:

        if (hash_head != NIL && strstart - hash_head <= MAX_DIST) {

and

        if (hash_head != NIL && prev_length < max_lazy_match &&
            strstart - hash_head <= MAX_DIST) {

preceeding calls to longest_match() do not actually ensure that the assertion:

    Assert(strstart <= window_size-MIN_LOOKAHEAD, "insufficient lookahead");

in longest_match() actually passes. I don't really understand what's going
on exactly, but the thoughtless solution of adding the extra check from
the assertion explicitly seems to work (and, afaics, should work).

The two tests (strstart <= window_size - MIN_LOOKAHEAD, and strstart -
hash_head <= MAX_DIST) are equivalent when hash_head > WSIZE, but there's
no particular reason for that to be true, that I can see. I don't think this
can result in corrupted data, and while a buffer is overflown I think it's
only by reading, so apart from the segfault I don't _think_ there are any
problems caused by this bug. I'm not really sure though.

The patch looks like:

--- gzip-1.3.5/deflate.c        2003-04-03 21:51:36.000000000 +1000
+++ gzip-1.3.5-aj/deflate.c     2003-04-03 21:56:38.000000000 +1000
@@ -643,7 +643,8 @@
         /* Find the longest match, discarding those <= prev_length.
          * At this point we have always match_length < MIN_MATCH
          */
-        if (hash_head != NIL && strstart - hash_head <= MAX_DIST) {
+        if (hash_head != NIL && strstart - hash_head <= MAX_DIST &&
+            strstart <= window_size - MIN_LOOKAHEAD) {
             /* To simplify the code, we prevent matches with the string
              * of window index 0 (in particular we have to avoid a match
              * of the string with itself at the start of the input file).
@@ -737,7 +738,8 @@
         match_length = MIN_MATCH-1;
 
         if (hash_head != NIL && prev_length < max_lazy_match &&
-            strstart - hash_head <= MAX_DIST) {
+            strstart - hash_head <= MAX_DIST && 
+            strstart <= window_size - MIN_LOOKAHEAD) {
             /* To simplify the code, we prevent matches with the string
              * of window index 0 (in particular we have to avoid a match
              * of the string with itself at the start of the input file).

As it stands, this patch decreases the effectiveness of gzip's deflate
implementation by, I guess, up to 258 bytes per file. For comparison:

$ echo `gzip <X | wc -c` `gzip <X | md5sum | cut -d\  -f1`
11912 4d02d6c8ee27d64a2cb773ad5cf9a086
$ echo `./gzip <X | wc -c` `./gzip <X | md5sum | cut -d\  -f1`
11981 499f30e71926490dd5404f9d38efeacd

Both files decompresses correctly, of course.

For files not affected by this bug, the output is exactly the same:

$ echo `gzip </etc/motd | wc -c` `gzip </etc/motd | md5sum | cut -d\  -f1`
277 e73b4f30720cf2a29c2b774078237ea4
$ echo `./gzip </etc/motd | wc -c` `./gzip </etc/motd | md5sum | cut -d\  -f1`
277 e73b4f30720cf2a29c2b774078237ea4

Note that i386 has an assembly implementation of longest_match() which
may or may not have the bug.

Cheers,
aj

-- 
Anthony Towns <aj@humbug.org.au> <http://azure.humbug.org.au/~aj/>
I don't speak for anyone save myself. GPG signed mail preferred.

  ``Dear Anthony Towns: [...] Congratulations -- 
        you are now certified as a Red Hat Certified Engineer!''



Information forwarded to debian-bugs-dist@lists.debian.org, Bdale Garbee <bdale@gag.com>, gzip@packages.qa.debian.org:
Bug#184057; Package gzip. Full text and rfc822 format available.

Acknowledgement sent to Anthony Towns <aj@azure.humbug.org.au>:
Extra info received and forwarded to list. Copy sent to Bdale Garbee <bdale@gag.com>, gzip@packages.qa.debian.org. Full text and rfc822 format available.

Message #25 received at 184057@bugs.debian.org (full text, mbox):

From: Anthony Towns <aj@azure.humbug.org.au>
To: 184057@bugs.debian.org
Subject: gzip/alpha bug
Date: Thu, 3 Apr 2003 22:30:26 +1000
Oh, I notice Falk said:

> please have a look at http://bugs.debian.org/184057. A certain file
> makes gzip crash on Alpha because of insufficient lookahead. This does
> not occur with 1.2.4, but with 1.3.5.

I can reproduce this on powerpc with 1.2.4, compiling with -DDEBUG:

] $ ./gzip --version
] gzip 1.2.4 (18 Aug 93)
] Compilation options:
] DIRENT UTIME STDC_HEADERS HAVE_UNISTD_H DEBUG 

] $ ./gzip < ../../X -f
] 
] last_lit 4096, last_dist 2393, in 28529, out ~6845(77%) 
] last_lit 8192, last_dist 3668, in 59294, out ~11995(80%) 
] opt 9946(79561) stat 11906(95238) stored 59294 lit 8192 dist 3668 
] gzip: stdin: insufficient lookahead

Cheers,
aj

-- 
Anthony Towns <aj@humbug.org.au> <http://azure.humbug.org.au/~aj/>
I don't speak for anyone save myself. GPG signed mail preferred.

  ``Dear Anthony Towns: [...] Congratulations -- 
        you are now certified as a Red Hat Certified Engineer!''



Information forwarded to debian-bugs-dist@lists.debian.org, Bdale Garbee <bdale@gag.com>, gzip@packages.qa.debian.org:
Bug#184057; Package gzip. Full text and rfc822 format available.

Acknowledgement sent to Anthony Towns <aj@azure.humbug.org.au>:
Extra info received and forwarded to list. Copy sent to Bdale Garbee <bdale@gag.com>, gzip@packages.qa.debian.org. Full text and rfc822 format available.

Message #30 received at 184057@bugs.debian.org (full text, mbox):

From: Anthony Towns <aj@azure.humbug.org.au>
To: Falk Hueffner <falk.hueffner@student.uni-tuebingen.de>, 184057@bugs.debian.org, bug-gzip@gnu.org
Subject: gzip/alpha crash due to insufficient lookahead
Date: Thu, 3 Apr 2003 22:32:41 +1000
(repeat, this time cc'ing people)

I notice Falk said:

> please have a look at http://bugs.debian.org/184057. A certain file
> makes gzip crash on Alpha because of insufficient lookahead. This does
> not occur with 1.2.4, but with 1.3.5.

I can reproduce this on powerpc with 1.2.4, compiling with -DDEBUG:

] $ ./gzip --version
] gzip 1.2.4 (18 Aug 93)
] Compilation options:
] DIRENT UTIME STDC_HEADERS HAVE_UNISTD_H DEBUG 

] $ ./gzip < ../../X -f
] 
] last_lit 4096, last_dist 2393, in 28529, out ~6845(77%) 
] last_lit 8192, last_dist 3668, in 59294, out ~11995(80%) 
] opt 9946(79561) stat 11906(95238) stored 59294 lit 8192 dist 3668 
] gzip: stdin: insufficient lookahead

There's a diagnosis and a patch in http://bugs.debian.org/184057 now (or
there will be in a few minutes when it gets my mail), fwiw.

Cheers,
aj

-- 
Anthony Towns <aj@humbug.org.au> <http://azure.humbug.org.au/~aj/>
I don't speak for anyone save myself. GPG signed mail preferred.

  ``Dear Anthony Towns: [...] Congratulations -- 
        you are now certified as a Red Hat Certified Engineer!''



Severity set to `serious'. Request was from Anthony Towns <aj@azure.humbug.org.au> to control@bugs.debian.org. Full text and rfc822 format available.

Merged 184057 187417. Request was from Anthony Towns <aj@azure.humbug.org.au> to control@bugs.debian.org. Full text and rfc822 format available.

Reply sent to Bdale Garbee <bdale@gag.com>:
You have taken responsibility. Full text and rfc822 format available.

Notification sent to Jochen Friedrich <jochen@scram.de>:
Bug acknowledged by developer. Full text and rfc822 format available.

Message #39 received at 184057-close@bugs.debian.org (full text, mbox):

From: Bdale Garbee <bdale@gag.com>
To: 184057-close@bugs.debian.org
Subject: Bug#184057: fixed in gzip 1.3.5-5
Date: Wed, 16 Apr 2003 14:02:07 -0400
We believe that the bug you reported is fixed in the latest version of
gzip, which is due to be installed in the Debian FTP archive:

gzip_1.3.5-5.diff.gz
  to pool/main/g/gzip/gzip_1.3.5-5.diff.gz
gzip_1.3.5-5.dsc
  to pool/main/g/gzip/gzip_1.3.5-5.dsc
gzip_1.3.5-5_ia64.deb
  to pool/main/g/gzip/gzip_1.3.5-5_ia64.deb



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 184057@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Bdale Garbee <bdale@gag.com> (supplier of updated gzip package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.7
Date: Wed, 16 Apr 2003 11:24:23 -0600
Source: gzip
Binary: gzip
Architecture: source ia64
Version: 1.3.5-5
Distribution: unstable
Urgency: low
Maintainer: Bdale Garbee <bdale@gag.com>
Changed-By: Bdale Garbee <bdale@gag.com>
Description: 
 gzip       - The GNU compression utility
Closes: 184057 187417
Changes: 
 gzip (1.3.5-5) unstable; urgency=low
 .
   * apply patch from Anthony Towns that fixes seg faults on alpha during
     build of Xfree86 at the expense of slightly decreasing the effectiveness
     of the deflate implementation.  closes: #184057, #187417
Files: 
 eebc836677d36fe39ea889fac9c3868d 584 base required gzip_1.3.5-5.dsc
 7f83e82f664da2a4b5bb95a3da66199b 9826 base required gzip_1.3.5-5.diff.gz
 3f8516034138efabdd9acb1cd8d301d0 90410 base required gzip_1.3.5-5_ia64.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQE+nZdnZKfAp/LPAagRAg69AJ99hUUUjmhBT+FMTkpLCwiyMn09vwCeOvTN
DvQfpCkXO7C2TPCqNuKRG5M=
=M92A
-----END PGP SIGNATURE-----




Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Fri Apr 18 17:04:34 2014; Machine Name: buxtehude.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.