Debian Bug report logs - #227621
perl: getgrent() crashes with "Out of memory" if /etc/group contains long lines

Display info messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Peter Paluch <peterp@frix.fri.utc.sk>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: perl: getgrnam() crashes with "Out of memory" if /etc/group contains long lines

Date: Wed, 14 Jan 2004 02:47:44 +0100

Package: perl
Version: 5.8.2-2
Severity: normal

Hello,
=-==-=

There is a problem with the getgrent() function if the /etc/group file
contains at least one line longer than 4088 characters, including the
newline character. In this case, the script execution will fail with
a "Out of memory!" message.

It is very easy to demonstrate it, just create a line long enough in the
/etc/group, and then execute this simple script:

#!/usr/bin/perl

while (my @gr=getgrent()) {
  print $gr[0] . "\n"
}
print "End\n"

The last line will not be executed, and you will get the "Out of memory!"
message.

This bug resembles another two already closed bugs, #208428 and #212541,
that were present in glibc. However, these bugs are already solved and there
is no connection between them and this bug - while the glibc's getgrent()
works perfectly fine even with overly long lines in /etc/group, the perl's
implementation does not.

I have not tried other functions similar to getgrent() but I presume that
they may suffer from this behavior as well.

This bug appears both in sarge and in sid (I have not tested the woody).

Regards,
Peter

-- System Information:
Debian Release: testing/unstable
Architecture: i386
Kernel: Linux gericom 2.4.23-xfs #1 Po dec 29 23:38:10 CET 2003 i686
Locale: LANG=sk_SK, LC_CTYPE=sk_SK

Versions of packages perl depends on:
ii  libc6                       2.3.2.ds1-10 GNU C Library: Shared libraries an
ii  libdb4.0                    4.0.14-1.3   Berkeley v4.0 Database Libraries [
ii  libgdbm3                    1.8.3-2      GNU dbm database routines (runtime
ii  perl-base                   5.8.2-2      The Pathologically Eclectic Rubbis
ii  perl-modules                5.8.2-2      Core Perl modules.

-- no debconf information

Message #12 received at 227621@bugs.debian.org (full text, mbox, reply):

From: "Steinar H. Gunderson" <sgunderson@bigfoot.com>

To: Peter Paluch <peterp@frix.fri.utc.sk>

Cc: 227621@bugs.debian.org

Subject: Re: perl: getgrnam() crashes with "Out of memory" if /etc/group contains long lines

Date: Sat, 4 Jun 2005 23:24:48 +0200

On Wed, Jan 14, 2004 at 02:47:44AM +0100, Peter Paluch wrote:
> There is a problem with the getgrent() function if the /etc/group file
> contains at least one line longer than 4088 characters, including the
> newline character. In this case, the script execution will fail with
> a "Out of memory!" message.

We were bitten by this as well, but in the form of a segfault, and with
significantly shorter lines (about 1500 characters). Shouldn't this be
upgraded to at least important?

FWIW, this happened after an upgrade from woody to sarge, so the woody
version is OK.

/* Steinar */
-- 
Homepage: http://www.sesse.net/

Message #17 received at 227621@bugs.debian.org (full text, mbox, reply):

From: Peter Palúch <peterp@frix.fri.utc.sk>

To: "Steinar H. Gunderson" <sgunderson@bigfoot.com>

Cc: Peter Paluch <peterp@frix.fri.utc.sk>, 227621@bugs.debian.org

Subject: Re: perl: getgrnam() crashes with "Out of memory" if /etc/group contains long lines

Date: Mon, 6 Jun 2005 13:10:01 +0200

[Message part 1 (text/plain, inline)]

Hello,
=-==-=

> On Wed, Jan 14, 2004 at 02:47:44AM +0100, Peter Paluch wrote:
> > There is a problem with the getgrent() function if the /etc/group file
> > contains at least one line longer than 4088 characters, including the
> > newline character. In this case, the script execution will fail with
> > a "Out of memory!" message.
> 
> We were bitten by this as well, but in the form of a segfault, and with
> significantly shorter lines (about 1500 characters). Shouldn't this be
> upgraded to at least important?

Maybe yes - it's up to maintainers. Originally I did not intend to mark this
bug as severity important as this bug appears only under not-so-common
conditions (I have been told that to have such a long line in /etc/group is
a sign of a bad system design - so many users and groups should already be
placed in a database). Nevertheless, this bug can be very nasty.

There was a very similar bug in glibc some time ago. The problem appeared
when a line in /etc/group was longer than 1023 characters - the routines in
glibc allocated more and more memory in an infinite loop until all free
memory was exhausted. It was reported as bug #208428 and I have contributed
some information to it. However, the problem in glibc has been solved since
then. You will find some detailed information in the bug report archive. The
bug was finally solved upstream in an other way that I suggested - but
surely it was a better solution.

I thought it may be possible that perl routines use some code snippets from
glibc library or maybe they use them in some curious way that makes the
problem from glibc reappear. However, I am not at all familiar with perl
internals - I just found it interesting that this bug appeared only some
time after the nearly identical bug in glibc was found.

Regards,
Peter

[signature.asc (application/pgp-signature, inline)]

Message #22 received at 227621@bugs.debian.org (full text, mbox, reply):

From: "Steinar H. Gunderson" <sgunderson@bigfoot.com>

To: Peter Palúch <peterp@frix.fri.utc.sk>

Cc: 227621@bugs.debian.org, control@bugs.debian.org

Subject: Re: perl: getgrnam() crashes with "Out of memory" if /etc/group contains long lines

Date: Thu, 9 Jun 2005 23:20:38 +0200

severity 227621 important
thanks

On Mon, Jun 06, 2005 at 01:10:01PM +0200, Peter Palúch wrote:
> Maybe yes - it's up to maintainers. Originally I did not intend to mark this
> bug as severity important as this bug appears only under not-so-common
> conditions (I have been told that to have such a long line in /etc/group is
> a sign of a bad system design - so many users and groups should already be
> placed in a database). Nevertheless, this bug can be very nasty.

We're bitten really hard by it now -- we do getgrent() in several
system-critical scripts, and we cannot really migrate away from that group
structure anytime soon, so we're stuck with trying to rewrite the scripts...

When I asked the RMs about this before the release, they judged it as
severity important, so I'm upgrading it. It still won't go into sarge AFAICS,
but hey...

/* Steinar */
-- 
Homepage: http://www.sesse.net/

Message #29 received at 227621@bugs.debian.org (full text, mbox, reply):

From: "Steinar H. Gunderson" <sgunderson@bigfoot.com>

To: Peter Palúch <peterp@frix.fri.utc.sk>

Cc: control@bugs.debian.org, 227621@bugs.debian.org, debian-security@lists.debian.org

Subject: Re: perl: getgrnam() crashes with "Out of memory" if /etc/group contains long lines

Date: Fri, 10 Jun 2005 15:03:02 +0200

tags 227621 + patch
thanks

On Mon, Jun 06, 2005 at 01:10:01PM +0200, Peter Palúch wrote:
> There was a very similar bug in glibc some time ago. The problem appeared
> when a line in /etc/group was longer than 1023 characters - the routines in
> glibc allocated more and more memory in an infinite loop until all free
> memory was exhausted. It was reported as bug #208428 and I have contributed
> some information to it. However, the problem in glibc has been solved since
> then. You will find some detailed information in the bug report archive. The
> bug was finally solved upstream in an other way that I suggested - but
> surely it was a better solution.

It's about the same bug in perl as it was in glibc. reentr.pl line 698 reads:

  $call = qq[((PL_REENTRANT_RETINT = $call)$test ? $true : (((PL_REENTRANT_RETINT == ERANGE) || (errno == ERANGE)) ?  ($seenm{$func}{$seenr{$func}})Perl_reentrant_retry("$func"$rv) : 0))];
  
The problem here is "errno == ERANGE". If, at any time, there's a line longer
than the initial buffer, getgrent() (or any in the same family) will get
ERANGE back (and errno will be set to ERANGE). However, this is never reset.
Thus, when getgrent_r() hits EOF, it returns ENOENT, _but errno is still
ERANGE_. Perl figures the buffer was too small, doubles it and tries again,
but still gets ENOENT, of course (and errno is still ERANGE). This goes on
forever and ever until you run out of memory (which happens quite fast).

The solution is simply to remove "errno == ERANGE" AFAICS; getgrent_r() does
not define what happens to errno, and the return message will always be
ERANGE if the buffer is too small.

I'm a bit tempted to tag this "security"; if a user can (say) change his or
her own GECOS field to make it long enough, Perl programs using getpwent()
will crash, for instance. I can't find any direct way to exploit it (chfn
limits the length of the fields, for instance), but I'm still slightly
concerned over the possibilities of a DoS; Cc-ing debian-security.

/* Steinar */
-- 
Homepage: http://www.sesse.net/

Message #36 received at 227621@bugs.debian.org (full text, mbox, reply):

From: Peter Gervai <grin@grin.hu>

To: 227621@bugs.debian.org

Subject: damn bug

Date: Thu, 30 Jun 2005 20:25:32 +0200

I have been hunting this for at least 3 hours now and got the same
result much harder, since my program was openwebmail, a cgi script.
Dies on a group which contains plenty of users.

I don't really see what to patch (I ain't no perl internals hacker)
but I found an ugly hack till it gets fixed (which I hope sooner than
later):


#!/usr/bin/perl

while (my @gr=getgrent()) {
  print $gr[0] . "\n"
  open(F,"</etc/passwd");close(F);
}
print "End\n"


any file will do, if it exists. Don't ask me why, you probably could
guess better than me. And don't look at me with this face... :)

Peter

Message #41 received at 227621@bugs.debian.org (full text, mbox, reply):

From: Brendan O'Dea <bod@debian.org>

To: Perl5 Porters <perl5-porters@perl.org>

Cc: 227621@bugs.debian.org

Subject: Debian bug #227621: getgrent out of memory

Date: Wed, 6 Jul 2005 01:00:20 +1000

Debian byg 227621 (http://bugs.debian.org/227621) reports OOM problems
processing /etc/group files with large lines.

The bug reporter notes that the OOM is avoided by opening a random file
within the loop:

>#!/usr/bin/perl
>
>while (my @gr=getgrent()) {
>  print $gr[0] . "\n";
>  open(F,"</etc/passwd");close(F);
>}
>print "End\n"

Any suggestions?

--bod

Message #46 received at 227621@bugs.debian.org (full text, mbox, reply):

From: Brendan O'Dea <bod@debian.org>

To: Peter Gervai <grin@grin.hu>, 227621@bugs.debian.org

Subject: Re: Bug#227621: damn bug

Date: Wed, 6 Jul 2005 01:09:01 +1000

On Thu, Jun 30, 2005 at 08:25:32PM +0200, Peter Gervai wrote:
>#!/usr/bin/perl
>
>while (my @gr=getgrent()) {
>  print $gr[0] . "\n"
>  open(F,"</etc/passwd");close(F);
>}
>print "End\n"
>
>
>any file will do, if it exists. Don't ask me why, you probably could
>guess better than me. And don't look at me with this face... :)

Nothing that I can see, bug forwarded upstream.

--bod

Message #51 received at 227621@bugs.debian.org (full text, mbox, reply):

From: Rafael Garcia-Suarez <rgarciasuarez@gmail.com>

To: Perl5 Porters <perl5-porters@perl.org>, 227621@bugs.debian.org

Subject: Re: Debian bug #227621: getgrent out of memory

Date: Tue, 5 Jul 2005 18:46:51 +0200

On 7/5/05, Brendan O'Dea <bod@debian.org> wrote:
> Debian byg 227621 (http://bugs.debian.org/227621) reports OOM problems
> processing /etc/group files with large lines.

I can't find this in the change log now, but I note in perl590delta :

For threaded Perls certain system database functions like getpwent()
and getgrent() now grow their result buffer dynamically, instead of
failing.  This means that at sites with lots of users and groups the
functions no longer fail by returning only partial results.

So I'd suggest to try to reproduce with bleadperl; if fixed in blead,
find and backport the fix. (should be straightforward :)

Message #58 received at 227621-close@bugs.debian.org (full text, mbox, reply):

From: Brendan O'Dea <bod@debian.org>

To: 227621-close@bugs.debian.org

Subject: Bug#227621: fixed in perl 5.8.7-4

Date: Fri, 08 Jul 2005 23:32:18 -0400

Source: perl
Source-Version: 5.8.7-4

We believe that the bug you reported is fixed in the latest version of
perl, which is due to be installed in the Debian FTP archive:

libcgi-fast-perl_5.8.7-4_all.deb
  to pool/main/p/perl/libcgi-fast-perl_5.8.7-4_all.deb
libperl-dev_5.8.7-4_i386.deb
  to pool/main/p/perl/libperl-dev_5.8.7-4_i386.deb
libperl-dev_5.8.7-4_powerpc.deb
  to pool/main/p/perl/libperl-dev_5.8.7-4_powerpc.deb
libperl-dev_5.8.7-4_sparc.deb
  to pool/main/p/perl/libperl-dev_5.8.7-4_sparc.deb
libperl5.8_5.8.7-4_i386.deb
  to pool/main/p/perl/libperl5.8_5.8.7-4_i386.deb
libperl5.8_5.8.7-4_powerpc.deb
  to pool/main/p/perl/libperl5.8_5.8.7-4_powerpc.deb
libperl5.8_5.8.7-4_sparc.deb
  to pool/main/p/perl/libperl5.8_5.8.7-4_sparc.deb
perl-base_5.8.7-4_i386.deb
  to pool/main/p/perl/perl-base_5.8.7-4_i386.deb
perl-base_5.8.7-4_powerpc.deb
  to pool/main/p/perl/perl-base_5.8.7-4_powerpc.deb
perl-base_5.8.7-4_sparc.deb
  to pool/main/p/perl/perl-base_5.8.7-4_sparc.deb
perl-debug_5.8.7-4_i386.deb
  to pool/main/p/perl/perl-debug_5.8.7-4_i386.deb
perl-debug_5.8.7-4_powerpc.deb
  to pool/main/p/perl/perl-debug_5.8.7-4_powerpc.deb
perl-debug_5.8.7-4_sparc.deb
  to pool/main/p/perl/perl-debug_5.8.7-4_sparc.deb
perl-doc_5.8.7-4_all.deb
  to pool/main/p/perl/perl-doc_5.8.7-4_all.deb
perl-modules_5.8.7-4_all.deb
  to pool/main/p/perl/perl-modules_5.8.7-4_all.deb
perl-suid_5.8.7-4_i386.deb
  to pool/main/p/perl/perl-suid_5.8.7-4_i386.deb
perl-suid_5.8.7-4_powerpc.deb
  to pool/main/p/perl/perl-suid_5.8.7-4_powerpc.deb
perl-suid_5.8.7-4_sparc.deb
  to pool/main/p/perl/perl-suid_5.8.7-4_sparc.deb
perl_5.8.7-4.diff.gz
  to pool/main/p/perl/perl_5.8.7-4.diff.gz
perl_5.8.7-4.dsc
  to pool/main/p/perl/perl_5.8.7-4.dsc
perl_5.8.7-4_i386.deb
  to pool/main/p/perl/perl_5.8.7-4_i386.deb
perl_5.8.7-4_powerpc.deb
  to pool/main/p/perl/perl_5.8.7-4_powerpc.deb
perl_5.8.7-4_sparc.deb
  to pool/main/p/perl/perl_5.8.7-4_sparc.deb



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 227621@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Brendan O'Dea <bod@debian.org> (supplier of updated perl package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.7
Date: Sat,  9 Jul 2005 11:14:13 +1000
Source: perl
Binary: perl-base libcgi-fast-perl libperl-dev perl-debug perl-modules perl libperl5.8 perl-suid perl-doc
Architecture: all i386 powerpc source sparc 
Version: 5.8.7-4
Distribution: unstable
Urgency: low
Maintainer: Brendan O'Dea <bod@debian.org>
Changed-By: Brendan O'Dea <bod@debian.org>
Description: 
 libperl-dev - Perl library: development files
 libperl5.8 - Shared Perl library
 perl       - Larry Wall's Practical Extraction and Report Language
 perl-base  - The Pathologically Eclectic Rubbish Lister
 perl-debug - Debug-enabled Perl interpreter
 perl-suid  - Runs setuid Perl scripts
Closes: 227621 304640
Changes: 
 perl (5.8.7-4) unstable; urgency=low
 .
   * Don't generate broken md5sums for libperl5.8 (closes: #304640).
   * Build with gcc 4.0; restore standard (-O2) optimisation on arm, ia64
     and powerpc.  Remove gcc-2.95 build work-around for m68k.
   * Fix test of reenterant function return values which was causing
     perl to malloc itself to death if ERANGE was encountered before
     ENOENT (such as a long line in /etc/group; closes: #227621).
Files: 
 0121c103ef3488fbc8819183ae8ca110 646428 libdevel optional libperl-dev_5.8.7-4_powerpc.deb
 0556c26500a2140e1ad7254d35e7f73d 31220 perl optional perl-suid_5.8.7-4_powerpc.deb
 06e604d7c7cf77d91966433e2dd8fd21 38586 perl optional libcgi-fast-perl_5.8.7-4_all.deb
 7e69f9b4e5f69a355d361b0d381dab65 705 perl standard perl_5.8.7-4.dsc
 1ef9b1c5ff83062dc790e004ce746e7a 792510 base required perl-base_5.8.7-4_sparc.deb
 1f92413d12f363e2e5f9457bda5c3723 3691762 perl standard perl_5.8.7-4_sparc.deb
 2211e476db721a1032ad81dcd6dcee15 602496 libdevel optional libperl-dev_5.8.7-4_sparc.deb
 32f9e539b129f9ef1b3e1ce0b588869a 3377520 perl standard perl_5.8.7-4_i386.deb
 3d36ed1c7af85b89b0a33ff161dd8532 994 libs optional libperl5.8_5.8.7-4_i386.deb
 4dab11f3099e188ca97c95c14e3d314e 2479382 perl optional perl-debug_5.8.7-4_i386.deb
 76dc8b0d59a650458db08db92de56bb2 7208074 doc optional perl-doc_5.8.7-4_all.deb
 869b0e5d52d1204cf6561e93525f0053 2325656 perl standard perl-modules_5.8.7-4_all.deb
 928b21143c843c6fc3addfd10708910c 998 libs optional libperl5.8_5.8.7-4_sparc.deb
 988f0633985145aab744f3f548d5e754 2663788 perl optional perl-debug_5.8.7-4_powerpc.deb
 9e49e9fb1e534488b67d5c39b726396c 30174 perl optional perl-suid_5.8.7-4_i386.deb
 a67e0a0c1a9957904bf0c2306c88e035 3678170 perl standard perl_5.8.7-4_powerpc.deb
 b3ac5821695fb67f3c213bc138d7d7c0 806278 base required perl-base_5.8.7-4_powerpc.deb
 d074285e7d86d7877584bd7bfead79ab 1000 libs optional libperl5.8_5.8.7-4_powerpc.deb
 d2649d03a2731eebf3f7abf90a40f918 2553608 perl optional perl-debug_5.8.7-4_sparc.deb
 d759a93851f87fb0fdad80b8dc42c289 780858 base required perl-base_5.8.7-4_i386.deb
 dbfbccdb980a4bb18f61602ff065f3af 573002 libdevel optional libperl-dev_5.8.7-4_i386.deb
 e95f5330208a39f5a0468889012501d6 30152 perl optional perl-suid_5.8.7-4_sparc.deb
 fcb17a7cb7ad5c71777fa15929ef9f1e 82079 perl standard perl_5.8.7-4.diff.gz

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFCz0E/8NyOALKMWZURAvPDAJ92L95y4bXo4jR/Yb7/89JwcN1wwACeOd5u
v1fFijh2fa1cbT7ek4F9c2Q=
=YHde
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.