Debian Bug report logs - #281313
if VM is low, setlocale fails silentely

version graph

Package: libc6; Maintainer for libc6 is GNU Libc Maintainers <debian-glibc@lists.debian.org>; Source for libc6 is src:eglibc.

Reported by: James Youngman <jay@gnu.org>

Date: Mon, 15 Nov 2004 02:48:01 UTC

Severity: normal

Tags: confirmed, fixed-upstream, upstream

Fixed in version 2.5

Done: Pierre HABOUZIT <madcoder@debian.org>

Bug is archived. No further changes may be made.

Forwarded to http://sourceware.org/bugzilla/show_bug.cgi?id=4416

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, Michael Stone <mstone@debian.org>:
Bug#281313; Package coreutils. Full text and rfc822 format available.

Acknowledgement sent to James Youngman <jay@gnu.org>:
New Bug report received and forwarded. Copy sent to Michael Stone <mstone@debian.org>. Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: James Youngman <jay@gnu.org>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: coreutils: Output of sort fails sort -c check if LANG is set and memory is low
Date: Mon, 15 Nov 2004 02:34:27 +0000
Package: coreutils
Version: 5.2.1-2
Severity: normal


If $LANG is set and VM is low (in this case artificially), "sort"
produces output that will fail the check made by "sort -c".

If available VM is increased or LANG is unset, the problem does not
occur.  Looking at the strace output for successful invocations (where
VM was plentiful) and the failure case, I notice that the issue relates
to a failing mmap() call:-


 open("/usr/lib/locale/locale-archive", O_RDONLY|O_LARGEFILE) = 3
  fstat64(3, {st_mode=S_IFREG|0644, st_size=1814064, ...}) = 0
   mmap2(NULL, 1814064, PROT_READ, MAP_PRIVATE, 3, 0) = -1 ENOMEM
   (Cannot allocate memory)

Locale information is as per below.  I have also reported this at 
https://savannah.gnu.org/bugs/index.php?func=detailitem&item_id=11004,
but since this might actually be a glibc issue, I have reported it here
also.


-- System Information:
Debian Release: 3.1
  APT prefers unstable
  APT policy: (500, 'unstable'), (500, 'testing')
Architecture: i386 (i686)
Kernel: Linux 2.6.8-1-k7-smp
Locale: LANG=en_GB.ISO-8859-15, LC_CTYPE=en_GB.ISO-8859-15 (charmap=ISO-8859-15)

Versions of packages coreutils depends on:
ii  libacl1                     2.2.26-1     Access control list shared library
ii  libc6                       2.3.2.ds1-18 GNU C Library: Shared libraries an

-- no debconf information



Information forwarded to debian-bugs-dist@lists.debian.org, Michael Stone <mstone@debian.org>:
Bug#281313; Package coreutils. Full text and rfc822 format available.

Acknowledgement sent to Jim Meyering <jim@meyering.net>:
Extra info received and forwarded to list. Copy sent to Michael Stone <mstone@debian.org>. Full text and rfc822 format available.

Message #10 received at submit@bugs.debian.org (full text, mbox):

From: Jim Meyering <jim@meyering.net>
To: James Youngman <jay@gnu.org>
Cc: 281313@bugs.debian.org, Debian Bug Tracking System <submit@bugs.debian.org>
Subject: Re: Bug#281313: coreutils: Output of sort fails sort -c check if LANG is set and memory is low
Date: Mon, 15 Nov 2004 09:44:02 +0100
James Youngman <jay@gnu.org> wrote:
> If $LANG is set and VM is low (in this case artificially), "sort"
> produces output that will fail the check made by "sort -c".
>
> If available VM is increased or LANG is unset, the problem does not
> occur.  Looking at the strace output for successful invocations (where
> VM was plentiful) and the failure case, I notice that the issue relates
> to a failing mmap() call:-
>
>  open("/usr/lib/locale/locale-archive", O_RDONLY|O_LARGEFILE) = 3
>   fstat64(3, {st_mode=S_IFREG|0644, st_size=1814064, ...}) = 0
>    mmap2(NULL, 1814064, PROT_READ, MAP_PRIVATE, 3, 0) = -1 ENOMEM
>    (Cannot allocate memory)

Thank you for the reports.  I can reproduce the problem.
Comparing the failing and succeeding strace output,
I see that in the failing (low-mem) case, setlocale malfunctions like this:

  open("/usr/lib/locale/locale-archive", O_RDONLY|O_LARGEFILE) = 3
  fstat64(3, {st_mode=S_IFREG|0644, st_size=6323664, ...}) = 0
  mmap2(NULL, 2097152, PROT_READ, MAP_PRIVATE, 3, 0) = -1 ENOMEM (Cannot allocate memory)
  close(3)                                = 0
  open("/usr/share/locale/locale.alias", O_RDONLY) = 3
  fstat64(3, {st_mode=S_IFREG|0644, st_size=2539, ...}) = 0
  mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7c93000
  read(3, "# Locale name alias data base.\n#"..., 4096) = 2539
  read(3, "", 4096)                       = 0
  close(3)                                = 0
  munmap(0xb7c93000, 4096)                = 0
  open("/usr/lib/locale/en_GB.iso88591/LC_IDENTIFICATION", O_RDONLY) = -1 ENOENT (No such file or directory)
  open("/usr/lib/locale/en_GB/LC_IDENTIFICATION", O_RDONLY) = -1 ENOENT (No such file or directory)
  open("/usr/lib/locale/en.iso88591/LC_IDENTIFICATION", O_RDONLY) = -1 ENOENT (No such file or directory)
  open("/usr/lib/locale/en/LC_IDENTIFICATION", O_RDONLY) = -1 ENOENT (No such file or directory)

and resorts to setting to the "C" locale.

The trouble seems to be that when the mmap fails, it falls back
on using LC_IDENTIFICATION files like this one:

  /usr/lib/locale/en_GB.iso88591/LC_IDENTIFICATION

but `locate' says I have no file named LC_IDENTIFICATION on my system.



Information forwarded to debian-bugs-dist@lists.debian.org, Michael Stone <mstone@debian.org>:
Bug#281313; Package coreutils. Full text and rfc822 format available.

Acknowledgement sent to Jim Meyering <jim@meyering.net>:
Extra info received and forwarded to list. Copy sent to Michael Stone <mstone@debian.org>. Full text and rfc822 format available.

Tags added: confirmed, upstream Request was from Thomas Hood <jdthood@yahoo.co.uk> to control@bugs.debian.org. Full text and rfc822 format available.

Bug reassigned from package `coreutils' to `libc6'. Request was from Michael Stone <mstone@debian.org> to control@bugs.debian.org. Full text and rfc822 format available.

Reply sent to Pierre HABOUZIT <madcoder@debian.org>:
You have taken responsibility. Full text and rfc822 format available.

Notification sent to James Youngman <jay@gnu.org>:
Bug acknowledged by developer. Full text and rfc822 format available.

Message #24 received at 281313-done@bugs.debian.org (full text, mbox):

From: Pierre HABOUZIT <madcoder@debian.org>
To: James Youngman <jay@gnu.org>, 281313-done@bugs.debian.org
Subject: Re: Bug#281313: coreutils: Output of sort fails sort -c check if LANG is set and memory is low
Date: Tue, 17 Apr 2007 01:08:18 +0200
[Message part 1 (text/plain, inline)]
On Mon, Nov 15, 2004 at 02:34:27AM +0000, James Youngman wrote:
> Package: coreutils
> Version: 5.2.1-2
> Severity: normal
> 
> 
> If $LANG is set and VM is low (in this case artificially), "sort"
> produces output that will fail the check made by "sort -c".
> 
> If available VM is increased or LANG is unset, the problem does not
> occur.  Looking at the strace output for successful invocations (where
> VM was plentiful) and the failure case, I notice that the issue relates
> to a failing mmap() call:-
> 
> 
>  open("/usr/lib/locale/locale-archive", O_RDONLY|O_LARGEFILE) = 3
>   fstat64(3, {st_mode=S_IFREG|0644, st_size=1814064, ...}) = 0
>    mmap2(NULL, 1814064, PROT_READ, MAP_PRIVATE, 3, 0) = -1 ENOMEM
>    (Cannot allocate memory)
> 
> Locale information is as per below.  I have also reported this at 
> https://savannah.gnu.org/bugs/index.php?func=detailitem&item_id=11004,
> but since this might actually be a glibc issue, I have reported it here
> also.

  Correct, but /usr/lib/locale/locale-archive is about 67Mo. If your
system isn't able to map 67M, then your system is likely to have many
problems anyway. Glibc is not meant for embeded systems, especially due
to the locales (and other memory-costly features). Please use dietlibc,
uclibc or custom builds of the libc for that.

  I don't consider it a bug, especially since the real problem is not
that the sort is incorrect, but that it has been done with the C locale
(as it's the default, and that the settings for your $LANG one have not
been found). That's quite a sensible behaviour I'd say.

  Closing the bug, if you feel like it should be kept open, please also
tag it wontfix.

Thanks.
-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org
[Message part 2 (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, GNU Libc Maintainers <debian-glibc@lists.debian.org>:
Bug#281313; Package libc6. Full text and rfc822 format available.

Acknowledgement sent to "James Youngman" <jay@gnu.org>:
Extra info received and forwarded to list. Copy sent to GNU Libc Maintainers <debian-glibc@lists.debian.org>. Full text and rfc822 format available.

Message #29 received at 281313@bugs.debian.org (full text, mbox):

From: "James Youngman" <jay@gnu.org>
To: 281313@bugs.debian.org
Subject: Re: Bug#281313 closed by Pierre HABOUZIT <madcoder@debian.org> (Re: Bug#281313: coreutils: Output of sort fails sort -c check if LANG is set and memory is low)
Date: Tue, 17 Apr 2007 08:29:23 +0100
On 4/17/07, Debian Bug Tracking System <owner@bugs.debian.org> wrote:
> It has been closed by Pierre HABOUZIT <madcoder@debian.org>.

>   Correct, but /usr/lib/locale/locale-archive is about 67Mo. If your
> system isn't able to map 67M, then your system is likely to have many
> problems anyway. Glibc is not meant for embeded systems, especially due
> to the locales (and other memory-costly features). Please use dietlibc,
> uclibc or custom builds of the libc for that.

With all due respect, you have entirely missed the point of the bug.
The bug is *not* that the mmap is necessary.  It is *not* that the
amount of data is large, or that this is difficult on small systems.

The bug s that the failure is *not detected*.    If the mmap() fails,
setlocale() should fail.

James.



Information forwarded to debian-bugs-dist@lists.debian.org, GNU Libc Maintainers <debian-glibc@lists.debian.org>:
Bug#281313; Package libc6. Full text and rfc822 format available.

Acknowledgement sent to Pierre Habouzit <madcoder@debian.org>:
Extra info received and forwarded to list. Copy sent to GNU Libc Maintainers <debian-glibc@lists.debian.org>. Full text and rfc822 format available.

Message #34 received at 281313@bugs.debian.org (full text, mbox):

From: Pierre Habouzit <madcoder@debian.org>
To: James Youngman <jay@gnu.org>, 281313@bugs.debian.org
Cc: control@bugs.debian.org
Subject: Re: Bug#281313: closed by Pierre HABOUZIT <madcoder@debian.org> (Re: Bug#281313: coreutils: Output of sort fails sort -c check if LANG is set and memory is low)
Date: Tue, 17 Apr 2007 11:00:02 +0200
[Message part 1 (text/plain, inline)]
reopen 281313
retitle 281313 if VM is low, setlocale fails silentely
thanks

On Tue, Apr 17, 2007 at 08:29:23AM +0100, James Youngman wrote:
> On 4/17/07, Debian Bug Tracking System <owner@bugs.debian.org> wrote:
> >It has been closed by Pierre HABOUZIT <madcoder@debian.org>.
> 
> >  Correct, but /usr/lib/locale/locale-archive is about 67Mo. If your
> >system isn't able to map 67M, then your system is likely to have many
> >problems anyway. Glibc is not meant for embeded systems, especially due
> >to the locales (and other memory-costly features). Please use dietlibc,
> >uclibc or custom builds of the libc for that.
> 
> With all due respect, you have entirely missed the point of the bug.
> The bug is *not* that the mmap is necessary.  It is *not* that the
> amount of data is large, or that this is difficult on small systems.
> 
> The bug s that the failure is *not detected*.    If the mmap() fails,
> setlocale() should fail.

  ooh, that wasn't obvious at all in your report. Agreed, reopened, and
bug retitled to be more obvious. I'll try to work on a fix.

-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org
[Message part 2 (application/pgp-signature, inline)]

Bug reopened, originator not changed. Request was from Pierre Habouzit <madcoder@debian.org> to control@bugs.debian.org. (Tue, 17 Apr 2007 09:03:02 GMT) Full text and rfc822 format available.

Changed Bug title to if VM is low, setlocale fails silentely from coreutils: Output of sort fails sort -c check if LANG is set and memory is low. Request was from Pierre Habouzit <madcoder@debian.org> to control@bugs.debian.org. (Tue, 17 Apr 2007 09:03:03 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, GNU Libc Maintainers <debian-glibc@lists.debian.org>:
Bug#281313; Package libc6. Full text and rfc822 format available.

Acknowledgement sent to "James Youngman" <jay@gnu.org>:
Extra info received and forwarded to list. Copy sent to GNU Libc Maintainers <debian-glibc@lists.debian.org>. Full text and rfc822 format available.

Message #43 received at 281313@bugs.debian.org (full text, mbox):

From: "James Youngman" <jay@gnu.org>
To: "Pierre Habouzit" <madcoder@debian.org>
Cc: 281313@bugs.debian.org
Subject: Re: Bug#281313: closed by Pierre HABOUZIT <madcoder@debian.org> (Re: Bug#281313: coreutils: Output of sort fails sort -c check if LANG is set and memory is low)
Date: Tue, 17 Apr 2007 10:11:54 +0100
On 4/17/07, Pierre Habouzit <madcoder@debian.org> wrote:
>   ooh, that wasn't obvious at all in your report. Agreed, reopened, and
> bug retitled to be more obvious. I'll try to work on a fix.

Apologies for my earlier poor communication.  Thanks for taking the
time to look at the problem.  FYI though, I haven't retested this in
ages.



Noted your statement that Bug has been forwarded to http://sourceware.org/bugzilla/show_bug.cgi?id=4416. Request was from Pierre Habouzit <madcoder@debian.org> to control@bugs.debian.org. (Mon, 23 Apr 2007 22:00:05 GMT) Full text and rfc822 format available.

Reply sent to Pierre HABOUZIT <madcoder@debian.org>:
You have taken responsibility. Full text and rfc822 format available.

Notification sent to James Youngman <jay@gnu.org>:
Bug acknowledged by developer. Full text and rfc822 format available.

Message #50 received at 281313-done@bugs.debian.org (full text, mbox):

From: Pierre HABOUZIT <madcoder@debian.org>
To: James Youngman <jay@gnu.org>, 281313-done@bugs.debian.org
Subject: Re: Bug#281313: closed by Pierre HABOUZIT <madcoder@debian.org> (Re: Bug#281313: coreutils: Output of sort fails sort -c check if LANG is set and memory is low)
Date: Tue, 24 Apr 2007 12:16:44 +0200
[Message part 1 (text/plain, inline)]
Version: 2.5

On Tue, Apr 17, 2007 at 10:11:54AM +0100, James Youngman wrote:
> On 4/17/07, Pierre Habouzit <madcoder@debian.org> wrote:
> >  ooh, that wasn't obvious at all in your report. Agreed, reopened, and
> >bug retitled to be more obvious. I'll try to work on a fix.
> 
> Apologies for my earlier poor communication.  Thanks for taking the
> time to look at the problem.  FYI though, I haven't retested this in
> ages.


  Allright, I tested it on a 2.5, removing
/usr/lib/locale/locale-archive (which has roughly the same effect than a
failed mmap) and now setlocale returns NULL properly (hence failing
loudly like it should always have).

  Closing the bug.
-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org
[Message part 2 (application/pgp-signature, inline)]

Tags added: fixed-upstream Request was from bts-link-upstream@lists.alioth.debian.org to control@bugs.debian.org. (Wed, 25 Apr 2007 20:24:06 GMT) Full text and rfc822 format available.

Bug archived. Request was from Debbugs Internal Request <owner@bugs.debian.org> to internal_control@bugs.debian.org. (Sun, 24 Jun 2007 15:31:48 GMT) Full text and rfc822 format available.

Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Wed Apr 16 13:17:34 2014; Machine Name: buxtehude.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.