Debian Bug report logs - #574990
nscd crashes after moderate use

version graph

Package: nscd; Maintainer for nscd is GNU Libc Maintainers <debian-glibc@lists.debian.org>; Source for nscd is src:eglibc.

Reported by: Lukas Kolbe <lkolbe@TechFak.Uni-Bielefeld.DE>

Date: Mon, 22 Mar 2010 17:57:01 UTC

Severity: important

Tags: squeeze-ignore

Found in versions 2.7-18lenny2, glibc/2.7-18

Reply or subscribe to this bug.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, GNU Libc Maintainers <debian-glibc@lists.debian.org>:
Bug#574990; Package nscd. (Mon, 22 Mar 2010 17:57:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Lukas Kolbe <lkolbe@TechFak.Uni-Bielefeld.DE>:
New Bug report received and forwarded. Copy sent to GNU Libc Maintainers <debian-glibc@lists.debian.org>. (Mon, 22 Mar 2010 17:57:04 GMT) Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: Lukas Kolbe <lkolbe@TechFak.Uni-Bielefeld.DE>
To: submit@bugs.debian.org
Subject: nscd crashes after moderate use
Date: Mon, 22 Mar 2010 18:55:00 +0100
Package: nscd
Version: 2.7-18lenny2
Severity: grave

Hi,

on all (?) our servers we see repeated crashes of nscd:

[ 3687.306397] nscd[3340]: segfault at 10 ip 00007f6b22ce3685 sp 00007fffc977bb60 error 6 in nscd[7f6b22cdd000+1c000]

This happens mostly between 3000 and 6000 seconds after starting. We
have 3003 users in our LDAP, so we increased the 'suggested-size' to a
high prime number 99991, removed the dbs and restarted nscd. The crashes
happen nevertheless, rendering nscd completely unusable on our systems.
I'd like to provide more debugging information, but I don't really know
how at the moment ...

Thanks,
Lukas






Information forwarded to debian-bugs-dist@lists.debian.org, GNU Libc Maintainers <debian-glibc@lists.debian.org>:
Bug#574990; Package nscd. (Thu, 29 Jul 2010 03:24:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Moritz Muehlenhoff <jmm@inutil.org>:
Extra info received and forwarded to list. Copy sent to GNU Libc Maintainers <debian-glibc@lists.debian.org>. (Thu, 29 Jul 2010 03:24:04 GMT) Full text and rfc822 format available.

Message #10 received at 574990@bugs.debian.org (full text, mbox):

From: Moritz Muehlenhoff <jmm@inutil.org>
To: Lukas Kolbe <lkolbe@TechFak.Uni-Bielefeld.DE>
Cc: 574990@bugs.debian.org
Subject: Re: nscd crashes after moderate use
Date: Wed, 28 Jul 2010 23:20:22 -0400
On Mon, Mar 22, 2010 at 06:55:00PM +0100, Lukas Kolbe wrote:
> Package: nscd
> Version: 2.7-18lenny2
> Severity: grave
> 
> Hi,
> 
> on all (?) our servers we see repeated crashes of nscd:
> 
> [ 3687.306397] nscd[3340]: segfault at 10 ip 00007f6b22ce3685 sp 00007fffc977bb60 error 6 in nscd[7f6b22cdd000+1c000]
> 
> This happens mostly between 3000 and 6000 seconds after starting. We
> have 3003 users in our LDAP, so we increased the 'suggested-size' to a
> high prime number 99991, removed the dbs and restarted nscd. The crashes
> happen nevertheless, rendering nscd completely unusable on our systems.
> I'd like to provide more debugging information, but I don't really know
> how at the moment ...

I'm currently setting up a similar test environment with Lenny
and Squeeze to see if I can reproduce it.

nscd has configuration options to force a continuous restart, see
the options "paranoia" and "restart-interval" in nscd.conf. Does
nscd keep up if you set it to something like 600?

You should probably use this in conjunction with the "persistent"
settings.

Cheers,
        Moritz




Information forwarded to debian-bugs-dist@lists.debian.org, GNU Libc Maintainers <debian-glibc@lists.debian.org>:
Bug#574990; Package nscd. (Thu, 29 Jul 2010 08:06:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Lukas Kolbe <lkolbe@techfak.uni-bielefeld.de>:
Extra info received and forwarded to list. Copy sent to GNU Libc Maintainers <debian-glibc@lists.debian.org>. (Thu, 29 Jul 2010 08:06:03 GMT) Full text and rfc822 format available.

Message #15 received at 574990@bugs.debian.org (full text, mbox):

From: Lukas Kolbe <lkolbe@techfak.uni-bielefeld.de>
To: Moritz Muehlenhoff <jmm@inutil.org>
Cc: 574990@bugs.debian.org
Subject: Re: nscd crashes after moderate use
Date: Thu, 29 Jul 2010 09:54:23 +0200
Am Mittwoch, den 28.07.2010, 23:20 -0400 schrieb Moritz Muehlenhoff:
> On Mon, Mar 22, 2010 at 06:55:00PM +0100, Lukas Kolbe wrote:
> > Package: nscd
> > Version: 2.7-18lenny2
> > Severity: grave
> > 
> > Hi,
> > 
> > on all (?) our servers we see repeated crashes of nscd:
> > 
> > [ 3687.306397] nscd[3340]: segfault at 10 ip 00007f6b22ce3685 sp 00007fffc977bb60 error 6 in nscd[7f6b22cdd000+1c000]
> > 
> > This happens mostly between 3000 and 6000 seconds after starting. We
> > have 3003 users in our LDAP, so we increased the 'suggested-size' to a
> > high prime number 99991, removed the dbs and restarted nscd. The crashes
> > happen nevertheless, rendering nscd completely unusable on our systems.
> > I'd like to provide more debugging information, but I don't really know
> > how at the moment ...
> 
> I'm currently setting up a similar test environment with Lenny
> and Squeeze to see if I can reproduce it.
> 
> nscd has configuration options to force a continuous restart, see
> the options "paranoia" and "restart-interval" in nscd.conf. Does
> nscd keep up if you set it to something like 600?
> 
> You should probably use this in conjunction with the "persistent"
> settings.

Sorry, I forgot to mention that - of course we set paranoia mode on, but
that doesn't help at all. I'll set the restart-interval down to 600 to
see if it helps. Here's our default configuration for reference:

        debug-level             0
        paranoia                yes
        enable-cache            passwd          yes
        positive-time-to-live   passwd          600
        negative-time-to-live   passwd          20
        suggested-size          passwd          99991
        check-files             passwd          yes
        persistent              passwd          yes
        shared                  passwd          yes
        max-db-size             passwd          134217728
        auto-propagate          passwd          yes
        enable-cache            group           yes
        positive-time-to-live   group           3600
        negative-time-to-live   group           60
        suggested-size          group           99991
        check-files             group           yes
        persistent              group           yes
        shared                  group           yes
        max-db-size             group           134217728
        auto-propagate          group           yes
        enable-cache            hosts           no
        positive-time-to-live   hosts           3600
        negative-time-to-live   hosts           20
        suggested-size          hosts           99991
        check-files             hosts           yes
        persistent              hosts           yes
        shared                  hosts           yes
        max-db-size             hosts           134217728
        enable-cache            services        yes
        positive-time-to-live   services        28800
        negative-time-to-live   services        20
        suggested-size          services        99991
        check-files             services        yes
        persistent              services        yes
        shared                  services        yes
        max-db-size             services        134217728

As soon as we test our first fileserver with Squeeze (which will take a
few more months probably), I hope to report this bug is non-existant
there ;)

Thanks for your help!

-- 
Lukas





Information forwarded to debian-bugs-dist@lists.debian.org, GNU Libc Maintainers <debian-glibc@lists.debian.org>:
Bug#574990; Package nscd. (Thu, 29 Jul 2010 08:33:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to Lukas Kolbe <lkolbe@techfak.uni-bielefeld.de>:
Extra info received and forwarded to list. Copy sent to GNU Libc Maintainers <debian-glibc@lists.debian.org>. (Thu, 29 Jul 2010 08:33:05 GMT) Full text and rfc822 format available.

Message #20 received at 574990@bugs.debian.org (full text, mbox):

From: Lukas Kolbe <lkolbe@techfak.uni-bielefeld.de>
To: Moritz Muehlenhoff <jmm@inutil.org>
Cc: 574990@bugs.debian.org
Subject: Re: nscd crashes after moderate use
Date: Thu, 29 Jul 2010 10:30:51 +0200
Hi again,

> I'm currently setting up a similar test environment with Lenny
> and Squeeze to see if I can reproduce it.
> 
> nscd has configuration options to force a continuous restart, see
> the options "paranoia" and "restart-interval" in nscd.conf. Does
> nscd keep up if you set it to something like 600?
> 
> You should probably use this in conjunction with the "persistent"
> settings.

Hm, even after setting restart-interval to 600 nscd keeps crashing:

Jul 29 10:06:42 niska kernel: [29035521.984129] nscd[24114]: segfault at 10 ip 00007fa46659b685 sp 00007fffec52f6b0 error 6 in nscd[7fa466595000+1c000]

-- 
Lukas






Information forwarded to debian-bugs-dist@lists.debian.org, GNU Libc Maintainers <debian-glibc@lists.debian.org>:
Bug#574990; Package nscd. (Thu, 29 Jul 2010 10:39:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Hermann Lauer <Hermann.Lauer@iwr.uni-heidelberg.de>:
Extra info received and forwarded to list. Copy sent to GNU Libc Maintainers <debian-glibc@lists.debian.org>. (Thu, 29 Jul 2010 10:39:05 GMT) Full text and rfc822 format available.

Message #25 received at 574990@bugs.debian.org (full text, mbox):

From: Hermann Lauer <Hermann.Lauer@iwr.uni-heidelberg.de>
To: 574990@bugs.debian.org
Subject: nscd crashes after restart after sshd hangs
Date: Thu, 29 Jul 2010 12:04:46 +0200
Hello,

with libnss-ldap and nscd on lenny latest from time to time we have
unresolved bug #552431, which makes sshd hanging due to suddenly unresolvable
"sshd" system user.

Today nscd crashed after sshd hanging and login was possible.
I set "paranoia	yes" in nscd.conf without setting "restart-interval" so it
should be default 3600.

Now nscd crashed, probably while restarting itself:

Jul 29 11:00:43 quadopt3 kernel: [3616361.904197] nscd[6891]: segfault at 10 ip 7f587be30685 sp 7fffffffd960 error 6 in nscd[7f587be2a000+1c000]

Set "persistent no" now and restarted nscd. Current config is below.

Any help for solving this and #552431 is most welcome !

Thanks,
  Hermann

nscd.conf:
	debug-level		0
#	reload-count		5
	paranoia		yes
#	restart-interval	3600

	enable-cache		passwd		yes
	positive-time-to-live	passwd		600
	negative-time-to-live	passwd		20
	suggested-size		passwd		211
	check-files		passwd		yes
	persistent		passwd		no
	shared			passwd		yes
	max-db-size		passwd		33554432
	auto-propagate		passwd		yes

	enable-cache		group		yes
	positive-time-to-live	group		3600
	negative-time-to-live	group		60
	suggested-size		group		211
	check-files		group		yes
	persistent		group		no
	shared			group		yes
	max-db-size		group		33554432
	auto-propagate		group		yes

# hosts caching is broken with gethostby* calls, hence is now disabled
# per default.  See /usr/share/doc/nscd/NEWS.Debian.
	enable-cache		hosts		no
	positive-time-to-live	hosts		3600
	negative-time-to-live	hosts		20
	suggested-size		hosts		211
	check-files		hosts		yes
	persistent		hosts		no
	shared			hosts		yes
	max-db-size		hosts		33554432

	enable-cache		services	yes
	positive-time-to-live	services	28800
	negative-time-to-live	services	20
	suggested-size		services	211
	check-files		services	yes
	persistent		services	no
	shared			services	yes
	max-db-size		services	33554432




Information forwarded to debian-bugs-dist@lists.debian.org, GNU Libc Maintainers <debian-glibc@lists.debian.org>:
Bug#574990; Package nscd. (Fri, 30 Jul 2010 01:03:06 GMT) Full text and rfc822 format available.

Acknowledgement sent to Moritz Muehlenhoff <jmm@inutil.org>:
Extra info received and forwarded to list. Copy sent to GNU Libc Maintainers <debian-glibc@lists.debian.org>. (Fri, 30 Jul 2010 01:03:06 GMT) Full text and rfc822 format available.

Message #30 received at 574990@bugs.debian.org (full text, mbox):

From: Moritz Muehlenhoff <jmm@inutil.org>
To: Lukas Kolbe <lkolbe@techfak.uni-bielefeld.de>
Cc: jmm@inutil.org, 574990@bugs.debian.org
Subject: Re: nscd crashes after moderate use
Date: Thu, 29 Jul 2010 21:00:27 -0400
On Thu, Jul 29, 2010 at 10:30:51AM +0200, Lukas Kolbe wrote:
> Hi again,
> 
> > I'm currently setting up a similar test environment with Lenny
> > and Squeeze to see if I can reproduce it.
> > 
> > nscd has configuration options to force a continuous restart, see
> > the options "paranoia" and "restart-interval" in nscd.conf. Does
> > nscd keep up if you set it to something like 600?
> > 
> > You should probably use this in conjunction with the "persistent"
> > settings.
> 
> Hm, even after setting restart-interval to 600 nscd keeps crashing:
> 
> Jul 29 10:06:42 niska kernel: [29035521.984129] nscd[24114]: segfault at 10 ip 00007fa46659b685 sp 00007fffec52f6b0 error 6 in nscd[7fa466595000+1c000]

Could you please run an additional test: If you set the cache-size
to a really small value, does is crash faster?

I have the suspicion that the crash might occur the first time a
cache slot is replaced.

Cheers,
        Moritz




Information forwarded to debian-bugs-dist@lists.debian.org, GNU Libc Maintainers <debian-glibc@lists.debian.org>:
Bug#574990; Package nscd. (Fri, 30 Jul 2010 09:03:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Lukas Kolbe <lkolbe@techfak.uni-bielefeld.de>:
Extra info received and forwarded to list. Copy sent to GNU Libc Maintainers <debian-glibc@lists.debian.org>. (Fri, 30 Jul 2010 09:03:03 GMT) Full text and rfc822 format available.

Message #35 received at 574990@bugs.debian.org (full text, mbox):

From: Lukas Kolbe <lkolbe@techfak.uni-bielefeld.de>
To: Moritz Muehlenhoff <jmm@inutil.org>
Cc: 574990@bugs.debian.org
Subject: Re: nscd crashes after moderate use
Date: Fri, 30 Jul 2010 10:59:35 +0200
Am Donnerstag, den 29.07.2010, 21:00 -0400 schrieb Moritz Muehlenhoff:
> On Thu, Jul 29, 2010 at 10:30:51AM +0200, Lukas Kolbe wrote:
> > Hi again,
> > 
> > > I'm currently setting up a similar test environment with Lenny
> > > and Squeeze to see if I can reproduce it.
> > > 
> > > nscd has configuration options to force a continuous restart, see
> > > the options "paranoia" and "restart-interval" in nscd.conf. Does
> > > nscd keep up if you set it to something like 600?
> > > 
> > > You should probably use this in conjunction with the "persistent"
> > > settings.
> > 
> > Hm, even after setting restart-interval to 600 nscd keeps crashing:
> > 
> > Jul 29 10:06:42 niska kernel: [29035521.984129] nscd[24114]: segfault at 10 ip 00007fa46659b685 sp 00007fffec52f6b0 error 6 in nscd[7fa466595000+1c000]
> 
> Could you please run an additional test: If you set the cache-size
> to a really small value, does is crash faster?
> 
> I have the suspicion that the crash might occur the first time a
> cache slot is replaced.

I have set suggested-size to 11, rm /var/cache/nscd/* and a few minutes
later it crashed:
Jul 30 10:11:32 niska kernel: [29122212.408089] nscd[27240]: segfault at 10 ip 00007f1acbd48685 sp 00007fff24972820 error 6 in nscd[7f1acbd42000+1c000]

Also setting the max-db-size to 11416 (the sizes of the files
in /var/cache/nscd) didn't change the behaviour. 

FYI: 
niska:~# getent passwd | wc -l
3074

This alone doesn't trigger the crash, it happens only a few minutes
afterwards. I've now set debug-level to 99, maybe it helps ...

-- 
Lukas






Information forwarded to debian-bugs-dist@lists.debian.org, GNU Libc Maintainers <debian-glibc@lists.debian.org>:
Bug#574990; Package nscd. (Mon, 09 Aug 2010 17:18:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Moritz Muehlenhoff <jmm@inutil.org>:
Extra info received and forwarded to list. Copy sent to GNU Libc Maintainers <debian-glibc@lists.debian.org>. (Mon, 09 Aug 2010 17:18:03 GMT) Full text and rfc822 format available.

Message #40 received at 574990@bugs.debian.org (full text, mbox):

From: Moritz Muehlenhoff <jmm@inutil.org>
To: Lukas Kolbe <lkolbe@techfak.uni-bielefeld.de>
Cc: 574990@bugs.debian.org
Subject: Re: nscd crashes after moderate use
Date: Mon, 9 Aug 2010 19:15:55 +0200
On Fri, Jul 30, 2010 at 10:59:35AM +0200, Lukas Kolbe wrote:
> Am Donnerstag, den 29.07.2010, 21:00 -0400 schrieb Moritz Muehlenhoff:

I couldn't test a bigger LDAP setup with sid during DebConf, since my 
notebook turned out to be too short of memory for such a setup inside
virtualbox.

In the mean time, you might want to test whether unscd works better
for you (as suggested in my initial email). There's even preliminary
Debian packaging in the ITP:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=513305

Cheers,
        Moritz





Bug Marked as found in versions glibc/2.7-18. Request was from Don Armstrong <don@debian.org> to control@bugs.debian.org. (Thu, 25 Nov 2010 17:39:27 GMT) Full text and rfc822 format available.

Added tag(s) squeeze-ignore. Request was from Mehdi Dogguy <mehdi@dogguy.org> to control@bugs.debian.org. (Thu, 09 Dec 2010 12:09:12 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, GNU Libc Maintainers <debian-glibc@lists.debian.org>:
Bug#574990; Package nscd. (Thu, 09 Dec 2010 12:27:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Holger Levsen <holger@layer-acht.org>:
Extra info received and forwarded to list. Copy sent to GNU Libc Maintainers <debian-glibc@lists.debian.org>. (Thu, 09 Dec 2010 12:27:03 GMT) Full text and rfc822 format available.

Message #49 received at 574990@bugs.debian.org (full text, mbox):

From: Holger Levsen <holger@layer-acht.org>
To: 574990@bugs.debian.org
Subject: Re: Bug#603982: unblock: unscd/0.47-1 (New Package)
Date: Thu, 9 Dec 2010 13:23:42 +0100
[Message part 1 (text/plain, inline)]
to make life easier for people in the future :)

On Donnerstag, 9. Dezember 2010, Mehdi Dogguy wrote:
> tags 574990 + squeeze-ignore
[...]
> Anyway… [unscd]'s unblocked now and will migrate to Squeeze tonight. I'm 
> also squeeze-ignoring 574990 since I don't imagine it to be fixed in time,
> and now that we have a "replacement". I hope that we won't discover critical
> issues in unscd that will affect Squeeze…


[signature.asc (application/pgp-signature, inline)]

Severity set to 'important' from 'grave' Request was from edward@allcutt.me.uk to control@bugs.debian.org. (Sat, 03 Mar 2012 15:51:15 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, GNU Libc Maintainers <debian-glibc@lists.debian.org>:
Bug#574990; Package nscd. (Sat, 03 Mar 2012 15:52:47 GMT) Full text and rfc822 format available.

Acknowledgement sent to Edward Allcutt <edward@allcutt.me.uk>:
Extra info received and forwarded to list. Copy sent to GNU Libc Maintainers <debian-glibc@lists.debian.org>. (Sat, 03 Mar 2012 15:52:47 GMT) Full text and rfc822 format available.

Message #56 received at 574990@bugs.debian.org (full text, mbox):

From: Edward Allcutt <edward@allcutt.me.uk>
To: Lukas Kolbe <lkolbe@TechFak.Uni-Bielefeld.DE>, Hermann Lauer <Hermann.Lauer@iwr.uni-heidelberg.de>
Cc: 574990@bugs.debian.org
Subject: Ping: nscd crashes after moderate use
Date: Sat, 3 Mar 2012 15:44:38 +0000 (GMT)
Is this reproducible in squeeze or later?

-- 
Edward Allcutt




Information forwarded to debian-bugs-dist@lists.debian.org, GNU Libc Maintainers <debian-glibc@lists.debian.org>:
Bug#574990; Package nscd. (Mon, 05 Mar 2012 09:48:14 GMT) Full text and rfc822 format available.

Acknowledgement sent to Hermann Lauer <Hermann.Lauer@iwr.uni-heidelberg.de>:
Extra info received and forwarded to list. Copy sent to GNU Libc Maintainers <debian-glibc@lists.debian.org>. (Mon, 05 Mar 2012 09:48:14 GMT) Full text and rfc822 format available.

Message #61 received at 574990@bugs.debian.org (full text, mbox):

From: Hermann Lauer <Hermann.Lauer@iwr.uni-heidelberg.de>
To: Edward Allcutt <edward@allcutt.me.uk>
Cc: 574990@bugs.debian.org, lkolbe@TechFak.Uni-Bielefeld.DE
Subject: Re: Ping: nscd crashes after moderate use
Date: Mon, 5 Mar 2012 09:25:28 +0100
On Sat, Mar 03, 2012 at 03:44:38PM +0000, Edward Allcutt wrote:
> Is this reproducible in squeeze or later?

I got one report since using squeeze nscd with ldap, after heavy usage.
We didn't track that down, so no clue at the moment if it's related.

If you don't get another feedback, this lenny related bug could be closed
from my point of view.

  Hermann Lauer

-- 
Netzwerkadministration/Zentrale Dienste, Interdiziplinaeres 
Zentrum fuer wissenschaftliches Rechnen der Universitaet Heidelberg
IWR; INF 368; 69120 Heidelberg; Tel: (06221)54-8236 Fax: -5224
Email: Hermann.Lauer@iwr.uni-heidelberg.de




Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Fri Apr 18 16:15:16 2014; Machine Name: beach.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.