Debian Bug report logs - #482439
cfengine2: segfaults at instrument.c:719

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: Richard A Nelson <cowboy@debian.org>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: cfengine2: There are still a ton of segfaults
Date: Thu, 22 May 2008 18:41:44 +0000

Package: cfengine2
Version: 2.2.6-1
Severity: grave
Justification: renders package unusable + possible data loss

==12662== Invalid read of size 1
==12662==    at 0x80778C8: (within /usr/sbin/cfagent)
==12662==    by 0x8077A8A: (within /usr/sbin/cfagent)
==12662==    by 0x807A48E: (within /usr/sbin/cfagent)
==12662==    by 0x8053AC0: (within /usr/sbin/cfagent)
==12662==    by 0x8053ECC: (within /usr/sbin/cfagent)
==12662==    by 0x431A44F: (below main) (libc-start.c:222)
==12662==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==12662==
==12662== Process terminating with default action of signal 11 (SIGSEGV)
==12662==  Access not within mapped region at address 0x0

This happens on all (or most all) of my cf files - and nearly every
cron cfagent run

It also creates output like this on a few systems:
sparks-ave: Running process command /bin/ps auxw
---------------------------------------------------------------------
Alerts
-------------------------------------------------Ãyð¿^R: Successful
return: 0
Ãyð¿^R: Successful return: 0

-- System Information:
Debian Release: 4.0
  APT prefers testing-proposed-updates
  APT policy: (500, 'testing-proposed-updates'), (500, 'proposed-updates'), (500, 'unstable'), (500, 'testing'), (500, 'stable')
Architecture: i386 (i686)

Kernel: Linux 2.6.25.3 (SMP w/2 CPU cores)
Locale: LANG=en_US, LC_CTYPE=en_US (charmap=ISO-8859-1)
Shell: /bin/sh linked to /bin/bash

Versions of packages cfengine2 depends on:
ii  debconf [debconf-2.0]         1.5.22     Debian configuration management sy
ii  debianutils                   2.28.6     Miscellaneous utilities specific t
ii  libc6                         2.7-11     GNU C Library: Shared libraries
ii  libdb4.6                      4.6.21-8   Berkeley v4.6 Database Libraries [
ii  libssl0.9.8                   0.9.8g-10  SSL shared libraries
ii  lsb-base                      3.2-12     Linux Standard Base 3.2 init scrip
ii  perl                          5.8.8-12   Larry Wall's Practical Extraction 

cfengine2 recommends no packages.

-- debconf information:
* cfengine2/run_cfservd: true
* cfengine2/run_cfenvd: false
* cfengine2/run_cfexecd: false

Message #10 received at 482439@bugs.debian.org (full text, mbox):

From: Morten Werner Forsbring <werner@debian.org>
To: Richard A Nelson <cowboy@debian.org>
Cc: 482439@bugs.debian.org
Subject: Re: Bug#482439: cfengine2: There are still a ton of segfaults
Date: Wed, 25 Jun 2008 20:26:18 +0200

Richard A Nelson <cowboy@debian.org> writes:

> Package: cfengine2
> Version: 2.2.6-1
> Severity: grave
> Justification: renders package unusable + possible data loss
>
> ==12662== Invalid read of size 1
> ==12662==    at 0x80778C8: (within /usr/sbin/cfagent)
> ==12662==    by 0x8077A8A: (within /usr/sbin/cfagent)
> ==12662==    by 0x807A48E: (within /usr/sbin/cfagent)
> ==12662==    by 0x8053AC0: (within /usr/sbin/cfagent)
> ==12662==    by 0x8053ECC: (within /usr/sbin/cfagent)
> ==12662==    by 0x431A44F: (below main) (libc-start.c:222)
> ==12662==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
> ==12662==
> ==12662== Process terminating with default action of signal 11 (SIGSEGV)
> ==12662==  Access not within mapped region at address 0x0
>
> This happens on all (or most all) of my cf files - and nearly every
> cron cfagent run

Hi,

and sorry for this late reply. Have you tested if this is improved
with 2.2.7-1 of cfengine?


- Werner

Message #15 received at 482439@bugs.debian.org (full text, mbox):

From: Richard A Nelson <cowboy@debian.org>
To: Morten Werner Forsbring <werner@debian.org>
Cc: 482439@bugs.debian.org
Subject: Re: Bug#482439: cfengine2: There are still a ton of segfaults
Date: Wed, 25 Jun 2008 18:23:26 -0700 (PDT)

On Wed, 25 Jun 2008, Morten Werner Forsbring wrote:

>> ==12662== Invalid read of size 1
>> ==12662==    at 0x80778C8: (within /usr/sbin/cfagent)
>> ==12662==    by 0x8077A8A: (within /usr/sbin/cfagent)
>> ==12662==    by 0x807A48E: (within /usr/sbin/cfagent)
>> ==12662==    by 0x8053AC0: (within /usr/sbin/cfagent)
>> ==12662==    by 0x8053ECC: (within /usr/sbin/cfagent)
>> ==12662==    by 0x431A44F: (below main) (libc-start.c:222)
>> ==12662==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
>> ==12662==
>> ==12662== Process terminating with default action of signal 11 (SIGSEGV)
>> ==12662==  Access not within mapped region at address 0x0

> Hi,
>
> and sorry for this late reply. Have you tested if this is improved
> with 2.2.7-1 of cfengine?

I don't see it on the servers (amd64), but it persists on the
client (x86-32) boxen - now with about the same regularity (ie, not
every single run, but a couple times a day) - so much better than
before :)

The work clients get this:
Jun 26 00:31:37 corkscrew kernel: cfagent[18939]: segfault at 0 ip
00000000 sp bff455bc error 4 in cfagent[8048000+b3000]
Jun 26 01:01:46 corkscrew kernel: cfagent[24563]: segfault at 0 ip
00000000 sp bfd19b8c error 4 in cfagent[8048000+b3000]
and
Jun 25 17:31:21 gothic-ave kernel: cfagent[8895]: segfault at 1f201 ip
0001f201 sp bfe524ac error 4 in cfagent[8048000+b3000]
Jun 25 18:01:37 gothic-ave kernel: cfagent[15938]: segfault at 1f201 ip
0001f201 sp bfb0a16c error 4 in cfagent[8048000+b3000]

At home, I see this instead:
Jun 26 00:31:21 sparks-ave kernel: cfagent[24499]: segfault at dbdbdbdb
ip dbdbdbdb sp bfa5c0cc error 15
Jun 26 01:01:28 sparks-ave kernel: cfagent[27125]: segfault at dbdbdbdb
ip dbdbdbdb sp bf830eac error 15

The shining light, if there is one is that they problems look to be
always the same failure (address/ip) - looks to be a bad branch ?

I can run a bit via valgrind if it'll help

-- 
Rick Nelson
* |Rain| prepares for polygon soup
<|Rain|> sweet merciful crap, it works?
* |Rain| faints

Message #20 received at 482439@bugs.debian.org (full text, mbox):

From: Petter Reinholdtsen <pere@hungry.com>
To: Richard A Nelson <cowboy@debian.org>, 482439@bugs.debian.org
Subject: Re: cfengine2: There are still a ton of segfaults
Date: Fri, 1 Aug 2008 23:04:34 +0200

Hi.  I rebuild cfengine 2 with DEB_BUILD_OPTIONS=nostrip to get debug
information included in the binaries.  Are you able to rerun your test
with valgrind using these binaries (or binaries you build yourself
using the above environment variable)?  It hopefully will provide more
information on where in the cfagent binary the crash happen.

Happy hacking,
-- 
Petter Reinholdtsen

Message #25 received at 482439@bugs.debian.org (full text, mbox):

From: Morten Werner Forsbring <werner@debian.org>
To: Richard A Nelson <cowboy@debian.org>
Cc: 482439@bugs.debian.org
Subject: Re: Bug#482439: cfengine2: There are still a ton of segfaults
Date: Mon, 25 Aug 2008 11:34:49 +0200

Richard A Nelson <cowboy@debian.org> writes:

>>> ==12662== Invalid read of size 1
>>> ==12662==    at 0x80778C8: (within /usr/sbin/cfagent)
>>> ==12662==    by 0x8077A8A: (within /usr/sbin/cfagent)
>>> ==12662==    by 0x807A48E: (within /usr/sbin/cfagent)
>>> ==12662==    by 0x8053AC0: (within /usr/sbin/cfagent)
>>> ==12662==    by 0x8053ECC: (within /usr/sbin/cfagent)
>>> ==12662==    by 0x431A44F: (below main) (libc-start.c:222)
>>> ==12662==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
>>> ==12662==
>>> ==12662== Process terminating with default action of signal 11 (SIGSEGV)
>>> ==12662==  Access not within mapped region at address 0x0
>
>> Hi,
>>
>> and sorry for this late reply. Have you tested if this is improved
>> with 2.2.7-1 of cfengine?
>
> I don't see it on the servers (amd64), but it persists on the
> client (x86-32) boxen - now with about the same regularity (ie, not
> every single run, but a couple times a day) - so much better than
> before :)

Cfengine2 2.2.8-1 was uploaded to unstable a few days ago, are you able
to test this version as well? Upstream claims that they have been
"fixing an important threading error that has become apparent with the
influx of multicore processors".


- Werner

Message #30 received at 482439@bugs.debian.org (full text, mbox):

From: Richard A Nelson <cowboy@debian.org>
To: Morten Werner Forsbring <werner@debian.org>
Cc: 482439@bugs.debian.org
Subject: Re: Bug#482439: cfengine2: There are still a ton of segfaults
Date: Mon, 25 Aug 2008 09:52:03 -0700 (PDT)

On Mon, 25 Aug 2008, Morten Werner Forsbring wrote:

> Cfengine2 2.2.8-1 was uploaded to unstable a few days ago, are you able
> to test this version as well? Upstream claims that they have been
> "fixing an important threading error that has become apparent with the
> influx of multicore processors".

It seems to have reduced the frequency of segfaults, but do still see
them, I'll be happy to run more tests - if they provide anything of
value

Aug 25 13:31:39 gothic-ave kernel: cfagent[17766]: segfault at 1f201 ip
0001f201 sp bfe5d4dc error 4 in cfagent[8048000+a3000]

Aug 25 07:33:01 corkscrew kernel: cfagent[30723]: segfault at 0 ip
00000000 sp bfa660fc error 4 in cfagent[8048000+a3000]

Aug 25 16:33:45 bandit-hall kernel: cfagent[14429]: segfault at 1e03e ip
0001e03e sp bf94277c error 4 in cfagent[8048000+a3000]

-- 
Rick Nelson
Well, since MS cant be sure of the username of someone downloading
things, they are going to play it safe and have everything dowloaded
and executed by Explorer as suid root. That way, it will run on ANY
system anywhere. :)
		-- George Bonser <grep@cris.com>

Message #35 received at 482439@bugs.debian.org (full text, mbox):

From: Morten Werner Forsbring <werner@debian.org>
To: Richard A Nelson <cowboy@debian.org>
Cc: 482439@bugs.debian.org
Subject: Re: Bug#482439: cfengine2: There are still a ton of segfaults
Date: Fri, 29 Aug 2008 18:05:08 +0200

Richard A Nelson <cowboy@debian.org> writes:

> On Mon, 25 Aug 2008, Morten Werner Forsbring wrote:
>
>> Cfengine2 2.2.8-1 was uploaded to unstable a few days ago, are you able
>> to test this version as well? Upstream claims that they have been
>> "fixing an important threading error that has become apparent with the
>> influx of multicore processors".
>
> It seems to have reduced the frequency of segfaults, but do still see
> them, I'll be happy to run more tests - if they provide anything of
> value
>
> Aug 25 13:31:39 gothic-ave kernel: cfagent[17766]: segfault at 1f201 ip
> 0001f201 sp bfe5d4dc error 4 in cfagent[8048000+a3000]
>
> Aug 25 07:33:01 corkscrew kernel: cfagent[30723]: segfault at 0 ip
> 00000000 sp bfa660fc error 4 in cfagent[8048000+a3000]
>
> Aug 25 16:33:45 bandit-hall kernel: cfagent[14429]: segfault at 1e03e ip
> 0001e03e sp bf94277c error 4 in cfagent[8048000+a3000]

Hi,

I've fetched a patch from upstream which fixes a leaking file
descriptor, can you test the packages on my homepage [1]?


- Werner

[1] http://people.debian.org/~werner/cfengine2/

Message #40 received at 482439@bugs.debian.org (full text, mbox):

From: Richard A Nelson <cowboy@debian.org>
To: Morten Werner Forsbring <werner@debian.org>
Cc: 482439@bugs.debian.org
Subject: Re: Bug#482439: cfengine2: There are still a ton of segfaults
Date: Fri, 29 Aug 2008 09:50:46 -0700 (PDT)

On Fri, 29 Aug 2008, Morten Werner Forsbring wrote:

> Hi,

Howdy

> I've fetched a patch from upstream which fixes a leaking file
> descriptor, can you test the packages on my homepage [1]?

Sure, but not until Tuesday ...  the home machines seem to be doing
ok after the last round of updates - and the office is powered down
this 3day weekdend

-- 
Rick Nelson
<SomeLamer> what's the difference between chattr and chmod?
<SomeGuru> SomeLamer: man chattr > 1; man chmod > 2; diff -u 1 2 | less
		-- Seen on #linux on irc

Message #45 received at 482439@bugs.debian.org (full text, mbox):

From: Morten Werner Forsbring <werner@forsbring.no>
To: Richard A Nelson <cowboy@debian.org>
Cc: 482439@bugs.debian.org
Subject: Re: Bug#482439: cfengine2: There are still a ton of segfaults
Date: Fri, 05 Sep 2008 00:47:10 +0200

Richard A Nelson <cowboy@debian.org> writes:

> Howdy
>
>> I've fetched a patch from upstream which fixes a leaking file
>> descriptor, can you test the packages on my homepage [1]?
>
> Sure, but not until Tuesday ...  the home machines seem to be doing
> ok after the last round of updates - and the office is powered down
> this 3day weekdend

Hi, have you been able to test the new package on your other machines?
Or any other results from your home machines?


- Werner

Message #50 received at 482439@bugs.debian.org (full text, mbox):

From: Richard A Nelson <cowboy@debian.org>
To: Morten Werner Forsbring <werner@forsbring.no>
Cc: 482439@bugs.debian.org
Subject: Re: Bug#482439: cfengine2: There are still a ton of segfaults
Date: Thu, 4 Sep 2008 16:53:14 -0700 (PDT)

On Fri, 5 Sep 2008, Morten Werner Forsbring wrote:

>> Sure, but not until Tuesday ...  the home machines seem to be doing
>> ok after the last round of updates - and the office is powered down
>> this 3day weekdend
>
> Hi, have you been able to test the new package on your other machines?
> Or any other results from your home machines?

Thanks for the prod !

With the current packages (in Debian), I still see the failures
on *all* 32bit systems, but none on the amd64 system (which are
also the cfengine servers).  All (32&64bit) are running cfservd
and cron scheduled cfagent (every 30min).

I've just installed your updated package on the 32bit systems -
fortunately my problem & your build match :)

I'll let it run a while and watch the logs...

In case it helps, here's what I have been seeing with the
stock packages (I'm not sure exactly what the kernel segfault message is
telling me) - but there are some unique patterns:

$zgrep -e 'cfagent.* segfault' /var/log/syslog* | grep -v 'error 4'
/var/log/syslog.5.gz:Aug 30 16:39:27 sparks-ave kernel: cfagent[14488]:
segfault at dbdbdbdb ip dbdbdbdb sp bfaa0a7c error 15

/var/log/syslog.3.gz:Aug 27 16:32:22 bandit-hall kernel: cfagent[21515]:
segfault at dbdbdbdb ip dbdbdbdb sp bfb1994c error 5

/var/log/syslog.2.gz:Aug 28 21:31:49 corkscrew kernel: cfagent[30815]:
segfault at 1166c548 ip 08b3561e sp bfb169ac error 6


This error, however is ~ 99% of them - and should be trackable
to the actual source (with linkmap & listings):
$zgrep -e 'cfagent.* segfault' /var/log/syslog*
/var/log/syslog.3.gz:Aug 27 07:31:52 bandit-hall kernel: cfagent[21610]:
segfault at 1e03e ip 0001e03e sp bfc3e26c error 4 in cfagent[8048000+a3000]

/var/log/syslog:Sep  4 07:01:46 corkscrew kernel: cfagent[1523]:
segfault at 0 ip 00000000 sp bfed556c error 4 in cfagent[8048000+a3000]

/var/log/syslog:Sep  4 07:01:35 gothic-ave kernel: cfagent[15879]:
segfault at 1f201 ip 0001f201 sp bfea7d2c error 4 in cfagent[8048000+a3000]

-- 
Rick Nelson
* Dry-ice can't code his way out of a paper bag
<Coderjoe> dry-ice: int main() { ExitPaperBag(); return 0; }
<Knghtbrd> Is that how that's done then?  *takes notes*

Message #55 received at 482439@bugs.debian.org (full text, mbox):

From: Luk Claes <luk@debian.org>
To: 482439@bugs.debian.org
Subject: Re: cfengine2: There are still a ton of segfaults
Date: Sun, 05 Oct 2008 19:22:11 +0200

Hi

Can we soon expect an upload fixing this bug? If not, please explain
what's the status and how you think you will be able to fix it.

Cheers

Luk

Message #60 received at 482439@bugs.debian.org (full text, mbox):

From: Richard A Nelson <cowboy@debian.org>
To: Morten Werner Forsbring <werner@forsbring.no>
Cc: 482439@bugs.debian.org
Subject: Re: Bug#482439: cfengine2: There are still a ton of segfaults
Date: Mon, 6 Oct 2008 13:27:32 -0700 (PDT)

I was, fortunately, just prodded about this issue on IRC ;)

The current state (for me), is my amd64 servers (also clients) are
running fine...  some of my x86 machines are still experiencing 
a plethora of segfaults daily

The private builds you did were gone, so I rebuilt the current
package sans the strip (to get meaningful data from cfagent)

Here is one such failure - this one is repeatable on one machine:
---------------------------------------------------------------------------
*********************************************************************
 Main Tree Sched: editfiles pass 3 @ Mon Oct  6 19:47:32 2008
*********************************************************************

Checking for potential rule:: Tidy </var/lib/cfengine2/ppkeys>

*********************************************************************
 ==13134==
==13134== Invalid read of size 4
==13134==    at 0x80619DD: CheckFriendReliability (instrument.c:719)
==13134==    by 0x8077264: HandleFriendStatus (functions.c:1847)
==13134==    by 0x807AF73: EvaluateFunction (functions.c:177)
==13134==    by 0x805B3F7: DoAlerts (alerts.c:64)
==13134==    by 0x804F1C6: main (cfagent.c:242)
==13134==  Address 0x463c8fc is 332 bytes inside a block of size 636 free'd
==13134==    at 0x4022B8A: free (vg_replace_malloc.c:323)
==13134==    by 0x41406CF: __os_free (in /usr/lib/libdb-4.6.so)
==13134==    by 0x40E48A3: __db_close (in /usr/lib/libdb-4.6.so)
==13134==    by 0x40F8E21: __db_close_pp (in /usr/lib/libdb-4.6.so)
==13134==    by 0x80617CA: CheckFriendReliability (instrument.c:676)
==13134==    by 0x8077264: HandleFriendStatus (functions.c:1847)
==13134==    by 0x807AF73: EvaluateFunction (functions.c:177)
==13134==    by 0x805B3F7: DoAlerts (alerts.c:64)
==13134==    by 0x804F1C6: main (cfagent.c:242)
==13134==
==13134== Jump to the invalid address stated on the next line
==13134==    at 0xDBDBDBDB: ???
==13134==    by 0x8077264: HandleFriendStatus (functions.c:1847)
==13134==    by 0x807AF73: EvaluateFunction (functions.c:177)
==13134==    by 0x805B3F7: DoAlerts (alerts.c:64)
==13134==    by 0x804F1C6: main (cfagent.c:242)
==13134==  Address 0xdbdbdbdb is not stack'd, malloc'd or (recently) free'd
==13134==
==13134== Process terminating with default action of signal 11 (SIGSEGV)
==13134==  Bad permissions for mapped region at address 0xDBDBDBDB
==13134==    at 0xDBDBDBDB: ???
==13134==    by 0x8077264: HandleFriendStatus (functions.c:1847)
==13134==    by 0x807AF73: EvaluateFunction (functions.c:177)
==13134==    by 0x805B3F7: DoAlerts (alerts.c:64)
==13134==    by 0x804F1C6: main (cfagent.c:242)
Main Tree Sched: shellcommands pass 3 @ Mon Oct  6 19:47:32 2008
*********************************************************************
---------------------------------------------------------------------------

-- 
Rick Nelson
<_Anarchy_> Argh.. who's handing out the paper bags  8)

Message #65 received at 482439@bugs.debian.org (full text, mbox):

From: Richard A Nelson <cowboy@debian.org>
To: Morten Werner Forsbring <werner@forsbring.no>
Cc: 482439@bugs.debian.org
Subject: Re: Bug#482439: cfengine2: There are still a ton of segfaults
Date: Tue, 7 Oct 2008 10:53:57 -0700 (PDT)

Further testing shows this same failure hitting on most of my x86 boxen
I'm not yet sure it is the only one, but is deffinitely common.

-- 
Rick Nelson
"Linux was made by foreign terrorists to take money from true US companies
like Microsoft." - Some AOL'er.
"To this end we dedicate ourselves..." -Don
		-- From the sig of "Don", don@cs.byu.edu

Message #70 received at 482439@bugs.debian.org (full text, mbox):

From: Morten Werner Forsbring <werner@forsbring.no>
To: Richard A Nelson <cowboy@debian.org>
Cc: 482439@bugs.debian.org
Subject: Re: Bug#482439: cfengine2: There are still a ton of segfaults
Date: Thu, 09 Oct 2008 09:49:21 +0200

Richard A Nelson <cowboy@debian.org> writes:

> The current state (for me), is my amd64 servers (also clients) are
> running fine...  some of my x86 machines are still experiencing a
> plethora of segfaults daily
>
> The private builds you did were gone, so I rebuilt the current
> package sans the strip (to get meaningful data from cfagent)

They are back again (was gone due to change of server for
people.debian.org). Are you able to test this package again now?

> Here is one such failure - this one is repeatable on one machine:

Thanks a lot, I'll try to look at them asap.

In the bugreport [1], you wrote 25th of August that the 2.2.8-upload
"seems to have reduced the frequency of segfaults, but do still see
them". I think the severity of the bug is not grave after that upload,
but more important. Do you agree in lowering the severity? That said, we
should definitively try to fix this bug for lenny!


- Werner

Message #75 received at 482439@bugs.debian.org (full text, mbox):

From: Richard A Nelson <cowboy@debian.org>
To: Morten Werner Forsbring <werner@forsbring.no>
Cc: 482439@bugs.debian.org
Subject: Re: Bug#482439: cfengine2: There are still a ton of segfaults
Date: Thu, 9 Oct 2008 08:48:19 -0700 (PDT)

On Thu, 9 Oct 2008, Morten Werner Forsbring wrote:

> They are back again (was gone due to change of server for
> people.debian.org). Are you able to test this package again now?

cool, downloaded and installed - hopefully there is debugging info
in the package...  I'll do some runs and see that happens.

I definitely notice a difference due to the FD leak patch that was
in the earlier update; it made a big difference ;)

>> Here is one such failure - this one is repeatable on one machine:
>
> Thanks a lot, I'll try to look at them asap.

great, as I think I mentioned in a follow-up, that particular bug
was repeatable on several machines

> In the bugreport [1], you wrote 25th of August that the 2.2.8-upload
> "seems to have reduced the frequency of segfaults, but do still see
> them". I think the severity of the bug is not grave after that upload,
> but more important. Do you agree in lowering the severity?

Yes, I do agree - you weren't on IRC, but there I mentioned that while
I am (now less so) worried about the issue, I can, in no way, do without
cfengine.

> That said, we
> should definitively try to fix this bug for lenny!

That'd be great...  I had hoped v3 would be out in time, but that isn't
going to happen soon, if ever (and I'm really anxious for some of the
proposed features - like proper imbed handling)

-- 
Rick Nelson
| |-sshd---tcsh-+-dpkg-buildpacka---rules---sh---make---make---sh---make---sh---make---sh---make---sh---make---sh---make
		-- While packaging XFree86 for Debian GNU/Linux

Message #84 received at 482439@bugs.debian.org (full text, mbox):

From: Morten Werner Forsbring <werner@forsbring.no>
To: Richard A Nelson <cowboy@debian.org>
Cc: 482439@bugs.debian.org
Subject: Re: Bug#482439: cfengine2: There are still a ton of segfaults
Date: Sun, 26 Oct 2008 23:21:10 +0100

Richard A Nelson <cowboy@debian.org> writes:

> cool, downloaded and installed - hopefully there is debugging info
> in the package...  I'll do some runs and see that happens.

Sorry, I forgot to build with debugging info. :/ Have you noticed any
difference?

>>> Here is one such failure - this one is repeatable on one machine:
>>
>> Thanks a lot, I'll try to look at them asap.
>
> great, as I think I mentioned in a follow-up, that particular bug
> was repeatable on several machines

Unfortunately I have not had any time to dig into your results. :/ I've
added the help-tag to this bug.

> Yes, I do agree - you weren't on IRC, but there I mentioned that while
> I am (now less so) worried about the issue, I can, in no way, do without
> cfengine.

I also retitled the bug to reflect the severity.


- Werner

Message #91 received at 482439@bugs.debian.org (full text, mbox):

From: Antonio Radici <antonio@dyne.org>
To: 482439@bugs.debian.org, control@bugs.debian.rog
Subject: cfengine2 still segfaults
Date: Tue, 10 Feb 2009 19:18:30 +0000

tag 482439 +moreinfo
tag 482439 -help
thanks

Hi there,
I'm having a look at this bug and we are claiming that this
is not fixed yet and unfortunately we are not going to get
this fixed for the upcoming stable.

Anyway, what I need for this bug is a compressed corefile,
you can obtain it setting the core file limit to unlimited
(ulimit -c ulimited) and then let cfengine segfault.

Apart from this it would be great if you can include your
inputs directory and your control file for cfengine, if you
can't for privacy reasons or anything else please let me know.

The corefile is crucial to understand where the problem is and
fix it, so please attach it to this bug report.

Cheers
Antonio

Message #100 received at 482439@bugs.debian.org (full text, mbox):

From: Richard Nelson <cowboy@debian.org>
To: 482439@bugs.debian.org
Subject: cfengine2 segfaults
Date: Sun, 01 Mar 2009 19:33:44 -0800

Odd, I wasn't copied (or missed) your reply and request for help...

Thanks, by the way, for the picking up the package - it is an important
tool for me, even with the volume of failures... by and large it does
the right thing (just leaves me nervous about what it isn't doing).

Well, the issue remains, even with the new version, but unfortunately, the package doesn't also build
cfengine2-dbg, so the current core file is pretty much useless.

# gdb /usr/sbin/cfagent /home/core
GNU gdb 6.8-debian
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i486-linux-gnu"...
(no debugging symbols found)

warning: Can't read pathname for load map: Input/output error.
Reading symbols from /usr/lib/libdb-4.7.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libdb-4.7.so
Reading symbols from /usr/lib/i686/cmov/libcrypto.so.0.9.8...(no debugging symbols found).
..done.
Loaded symbols for /usr/lib/i686/cmov/libcrypto.so.0.9.8
Reading symbols from /lib/i686/cmov/libnss_nis.so.2...
(no debugging symbols found)...done.
Loaded symbols for /lib/i686/cmov/libnss_nis.so.2
Reading symbols from /lib/i686/cmov/libpthread.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib/i686/cmov/libpthread.so.0
Reading symbols from /lib/i686/cmov/librt.so.1...
(no debugging symbols found)...done.
Loaded symbols for /lib/i686/cmov/librt.so.1
Reading symbols from /lib/i686/cmov/libm.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/i686/cmov/libm.so.6
Reading symbols from /lib/i686/cmov/libc.so.6...
(no debugging symbols found)...done.
Loaded symbols for /lib/i686/cmov/libc.so.6
Reading symbols from /lib/i686/cmov/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/i686/cmov/libdl.so.2
Reading symbols from /usr/lib/libz.so.1...
(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libz.so.1
Reading symbols from /lib/i686/cmov/libnsl.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/i686/cmov/libnsl.so.1
Reading symbols from /lib/i686/cmov/libnss_files.so.2...
(no debugging symbols found)...done.
Loaded symbols for /lib/i686/cmov/libnss_files.so.2
Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /lib/i686/cmov/libnss_dns.so.2...
(no debugging symbols found)...done.
Loaded symbols for /lib/i686/cmov/libnss_dns.so.2
Reading symbols from /lib/i686/cmov/libresolv.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/i686/cmov/libresolv.so.2

(no debugging symbols found)
Core was generated by `cfagent -IqvK'.
Program terminated with signal 11, Segmentation fault.
[New process 7081]
#0  0x00000000 in ?? ()
(gdb) where
#0  0x00000000 in ?? ()
#1  0x08061a43 in ?? ()
#2  0x08388bf0 in ?? ()
#3  0x00000000 in ?? ()
(gdb)

I've since installed the libc6-dbg package and so have symbols for everything but cfengine2,
but the stack frames are still unknown...  which is why I resorted to valgrind before.

-- 
Rick

Message #105 received at 482439@bugs.debian.org (full text, mbox):

From: Richard Nelson <cowboy@cavein.org>
To: 482439@bugs.debian.org, 511666@bugs.debian.org
Cc: Antonio Radici <antonio@dyne.org>, martin <martin@clues.ltd.uk>
Subject: Re: cfengine2 segfaults
Date: Sun, 01 Mar 2009 20:13:00 -0800

[Message part 1 (text/plain, inline)]

I'd wager a significant amount that these items are in fact
one and the same - or at least related...

After a local rebuild, this is what I see (three in a row):

Core was generated by `cfagent -IqvK'.
Program terminated with signal 11, Segmentation fault.
[New process 13287]
#0  0x00000000 in ?? ()
(gdb) bt
#0  0x00000000 in ?? ()
#1  0x08061a43 in CheckFriendReliability () at instrument.c:719
#2  0x080773e5 in HandleFriendStatus (args=0xbfadc44c "12", value=0xbfade88c "")
    at functions.c:1847
#3  0x0807b0e4 in EvaluateFunction (f=0x90ca390 "FriendStatus(12)", value=0xbfade88c "")
    at functions.c:177
#4  0x0805b468 in DoAlerts () at alerts.c:64
#5  0x0804f1a7 in main (argc=2, argv=0xbfae0954) at cfagent.c:242

I do have this in the common cf file - so it gets driven by both the server
and clients (though only the clients are crapping out on it):
cf.cfengine:        FriendStatus(12) ifelapsed=719

I'm still attempting to capture the failure on some other machines to see
if they, too, fail in the same place
-- 
Rick

[signature.asc (application/pgp-signature, attachment)]

Message #110 received at 482439@bugs.debian.org (full text, mbox):

From: antonio <antonio@dyne.org>
To: Richard Nelson <cowboy@cavein.org>, 511666@bugs.debian.org
Cc: 482439@bugs.debian.org, martin <martin@clues.ltd.uk>
Subject: Re: Bug#511666: cfengine2 segfaults
Date: Mon, 02 Mar 2009 10:11:29 +0000

Richard Nelson wrote:
> I'd wager a significant amount that these items are in fact
> one and the same - or at least related...
>
> After a local rebuild, this is what I see (three in a row):
>   
Hi,
please include the core file (bzip2-ed) so I can have a look,
also let me know which arch you're running cfengine2 on.

Cheers
Antonio

Message #115 received at 482439@bugs.debian.org (full text, mbox):

From: Richard Nelson <cowboy@debian.org>
To: antonio <antonio@dyne.org>
Cc: 511666@bugs.debian.org, 482439@bugs.debian.org,
        martin <martin@clues.ltd.uk>
Subject: Re: Bug#511666: cfengine2 segfaults
Date: Mon, 02 Mar 2009 06:20:32 -0800

[Message part 1 (text/plain, inline)]

antonio wrote:

> Hi,

Hello

> please include the core file (bzip2-ed) so I can have a look,
> also let me know which arch you're running cfengine2 on.

attached (gzip'd), have fun ;)

>  Cheers
> Antonio

-- 
Rick

[core.gz (application/gzip, inline)]

Message #120 received at 482439@bugs.debian.org (full text, mbox):

From: Richard Nelson <cowboy@debian.org>
To: antonio <antonio@dyne.org>
Cc: 511666@bugs.debian.org, 482439@bugs.debian.org,
        martin <martin@clues.ltd.uk>
Subject: Re: Bug#511666: cfengine2 segfaults
Date: Mon, 02 Mar 2009 06:57:22 -0800

antonio wrote:

> also let me know which arch you're running cfengine2 on.

thusfar, all the failing scenarios have been on i386
though I also run amd64

Message #125 received at 482439@bugs.debian.org (full text, mbox):

From: Richard A Nelson <cowboy@debian.org>
To: Richard Nelson <cowboy@debian.org>
Cc: antonio <antonio@dyne.org>, 482439@bugs.debian.org,
        martin <martin@clues.ltd.uk>
Subject: Re: Bug#511666: cfengine2 segfaults
Date: Mon, 02 Mar 2009 12:24:15 -0800

A likely unrelated issue... also on i386 - another machine
routine spews dozens+ lines like this:


*********************************************************************
 Main Tree Sched: editfiles pass 3 @ Mon Mar  2 04:11:27 2009
*********************************************************************

Checking for potential rule:: Tidy </P<8e><9a>¿^R: Successful return: 0
P<8e><9a>¿^R: Successful return: 0
P<8e><9a>¿^R: Successful return: 0
P<8e><9a>¿^R: Successful return: 0
P<8e><9a>¿^R: Successful return: 0
P<8e><9a>¿^R: Successful return: 0
P<8e><9a>¿^R: Successful return: 0
P<8e><9a>¿^R: Successful return: 0
P<8e><9a>¿^R: Successful return: 0
....

so the output buffer or pointer to it seems to have been corrupted
to a string I can't find in the package source :(

Message #138 received at 482439@bugs.debian.org (full text, mbox):

From: Antonio Radici <antonio@dyne.org>
To: cowboy@debian.org
Cc: 482439@bugs.debian.org
Subject: cfengine2: segfaults at instrument.c:719
Date: Sun, 15 Mar 2009 11:26:08 +0000

Hi Richard,
any update on testing the new (custom) cfengine2 package
from mentors?

Cheers
Antonio

Message #143 received at 482439@bugs.debian.org (full text, mbox):

From: Richard A Nelson <cowboy@debian.org>
To: Antonio Radici <antonio@dyne.org>
Cc: 482439@bugs.debian.org
Subject: Re: cfengine2: segfaults at instrument.c:719
Date: Mon, 23 Mar 2009 07:23:55 -0700 (PDT)

On Sun, 15 Mar 2009, Antonio Radici wrote:

> Hi Richard,

Hello, and sorry for the delay - I had an unexpected trip
and was out of town for a week :(

> any update on testing the new (custom) cfengine2 package
> from mentors?

On finally making some headway on my mail backlog, I saw your
prompt (thanks!) and blasted the newer packages on all the 
x86 boxen - letting it run overnight.

The segv, and the oft' extranous output both seem to be gone :)

> Cheers

Indeed, and thanks !!

-- 
Rick Nelson
<james> abuse me.  I'm so lame I sent a bug report to
        debian-devel-changes

Message #146 received at 482439-submitter@bugs.debian.org (full text, mbox):

From: Antonio Radici <antonio@dyne.org>
To: 482439-submitter@bugs.debian.org, cowboy@debian.org
Subject: cfengine2: segfaults at instrument.c:719
Date: Tue, 12 May 2009 21:52:34 +0100

Hi Richard,
just a quick update: are you still getting segfaults using the fixed version?
I'd like to release this fix with the new upstream version of cfengine.

Cheers
Antonio

Message #153 received at 482439-close@bugs.debian.org (full text, mbox):

From: Antonio Radici <antonio@dyne.org>
To: 482439-close@bugs.debian.org
Subject: Bug#482439: fixed in cfengine2 2.2.9-3
Date: Sun, 24 May 2009 10:32:05 +0000

Source: cfengine2
Source-Version: 2.2.9-3

We believe that the bug you reported is fixed in the latest version of
cfengine2, which is due to be installed in the Debian FTP archive:

cfengine2-dbg_2.2.9-3_amd64.deb
  to pool/main/c/cfengine2/cfengine2-dbg_2.2.9-3_amd64.deb
cfengine2_2.2.9-3.diff.gz
  to pool/main/c/cfengine2/cfengine2_2.2.9-3.diff.gz
cfengine2_2.2.9-3.dsc
  to pool/main/c/cfengine2/cfengine2_2.2.9-3.dsc
cfengine2_2.2.9-3_amd64.deb
  to pool/main/c/cfengine2/cfengine2_2.2.9-3_amd64.deb



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 482439@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Antonio Radici <antonio@dyne.org> (supplier of updated cfengine2 package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.8
Date: Wed, 13 May 2009 00:01:36 +0100
Source: cfengine2
Binary: cfengine2 cfengine2-dbg
Architecture: source amd64
Version: 2.2.9-3
Distribution: unstable
Urgency: low
Maintainer: Antonio Radici <antonio@dyne.org>
Changed-By: Antonio Radici <antonio@dyne.org>
Description: 
 cfengine2  - Tool for configuring and maintaining network machines
 cfengine2-dbg - debugging symbols for cfengine2
Closes: 482439 518445 520696
Changes: 
 cfengine2 (2.2.9-3) unstable; urgency=low
 .
   * debian/patches/admit-noclass-520696.patch
     + when admit: contains multiple hostnames and the class does
       not exist, cfengine will fail (Closes: 520696)
   * debian/patches/511666-segfault.patch
     + fix a bug involving CheckFriendReliability (Closes: 482439)
   * debian/po/bg.po:
     + bulgarian translation by dmn@debian.org (Closes: 518445)
   * debian/control:
     + Standards-Version updated to 3.8.1
     + cfengine2-dbg section is now debug - extra
Checksums-Sha1: 
 ba1c111b2ff15e350749caf7407c13d4b0c49cc6 1247 cfengine2_2.2.9-3.dsc
 8662e8b264f95803a469cfe2f36214a9af5c1047 29241 cfengine2_2.2.9-3.diff.gz
 b308f78e23762fa369f861d58670cd666b866a90 2878138 cfengine2_2.2.9-3_amd64.deb
 a46a95a3f45d708e6719acc157311f88b4ede93b 3200386 cfengine2-dbg_2.2.9-3_amd64.deb
Checksums-Sha256: 
 299612aecbcfb8b58ed75fbcce9962924ea06813f736e353a3ccde7d4a5f13f8 1247 cfengine2_2.2.9-3.dsc
 c7523c1029232688b05805c92dcdc99367406a10288f5758076e07bcc0b82799 29241 cfengine2_2.2.9-3.diff.gz
 4e2e85ead47ed595d708abc7da21fb50fea0341a518d48e6c7f5bbdc32bbc750 2878138 cfengine2_2.2.9-3_amd64.deb
 f1702317ba88fe785bd5f30565ab77db64f918a3dc16bcdfa31ff396b84162aa 3200386 cfengine2-dbg_2.2.9-3_amd64.deb
Files: 
 2f09a0bf18d69d472c1fc029ff92767b 1247 admin optional cfengine2_2.2.9-3.dsc
 1360da97d6d5f9351474156e2c30d8f3 29241 admin optional cfengine2_2.2.9-3.diff.gz
 82811a4af58ec00d5ab6f8a24d7cdebd 2878138 admin optional cfengine2_2.2.9-3_amd64.deb
 c5f610f841e910b9bb1b0c5615fbdb3c 3200386 debug extra cfengine2-dbg_2.2.9-3_amd64.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkoZH08ACgkQABzeamt51AFbywCeJh/KxfRPLkUndivquu1IByn+
qbUAn27lYpNzF6UUfhX1cMraSIH3QBlO
=xtaq
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.