Debian Bug report logs - #679630
startpar: Concurrent booting change SigInt mask of daemons

version graph

Package: startpar; Maintainer for startpar is Debian sysvinit maintainers <pkg-sysvinit-devel@lists.alioth.debian.org>; Source for startpar is src:startpar.

Reported by: Michael Hanke <mih@debian.org>

Date: Sat, 30 Jun 2012 10:39:01 UTC

Severity: important

Found in version startpar/0.58-2

Reply or subscribe to this bug.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, Petter Reinholdtsen <pere@debian.org>:
Bug#679630; Package insserv. (Sat, 30 Jun 2012 10:39:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Michael Hanke <mih@debian.org>:
New Bug report received and forwarded. Copy sent to Petter Reinholdtsen <pere@debian.org>. (Sat, 30 Jun 2012 10:39:07 GMT) Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: Michael Hanke <mih@debian.org>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: insserv: Dependency-based boot changes SigIgn mask of daemons
Date: Sat, 30 Jun 2012 12:35:27 +0200
Package: insserv
Version: 1.14.0-2
Severity: important

Hi,

In our squeeze-based cluster enabling dependency-based booting causes
certain daemons to have different SigIgn masks -- more to the point,
they start ignoring SIGINT. As you can imagine that has all kinds of
implications (for the daemons and the processes they start).

Here is the symptom (ran on one of our compute nodes):

root@snake1:~# cat /proc/*/status | grep "\(SigIgn\|Name\)" |egrep -B1 "[0-9a-f][0-9a-f]+2$"
Name:	ypbind
SigIgn:	0000000000000002
--
Name:	condor_master
SigIgn:	0000000000001002
--
Name:	condor_startd
SigIgn:	0000000000001002
--
Name:	exim4
SigIgn:	0000000000001002
--
Name:	cron
SigIgn:	0000000000000002
--
Name:	sshd
SigIgn:	0000000000001002


root@snake1:~# grep CONCURRENCY /etc/init.d/rc |head -n1
CONCURRENCY=makefile

As soon as I set CONCURRENCY to 'none' there is not a single process
ignoring SIGINT running on the fully booted system. The SigIgn mask can
be fixed by simply restarting the daemons after the system is fully
booted -- for each and every single one:


root@snake1:~# invoke-rc.d condor restart
Shutting down Condor (fast-shutdown mode)...  done.
Starting up Condor...    done.
root@snake1:~# cat /proc/*/status | grep "\(SigIgn\|Name\)" |egrep -B1
"[0-9a-f][0-9a-f]+2$"
Name:	ypbind
SigIgn:	0000000000000002
--
Name:	exim4
SigIgn:	0000000000001002
--
Name:	cron
SigIgn:	0000000000000002
--
Name:	sshd
SigIgn:	0000000000001002
root@snake1:~# invoke-rc.d exim4 restart
Stopping MTA for restart: exim4_listener.
Restarting MTA: exim4.
root@snake1:~# cat /proc/*/status | grep "\(SigIgn\|Name\)" |egrep -B1
"[0-9a-f][0-9a-f]+2$"
Name:	ypbind
SigIgn:	0000000000000002
--
Name:	cron
SigIgn:	0000000000000002
--
Name:	sshd
SigIgn:	0000000000001002
root@snake1:~# invoke-rc.d crom restart 
invoke-rc.d: unknown initscript, /etc/init.d/crom not found.
root@snake1:~# invoke-rc.d cron restart 
Restarting periodic command scheduler: cron.
root@snake1:~# cat /proc/*/status | grep "\(SigIgn\|Name\)" |egrep -B1
"[0-9a-f][0-9a-f]+2$"
Name:	ypbind
SigIgn:	0000000000000002
--
Name:	sshd
SigIgn:	0000000000001002
root@snake1:~# invoke-rc.d ypbind restart
invoke-rc.d: unknown initscript, /etc/init.d/ypbind not found.
root@snake1:~# invoke-rc.d nis restart
Stopping NIS services: ypbind ypserv ypppasswdd ypxfrd.
Starting NIS services: ypbind.
root@snake1:~# cat /proc/*/status | grep "\(SigIgn\|Name\)" |egrep -B1
"[0-9a-f][0-9a-f]+2$"
Name:	sshd
SigIgn:	0000000000001002


We have some indication that not only our cluster is affected (several
condor user report SIGINT-ignore related problems on Debian squeeze).
However there are also clusters that are not affected -- even with
dependency-based booting enabled. Therefore there a chance that this is
caused by something other than insserv. However, given the number of
daemons affected, I decided to report this here.

In any case, I can reliably change the SigIgn masks for all these
daemons by enabled/disabling dependency-based booting -- and this should
probably not be happening.

Thanks,

Michael


-- System Information:
Debian Release: 6.0.5
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.32-5-amd64 (SMP w/24 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages insserv depends on:
ii  libc6                         2.11.3-3   Embedded GNU C Library: Shared lib

insserv recommends no packages.

Versions of packages insserv suggests:
pn  bootchart                     <none>     (no description available)

-- no debconf information




Information forwarded to debian-bugs-dist@lists.debian.org, Petter Reinholdtsen <pere@debian.org>:
Bug#679630; Package insserv. (Sat, 30 Jun 2012 11:03:21 GMT) Full text and rfc822 format available.

Acknowledgement sent to Petter Reinholdtsen <pere@hungry.com>:
Extra info received and forwarded to list. Copy sent to Petter Reinholdtsen <pere@debian.org>. (Sat, 30 Jun 2012 11:03:26 GMT) Full text and rfc822 format available.

Message #10 received at 679630@bugs.debian.org (full text, mbox):

From: Petter Reinholdtsen <pere@hungry.com>
To: Michael Hanke <mih@debian.org>, 679630@bugs.debian.org
Subject: Re: Bug#679630: insserv: Dependency-based boot changes SigIgn mask of daemons
Date: Sat, 30 Jun 2012 13:02:25 +0200
reassign 679630 sysvinit-utils
retitle 679630 startpar: Concurrent booting change SigInt mask of daemons
thanks

[Michael Hanke]
> In our squeeze-based cluster enabling dependency-based booting causes
> certain daemons to have different SigIgn masks -- more to the point,
> they start ignoring SIGINT. As you can imagine that has all kinds of
> implications (for the daemons and the processes they start).

Thank you for your report.  It is most likely reported against the wrong
package, as insserv only update the rc#.d/ symlinks, while startpar is
the program starting services during boot.  So it is not caused by
Dependency-based boot ordering, but by concurrent booting.
-- 
Happy hacking
Petter Reinholdtsen




Bug reassigned from package 'insserv' to 'sysvinit-utils'. Request was from Petter Reinholdtsen <pere@hungry.com> to control@bugs.debian.org. (Sat, 30 Jun 2012 11:03:33 GMT) Full text and rfc822 format available.

No longer marked as found in versions insserv/1.14.0-2. Request was from Petter Reinholdtsen <pere@hungry.com> to control@bugs.debian.org. (Sat, 30 Jun 2012 11:03:34 GMT) Full text and rfc822 format available.

Changed Bug title to 'startpar: Concurrent booting change SigInt mask of daemons' from 'insserv: Dependency-based boot changes SigIgn mask of daemons' Request was from Petter Reinholdtsen <pere@hungry.com> to control@bugs.debian.org. (Sat, 30 Jun 2012 11:03:35 GMT) Full text and rfc822 format available.

Added indication that 679630 affects condor Request was from Michael Hanke <mih@debian.org> to control@bugs.debian.org. (Sun, 01 Jul 2012 07:54:25 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Debian sysvinit maintainers <pkg-sysvinit-devel@lists.alioth.debian.org>:
Bug#679630; Package sysvinit-utils. (Fri, 07 Feb 2014 08:09:09 GMT) Full text and rfc822 format available.

Acknowledgement sent to Petter Reinholdtsen <pere@hungry.com>:
Extra info received and forwarded to list. Copy sent to Debian sysvinit maintainers <pkg-sysvinit-devel@lists.alioth.debian.org>. (Fri, 07 Feb 2014 08:09:09 GMT) Full text and rfc822 format available.

Message #23 received at 679630@bugs.debian.org (full text, mbox):

From: Petter Reinholdtsen <pere@hungry.com>
To: Michael Hanke <mih@debian.org>, 679630@bugs.debian.org
Subject: Re: Bug#679630: insserv: Dependency-based boot changes SigIgn mask of daemons
Date: Fri, 07 Feb 2014 09:04:49 +0100
Perhaps this problem with SIGINT and startpar was caused by the fix
applied in <URL: http://bugs.debian.org/582442 > in 2010?

-- 
Happy hacking
Petter Reinholdtsen



Bug reassigned from package 'sysvinit-utils' to 'startpar'. Request was from Petter Reinholdtsen <pere@hungry.com> to control@bugs.debian.org. (Fri, 28 Mar 2014 18:36:20 GMT) Full text and rfc822 format available.

Marked as found in versions startpar/0.58-2. Request was from Petter Reinholdtsen <pere@hungry.com> to control@bugs.debian.org. (Fri, 28 Mar 2014 18:36:20 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Debian sysvinit maintainers <pkg-sysvinit-devel@lists.alioth.debian.org>:
Bug#679630; Package startpar. (Thu, 10 Apr 2014 13:15:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Petter Reinholdtsen <pere@hungry.com>:
Extra info received and forwarded to list. Copy sent to Debian sysvinit maintainers <pkg-sysvinit-devel@lists.alioth.debian.org>. (Thu, 10 Apr 2014 13:15:04 GMT) Full text and rfc822 format available.

Message #32 received at 679630@bugs.debian.org (full text, mbox):

From: Petter Reinholdtsen <pere@hungry.com>
To: Michael Hanke <mih@debian.org>, 679630@bugs.debian.org
Subject: Re: Bug#679630: insserv: Dependency-based boot changes SigIgn mask of daemons
Date: Thu, 10 Apr 2014 15:10:54 +0200
[Michael Hanke 2012-06-30]
> Hi,

Hi again.

> In our squeeze-based cluster enabling dependency-based booting causes
> certain daemons to have different SigIgn masks -- more to the point,
> they start ignoring SIGINT. As you can imagine that has all kinds of
> implications (for the daemons and the processes they start).

Are you still able to reproduce this?  I'm trying to write a test case
to trigger this bug, but am unable to do so.  This is the test code I
use, and it always report SigIgn: 0000000000000000 for both test
scripts.  What am I doing wrong?  Note that you need startpar version
0.59 just uploaded to unstable to have support for the -e and -d options
to use startpar without running scripts in /etc/init.d/. :)

#!/bin/sh
set -e
if [ -z "$STARTPAR" ] ; then
    STARTPAR=../startpar
fi
mkdir -p etc/init.d
touch etc/insserv.conf
cat > etc/init.d/test <<'EOF'
#!/bin/sh
set -e
### BEGIN INIT INFO
# Provides:          test
# Required-Start:    $remote_fs
# Required-Stop:     $remote_fs
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: test script
### END INIT INFO
echo success: the test script is running $1
echo signal mask for $$:
cat /proc/$$/status | grep "\(SigIgn\|Name\)"
EOF
chmod a+rx etc/init.d/test

cat > etc/init.d/test2 <<'EOF'
#!/bin/sh
set -e
### BEGIN INIT INFO
# Provides:          test2
# Required-Start:    $remote_fs
# Required-Stop:     $remote_fs
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: test script
### END INIT INFO
echo success: the test2 script is running $1
echo signal mask for $$:
cat /proc/$$/status | grep "\(SigIgn\|Name\)"
EOF
chmod a+rx etc/init.d/test2

/sbin/insserv -p etc/init.d test
/sbin/insserv -p etc/init.d test2
$STARTPAR -d etc/init.d -e etc -P 1 -R 2 -M start

rm -rf etc

-- 
Happy hacking
Petter Reinholdtsen



Information forwarded to debian-bugs-dist@lists.debian.org, Debian sysvinit maintainers <pkg-sysvinit-devel@lists.alioth.debian.org>:
Bug#679630; Package startpar. (Thu, 10 Apr 2014 13:27:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Michael Hanke <mih@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian sysvinit maintainers <pkg-sysvinit-devel@lists.alioth.debian.org>. (Thu, 10 Apr 2014 13:27:04 GMT) Full text and rfc822 format available.

Message #37 received at 679630@bugs.debian.org (full text, mbox):

From: Michael Hanke <mih@debian.org>
To: Petter Reinholdtsen <pere@hungry.com>
Cc: 679630@bugs.debian.org
Subject: Re: Bug#679630: insserv: Dependency-based boot changes SigIgn mask of daemons
Date: Thu, 10 Apr 2014 15:24:50 +0200
Hi,

On Thu, Apr 10, 2014 at 03:10:54PM +0200, Petter Reinholdtsen wrote:
> Are you still able to reproduce this?  I'm trying to write a test case
> to trigger this bug, but am unable to do so.  This is the test code I
> use, and it always report SigIgn: 0000000000000000 for both test
> scripts.  What am I doing wrong?  Note that you need startpar version
> 0.59 just uploaded to unstable to have support for the -e and -d options
> to use startpar without running scripts in /etc/init.d/. :)

Thanks for keeping track of this one!

Unfortunately, the issue is still present -- on an up-to-date Debian
wheezy. I still have no clue who to trigger this behavior other than
a reboot of those machines. I can still reliably fix it by a restart
of the condor daemons once the machines are fully up.

If there is anything I can try on those machines that would help you
determine the problem please let me know.

Thanks,

Michael



Information forwarded to debian-bugs-dist@lists.debian.org, Debian sysvinit maintainers <pkg-sysvinit-devel@lists.alioth.debian.org>:
Bug#679630; Package startpar. (Mon, 14 Apr 2014 21:06:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Petter Reinholdtsen <pere@hungry.com>:
Extra info received and forwarded to list. Copy sent to Debian sysvinit maintainers <pkg-sysvinit-devel@lists.alioth.debian.org>. (Mon, 14 Apr 2014 21:06:04 GMT) Full text and rfc822 format available.

Message #42 received at 679630@bugs.debian.org (full text, mbox):

From: Petter Reinholdtsen <pere@hungry.com>
To: Michael Hanke <mih@debian.org>, 679630@bugs.debian.org
Subject: Re: Bug#679630: insserv: Dependency-based boot changes SigIgn mask of daemons
Date: Mon, 14 Apr 2014 23:04:02 +0200
[Michael Hanke]
> Unfortunately, the issue is still present -- on an up-to-date Debian
> wheezy. I still have no clue who to trigger this behavior other than
> a reboot of those machines. I can still reliably fix it by a restart
> of the condor daemons once the machines are fully up.

Right.  Then we should try to fix it. :)

> If there is anything I can try on those machines that would help you
> determine the problem please let me know.

Please try this patch.  It make sure run_single() and detach() both
reset the SIGINT signal.

Index: startpar.c
===================================================================
--- startpar.c  (revision 192)
+++ startpar.c  (working copy)
@@ -732,6 +732,7 @@
          (void)sigfillset(&nmask);
          sigprocmask(SIG_UNBLOCK, &nmask, NULL);
 
+         (void)signal(SIGINT,  SIG_DFL);
          (void)signal(SIGHUP,  SIG_DFL);
          (void)signal(SIGQUIT, SIG_DFL);
          (void)signal(SIGSEGV, SIG_DFL);

I suspect my test cases all ended up using the "fixed" code path.  Not
sure how to trigger the detach() code path in a test case. :)

-- 
Happy hacking
Petter Reinholdtsen



Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Sun Apr 20 11:42:40 2014; Machine Name: buxtehude.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.