Debian Bug report logs - #248125
sshd: processes keep alive after connection break

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: "Mario 'BitKoenig' Holbe" <Mario.Holbe@RZ.TU-Ilmenau.DE>

To: submit@bugs.debian.org

Subject: sshd: processes keep alive after connection break

Date: Sun, 9 May 2004 15:40:21 +0200

Package: ssh
Version: 1:3.8p1-3
Severity: critical

Hello,

sshd leaves processes alive, if a connection breaks while
authentication phase:

Initial state is:

| root@darkside:~# ps -ef | grep ssh
| root     27981     1  0 15:29 ?        00:00:00 /usr/sbin/sshd
| root@darkside:~#

Now I do:

| holbe@darkside:/home/holbe% ssh holbe@localhost
| Password:

Which results in:

| root@darkside:~# ps -ef | grep ssh
| root     27981     1  0 15:28 ?        00:00:00 /usr/sbin/sshd
| holbe    28162  1398  0 15:31 tty5     00:00:00 ssh holbe@localhost
| root     28163 27981  0 15:31 ?        00:00:00 sshd: holbe [priv]
| sshd     28165 28163  0 15:31 ?        00:00:00 sshd: holbe [net]
| root     28166 28163  0 15:31 ?        00:00:00 sshd: holbe [pam]
| root@darkside:~#

Now I break the client with Ctrl-C:

| holbe@darkside:/home/holbe% ssh holbe@localhost
| Password:
| 
| holbe@darkside:/home/holbe%

And the result is:

| root@darkside:~# ps -ef | grep ssh
| root     27981     1  0 15:28 ?        00:00:00 /usr/sbin/sshd
| root     28163 27981  0 15:31 ?        00:00:00 sshd: holbe [priv]
| sshd     28165 28163  0 15:31 ?        00:00:00 [sshd] <defunct>
| root     28166 28163  0 15:31 ?        00:00:00 sshd: holbe [pam]
| root@darkside:~#

Those processes remain running until I manually kill them.

This could very easily be exploited to a Denial-of-Service
attack against system ressources (processes). There is no
special knowledge needed about the victim system, this works
also with uids that don't exist.
That's why I set the severity to critical.


regards,
   Mario
-- 
<jv> Oh well, config
<jv> one actually wonders what force in the universe is holding it
<jv> and makes it working
<Beeth> chances and accidents :)

Message #10 received at 248125@bugs.debian.org (full text, mbox, reply):

From: "Mario 'BitKoenig' Holbe" <Mario.Holbe@RZ.TU-Ilmenau.DE>

To: 248125@bugs.debian.org

Subject: Re: sshd: processes keep alive after connection break

Date: Sun, 9 May 2004 15:57:43 +0200

On Sun, May 09, 2004 at 03:40:21PM +0200, Mario 'BitKoenig' Holbe wrote:
> sshd leaves processes alive, if a connection breaks while
> authentication phase:

Ah, I forgot: config is current packages default.


Mario
-- 
I thought the only thing the internet was good for was porn.  -- Futurama

Message #15 received at 248125@bugs.debian.org (full text, mbox, reply):

From: "Mario 'BitKoenig' Holbe" <Mario.Holbe@RZ.TU-Ilmenau.DE>

To: 248125@bugs.debian.org

Subject: Re: sshd: processes keep alive after connection break

Date: Sun, 9 May 2004 16:04:28 +0200

On Sun, May 09, 2004 at 03:57:43PM +0200, Mario 'BitKoenig' Holbe wrote:
> Ah, I forgot: config is current packages default.

And a second addendum: without UsePAM yes (and with
PasswordAuthentication yes), everything works well.


regards,
   Mario
-- 
User sind wie ideale Gase - sie verteilen sich gleichmaessig ueber alle Platten

Message #22 received at 248125@bugs.debian.org (full text, mbox, reply):

From: Colin Watson <cjwatson@debian.org>

To: Darren Tucker <dtucker@zip.com.au>

Cc: 248125@bugs.debian.org

Subject: sshd processes hanging around after Ctrl-C-ed authentication

Date: Thu, 20 May 2004 14:20:27 +0100

Hi Darren,

Could I impose on you to have a quick look at Debian bug #248125, in
which Ctrl-C-ing ssh at the Password: prompt causes several stray
processes to remain around, including a zombie sshd? I've been looking
through portable Bugzilla for this, and can't find anything, although
bug #839 looks somewhat similar (which is why I'm mailing you).

I can reproduce the problem.

Thanks,

-- 
Colin Watson                                  [cjwatson@flatline.org.uk]

Message #27 received at 248125@bugs.debian.org (full text, mbox, reply):

From: Darren Tucker <dtucker@zip.com.au>

To: Colin Watson <cjwatson@debian.org>

Cc: 248125@bugs.debian.org

Subject: Re: sshd processes hanging around after Ctrl-C-ed authentication

Date: Thu, 20 May 2004 23:32:21 +1000

Colin Watson wrote:
> Could I impose on you to have a quick look at Debian bug #248125, in
> which Ctrl-C-ing ssh at the Password: prompt causes several stray
> processes to remain around, including a zombie sshd? I've been looking
> through portable Bugzilla for this, and can't find anything, although
> bug #839 looks somewhat similar (which is why I'm mailing you).

No problem.  I'm pretty sure that's this one which was fixed in 3.8.1p1:
Log:
 - (dtucker) [auth-pam.c] Reset signal status when starting pam auth
   thread, prevent hanging during PAM keyboard-interactive
   authentications.  ok djm@

Members:
        ChangeLog:1.3274->1.3275
        auth-pam.c:1.96->1.97

-- 
Darren Tucker (dtucker at zip.com.au)
GPG key 8FF4FA69 / D9A3 86E9 7EEE AF4B B2D4  37C9 C982 80C7 8FF4 FA69
    Good judgement comes with experience. Unfortunately, the experience
usually comes from bad judgement.

Message #32 received at 248125@bugs.debian.org (full text, mbox, reply):

From: Colin Watson <cjwatson@debian.org>

To: Darren Tucker <dtucker@zip.com.au>, 248125@bugs.debian.org

Subject: Re: Bug#248125: sshd processes hanging around after Ctrl-C-ed authentication

Date: Thu, 20 May 2004 16:35:59 +0100

On Thu, May 20, 2004 at 11:32:21PM +1000, Darren Tucker wrote:
> Colin Watson wrote:
> >Could I impose on you to have a quick look at Debian bug #248125, in
> >which Ctrl-C-ing ssh at the Password: prompt causes several stray
> >processes to remain around, including a zombie sshd? I've been looking
> >through portable Bugzilla for this, and can't find anything, although
> >bug #839 looks somewhat similar (which is why I'm mailing you).
> 
> No problem.  I'm pretty sure that's this one which was fixed in 3.8.1p1:
> Log:
>  - (dtucker) [auth-pam.c] Reset signal status when starting pam auth
>    thread, prevent hanging during PAM keyboard-interactive
>    authentications.  ok djm@
> 
> Members:
>         ChangeLog:1.3274->1.3275
>         auth-pam.c:1.96->1.97

I'm afraid I'm currently on 3.8.1p1 and can still reproduce it.

Cheers,

-- 
Colin Watson                                  [cjwatson@flatline.org.uk]

Message #37 received at 248125@bugs.debian.org (full text, mbox, reply):

From: Darren Tucker <dtucker@zip.com.au>

To: Colin Watson <cjwatson@debian.org>

Cc: 248125@bugs.debian.org

Subject: Re: Bug#248125: sshd processes hanging around after Ctrl-C-ed authentication

Date: Fri, 21 May 2004 02:45:57 +1000

[Message part 1 (text/plain, inline)]

Colin Watson wrote:
> I'm afraid I'm currently on 3.8.1p1 and can still reproduce it.

Damn, you're right.

It appears that what is happening is that the client exits, breaking the 
TCP connection.  When that happens, the privsep slave exits too, which 
causes a SIGCHLD to be delivered to the monitor.  The monitor then 
attempts to waitpid() on the PAM "thread" which is still alive and 
blissfully unaware of a problem (because nobody told it to die).  That 
waitpid hangs the monitor's cleanup.

The attached patch tests adds a test for this case to the signal handler 
to shoot the PAM  thread itself if it has to.  It needs a bit more 
thought, but works for me in limited testing.

-- 
Darren Tucker (dtucker at zip.com.au)
GPG key 8FF4FA69 / D9A3 86E9 7EEE AF4B B2D4  37C9 C982 80C7 8FF4 FA69
    Good judgement comes with experience. Unfortunately, the experience
usually comes from bad judgement.

[openssh-pam-hang.patch (text/plain, inline)]

Index: auth-pam.c
===================================================================
RCS file: /usr/local/src/security/openssh/cvs/openssh_cvs/auth-pam.c,v
retrieving revision 1.101
diff -u -p -r1.101 auth-pam.c
--- auth-pam.c	13 May 2004 07:29:35 -0000	1.101
+++ auth-pam.c	20 May 2004 16:34:11 -0000
@@ -95,8 +95,14 @@ sshpam_sigchld_handler(int sig)
 {
 	if (cleanup_ctxt == NULL)
 		return;	/* handler called after PAM cleanup, shouldn't happen */
-	if (waitpid(cleanup_ctxt->pam_thread, &sshpam_thread_status, 0) == -1)
-		return;	/* couldn't wait for process */
+	if (waitpid(cleanup_ctxt->pam_thread, &sshpam_thread_status, WNOHANG)
+	     == -1) {
+		/* PAM thread has not exitted, privsep slave must have */
+		kill(cleanup_ctxt->pam_thread, SIGTERM);
+		if (waitpid(cleanup_ctxt->pam_thread, &sshpam_thread_status, 0)
+		    == -1)
+			return; /* could not wait */
+	}
 	if (WIFSIGNALED(sshpam_thread_status) &&
 	    WTERMSIG(sshpam_thread_status) == SIGTERM)
 		return;	/* terminated by pthread_cancel */

Message #42 received at 248125@bugs.debian.org (full text, mbox, reply):

From: Darren Tucker <dtucker@zip.com.au>

To: 248125@bugs.debian.org, "Mario 'BitKoenig' Holbe" <Mario.Holbe@RZ.TU-Ilmenau.DE>

Subject: Debian bug#248125: now fixed upstream

Date: Fri, 28 May 2004 20:56:33 +1000

Hi.
	This bug has now been fixed upstream with a slightly different patch to 
the one I sent earlier.  The upstream bug report is at 
http://bugzilla.mindrot.org/show_bug.cgi?id=839.  The patch has been 
applied to the main OpenSSH tree but the bug is not yet closed (not 
confirmed by reporter).

-- 
Darren Tucker (dtucker at zip.com.au)
GPG key 8FF4FA69 / D9A3 86E9 7EEE AF4B B2D4  37C9 C982 80C7 8FF4 FA69
    Good judgement comes with experience. Unfortunately, the experience
usually comes from bad judgement.

Message #47 received at 248125@bugs.debian.org (full text, mbox, reply):

From: "Mario 'BitKoenig' Holbe" <Mario.Holbe@RZ.TU-Ilmenau.DE>

To: Darren Tucker <dtucker@zip.com.au>

Cc: 248125@bugs.debian.org

Subject: Re: Debian bug#248125: now fixed upstream

Date: Fri, 28 May 2004 14:22:49 +0200

Hello Darren,

On Fri, May 28, 2004 at 08:56:33PM +1000, Darren Tucker wrote:
> 	to the one I sent earlier.  The upstream bug report is at 
> http://bugzilla.mindrot.org/show_bug.cgi?id=839.  The patch has been 
> applied to the main OpenSSH tree but the bug is not yet closed (not 
> confirmed by reporter).

I'll retest the issue as soon as the new version goes to
debian unstable. I'll contact you then :)


regards,
   Mario
-- 
Independence Day: Fortunately, the alien computer operating system works just
fine with the laptop. This proves an important point which Apple enthusiasts
have known for years. While the evil empire of Microsoft may dominate the
computers of Earth people, more advanced life forms clearly prefer Mac's.

Message #52 received at 248125@bugs.debian.org (full text, mbox, reply):

From: Colin Watson <cjwatson@debian.org>

To: Darren Tucker <dtucker@zip.com.au>, 248125@bugs.debian.org

Cc: Mario 'BitKoenig' Holbe <Mario.Holbe@RZ.TU-Ilmenau.DE>

Subject: Re: Bug#248125: Debian bug#248125: now fixed upstream

Date: Fri, 28 May 2004 21:55:56 +0100

On Fri, May 28, 2004 at 08:56:33PM +1000, Darren Tucker wrote:
> 	This bug has now been fixed upstream with a slightly different patch 
> 	to the one I sent earlier.  The upstream bug report is at 
> http://bugzilla.mindrot.org/show_bug.cgi?id=839.  The patch has been 
> applied to the main OpenSSH tree but the bug is not yet closed (not 
> confirmed by reporter).

Thanks for that! It works for me, so I'm preparing an upload now.

Cheers,

-- 
Colin Watson                                  [cjwatson@flatline.org.uk]

Message #57 received at 248125-close@bugs.debian.org (full text, mbox, reply):

From: Colin Watson <cjwatson@debian.org>

To: 248125-close@bugs.debian.org

Subject: Bug#248125: fixed in openssh 1:3.8.1p1-4

Date: Fri, 28 May 2004 17:32:10 -0400

Source: openssh
Source-Version: 1:3.8.1p1-4

We believe that the bug you reported is fixed in the latest version of
openssh, which is due to be installed in the Debian FTP archive:

openssh-client-udeb_3.8.1p1-4_powerpc.udeb
  to pool/main/o/openssh/openssh-client-udeb_3.8.1p1-4_powerpc.udeb
openssh-server-udeb_3.8.1p1-4_powerpc.udeb
  to pool/main/o/openssh/openssh-server-udeb_3.8.1p1-4_powerpc.udeb
openssh_3.8.1p1-4.diff.gz
  to pool/main/o/openssh/openssh_3.8.1p1-4.diff.gz
openssh_3.8.1p1-4.dsc
  to pool/main/o/openssh/openssh_3.8.1p1-4.dsc
ssh-askpass-gnome_3.8.1p1-4_powerpc.deb
  to pool/main/o/openssh/ssh-askpass-gnome_3.8.1p1-4_powerpc.deb
ssh_3.8.1p1-4_powerpc.deb
  to pool/main/o/openssh/ssh_3.8.1p1-4_powerpc.deb



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 248125@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Colin Watson <cjwatson@debian.org> (supplier of updated openssh package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.7
Date: Fri, 28 May 2004 17:58:45 -0300
Source: openssh
Binary: ssh-askpass-gnome openssh-client-udeb ssh openssh-server-udeb
Architecture: source powerpc
Version: 1:3.8.1p1-4
Distribution: unstable
Urgency: medium
Maintainer: Matthew Vernon <matthew@debian.org>
Changed-By: Colin Watson <cjwatson@debian.org>
Description: 
 openssh-client-udeb - Secure shell client for the Debian installer (udeb)
 openssh-server-udeb - Secure shell server for the Debian installer (udeb)
 ssh        - Secure rlogin/rsh/rcp replacement (OpenSSH)
 ssh-askpass-gnome - under X, asks user for a passphrase for ssh-add
Closes: 248125
Changes: 
 openssh (1:3.8.1p1-4) unstable; urgency=medium
 .
   * Kill off PAM thread if privsep slave dies (closes: #248125).
Files: 
 8dce3b0bc4cdc70093d8dbdc473e9bd8 890 net standard openssh_3.8.1p1-4.dsc
 313bb10cb79d9677e887935de39c7178 145574 net standard openssh_3.8.1p1-4.diff.gz
 d56bb8a20deefd960104e0a11d6bd23e 730442 net standard ssh_3.8.1p1-4_powerpc.deb
 08f2e260a229e3886bb06ff3dec6a553 51610 gnome optional ssh-askpass-gnome_3.8.1p1-4_powerpc.deb
 0c181ed3e4c6496eb3bf725543cafae2 100746 debian-installer optional openssh-client-udeb_3.8.1p1-4_powerpc.udeb
 f2dd9a38bcd13f6beab183583db5a1b2 160116 debian-installer optional openssh-server-udeb_3.8.1p1-4_powerpc.udeb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Colin Watson <cjwatson@debian.org> -- Debian developer

iD8DBQFAt6s39t0zAhD6TNERAh32AJ4+34IeBeOc/4toCW8c478PQr5b9ACfSMQD
l/NRDsnwai0LTXXpA0RhWaU=
=JvF4
-----END PGP SIGNATURE-----

Message #62 received at 248125@bugs.debian.org (full text, mbox, reply):

From: "Mario 'BitKoenig' Holbe" <Mario.Holbe@RZ.TU-Ilmenau.DE>

To: Darren Tucker <dtucker@zip.com.au>

Cc: 248125@bugs.debian.org

Subject: Re: Bug#248125: Debian bug#248125: now fixed upstream

Date: Sun, 30 May 2004 08:55:37 +0200

Hello Darren,

On Fri, May 28, 2004 at 09:55:56PM +0100, Colin Watson wrote:
> On Fri, May 28, 2004 at 08:56:33PM +1000, Darren Tucker wrote:
> > http://bugzilla.mindrot.org/show_bug.cgi?id=839.  The patch has been 
> > applied to the main OpenSSH tree but the bug is not yet closed (not 
> > confirmed by reporter).
> Thanks for that! It works for me, so I'm preparing an upload now.

Same here - it works well, no more zombies :)


Thanks for your work & regards,
   Mario
-- 
I heard, if you play a NT-CD backwards, you get satanic messages...
That's nothing. If you play it forwards, it installs NT.

Send a report that this bug log contains spam.