Debian Bug report logs - #341767
simple script causes sshd to run out of memory and die

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: "Mark K. Gardner" <mkg@lanl.gov>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: simple script causes sshd to run out of memory and die

Date: Fri, 02 Dec 2005 14:47:37 -0700

Package: ssh
Version: 1:3.8.1p1-8.sarge.4
Severity: important

I have isolated the problem to running the following script on a
machine logged into via ssh:

===
#!/bin/sh

#
# crashsshd - attempt to crash sshd by making the command line overflow
#

crashsshd a $*
===

  ssh sacrificialhost
  $ crashsshd a

While the script takes a while to run, it eventually causes sshd to
die. The following message was found in the syslog:

Dec  1 21:37:40 mpiblaster kernel: DMA per-cpu:
Dec  1 21:37:40 mpiblaster kernel: cpu 0 hot: low 2, high 6, batch 1
Dec  1 21:37:40 mpiblaster kernel: cpu 0 cold: low 0, high 2, batch 1
Dec  1 21:37:40 mpiblaster kernel: Normal per-cpu:
Dec  1 21:37:40 mpiblaster kernel: cpu 0 hot: low 32, high 96, batch 16
Dec  1 21:37:40 mpiblaster kernel: cpu 0 cold: low 0, high 32, batch 16
Dec  1 21:37:40 mpiblaster kernel: HighMem per-cpu: empty
Dec  1 21:37:40 mpiblaster kernel: 
Dec  1 21:37:40 mpiblaster kernel: Free pages:        4212kB (0kB HighMem)
Dec  1 21:37:40 mpiblaster kernel: Active:202202 inactive:1128 dirty:0 writeback:0 unstable:0 free:1053 slab:13703 mapped:203771 pagetables:5905
Dec  1 21:37:40 mpiblaster kernel: DMA free:1900kB min:16kB low:32kB high:48kB active:11128kB inactive:0kB present:16384kB
Dec  1 21:37:40 mpiblaster kernel: protections[]: 8 476 476
Dec  1 21:37:40 mpiblaster kernel: Normal free:2312kB min:936kB low:1872kB high:2808kB active:797680kB inactive:4512kB present:901120kB
Dec  1 21:37:40 mpiblaster kernel: protections[]: 0 468 468
Dec  1 21:37:40 mpiblaster kernel: HighMem free:0kB min:128kB low:256kB high:384kB active:0kB inactive:0kB present:0kB
Dec  1 21:37:40 mpiblaster kernel: protections[]: 0 0 0
Dec  1 21:37:40 mpiblaster kernel: DMA: 1*4kB 1*8kB 0*16kB 1*32kB 1*64kB 0*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 1900kB
Dec  1 21:37:40 mpiblaster kernel: Normal: 128*4kB 5*8kB 0*16kB 1*32kB 1*64kB 1*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2312kB
Dec  1 21:37:40 mpiblaster kernel: HighMem: empty
Dec  1 21:37:40 mpiblaster kernel: Swap cache: add 0, delete 0, find 0/0, race 0+0
Dec  1 21:37:40 mpiblaster kernel: Out of Memory: Killed process 19833 (sshd).

While infinite recursion is certainly a error in the script, it should
not cause sshd to die. Because it kills sshd, a malicious user can
prevent anyone from logging in via ssh until the daemon is restarted.

-- System Information:
Debian Release: 3.1
Architecture: i386 (i686)
Kernel: Linux 2.6.8-2-386
Locale: LANG=en_US, LC_CTYPE=en_US (charmap=ANSI_X3.4-1968) (ignored: LC_ALL set to C)

Versions of packages ssh depends on:
ii  adduser                3.63              Add and remove users and groups
ii  debconf                1.4.30.13         Debian configuration management sy
ii  dpkg                   1.10.28           Package maintenance system for Deb
ii  libc6                  2.3.2.ds1-22      GNU C Library: Shared libraries an
ii  libpam-modules         0.76-22           Pluggable Authentication Modules f
ii  libpam-runtime         0.76-22           Runtime support for the PAM librar
ii  libpam0g               0.76-22           Pluggable Authentication Modules l
ii  libssl0.9.7            0.9.7e-3sarge1    SSL shared libraries
ii  libwrap0               7.6.dbs-8         Wietse Venema's TCP wrappers libra
ii  zlib1g                 1:1.2.2-4.sarge.2 compression library - runtime

-- debconf information:
  ssh/insecure_rshd:
  ssh/user_environment_tell:
  ssh/ssh2_keys_merged:
* ssh/forward_warning:
  ssh/insecure_telnetd:
  ssh/new_config: true
* ssh/use_old_init_script: true
* ssh/protocol2_only: true
  ssh/encrypted_host_key_but_no_keygen:
* ssh/run_sshd: true
* ssh/SUID_client: true
  ssh/disable_cr_auth: false

Message #10 received at 341767@bugs.debian.org (full text, mbox, reply):

From: Matthew Vernon <matthew@sel.cam.ac.uk>

To: "Mark K. Gardner" <mkg@lanl.gov>, 341767@bugs.debian.org

Subject: Bug#341767: simple script causes sshd to run out of memory and die

Date: Tue, 6 Dec 2005 00:52:44 +0000

 > Dec  1 21:37:40 mpiblaster kernel: HighMem: empty
 > Dec  1 21:37:40 mpiblaster kernel: Swap cache: add 0, delete 0, find 0/0, race 0+0
 > Dec  1 21:37:40 mpiblaster kernel: Out of Memory: Killed process 19833 (sshd).
 > 
 > While infinite recursion is certainly a error in the script, it should
 > not cause sshd to die. Because it kills sshd, a malicious user can
 > prevent anyone from logging in via ssh until the daemon is restarted.
 
What is happening here is that you are running your entire system out
of memory, and the kernel is then killing a process (as it will do if
it runs out of system memory).

This is the expected behaviour of the system.

Matthew 

-- 
Rapun.sel - outermost outpost of the Pick Empire
http://www.pick.ucam.org

Message #15 received at 341767@bugs.debian.org (full text, mbox, reply):

From: Florian Weimer <fw@deneb.enyo.de>

To: Matthew Vernon <matthew@sel.cam.ac.uk>

Cc: 341767@bugs.debian.org, "Mark K. Gardner" <mkg@lanl.gov>

Subject: Re: Bug#341767: simple script causes sshd to run out of memory and die

Date: Tue, 13 Dec 2005 17:56:51 +0100

* Matthew Vernon:

>  > Dec  1 21:37:40 mpiblaster kernel: HighMem: empty
>  > Dec  1 21:37:40 mpiblaster kernel: Swap cache: add 0, delete 0, find 0/0, race 0+0
>  > Dec  1 21:37:40 mpiblaster kernel: Out of Memory: Killed process 19833 (sshd).
>  > 
>  > While infinite recursion is certainly a error in the script, it should
>  > not cause sshd to die. Because it kills sshd, a malicious user can
>  > prevent anyone from logging in via ssh until the daemon is restarted.
>  
> What is happening here is that you are running your entire system out
> of memory, and the kernel is then killing a process (as it will do if
> it runs out of system memory).
>
> This is the expected behaviour of the system.

Yes, but "expected" in the sense of "we know that this is a problem".

If you run a recent kernel and put "vm.overcommit_memory = 2" into
/etc/sysctl.conf (and rebot or run "sysctl vm.overcommit_memory=2"),
the kernel should terminate the process which is the real culprit, and
not kill some innocent bystander.

(By the way, it might make sense to change the OOM priority for sshd.)

Message #22 received at 341767-close@bugs.debian.org (full text, mbox, reply):

From: Colin Watson <cjwatson@debian.org>

To: 341767-close@bugs.debian.org

Subject: Bug#341767: fixed in openssh 1:4.7p1-6

Date: Sun, 30 Mar 2008 22:02:15 +0000

Source: openssh
Source-Version: 1:4.7p1-6

We believe that the bug you reported is fixed in the latest version of
openssh, which is due to be installed in the Debian FTP archive:

openssh-client-udeb_4.7p1-6_i386.udeb
  to pool/main/o/openssh/openssh-client-udeb_4.7p1-6_i386.udeb
openssh-client_4.7p1-6_i386.deb
  to pool/main/o/openssh/openssh-client_4.7p1-6_i386.deb
openssh-server-udeb_4.7p1-6_i386.udeb
  to pool/main/o/openssh/openssh-server-udeb_4.7p1-6_i386.udeb
openssh-server_4.7p1-6_i386.deb
  to pool/main/o/openssh/openssh-server_4.7p1-6_i386.deb
openssh_4.7p1-6.diff.gz
  to pool/main/o/openssh/openssh_4.7p1-6.diff.gz
openssh_4.7p1-6.dsc
  to pool/main/o/openssh/openssh_4.7p1-6.dsc
ssh-askpass-gnome_4.7p1-6_i386.deb
  to pool/main/o/openssh/ssh-askpass-gnome_4.7p1-6_i386.deb
ssh-krb5_4.7p1-6_all.deb
  to pool/main/o/openssh/ssh-krb5_4.7p1-6_all.deb
ssh_4.7p1-6_all.deb
  to pool/main/o/openssh/ssh_4.7p1-6_all.deb



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 341767@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Colin Watson <cjwatson@debian.org> (supplier of updated openssh package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.7
Date: Sun, 30 Mar 2008 21:14:12 +0100
Source: openssh
Binary: openssh-client openssh-server ssh ssh-krb5 ssh-askpass-gnome openssh-client-udeb openssh-server-udeb
Architecture: source all i386
Version: 1:4.7p1-6
Distribution: unstable
Urgency: low
Maintainer: Debian OpenSSH Maintainers <debian-ssh@lists.debian.org>
Changed-By: Colin Watson <cjwatson@debian.org>
Description: 
 openssh-client - secure shell client, an rlogin/rsh/rcp replacement
 openssh-client-udeb - secure shell client for the Debian installer (udeb)
 openssh-server - secure shell server, an rshd replacement
 openssh-server-udeb - secure shell server for the Debian installer (udeb)
 ssh        - secure shell client and server (metapackage)
 ssh-askpass-gnome - interactive X program to prompt users for a passphrase for ssh-ad
 ssh-krb5   - secure shell client and server (transitional package)
Closes: 341767
Changes: 
 openssh (1:4.7p1-6) unstable; urgency=low
 .
   * Disable the Linux kernel's OOM-killer for the sshd parent; tweak
     SSHD_OOM_ADJUST in /etc/default/ssh to change this (closes: #341767).
Files: 
 d4e30527835f6840f263f22efef4950c 1104 net standard openssh_4.7p1-6.dsc
 a396bfba7f9ee38764764289b83c1d23 187796 net standard openssh_4.7p1-6.diff.gz
 366a5ad2f69b7c791e8c96b4b9589318 1046 net extra ssh_4.7p1-6_all.deb
 3fb982edc74f9dc60ff61e7c076f079b 87926 net extra ssh-krb5_4.7p1-6_all.deb
 44f558373ded65347c99a72d3ec8e7f9 662328 net standard openssh-client_4.7p1-6_i386.deb
 ff14132a6233c126b41fdc967b43e525 245532 net optional openssh-server_4.7p1-6_i386.deb
 855043e08d7a157bde882676504dd603 95406 gnome optional ssh-askpass-gnome_4.7p1-6_i386.deb
 d0d78970878da78111ddaf5768f778a4 158524 debian-installer optional openssh-client-udeb_4.7p1-6_i386.udeb
 d337b87c7b20497fd1930abf42b5c831 169112 debian-installer optional openssh-server-udeb_4.7p1-6_i386.udeb
Package-Type: udeb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Colin Watson <cjwatson@debian.org> -- Debian developer

iD8DBQFH7/gH9t0zAhD6TNERAuzFAJsFG2Kvym59te86EnA27sjcw+BRfgCeLXqD
THXOKefShGTjaNsnMv0XtFM=
=V69c
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.