Debian Bug report logs - #751636
ssh sessions are not cleanly terminated on shutdown/restart with systemd

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Christoph Anton Mitterer <calestyo@scientia.net>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Sun, 15 Jun 2014 01:20:41 +0200

Package: openssh-server
Version: 1:6.6p1-5
Severity: normal


Hi.

Since openssh-server comes with systemd support, whenever a host
is shut down or restarted, ssh connections to that host just hang
and are no longer cleanly terminated (one also doesn't see the
shutdown message anymore).

I'd suspect that systemd might shut down the network before it kills
the ssh session (or perhaps never kills them at all?)

Cheers,
Chris.


-- System Information:
Debian Release: jessie/sid
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: amd64 (x86_64)

Kernel: Linux 3.14-1-amd64 (SMP w/8 CPU cores)
Locale: LANG=en_DE.utf8, LC_CTYPE=en_DE.utf8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages openssh-server depends on:
ii  adduser                3.113+nmu3
ii  debconf [debconf-2.0]  1.5.53
ii  dpkg                   1.17.10
ii  init-system-helpers    1.19
ii  libc6                  2.19-1
ii  libcomerr2             1.42.10-1
ii  libgssapi-krb5-2       1.12.1+dfsg-2
ii  libkrb5-3              1.12.1+dfsg-2
ii  libpam-modules         1.1.8-3
ii  libpam-runtime         1.1.8-3
ii  libpam0g               1.1.8-3
ii  libselinux1            2.3-1
ii  libssl1.0.0            1.0.1h-2
ii  libwrap0               7.6.q-25
ii  lsb-base               4.1+Debian13
ii  openssh-client         1:6.6p1-5
ii  openssh-sftp-server    1:6.6p1-5
ii  procps                 1:3.3.9-5
ii  zlib1g                 1:1.2.8.dfsg-1

Versions of packages openssh-server recommends:
ii  ncurses-term  5.9+20140118-1
ii  xauth         1:1.0.7-1

Versions of packages openssh-server suggests:
pn  molly-guard   <none>
pn  monkeysphere  <none>
ii  rssh          2.3.4-4
pn  ssh-askpass   <none>
pn  ufw           <none>

-- debconf information:
  ssh/disable_cr_auth: false
  ssh/encrypted_host_key_but_no_keygen:
  openssh-server/permit-root-login: false
* ssh/use_old_init_script: true
  ssh/vulnerable_host_keys:
  ssh/new_config: true

Message #10 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Christoph Anton Mitterer <calestyo@scientia.net>

To: 751636@bugs.debian.org

Subject: Re: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Sun, 15 Jun 2014 04:12:24 +0200

[Message part 1 (text/plain, inline)]

See Also:
https://bugzilla.redhat.com/show_bug.cgi?id=626477

[smime.p7s (application/x-pkcs7-signature, attachment)]

Message #15 received at 751636@bugs.debian.org (full text, mbox, reply):

From: "I. Schrey" <sy@schreyben.de>

To: 751636@bugs.debian.org

Subject: Re: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Mon, 15 Sep 2014 12:03:06 +0200

[Message part 1 (text/plain, inline)]

Hi,

I'm also getting this behaviour from a stripped-down Jessie VM that uses 
systemd.
It started when I switched that VM from sysv to systemd a couple of 
months ago.
It's reproducible on that VM - happens every time.

openssh-server package version: 1:6.6p1-7.

Funny enough, this doesn't happen with another Jessie VM
(same package versions, just a lot more packages installed)  -
that one manages to cleanly terminate ssh sessions upon reboot.


In addition, what both VMs fail to produce is the usual message, like

---snip---
Broadcast message from root@host (Mon Sep 15 10:29:06 2014):

Power button pressed
The system is going down for system halt NOW!
---snip---



Attached are two csv files containing the syslog messages on shutdown
(loglevel info), for comparison.

For example, missing from the stripped-down VM:
"sshd: Received signal 15; terminating"
whereas
"Stopping OpenBSD Secure Shell server..."
is present in both logs.


Regards
Ingmar

[jessie.csv (text/plain, attachment)]

[jessie-stripped.csv (text/plain, attachment)]

Message #20 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Tom Hutter <tom.hutter@unixum.de>

To: 751636@bugs.debian.org

Subject: Re: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Fri, 10 Oct 2014 11:52:24 +0200

[Message part 1 (text/plain, inline)]

Hi,

I had the same issue under Debian Wheezy with systemd 204-14~bpo70+1
As systemd is in constant development and Jessie is currently running 
version 215-5+b1 it maybe resolved in more recent versions than 204-14.

The solution provided under 
https://bugzilla.redhat.com/show_bug.cgi?id=626477 suggests to killall ssh 
sessions, when stopping sshd. I prefer to have at least one ssh session 
open to my server when restarting ssh, if something goes wrong.

Therefore I added a service, to solve this under the current Debian 
Wheezy version I am running.

/etc/systemd/system/ssh-user-sessions.service:

[Unit]
Description=Shutdown all ssh sessions before network
After=network.target

[Service]
TimeoutStartSec=0
Type=oneshot
RemainAfterExit=yes
ExecStart=/bin/true
ExecStop=/usr/bin/killall sshd

When starting, this service does simply nothing (/bin/true). But due to the 
statement "After=network.target" it kills all ssh processes before shutting 
down the network.

Works for me :-)

Cheers

Tom

[Message part 2 (text/html, inline)]

Message #25 received at 751636@bugs.debian.org (full text, mbox, reply):

From: "I. Schrey" <sy@schreyben.de>

To: 751636@bugs.debian.org

Subject: Re: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Fri, 10 Oct 2014 12:54:59 +0200

Hi,

I also created a workaround for my minimal environment, just yesterday.

It involves patching /etc/acpi/powerbtn-acpi-support.sh
(I use acpid/acpi-support-base for power events)
and adding aliases for 'shutdown' and 'reboot':

http://www.schreyben.de/Systemd_Haengende_SSH-Verbindungen_beim_Neustart

Also seems to work well so far.

Regards
Ingmar

Message #30 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Christoph Anton Mitterer <calestyo@scientia.net>

To: 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Fri, 10 Oct 2014 17:44:14 +0200

[Message part 1 (text/plain, inline)]

On Fri, 2014-10-10 at 11:52 +0200, Tom Hutter wrote:
> Therefore I added a service, to solve this under the current Debian
> Wheezy version I am running.
Hmm I'm a bit torn apart between thinking that this is either an ugly
hack or a clean solution ^^

> ExecStart=/bin/true
At least this seems a hack ;-)


Let me see, on the one hand I think there should be one ssh.service file
which should be able to handle everything (isn't systemd so powerful?!
)... on the other hand I like the idea of having a service for the user
sessions, which should ideally allow to stop them (which is, AFAICS,
however not achieved by your version).


Let's perhaps collect what we want:

a) if sshd crashes or when it is restarted/reloaded (or when the network
is restarted), we do not want ssh sessions to terminate, right?

Of course this has the problem, that long running user sessions (or
consider something like ssh control channel multiplexing) are typically
*never* killed - which can be quite of a security problem.
Imagine a flaw found in ssh, which also affects running connections -
even when the main daemon would be restarted after package upgrade,..
the user session processes would be still vulnerable.
But let's say that this should be the job of a tool like needrestart[0]

b) we want a way to actually stop user sessions... not only for this
particular bug (i.e. on shutdown), but as a locally logged in sysadmin
I'd also like to say "okay... away with sshd and it's users".


Maybe the way to go would be the following:
- only restart/reload actions keep the user sessions a live
- stop however kills them as well

AFAICS, this should solve (a) and (b), the only difference to now would
be, that we need to educate our users/admins, that "systemctl stop ssh"
really means "all ssh stops" and not just "the main ssh daemon stops but
old connections remain".
Actually I feel that would be much cleaner thins the action is called
"stop" and not "stop-but-there-are-exceptions" and one should be able to
expect stop to really stop it (on the other hand though, one could argue
the same for restart/reload).


Cheers,
Chris.



[0] Note that needrestart currently also suffers this problem: it only
tracks the daemons, not the running sessions.

[smime.p7s (application/x-pkcs7-signature, attachment)]

Message #35 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Christoph Anton Mitterer <calestyo@scientia.net>

To: 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Fri, 10 Oct 2014 17:45:53 +0200

[Message part 1 (text/plain, inline)]

On Fri, 2014-10-10 at 12:54 +0200, I. Schrey wrote: 
> I also created a workaround for my minimal environment, just yesterday.
> 
> It involves patching /etc/acpi/powerbtn-acpi-support.sh
> (I use acpid/acpi-support-base for power events)
> and adding aliases for 'shutdown' and 'reboot':
> 
> http://www.schreyben.de/Systemd_Haengende_SSH-Verbindungen_beim_Neustart
> 
> Also seems to work well so far.

Well that's really a hack, especially since it only works with acpi*
packages installed.

[smime.p7s (application/x-pkcs7-signature, attachment)]

Message #40 received at 751636@bugs.debian.org (full text, mbox, reply):

From: David Lechner <david@lechnology.com>

To: 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Fri, 10 Oct 2014 10:51:04 -0500

I have found that using ssh.socket instead of ssh.service makes clients 
shut down cleanly most of the time (sometimes they will still hang, but 
it is rare).

    systemctl disable ssh.service
    systemctl enable ssh.socket

Message #45 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Tom Hutter <tom.hutter@unixum.de>

To: Christoph Anton Mitterer <calestyo@scientia.net>, 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Fri, 10 Oct 2014 18:51:20 +0200

[Message part 1 (text/plain, inline)]

On Friday 10 October 2014 17:44:14 Christoph Anton Mitterer wrote:
> On Fri, 2014-10-10 at 11:52 +0200, Tom Hutter wrote:
> > Therefore I added a service, to solve this under the current Debian
> > Wheezy version I am running.
> 
> Hmm I'm a bit torn apart between thinking that this is either an ugly
> hack or a clean solution ^^
> 
> > ExecStart=/bin/true
> 
> At least this seems a hack ;-)

Never said it's not a hack. I may disagree in the word "ugly" ;-)

> 
> 
> Let me see, on the one hand I think there should be one ssh.service file
> which should be able to handle everything (isn't systemd so powerful?!
> )... on the other hand I like the idea of having a service for the user
> sessions, which should ideally allow to stop them (which is, AFAICS,
> however not achieved by your version).
> 
> 
> Let's perhaps collect what we want:
> 
> a) if sshd crashes or when it is restarted/reloaded (or when the network
> is restarted), we do not want ssh sessions to terminate, right?
> 

If ssh is restarted, my hack does exactly the expected. The ssh user 
sessions stay alive. Only, if the network is shut down, the sessions will be 
terminated. Looking at this excerpt from systemd.special(7), I would say, 
my hack does exactly the expected. What is a session depending on 
network good, once the network has gone away?

       network.target
           This unit is supposed to indicate when network functionality is
           available, but it is only very weakly defined what that is supposed
           to mean, with one exception: at shutdown, a unit that is ordered
           after network.target will be stopped before the network -- to
           whatever level it might be set up then -- is shut down. Also see
           Running Services After the Network is up[1] for more information.
           Also see network-online.target described above.

> Of course this has the problem, that long running user sessions (or
> consider something like ssh control channel multiplexing) are typically
> *never* killed - which can be quite of a security problem.
> Imagine a flaw found in ssh, which also affects running connections -
> even when the main daemon would be restarted after package 
upgrade,..
> the user session processes would be still vulnerable.
> But let's say that this should be the job of a tool like needrestart[0]

As mentioned above, restarting of sshd will not shut down the existing ssh 
sessions. Security flaw or not.

> 
> b) we want a way to actually stop user sessions... not only for this
> particular bug (i.e. on shutdown), but as a locally logged in sysadmin
> I'd also like to say "okay... away with sshd and it's users".

If you want to stop the ssh sessions during shutdown and signalling it to 
the clients, you have to have the network stack still available. Therefore 
killing all ssh sessions before network.target is going down seems to me 
the last possible moment. Otherwise the ssh sessions will be closed but 
your client will not be signalled, which was the reason I made this hack.

> 
> 
> Maybe the way to go would be the following:
> - only restart/reload actions keep the user sessions a live
> - stop however kills them as well

systemd distinguishes between restart and reload. Reloading of the 
network.target (if reloading a target is even possible) shouldn't affect the 
ssh sessions. Therefor I would need something like 
ReloadPropagatedFrom= in the unit section.

I think it's more a philosophical question, if running processes depending 
on the network should survive a network restart - I would say: No
Then again if you say "Yes", I think it's hard to find a solution, to terminate 
the ssh sessions only on network stop and not on network restart. If I am 
correct, restart in systemd means stop and start. So how distinguish 
between stop to shut down an stop to restart?

Cheers

Tom

[Message part 2 (text/html, inline)]

[signature.asc (application/pgp-signature, inline)]

Message #50 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Christoph Anton Mitterer <calestyo@scientia.net>

To: 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Fri, 10 Oct 2014 22:50:31 +0200

[Message part 1 (text/plain, inline)]

On Fri, 2014-10-10 at 18:51 +0200, Tom Hutter wrote:
> > Hmm I'm a bit torn apart between thinking that this is either an
> > ugly hack or a clean solution ^^
> Never said it's not a hack. I may disagree in the word "ugly" ;-)
;-)


> If ssh is restarted, my hack does exactly the expected. The ssh user
> sessions stay alive. Only, if the network is shut down, the sessions
> will be terminated. Looking at this excerpt from systemd.special(7), I
> would say, my hack does exactly the expected.
Sure,... I never claimed it wouldn't work... just that I'm not sure
whether I like the idea of having a separate unit file for the ssh
"sessions".

I mean you have a similar concepts in many other daemons (like httpd or
some DB servers have their open connections)... and none of them will
have these managed via a separate unit file.


> network.target
> This unit is supposed to indicate when network functionality is
> available, but it is only very weakly defined what that is supposed
Well that's another point,.. I personally really hate the idea of
network.target... as even upstream admits it's weakly defined... and I
think this brings all kinds of problem with it.
What does "indicate when network functionality is available" really mean
in a even driven world with many possible network connections per
computer and services networking depends upon.

So the problem is that many things may hook into network.target or
network-pre.target,... starting from just the bare question whether a
device is up or not, over things like "do we have DNS?" to stuff like
firewalls, or VPN.
Of course this doesn't directly affect us, I just want to point out that
depending on network.target always means some serialisation point, which
is IMHO against the ideas of systemd.
At least from my (though non-expert PoV) I'd suggest to avoid
dependencies on it when possible.


> As mentioned above, restarting of sshd will not shut down the existing
> ssh sessions. Security flaw or not.
Well you're talking about restarting the sshd process - sure it won't
kill the existing sessions.
I was referring to restarting the ssh[d] service (i.e. systemd
service)... and was generally questioning, what do we want to happen,
when people restart respectively stop it. :)


> If you want to stop the ssh sessions during shutdown and signalling it
> to the clients, you have to have the network stack still available.
> Therefore killing all ssh sessions before network.target is going down
> seems to me the last possible moment. Otherwise the ssh sessions will
> be closed but your client will not be signalled, which was the reason
> I made this hack.
Again here,.. my point (b) was more a general question, not only
specific to shutdown.

My idea was: When we'd agree that stopping the ssh service (not
restarting/reloading) should actually also stop the connections, then
one could AFAIU simply do something like this:
Remove the "KillMode=process" line from the ssh.service unit file.

Or is KillMode also used on systemd's restart/reload of services?



> I think it's more a philosophical question, if running processes
> depending on the network should survive a network restart - I would
> say: No
I'd say: depends
Many services just happily survive a network restart... ssh is a good
example (unless sshd kills the connection beacause of TCP ClientAlive or
SSH ClientAlive).



> Then again if you say "Yes", I think it's hard to find a solution, to
> terminate the ssh sessions only on network stop and not on network
> restart.
Well all this question wouldn't arise when there was no separate
service, right?
If the normal ssh.service would just kill existing connections, than,
AFAIU, the only thing needed is:
After=networking.target (which we have anyway)
and removal of:
KillMode=process

If network is just restarted, then this shouldn't affect ssh.service at
all. But if the machine is shut down, then systemd should stop
ssh.service, thereby calling the default KillMode=control-group.
And because of the After= it should happen before networking.target is
"stopped".
Or do I miss something?

> If I am correct, restart in systemd means stop and start. So how
> distinguish between stop to shut down an stop to restart?
Mhhh... that would be a problem though in my idea,.. cause I base my
assumption that on restart KillMode=control-group wouldn't happen. :/


Cheers,
Chris.

[smime.p7s (application/x-pkcs7-signature, attachment)]

Message #55 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Tom Hutter <tom.hutter@unixum.de>

To: Christoph Anton Mitterer <calestyo@scientia.net>, 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Sat, 11 Oct 2014 11:23:39 +0200

[Message part 1 (text/plain, inline)]

On Friday 10 October 2014 22:50:31 Christoph Anton Mitterer wrote:
> Sure,... I never claimed it wouldn't work... just that I'm not sure
> whether I like the idea of having a separate unit file for the ssh
> "sessions".
> 
> I mean you have a similar concepts in many other daemons (like httpd or
> some DB servers have their open connections)... and none of them will
> have these managed via a separate unit file.

Exactly. Stop apache, nginx, mysql, postgres and tell me, how many 
children processes will be left, once you stopped it.
As ssh behaves differently and keeps the children alive, you have to 
handle it differently :-)

> 
> > network.target
> > This unit is supposed to indicate when network functionality is
> > available, but it is only very weakly defined what that is supposed
> 
> Well that's another point,.. I personally really hate the idea of
> network.target... as even upstream admits it's weakly defined... and I
> think this brings all kinds of problem with it.
> What does "indicate when network functionality is available" really mean
> in a even driven world with many possible network connections per
> computer and services networking depends upon.
> 
> So the problem is that many things may hook into network.target or
> network-pre.target,... starting from just the bare question whether a
> device is up or not, over things like "do we have DNS?" to stuff like
> firewalls, or VPN.
> Of course this doesn't directly affect us, I just want to point out that
> depending on network.target always means some serialisation point, 
which
> is IMHO against the ideas of systemd.
> At least from my (though non-expert PoV) I'd suggest to avoid
> dependencies on it when possible.

What can I say. You made your point clear. If you don't want to rely on 
network.target my solution will not help you :-)
> 
> > As mentioned above, restarting of sshd will not shut down the existing
> > ssh sessions. Security flaw or not.
> 
> Well you're talking about restarting the sshd process - sure it won't
> kill the existing sessions.
> I was referring to restarting the ssh[d] service (i.e. systemd
> service)... and was generally questioning, what do we want to happen,
> when people restart respectively stop it. :)

systemctl stop ssh.service => all existing user sessions will stay alive
Try it. That is why I made this hack in the first place. The solution provided 
in the RedHat forum would shut down all ssh sessions and I want to keep 
at least one ssh session open to my server, when restarting sshd.

Cheers

Tom

[Message part 2 (text/html, inline)]

Message #60 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Christoph Anton Mitterer <calestyo@scientia.net>

To: 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Sun, 12 Oct 2014 02:24:11 +0200

[Message part 1 (text/plain, inline)]

On Sat, 2014-10-11 at 11:23 +0200, Tom Hutter wrote:
> Exactly. Stop apache, nginx, mysql, postgres and tell me, how many
> children processes will be left, once you stopped it.
> 
> As ssh behaves differently and keeps the children alive, you have to
> handle it differently :-)
Sure... 

But I mean before thinking about "how to handle it technically",... it
should probably decided "what do we actually want".
Like "even if ssh itself (the binary) behaves different from apache/etc.
- do we want to keep that in the unit-files   or   do we want "stop" to
generally mean that everything from that service is stopped".

I personally would tend to the later, though this may have many
implications...

And when you're right, that systemd restart == stop + start... then this
would be anyway difficult.


> systemctl stop ssh.service => all existing user sessions will stay
> alive
Sure, I know,... due to the
KillMode=process


Cheers,
Chris.

[smime.p7s (application/x-pkcs7-signature, attachment)]

Message #65 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Russ Allbery <rra@debian.org>

To: Christoph Anton Mitterer <calestyo@scientia.net>

Cc: 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Sat, 11 Oct 2014 18:33:39 -0700

Christoph Anton Mitterer <calestyo@scientia.net> writes:

> But I mean before thinking about "how to handle it technically",... it
> should probably decided "what do we actually want".  Like "even if ssh
> itself (the binary) behaves different from apache/etc.  - do we want to
> keep that in the unit-files or do we want "stop" to generally mean that
> everything from that service is stopped".

> I personally would tend to the later, though this may have many
> implications...

Yes, a lot of people being *quite* upset when they stop ssh to restart it
with debugging or to temporarily bring it down while working on something,
discover that their session was terminated in a way that's never happened
with ssh in the past, and now be unable to connect to the system since it
was a remote server.

Let's not do that.  That would be really unpleasant.  We need to preserve
the current sshd behavior that stopping the service does *not* kill open
sessions.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>

Message #70 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Christoph Anton Mitterer <calestyo@scientia.net>

To: 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Sun, 12 Oct 2014 04:12:07 +0200

[Message part 1 (text/plain, inline)]

On Sat, 2014-10-11 at 18:33 -0700, Russ Allbery wrote: 
> Yes, a lot of people being *quite* upset when they stop ssh to restart it
> with debugging or to temporarily bring it down while working on something,
> discover that their session was terminated in a way that's never happened
> with ssh in the past, and now be unable to connect to the system since it
> was a remote server.
> 
> Let's not do that.  That would be really unpleasant.  We need to preserve
> the current sshd behavior that stopping the service does *not* kill open
> sessions.

Well never too late to change something *if* it was the cleaner way to
handle it. =)

Anyway,... what would you propose then to bring everything under one
hat?

2 service files?

Or even three? Like
ssh.service
sshd.service
ssh-sessions.service
with:
- sshd.service being the daemon, that, if stopped/restarted, leaves
ssh-sessions.service alone
- ssh-session.service being the one that stops the ssh sessions (either
manually, when the admin wants it, or on shutdown)
- and ssh.service being simply something that depends on both (i.e.
stops both and starts sshd.serice)
?


Cheers,
Chris.

[smime.p7s (application/x-pkcs7-signature, attachment)]

Message #75 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Russ Allbery <rra@debian.org>

To: Christoph Anton Mitterer <calestyo@scientia.net>

Cc: 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Sat, 11 Oct 2014 20:12:56 -0700

Christoph Anton Mitterer <calestyo@scientia.net> writes:

> Well never too late to change something *if* it was the cleaner way to
> handle it. =)

But having stopping sshd kill open sessions is not cleaner.  It would be a
pretty serious regression.

> Anyway,... what would you propose then to bring everything under one
> hat?

I have no problems with the current behavior under systemd, so I'm not the
one to ask for a solution.  It makes no difference at all to me whether I
get a clean connection shutdown from a host when it's being rebooted.
That didn't reliably happen even under sysvinit-started sshd.

If you can find a way to improve the behavior along some axis that you
care about, I'm certainly fine with that, but given that I don't even
consider the problem that you're trying to solve to be a problem, I'm
going to have a low tolerance for regressions.  :)

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>

Message #80 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Christoph Anton Mitterer <calestyo@scientia.net>

To: 751636@bugs.debian.org

Cc: control@bugs.debian.org

Subject: Re: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Wed, 12 Nov 2014 19:33:00 +0100

[Message part 1 (text/plain, inline)]

retitle 751636 ssh sessions are not cleanly terminated on shutdown/restart with systemd
stop

Hey.

Some updates on this:

1) I recently completely reworked my sshd_config, and since then, when I
shutdown the server where sshd runs upon, I do get logouts on the
clients that are connected to it.
The strange thing though is, that the way the client is logged out
differs from time to time.
Sometimes it says:
Terminated
Connection to myhost closed by remote host.
Connection to myhost closed.

Sometimes just:
Connection to myhost closed by remote host.
Connection to myhost closed.

So it still seems that something is a bit fishy there.



2) I've just experienced another behaviour though I'm not sure whether
it's actually systemd related or some other bug in ssh itself.
What happened is, that I rebooted the client node and after log in back
to the server node, I accidentally noted that the former ssh connected
was still running there (since aptitude was run in it, and it hat dpkg
locked).
This also shows the potential deeper effects of this issue, i.e. apps
not working correctly anymore due to locking and similar things.
Now I do have TCPKeepAlives disabled, but I have ClientAlive* enabled...
so it's just a matter of time till sshd would have killed it,..
nevertheless it's not quite nice the way it is.

I just cannot reproduce right now whether this was actually caused by
systemd on the client (i.e. networking being stopped, before the ssh
process was terminated, and thus no clean disconnection happening) or
whether it's some issue in ssh itself.

The later could be the case, because I was trying the newest version of
needrestart, which suggested detected gdm.service to require a restart,
and even though it was not checked, it still killed my Cinnamon session,
and with that it should have also killed any gnome-terminal process in
it and thus also the ssh.
Only after that happened I've rebooted, so it could also be the case
that the session was already dead but not cleanly disconnected, long
before systemd came into play.


Cheers,
Chris.

[smime.p7s (application/x-pkcs7-signature, attachment)]

Message #87 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Christoph Anton Mitterer <calestyo@scientia.net>

To: 751636@bugs.debian.org

Subject: Re: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Thu, 13 Nov 2014 04:18:31 +0100

[Message part 1 (text/plain, inline)]

On Wed, 2014-11-12 at 19:33 +0100, Christoph Anton Mitterer wrote: 
> 1) ...
> So it still seems that something is a bit fishy there.

I've now also seen the following:
root@otherHost:~# poweroff 
Terminated
You have mail in /var/mail/root
root@otherHost:~# Connection to otherHost closed by remote host.
Connection to otherHost closed.
user@localHost:~$ 

Which seems even more strange:
1st - Something is Terminated (okay at first I'd have guessed this is
bash on the remote host

2nd - "You have mail in /var/mail/root" from the *remote host* is given,
so bash is still alive, the ssh process there is obviously still alive
as well... so what's Terminated?

o.O

[smime.p7s (application/x-pkcs7-signature, attachment)]

Message #92 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Russ Allbery <rra@debian.org>

To: Christoph Anton Mitterer <calestyo@scientia.net>

Cc: 751636@bugs.debian.org

Subject: Re: Bug#751636: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Wed, 12 Nov 2014 21:32:42 -0800

Christoph Anton Mitterer <calestyo@scientia.net> writes:

> 1) I recently completely reworked my sshd_config, and since then, when I
> shutdown the server where sshd runs upon, I do get logouts on the
> clients that are connected to it.
> The strange thing though is, that the way the client is logged out
> differs from time to time.
> Sometimes it says:
> Terminated
> Connection to myhost closed by remote host.
> Connection to myhost closed.

> Sometimes just:
> Connection to myhost closed by remote host.
> Connection to myhost closed.

> So it still seems that something is a bit fishy there.

I believe the former happens when process shutdown kills your shell before
your sshd, and the latter happens when the processes get killed in the
opposite order.  (bash prints "Terminated" when it catches a signal and
exits.)  Usually all running processes are killed "at the same time"
during shutdown, so the order is not deterministic.  I think this is
purely cosmetic.

> I just cannot reproduce right now whether this was actually caused by
> systemd on the client (i.e. networking being stopped, before the ssh
> process was terminated, and thus no clean disconnection happening) or
> whether it's some issue in ssh itself.

It doesn't really matter, since the client can go away without sending a
FIN in a ton of other ways.  If you care, you should set ClientAlive* or
TCPKeepAlive, like you have.  I've seen this behavior for years, with or
without systemd.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>

Message #97 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Russ Allbery <rra@debian.org>

To: Christoph Anton Mitterer <calestyo@scientia.net>

Cc: 751636@bugs.debian.org

Subject: Re: Bug#751636: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Wed, 12 Nov 2014 21:34:57 -0800

Christoph Anton Mitterer <calestyo@scientia.net> writes:

> I've now also seen the following:
> root@otherHost:~# poweroff 
> Terminated
> You have mail in /var/mail/root
> root@otherHost:~# Connection to otherHost closed by remote host.
> Connection to otherHost closed.
> user@localHost:~$ 

> Which seems even more strange:
> 1st - Something is Terminated (okay at first I'd have guessed this is
> bash on the remote host

> 2nd - "You have mail in /var/mail/root" from the *remote host* is given,
> so bash is still alive, the ssh process there is obviously still alive
> as well... so what's Terminated?

I believe the "Terminated" message is from poweroff itself getting killed,
which bash notices and prints a "Terminated" and then runs its normal
prompt commands.  That results in the mail notification, and then sshd is
killed and you lose the connection.

In other words, this is just the same issue, only there are three
processes that may be killed in some random order instead of two.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>

Message #102 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Luca Falavigna <dktrkranz@debian.org>

To: 751636@bugs.debian.org

Subject: Severity bump

Date: Thu, 27 Nov 2014 17:30:59 +0100

severity 751636 important
thanks


I believe this bug is quite important, and should deserve a fix in
time for Jessie, hence the severity bump.

Cheers,
Luca

Message #109 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Russ Allbery <rra@debian.org>

To: Luca Falavigna <dktrkranz@debian.org>

Cc: 751636@bugs.debian.org

Subject: Re: Bug#751636: Severity bump

Date: Thu, 27 Nov 2014 10:29:44 -0800

Luca Falavigna <dktrkranz@debian.org> writes:

> I believe this bug is quite important, and should deserve a fix in
> time for Jessie, hence the severity bump.

Er, why?  Have you read the discussion in the bug?

I continue to believe that this is not even a bug at all, let alone an
important one.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>

Message #114 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Marc Haber <mh+debian-packages@zugschlus.de>

To: Tom Hutter <tom.hutter@unixum.de>, 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Sat, 13 Dec 2014 14:58:57 +0100

On Fri, Oct 10, 2014 at 11:52:24AM +0200, Tom Hutter wrote:
> The solution provided under 
> https://bugzilla.redhat.com/show_bug.cgi?id=626477 suggests to killall ssh 
> sessions, when stopping sshd. I prefer to have at least one ssh session 
> open to my server when restarting ssh, if something goes wrong.

Agreed. This change would not be acceptable.

> Therefore I added a service, to solve this under the current Debian 
> Wheezy version I am running.
> 
> /etc/systemd/system/ssh-user-sessions.service:
> 
> [Unit]
> Description=Shutdown all ssh sessions before network
> After=network.target
> 
> [Service]
> TimeoutStartSec=0
> Type=oneshot
> RemainAfterExit=yes
> ExecStart=/bin/true
> ExecStop=/usr/bin/killall sshd
> 
> When starting, this service does simply nothing (/bin/true). But due to the 
> statement "After=network.target" it kills all ssh processes before shutting 
> down the network.

I had always thought that systemd would group processes together in
cgroups to be able to kill them more elegantly on shutdown. I'd like
to see that feature used instead of resorting to the old sledgehammer
method.

Greetings
Marc


-- 
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany    |  lose things."    Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600420

Message #119 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Marc Haber <mh+debian-packages@zugschlus.de>

To: Christoph Anton Mitterer <calestyo@scientia.net>, 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Sat, 13 Dec 2014 15:01:36 +0100

On Fri, Oct 10, 2014 at 05:44:14PM +0200, Christoph Anton Mitterer wrote:
> a) if sshd crashes or when it is restarted/reloaded (or when the network
> is restarted), we do not want ssh sessions to terminate, right?

At least that's what we always had. I would like to have a method to
kill all ssh sessions with the exception of my own ones, or the single
session that I happen to type the command restarting sshd in.

> b) we want a way to actually stop user sessions... not only for this
> particular bug (i.e. on shutdown), but as a locally logged in sysadmin
> I'd also like to say "okay... away with sshd and it's users".

Agreed. We didn't have that until jessie though.

> AFAICS, this should solve (a) and (b), the only difference to now would
> be, that we need to educate our users/admins, that "systemctl stop ssh"
> really means "all ssh stops" and not just "the main ssh daemon stops but
> old connections remain".

That would be a pretty severe change from the behavior we used to have
for fifteen years. I also guess it would be used as an argument
against systemd as a whole.

Please consider not changing this behavior to keep system acceptance up.

Greetings
Marc

-- 
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany    |  lose things."    Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600420

Message #124 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Marc Haber <mh+debian-packages@zugschlus.de>

To: Tom Hutter <tom.hutter@unixum.de>, 751636@bugs.debian.org

Cc: Christoph Anton Mitterer <calestyo@scientia.net>

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Sat, 13 Dec 2014 15:06:46 +0100

On Fri, Oct 10, 2014 at 06:51:20PM +0200, Tom Hutter wrote:
> If ssh is restarted, my hack does exactly the expected. The ssh user 
> sessions stay alive. Only, if the network is shut down, the sessions will be 
> terminated. Looking at this excerpt from systemd.special(7), I would say, 
> my hack does exactly the expected. What is a session depending on 
> network good, once the network has gone away?
> 
>        network.target
>            This unit is supposed to indicate when network functionality is
>            available, but it is only very weakly defined what that is supposed
>            to mean, with one exception: at shutdown, a unit that is ordered
>            after network.target will be stopped before the network -- to
>            whatever level it might be set up then -- is shut down. Also see
>            Running Services After the Network is up[1] for more information.
>            Also see network-online.target described above.

/lib/systemd/system/ssh.service in current sid has
"After=network.target" in its Unit stanza and still not cleanly kills
off ssh sessions.

There is also /lib/systemd/system/ssh@.service which seems to be
contrary to /lib/systemd/system/ssh.service which I do not understand.

> I think it's more a philosophical question, if running processes depending 
> on the network should survive a network restart

They usually did in the past, /etc/init.d/network stop;
/etc/init.d/network start returned you to the shell and did not kill
the session.

Greetings
Marc

-- 
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany    |  lose things."    Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600420

Message #129 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Marc Haber <mh+debian-packages@zugschlus.de>

To: Russ Allbery <rra@debian.org>, 751636@bugs.debian.org

Cc: Christoph Anton Mitterer <calestyo@scientia.net>

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Sat, 13 Dec 2014 15:09:15 +0100

On Sat, Oct 11, 2014 at 06:33:39PM -0700, Russ Allbery wrote:
> Christoph Anton Mitterer <calestyo@scientia.net> writes:
> > But I mean before thinking about "how to handle it technically",... it
> > should probably decided "what do we actually want".  Like "even if ssh
> > itself (the binary) behaves different from apache/etc.  - do we want to
> > keep that in the unit-files or do we want "stop" to generally mean that
> > everything from that service is stopped".
> 
> > I personally would tend to the later, though this may have many
> > implications...
> 
> Yes, a lot of people being *quite* upset when they stop ssh to restart it
> with debugging or to temporarily bring it down while working on something,
> discover that their session was terminated in a way that's never happened
> with ssh in the past, and now be unable to connect to the system since it
> was a remote server.
> 
> Let's not do that.  That would be really unpleasant.  We need to preserve
> the current sshd behavior that stopping the service does *not* kill open
> sessions.

Amen.

Greetings
Marc

-- 
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany    |  lose things."    Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600420

Message #134 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Marc Haber <mh+debian-packages@zugschlus.de>

To: Christoph Anton Mitterer <calestyo@scientia.net>, 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Sat, 13 Dec 2014 15:10:44 +0100

On Sun, Oct 12, 2014 at 04:12:07AM +0200, Christoph Anton Mitterer wrote:
> On Sat, 2014-10-11 at 18:33 -0700, Russ Allbery wrote: 
> > Yes, a lot of people being *quite* upset when they stop ssh to restart it
> > with debugging or to temporarily bring it down while working on something,
> > discover that their session was terminated in a way that's never happened
> > with ssh in the past, and now be unable to connect to the system since it
> > was a remote server.
> > 
> > Let's not do that.  That would be really unpleasant.  We need to preserve
> > the current sshd behavior that stopping the service does *not* kill open
> > sessions.
> 
> Well never too late to change something *if* it was the cleaner way to
> handle it. =)

That would be the systemd way to do it and instantly spawn a new hate
wave. After all, it was systemd locking people out of their headless,
remote systems during an urgent security update.

Don't change this behavior. Please.

Greetings
Marc


-- 
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany    |  lose things."    Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600420

Message #139 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Marc Haber <mh+debian-packages@zugschlus.de>

To: Russ Allbery <rra@debian.org>, 751636@bugs.debian.org

Cc: Christoph Anton Mitterer <calestyo@scientia.net>

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Sat, 13 Dec 2014 15:14:22 +0100

On Sat, Oct 11, 2014 at 08:12:56PM -0700, Russ Allbery wrote:
> I have no problems with the current behavior under systemd, so I'm not the
> one to ask for a solution.  It makes no difference at all to me whether I
> get a clean connection shutdown from a host when it's being rebooted.

It makes a big difference to me when working with multiple hosts at
once. I will only get back my shell after a manual action or after
waiting for the host to return to the network to finally send an RST.

> That didn't reliably happen even under sysvinit-started sshd.

It happened reliably in so many cases that I don't even remember the
last time where a sysvinit system didn't return me cleanly and quickly
to my shell on shutdown. It was a big WTF when my first systemd system
behaved that way.

If sysvinit were so unreliable in its behavior on shutdown, I would
have been at least a bit familiar with the "new" behavior, and I do
not like it at all.

This change will not improve systemd's behavior, as it is one more
place where the self-proclaimed "drop-in replacement to sysvinit"
changes system behavior in a _very_ prominent way.

> If you can find a way to improve the behavior along some axis that you
> care about, I'm certainly fine with that, but given that I don't even
> consider the problem that you're trying to solve to be a problem, I'm
> going to have a low tolerance for regressions.  :)

jftr, I see the changed behavior as a problem.

Greetings
Marc


-- 
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany    |  lose things."    Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600420

Message #144 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Marc Haber <mh+debian-packages@zugschlus.de>

To: David Lechner <david@lechnology.com>, 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Sat, 13 Dec 2014 15:16:19 +0100

On Fri, Oct 10, 2014 at 10:51:04AM -0500, David Lechner wrote:
> I have found that using ssh.socket instead of ssh.service makes
> clients shut down cleanly most of the time (sometimes they will
> still hang, but it is rare).
> 
>     systemctl disable ssh.service
>     systemctl enable ssh.socket

Do I understand correctly, that ssh in jessie/sid allows the local
admin to run sshd as a traditional daemon, with the new (undesired)
behavior, or as a systemd service with socket activation, which
gives a better emulation of traditional behavior?

If this is really seriously meant that way, people will see this as a
conspiracy to coax people into using socket activation. Danger, Will
Robinson.

Greetings
Marc

-- 
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany    |  lose things."    Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600420

Message #149 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Marc Haber <mh+debian-packages@zugschlus.de>

To: Russ Allbery <rra@debian.org>, 751636@bugs.debian.org

Cc: Luca Falavigna <dktrkranz@debian.org>

Subject: Re: Bug#751636: Severity bump

Date: Sat, 13 Dec 2014 15:19:29 +0100

On Thu, Nov 27, 2014 at 10:29:44AM -0800, Russ Allbery wrote:
> Luca Falavigna <dktrkranz@debian.org> writes:
> > I believe this bug is quite important, and should deserve a fix in
> > time for Jessie, hence the severity bump.
> 
> Er, why?  Have you read the discussion in the bug?
> 
> I continue to believe that this is not even a bug at all, let alone an
> important one.

jftr, I see this as an unnecessary behavioral change in a quite
prominent place which will be seen as systemd's fault by a gazillion
of people, including me. I do not like the new behavior at all, see
this as a clear bug and find the important severity justified.

Greetings
Marc

-- 
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany    |  lose things."    Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600420

Message #154 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Christoph Anton Mitterer <calestyo@scientia.net>

To: 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Sat, 13 Dec 2014 19:04:30 +0100

[Message part 1 (text/plain, inline)]

On Sat, 2014-12-13 at 15:01 +0100, Marc Haber wrote: 
> I would like to have a method to
> kill all ssh sessions with the exception of my own ones, or the single
> session that I happen to type the command restarting sshd in.
Since there is no definition of "which are yours" (the ones logged in to
your current user? sessions started by your current user?) this is going
to be difficult, apart from the fact, that one would need a way to find
out such information.

Anyway. This kind of process management is not what the init-system
should be there for.


> > b) we want a way to actually stop user sessions... not only for this
> > particular bug (i.e. on shutdown), but as a locally logged in sysadmin
> > I'd also like to say "okay... away with sshd and it's users".
> Agreed. We didn't have that until jessie though.
Sure,... I didn't claim that we had that before.


> > AFAICS, this should solve (a) and (b), the only difference to now would
> > be, that we need to educate our users/admins, that "systemctl stop ssh"
> > really means "all ssh stops" and not just "the main ssh daemon stops but
> > old connections remain".
> 
> That would be a pretty severe change from the behavior we used to have
> for fifteen years. I also guess it would be used as an argument
> against systemd as a whole.
First of all,... I just put that up for open discussion... i.e. the
questions:
When we'd start from scratch with the OS, and would ask ourselves "what
should happen when I type 'stop service XYZ'"... would that be to only
stop the listener, or to stop anything related to that service (i.e.
also any sessions, like ongoing httpd connections or that like).

I think the later would be the cleaner way of (generally) handling
things... and I'm well aware that this would change current behaviour.
OTOH, people aren't stupid and when you tell them why a current
behaviour changes and that it's for a better design of the system,..
they likely can adapt to it.
And in general, one cannot stop progress just because there are some
backwards-minded people who never want to have ill-designed things to
change.

Apart from that - *if* such change in behaviour would really come, it
would have nothing to do with systemd.

[smime.p7s (application/x-pkcs7-signature, attachment)]

Message #159 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Christoph Anton Mitterer <calestyo@scientia.net>

To: 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Sat, 13 Dec 2014 19:14:53 +0100

[Message part 1 (text/plain, inline)]

On Sat, 2014-12-13 at 15:06 +0100, Marc Haber wrote: 
> /lib/systemd/system/ssh.service in current sid has
> "After=network.target" in its Unit stanza and still not cleanly kills
> off ssh sessions.
Since the ssh.service unit file only starts the listener daemons and not
the sessions neither explicitly stops the session processes... this is
absolutely expected behaviour.


> There is also /lib/systemd/system/ssh@.service which seems to be
> contrary to /lib/systemd/system/ssh.service which I do not understand.
I guess it should be easy to find out what this is about, in any way,
explaining systemd functionality doesn't belong to this bug.




On Sat, 2014-12-13 at 15:10 +0100, Marc Haber wrote:
> That would be the systemd way to do it
No, it would be the way to improve things when the were done wrong in
the past,... which you've had always during the whole history of UNIX,
and which neither contradicts any of UNIX' paradigms like "small is
nice" and that like.

> and instantly spawn a new hate
> wave. After all, it was systemd locking people out of their headless,
> remote systems during an urgent security update.
Since such behavioural change wouldn't depend on the initsystem, just
stubborn people who annoy us with their systemd FUD and hatred anyway
would do so...




On Sat, 2014-12-13 at 15:16 +0100, Marc Haber wrote:
> Do I understand correctly, that ssh in jessie/sid allows the local
> admin to run sshd as a traditional daemon, with the new (undesired)
> behavior, or as a systemd service with socket activation, which
> gives a better emulation of traditional behavior?
No, you don't understand correctly.
In daemon mode, the "new" behaviour is a bug, reported in this very
ticket.
In socket mode, each session is controlled by what compares to the
listener daemon (i.e. the process which is directly managed by systemd),
therefore one get's the fix for free, as this process is stopped by
systemd.


> If this is really seriously meant that way, people will see this as a
> conspiracy to coax people into using socket activation.
There is no conspiracy, the default mode of ssh is still the daemon
based.

[smime.p7s (application/x-pkcs7-signature, attachment)]

Message #164 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Christoph Anton Mitterer <calestyo@scientia.net>

To: 751636@bugs.debian.org

Subject: Re: Bug#751636: Severity bump

Date: Sat, 13 Dec 2014 19:15:40 +0100

[Message part 1 (text/plain, inline)]

On Sat, 2014-12-13 at 15:19 +0100, Marc Haber wrote: 
> prominent place which will be seen as systemd's fault by a gazillion
> of people, including me.

Could you please place that systemd hatred somewhere else (we have
countless of such threads open where people could place there nonsense)?

ssh sessions were in principle not separately killed off during sysvinit
either, and just got so by accident early enough, which is not the case
in systemd, as it shut downs the system more concurrently.

There is a bug open to deal with that issue (which won't affect you
anyway when you don't switch to it).

People are not helped at all when you clutter this bug with anti-systemd
stuff.


Cheers,
Chris.

[smime.p7s (application/x-pkcs7-signature, attachment)]

Message #169 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Marc Haber <mh+debian-bugs@zugschlus.de>

To: Christoph Anton Mitterer <calestyo@scientia.net>, 751636@bugs.debian.org

Subject: Re: Bug#751636: Severity bump

Date: Sun, 14 Dec 2014 15:40:35 +0100

Hi,

On Sat, Dec 13, 2014 at 07:15:40PM +0100, Christoph Anton Mitterer wrote:
> On Sat, 2014-12-13 at 15:19 +0100, Marc Haber wrote: 
> > prominent place which will be seen as systemd's fault by a gazillion
> > of people, including me.
> 
> Could you please place that systemd hatred somewhere else

Please try to take matter-of-fact criticism not as hatred. I do
actually like some of systemd's concepts.

> ssh sessions were in principle not separately killed off during sysvinit
> either, and just got so by accident early enough, which is not the case
> in systemd, as it shut downs the system more concurrently.

systemd's behavior has some clear disadvantages which show themselves
evidently in server management.

> There is a bug open to deal with that issue (which won't affect you
> anyway when you don't switch to it).

I _want_ to switch to systemd. Before I do so, I'd like to get rid of
some nuisances. Nuisances are not the side effect of switching to
systemd, they are the side effect of every switch that is done. Most
software projects deal with those reports professionally without
taking the critique personally.

Greetings
Marc

-- 
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany    |  lose things."    Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600420

Message #174 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Marc Haber <mh+debian-bugs@zugschlus.de>

To: Christoph Anton Mitterer <calestyo@scientia.net>, 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Sun, 14 Dec 2014 15:46:26 +0100

Hi,

On Sat, Dec 13, 2014 at 07:04:30PM +0100, Christoph Anton Mitterer wrote:
> On Sat, 2014-12-13 at 15:01 +0100, Marc Haber wrote: 
> > I would like to have a method to
> > kill all ssh sessions with the exception of my own ones, or the single
> > session that I happen to type the command restarting sshd in.
> Since there is no definition of "which are yours" (the ones logged in to
> your current user? sessions started by your current user?) this is going
> to be difficult, apart from the fact, that one would need a way to find
> out such information.
> 
> Anyway. This kind of process management is not what the init-system
> should be there for.

I am aware of that. I was just dreaming of a perfect solution, very
well knowing that I won't get it RSN.

> > That would be a pretty severe change from the behavior we used to have
> > for fifteen years. I also guess it would be used as an argument
> > against systemd as a whole.
> First of all,... I just put that up for open discussion... i.e. the
> questions:
> When we'd start from scratch with the OS, and would ask ourselves "what
> should happen when I type 'stop service XYZ'"... would that be to only
> stop the listener, or to stop anything related to that service (i.e.
> also any sessions, like ongoing httpd connections or that like).

Yes, but GNU/Linux is not a "grüne wiese" approach, it is migrating
millions of existing systems.

Even a classic Unix leaves many things to be hated, but it's just what
we are used to, and which is not easily changed.

> OTOH, people aren't stupid and when you tell them why a current
> behaviour changes and that it's for a better design of the system,..
> they likely can adapt to it.

Locking a shell for a two-digit number of seconds in a ordinare usage
situation is something I would not want to adapt to even if the
underlying system was perfect.

> And in general, one cannot stop progress just because there are some
> backwards-minded people who never want to have ill-designed things to
> change.

I fail to see why it is "progress" to save five seconds of shutdown
time for the machine while the same change wastes half a minute or
more for the human.

Greetings
Marc, well aware that the last sentence will be misinterpreted as
systemd hatred again, which it is not


-- 
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany    |  lose things."    Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600420

Message #179 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Marc Haber <mh+debian-bugs@zugschlus.de>

To: Christoph Anton Mitterer <calestyo@scientia.net>, 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Sun, 14 Dec 2014 16:19:43 +0100

On Sat, Dec 13, 2014 at 07:14:53PM +0100, Christoph Anton Mitterer wrote:
> On Sat, 2014-12-13 at 15:06 +0100, Marc Haber wrote: 
> > /lib/systemd/system/ssh.service in current sid has
> > "After=network.target" in its Unit stanza and still not cleanly kills
> > off ssh sessions.
> Since the ssh.service unit file only starts the listener daemons and not
> the sessions neither explicitly stops the session processes... this is
> absolutely expected behaviour.

It might be expected by somebody very familiar with how new init
works. It is surprising to people who aren't and undresired by some of
them.

> On Sat, 2014-12-13 at 15:10 +0100, Marc Haber wrote:
> > That would be the systemd way to do it
> No, it would be the way to improve things when the were done wrong in
> the past,

"Wrong" is subjective. It might also be something people are used to
for decades.

> > and instantly spawn a new hate
> > wave. After all, it was systemd locking people out of their headless,
> > remote systems during an urgent security update.
> Since such behavioural change wouldn't depend on the initsystem, just
> stubborn people who annoy us with their systemd FUD and hatred anyway
> would do so...

Unfortunately, there are many of those people.

Btw, with systemd sticking its head in many parts of the system that
are traditionally not an init's job, it is hard to judge whether a
change is introduced by systemd or not. People are going to take the
easiest route to vent.

> On Sat, 2014-12-13 at 15:16 +0100, Marc Haber wrote:
> > Do I understand correctly, that ssh in jessie/sid allows the local
> > admin to run sshd as a traditional daemon, with the new (undesired)
> > behavior, or as a systemd service with socket activation, which
> > gives a better emulation of traditional behavior?
> No, you don't understand correctly.
> In daemon mode, the "new" behaviour is a bug, reported in this very
> ticket.

As long as we agree on this being a bug, we are fine with each other.

> In socket mode, each session is controlled by what compares to the
> listener daemon (i.e. the process which is directly managed by systemd),
> therefore one get's the fix for free, as this process is stopped by
> systemd.

After typing systemctl disable ssh.service and systemctl enable
ssh.socket, I still have a running sshd -D, and rebooting the system
results in a hanging shell.

After the reboot, there is no sshd -D under systemd, my connection
closes immediately after typing shutdown -r now, but the shutdown
message is still not printed. I can therfore not be sure whether it
was a random ion storm in the kernel that disconnected me or the
shutdown that I initiated.

> > If this is really seriously meant that way, people will see this as a
> > conspiracy to coax people into using socket activation.
> There is no conspiracy, the default mode of ssh is still the daemon
> based.

You know that, I know that. But if the daemon mode has new
shortcomings compared to what we are used to, and the canonical answer
is "switch to socket activation", we're screwed.

Greetings
Marc

-- 
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany    |  lose things."    Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600420

Message #184 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Christoph Anton Mitterer <calestyo@scientia.net>

To: 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Sun, 14 Dec 2014 17:04:13 +0100

[Message part 1 (text/plain, inline)]

Answering two of your mails in one, since all of this got really off
topic.



On Sun, 2014-12-14 at 15:46 +0100, Marc Haber wrote: 
> Yes, but GNU/Linux is not a "grüne wiese" approach, it is migrating
> millions of existing systems.
Sure, but this doesn't mean that one cannot pick out single points,
think about whether there is a conceptually better way (I didn't even
say that I'm 100% sure that my proposed way is really the conceptually
better one)... and migrate to this when feasible.
We had much harder migrations (multiarch, etc.) than something like
telling people "if tell your init systemd to stop XYZ, it's sessions
will stop as well".


> Locking a shell for a two-digit number of seconds in a ordinare usage
> situation is something I would not want to adapt to even if the
> underlying system was perfect.
I don't get what you want to say... *if* my proposed change would be
chosen, then stopping ssh would stop all active sessions... not just
freeze them for n seconds.


> I fail to see why it is "progress" to save five seconds of shutdown
> time for the machine while the same change wastes half a minute or
> more for the human.
Again I don't really see where anything here stops something for half a
minute or more... but personally said: being a faster init system (in
some occasions) is the least interesting thing for me about systemd
(actually I don't care at all about boot speed).
The real advancements are clean dependency expression mechanisms,
guarantees like "I can prevent my daemon to start if the firewall rules
didn't load", etc.



On Sun, 2014-12-14 at 15:40 +0100, Marc Haber wrote:
Please try to take matter-of-fact criticism not as hatred. I do
> actually like some of systemd's concepts.
> Well I think I've showed that most of the things you've criticised
have nothing to with systemd itself, especially not any proposals to
change the behaviour of "ssh stop" - which was just a proposal by myself
and nothing form the package maintainers.

The only "valid" criticism was that due to how systemd shuts down (i.e.
more parallel) we see this bug.
And I doubt that other software (neither upstart nor sysvinit) is free
of bugs.


systemd's behavior has some clear disadvantages which show themselves
> evidently in server management.
> Which was discussed over and over again on general lists, and which
has really nothing to do with this bug.


I _want_ to switch to systemd. Before I do so, I'd like to get rid of
> some nuisances. Nuisances are not the side effect of switching to
> systemd, they are the side effect of every switch that is done. Most
> software projects deal with those reports professionally without
> taking the critique personally.
> Well that's why this bug (and others) are reported, and I haven't seen
any of the systemd maintainers here, complaining against the bug (which
is btw. rather a bug in ssh in that it never had a clean way of session
termination).
You in contrast mentioned every second sentence that this was all
systemd's fault, that there would be conspiracies, etc..

I personally do have several points of critic in systemd as well
(starting from that we more or less only try to rebuild sysvinit in
systemd, without really using the full range of systemd features... over
political questions that more and more base things are concentrated in
one projects).
These are things which one can objectively discuss in the appropriate
places.
But even I'm really fed up with all the FUD spread at in all kinds of
unrelated places like this.

[smime.p7s (application/x-pkcs7-signature, attachment)]

Message #189 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Christoph Anton Mitterer <calestyo@scientia.net>

To: 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Sun, 14 Dec 2014 17:04:17 +0100

[Message part 1 (text/plain, inline)]

On Sun, 2014-12-14 at 16:19 +0100, Marc Haber wrote: 
> It might be expected by somebody very familiar with how new init
> works. It is surprising to people who aren't and undresired by some of
> them.
Again, this is why we have this bug. And when I talked about "absolutely
expected behaviour" I meant when you look at the system and wonder why
does XYZ happen... if you now then how things work, it's not unexpected
or strange that systemd works as it does.


> As long as we agree on this being a bug, we are fine with each other.
Well sure... haven't you seen *who* reported that issue first? Me.


> After typing systemctl disable ssh.service and systemctl enable
> ssh.socket, I still have a running sshd -D
systemd's enable and disable actions are just the same as update-rc.d's
were with sysvinit. They don't start or stop an already running service.

> , and rebooting the system
> results in a hanging shell.
What exactly do you mean,... the shell which you were using to connect
to the host that you're just rebooting from a remote node?
Then again, this is this very bug (i.e. ssh has no init-system
configuration to properly stop it's sessions).

>  but the shutdown
> message is still not printed. I can therfore not be sure whether it
> was a random ion storm in the kernel that disconnected me or the
> shutdown that I initiated.
mhh


> and the canonical answer
> is "switch to socket activation", we're screwed.
For some daemons, socket activation may actually be the preferred way
that should become default (e.g. why should I need to have a local cups
daemon running all time, if I only print every once in a while)... for
others (e.g. ssh) I doubt that socket based activation will get default,
because it have disadvantages (like the delay in ssh parsing the
configuration file)

Also note that systemd provides here actually more than two ways of
starting things up:

/lib/systemd/system/ssh.service
=> the "classic" way of having a daemon run at boot, which listens for
incomming connections, spawning session processes

/lib/systemd/system/ssh.socket
=> socket based activation which starts the plain old listener
(ssh.service) the first time someone uses the socket

/lib/systemd/system/ssh@.service
=> spawn an inetd-style ssh just for that single connection, when
someone connects to the socket which systemd listens on.

[smime.p7s (application/x-pkcs7-signature, attachment)]

Message #194 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Russ Allbery <rra@debian.org>

To: Marc Haber <mh+debian-bugs@zugschlus.de>

Cc: Christoph Anton Mitterer <calestyo@scientia.net>, 751636@bugs.debian.org

Subject: Re: Bug#751636: Severity bump

Date: Sun, 14 Dec 2014 09:28:02 -0800

Marc Haber <mh+debian-bugs@zugschlus.de> writes:

> I _want_ to switch to systemd. Before I do so, I'd like to get rid of
> some nuisances. Nuisances are not the side effect of switching to
> systemd, they are the side effect of every switch that is done. Most
> software projects deal with those reports professionally without
> taking the critique personally.

On the point of your last sentence, please note that no one who has
commented on this bug to date (including both Christoph and I) are
involved in development of either systemd, openssh, or the Debian
packaging of either.  This is all just the peanut gallery.  :)

Personally, I would be happy to see this bug fixed; I just don't consider
it to be of important severity, since I routinely see the same behavior
when shutting down servers right now, in wheezy, using sysvinit.  I don't
have a good explanation for why you haven't seen this behavior under
sysvinit, but for me it's not a regression, and Christoph's proposed fix
*would* have been a regression, which is why I spoke up just to defend my
own interests.  :)  If the problem can be resolved without regressing,
that would be a clear improvement and I'm all in favor of that.  Whether
it's important enough to go into the next release is, fundamentally, the
call of the release managers, not any of the peanut gallery debating it on
this thread.  And, of course, we need a fix first before we can even talk
about whether it can go into the release.

Given that this bothers you, I'd love to see a fix, since I don't like
seeing people running into undesired behavior!  I'm just not sure how best
to fix it.  The best idea that I can think of is a separate unit that
doesn't run on upgrades but that runs on shutdown prior to the network
being shut down and kills all the ssh child processes.

Note, on a point you made in one of your other messages, that I don't
think systemd can use cgroups to cleanly shut this down because systemd
explicitly uses KillMode=process for sshd precisely to avoid killing the
child processes.  One needs to use one KillMode on a regular shutdown and
a different one when shutting down the system, which is what makes this
tricky.

To answer another question elsewhere in this thread, the ssh@.service runs
sshd inetd-style, with a separate sshd for each incoming connection.
ssh.service (the default) runs it in traditional daemon mode.  Both are
provided so that the local system administrator can switch to inetd-style
if they wish (usually for systems with minimal resources that don't want
to have another long-running daemon), but I believe only ssh.service is
enabled by default.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>

Message #199 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Christoph Anton Mitterer <calestyo@scientia.net>

To: 751636@bugs.debian.org

Subject: Re: Bug#751636: Severity bump

Date: Wed, 17 Dec 2014 01:30:24 +0100

[Message part 1 (text/plain, inline)]

On Sun, 2014-12-14 at 09:28 -0800, Russ Allbery wrote: 
> since I routinely see the same behavior
> when shutting down servers right now, in wheezy, using sysvinit.
This is quite interesting, btw,... cause I've never seen that during
times I've sysvinit.
Have you had any special modifications that could explain that?

>  and Christoph's proposed fix
> *would* have been a regression, which is why I spoke up just to defend my
> own interests.  :)
Well I don't like calling my fix (and it was just one of the proposals)
a regression.
It would be simply a different behaviour.

I simply think, that now when we anyway "massively" change the
init-system, it would be a good opportunity to re-consider what we
actually want.
That was the sole idea of my proposal: I that we should ask ourselfs
what should (all and not just ssh) services do, when they're stopped.


>   If the problem can be resolved without regressing,
> that would be a clear improvement and I'm all in favor of that.  Whether
> it's important enough to go into the next release is, fundamentally, the
> call of the release managers, not any of the peanut gallery debating it on
> this thread.  And, of course, we need a fix first before we can even talk
> about whether it can go into the release.
I'm a bit unsure whether it's that important or not.
But since I'm experiencing that issue, it happens very often to me, that
I log on back to my server, and things like aptitude give me locking
errors, because an old process is still running - my ssh is configured
to timeout after 2min[0],... but even then this is always quite
annoying.


> Given that this bothers you, I'd love to see a fix, since I don't like
> seeing people running into undesired behavior!  I'm just not sure how best
> to fix it.  The best idea that I can think of is a separate unit that
> doesn't run on upgrades but that runs on shutdown prior to the network
> being shut down and kills all the ssh child processes.
I'd probably agree, at least in the likely case, that not general "stop
behaviour" for all services will be introduced which is defined to also
stop the sessions.

Such unit should perhaps be called "ssh-sessions" and it would be nice
if one could use it generally top stop all running ssh-sessions.
As such it would of course need sshd to be stopped already.

And the whole thing should be implemented for sysvinit as well :)


> Note, on a point you made in one of your other messages, that I don't
> think systemd can use cgroups to cleanly shut this down because systemd
> explicitly uses KillMode=process for sshd precisely to avoid killing the
> child processes.  One needs to use one KillMode on a regular shutdown and
> a different one when shutting down the system, which is what makes this
> tricky.
Even then when you change the KillMode,.. would that change anything?
Cause AFAIR, ssh sessions run in their own new cgroups.



Cheers,
Chris.

[0] Though I've experienced at least one case, where it remained living
for at least 20mins... maybe a bug in ssh.. :/

[smime.p7s (application/x-pkcs7-signature, attachment)]

Message #204 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Russ Allbery <rra@debian.org>

To: Christoph Anton Mitterer <calestyo@scientia.net>

Cc: 751636@bugs.debian.org

Subject: Re: Bug#751636: Severity bump

Date: Tue, 16 Dec 2014 17:51:21 -0800

Christoph Anton Mitterer <calestyo@scientia.net> writes:
> On Sun, 2014-12-14 at 09:28 -0800, Russ Allbery wrote: 

>> since I routinely see the same behavior when shutting down servers
>> right now, in wheezy, using sysvinit.

> This is quite interesting, btw,... cause I've never seen that during
> times I've sysvinit.  Have you had any special modifications that could
> explain that?

Not that I'm aware of.  (It was my previous work environment, so I can't
easily experiment right now -- I'm temporarily on a much older environment
using upstart, which is less interesting for this whole discussion.)  We
were running OpenAFS as well, but I don't see why that would make a
difference.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>

Message #207 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Christoph Berg <myon@debian.org>

To: Debian Bug Tracking System <751636@bugs.debian.org>

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Mon, 20 Apr 2015 12:52:57 +0200

[Message part 1 (text/plain, inline)]

Re: Christoph Anton Mitterer 2014-06-15 <20140614232041.29765.83096.reportbug@heisenberg.scientia.net>
> Since openssh-server comes with systemd support, whenever a host
> is shut down or restarted, ssh connections to that host just hang
> and are no longer cleanly terminated (one also doesn't see the
> shutdown message anymore).

Fwiw, I'm also seeing this behavior and I find it utterly annoying.
Sometimes not even <enter>~. works (possibly when ssh is waiting for
a reply from the remote machine). It's the same on CentOS, so it's not
Debian-specific, but we should do something about it.

Christoph
-- 
cb@df7cb.de | http://www.df7cb.de/

[signature.asc (application/pgp-signature, inline)]

Message #212 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Matthew Dawson <matthew@mjdsystems.ca>

To: 751636@bugs.debian.org

Subject: Re: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Wed, 29 Apr 2015 02:57:28 -0400

[Message part 1 (text/plain, inline)]

On Sun, 15 Jun 2014 01:20:41 +0200 Christoph Anton Mitterer
> Hi.
> 
> Since openssh-server comes with systemd support, whenever a host
> is shut down or restarted, ssh connections to that host just hang
> and are no longer cleanly terminated (one also doesn't see the
> shutdown message anymore).
> 
> I'd suspect that systemd might shut down the network before it kills
> the ssh session (or perhaps never kills them at all?)
Hi all,

I ran into this issue on a freshly upgraded VM.  I think I found an 
appropriate work around, installing libpam-systemd.  With that, systemd seems 
to know about my ssh sessions and will close them down before killing the 
network.

I'm not sure if something is still missing, as mosh sessions will sometimes 
linger.  But ssh seems solved.
-- 
Matthew

[smime.p7s (application/pkcs7-signature, attachment)]

Message #217 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Steven Monai <steve@monai.ca>

To: 751636@bugs.debian.org

Subject: Re: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Sat, 02 May 2015 18:55:15 -0700

On Wed, 29 Apr 2015 02:57:28 -0400 Matthew Dawson
<matthew@mjdsystems.ca> wrote:
> On Sun, 15 Jun 2014 01:20:41 +0200 Christoph Anton Mitterer
>> 
>> Since openssh-server comes with systemd support, whenever a host 
>> is shut down or restarted, ssh connections to that host just
>> hang and are no longer cleanly terminated (one also doesn't see
>> the shutdown message anymore).
>> 
>> I'd suspect that systemd might shut down the network before it
>> kills the ssh session (or perhaps never kills them at all?)
> 
> I ran into this issue on a freshly upgraded VM.  I think I found an
>  appropriate work around, installing libpam-systemd.  With that,
> systemd seems to know about my ssh sessions and will close them
> down before killing the network.

Just writing to confirm that this workaround worked for me.

I was having this same problem on a new, minimal Jessie installation.
Installing libpam-systemd (and rebooting) fixed it.

As an aside, it seems strange that libpam-systemd was not installed in
a minimal Jessie install, since systemd recommends it. But perhaps
that's a quirk of the Debian Installer.

-Steven M.

Message #222 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Christoph Anton Mitterer <calestyo@scientia.net>

To: 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Sun, 03 May 2015 04:42:59 +0200

[Message part 1 (text/plain, inline)]

On Sat, 2015-05-02 at 18:55 -0700, Steven Monai wrote: 
> I was having this same problem on a new, minimal Jessie installation.
> Installing libpam-systemd (and rebooting) fixed it.
I somewhat doubt that this is really a fix.
When I reported the problem in the beginning, I've always had
libpam-systemd installed.

At some later point I've also seen the problem occurring far less... but
from time to time it still happens that I have a hanging connection.

So I'd guess it's just coincidence that this improves things a bit.


Cheers,
Chris.

[smime.p7s (application/x-pkcs7-signature, attachment)]

Message #227 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Matt Black <dev@mafro.net>

To: 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Tue, 12 May 2015 10:20:47 +1000

[Message part 1 (text/plain, inline)]

I've been struggling with an appropriate fix for this bug.

I've also found that installing libpam-systemd seems to fix this for me in
my testing. (If I discover it still to be a problem, I'll post again).
However, I'd like to understand why libpam-systemd makes a difference if
anyone has any insight?

Finally, there are issues with using the ssh.socket approach which are not
at first apparent. When exiting an SSH session, all processes created under
that session are terminated - which means tools like screen and tmux will
not work. This may or may not be a problem for you, but it's certainly
means ssh.socket is also not a drop-in replacement like ssh.service.

Cheers
Matt Black

[Message part 2 (text/html, inline)]

Message #232 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Christoph Anton Mitterer <calestyo@scientia.net>

To: Matt Black <dev@mafro.net>, 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Tue, 12 May 2015 02:45:10 +0200

[Message part 1 (text/plain, inline)]

On Tue, 2015-05-12 at 10:20 +1000, Matt Black wrote: 
> I've also found that installing libpam-systemd seems to fix this for
> me in my testing. (If I discover it still to be a problem, I'll post
> again). However, I'd like to understand why libpam-systemd makes a
> difference if anyone has any insight?
It doesn't, I still have the problem occasionally on multiple systems,
all with libpam-systemd installed.
It's pure coincidence. 

> Finally, there are issues with using the ssh.socket approach which are
> not at first apparent. When exiting an SSH session, all processes
> created under that session are terminated - which means tools like
> screen and tmux will not work. This may or may not be a problem for
> you, but it's certainly means ssh.socket is also not a drop-in
> replacement like ssh.service.
please open a separate bug for this.


Cheers.
Chris 

[smime.p7s (application/x-pkcs7-signature, attachment)]

Message #237 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Matt Black <dev@mafro.net>

To: Christoph Anton Mitterer <calestyo@scientia.net>

Cc: 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Tue, 12 May 2015 10:49:38 +1000

[Message part 1 (text/plain, inline)]

>> I've also found that installing libpam-systemd seems to fix this for
>> me in my testing. (If I discover it still to be a problem, I'll post
>> again). However, I'd like to understand why libpam-systemd makes a
>> difference if anyone has any insight?
>
> It doesn't, I still have the problem occasionally on multiple systems,
> all with libpam-systemd installed.
> It's pure coincidence.

Interesting. The important thing here then is my question - WHY does
libpam-systemd make a difference in some cases?

>> Finally, there are issues with using the ssh.socket approach which are
>> not at first apparent. When exiting an SSH session, all processes
>> created under that session are terminated - which means tools like
>> screen and tmux will not work. This may or may not be a problem for
>> you, but it's certainly means ssh.socket is also not a drop-in
>> replacement like ssh.service.
>
> please open a separate bug for this.

IMO this isn't a bug, it's intended behaviour of ssh.socket.


On 12 May 2015 at 10:45, Christoph Anton Mitterer <calestyo@scientia.net>
wrote:

>
> On Tue, 2015-05-12 at 10:20 +1000, Matt Black wrote:
> > I've also found that installing libpam-systemd seems to fix this for
> > me in my testing. (If I discover it still to be a problem, I'll post
> > again). However, I'd like to understand why libpam-systemd makes a
> > difference if anyone has any insight?
> It doesn't, I still have the problem occasionally on multiple systems,
> all with libpam-systemd installed.
> It's pure coincidence.
>
> > Finally, there are issues with using the ssh.socket approach which are
> > not at first apparent. When exiting an SSH session, all processes
> > created under that session are terminated - which means tools like
> > screen and tmux will not work. This may or may not be a problem for
> > you, but it's certainly means ssh.socket is also not a drop-in
> > replacement like ssh.service.
> please open a separate bug for this.
>
>
> Cheers.
> Chris
>

[Message part 2 (text/html, inline)]

Message #242 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Christoph Anton Mitterer <calestyo@scientia.net>

To: Matt Black <dev@mafro.net>

Cc: 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Tue, 12 May 2015 03:08:57 +0200

[Message part 1 (text/plain, inline)]

On Tue, 2015-05-12 at 10:49 +1000, Matt Black wrote: 
> Interesting. The important thing here then is my question - WHY does
> libpam-systemd make a difference in some cases?
I think Russ mentioned it before already,... it may be simply some
timing issues i.e. libpam-systemd's presence slowing the shutdown
somehow down a bit, and thus your sessions get properly killed.


Cheers,
Chris. 

[smime.p7s (application/x-pkcs7-signature, attachment)]

Message #247 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Russ Allbery <rra@debian.org>

To: Christoph Anton Mitterer <calestyo@scientia.net>

Cc: Matt Black <dev@mafro.net>, 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Mon, 11 May 2015 18:49:58 -0700

Christoph Anton Mitterer <calestyo@scientia.net> writes:
> On Tue, 2015-05-12 at 10:49 +1000, Matt Black wrote: 

>> Interesting. The important thing here then is my question - WHY does
>> libpam-systemd make a difference in some cases?

> I think Russ mentioned it before already,... it may be simply some
> timing issues i.e. libpam-systemd's presence slowing the shutdown
> somehow down a bit, and thus your sessions get properly killed.

Nope, I haven't mentioned anything about this, and I'm not sure why
libpam-systemd would make a difference.

For your explanation to be the case, that would mean there's some race,
but I'm not sure what that race would be in my model of how the system
shutdown works.  Do you think that killall is racing the kernel shutting
down the network device, maybe?  I wouldn't have expected that.

There have been LOTS of words written on this bug report, but having
skimmed through it again, I think some really basic information is still
missing.

What, *exactly*, is the sequence of process operations on the server that
causes the client to get a clean shutdown?  What, *exactly*, is the
sequence of process operations on the server that causes the connection to
apparently hang?

It's really hard to design a solution to this problem until one has a good
answer to those two questions, and I haven't seen that information on this
thread yet.  I suspect it's going to require some manual experimentation
with attempting to duplicate the shutdown behavior in a controlled way.

Clearly something changed in systemd vs. sysvinit, which is a clue, but I
don't think we've yet established *what* changed, and therefore have no
idea whether it's intentional, a side effect of something else, a bug, a
race condition, or something else we haven't anticipated.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>

Message #252 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Daniel Kauffman <daniel.kauffman@rocksolidsolutions.org>

To: 751636@bugs.debian.org

Subject: ssh sessions are not cleanly terminated on shutdown/restart with systemd

Date: Sat, 04 Jul 2015 14:40:32 -0700

[Message part 1 (text/plain, inline)]

If sshd is configured with UsePAM yes, then after installing 
libpam-systemd to a remote system and rebooting, ssh sessions are 
cleanly terminated, but after purging libpam-systemd and rebooting, ssh 
session are not cleanly terminated.

If sshd is configured with UsePAM no, then installing/purging 
libpam-systemd has no effect, and ssh session are not cleanly terminated.

*When sshd is configured with UsePAM yes, and libpam-systemd is 
installed, **/usr/share/pam-configs/systemd refers to registering user 
sessions in the systemd control group hierarchy**.**Could this explain 
how ssh sessions are being shutdown cleanly when using PAM?  Does the 
ssh service, apart from PAM, similarly register sessions in the systemd 
control group hierarchy?  If the ssh service does not register sessions 
with the systemd control group hierarchy, could that explain this issue?*

Steps to reproduce (noting that the install/purge AND the reboot must be 
completed before the ssh session behavior changes):

Use ssh to connect to a remote system where ssh sessions are not cleanly 
terminated on remote system reboot.
Configure sshd with UsePAM yes.
Install libpam-systemd.
Reboot.
Observe ssh session is not cleanly terminated.
Use ssh to connect to remote system.
Reboot.
Observe ssh session is cleanly terminated.
Use ssh to connect to remote system.
Remove libpam-systemd.
Reboot.
Observe ssh session is cleanly terminated.
Use ssh to connect to remote system.
Reboot.
Observe ssh session is not cleanly terminated.

-- 
Daniel Kauffman
Lead Developer
Rock Solid Solutions, LLC
877.239.9195 toll-free
208.699.9699 mobile

[Message part 2 (text/html, inline)]

Message #257 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Christoph Anton Mitterer <calestyo@scientia.net>

To: Daniel Kauffman <daniel.kauffman@rocksolidsolutions.org>, 751636@bugs.debian.org

Subject: Re: Bug#751636: ssh sessions are not cleanly terminated on shutdown/restart with systemd

Date: Mon, 06 Jul 2015 17:09:16 +0200

[Message part 1 (text/plain, inline)]

On Sat, 2015-07-04 at 14:40 -0700, Daniel Kauffman wrote:
> If sshd is configured with UsePAM yes, then after installing libpam
> -systemd to a remote system and rebooting, ssh sessions are cleanly 
> terminated, but after purging libpam-systemd and rebooting, ssh 
> session are not cleanly terminated.
I'd guess that this is just some timing coincidence.

I still see the issue from time to time, even though I have PAM
enabled.

E.g. just a few minutes ago it happened again with:
# reboot 
PolicyKit daemon disconnected from the bus.
We are no longer a registered authentication agent.
g_dbus_connection_real_closed: Remote peer vanished with error: Underlying GIOStream returned 0 bytes on an async read (g-io-error-quark, 0). Exiting.
<here it hangs>


Cheer,
Chris.

[smime.p7s (application/x-pkcs7-signature, attachment)]

Message #262 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Daniel Kauffman <daniel.kauffman@rocksolidsolutions.org>

To: Christoph Anton Mitterer <calestyo@scientia.net>, 751636@bugs.debian.org

Subject: Re: Bug#751636: ssh sessions are not cleanly terminated on shutdown/restart with systemd

Date: Mon, 06 Jul 2015 08:48:33 -0700

On 07/06/2015 08:09 AM, Christoph Anton Mitterer wrote:
> On Sat, 2015-07-04 at 14:40 -0700, Daniel Kauffman wrote:
>> If sshd is configured with UsePAM yes, then after installing libpam
>> -systemd to a remote system and rebooting, ssh sessions are cleanly
>> terminated, but after purging libpam-systemd and rebooting, ssh
>> session are not cleanly terminated.
> I'd guess that this is just some timing coincidence.
>
> I still see the issue from time to time, even though I have PAM
> enabled.
>
> E.g. just a few minutes ago it happened again with:
> # reboot
> PolicyKit daemon disconnected from the bus.
> We are no longer a registered authentication agent.
> g_dbus_connection_real_closed: Remote peer vanished with error: Underlying GIOStream returned 0 bytes on an async read (g-io-error-quark, 0). Exiting.
> <here it hangs>
In that case, we're looking at two separate issues.  I tested the 
add/reboot/reboot/purge/reboot/reboot cycle on multiple systems and 
found the behavior to be consistently reproducible, where installing 
libpam-systemd would consistently and completely resolve the issue I was 
seeing.  The remote systems I tested are all running fresh, minimal 
installs of Debian 8.

Before installing libpam-systemd, on reboot, the terminal would hang for 
some time, then display:

Write failed: Broken pipe

After which the terminal would return to the local prompt.

After installing libpam-systemd and rebooting, on the next reboot, the 
terminal immediately displays:

Connection to N.N.N.N closed by remote host.
Connection to N.N.N.N closed.

After which the terminal immediately returns to the local prompt.

--
Daniel Kauffman
Lead Developer
Rock Solid Solutions, LLC
877.239.9195 toll-free
208.699.9699 mobile

Message #267 received at 751636@bugs.debian.org (full text, mbox, reply):

From: "R.S." <brassmold@yahoo.com>

To: 751636@bugs.debian.org

Subject: Re: Bug#751636: ssh sessions are not cleanly terminated on shutdown/restart with systemd

Date: Sun, 26 Jul 2015 00:54:15 -0700

I found a serverfault answer that led me down the path of reasoning below. Have a look: http://serverfault.com/a/706494

> >> If sshd is configured with UsePAM yes, then after installing libpam
> >> -systemd to a remote system and rebooting, ssh sessions are cleanly
> >> terminated…

Given the above and that:

* openssh-server comes configured _out of the box_ with UsePAM=yes enabled
* Openssh-server's installation default is as a systemd service
* libpam-systemd adds functionality to openssh-server in that connections are properly terminated upon shutdown (while the ‘maintain connections’ behavior if the sshd process is stopped/restarted is preserved)
* There is no mention of the package libpam-systemd as being related in any way at all to openssh-server on https://packages.debian.org/jessie/openssh-server (rec/sug/enh, much less as a hard dependency)

…My feeling is, I’d construe THAT to be the real bug here. I don’t know enough about debian’s package relationships to say what relationship libpam-systemd should have to openssh-server, but I’m pretty sure that ‘none whatsoever’ is not the answer.

Similarly,

> > I still see the issue from time to time, even though I have PAM
> > enabled.
> >
> > E.g. just a few minutes ago it happened again with:
> > # reboot
> > PolicyKit daemon disconnected from the bus.
> > We are no longer a registered authentication agent.
> > g_dbus_connection_real_closed: Remote peer vanished with error: Underlying GIOStream returned 0 bytes on an async read (g-io-error-quark, 0). Exiting.
> > <here it hangs>

As suggested in the serverfault answer, and possibly by the above error, does dbus need a relationship either to systemd or openssh-server? ArchLinux and FedoraCore both list dbus as a _requirement_ for systemd. Debian does not AFAICT, and I’m wondering if that should change. [Yes, this is me saying ‘Others do it, why don’t we?’, but I think it’s a fair question to ask here. If dbus would provide information to systemd to manage openssh-server client connections more effectively, shouldn’t it be a dependency to one or the other?]

Best,

R.S.

Message #272 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Christoph Anton Mitterer <calestyo@scientia.net>

To: 751636@bugs.debian.org

Cc: brassmold@yahoo.com

Subject: Re: Bug#751636: ssh sessions are not cleanly terminated on shutdown/restart with systemd

Date: Sun, 26 Jul 2015 22:50:57 +0200

[Message part 1 (text/plain, inline)]

Well,...

By chance, I just had it again:
# reboot 
PolicyKit daemon disconnected from the bus.
We are no longer a registered authentication agent.
g_dbus_connection_real_closed: Remote peer vanished with error: Underlying GIOStream returned 0 bytes on an async read (g-io-error-quark, 0). Exiting.

Timeout, server kronecker not responding.


So in any case, this isn't fixed with libpam-systemd.

Apart from that:
- The "openssh-server comes configured _out of the box_ with UsePAM=yes
enabled" is just a Debian speciality, the [upstream] default is "no".
- There is no reason why people shouldn't be allowed to use UsePAM=no,
so even if the pam module would fix this for the =yes case (which it
apparently doesn't) we'd still need to cover the =no case.
- Since Debian supports diversity wrt init systems I don't see that it
would be good to depend on libpam-systemd.
- I wouldn't be aware that SSH mandatorily uses dbus (which is I guess
the reason that we don't have a dependency on it).

Cheers,
Chris.

[smime.p7s (application/x-pkcs7-signature, attachment)]

Message #277 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Laurent Bigonville <bigon@debian.org>

To: 751636@bugs.debian.org

Subject: Re: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Mon, 17 Aug 2015 12:23:33 +0200

Hey,

OK looking at this bug, if I understood the issue properly, I think I
found what the problem is.

With libpam-systemd installed, UsePAM set to yes, the ssh process with
lower privileges is assign to the user session, when shutting down,
systemd is going through all the user sessions and kill them one by one
properly.

Without libpam-systemd (or UsePAM set to no), the low privileged process
stays in the sshd cgroup. As the ssh.service file explicitly contains
KillMode=process, only the main (privileged) process is killed. At the
end of the shutdown procedure, systemd is going into a killing spree
and SIGKILL all the remaining processes.

The correct solution is IMVHO is to use libpam-systemd with UsePAM set
to yes. On other solution is to change the KillMode, but doing so,
you'll probably loose the connection if the ssh service is restarted.

my 2¢

Laurent Bigonville

Message #282 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Christoph Anton Mitterer <calestyo@scientia.net>

To: Laurent Bigonville <bigon@debian.org>, 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Mon, 17 Aug 2015 16:51:42 +0200

[Message part 1 (text/plain, inline)]

On Mon, 2015-08-17 at 12:23 +0200, Laurent Bigonville wrote:
> The correct solution is IMVHO is to use libpam-systemd with UsePAM
> set
> to yes. On other solution is to change the KillMode, but doing so,
> you'll probably loose the connection if the ssh service is restarted.
Both doesn't seem to be ideal...

I think we should simply add a new unit file, that cleans up any "left
over" sessions which haven't been killed via the cgroup (either because
of the pam module not installed or simply because the user doesn't want
to use UsePAM.

Cheers,
Chris

[smime.p7s (application/x-pkcs7-signature, attachment)]

Message #287 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Laurent Bigonville <bigon@debian.org>

To: 751636@bugs.debian.org

Subject: Re: Bug#751636: openssh-server: ssh sessions are not cleanly termined on shutdown/restart with systemd

Date: Mon, 17 Aug 2015 19:17:19 +0200

On Mon, 17 Aug 2015 16:51:42 +0200 Christoph Anton Mitterer
<calestyo@scientia.net> wrote:
> On Mon, 2015-08-17 at 12:23 +0200, Laurent Bigonville wrote:
> > The correct solution is IMVHO is to use libpam-systemd with UsePAM
> > set
> > to yes. On other solution is to change the KillMode, but doing so,
> > you'll probably loose the connection if the ssh service is
> > restarted.
> Both doesn't seem to be ideal...
> 
> I think we should simply add a new unit file, that cleans up any "left
> over" sessions which haven't been killed via the cgroup (either
> because of the pam module not installed or simply because the user
> doesn't want to use UsePAM.

I personally don't like this solution. IMO, if you are using systemd
you need to call libpam-systemd in your entry point service (sshd,
login, xDM,...) otherwise you are loosing 1/2 of the nice features. We
need to be certain that if systemd is used as pid1 the pam module is
also installed.

And if you are setting UsePAM to false there will be other stuffs
missing/being broken, (like missing entries in the lastlog, the loginuid
using for auditing not being set properly,...).

BTW, if I understand the things correctly (I didn't check the code),
but disabling PAM in sshd or removing libpam-systemd would also mean
that the user processes started via ssh will not be killed gracefully if
somebody is rebooting/shutting down the machine.

Cheers,

Laurent Bigonville

Message #292 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Jephte Clain <jephte.clain@univ-reunion.fr>

To: 751636@bugs.debian.org

Subject: script to gracefully kill ssh user session on reboot only

Date: Thu, 20 Aug 2015 16:23:31 +0400

hello,

to follow up message #20, this is the unit I use:

# cat >/etc/systemd/system/kill-ssh-user-sessions-on-shutdown.service <<EOF
[Unit]
Description=Shutdown all ssh sessions before network
DefaultDependencies=no
Before=network.target shutdown.target

[Service]
Type=oneshot
ExecStart=/usr/bin/killall sshd

[Install]
WantedBy=poweroff.target halt.target reboot.target
EOF

# systemctl enable kill-ssh-user-sessions-on-shutdown

this allow me to reboot or poweroff the system without the ssh user 
session being stuck
FYI, installing libpam-systemd doesn't solve the problem for me

best regards,

-- 
*Jephté CLAIN | Développeur, Intégrateur d'applications*
Service Systèmes d'Information
Direction des Systèmes d'Information <http://numerique.univ-reunion.fr>
Tél: +262 262 93 86 31 <tel:+262262938631> || Gsm: +262 692 29 58 24 
<tel:+262692295824>
www.univ-reunion.fr <http://www.univ-reunion.fr> || Facebook 
<http://www.facebook.com/pages/Universit%C3%A9-de-La-R%C3%A9union-OFFICIEL/197176816990430> 
|| Twitter <http://twitter.com/univ_reunion>

Message #297 received at 751636@bugs.debian.org (full text, mbox, reply):

From: rihad <rihad@mail.ru>

To: 751636@bugs.debian.org

Subject: ssh sessions are not cleanly terminated on shutdown/restart with systemd

Date: Sun, 20 Sep 2015 12:42:04 +0500

It's very surprising that this bug hasn't been addressed for more than a 
year. This ssh connection hangup never happens under FreeBSD, for 
instance. systemd may just be too buggy, since it has another design 
deficiency, namely being unable to save system time upon reboots (along 
the lines of "hwclock --systohc") because its stupid design prevents 
/etc/init.d/hwclock.sh from running out of the box. Just mentioning 
another unrelated core bug that I have personally encountered.

Message #302 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Christoph Anton Mitterer <calestyo@scientia.net>

To: rihad <rihad@mail.ru>, 751636@bugs.debian.org

Subject: Re: Bug#751636: ssh sessions are not cleanly terminated on shutdown/restart with systemd

Date: Sun, 20 Sep 2015 14:44:04 +0200

[Message part 1 (text/plain, inline)]

On Sun, 2015-09-20 at 12:42 +0500, rihad wrote:
> systemd may just be too buggy, since it has another design
> deficiency
This isn't a bug in systemd... o.O

[smime.p7s (application/x-pkcs7-signature, attachment)]

Message #307 received at 751636@bugs.debian.org (full text, mbox, reply):

From: "Alexander Afonyashin" <firm@iname.com>

To: 751636@bugs.debian.org

Subject: Re: Bug#751636: ssh sessions are not cleanly terminated on shutdown/restart with systemd

Date: Sun, 7 Feb 2016 10:17:12 +0100

[Message part 1 (text/html, inline)]

Message #312 received at 751636@bugs.debian.org (full text, mbox, reply):

From: "Alexander Afonyashin" <firm@iname.com>

To: 751636@bugs.debian.org

Subject: Re: Bug#751636: ssh sessions are not cleanly terminated on shutdown/restart with systemd

Date: Sun, 7 Feb 2016 14:36:24 +0100

Hi,
 
The following works for me on Debian 8.3:
 
1. Remove symlink /etc/systemd/system/sshd.service -> /lib/systemd/system/ssh.service - who knows what does symlink do here?
2. Copy /lib/systemd/system/ssh.service to /etc/systemd/system/ssh.service.
3. Edit /etc/systemd/system/ssh.service, add ExecStop=/usr/bin/killall sshd to [Service] section:
 

[Service]
EnvironmentFile=-/etc/default/ssh
ExecStart=/usr/sbin/sshd -D $SSHD_OPTS
ExecReload=/bin/kill -HUP $MAINPID
ExecStop=/usr/bin/killall sshd
KillMode=process
Restart=on-failure
 
4. Reload systemd: systemctl daemon-reload
 
Best regards
Alexander Afonyashin

Message #317 received at 751636@bugs.debian.org (full text, mbox, reply):

From: "Alexander Afonyashin" <firm@iname.com>

To: 751636@bugs.debian.org

Subject: Re: Bug#751636: ssh sessions are not cleanly terminated on shutdown/restart with systemd

Date: Tue, 9 Feb 2016 00:09:25 +0100

Hi,

Small update to previous letter: it's better to replace 'killall sshd' with 'pkill sshd' as psmisc package isn't installed by default if no packages were selected during setup. So ExecStop should be:

ExecStop=/usr/bin/pkill sshd
 
Best regards,
Alexander Afonyashin

Message #322 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Olaf Zaplinski <olaf@zaplinski.de>

To: 751636@bugs.debian.org

Subject: .

Date: Wed, 17 Feb 2016 14:24:56 +0100

tl;dr
but
how can I get the nice and non-problematic Debian 7 behaviour back?

Message #327 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Sven Giermann <giermann@funke.de>

To: 751636@bugs.debian.org

Subject: Re: .

Date: Thu, 23 Jun 2016 15:53:02 +0200

[Message part 1 (text/plain, inline)]

On Wed, 17 Feb 2016 14:24:56 +0100 Olaf Zaplinski <olaf@zaplinski.de> wrote:
 > how can I get the nice and non-problematic Debian 7 behaviour back?

I second this request!
Although I implemented the suggested workaround with ExecStop, I 
recently encountered a broken session while updating sshd, which 
obviously implied stopping and starting the service.
This didn't happen with InitD in the past...

Regards,
Sven



----------------



	FUNKE Wärmeaustauscher Apparatebau GmbH
	 
	Zur Dessel 1
31028 Gronau (Leine)
Germany	 	Tel:
Fax:	 +49 5182 / 582 - 0
 +49 5182 / 582 - 48
	 	www.funke.de
	 
	Geschäftsführer	|	Managing director:	 	Bernd Timmermann
	Registergericht	|	Registered at:	 	Hildesheim, HRB 15080
	Ust.-ID-Nr.	|	VAT no.	 	DE178234018
	Steuer-Nr.	|	Tax no.	 	11/201/19410
----------------
Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese E-Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser E-Mail ist nicht gestattet.

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

[Message part 2 (text/html, inline)]

[8f94ebbb.606a7474.gif (image/gif, inline)]

Message #332 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Simon Deziel <simon@sdeziel.info>

To: 751636@bugs.debian.org

Subject: Re: Bug#751636: ssh sessions are not cleanly terminated on shutdown/restart with systemd

Date: Thu, 23 Jun 2016 12:44:58 -0400

On Sun, 7 Feb 2016 14:36:24 +0100 "Alexander Afonyashin"
<firm@iname.com> wrote:
> 1. Remove symlink /etc/systemd/system/sshd.service -> /lib/systemd/system/ssh.service - who knows what does symlink do here?
> 2. Copy /lib/systemd/system/ssh.service to /etc/systemd/system/ssh.service.

Forking the whole file can by avoided by overriding just the desired
part. In that case, using "systemctl edit ssh" or running those as root
would have been enough:

mkdir -p /etc/systemd/system/ssh.service.d/
cat << EOF >> /etc/systemd/system/ssh.service.d/override.conf
[Service]
ExecStop=/usr/bin/pkill sshd
EOF
systemctl daemon-reload

> 3. Edit /etc/systemd/system/ssh.service, add ExecStop=/usr/bin/killall sshd to [Service] section:

Unfortunately, killing every sshd instances is dangerous. Anyone
stopping the service remotely would be locked out.

I think that another service would be needed to cleanup SSH sessions on
shutdown before they are forcibly killed.

Regards,
Simon

Message #337 received at 751636-close@bugs.debian.org (full text, mbox, reply):

From: Colin Watson <cjwatson@debian.org>

To: 751636-close@bugs.debian.org

Subject: Bug#751636: fixed in openssh 1:7.2p2-6

Date: Fri, 22 Jul 2016 17:26:14 +0000

Source: openssh
Source-Version: 1:7.2p2-6

We believe that the bug you reported is fixed in the latest version of
openssh, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 751636@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Colin Watson <cjwatson@debian.org> (supplier of updated openssh package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Fri, 22 Jul 2016 17:06:19 +0100
Source: openssh
Binary: openssh-client openssh-client-ssh1 openssh-server openssh-sftp-server ssh ssh-krb5 ssh-askpass-gnome openssh-client-udeb openssh-server-udeb
Architecture: source
Version: 1:7.2p2-6
Distribution: unstable
Urgency: medium
Maintainer: Debian OpenSSH Maintainers <debian-ssh@lists.debian.org>
Changed-By: Colin Watson <cjwatson@debian.org>
Description:
 openssh-client - secure shell (SSH) client, for secure access to remote machines
 openssh-client-ssh1 - secure shell (SSH) client for legacy SSH1 protocol
 openssh-client-udeb - secure shell client for the Debian installer (udeb)
 openssh-server - secure shell (SSH) server, for secure access from remote machines
 openssh-server-udeb - secure shell server for the Debian installer (udeb)
 openssh-sftp-server - secure shell (SSH) sftp server module, for SFTP access from remot
 ssh        - secure shell client and server (metapackage)
 ssh-askpass-gnome - interactive X program to prompt users for a passphrase for ssh-ad
 ssh-krb5   - secure shell client and server (transitional package)
Closes: 714526 751636 766887 822997 823827 831902
Changes:
 openssh (1:7.2p2-6) unstable; urgency=medium
 .
   * debian/watch: Switch to HTTP (thanks, Nicholas Luedtke; closes:
     #822997).
   * Copy summary of supported SFTP protocol versions from upstream's
     PROTOCOL file into the openssh-sftp-server package description (closes:
     #766887).
   * Set SSH_PROGRAM=/usr/bin/ssh1 when building openssh-client-ssh1 so that
     scp1 works (reported by Olivier MATZ).
   * Retroactively add a NEWS.Debian entry for the UseDNS change in 6.9 (see
     LP #1588457).
   * CVE-2016-6210: Mitigate user enumeration via covert timing channel
     (closes: #831902).
   * Backport upstream patch to close ControlPersist background process
     stderr when not in debug mode or when logging to a file or syslog
     (closes: #714526).
   * Add a session cleanup script and a systemd unit file to trigger it,
     which serves to terminate SSH sessions cleanly if systemd doesn't do
     that itself, often because libpam-systemd is not installed (thanks,
     Vivek Das Mohapatra, Tom Hutter, and others; closes: #751636).
   * Stop generating DSA host keys by default (thanks, Santiago Vila; closes:
     #823827).
Checksums-Sha1:
 2170a722d423c610aebff6c7d46851fb88316348 2837 openssh_7.2p2-6.dsc
 74c23afda7155665754613e32106434aa5ae105f 154028 openssh_7.2p2-6.debian.tar.xz
Checksums-Sha256:
 2e071288cb930a73414d8cd2c4050b8db583970df13ec7ee47a0150c87b8382e 2837 openssh_7.2p2-6.dsc
 d02a0ad674537b470348807e522496f3c06f7893bfd11b5de809a9cfa5b1176f 154028 openssh_7.2p2-6.debian.tar.xz
Files:
 6b199afe03c15f81d0e758383fee1200 2837 net standard openssh_7.2p2-6.dsc
 15f3b542b8e3378a329acd5eb86ac9a8 154028 net standard openssh_7.2p2-6.debian.tar.xz

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Colin Watson <cjwatson@debian.org> -- Debian developer

iQIVAwUBV5JHkDk1h9l9hlALAQj+hQ/+PBz3IIxgy7hxdy/H+j3nIBIk6nhRV3By
gVQIkNo7t0mnSlfQmQtyVrdzuPuiwItLgYzhNOODIyAqNBxuQVLbv3bNH1Y9n3Do
2SB+KTAxHoKIqF2323r1n0OjIlpot7aI8uX6jZ+Cv5BMQwwtZMqgmlwJll1PUeSL
LVbGVOwy3iOZcNpSAWi7LRcoRMdsNi5myqFB03exOq3026zDmjGEUgrpJC6Dhjyi
lQ64vS/neB97Ww8XtWRclmdnTA0d2VuTS5uvzTGVihV2VZtaRi6WTl4kbiABYRES
YnYFcKQnV4rI7yXTGLXzBeSmWowanzeGW8ppFg3HYQx43rR9pJ/UYX/g2PVVqVvx
Y+W9t10AMX7Fn9PnbY905nAig4kp3GgOWqV5tTaaupVzWXlPIlOmKNvvfnuCJJDk
P3U5vIMueTt+7FOapcbj6MnMktLeoqwrij0alD4CiLF0KTo1PCLM/mkQY76tBwEP
UcbSMBlFhsy4OBLq3qOR017NSo1TXdfSF7CpA/VB9bzbJkHULuVs17G/oHhBV4X/
6QH1AiH0bLReuTxCqbD/uoIjMbOkCjvUHCUbXGdXSMctyv1zYiw7uZJwpB6XU0Do
vmuKMn2IoDtow5OlSD0akLRaXTaNCH2N8Kg5pgPVzIG5XUiJZWoS4U6b8lzuzctz
/2yteQ0fDcQ=
=33BL
-----END PGP SIGNATURE-----

Message #342 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Christoph Anton Mitterer <calestyo@scientia.net>

To: 751636@bugs.debian.org, cjwatson@debian.org

Subject: Re: Bug#751636 closed by Colin Watson <cjwatson@debian.org> (Bug#751636: fixed in openssh 1:7.2p2-6)

Date: Sat, 23 Jul 2016 00:46:29 +0200

[Message part 1 (text/plain, inline)]

Hey Colin.

Not really an issue, but:
>Description=OpenBSD Secure Shell session cleanup

I thought the official name was "OpenSSH", so rather "Open Secure
Shell" (if at all) and not "OpenBSD Secure Shell"?

Cheers,
Chris

[smime.p7s (application/x-pkcs7-signature, attachment)]

Message #347 received at 751636@bugs.debian.org (full text, mbox, reply):

From: Colin Watson <cjwatson@debian.org>

To: Christoph Anton Mitterer <calestyo@scientia.net>

Cc: 751636@bugs.debian.org

Subject: Re: Bug#751636 closed by Colin Watson <cjwatson@debian.org> (Bug#751636: fixed in openssh 1:7.2p2-6)

Date: Sat, 23 Jul 2016 11:02:44 +0100

On Sat, Jul 23, 2016 at 12:46:29AM +0200, Christoph Anton Mitterer wrote:
> Not really an issue, but:
> >Description=OpenBSD Secure Shell session cleanup
> 
> I thought the official name was "OpenSSH", so rather "Open Secure
> Shell" (if at all) and not "OpenBSD Secure Shell"?

It's primarily an OpenBSD project; I have no issues with giving them due
credit.  It also seems more specific and useful as a descriptor than
just "Open".

-- 
Colin Watson                                       [cjwatson@debian.org]

Send a report that this bug log contains spam.