Debian Bug report logs - #809669
unattended-upgrades: files got created under /var/ mountpoint

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Alexandre Detiste <alexandre.detiste@gmail.com>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: unattended-upgrades: files got created under /var/ mountpoint

Date: Sat, 02 Jan 2016 16:53:31 +0100

Package: unattended-upgrades
Version: 0.86.5
Severity: normal

Hi,

I usually have / on a cheap SSD and /var & /home on a HDD.

I noticed that files got created in /var/ in the root
partition, thus before /var got mounted or
more likely after /var got unmounted.

/mnt/var/run/unattended-upgrades.lock
/mnt/var/log/unattended-upgrades/unattended-upgrades-shutdown.log

There's a specific systemd option to fix this:
"RequiresMountsFor=/var/log"

This will also fix /var/run,
as this should always be a link to /run.

---

Also, the line "if apt_pkg.config.find_b("Unattended-
Upgrade::InstallOnShutdown", False):" should be duplicated
before the previous paragrah to avoid creating
an always-empty unattended-upgrades-shutdown.log
if this feature is not used.



-- System Information:
Debian Release: stretch/sid
  APT prefers testing
  APT policy: (500, 'testing'), (450, 'unstable'), (400, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.3.0-1-amd64 (SMP w/6 CPU cores)
Locale: LANG=fr_BE.UTF-8, LC_CTYPE=fr_BE.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages unattended-upgrades depends on:
ii  apt                    1.1.10
ii  apt-utils              1.1.10
ii  debconf [debconf-2.0]  1.5.58
ii  init-system-helpers    1.24
ii  lsb-base               9.20150917
ii  lsb-release            9.20150917
ii  python3                3.4.3-7
ii  python3-apt            1.1.0~beta1
ii  ucf                    3.0031
ii  xz-utils               5.1.1alpha+20120614-2.1

Versions of packages unattended-upgrades recommends:
ii  systemd-cron [cron-daemon]  1.5.3-1

Versions of packages unattended-upgrades suggests:
pn  bsd-mailx                          <none>
ii  nullmailer [mail-transport-agent]  1:1.13-1

-- debconf information:
  unattended-upgrades/origins_pattern: "origin=Debian,codename=${distro_codename},label=Debian-Security";
* unattended-upgrades/enable_auto_updates: true

Message #10 received at 809669@bugs.debian.org (full text, mbox, reply):

From: Alexandre Detiste <alexandre.detiste@gmail.com>

To: 809669@bugs.debian.org

Subject: Re: Bug#809669: unattended-upgrades: files got created under /var/ mountpoint

Date: Sat, 02 Jan 2016 17:04:36 +0100

control: severity -1 important

Let's tag this important: all packages updated in "at shutdown mode"
will have their new /var files stuffed on the wrong hard disk
and these file will be invisible after a reboot;
and package will then consist of a mix of new assets in /
+ old assets in /var.

Message #17 received at 809669@bugs.debian.org (full text, mbox, reply):

From: Scott Leggett <scott@sl.id.au>

To: 809669@bugs.debian.org, control@bugs.debian.org

Subject: unattended-upgrades: files got created under /var/ mountpoint

Date: Mon, 6 Feb 2017 21:14:58 +1100

[Message part 1 (text/plain, inline)]

severity 809669 critical
tags 809669 + patch
--

Hi,

I bumped the severity of this bug because I have seen it cause at least
three systems to hang (possibly until the Unattended Upgrades Shutdown
job times out - I've never waited that long) on every single
shutdown/reboot. This makes some systems which reboot often almost
unusable.

The hang is apparently because /var is unmounted before
unattended-upgrade-shutdown runs, which causes it to falsely believe
that there's an unattended-upgrade running.

In any case, it causes restart/shutdown to hang every time with the only
way to get anything to happen in a timely manner to cut the power or
Alt-SysRq-b.

The problem is caused by an incorrect unattended-upgrades.service file.
According to upstream[0] rather than targeting shutodwn.target, the
correct way to order jobs on shutdown is to use an empty ExecStart
with the desired command in ExecStop. I've tested the following
/etc/systemd/system/unattended-upgrades.service file and it fixes the
problem for me:

  [Unit]
  Description=Unattended Upgrades Shutdown
  After=network.target
  Documentation=man:unattended-upgrade(8)
  
  [Service]
  Type=oneshot
  RemainAfterExit=yes
  ExecStop=/usr/share/unattended-upgrades/unattended-upgrade-shutdown
  TimeoutStopSec=900
  
  [Install]
  WantedBy=multi-user.target


[0]
https://lists.freedesktop.org/archives/systemd-devel/2014-October/023860.html

-- 
Regards,
Scott.

[signature.asc (application/pgp-signature, inline)]

Message #26 received at 809669@bugs.debian.org (full text, mbox, reply):

From: Louis Bouchard <louis.bouchard@ubuntu.com>

To: 809669@bugs.debian.org

Subject: unattended-upgrades: files got created under /var/ mountpoint

Date: Thu, 9 Feb 2017 17:14:44 +0100

Hello,

For info, this also has the potential effect of blocking shutdown (see Ubuntu's
LP: #1654600 [1]) for details.

The unattended-upgrade-shutdown script uses a lock in /var/run to check if an
upgrade job is running. After /var is unmounted, /var/run is no longer present
and  apt_pkg.get_lock() return -1 which normally means that the lock cannot be
taken. In this case, -1 is caused by the fact that the /var/run path no longer
exists. The lock appears to be present, so unattended-upgrade-shutdown waits for
it to go away. The delay to timeout is 10 minutes so the shutdown may block for
10 minutes.

Thought it was worth mentionning.

Kind regards,

...Louis

[1] https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1654600
-- 
Louis Bouchard
Software engineer,
Ubuntu Developer / Debian Maintainer
GPG : 429D 7A3B DD05 B6F8 AF63  B9C4 8B3D 867C 823E 7A61

Message #31 received at 809669@bugs.debian.org (full text, mbox, reply):

From: Alexandre Detiste <alexandre.detiste@gmail.com>

To: Louis Bouchard <louis.bouchard@ubuntu.com>, 809669@bugs.debian.org

Subject: Re: Bug#809669: unattended-upgrades: files got created under /var/ mountpoint

Date: Thu, 09 Feb 2017 18:23:39 +0100

Le jeudi 9 février 2017, 17 h 14 min 44 s CET Louis Bouchard a écrit :
> The unattended-upgrade-shutdown script uses a lock in /var/run to check if an
> upgrade job is running. After /var is unmounted, /var/run is no longer present

Hi,

I thought that by now all reference to /var/run could be replaced by simply "/run".

This would solve this problem.

Alexandre

Message #36 received at 809669@bugs.debian.org (full text, mbox, reply):

From: Louis Bouchard <louis.bouchard@canonical.com>

To: 809669@bugs.debian.org

Subject: Re: unattended-upgrades: files got created under /var/ mountpoint

Date: Fri, 10 Feb 2017 16:31:25 +0100

Hi,

The proposed systemd unit change would break :

Unattended-Upgrade::InstallOnShutdown "true";

as the network is no longer available to fetch the archive.

As outlined in the systemd documentation :

"Given two units with any ordering dependency between them, if one unit is shut
down and the other is started up, the shutdown is ordered before the start-up.
It doesn't matter if the ordering dependency is After= or Before=."

In that context, the network needs to remain available until completion of
unattended-upgrade.service.

I'm still looking for a combination that will work in all cases.

Kind regards,

...Louis

-- 
Louis Bouchard
Software engineer, Cloud & Sustaining eng.
Canonical Ltd
Ubuntu developer                       Debian Maintainer
GPG : 429D 7A3B DD05 B6F8 AF63  B9C4 8B3D 867C 823E 7A61

Message #41 received at 809669@bugs.debian.org (full text, mbox, reply):

From: Scott Leggett <scott@sl.id.au>

To: 809669@bugs.debian.org

Subject: Re: unattended-upgrades: files got created under /var/ mountpoint

Date: Sat, 11 Feb 2017 21:08:41 +1100

[Message part 1 (text/plain, inline)]

On Fri, 10 Feb 2017 16:31:25 +0100 Louis Bouchard <louis.bouchard@canonical.com> wrote:
> Hi,
>·
> The proposed systemd unit change would break :
>·
> Unattended-Upgrade::InstallOnShutdown "true";
>·
> as the network is no longer available to fetch the archive.
>·

No, it wouldn't.

> As outlined in the systemd documentation :
>·
> "Given two units with any ordering dependency between them, if one unit is shut
> down and the other is started up, the shutdown is ordered before the start-up.
> It doesn't matter if the ordering dependency is After= or Before=."
>·

Both units in this case are being shut down, so that paragraph doesn't
apply.

You can test that the network is available for yourself by adding
anonther ExecStop line below the first:

  ExecStop=/bin/ping -c 4 google.com

--·
Regards,
Scott.

[signature.asc (application/pgp-signature, inline)]

Message #46 received at 809669@bugs.debian.org (full text, mbox, reply):

From: Louis Bouchard <louis.bouchard@canonical.com>

To: 809669@bugs.debian.org

Subject: unattended-upgrades: files got created under /var/ mountpoint

Date: Wed, 15 Feb 2017 14:34:58 +0100

Hello,

I may be wrong, but this clearly shows that the Unattended Upgrades Shutdown
unit starts once the target Network is being brought down :

> Feb 15 12:35:52 ZunattendedUpgrade systemd[1]: Stopped target Network is Online.
> Feb 15 12:35:52 ZunattendedUpgrade systemd[1]: Stopped target Network.
> Feb 15 12:35:52 ZunattendedUpgrade systemd[1]: Started Unattended Upgrades Shutdown.
> Feb 15 12:35:52 ZunattendedUpgrade systemd[1]: Stopping ifup for ens3...
> Feb 15 12:35:52 ZunattendedUpgrade systemd[1]: Stopping Raise network interfaces...
> Feb 15 12:35:52 ZunattendedUpgrade systemd[1]: Reloading.
> Feb 15 12:35:52 ZunattendedUpgrade systemd[1]: Reloading.
> Feb 15 12:35:52 ZunattendedUpgrade dhclient[2799]: Killed old client process
> Feb 15 12:35:52 ZunattendedUpgrade ifdown[2765]: Killed old client process
> Feb 15 12:35:52 ZunattendedUpgrade systemd[1]: Reloading.
> Feb 15 12:35:52 ZunattendedUpgrade systemd[1]: Stopped Raise network interfaces.
> Feb 15 12:35:53 ZunattendedUpgrade dhclient[2799]: Internet Systems Consortium DHCP Client 4.3.3
> Feb 15 12:35:53 ZunattendedUpgrade ifdown[2765]: Internet Systems Consortium DHCP Client 4.3.3
> Feb 15 12:35:53 ZunattendedUpgrade ifdown[2765]: Copyright 2004-2015 Internet Systems Consortium.
> Feb 15 12:35:53 ZunattendedUpgrade ifdown[2765]: All rights reserved.
> Feb 15 12:35:53 ZunattendedUpgrade ifdown[2765]: For info, please visit https://www.isc.org/software/dhcp/
> Feb 15 12:35:53 ZunattendedUpgrade dhclient[2799]: Copyright 2004-2015 Internet Systems Consortium.
> Feb 15 12:35:53 ZunattendedUpgrade dhclient[2799]: All rights reserved.
> Feb 15 12:35:53 ZunattendedUpgrade dhclient[2799]: For info, please visit https://www.isc.org/software/dhcp/
> Feb 15 12:35:53 ZunattendedUpgrade dhclient[2799]: 
> Feb 15 12:35:53 ZunattendedUpgrade dhclient[2799]: Listening on LPF/ens3/52:54:00:69:a4:c4
> Feb 15 12:35:53 ZunattendedUpgrade ifdown[2765]: Listening on LPF/ens3/52:54:00:69:a4:c4
> Feb 15 12:35:53 ZunattendedUpgrade ifdown[2765]: Sending on   LPF/ens3/52:54:00:69:a4:c4
> Feb 15 12:35:53 ZunattendedUpgrade ifdown[2765]: Sending on   Socket/fallback
> Feb 15 12:35:53 ZunattendedUpgrade dhclient[2799]: Sending on   LPF/ens3/52:54:00:69:a4:c4
> Feb 15 12:35:53 ZunattendedUpgrade dhclient[2799]: Sending on   Socket/fallback
> Feb 15 12:35:53 ZunattendedUpgrade dhclient[2799]: DHCPRELEASE on ens3 to 192.168.1.1 port 67 (xid=0x3e07e0e5)
> Feb 15 12:35:53 ZunattendedUpgrade ifdown[2765]: DHCPRELEASE on ens3 to 192.168.1.1 port 67 (xid=0x3e07e0e5)
> Feb 15 12:35:53 ZunattendedUpgrade systemd[1]: Stopped ifup for ens3.

Pinging google for 4 seconds is not sufficient, the Unattended upgrade shutdown
can run for saveral minutes before completing.

HTH,

Kind regards,

...Louis


-- 
Louis Bouchard
Software engineer, Cloud & Sustaining eng.
Canonical Ltd
Ubuntu developer                       Debian Maintainer
GPG : 429D 7A3B DD05 B6F8 AF63  B9C4 8B3D 867C 823E 7A61

Message #51 received at 809669@bugs.debian.org (full text, mbox, reply):

From: Scott Leggett <scott@sl.id.au>

To: 809669@bugs.debian.org

Subject: Re: unattended-upgrades: files got created under /var/ mountpoint

Date: Tue, 21 Feb 2017 22:31:21 +1100

[Message part 1 (text/plain, inline)]

Hi Louis,

On Wed, 15 Feb 2017 14:34:58 +0100 Louis Bouchard <louis.bouchard@canonical.com> wrote:
> Hello,
> 
> I may be wrong, but this clearly shows that the Unattended Upgrades Shutdown
> unit starts once the target Network is being brought down :

I don't think the replacement unit I proposed was installed correctly on
your system. Could you double check?

> Pinging google for 4 seconds is not sufficient, the Unattended upgrade shutdown
> can run for saveral minutes before completing.

This is the express purpose of network.target. Here's the relevant
snippet from `man systemd.special`:

  network.target
    This unit is supposed to indicate when network functionality is
    available, but it is only very weakly defined what that is supposed
    to mean, with one exception: at shutdown, a unit that is ordered
    after network.target will be stopped before the network — to
    whatever level it might be set up then — is shut down. It is hence
    useful when writing service files that require network access on
    shutdown, which should order themselves after this target, but not
    pull it in. Also see Running Services After the Network is up[1] for
    more information. Also see network-online.target described above.

-- 
Regards,
Scott.

[signature.asc (application/pgp-signature, inline)]

Message #56 received at 809669@bugs.debian.org (full text, mbox, reply):

From: Louis Bouchard <louis.bouchard@canonical.com>

To: Scott Leggett <scott@sl.id.au>, 809669@bugs.debian.org

Subject: Re: Bug#809669: unattended-upgrades: files got created under /var/ mountpoint

Date: Thu, 2 Mar 2017 17:59:02 +0100

[Message part 1 (text/plain, inline)]

Hello,

Le 21/02/2017 à 12:31, Scott Leggett a écrit :
> Hi Louis,
> 
> On Wed, 15 Feb 2017 14:34:58 +0100 Louis Bouchard <louis.bouchard@canonical.com> wrote:
>> Hello,
>>
>> I may be wrong, but this clearly shows that the Unattended Upgrades Shutdown
>> unit starts once the target Network is being brought down :
> 
> I don't think the replacement unit I proposed was installed correctly on
> your system. Could you double check?
> 
>> Pinging google for 4 seconds is not sufficient, the Unattended upgrade shutdown
>> can run for saveral minutes before completing.
> 
> This is the express purpose of network.target. Here's the relevant
> snippet from `man systemd.special`:
> 
>   network.target
>     This unit is supposed to indicate when network functionality is
>     available, but it is only very weakly defined what that is supposed
>     to mean, with one exception: at shutdown, a unit that is ordered
>     after network.target will be stopped before the network — to
>     whatever level it might be set up then — is shut down. It is hence
>     useful when writing service files that require network access on
>     shutdown, which should order themselves after this target, but not
>     pull it in. Also see Running Services After the Network is up[1] for
>     more information. Also see network-online.target described above.
> 

The unit was correctly installed but another issue made it show as an incorrect
behavior.

It now tests correctly and I am preparing an upload to our development release.

Thanks

Kind regards,

...Louis

-- 
Louis Bouchard
Software engineer, Cloud & Sustaining eng.
Canonical Ltd
Ubuntu developer                       Debian Maintainer
GPG : 429D 7A3B DD05 B6F8 AF63  B9C4 8B3D 867C 823E 7A61

[signature.asc (application/pgp-signature, attachment)]

Message #61 received at 809669@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: Louis Bouchard <louis.bouchard@canonical.com>, 809669@bugs.debian.org

Subject: Re: Bug#809669: unattended-upgrades: files got created under /var/ mountpoint

Date: Thu, 9 Mar 2017 10:35:51 +0000

On Fri, 10 Feb 2017 at 16:31:25 +0100, Louis Bouchard wrote:
> The proposed systemd unit change would break :
> 
> Unattended-Upgrade::InstallOnShutdown "true";
> 
> as the network is no longer available to fetch the archive.

You might be interested in unattended-upgrades patches that I wrote for
SteamOS a few years ago: https://bugs.debian.org/741356

The SteamOS developers were concerned that an expectation that the
network is up during shutdown was unrealistic, so I changed the behaviour
to prepare and download an entire proposed upgrade and stash it in /var
before notifying the user that they should reboot, invalidating the
proposed upgrade if the apt state subsequently changes. A modified
version of my patches seems to be still in production use.

Unfortunately it seems the unattended-upgrades maintainer hasn't been
able to review or integrate these.

    S

Message #66 received at 809669@bugs.debian.org (full text, mbox, reply):

From: Joshua Sanchez <sanchezjoshua343@gmail.com>

To: 809669@bugs.debian.org

Subject: Re: Re: Bug

Date: Mon, 20 Mar 2017 12:50:28 -0700

[Message part 1 (text/plain, inline)]

Fix

[Message part 2 (text/html, inline)]

Message #71 received at 809669@bugs.debian.org (full text, mbox, reply):

From: Niels Thykier <niels@thykier.net>

To: 809669@bugs.debian.org, Louis Bouchard <louis.bouchard@canonical.com>

Subject: Re: Bug#809669: unattended-upgrades: files got created under /var/ mountpoint

Date: Tue, 11 Apr 2017 17:25:00 +0000

On Thu, 2 Mar 2017 17:59:02 +0100 Louis Bouchard
<louis.bouchard@canonical.com> wrote:
> Hello,
> 
> [...]
> 
> The unit was correctly installed but another issue made it show as an incorrect
> behavior.
> 
> It now tests correctly and I am preparing an upload to our development release.
> 
> Thanks
> 
> Kind regards,
> 
> ...Louis
> 
> -- 
> Louis Bouchard
> Software engineer, Cloud & Sustaining eng.
> Canonical Ltd
> Ubuntu developer                       Debian Maintainer
> GPG : 429D 7A3B DD05 B6F8 AF63  B9C4 8B3D 867C 823E 7A61
> 

Hi Louis,

Any news on this?  I cannot see an upload to unstable yet, did something
hold you up?

Thanks,
~Niels

Message #76 received at 809669@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: Niels Thykier <niels@thykier.net>, 809669@bugs.debian.org

Cc: Louis Bouchard <louis.bouchard@canonical.com>, Scott Leggett <scott@sl.id.au>

Subject: Re: Bug#809669: unattended-upgrades: files got created under /var/ mountpoint

Date: Thu, 20 Apr 2017 11:27:18 +0100

On Tue, 11 Apr 2017 at 17:25:00 +0000, Niels Thykier wrote:
> On Thu, 2 Mar 2017 17:59:02 +0100 Louis Bouchard
> <louis.bouchard@canonical.com> wrote:
> > It now tests correctly and I am preparing an upload to our development release.
> 
> Any news on this?  I cannot see an upload to unstable yet, did something
> hold you up?

The "us" referred to in "our development release" appears to have been
Ubuntu, not Debian.

However, the systemd unit proposed by Scott Leggett is not actually the
same as the one now shipped in Ubuntu zesty. Is this deliberate? Here
is a diff:

--- ubuntu/debian/unattended-upgrades.service
+++ scott/debian/unattended-upgrades.service
@@ -1,7 +1,6 @@
 [Unit]
 Description=Unattended Upgrades Shutdown
-DefaultDependencies=no
-Before=shutdown.target reboot.target halt.target network.target local-fs.target
+After=network.target
 Documentation=man:unattended-upgrade(8)
 
 [Service]
@@ -11,4 +10,4 @@ ExecStop=/usr/share/unattended-upgrades/unattended-upgrade-shutdown
 TimeoutStopSec=900
 
 [Install]
-WantedBy=shutdown.target
+WantedBy=multi-user.target

I'm concerned that the version now shipped in Ubuntu might in fact be shut
down *after* the network is taken down, because Before/After dependencies
are about the order of startup: shutdown happens in the reverse order.
So if unattended-upgrades.service has After=network.target, as in Scott's
proposed unit, it will be started (which now does nothing) after
network.target is started, but stopped (which is where the real work
happens) before network.target is stopped. That seems like the right
thing, and the version in Ubuntu zesty seems like only a partial fix.

    S

Message #81 received at 809669@bugs.debian.org (full text, mbox, reply):

From: Louis Bouchard <louis.bouchard@canonical.com>

To: Simon McVittie <smcv@debian.org>, Niels Thykier <niels@thykier.net>, 809669@bugs.debian.org

Cc: Scott Leggett <scott@sl.id.au>

Subject: Re: Bug#809669: unattended-upgrades: files got created under /var/ mountpoint

Date: Thu, 20 Apr 2017 14:05:39 +0200

Hello,

Le 20/04/2017 à 12:27, Simon McVittie a écrit :
> On Tue, 11 Apr 2017 at 17:25:00 +0000, Niels Thykier wrote:
>> On Thu, 2 Mar 2017 17:59:02 +0100 Louis Bouchard
>> <louis.bouchard@canonical.com> wrote:
>>> It now tests correctly and I am preparing an upload to our development release.
>>
>> Any news on this?  I cannot see an upload to unstable yet, did something
>> hold you up?
> 
> The "us" referred to in "our development release" appears to have been
> Ubuntu, not Debian.
> 
> However, the systemd unit proposed by Scott Leggett is not actually the
> same as the one now shipped in Ubuntu zesty. Is this deliberate? Here
> is a diff:
> 
> --- ubuntu/debian/unattended-upgrades.service
> +++ scott/debian/unattended-upgrades.service
> @@ -1,7 +1,6 @@
>  [Unit]
>  Description=Unattended Upgrades Shutdown
> -DefaultDependencies=no
> -Before=shutdown.target reboot.target halt.target network.target local-fs.target
> +After=network.target
>  Documentation=man:unattended-upgrade(8)
>  
>  [Service]
> @@ -11,4 +10,4 @@ ExecStop=/usr/share/unattended-upgrades/unattended-upgrade-shutdown
>  TimeoutStopSec=900
>  
>  [Install]
> -WantedBy=shutdown.target
> +WantedBy=multi-user.target
> 
> I'm concerned that the version now shipped in Ubuntu might in fact be shut
> down *after* the network is taken down, because Before/After dependencies
> are about the order of startup: shutdown happens in the reverse order.
> So if unattended-upgrades.service has After=network.target, as in Scott's
> proposed unit, it will be started (which now does nothing) after
> network.target is started, but stopped (which is where the real work
> happens) before network.target is stopped. That seems like the right
> thing, and the version in Ubuntu zesty seems like only a partial fix.
> 
>     S
> 

First of all, I am sorry for not keeping up with my work in both bug reports.
Recent events at my employer have made things a bit more difficult, including
the recent release of Ubuntu Zesty.

The fix in Zesty is indeed incomplete, but final freeze prevented me for
including the definitive fix in Zesty. This should happen shortly.

I am also preparing a .debdiff of a proposed debian fix that I will create and
test today and tomorrow. In the meantime, here is the final unit that will be
shiped :

> [Unit]                                                                                                             
> Description=Unattended Upgrades Shutdown                                                                           
> After=network.target local-fs.target                                                                               
> RequiresMountsFor=/var/log /var/run /var/lib /boot                                                                 
> Documentation=man:unattended-upgrade(8)                                                                            
>                                                                                                                    
> [Service]                                                                                                          
> Type=oneshot                                                                                                       
> RemainAfterExit=yes                                                                                                
> ExecStop=/usr/share/unattended-upgrades/unattended-upgrade-shutdown                                                
> TimeoutStopSec=900                                                                                                 
>                                                                                                                    
> [Install]                                                                                                          
> WantedBy=multi-user.target                                                                                         

DefaultDependencies=no Needs to be removed as switching from shutdown.target to
multi-user.target requires to have the DefaultDependencies

RequireMountsFor= provides adequate dependencies when /var is separate.

In changing WantedBy= I ran into Debian Bug #797108 which causes the service to
not be properly enabled on upgrade. Working around that took longer than expected.

So hopefully I will provide a final proposition for this by End of week.

Kind regards,

...Louis


-- 
Louis Bouchard
Software engineer, Cloud & Sustaining eng.
Canonical Ltd
Ubuntu developer                       Debian Maintainer
GPG : 429D 7A3B DD05 B6F8 AF63  B9C4 8B3D 867C 823E 7A61

Message #86 received at 809669@bugs.debian.org (full text, mbox, reply):

From: Scott Leggett <scott@sl.id.au>

To: Louis Bouchard <louis.bouchard@canonical.com>

Cc: Simon McVittie <smcv@debian.org>, Niels Thykier <niels@thykier.net>, 809669@bugs.debian.org

Subject: Re: Bug#809669: unattended-upgrades: files got created under /var/ mountpoint

Date: Thu, 20 Apr 2017 23:25:49 +1000

[Message part 1 (text/plain, inline)]

Hi Louis, thanks for your work on this.

On 2017-04-20.14:05, Louis Bouchard wrote:
> > [Unit]                                                                                                             
> > Description=Unattended Upgrades Shutdown                                                                           
> > After=network.target local-fs.target                                                                               
> > RequiresMountsFor=/var/log /var/run /var/lib /boot                                                                 
> > Documentation=man:unattended-upgrade(8)                                                                            
> >                                                                                                                    
> > [Service]                                                                                                          
> > Type=oneshot                                                                                                       
> > RemainAfterExit=yes                                                                                                
> > ExecStop=/usr/share/unattended-upgrades/unattended-upgrade-shutdown                                                
> > TimeoutStopSec=900                                                                                                 
> >                                                                                                                    
> > [Install]                                                                                                          
> > WantedBy=multi-user.target                                                                                         
> 
> DefaultDependencies=no Needs to be removed as switching from shutdown.target to
> multi-user.target requires to have the DefaultDependencies

I think the After=local-fs.target is unnecessary, as DefaultDependencies
pulls in After=sysinit.target, which itself is After=local-fs.target.

-- 
Regards,
Scott.

[signature.asc (application/pgp-signature, inline)]

Message #91 received at 809669@bugs.debian.org (full text, mbox, reply):

From: Niels Thykier <niels@thykier.net>

To: 809669@bugs.debian.org, Louis Bouchard <louis.bouchard@canonical.com>

Subject: Re: Bug#809669: unattended-upgrades: files got created under /var/ mountpoint

Date: Sat, 22 Apr 2017 20:29:00 +0000

On Thu, 20 Apr 2017 14:05:39 +0200 Louis Bouchard
<louis.bouchard@canonical.com> wrote:
> Hello,
> 
> Le 20/04/2017 à 12:27, Simon McVittie a écrit :
> > [...]
> 
> First of all, I am sorry for not keeping up with my work in both bug reports.
> Recent events at my employer have made things a bit more difficult, including
> the recent release of Ubuntu Zesty.
> 

Hi,

Congrats on the Zesty release, btw. :)

> [...]
> 
> I am also preparing a .debdiff of a proposed debian fix that I will create and
> test today and tomorrow. In the meantime, here is the final unit that will be
> shiped :
> 
>

Any news from the test and about the debdiff? :)

Thanks,
~Niels

Message #96 received at 809669@bugs.debian.org (full text, mbox, reply):

From: Louis Bouchard <louis.bouchard@canonical.com>

To: Niels Thykier <niels@thykier.net>, 809669@bugs.debian.org

Subject: Re: Bug#809669: unattended-upgrades: files got created under /var/ mountpoint

Date: Mon, 24 Apr 2017 17:30:34 +0200

[Message part 1 (text/plain, inline)]

Hello,

Here is the patch for this issue that I uploaded to Ubuntu. It is slightly
adapted to take into account the possibility of being install on a non-systemd
installation.

I tried to make the changelog entry as explicit as possible but here are some
details.

As outlined previously, the systemd unit is changed to be an ExecStart instead
of an ExecStop. RequiresMountsFor= are added for /var/log, /var/run, /var/lib &
/boot. Disabling DefaultDependencies is removed so, as outlined previously,
local-fs.target might be superfluous.  RemainAfterExit is set to Yes so the unit
is seen as started. WantedBy is set to multi-user.target

This introduces a problem since Bug #797108[1] is causing the unit not to be
enabled upon upgrade. The following is done to work around this issue:

1) On systemd enabled system, postinst forces the unattended-upgrades service to
be disabled before deb-systemd-helper is executed so the previous
shutdown.target symlink does not remain.

2) At the end of the postinst script, after the deb-systemd-helper has been run,
manually enable and start the service. This will leave the service correctly
configured, as if the deb-systemd-helper had no bug.

3) systemctl enable requires the SysV init script to have a Default-Start
statement in the header otherwise it fails. Add the header in the script.

4) Remove the override_dh_installinit since it uses the 'stop' option which is
no longer available hence switching to 'default' which is the normal installinit
behavior. The postinst script also needs to cleanup the faulty stop symlink
created previously otherwise the systemclt enable fails.

5) Add DEP8 tests to verify that the unit is correctly started and that
InstallOnShutdown works as expected.

I have only tested the upgrade from 0.93.1 to 0.93.2 on Debian/Sid but I have
done extensive testing on Ubuntu which includes :

 * do-release-upgrade from Trusty(upstart) to Xenial(systemd)
 * upgrade on Xenial, Yakkety, Zesty & Artful

Please let me know if I can help with any more testing.

Kind regards,

...Louis

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=797108
--
Louis Bouchard
Software engineer,
Ubuntu Developer / Debian Maintainer
GPG : 429D 7A3B DD05 B6F8 AF63  B9C4 8B3D 867C 823E 7A61

[bug809669_fix_systemd_unit.debdiff (text/plain, attachment)]

[signature.asc (application/pgp-signature, attachment)]

Message #101 received at 809669@bugs.debian.org (full text, mbox, reply):

From: Louis Bouchard <louis.bouchard@canonical.com>

To: Niels Thykier <niels@thykier.net>, 809669@bugs.debian.org

Subject: Re: Bug#809669: unattended-upgrades: files got created under /var/ mountpoint

Date: Fri, 28 Apr 2017 09:14:16 +0200

[Message part 1 (text/plain, inline)]

Hello,

The debdiff that I provided contains two typos in a comment :

> 64 +# Explicitely enable and start the service.i Debian Bug #797108 for

It should read Explicitly and "start the service. Debian"

Just let me know if you want an updated debdiff or if you will be fixing it up
yourself.

Kind regards,

...Louis


-- 
Louis Bouchard
Software engineer, Cloud & Sustaining eng.
Canonical Ltd
Ubuntu developer                       Debian Maintainer
GPG : 429D 7A3B DD05 B6F8 AF63  B9C4 8B3D 867C 823E 7A61

[signature.asc (application/pgp-signature, attachment)]

Message #106 received at 809669@bugs.debian.org (full text, mbox, reply):

From: Gaudenz Steinlin <gaudenz@debian.org>

To: 809669@bugs.debian.org

Subject: RC Bug NMU diff

Date: Sat, 06 May 2017 20:09:39 +0200

[Message part 1 (text/plain, inline)]

Hi

I uplaoded an NMU to unstable to fix this bug. I mostly used the debdiff
prepared by Louis Bouchard but fixed the version number to be
0.93.1+nmu1 instead of 0.93.2 and actually fixed the systemd unit in the
way proposed in this bug. The debdiff attached to the bug missed the
critical changes.

Gaudenz

[bug_809669.debdiff (application/octet-stream, attachment)]

[Message part 3 (text/plain, inline)]

-- 
PGP: 836E 4F81 EFBB ADA7 0852 79BF A97A 7702 BAF9 1EF5

[signature.asc (application/pgp-signature, inline)]

Message #111 received at 809669-close@bugs.debian.org (full text, mbox, reply):

From: Gaudenz Steinlin <gaudenz@debian.org>

To: 809669-close@bugs.debian.org

Subject: Bug#809669: fixed in unattended-upgrades 0.93.1+nmu1

Date: Sat, 06 May 2017 18:19:39 +0000

Source: unattended-upgrades
Source-Version: 0.93.1+nmu1

We believe that the bug you reported is fixed in the latest version of
unattended-upgrades, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 809669@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Gaudenz Steinlin <gaudenz@debian.org> (supplier of updated unattended-upgrades package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.8
Date: Sat, 06 May 2017 19:42:14 +0200
Source: unattended-upgrades
Binary: unattended-upgrades
Architecture: source all
Version: 0.93.1+nmu1
Distribution: unstable
Urgency: medium
Maintainer: Michael Vogt <mvo@debian.org>
Changed-By: Gaudenz Steinlin <gaudenz@debian.org>
Description:
 unattended-upgrades - automatic installation of security upgrades
Closes: 809669
Changes:
 unattended-upgrades (0.93.1+nmu1) unstable; urgency=medium
 .
   * Non-maintainer upload.
 .
   [ Louis Bouchard ]
   * Fix the unattended-upgrades.service unit not correctly working:
     - d/rules : Remove the override_dh_installinit. The stop option is no longer
       available so the command falls back to default. This is the normal
       behavior so the override is not required
     - d/unattended-upgrades.init : Add Default-Start runlevels, otherwise the
       unattended-upgrades.service unit cannot be enabled on boot
     - d/postinst : Cleanup the stop symlinks created by the wrong
       override_dh_installinit. Without that, the systemd unit cannot be
       enabled correctly.
       Force disable the service before deb-systemd-helper runs so the old
       symlink is not left dangling (workaround for Debian Bug #797108).
       Force enable and start of the systemd unit to work around Debian Bug
       #797108 which fails to enable systemd units correctly when WantedBy=
       statement is changed which is the case here.
     - d/unattended-upgrades.service : Fix the service so it runs correctly on
       shutdown :
         Remove DefaultDependencies=no : Breaks normal shutdown dependencies
         Set After= to network.target and local-fs.target. Since our service is
         now ExecStop, it will run before network and local-fs become
         unavailable. Add RequiresMountsFor=/var/log /var/run /var/lib /boot :
         Necessary if /var is a separate file system. Set WantedBy= to
         multi-user.target
     - Add DEP8 tests to verify the following :
       Verify that the unattended-upgrades.service unit is enabled and started.
       Verify that InstallOnShutdown works when configured.
     (Closes: #809669)
Checksums-Sha1:
 1f280837bf9f42bf9d24a279fd2e04b3a8463c3e 1463 unattended-upgrades_0.93.1+nmu1.dsc
 da0f8ab65f87663edbc86921a4da201753b0d7d5 91872 unattended-upgrades_0.93.1+nmu1.tar.xz
 aaeff4cbcbf343e585f84284f7e79db105908897 61690 unattended-upgrades_0.93.1+nmu1_all.deb
 8d1d70dd227fbec049525ac90d38f06d65afff2f 8253 unattended-upgrades_0.93.1+nmu1_amd64.buildinfo
Checksums-Sha256:
 3e19a1912c3f9e4a4995229968d0890a00edfa9e4092ded23242c8068ffb33e4 1463 unattended-upgrades_0.93.1+nmu1.dsc
 753be17e5376f1934c225a3657dd9ca58c2adbd2ffdeb76fa64b5298e90e4900 91872 unattended-upgrades_0.93.1+nmu1.tar.xz
 90a26aa904b96a1e913c429fda03b3e61a685c2c2cc303a08bc8002474c339c8 61690 unattended-upgrades_0.93.1+nmu1_all.deb
 260b3ce7eb9788027e3b8b8330d01f716c0a987c0684d3985ccd79c18a946367 8253 unattended-upgrades_0.93.1+nmu1_amd64.buildinfo
Files:
 4fa33bdb52e199a24befc0bcf03a24de 1463 admin optional unattended-upgrades_0.93.1+nmu1.dsc
 059027850baab7320f7319582bb360bc 91872 admin optional unattended-upgrades_0.93.1+nmu1.tar.xz
 11b024d695da7bbc3cbc3534b749b0f1 61690 admin optional unattended-upgrades_0.93.1+nmu1_all.deb
 9a6dd6a6897e2a531bafffe0525ea3b0 8253 admin optional unattended-upgrades_0.93.1+nmu1_amd64.buildinfo

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEXtjbd32AqFIO1HzsOrL5guAQm9UFAlkOD7MACgkQOrL5guAQ
m9WOfwf/XHSDgouzAMUUMNdzLTiBjvTgUN9u9+U50UYYdVZfJ/N4pJa30zHMKOQM
Qj9W8VLbNB/N219TQUgDLd/MyE+R1pmdIEQ3+I9K8tS4X5X0zkiQe9oPa0rTz9Qh
Smyi/trDEqE142vWToAqemVBrcG2kLVUqxLpTNRpo4ZjqpP9lEWU6yk0fbYETQTc
0IYQ3cJDMlxsDVJqI8M2MGhUhp5sUB3mXuTwE3AX4OSto6hjMgNaRmmRCrxJYkBO
7zT9L5wKkUKujVdxLuzTFNmfFDuf8iMn2ShFfa/qEChbXoUGtUnuc0I9uBimViLw
Gxph2y822qNp8/21Rflmt7CCqjOIYw==
=cDt5
-----END PGP SIGNATURE-----

Message #116 received at 809669@bugs.debian.org (full text, mbox, reply):

From: Louis Bouchard <louis.bouchard@canonical.com>

To: Gaudenz Steinlin <gaudenz@debian.org>, 809669@bugs.debian.org

Subject: Re: Bug#809669: RC Bug NMU diff

Date: Wed, 10 May 2017 11:41:52 +0200

Hello,

Le 06/05/2017 à 20:09, Gaudenz Steinlin a écrit :
> 
> Hi
> 
> I uplaoded an NMU to unstable to fix this bug. I mostly used the debdiff
> prepared by Louis Bouchard but fixed the version number to be
> 0.93.1+nmu1 instead of 0.93.2 and actually fixed the systemd unit in the
> way proposed in this bug. The debdiff attached to the bug missed the
> critical changes.
> 
> Gaudenz
> 
> 
> 
> 

Thank you for the upload. Sorry for the missing bits; I new I'd get bitten by my
two separate Ubuntu uploads.

I will send the fix upsteam in a few minutes.

Kind regards,

...Louis

-- 
Louis Bouchard
Software engineer,
Ubuntu Developer / Debian Maintainer
GPG : 429D 7A3B DD05 B6F8 AF63  B9C4 8B3D 867C 823E 7A61

Send a report that this bug log contains spam.