Debian Bug report logs - #841763
unattended-upgrades: Breaks hard when apt is upgraded

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Stefan Fritsch <sf@sfritsch.de>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: unattended-upgrades: Breaks hard when apt is upgraded

Date: Sun, 23 Oct 2016 12:08:53 +0200

Package: unattended-upgrades
Version: 0.92
Severity: grave

Dear Maintainer,


When unattended-upgrades has to upgrade apt itself, it will be
terminated and leaves the system in a state that requires manual
intervention, like

  dpkg --reconfigure --pending
  apt-get -f install


A second bug is that this broken state does not lead to any mails being
sent to root, and it may take a long time until it is discovered that
updates do not work.

Possibly, apt is killed because systemd is killing all processes
belonging to a service when that service is stopped/restarted and maybe
apt restarts the daily apt service. 

Cheers,
Stefan

Log file extract:

================ unattended-upgrades-dpkg.log
Log started: 2016-10-10  15:35:41
apt-listchanges warning: Cannot reopen /dev/tty for stdin: [Errno 6] No
such device or address: '/dev/tty'
apt-listchanges: Reading changelogs...
apt-listchanges: Mailing root: apt-listchanges: news for XXX
(Reading database ... ^M(Reading database ... 5%^M(Reading database ...
10%^M(Reading database ... 15%^M(Reading database ... 20%^M(Reading
database ... 25%^M(Reading database ... 30%^M(Reading database ...
35%^M(Reading database ... 40%^M(Reading database ... 45%^M(Reading
database ... 50%^M(Reading database ... 55%^M(Reading database ...
60%^M(Reading database ... 65%^M(Reading database ... 70%^M(Reading
database ... 75%^M(Reading database ... 80%^M(Reading database ...
85%^M(Reading database ... 90%^M(Reading database ... 95%^M(Reading
database ... 100%^M(Reading database ... 55308 files and directories
currently installed.)
Preparing to unpack .../libapt-pkg5.0_1.3.1_amd64.deb ...
Unpacking libapt-pkg5.0:amd64 (1.3.1) over (1.3) ...
Setting up libapt-pkg5.0:amd64 (1.3.1) ...
(Reading database ... ^M(Reading database ... 5%^M(Reading database ...
10%^M(Reading database ... 15%^M(Reading database ... 20%^M(Reading
database ... 25%^M(Reading database ... 30%^M(Reading database ...
35%^M(Reading database ... 40%^M(Reading database ... 45%^M(Reading
database ... 50%^M(Reading database ... 55%^M(Reading database ...
60%^M(Reading database ... 65%^M(Reading database ... 70%^M(Reading
database ... 75%^M(Reading database ... 80%^M(Reading database ...
85%^M(Reading database ... 90%^M(Reading database ... 95%^M(Reading
database ... 100%^M(Reading database ... 55308 files and directories
currently installed.)
Preparing to unpack .../0-libapt-inst2.0_1.3.1_amd64.deb ...
Unpacking libapt-inst2.0:amd64 (1.3.1) over (1.3) ...
Preparing to unpack .../1-apt_1.3.1_amd64.deb ...
Unpacking apt (1.3.1) over (1.3) ...
Setting up apt (1.3.1) ...
Log started: 2016-10-22  04:37:01
================ unattended-upgrades.log
2016-10-10 15:35:34,852 INFO Initial blacklisted packages: 
2016-10-10 15:35:34,852 INFO Initial whitelisted packages: 
2016-10-10 15:35:34,852 INFO Starting unattended upgrades script
2016-10-10 15:35:34,852 INFO Allowed origins are:
['origin=Debian,codename=stretch']
2016-10-10 15:35:41,215 INFO Packages that will be upgraded: apache2
apache2-bin apache2-data apache2-utils apt apt-utils libapt-inst2.0
libapt-pkg5.0 libreadline6 libslang2 readline-common vim vim-common
vim-runtime xxd
2016-10-10 15:35:41,215 INFO Writing dpkg log to
'/var/log/unattended-upgrades/unattended-upgrades-dpkg.log'
2016-10-21 08:08:47,884 INFO Initial blacklisted packages: 
================



-- System Information:
Debian Release: stretch/sid
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 4.7.0-1-amd64 (SMP w/2 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages unattended-upgrades depends on:
ii  apt                    1.3.1
ii  apt-utils              1.3.1
ii  debconf [debconf-2.0]  1.5.59
ii  init-system-helpers    1.45
ii  lsb-base               9.20160629
ii  lsb-release            9.20160629
ii  python3                3.5.1-4
ii  python3-apt            1.1.0~beta5
ii  ucf                    3.0036
ii  xz-utils               5.2.2-1.2

Versions of packages unattended-upgrades recommends:
ii  cron [cron-daemon]  3.0pl1-128

Versions of packages unattended-upgrades suggests:
ii  bsd-mailx                                  8.1.2-0.20160123cvs-3
ii  exim4-daemon-light [mail-transport-agent]  4.87-3+b1

-- debconf information:
* unattended-upgrades/enable_auto_updates: true
* unattended-upgrades/origins_pattern: origin=Debian,codename=${distro_codename}

Message #12 received at 841763@bugs.debian.org (full text, mbox, reply):

From: Santiago Vila <sanvila@unex.es>

To: Stefan Fritsch <sf@sfritsch.de>, 841763@bugs.debian.org

Subject: Re: Bug#841763: unattended-upgrades: Breaks hard when apt is upgraded

Date: Sun, 23 Oct 2016 13:43:59 +0200

On Sun, Oct 23, 2016 at 12:08:53PM +0200, Stefan Fritsch wrote:
> Package: unattended-upgrades
> Version: 0.92
> Severity: grave
> 
> Dear Maintainer,
> 
> 
> When unattended-upgrades has to upgrade apt itself, it will be
> terminated and leaves the system in a state that requires manual
> intervention, like
> 
>   dpkg --reconfigure --pending
>   apt-get -f install
> 
> 
> A second bug is that this broken state does not lead to any mails being
> sent to root, and it may take a long time until it is discovered that
> updates do not work.
> 
> Possibly, apt is killed because systemd is killing all processes
> belonging to a service when that service is stopped/restarted and maybe
> apt restarts the daily apt service. 

Aha! I can reproduce this too. On the machines where this happens,
I believe I answered "yes" to a debconf question about restarting
services automatically (one of the times I upgraded by hand and not
using unattended-upgrades).

Maybe something like this would work as a workaround?

echo "libc6 libraries/restart-without-asking boolean false" | debconf-set-selections

Thanks a lot.

Message #17 received at 841763@bugs.debian.org (full text, mbox, reply):

From: Alexandre Detiste <alexandre.detiste@gmail.com>

To: Santiago Vila <sanvila@unex.es>, 841763@bugs.debian.org

Cc: Stefan Fritsch <sf@sfritsch.de>

Subject: Re: Bug#841763: unattended-upgrades: Breaks hard when apt is upgraded

Date: Sun, 23 Oct 2016 13:55:37 +0200

Le dimanche 23 octobre 2016, 13 h 43 min 59 s CEST Santiago Vila a écrit :
> Aha! I can reproduce this too. On the machines where this happens,
> I believe I answered "yes" to a debconf question about restarting
> services automatically (one of the times I upgraded by hand and not
> using unattended-upgrades).
> 
> Maybe something like this would work as a workaround?
> 
> echo "libc6 libraries/restart-without-asking boolean false" | debconf-set-selections
> 
> Thanks a lot.

Hi,

I think that adding this snippet to apt's debian/rules would fix this problem,
not tested tough.



override_dh_systemd_start:
        dh_systemd_start apt-daily.timer



Without this override, we get this: apt got -daily.service got "try-restart"ed
bit this statement in atp.postinst:
-------------


# Automatically added by dh_systemd_start
if [ -d /run/systemd/system ]; then
        systemctl --system daemon-reload >/dev/null || true
        if [ -n "$2" ]; then
                _dh_action=try-restart
        else
                _dh_action=start
        fi
        deb-systemd-invoke $_dh_action apt-daily.service apt-daily.timer >/dev/null || true
fi
# End automatically added section




Greets,

Alexandre Detiste

Message #22 received at 841763@bugs.debian.org (full text, mbox, reply):

From: Michael Vogt <mvo@debian.org>

To: Alexandre Detiste <alexandre.detiste@gmail.com>, 841763@bugs.debian.org

Subject: Re: Bug#841763: unattended-upgrades: Breaks hard when apt is upgraded

Date: Sun, 30 Oct 2016 15:43:16 +0100

On Sun, Oct 23, 2016 at 01:55:37PM +0200, Alexandre Detiste wrote:
> Le dimanche 23 octobre 2016, 13 h 43 min 59 s CEST Santiago Vila a écrit :
> > Aha! I can reproduce this too. On the machines where this happens,
> > I believe I answered "yes" to a debconf question about restarting
> > services automatically (one of the times I upgraded by hand and not
> > using unattended-upgrades).
> > 
> > Maybe something like this would work as a workaround?
> > 
> > echo "libc6 libraries/restart-without-asking boolean false" | debconf-set-selections
> > 
> > Thanks a lot.
> 
> Hi,
> 
> I think that adding this snippet to apt's debian/rules would fix this problem,
> not tested tough.
> 
> 
> 
> override_dh_systemd_start:
>         dh_systemd_start apt-daily.timer
> 

Yes, I think you are right. This appears to be what is happening. I
have commited this fix to git and it will be part of the next upload.

Thanks,
 Michael

> 
> Without this override, we get this: apt got -daily.service got "try-restart"ed
> bit this statement in atp.postinst:
> -------------
> 
> 
> # Automatically added by dh_systemd_start
> if [ -d /run/systemd/system ]; then
>         systemctl --system daemon-reload >/dev/null || true
>         if [ -n "$2" ]; then
>                 _dh_action=try-restart
>         else
>                 _dh_action=start
>         fi
>         deb-systemd-invoke $_dh_action apt-daily.service apt-daily.timer >/dev/null || true
> fi
> # End automatically added section
> 
> 
> 
> 
> Greets,
> 
> Alexandre Detiste

Message #27 received at 841763@bugs.debian.org (full text, mbox, reply):

From: Stefan Fritsch <sf@sfritsch.de>

To: Alexandre Detiste <alexandre.detiste@gmail.com>

Cc: 841763@bugs.debian.org

Subject: Re: Bug#841763: unattended-upgrades: Breaks hard when apt is upgraded

Date: Wed, 2 Nov 2016 15:30:37 +0100 (CET)

Hi,

On Sun, 23 Oct 2016, Alexandre Detiste wrote:

> I think that adding this snippet to apt's debian/rules would fix this problem,
> not tested tough.
> 
> 
> 
> override_dh_systemd_start:
>         dh_systemd_start apt-daily.timer
> 

Not restarting it would be one way to fix it. I don't know if apt-daily 
does other things that need a restart when upgrading.

A different idea may be to use systemd-run to start apt (or the whole 
unattended-upgrades script) as a transient scope unit. AIUI, this should 
prevent systemd from killing apt, even if the parent process is 
terminated.

Cheers,
Stefan

Message #30 received at 841763-submitter@bugs.debian.org (full text, mbox, reply):

From: Michael Vogt <mvo@ubuntu.com>

To: 841763-submitter@bugs.debian.org

Subject: Bug#841763 in apt marked as pending

Date: Wed, 09 Nov 2016 15:02:22 +0000

Control: tag 841763 pending

Hello,

Bug #841763 in apt reported by you has been fixed in the Git repository. You can
see the commit message below, and you can check the diff of the fix at:

    https://anonscm.debian.org/cgit/apt/apt.git/diff/?id=e133bb5

(this message was generated automatically based on the git commit message)
---
commit e133bb5e81b10bf059b3abeab2d9e41f7206e446
Author: Michael Vogt <mvo@ubuntu.com>
Date:   Wed Nov 9 15:09:44 2016 +0100

    Do not (re)start "apt-daily.system"
    
    This unit runs unattended-upgrades. If apt itself is part of the
    upgrade a restart of the unit will kill unattended-upgrades. This
    will lead to an inconsistent dpkg status.
    
    Closes: #841763
    Thanks: Alexandre Detiste

Message #43 received at 841763-close@bugs.debian.org (full text, mbox, reply):

From: Julian Andres Klode <jak@debian.org>

To: 841763-close@bugs.debian.org

Subject: Bug#841763: fixed in apt 1.4~beta1

Date: Sat, 26 Nov 2016 00:03:23 +0000

Source: apt
Source-Version: 1.4~beta1

We believe that the bug you reported is fixed in the latest version of
apt, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 841763@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Julian Andres Klode <jak@debian.org> (supplier of updated apt package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.8
Date: Fri, 25 Nov 2016 23:49:54 +0100
Source: apt
Binary: apt libapt-pkg5.0 libapt-inst2.0 apt-doc libapt-pkg-dev libapt-pkg-doc apt-utils apt-transport-https
Architecture: source
Version: 1.4~beta1
Distribution: unstable
Urgency: medium
Maintainer: APT Development Team <deity@lists.debian.org>
Changed-By: Julian Andres Klode <jak@debian.org>
Description:
 apt        - commandline package manager
 apt-doc    - documentation for APT
 apt-transport-https - https download transport for APT
 apt-utils  - package management related utility programs
 libapt-inst2.0 - deb package format runtime library
 libapt-pkg-dev - development files for APT's libapt-pkg and libapt-inst
 libapt-pkg-doc - documentation for APT development
 libapt-pkg5.0 - package management runtime library
Closes: 272557 465572 644610 767891 837395 838779 840552 840757 841763 841874 842877 844724 845599
Changes:
 apt (1.4~beta1) unstable; urgency=medium
 .
   [ Chris Leick ]
   * Updated German documentation translation
   * fix three typos in sources & manpages
   * German translation proof read by Helge Kreutzmann
 .
   [ Frans Spiesschaert ]
   * Dutch program translation update (Closes: #840552)
   * Dutch manpages translation update (Closes: #840757)
 .
   [ David Kalnischkies ]
   * don't install new deps of candidates for kept back pkgs
   * keep Release.gpg on untrusted to trusted IMS-Hit (Closes: 838779)
   * fix testcase expecting incorrect remove log from dpkg
   * reset HOME, USER(NAME), TMPDIR & SHELL in DropPrivileges (Closes: 842877)
   * show the conflicting distribution warning again (Closes: 841874)
   * rename Checksum-FileSize to Filesize in hashsum mismatch
   * improve SOCKS error messages for http slightly
   * support 'apt build-dep .' (aka: without /)
   * add TMP/TEMP/TEMPDIR to the TMPDIR DropPrivileges dance
   * http: skip connection cleanup if we close it anyhow
   * add hidden config to set packages as Essential/Important.
     Thanks to Anthony Towns for initial patch (Closes: 767891)
   * don't warn if untransformed distribution matches.
     Thanks to Lukas Anzinger for initial patch (Closes: 644610)
   * show distribution mismatch for changed codenames
   * react to trig-pend only if we have nothing else to do
   * correct cross & disappear progress detection
   * improve arch-unqualified dpkg-progress parsing
   * don't perform implicit crossgrades involving M-A:same
   * do not configure unconfigured to be removed packages
   * skip unconfigure for unconfigured to-be removed pkgs
   * report apt-key errors via status-fd messages (LP: #1522988)
   * add apt-key support for armored GPG key files (*.asc)
   * document which keyring formats are supported by apt-key (Closes: 844724)
   * get pdiff files from the same mirror as the index
   * follow the googletest merge in build-depends
 .
   [ Michael Vogt ]
   * Do not (re)start "apt-daily.system"
     Thanks to Alexandre Detiste (Closes: #841763)
 .
   [ Johannes Schauer ]
   * add support for Build-Depends/Conflicts-Arch (Closes: #837395)
 .
   [ Edgar Fuß ]
   * http: clear content before reporting the failure (Closes: #465572)
 .
   [ James Clarke ]
   * apt-ftparchive: Support NotAutomatic and ButAutomaticUpgrades fields
     (Closes: #272557)
 .
   [ Julian Andres Klode ]
   * TagSection: Split AlphaIndexes into AlphaIndexes and BetaIndexes
   * TagSection: Extract Find() methods taking Pos instead of Key
   * Squashed 'triehash/' content from commit 16f59e1
   * TagSection: Introduce functions for looking up by key ids
   * debListParser: Convert to use pkgTagSection::Key-based lookup
   * Bump the cache major version for non-backportable changes
   * Introduce tolower_ascii_unsafe() and use it for hashing
   * Optimize VersionHash() to not need temporary copy of input
   * Compare size before data when ordering cache bucket entries
   * debListParser: Micro-optimize AvailableDescriptionLanguages()
   * Do not use MD5SumValue for Description_md5()
   * gpgv: Untrust SHA1, RIPE-MD/160, but allow downgrading to weak
 .
   [ Paul Wise ]
   * show output as documented for APT::Periodic::Verbose 2 (Closes: 845599)
Checksums-Sha1:
 9f49c6a174ec4da92b27041060cb3531dfa93337 2565 apt_1.4~beta1.dsc
 bde0ba2ec01caa7856bb02255ab96ed0bba6d56d 2053936 apt_1.4~beta1.tar.xz
Checksums-Sha256:
 59965ea9f2489feeb510d945f8ec681f8cedf6ff9ddc413c02423b2f48b0d9c1 2565 apt_1.4~beta1.dsc
 7321ec058d2a7d664e1b5123a1d73fc0d63738eb97783df8dce45e8a2b2898c1 2053936 apt_1.4~beta1.tar.xz
Files:
 0bd8495d92665d653fc0552a29a1e707 2565 admin important apt_1.4~beta1.dsc
 08e36fb472189a177a8f1b55949e41ae 2053936 admin important apt_1.4~beta1.tar.xz

-----BEGIN PGP SIGNATURE-----

iQJDBAEBCgAtFiEEzeVhi4gF/W4gLOnC1zw55WWAs4YFAlg4wOQPHGpha0BkZWJp
YW4ub3JnAAoJENc8OeVlgLOGoDcP/0TuSlspjqIdw0V7mP8jvDg69jH2WDVv4GGJ
8/+JTSry8k7x7X07GaZX7bPmSUbFB1JMeP1+SWyFYa/J3e/GwqTWhKllojjvCo79
XnduRT6KsMrzznzB4CIlS9MwpIn6Wz3hxogJYiJcST+8243m59QsqhdnnD+fWrHG
IpLk8cPgxrVPnw+9cSXt/zuJVuuZL2ExMBk+QaRYpSXXCdMN7rUuBCoWP4RE5hTy
isGA2K53tHt7QYpo6175N3zwcuFyOTzp4XuuOlemG2FcIHjIpZ978EhA8bLdh5QW
OSbW8suti7IwX7+TJyOjYOOGmZSwa+34f6lq4ED0N79SRQAaCpOKqx6OMWFnvDhN
vEeMXpaWZSsf1QIgQ6V27fMWHVncQJ5SO+Dz8E1VTmgqVU/kMSb6JKW06WCkH5fO
ebNfv1l9UndWQQAvVrLTS8SduR334lZN2Z1FfJLPVHLZTqfSNZjLQWuVwbnZReDR
GIEgmXge1MEmHFDW33/xISOe3kLJfWtSjQxe2NbvTKKKGvwEzawVQNlJxAOGmhMW
ZbGXSx4j7YWlUmBVObHSIsHqCLZov4pLA08CsYu6g3DE/YyuQqrzmIRC/cS6yR5K
lZ+6sJLnaWaTxjEsBsT5AKNpdoOB6vnodNJY4bY0malBcCDfv8BpnuApmyFRzyCw
sUscE3+G
=eM/Q
-----END PGP SIGNATURE-----

Message #46 received at 841763-submitter@bugs.debian.org (full text, mbox, reply):

From: Julian Andres Klode <jak@debian.org>

To: 841763-submitter@bugs.debian.org

Subject: Bug#841763 in apt marked as pending

Date: Wed, 14 Dec 2016 19:36:52 +0000

Control: tag 841763 pending

Hello,

Bug #841763 in apt reported by you has been fixed in the Git repository. You can
see the commit message below, and you can check the diff of the fix at:

    https://anonscm.debian.org/cgit/apt/apt.git/diff/?id=55bf597

(this message was generated automatically based on the git commit message)
---
commit 55bf597f0cbaa52013f0f01de7b5fd5b4e07f2aa
Author: Michael Vogt <mvo@ubuntu.com>
Date:   Wed Nov 9 15:09:44 2016 +0100

    Do not (re)start "apt-daily.system"
    
    This unit runs unattended-upgrades. If apt itself is part of the
    upgrade a restart of the unit will kill unattended-upgrades. This
    will lead to an inconsistent dpkg status.
    
    Closes: #841763
    Thanks: Alexandre Detiste
    (cherry picked from commit e133bb5e81b10bf059b3abeab2d9e41f7206e446)
    LP: #1649959

Send a report that this bug log contains spam.