Debian Bug report logs - #776910
apt: upgrade from wheezy to jessie breaks in the middle

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Rafal Pietrak <rafal@ztk-rp.eu>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: apt: upgrade from wheezy to jessie breaks in the middle

Date: Tue, 03 Feb 2015 10:20:26 +0100

Package: apt
Version: 1.0.9.6
Severity: grave
Tags: newcomer
Justification: renders package unusable

My guess is that some limit on number of errors was taken into account
unneceserly during an upgrade - upgrades are expected to rise trancient errors.



-- Package-specific info:

-- apt-config dump --

APT "";
APT::Architecture "i386";
APT::Build-Essential "";
APT::Build-Essential:: "build-essential";
APT::Install-Recommends "1";
APT::Install-Suggests "0";
APT::NeverAutoRemove "";
APT::NeverAutoRemove:: "^firmware-linux.*";
APT::NeverAutoRemove:: "^linux-firmware$";
APT::NeverAutoRemove:: "^linux-image-3\.16\.0-4-586$";
APT::NeverAutoRemove:: "^linux-image-3\.2\.0-4-486$";
APT::NeverAutoRemove:: "^linux-headers-3\.16\.0-4-586$";
APT::NeverAutoRemove:: "^linux-headers-3\.2\.0-4-486$";
APT::NeverAutoRemove:: "^linux-image-extra-3\.16\.0-4-586$";
APT::NeverAutoRemove:: "^linux-image-extra-3\.2\.0-4-486$";
APT::NeverAutoRemove:: "^linux-signed-image-3\.16\.0-4-586$";
APT::NeverAutoRemove:: "^linux-signed-image-3\.2\.0-4-486$";
APT::NeverAutoRemove:: "^kfreebsd-image-3\.16\.0-4-586$";
APT::NeverAutoRemove:: "^kfreebsd-image-3\.2\.0-4-486$";
APT::NeverAutoRemove:: "^kfreebsd-headers-3\.16\.0-4-586$";
APT::NeverAutoRemove:: "^kfreebsd-headers-3\.2\.0-4-486$";
APT::NeverAutoRemove:: "^gnumach-image-3\.16\.0-4-586$";
APT::NeverAutoRemove:: "^gnumach-image-3\.2\.0-4-486$";
APT::NeverAutoRemove:: "^.*-modules-3\.16\.0-4-586$";
APT::NeverAutoRemove:: "^.*-modules-3\.2\.0-4-486$";
APT::NeverAutoRemove:: "^.*-kernel-3\.16\.0-4-586$";
APT::NeverAutoRemove:: "^.*-kernel-3\.2\.0-4-486$";
APT::NeverAutoRemove:: "^linux-backports-modules-.*-3\.16\.0-4-586$";
APT::NeverAutoRemove:: "^linux-backports-modules-.*-3\.2\.0-4-486$";
APT::NeverAutoRemove:: "^linux-tools-3\.16\.0-4-586$";
APT::NeverAutoRemove:: "^linux-tools-3\.2\.0-4-486$";
APT::NeverAutoRemove:: "^postgresql-";
APT::VersionedKernelPackages "";
APT::VersionedKernelPackages:: "linux-image";
APT::VersionedKernelPackages:: "linux-headers";
APT::VersionedKernelPackages:: "linux-image-extra";
APT::VersionedKernelPackages:: "linux-signed-image";
APT::VersionedKernelPackages:: "kfreebsd-image";
APT::VersionedKernelPackages:: "kfreebsd-headers";
APT::VersionedKernelPackages:: "gnumach-image";
APT::VersionedKernelPackages:: ".*-modules";
APT::VersionedKernelPackages:: ".*-kernel";
APT::VersionedKernelPackages:: "linux-backports-modules-.*";
APT::VersionedKernelPackages:: "linux-tools";
APT::Never-MarkAuto-Sections "";
APT::Never-MarkAuto-Sections:: "metapackages";
APT::Never-MarkAuto-Sections:: "restricted/metapackages";
APT::Never-MarkAuto-Sections:: "universe/metapackages";
APT::Never-MarkAuto-Sections:: "multiverse/metapackages";
APT::Never-MarkAuto-Sections:: "oldlibs";
APT::Never-MarkAuto-Sections:: "restricted/oldlibs";
APT::Never-MarkAuto-Sections:: "universe/oldlibs";
APT::Never-MarkAuto-Sections:: "multiverse/oldlibs";
APT::Update "";
APT::Update::Post-Invoke-Success "";
APT::Update::Post-Invoke-Success:: "[ ! -f /var/run/dbus/system_bus_socket ] || /usr/bin/dbus-send --system --dest=org.debian.apt --type=signal /org/debian/apt org.debian.apt.CacheChanged || true";
APT::Update::Post-Invoke-Success:: "/usr/bin/test -e /usr/share/dbus-1/system-services/org.freedesktop.PackageKit.service && /usr/bin/test -S /var/run/dbus/system_bus_socket && /usr/bin/gdbus call --system --dest org.freedesktop.PackageKit --object-path /org/freedesktop/PackageKit --timeout 4 --method org.freedesktop.PackageKit.StateHasChanged cache-update > /dev/null; /bin/echo > /dev/null";
APT::Immediate-Configure "1";
APT::Default-Release "jessie";
APT::Architectures "";
APT::Architectures:: "i386";
APT::Compressor "";
APT::Compressor::. "";
APT::Compressor::.::Name ".";
APT::Compressor::.::Extension "";
APT::Compressor::.::Binary "";
APT::Compressor::.::Cost "1";
APT::Compressor::gzip "";
APT::Compressor::gzip::Name "gzip";
APT::Compressor::gzip::Extension ".gz";
APT::Compressor::gzip::Binary "gzip";
APT::Compressor::gzip::Cost "2";
APT::Compressor::gzip::CompressArg "";
APT::Compressor::gzip::CompressArg:: "-9n";
APT::Compressor::gzip::UncompressArg "";
APT::Compressor::gzip::UncompressArg:: "-d";
APT::Compressor::bzip2 "";
APT::Compressor::bzip2::Name "bzip2";
APT::Compressor::bzip2::Extension ".bz2";
APT::Compressor::bzip2::Binary "bzip2";
APT::Compressor::bzip2::Cost "3";
APT::Compressor::bzip2::CompressArg "";
APT::Compressor::bzip2::CompressArg:: "-9";
APT::Compressor::bzip2::UncompressArg "";
APT::Compressor::bzip2::UncompressArg:: "-d";
APT::Compressor::xz "";
APT::Compressor::xz::Name "xz";
APT::Compressor::xz::Extension ".xz";
APT::Compressor::xz::Binary "xz";
APT::Compressor::xz::Cost "4";
APT::Compressor::xz::CompressArg "";
APT::Compressor::xz::CompressArg:: "-6";
APT::Compressor::xz::UncompressArg "";
APT::Compressor::xz::UncompressArg:: "-d";
APT::Compressor::lzma "";
APT::Compressor::lzma::Name "lzma";
APT::Compressor::lzma::Extension ".lzma";
APT::Compressor::lzma::Binary "xz";
APT::Compressor::lzma::Cost "5";
APT::Compressor::lzma::CompressArg "";
APT::Compressor::lzma::CompressArg:: "--format=lzma";
APT::Compressor::lzma::CompressArg:: "-9";
APT::Compressor::lzma::UncompressArg "";
APT::Compressor::lzma::UncompressArg:: "--format=lzma";
APT::Compressor::lzma::UncompressArg:: "-d";
Dir "/";
Dir::State "var/lib/apt/";
Dir::State::lists "lists/";
Dir::State::cdroms "cdroms.list";
Dir::State::mirrors "mirrors/";
Dir::State::extended_states "extended_states";
Dir::State::status "/var/lib/dpkg/status";
Dir::Cache "var/cache/apt/";
Dir::Cache::archives "archives/";
Dir::Cache::srcpkgcache "srcpkgcache.bin";
Dir::Cache::pkgcache "pkgcache.bin";
Dir::Etc "etc/apt/";
Dir::Etc::sourcelist "sources.list";
Dir::Etc::sourceparts "sources.list.d";
Dir::Etc::vendorlist "vendors.list";
Dir::Etc::vendorparts "vendors.list.d";
Dir::Etc::main "apt.conf";
Dir::Etc::netrc "auth.conf";
Dir::Etc::parts "apt.conf.d";
Dir::Etc::preferences "preferences";
Dir::Etc::preferencesparts "preferences.d";
Dir::Etc::trusted "trusted.gpg";
Dir::Etc::trustedparts "trusted.gpg.d";
Dir::Bin "";
Dir::Bin::methods "/usr/lib/apt/methods";
Dir::Bin::solvers "";
Dir::Bin::solvers:: "/usr/lib/apt/solvers";
Dir::Bin::dpkg "/usr/bin/dpkg";
Dir::Bin::bzip2 "/bin/bzip2";
Dir::Bin::xz "/usr/bin/xz";
Dir::Bin::lzma "/usr/bin/xz";
Dir::Media "";
Dir::Media::MountPath "/media/apt";
Dir::Log "var/log/apt";
Dir::Log::Terminal "term.log";
Dir::Log::History "history.log";
Dir::Ignore-Files-Silently "";
Dir::Ignore-Files-Silently:: "~$";
Dir::Ignore-Files-Silently:: "\.disabled$";
Dir::Ignore-Files-Silently:: "\.bak$";
Dir::Ignore-Files-Silently:: "\.dpkg-[a-z]+$";
Dir::Ignore-Files-Silently:: "\.save$";
Dir::Ignore-Files-Silently:: "\.orig$";
Dir::Ignore-Files-Silently:: "\.distUpgrade$";
Acquire "";
Acquire::cdrom "";
Acquire::cdrom::mount "/media/cdrom/";
Acquire::Languages "";
Acquire::Languages:: "en";
Acquire::Languages:: "none";
DPkg "";
DPkg::Pre-Install-Pkgs "";
DPkg::Pre-Install-Pkgs:: "/usr/bin/apt-listchanges --apt || test $? -ne 10";
DPkg::Pre-Install-Pkgs:: "/usr/sbin/dpkg-preconfigure --apt || true";
DPkg::Tools "";
DPkg::Tools::Options "";
DPkg::Tools::Options::/usr/bin/apt-listchanges "";
DPkg::Tools::Options::/usr/bin/apt-listchanges::Version "2";
DPkg::Post-Invoke "";
DPkg::Post-Invoke:: "/usr/bin/test -e /usr/share/dbus-1/system-services/org.freedesktop.PackageKit.service && /usr/bin/test -S /var/run/dbus/system_bus_socket && /usr/bin/gdbus call --system --dest org.freedesktop.PackageKit --object-path /org/freedesktop/PackageKit --timeout 4 --method org.freedesktop.PackageKit.StateHasChanged cache-update > /dev/null; /bin/echo > /dev/null";
Unattended-Upgrade "";
Unattended-Upgrade::Origins-Pattern "";
Unattended-Upgrade::Origins-Pattern:: "origin=Debian,archive=${distro_codename},label=Debian-Security";
CommandLine "";
CommandLine::AsString "apt-config dump";

-- (no /etc/apt/preferences present) --


-- /etc/apt/sources.list --

#deb file:///cdrom/ sarge main

#deb cdrom:[Debian GNU/Linux 3.1 r0a _Sarge_ - Official i386 Binary-1 (20050607)]/ unstable contrib main

#2012-8-27 deb http://www.emdebian.org/debian/ squeeze main
#2012-8-27 deb-src http://www.emdebian.org/debian/ squeeze main

#deb http://buildd.emdebian.org/emdebian-tools lenny main
#deb-src http://buildd.emdebian.org/emdebian-tools lenny main

#--deb http://www.emdebian.org/emdebian-tools testing main
#--deb-src http://www.emdebian.org/emdebian-tools testing main

#deb http://www.emdebian.org/emdebian-tools/ stable main
#deb-src http://www.emdebian.org/emdebian-tools/ stable main

## deb http://www.emdebian.org/debian/ jessie main
## deb-src http://www.emdebian.org/debian/ jessie main

deb http://ftp.pl.debian.org/debian jessie-backports main contrib non-free
deb-src http://ftp.pl.debian.org/debian jessie-backports main contrib  non-free

deb http://ftp.pl.debian.org/debian jessie main contrib non-free
deb-src http://ftp.pl.debian.org/debian jessie main contrib  non-free

deb http://security.debian.org/ jessie/updates main contrib
deb-src http://security.debian.org/ jessie/updates main contrib

-- System Information:
Debian Release: 8.0
  APT prefers testing
  APT policy: (990, 'testing')
Architecture: i386 (i686)

Kernel: Linux 3.16.0-4-586
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash
Init: systemd (via /run/systemd/system)

Versions of packages apt depends on:
ii  debian-archive-keyring  2014.3
ii  gnupg                   1.4.18-6
ii  libapt-pkg4.12          1.0.9.6
ii  libc6                   2.19-13
ii  libgcc1                 1:4.9.1-19
ii  libstdc++6              4.9.1-19

apt recommends no packages.

Versions of packages apt suggests:
pn  apt-doc     <none>
ii  aptitude    0.6.11-1+b1
ii  dpkg-dev    1.17.23
ii  python-apt  0.9.3.11
ii  synaptic    0.81.2

-- no debconf information

Message #10 received at 776910@bugs.debian.org (full text, mbox, reply):

From: David Kalnischkies <david@kalnischkies.de>

To: Rafal Pietrak <rafal@ztk-rp.eu>, 776910@bugs.debian.org

Subject: Re: Bug#776910: apt: upgrade from wheezy to jessie breaks in the middle

Date: Tue, 3 Feb 2015 15:17:14 +0100

[Message part 1 (text/plain, inline)]

Control: tag -1 - newcomer

Hi Rafal,

On Tue, Feb 03, 2015 at 10:20:26AM +0100, Rafal Pietrak wrote:
> My guess is that some limit on number of errors was taken into account
> unneceserly during an upgrade - upgrades are expected to rise trancient errors.

Your report seems to error into the total opposite unfortunately by not
mentioning a single error. Upgrading is a tricky business and basically
different for everyone (as which packages you have installed can vary
widely as you have ~30000 to choose from). Your report is hence as
actionable as a weather report saying: "It is going to rain tomorrow
somewhere on earth". That isn't really telling me much about if I should
carry an umbrella around or not on my adventure around my little patch of
dirt. For this, as well as here, we need "details, details, details" to
actually do something about it.

But my crystalball tells me that you might mean #776063 as dpkg shows in
this context a "too many errors" message (litmus test: the word "dbus"
was printed all over the place, right?).

If not we need at the very least the actual error message(s). The
current system state (/var/lib/dpkg/status) as well as the state before
the upgrade (the /var/backups/dpkg.status* file dated before the update)
could also be helpful.


On a more general note: Try not to guess in bugreports. You are the
eyewitness, you know the facts. I am the guy on jury duty who has to
come up with a coherent story of what happened and why. I know its
tempting to "add" evidence as a witness, but that can spoil the whole
process.


Best regards

David Kalnischkies

P.S.: The 'newcomer' tag is for maintainers to indicate "bitsized" bugs
which a newcomer to the project/package can try to tackle to get
started. I wouldn't recommend these sort of upgrade bugs as a starting
point… and we certainly don't need to label bugs from newcomers as such
(which I guess is what you meant it to mean) as a bug is a bug, it isn't
worse just because a longtime contributor reported it.

[signature.asc (application/pgp-signature, inline)]

Message #17 received at 776910@bugs.debian.org (full text, mbox, reply):

From: Rafal Pietrak <rafal@ztk-rp.eu>

To: David Kalnischkies <david@kalnischkies.de>, 776910@bugs.debian.org

Subject: Re: Bug#776910: apt: upgrade from wheezy to jessie breaks in the middle

Date: Tue, 03 Feb 2015 16:34:54 +0100

W dniu 03.02.2015 o 15:17, David Kalnischkies pisze:
> Control: tag -1 - newcomer
>
> Hi Rafal,
>
> On Tue, Feb 03, 2015 at 10:20:26AM +0100, Rafal Pietrak wrote:
>> My guess is that some limit on number of errors was taken into account
>> unneceserly during an upgrade - upgrades are expected to rise trancient errors.
> Your report seems to error into the total opposite unfortunately by not
> mentioning a single error. Upgrading is a tricky business and basically
OK. I can see it looks that way.

> different for everyone (as which packages you have installed can vary
> widely as you have ~30000 to choose from). Your report is hence as
> actionable as a weather report saying: "It is going to rain tomorrow
> somewhere on earth". That isn't really telling me much about if I should
> carry an umbrella around or not on my adventure around my little patch of
> dirt. For this, as well as here, we need "details, details, details" to
> actually do something about it.
>
> But my crystalball tells me that you might mean #776063 as dpkg shows in
> this context a "too many errors" message (litmus test: the word "dbus"
> was printed all over the place, right?).

It might, but I don't think so.

1. in my case there was no such thing a long trail of lines with "dbus" 
word in them.

2. I do _vaguely_ remember last line above the shell prompt (after 
upgrade stopped) saying: "too many errors" (after quite long running 
upgrade).

3. to my surprise, "apt-get -f install; apt-get dist-upgrade" did help 
... to some extend: the upgrade finished without next stop, but the 
system is "not actually usable" (I've filed two more bugreports 
regarding that).

4. the entire process/failure resembles what I've experiences while 
upgrading to wheezy some two years ago: at that time, I didn't have 
correct cpufreq-control and the upgrade did overheated the system, which 
rebooted during the process. After that I did numerous manual "pushing" 
of the upgrade, and (again _vaguely_ ) remember finding some 
"apt-max-errors", which I've (remember) increasing then, which helped a lot.

5. I'm filing this bugreport, because this time there was no 
"reboot-in-the middle", so I'd expect, the "apt-max-errors" (if it 
really exists - I cannot find it now) is truely too low for "an average 
system" .... I'd imagine, that upgrade from one major release to another 
should set it temporarly to infinity ;7, but may be not.


>
> If not we need at the very least the actual error message(s). The
> current system state (/var/lib/dpkg/status) as well as the state before
> the upgrade (the /var/backups/dpkg.status* file dated before the update)
> could also be helpful.

The pervious dpkg.status is gone, sorry.

But current "dpkg.status" is c.a. 4.5MB. Is it all right to upload it 
anyway?

>
>
> On a more general note: Try not to guess in bugreports. You are the
> eyewitness, you know the facts. I am the guy on jury duty who has to
> come up with a coherent story of what happened and why. I know its
> tempting to "add" evidence as a witness, but that can spoil the whole
> process.

Yes, It is tempting :)

In that case I should say, that I've only witnessed an unexpected and 
unprovoked single break during the upgrade. Which was "fully" mended by 
only running "apt -f install" followed by a single "apt-get 
dist-upgrade". Yet, the system (although reported by apt-get as "fully 
up to date") is not fully functional.

>
>
> Best regards
>
> David Kalnischkies
>
> P.S.: The 'newcomer' tag is for maintainers to indicate "bitsized" bugs
> which a newcomer to the project/package can try to tackle to get
> started. I wouldn't recommend these sort of upgrade bugs as a starting
> point… and we certainly don't need to label bugs from newcomers as such
> (which I guess is what you meant it to mean) as a bug is a bug, it isn't
> worse just because a longtime contributor reported it.
Ups. (I thought it defines the reporting person). Sorry for misleading tag.

-R

Message #22 received at 776910@bugs.debian.org (full text, mbox, reply):

From: Niels Thykier <niels@thykier.net>

To: Rafal Pietrak <rafal@ztk-rp.eu>, 776910@bugs.debian.org, David Kalnischkies <david@kalnischkies.de>

Subject: Re: Bug#776910: apt: upgrade from wheezy to jessie breaks in the middle

Date: Mon, 23 Feb 2015 22:27:21 +0100

On 2015-02-03 16:34, Rafal Pietrak wrote:
> 
> W dniu 03.02.2015 o 15:17, David Kalnischkies pisze:
>> Control: tag -1 - newcomer
>>
>> Hi Rafal,
>>
>> [...]
> 
> It might, but I don't think so.
> 
> 1. in my case there was no such thing a long trail of lines with "dbus"
> word in them.
> 

Sadly, dbus was not the only trigger issue remaining.  We have had to
remove the "trigger cycle" check in dpkg for Jessie because we keep
seeing this issue.

> 2. I do _vaguely_ remember last line above the shell prompt (after
> upgrade stopped) saying: "too many errors" (after quite long running
> upgrade).
> 

I suspect that line is from dpkg.  There are example of:

"""
dpkg: too many errors, stopping
"""

Maybe you were looking for "dpkg <...> --abort-after=999999"?  At least
that is what a quick googling suggests.

> 3. to my surprise, "apt-get -f install; apt-get dist-upgrade" did help
> ... to some extend: the upgrade finished without next stop, but the
> system is "not actually usable" (I've filed two more bugreports
> regarding that).
> 

Trigger issues (among other?) tends be "solved" by simply re-trying, so
the above certainly does not rule out a trigger issue (but it does not
confirm it either).

> [...]
> 
> 5. I'm filing this bugreport, because this time there was no
> "reboot-in-the middle", so I'd expect, the "apt-max-errors" (if it
> really exists - I cannot find it now) is truely too low for "an average
> system" .... I'd imagine, that upgrade from one major release to another
> should set it temporarly to infinity ;7, but may be not.
> 

Rather, there should be no errors during an upgrade - ever.  That is one
of the goals for upgrades.  Somehow, setting such a (fictive?) option to
infinite seems to be working around a problem that should not exist in
the first place.

> 
>>
>> If not we need at the very least the actual error message(s). The
>> current system state (/var/lib/dpkg/status) as well as the state before
>> the upgrade (the /var/backups/dpkg.status* file dated before the update)
>> could also be helpful.
> 
> The pervious dpkg.status is gone, sorry.
> 
> But current "dpkg.status" is c.a. 4.5MB. Is it all right to upload it
> anyway?
> 
> [...]
> -R
> 


I believe the BTS will accept it (you may want to gzip it though), but
lists.d.o will silently discard it.  So follow up with a separate mail
afterwards to ensure we notice once you have done it.


Thanks,
~Niels

Message #27 received at 776910@bugs.debian.org (full text, mbox, reply):

From: Rafał Pietrak <rafal@ztk-rp.eu>

To: Niels Thykier <niels@thykier.net>, 776910@bugs.debian.org, David Kalnischkies <david@kalnischkies.de>

Subject: Re: Bug#776910: apt: upgrade from wheezy to jessie breaks in the middle

Date: Tue, 24 Feb 2015 09:53:11 +0100

[Message part 1 (text/plain, inline)]

W dniu 23.02.2015 22:27, Niels Thykier pisze:
> On 2015-02-03 16:34, Rafal Pietrak wrote:
[--------------------]
>> 2. I do _vaguely_ remember last line above the shell prompt (after
>> upgrade stopped) saying: "too many errors" (after quite long running
>> upgrade).
>>
> I suspect that line is from dpkg.  There are example of:
>
> """
> dpkg: too many errors, stopping
> """
>
> Maybe you were looking for "dpkg <...> --abort-after=999999"?  At least
> that is what a quick googling suggests.

Possibly.

[---------------]
>
> 5. I'm filing this bugreport, because this time there was no
> "reboot-in-the middle", so I'd expect, the "apt-max-errors" (if it
> really exists - I cannot find it now) is truely too low for "an average
> system" .... I'd imagine, that upgrade from one major release to another
> should set it temporarly to infinity ;7, but may be not.
>
> Rather, there should be no errors during an upgrade - ever.  That is one
> of the goals for upgrades.  Somehow, setting such a (fictive?) option to
> infinite seems to be working around a problem that should not exist in
> the first place.

Yes and no.

*actual errors* yes; but I don't think I've seen them during the upgrade.

1. During the upgrade I have seen some "errors"/complains about 
unresolved dependencies, or conflicts "... but continiueing anyway as 
requested" the apt-get said.

2. at one point it said "too many errors" and stopped.

3. then a simple "apt-get -f install; apt-get dist-upgrade" completed 
the task (sort of) cleanly.

MHC (e.g. Conclusion) is, that there were "kind of errors", which were 
*expected* during the *dist-upgrade* from release to release, but 
apt-get counted them anyway....and checked against a treshold ....  
which it shouldn't in that particular situation of major release change.

[------------------]
>
> I believe the BTS will accept it (you may want to gzip it though), but
> lists.d.o will silently discard it.  So follow up with a separate mail
> afterwards to ensure we notice once you have done it.

I've learned, that attachments are OK. So here is my (most relevant) 
dpkg.status after the upgrade. Regretably, at this point I don't have 
the one from the date of an upgrade.

-R

[dpkg.status.6.gz (application/gzip, attachment)]

Message #32 received at 776910@bugs.debian.org (full text, mbox, reply):

From: Niels Thykier <niels@thykier.net>

To: Rafał Pietrak <rafal@ztk-rp.eu>, 776910@bugs.debian.org, David Kalnischkies <david@kalnischkies.de>

Subject: Re: Bug#776910: apt: upgrade from wheezy to jessie breaks in the middle

Date: Mon, 02 Mar 2015 22:59:16 +0100

On 2015-02-24 09:53, Rafał Pietrak wrote:
> W dniu 23.02.2015 22:27, Niels Thykier pisze:
>> On 2015-02-03 16:34, Rafal Pietrak wrote:
> [...]
>>
>> 5. I'm filing this bugreport, because this time there was no
>> "reboot-in-the middle", so I'd expect, the "apt-max-errors" (if it
>> really exists - I cannot find it now) is truely too low for "an average
>> system" .... I'd imagine, that upgrade from one major release to another
>> should set it temporarly to infinity ;7, but may be not.
>>
>> Rather, there should be no errors during an upgrade - ever.  That is one
>> of the goals for upgrades.  Somehow, setting such a (fictive?) option to
>> infinite seems to be working around a problem that should not exist in
>> the first place.
> 
> Yes and no.
> 
> *actual errors* yes; but I don't think I've seen them during the upgrade.
> 
> 1. During the upgrade I have seen some "errors"/complains about
> unresolved dependencies, or conflicts "... but continiueing anyway as
> requested" the apt-get said.
> 

Those are from dpkg (though I do appreciate that telling the difference
between dpkg and apt-get output is difficult if you don't know them by
heart or the interplay between apt and dpkg).  However, I am fairly sure
dpkg does /not/ have a max counter on those.

I suspect what is happening here is that dpkg is complaining and would
/normally/ have aborted with an error, but continues anyway because apt
invoked dpkg with a --force-depends (or something similar).

> 2. at one point it said "too many errors" and stopped.
> 
> 3. then a simple "apt-get -f install; apt-get dist-upgrade" completed
> the task (sort of) cleanly.
> 

Again, smells like a trigger issue.  Though I will leave it to David (or
another apt maintainer) to analyse the dpkg status log you included.

For reference, the trigger check in dpkg was reverted.  If it turns out
that you were experiencing a trigger issue, then it is now solved already.

> [...]
>>
>> I believe the BTS will accept it (you may want to gzip it though), but
>> lists.d.o will silently discard it.  So follow up with a separate mail
>> afterwards to ensure we notice once you have done it.
> 
> I've learned, that attachments are OK. So here is my (most relevant)
> dpkg.status after the upgrade. Regretably, at this point I don't have
> the one from the date of an upgrade.
> 
> -R

Ack, thanks.  For reference, the file was too large to reach
deity@lists.debian.org (as I described).  To anyone reading the list but
not following the bug directly, the file is at [1].

~Niels

[1]
https://bugs.debian.org/cgi-bin/bugreport.cgi?msg=27;filename=dpkg.status.6.gz;att=1;bug=776910

Message #37 received at 776910@bugs.debian.org (full text, mbox, reply):

From: David Kalnischkies <david@kalnischkies.de>

To: Rafał Pietrak <rafal@ztk-rp.eu>

Cc: 776910@bugs.debian.org, Niels Thykier <niels@thykier.net>

Subject: Re: Bug#776910: apt: upgrade from wheezy to jessie breaks in the middle

Date: Sun, 8 Mar 2015 14:25:03 +0100

[Message part 1 (text/plain, inline)]

On Tue, Feb 24, 2015 at 09:53:11AM +0100, Rafał Pietrak wrote:
> W dniu 23.02.2015 22:27, Niels Thykier pisze:
> >On 2015-02-03 16:34, Rafal Pietrak wrote:
> [--------------------]
> >>2. I do _vaguely_ remember last line above the shell prompt (after
> >>upgrade stopped) saying: "too many errors" (after quite long running
> >>upgrade).
> >>
> >I suspect that line is from dpkg.  There are example of:
> >
> >"""
> >dpkg: too many errors, stopping
> >"""
> >
> >Maybe you were looking for "dpkg <...> --abort-after=999999"?  At least
> >that is what a quick googling suggests.
> 
> Possibly.

No, you are not looking for this, like ever. apt and dpkg are tidily
coupled so people usually have their problems figuring out which message
comes from which program, but that is okay as there is just on general
rule: If apt exits non-zero (= it fails) then something really bad
happened, which shouldn't have happened and which needs to be fixed for
the release. No strange workarounds for users. End of story.

apt and dpkg have various points at which they consider something to be
"really bad" and you can configure some of them to a certain degree, but
that isn't advisable. It will usually fail anyway in the best case and
open an all consuming crack in the space-time continuum in the worst
case. The reason is simply that they aren't failing just for fun and try
really hard to avoid any kind of problem to begin with.

They also have various points which they consider "bad" and complain
about it, but carry on anyway in the hope it will work out anyway. It
usually does, but if it doesn't, these messages give hints where it all
started to go downhill.


> >5. I'm filing this bugreport, because this time there was no
> >"reboot-in-the middle", so I'd expect, the "apt-max-errors" (if it
> >really exists - I cannot find it now) is truely too low for "an average
> >system" .... I'd imagine, that upgrade from one major release to another
> >should set it temporarly to infinity ;7, but may be not.
> >
> >Rather, there should be no errors during an upgrade - ever.  That is one
> >of the goals for upgrades.  Somehow, setting such a (fictive?) option to
> >infinite seems to be working around a problem that should not exist in
> >the first place.
> 
> Yes and no.
> 
> *actual errors* yes; but I don't think I've seen them during the upgrade.
> 
> 1. During the upgrade I have seen some "errors"/complains about unresolved
> dependencies, or conflicts "... but continiueing anyway as requested" the
> apt-get said.
> 
> 2. at one point it said "too many errors" and stopped.
> 
> 3. then a simple "apt-get -f install; apt-get dist-upgrade" completed the
> task (sort of) cleanly.
> 
> MHC (e.g. Conclusion) is, that there were "kind of errors", which were
> *expected* during the *dist-upgrade* from release to release, but apt-get
> counted them anyway....and checked against a treshold ....  which it
> shouldn't in that particular situation of major release change.

The "1." you mention are complains of dpkg about how apt is talking to
it. They are 'normal' and 'okay' and are unlimited.
(As a user) Ignore them.

"2." is very very likely dpkg running into a trigger loop and unrelated
to the "1.". (Well, they could be related potentially, but they aren't
counting for the 'too many errors', but just influence the formation of
loops). The errors leading to "2." look a lot like "1." through, but
they are different in such a way that they aren't recoverable if they
can't be resolved, so dpkg is trying hard to find a solution, but at
some point it has to give up or it would just run forever in circles
(it did, in certain buggy versions).

"3." works (sometimes) because apt rethinks the upgrade, might come to
a different solution (still with the same packages involved, but in
a different order) and in this one dpkg doesn't end in a loop.
That works ONLY in this case as apt and dpkg have a deep communication
problem here. Just rerunning apt usually doesn't work (otherwise we
could just do that by default, right?).

The think is that I figured out a while back that the way apt is talking
to dpkg, dpkg can create a loop for itself without apt knowing before it
is actually too late. That is made worse by the fact that apt doesn't
know that dpkg cares about these loops anyhow (it didn't until very
recently) and apt doesn't know such loops can exist – in fact it pretends
that the things (= triggers) which create these loops do not even
exist as they are in theory an implementation detail of dpkg. The other
RC bugs apt currently has deal with this looping, so if you really want
to you can look into them if you want more details, but I would avoid it
if I could. Also requires a fair bit of deep knowledge even most DDs do
not concern themselves with… so don't say I haven't warned you. ;)

The important part is that loop detection was dropped from jessie by
request of Nils (thanks a million), so that what you saw will not be
seen with released jessie and hence in a wheezy to jessie upgrade.


> >I believe the BTS will accept it (you may want to gzip it though), but
> >lists.d.o will silently discard it.  So follow up with a separate mail
> >afterwards to ensure we notice once you have done it.
> 
> I've learned, that attachments are OK. So here is my (most relevant)
> dpkg.status after the upgrade. Regretably, at this point I don't have the
> one from the date of an upgrade.

Thanks, but the status is indeed 'too late'. Its your system already
fully upgraded to jessie. Good would have been your wheezy status, or
the one in between, but well, that can be attributed to me being such
a slowpoke in replying, sorry. :/

I am inclined to close this as "not an issue anymore" instead of merging
it with the other trigger loop bugs as we have enough of them already
with very similar information (to be fair, I am inclined to close them
as well, but I guess it will be a jessie-ignore by Nils [or another
release teamer] instead to scare me).

Or is there anything left unanswered/open?


Best regards

David Kalnischkies

[signature.asc (application/pgp-signature, inline)]

Message #42 received at 776910-done@bugs.debian.org (full text, mbox, reply):

From: David Kalnischkies <david@kalnischkies.de>

To: 771428-done@bugs.debian.org, 776910-done@bugs.debian.org

Subject: Close resolved trigger loop bugs

Date: Mon, 9 Mar 2015 22:19:12 +0100

[Message part 1 (text/plain, inline)]

Control: notfound -1 apt/1.0.9.3
Control: severity -1 normal

Hi everyone,

as announced by Nils in the recent Jessie status update [0] trigger
cycle detection was pulled from the jessie release, which means the
reports here are no longer reproducible/valid and hence it was requested
that we reflect that accordingly.

Further more, especially the dbus issue reported in 3 out of 4 bugs here
was no longer present even before that as dbus in version 1.8.14-2
switched to -noawait triggers, which can't produce loops by definition.
See [1] if you are interested in how that happened with -await triggers
and for why it can't happen with -noawait (beware, technical).

This means that there is neither a new/fixed version of apt nor that
there exists a situation in which this is a bug, which is what I try to
model by marking as notfound and downgrading to normal – after all, if
it doesn't affect any release anymore at the moment, it can't effect
(and/or delay) the jessie release as a release critical bug.


The issue of trigger loops as such will be revisited after the release
of jessie. There are potentially massive changes needed in all levels of
package management to cope with issues found while the check was active
in dpkg, so there is even a chance that even the theories outlined in
the bugs here is no longer completely true, so that it is better to open
new bugs for then current issues rather than keeping these now historic
reports open for the possibility of them maybe still applying in the
post-jessie future.


At this point I would like to thank everyone reporting, discussing and
working on this bugclass! It might look at first like this was in vain
now that the feature itself is pulled and me closing these bugs, but the
jessie release benefited from all changes made in this context here and
this is absolutely something we will revisit soon on dpkg@ and deity@!

We are all just admitting that this will take longer and is harder to
complete than initially anticipated, which is a good step forward all by
itself already and helps focusing at the task of releasing jessie now.

So thanks again and "see you all on the other side".


Best regards

David Kalnischkies

[0] https://lists.debian.org/debian-devel-announce/2015/03/msg00002.html
[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=776063#64

[signature.asc (application/pgp-signature, inline)]

Send a report that this bug log contains spam.