Debian Bug report logs - #931566
Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Display info messages

Message #3 received at submit@bugs.debian.org (full text, mbox, reply):

From: Christoph Berg <myon@debian.org>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Date: Sun, 7 Jul 2019 20:39:10 +0200

Package: apt
Version: 1.8.1
Severity: important

Hi,

I and #debian-devel in general were pretty surprised that our "buster"
systems couldn't properly "apt-get update" anymore today:

+ apt-get update
Get:1 http://mirror.hetzner.de/debian/packages buster InRelease [118 kB]
Get:2 http://security.debian.org/debian-security buster/updates InRelease [39.1 kB]
Get:3 http://ftp.de.debian.org/debian buster InRelease [118 kB]
Hit:4 http://apt.postgresql.org/pub/repos/apt buster-pgdg InRelease
Hit:5 http://atalia.postgresql.org/pub/repos/apt buster-pgdg-testing InRelease
Reading package lists...
E: Repository 'http://mirror.hetzner.de/debian/packages buster InRelease' changed its 'Suite' value from 'testing' to 'stable'
E: Repository 'http://security.debian.org/debian-security buster/updates InRelease' changed its 'Suite' value from 'testing' to 'stable'
E: Repository 'http://ftp.de.debian.org/debian buster InRelease' changed its 'Suite' value from 'testing' to 'stable'

For me, this killed the update process for the build chroots for
apt.postgresql.org.

I guess the same thing will happen once buster is turning oldstable,
breaking out all our stable installations out there. IMHO that's bad.

It might make sense to send a signal to people tracking suites, but if
sources.list is on a codename, it doesn't make any sense to break
systems that are totally fine.

Please consider setting Acquire::AllowReleaseInfoChange::Suite "true";

Or maybe silently do the change when running non-interactively.
Thanks.

Christoph

PS: Please document AllowReleaseInfoChange.

Message #6 received at submit@bugs.debian.org (full text, mbox, reply):

From: Christoph Berg <myon@debian.org>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: Re: Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Date: Mon, 8 Jul 2019 09:11:48 +0200

Control: tags -1 buster sid

Re: To Debian Bug Tracking System 2019-07-07 <20190707183910.GA22754@msg.df7cb.de>
> Please consider setting Acquire::AllowReleaseInfoChange::Suite "true";

Fwiw, I think this needs fixing in buster, not just in unstable.

Christoph

Message #13 received at 931566@bugs.debian.org (full text, mbox, reply):

From: Julien Cristau <jcristau@debian.org>

To: Christoph Berg <myon@debian.org>, 931566@bugs.debian.org

Subject: Re: Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Date: Mon, 8 Jul 2019 10:17:27 +0200

Control: severity -1 serious

On Mon, Jul 08, 2019 at 09:11:48AM +0200, Christoph Berg wrote:
> Control: tags -1 buster sid
> 
> Re: To Debian Bug Tracking System 2019-07-07 <20190707183910.GA22754@msg.df7cb.de>
> > Please consider setting Acquire::AllowReleaseInfoChange::Suite "true";
> 
> Fwiw, I think this needs fixing in buster, not just in unstable.
> 
Agree.  The user experience from this is horrible.

Cheers,
Julien

Message #22 received at 931566@bugs.debian.org (full text, mbox, reply):

From: Andreas Beckmann <anbe@debian.org>

To: Debian Bug Tracking System <931566@bugs.debian.org>

Subject: Re: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Date: Mon, 08 Jul 2019 11:19:28 +0200

Followup-For: Bug #931566
Control: tag -1 experimental

IMHO, the intention behind this change is good.
The implementation was completely untested. Well, it's kind of hard to
properly test such a change.
The documentation is not really helpful.

I initially thought this was caused by my rather complicated setup.
Until I experienced it on a machine installed with buster two weeks ago
and not yet "customized" by me.

Multiple errors like this:
E: Repository 'http://security-cdn.debian.org stretch/updates InRelease'
changed its 'Suite' value from 'stable' to 'oldstable'
N: This must be accepted explicitly before updates for this repository
can be applied. See apt-secure(8) manpage for details.

apt-secure(8) says:

INFORMATION CHANGES
       A Release file contains beside the checksums for the files in the
       repository also general information about the repository like the
       origin, codename or version number of the release.

       This information is shown in various places so a repository owner
       should always ensure correctness. Further more user configuration
       like apt_preferences(5) can depend and make use of this
       information.
       Since version 1.5 the user must therefore explicitly confirm
       changes to signal that the user is sufficiently prepared e.g. for
       the new major release of the distribution shipped in the
       repository (as e.g. indicated by the codename).

OK, and what do I do now? How do I confirm that?

It took me some time to discover --allow-releaseinfo-change in
apt-get(8).


Andreas

PS: my apt pinning is exactly setup to prevent unwanted release
changes even if sources.list contains everything from wheezy to
experimental ;-)

Message #29 received at 931566@bugs.debian.org (full text, mbox, reply):

From: Julian Andres Klode <jak@debian.org>

To: Andreas Beckmann <anbe@debian.org>, 931566@bugs.debian.org

Subject: Re: Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Date: Mon, 8 Jul 2019 11:33:46 +0200

On Mon, Jul 08, 2019 at 11:19:28AM +0200, Andreas Beckmann wrote:
> Followup-For: Bug #931566
> Control: tag -1 experimental
> 
> IMHO, the intention behind this change is good.
> The implementation was completely untested. Well, it's kind of hard to
> properly test such a change.
> The documentation is not really helpful.
> 
> I initially thought this was caused by my rather complicated setup.
> Until I experienced it on a machine installed with buster two weeks ago
> and not yet "customized" by me.
> 
> Multiple errors like this:
> E: Repository 'http://security-cdn.debian.org stretch/updates InRelease'
> changed its 'Suite' value from 'stable' to 'oldstable'
> N: This must be accepted explicitly before updates for this repository
> can be applied. See apt-secure(8) manpage for details.
> 
> apt-secure(8) says:
> 
> INFORMATION CHANGES
>        A Release file contains beside the checksums for the files in the
>        repository also general information about the repository like the
>        origin, codename or version number of the release.
> 
>        This information is shown in various places so a repository owner
>        should always ensure correctness. Further more user configuration
>        like apt_preferences(5) can depend and make use of this
>        information.
>        Since version 1.5 the user must therefore explicitly confirm
>        changes to signal that the user is sufficiently prepared e.g. for
>        the new major release of the distribution shipped in the
>        repository (as e.g. indicated by the codename).
> 
> OK, and what do I do now? How do I confirm that?
> 
> It took me some time to discover --allow-releaseinfo-change in
> apt-get(8).

Please let's not discuss the manpage here. Bug#879786 has been filed for
that back in Oct 2017.

-- 
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer                              i speak de, en

Message #34 received at 931566@bugs.debian.org (full text, mbox, reply):

From: Julian Andres Klode <jak@debian.org>

To: Julien Cristau <jcristau@debian.org>, 931566@bugs.debian.org

Cc: Christoph Berg <myon@debian.org>

Subject: Re: Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Date: Mon, 8 Jul 2019 11:41:55 +0200

On Mon, Jul 08, 2019 at 10:17:27AM +0200, Julien Cristau wrote:
> Control: severity -1 serious
> 
> On Mon, Jul 08, 2019 at 09:11:48AM +0200, Christoph Berg wrote:
> > Control: tags -1 buster sid
> > 
> > Re: To Debian Bug Tracking System 2019-07-07 <20190707183910.GA22754@msg.df7cb.de>
> > > Please consider setting Acquire::AllowReleaseInfoChange::Suite "true";
> > 
> > Fwiw, I think this needs fixing in buster, not just in unstable.
> > 
> Agree.  The user experience from this is horrible.

We'll figure out what to do. That said, let's not rush things,
we've got until the next release to figure out what to do and
fix it (a fix does not help anyone affected by the change right
now anyway).

We likely do not want to allow arbitrary changes of Suite. One
option is to introduce a new field that specifies that the Suite
field will change to something else soon, and a field to indicate
release notes for that change.

These are important security and safety features we don't just want
to kill because they are inconvenient. (The big reason this was
introduced is that if you allow this to change, you can just serve
someone an "old" oldstable repository (that still says stable) instead
of stable, and thus starve the clients from updates).

-- 
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer                              i speak de, en

Message #37 received at 931566@bugs.debian.org (full text, mbox, reply):

From: Christoph Berg <myon@debian.org>

To: Julian Andres Klode <jak@debian.org>, Julien Cristau <jcristau@debian.org>, 931566@bugs.debian.org

Subject: Re: Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Date: Mon, 8 Jul 2019 11:49:39 +0200

Re: Julian Andres Klode 2019-07-08 <20190708113658.GA10368@debian.org>
> We likely do not want to allow arbitrary changes of Suite. One
> option is to introduce a new field that specifies that the Suite
> field will change to something else soon, and a field to indicate
> release notes for that change.

If you plan to add anything that relies on "notifications" in the
Release file, please keep in mind that the mechanism still needs to
work in setups that are supposed to run unattended. Build chroots in
my case, and probably most machines in any data center where there is
no admin typing "apt(-get) update" every other day.

Please don't over-engineer it.

Christoph

Message #42 received at 931566@bugs.debian.org (full text, mbox, reply):

From: Julien Cristau <jcristau@debian.org>

To: Julian Andres Klode <jak@debian.org>, 931566@bugs.debian.org, Christoph Berg <myon@debian.org>

Subject: Re: Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Date: Mon, 8 Jul 2019 11:50:49 +0200

On Mon, Jul  8, 2019 at 11:41:55 +0200, Julian Andres Klode wrote:

> On Mon, Jul 08, 2019 at 10:17:27AM +0200, Julien Cristau wrote:
> > Control: severity -1 serious
> > 
> > On Mon, Jul 08, 2019 at 09:11:48AM +0200, Christoph Berg wrote:
> > > Control: tags -1 buster sid
> > > 
> > > Re: To Debian Bug Tracking System 2019-07-07 <20190707183910.GA22754@msg.df7cb.de>
> > > > Please consider setting Acquire::AllowReleaseInfoChange::Suite "true";
> > > 
> > > Fwiw, I think this needs fixing in buster, not just in unstable.
> > > 
> > Agree.  The user experience from this is horrible.
> 
> We'll figure out what to do. That said, let's not rush things,
> we've got until the next release to figure out what to do and
> fix it (a fix does not help anyone affected by the change right
> now anyway).
> 
> We likely do not want to allow arbitrary changes of Suite. One
> option is to introduce a new field that specifies that the Suite
> field will change to something else soon, and a field to indicate
> release notes for that change.
> 
> These are important security and safety features we don't just want
> to kill because they are inconvenient. (The big reason this was
> introduced is that if you allow this to change, you can just serve
> someone an "old" oldstable repository (that still says stable) instead
> of stable, and thus starve the clients from updates).
> 
How does the current behaviour fix that?  If you replay an old repo the
user is still starved from updates (how could they not be, anyway).  But
in addition now users using a legitimate repo are *also* starved from
updates, so everyone loses?

Cheers,
Julien

Message #47 received at 931566@bugs.debian.org (full text, mbox, reply):

From: Julian Andres Klode <jak@debian.org>

To: Julien Cristau <jcristau@debian.org>

Cc: 931566@bugs.debian.org, Christoph Berg <myon@debian.org>

Subject: Re: Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Date: Mon, 8 Jul 2019 12:19:36 +0200

On Mon, Jul 08, 2019 at 11:50:49AM +0200, Julien Cristau wrote:
> On Mon, Jul  8, 2019 at 11:41:55 +0200, Julian Andres Klode wrote:
> 
> > On Mon, Jul 08, 2019 at 10:17:27AM +0200, Julien Cristau wrote:
> > > Control: severity -1 serious
> > > 
> > > On Mon, Jul 08, 2019 at 09:11:48AM +0200, Christoph Berg wrote:
> > > > Control: tags -1 buster sid
> > > > 
> > > > Re: To Debian Bug Tracking System 2019-07-07 <20190707183910.GA22754@msg.df7cb.de>
> > > > > Please consider setting Acquire::AllowReleaseInfoChange::Suite "true";
> > > > 
> > > > Fwiw, I think this needs fixing in buster, not just in unstable.
> > > > 
> > > Agree.  The user experience from this is horrible.
> > 
> > We'll figure out what to do. That said, let's not rush things,
> > we've got until the next release to figure out what to do and
> > fix it (a fix does not help anyone affected by the change right
> > now anyway).
> > 
> > We likely do not want to allow arbitrary changes of Suite. One
> > option is to introduce a new field that specifies that the Suite
> > field will change to something else soon, and a field to indicate
> > release notes for that change.
> > 
> > These are important security and safety features we don't just want
> > to kill because they are inconvenient. (The big reason this was
> > introduced is that if you allow this to change, you can just serve
> > someone an "old" oldstable repository (that still says stable) instead
> > of stable, and thus starve the clients from updates).
> > 
> How does the current behaviour fix that?  If you replay an old repo the
> user is still starved from updates (how could they not be, anyway).  But
> in addition now users using a legitimate repo are *also* starved from
> updates, so everyone loses?

Well, I guess I was off a bit, you replay a current oldstable - you can't
roll back to older Date values (also, Valid-Until).

Without the AllowReleaseInfo changes, all the user woudl get is a warning
that there's a mismatch, but no breakage and non-interactive systems would
thus silently not get updates. Now they'll have a failing systemd timer,
which I guess is better.

Sure, yes, in practice this means Codename changed as well. I'm not
sure there really are many repositories where only Suite can change in a
way that's security-relevant. Though, in any case, you might still end up
with broken pinning, thus it seems like a good idea to try to prevent this
as long as we don't mess up things.

Anyway, this was not my thing so I might be misrepresenting or missing
things :)

-- 
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer                              i speak de, en

Message #52 received at 931566@bugs.debian.org (full text, mbox, reply):

From: David Kalnischkies <david@kalnischkies.de>

To: 931566@bugs.debian.org

Cc: Julien Cristau <jcristau@debian.org>, Christoph Berg <myon@debian.org>

Subject: Re: Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Date: Tue, 9 Jul 2019 15:31:07 +0200

[Message part 1 (text/plain, inline)]

On Mon, Jul 08, 2019 at 12:19:36PM +0200, Julian Andres Klode wrote:
> Anyway, this was not my thing so I might be misrepresenting or missing
> things :)

First of all: I am happy that I still manage to "break the world" (FSVO
world) – in other words that is "my thing" , but I mostly forgot about
it as it was implemented two years ago at the start of the last
release cycle so that everyone would have enough time to cry at me …
worked brilliantly I guess… 😉

Anyway, that apt is enforcing the metadata isn't changing is a security
feature in that if you don't check an attacker can reply to a request to
foo-security with foo – perfectly signed and everything so apt has no
chance to detect this and the user believes everything is fine – even
seeing files downloaded from -security – while the user never receives
data for -security. foo has no Valid-Until header so the attacker can
keep that up for basically ever. Compared to that serving old versions
of -security itself is guarded by Valid-Until. Not serving any data has
basically the same result, but the errors involved might eventually
raise some alarm bells. So that is a good strategy to keep you from ever
installing security upgrades.


So, after establishing that metadata shouldn't change all the time, lets
look at the individual metadata pieces. Th recent thread on (not) using
suite/codename/version to refer to a release is not only done in
documentation and sources.list, but also in configuration for example in
apt_preferences (which apt could check theoretically) and unattended-
upgrades (which apt can't realistically check). Many of these
configuration pieces for break silently and/or run guard automatic
processes.

People are also not very consistent in that they refer to an release in
different ways depending on the situation: While the sources.list might
refer to buster, the config for unattended upgrades could easily refer
to packages from stable to install automatically.


A) For a user with "debian stable" in sources.list the change of the
Codename from buster to bullseye is a giant leap which should not
be done carelessly or even automatically in the background.

B) For a user with "debian buster" in sources.list the change of the
Suite from e.g. stable to oldstable is an important event as well; not
right at this moment as there is a grace period, but security support is
about to end and plans should be set in motion on how to deal with that.

(A & B assume Debian for brevity. In other repos changes can have even
bigger/different effects, but that mail would get even longer if I would
write yet more examples)

Both situations deserve nagging the user in my opinion – preferably with
a pointer to specific details [which apt clients do if a Release-Notes
field is present in the Release file] – even if we ignore the risk of
breaking existing configuration on the system.
APT isn't a newspaper, but at least for me it seems wrong that the one
tool people associate with getting packages has absolutely nothing to
say about big events at the vendor who issues my packages.


That said, while I might be able to hold my ground on A) with
the remark that setting Acquire::AllowReleaseInfoChange::Codename "true"
on system where you know that upgrade processes can be ignored. (setting
it to true by default and letting d-i disable it for >= standard feels
dirty).

On B) I got serious backlash through and I might agree given that I said
"grace period" myself and an apt client is basically removing that
period now. Not really what I wanted… As such I would propose what
Julian has hinted at already as we talked about it on IRC yesterday:


A Release file can declare that it will "soon" change its the value of
a field to foo and apt clients can notify users about this so
they can prepare (or, for that matter help testing if they feel like
it). In exchange, if the change happens and apt knows it announced it
before it can choose to accept the change without explicit user
confirmation & I would propose enabling that for the Suite field.


So, lets see how that might look for Debian buster if I "will have had"
deployed a time machine:

On 2019-06-11, perhaps a bit earlier:

$ cat buster/Release
Codename: buster
Future-Codename: bullseye
Suite: testing
Future-Suite: stable
Future-Version: 10.0
Future-Release-Notes: https://example.org/preparing-for-buster

$ apt update
N: Repository '…' changes its 'Suite' value from 'testing' to 'stable' soon.
N: Repository '…' changes its 'Codename' value from 'buster' to 'bullseye' soon.
N: Repository '…' changes its 'Version' value from '' to '10.0' soon.
N: More information about this can be found online in the Release notes at: https://example.org/preparing-for-buster

Some of these N: will be wrong depending on how you choose to refer
to the repository in your sources.list – I guess we could filter with
some heuristics – or not, I kinda like how that lists the options.
Anyway, as N are notices they do not change the exit status of clients
and might not even be shown – apt-get e.g. doesn't in non-interactive use.


On 2019-07-06, the release day:

$ cat buster/Release
Codename: buster
Suite: stable
Release-Notes: https://example.org/upgrading-to-buster

$ apt update  # from a system with the last update call after the 2019-06-11
N: Repository '…' changed its 'Suite' value from 'testing' to 'stable'.
N: More information about this can be found online in the Release notes at: https://example.org/upgrading-to-buster

If you have 'debian buster' in sources.list, if you have 'debian stable':

$ apt update
E: Repository '…' changed its 'Codename' value from 'buster' to 'bullseye'.
N: More information about this can be found online in the Release notes at: https://example.org/upgrading-to-buster
N: This must be accepted explicitly before updates for this repository can be applied. See apt-secure(8) manpage for details.
N: These changes can be acknowledged interactively with 'apt' or with --allow-releaseinfo-change.

If you happen to not upgrade between the announcement and the release
the later case will play out exactly the same, the first one will be an
error akin to the Codename case just with the Suite data instead.


I have the code for it mostly written, so that seems to work and isn't
too bad, but I am fairly open to feedback from everyone involved. Lets
just not wait again 2 years shall we? 😉

Code will be on salsa hopefully later today, if you want to have a look
or comment on specific implementation details (like field/option names,
wording of messages,… …) I would encourage you to comment there and
leave that bugreport for the overarching "this sucks!" and "greatest
thing since sliced bread!" on the whole infrastructure as for this to
work at least release, ftp & publicity team have to accept me imposing
work on them (which arguably they already do anyhow, but still) and
hence quickly derails if we argue about Soon/Upcoming/Next/Future-
in here, too.

Best regards

David Kalnischkies

[signature.asc (application/pgp-signature, inline)]

Message #55 received at 931566@bugs.debian.org (full text, mbox, reply):

From: Christoph Berg <myon@debian.org>

To: David Kalnischkies <david@kalnischkies.de>, 931566@bugs.debian.org, Julien Cristau <jcristau@debian.org>

Subject: Re: Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Date: Tue, 9 Jul 2019 15:55:02 +0200

Re: David Kalnischkies 2019-07-09 <20190709133107.gvib2ibj6w36j447@crossbow>
> A) For a user with "debian stable" in sources.list the change of the
> Codename from buster to bullseye is a giant leap which should not
> be done carelessly or even automatically in the background.

I agree that stopping "apt-get update" here makes sense.

> B) For a user with "debian buster" in sources.list the change of the
> Suite from e.g. stable to oldstable is an important event as well; not
> right at this moment as there is a grace period, but security support is
> about to end and plans should be set in motion on how to deal with that.

Refusing to continue with non-interactive (!) operation here is
totally wrong. The system is configured perfectly fine to use
"buster", and just because buster is moving from testing to stable, or
stable to oldstable doesn't justify breaking 100s of installations in
my data center.

The message that I should probably be looking into dist-upgrading my
data center will be sent to me in form of debian-announce, the general
news, or whatever other mechanism. I need to get that message once,
not once per system. What I don't need is 100s of systems breaking
"apt-get update" because that means I won't get monitoring messages
about security updates either (where one message per system makes
sense).

> APT isn't a newspaper, but at least for me it seems wrong that the one
> tool people associate with getting packages has absolutely nothing to
> say about big events at the vendor who issues my packages.

If you chose to send a message, that's ok, but the message must not
break non-interactive, unattended apt-get operation.

(At the very least, don't break "apt-get". For more fancy things,
there's "apt".)

> A Release file can declare that it will "soon" change its the value of
> a field to foo and apt clients can notify users about this so

Fwiw, I think this is seriously overengineered. We can't really test
it, and there's high chances the feature will break (itself or
something else) more than it might fix in practise.

Christoph

Message #60 received at 931566@bugs.debian.org (full text, mbox, reply):

From: Julien Cristau <jcristau@debian.org>

To: Christoph Berg <myon@debian.org>, David Kalnischkies <david@kalnischkies.de>, 931566@bugs.debian.org

Subject: Re: Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Date: Tue, 9 Jul 2019 18:52:58 +0200

On Tue, Jul  9, 2019 at 15:55:02 +0200, Christoph Berg wrote:

> Re: David Kalnischkies 2019-07-09 <20190709133107.gvib2ibj6w36j447@crossbow>
> > A Release file can declare that it will "soon" change its the value of
> > a field to foo and apt clients can notify users about this so
> 
> Fwiw, I think this is seriously overengineered. We can't really test
> it, and there's high chances the feature will break (itself or
> something else) more than it might fix in practise.
> 
Definitely agreed.

I think overall what you're trying to do here (the whole "notify the
user they're out of date" thing) does not belong in apt.  IMO it belongs
in higher level tools that are going to heavily depend on the use case
and so there's not really a good generic answer you can come up with at
the lowest (apt) level.

Cheers,
Julien

Message #65 received at 931566@bugs.debian.org (full text, mbox, reply):

From: David Kalnischkies <david@kalnischkies.de>

To: Julien Cristau <jcristau@debian.org>, 931566@bugs.debian.org

Cc: Christoph Berg <myon@debian.org>

Subject: Re: Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Date: Wed, 10 Jul 2019 12:05:03 +0200

[Message part 1 (text/plain, inline)]

On Tue, Jul 09, 2019 at 06:52:58PM +0200, Julien Cristau wrote:
> I think overall what you're trying to do here (the whole "notify the
> user they're out of date" thing) does not belong in apt.  IMO it belongs
> in higher level tools that are going to heavily depend on the use case
> and so there's not really a good generic answer you can come up with at
> the lowest (apt) level.

Well, all user-facing clients use libapt for their equivalent of "apt
update". So your one-off python script, apt, apt-get, aptitude,
synaptics, all software centers, unattended-upgrades, apt-file … they
all share the code – and the data which is downloaded by one of them for
them all.

It is the choice of the frontend performing the update to present
encountered problems and apt{,-get} chooses to push the messages to the
usual list at the end of the run as it is common for apt. It could just
as well do nothing or open a blicking popup dialog, whatever seems
sensible for both the user using the current frontend AND for the other
frontends – so yes, there needs to be a default generic good answer on
the lowest level as we otherwise need to tell users to pick ONE frontend
and use that for the rest of their life exclusively.

Earmarking potentially dangerous data in hope that all frontends will
implement proper safety procedures while dealing with them doesn't work.
That was quite (in)visible for unauthenticated packages and I am very
happy we got mostly right of it by now.


apt{,-get} could choose to implement checks if the changed metadata
breaks its configuration – and we might do if I am really bored – but
that would indeed be lots of code and we still have the problem that the
other frontends might be broken by the new data.


#931524 and #931649 alone show that user config is easily broken & that
users frequently miss obvious and well-known changes to repositories. Or
did we already forget the recent outcry as oldoldstable left the
mirrors? And that is the main archive of your distribution. I doubt
users are following changes in 3rdparty repos they are using any closer.

I agree that apt{,-get} isn't the best place to keep you posted about
this stuff, but libapt is the best (because it is the only) place to
download the data all frontends can use and if present apt{,-get} should
make use of it as users expect it to help them. Even better if there are
higher level tools making better use of the data!


BUT that isn't about notifying users or breaking configuration even if
that is the most visible in practice. You don't have to look particular
far for an opportunity to create disaster (at least in the eyes of the
user) if you allow an attacker to change one metadata field:

http://deb.devuan.org/devuan/dists/stable/Release
http://deb.devuan.org/merged/dists/stable/Release

The two release pockets differ by Suite only. [Okay, they differ by
Label, too, so apt would catch that – but that looks more like an
accident than deliberate. And how much impact can a Label change have…
that should be automatic, too! btw "Debian stable" and "Debian
stable-security" differ by Label only in buster].
(not an endorsement btw, just a semi-random pick from the census)

Or because everyone loves docker lets look there:
https://download.docker.com/linux/debian/dists/buster/Release
https://download.docker.com/linux/debian/dists/stretch/Release
https://download.docker.com/linux/raspbian/dists/buster/Release
https://download.docker.com/linux/ubuntu/dists/artful/Release
https://download.docker.com/linux/ubuntu/dists/zesty/Release
These indeed differ by Suite only and are therefore freely exchangeable
if we allow unsanctioned Suite changes.

Having Oracle in your trusted keyring? Great! Then I will serve you this
the next time you request the Debian unstable repository:
https://oss.oracle.com/debian/dists/unstable/main/binary-i386/Release
[caught by codename – or more realistically by signed-by, expect nobody
uses that. MR !33 to the rescue but this requires correct and relatively
stable metadata as well]

The last one is just the literally first hit on duckduckgo for '"Origin:
Debian" Release' for me. It isn't particularity hard to find others with
better usability factors.

So if the proposed solution is over engineered I am all ears for
alternatives which deal with these issues.


Best regards

David Kalnischkies

P.S.: Pinning and other actions in libapt by codename was the first big
feature I implemented a decade ago. So in Debian terms it isn't that old
and indeed lots of documentation still uses suite names for this – and
in many cases that isn't wrong. Other frontends got that feature even
later – assuming it is implemented at all by now. Reading mails worded
as if codenames were a universal truth is quite funny in that context.

[signature.asc (application/pgp-signature, inline)]

Message #68 received at 931566@bugs.debian.org (full text, mbox, reply):

From: Christoph Berg <myon@debian.org>

To: David Kalnischkies <david@kalnischkies.de>, Julien Cristau <jcristau@debian.org>, 931566@bugs.debian.org

Subject: Re: Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Date: Thu, 11 Jul 2019 21:50:14 +0200

Re: David Kalnischkies 2019-07-10 <20190710100502.yoe2umfxyiuglaw2@crossbow>
> http://deb.devuan.org/devuan/dists/stable/Release
> http://deb.devuan.org/merged/dists/stable/Release
> 
> The two release pockets differ by Suite only.

Hi,

this bug is not about fixing problems that devuan and the others might
have, this is about not breaking proper Debian Buster installations.
I appreciate the efforts put into sanely handling half-broken
repositories, but I'd still like to avoid the looming shitstorm on
the stable->oldstable switch in two years.

Christoph

Message #73 received at 931566@bugs.debian.org (full text, mbox, reply):

From: Julian Andres Klode <jak@debian.org>

To: David Kalnischkies <david@kalnischkies.de>, 931566@bugs.debian.org

Cc: Julien Cristau <jcristau@debian.org>, Christoph Berg <myon@debian.org>

Subject: Re: Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Date: Fri, 12 Jul 2019 16:52:36 +0200

[Message part 1 (text/plain, inline)]

On Tue, Jul 09, 2019 at 03:31:07PM +0200, David Kalnischkies wrote:
> On Mon, Jul 08, 2019 at 12:19:36PM +0200, Julian Andres Klode wrote:
> > Anyway, this was not my thing so I might be misrepresenting or missing
> > things :)
> 
> First of all: I am happy that I still manage to "break the world" (FSVO
> world) – in other words that is "my thing" , but I mostly forgot about
> it as it was implemented two years ago at the start of the last
> release cycle so that everyone would have enough time to cry at me …
> worked brilliantly I guess… 😉
> 
> Anyway, that apt is enforcing the metadata isn't changing is a security
> feature in that if you don't check an attacker can reply to a request to
> foo-security with foo – perfectly signed and everything so apt has no
> chance to detect this and the user believes everything is fine – even
> seeing files downloaded from -security – while the user never receives
> data for -security. foo has no Valid-Until header so the attacker can
> keep that up for basically ever. Compared to that serving old versions
> of -security itself is guarded by Valid-Until. Not serving any data has
> basically the same result, but the errors involved might eventually
> raise some alarm bells. So that is a good strategy to keep you from ever
> installing security upgrades.

This should not happen silently, as there'll be a warning that the
name of the distro in the sources.list entry matches neither Codename
nor Suite. 

We also can't do a rollback attack from current testing to an older
testing either, as the Date is not allowed to roll backwards.

I'm not convinced we are adding any actual security here - we should
just upgrade the warning for name mismatch between sources.list and
Release file to an error for that.


> So, after establishing that metadata shouldn't change all the time, lets
> look at the individual metadata pieces. Th recent thread on (not) using
> suite/codename/version to refer to a release is not only done in
> documentation and sources.list, but also in configuration for example in
> apt_preferences (which apt could check theoretically) and unattended-
> upgrades (which apt can't realistically check). Many of these
> configuration pieces for break silently and/or run guard automatic
> processes.
> 
> People are also not very consistent in that they refer to an release in
> different ways depending on the situation: While the sources.list might
> refer to buster, the config for unattended upgrades could easily refer
> to packages from stable to install automatically.
> 
> 
> A) For a user with "debian stable" in sources.list the change of the
> Codename from buster to bullseye is a giant leap which should not
> be done carelessly or even automatically in the background.

That's true, but leaving the user stranded without updates is not
helpful either.

Regarding the automatic upgrades: It seems that unattended-upgrades
only considers upgrades that match the system's codename, so it won't
automatically upgrade you to a new release.

> 
> B) For a user with "debian buster" in sources.list the change of the
> Suite from e.g. stable to oldstable is an important event as well; not
> right at this moment as there is a grace period, but security support is
> about to end and plans should be set in motion on how to deal with that.

It's not an important enough change to starve them from further security
updates until they acknowledge it.

> A Release file can declare that it will "soon" change its the value of
> a field to foo and apt clients can notify users about this so
> they can prepare (or, for that matter help testing if they feel like
> it). In exchange, if the change happens and apt knows it announced it
> before it can choose to accept the change without explicit user
> confirmation & I would propose enabling that for the Suite field.
> 
> 
> So, lets see how that might look for Debian buster if I "will have had"
> deployed a time machine:
> 
> On 2019-06-11, perhaps a bit earlier:
> 
> $ cat buster/Release
> Codename: buster
> Future-Codename: bullseye
> Suite: testing
> Future-Suite: stable
> Future-Version: 10.0
> Future-Release-Notes: https://example.org/preparing-for-buster

It turns out the only reasonable point in time to do that is during release,
that is, stable would always contain Future-Suite: oldstable (and similar for
others). Two reasons:

* It's super annoying having to do those changes for all releases at like
  freeze time
* If your machine was offline in the time between adding Future and the
  new stable release, it will still be broken.

-- 
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer                              i speak de, en

[signature.asc (application/pgp-signature, inline)]

Message #78 received at 931566@bugs.debian.org (full text, mbox, reply):

From: David Kalnischkies <david@kalnischkies.de>

To: Julian Andres Klode <jak@debian.org>, 931566@bugs.debian.org

Cc: Julien Cristau <jcristau@debian.org>, Christoph Berg <myon@debian.org>

Subject: Re: Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Date: Sun, 21 Jul 2019 12:20:49 +0200

[Message part 1 (text/plain, inline)]

On Fri, Jul 12, 2019 at 04:52:36PM +0200, Julian Andres Klode wrote:
> On Tue, Jul 09, 2019 at 03:31:07PM +0200, David Kalnischkies wrote:
> > On Mon, Jul 08, 2019 at 12:19:36PM +0200, Julian Andres Klode wrote:
> > Anyway, that apt is enforcing the metadata isn't changing is a security
> > feature in that if you don't check an attacker can reply to a request to
> > foo-security with foo – perfectly signed and everything so apt has no
> > chance to detect this and the user believes everything is fine – even
> > seeing files downloaded from -security – while the user never receives
> > data for -security. foo has no Valid-Until header so the attacker can
> > keep that up for basically ever. Compared to that serving old versions
> > of -security itself is guarded by Valid-Until. Not serving any data has
> > basically the same result, but the errors involved might eventually
> > raise some alarm bells. So that is a good strategy to keep you from ever
> > installing security upgrades.
> 
> This should not happen silently, as there'll be a warning that the
> name of the distro in the sources.list entry matches neither Codename
> nor Suite.

Codename and Suite match for Debian and -security:
http://deb.debian.org/debian/dists/buster/Release
http://security.debian.org/debian-security/dists/buster/updates/Release

They differ in Label (, Version) and Components only. As such I can
easily hand you the Release file for Debian if you ask for security.

Components has its own verify, but as both styles ("updates/main" and
"main") exist in realworld scenarios we can't be that strict about it.

Version is a fun one to verify as that would imply enforcing a specific
version scheme.

Yes, they have different keys, but while the option exists nobody really
configures signed-by per sources.list entry (or Release or …) for
various reasons. !33 wants to help with that, but will/does depend on
the metadata to match keys entries to sources.list entries.


bullseye has that "fixed" as Suite & Codename will vary for both
archives, but there weren't only positive replies about that and I am
not that arrogant to call all other repositories "broken" just because
they don't exactly copy the choices Debian made – and even if they would
they would be a candidate for the flipping.


I guess Julian was thinking of the Docker example I had in a later
message which is caught by having the wrong suite compared to the
sources.list; but that example I included mainly to show that Debians
interpretation of what Suite and Codename are might not be applicable to
other repositories – for Docker, it makes perfect sense to say that
"buster" is the suite they release stuff for. It is at least quite
obviously not the codename for their product.


> We also can't do a rollback attack from current testing to an older
> testing either, as the Date is not allowed to roll backwards.

Assuming I don't catch you on your first update I just have to pick
a stable release (update) after the last security update you have
to switch you over to never receiving security updates again (perhaps
staling -security in the bounds of Valid-Until until stable has caught
up with the next point release).

(Me being a MITM or an evil [https-]mirror operator of course)


> I'm not convinced we are adding any actual security here - we should
> just upgrade the warning for name mismatch between sources.list and
> Release file to an error for that.

So, to be clear, you think about reverting the entire thing for all
fields – which is considerably more than what this bugreport is asking
for.

It is a valid option of course, but personally I would like to hear
reasons to allowing Origin/Label to change – so far I have only seen
complains about (mostly) Suite and (some) Codename [and for the record,
I also got some positive replies for both, so regardless the default,
I would at the very least like to retain the option] – as that
jeopardises stuff like !33 as well.


> > A) For a user with "debian stable" in sources.list the change of the
> > Codename from buster to bullseye is a giant leap which should not
> > be done carelessly or even automatically in the background.
> 
> That's true, but leaving the user stranded without updates is not
> helpful either.

I would personally consider no-upgrades a better scenario than
not-for-me-upgrades, but then I wouldn't classify a failing automatic
process as "stranding" given that human intervention is needed in both
scenarios.


> > B) For a user with "debian buster" in sources.list the change of the
> > Suite from e.g. stable to oldstable is an important event as well; not
> > right at this moment as there is a grace period, but security support is
> > about to end and plans should be set in motion on how to deal with that.
> 
> It's not an important enough change to starve them from further security
> updates until they acknowledge it.

I said as much, perhaps a bit clearer a few lines below that, and I think
we have consent on that – it is "just" a matter of how to achieve that.


> It turns out the only reasonable point in time to do that is during release,
> that is, stable would always contain Future-Suite: oldstable (and similar for
> others). Two reasons:
>
> * It's super annoying having to do those changes for all releases at like
>   freeze time

On a sidenote, it is sad that we don't update releases more often in
terms of the metadata attached to them like Valid-Until which I think we
should solve eventually – but that is a different issue we should work
on nonetheless.


> * If your machine was offline in the time between adding Future and the
>   new stable release, it will still be broken.

Well, yes, … but that is a long time without security updates as well,
so if your automatic process hasn't worked for more than a month my
sympathy for breaking it "further" is limited.


Anyway, as said, we could hardcode as well if it is such a pain to do the
announcement dance for Debian currently. I had to update the MR anyhow
as I had used the wrong code format (") {\n" vs ")\n{").

It's marked WIP as it is a) on top of Future, but could be just as
easily done without it and b) has no tests included yet c) it is very
hardcoded – I would like to have at least an option to disable it.

It is limited to Debian by Origin as a safety measure as "testing" and
"stable" are way to generic terms to give special meaning for all
possible repositories, but if we consider that variant, a config option
to add other origins so derivatives can opt in would be nice. Not too
hard to implement, just not feeling like doing it while a complete
revert is on the table (and a bit -ENOTIME, too).


Best regards

David Kalnischkies

[signature.asc (application/pgp-signature, inline)]

Message #83 received at 931566@bugs.debian.org (full text, mbox, reply):

From: Paul Wise <pabs@debian.org>

To: 931566@bugs.debian.org

Subject: Re: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Date: Mon, 02 Sep 2019 12:03:23 +0800

[Message part 1 (text/plain, inline)]

On Sun, 7 Jul 2019 20:39:10 +0200 Christoph Berg wrote:

> I and #debian-devel in general were pretty surprised that our "buster"
> systems couldn't properly "apt-get update" anymore today:

I recently had a similar issue with the backports repos:

E: Repository 'https://incoming.debian.org/debian-buildd buildd-stretch-backports-sloppy InRelease' changed its default priority for apt_preferences(5) from 500 to 100.
E: Repository 'https://cdn-aws.deb.debian.org/debian stretch-backports-sloppy InRelease' changed its default priority for apt_preferences(5) from 500 to 100.

This was because those suites got a couple of settings added to their
Release files in the apt repository:

NotAutomatic: yes
ButAutomaticUpgrades: yes

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

[signature.asc (application/pgp-signature, inline)]

Message #86 received at 931566@bugs.debian.org (full text, mbox, reply):

From: Christoph Berg <myon@debian.org>

To: Paul Wise <pabs@debian.org>, 931566@bugs.debian.org

Subject: Re: Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Date: Mon, 2 Sep 2019 09:26:01 +0200

Re: Paul Wise 2019-09-02 <68737fa6fe19d3c2a71beadefbe69c661dd4d271.camel@debian.org>
> I recently had a similar issue with the backports repos:
> 
> E: Repository 'https://incoming.debian.org/debian-buildd buildd-stretch-backports-sloppy InRelease' changed its default priority for apt_preferences(5) from 500 to 100.
> E: Repository 'https://cdn-aws.deb.debian.org/debian stretch-backports-sloppy InRelease' changed its default priority for apt_preferences(5) from 500 to 100.
> 
> This was because those suites got a couple of settings added to their
> Release files in the apt repository:
> 
> NotAutomatic: yes
> ButAutomaticUpgrades: yes

I think this is an example of something that this mechanism *should*
catch, especially if the change would have gone the other way round.

But the pain caused by testing->stable and oldstable->stable is still
much more prevalent than the legitimate cases. If we don't find a
solution that can tell these apart in a robust way, we should disable
the whole thing in stable.

Christoph

Message #91 received at 931566@bugs.debian.org (full text, mbox, reply):

From: Al Mamun <mamun18barist@gmail.com>

To: 931566@bugs.debian.org

Date: Tue, 23 Jun 2020 01:58:57 +0600

[Message part 1 (text/plain, inline)]

[Message part 2 (text/html, inline)]

Message #94 received at 931566-submitter@bugs.debian.org (full text, mbox, reply):

From: Julian Andres Klode <noreply@salsa.debian.org>

To: 931566-submitter@bugs.debian.org

Subject: Bug#931566 marked as pending in apt

Date: Mon, 10 Aug 2020 14:24:21 +0000

Control: tag -1 pending

Hello,

Bug #931566 in apt reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:

https://salsa.debian.org/apt-team/apt/-/commit/64b45e294f0c6931a9b57ae6cc99ecded8f6a2d3

------------------------------------------------------------------------
Default Acquire::AllowReleaseInfoChange::Suite to "true"

Closes: #931566
------------------------------------------------------------------------

(this message was generated automatically)
-- 
Greetings

https://bugs.debian.org/931566

Message #101 received at 931566-close@bugs.debian.org (full text, mbox, reply):

From: Debian FTP Masters <ftpmaster@ftp-master.debian.org>

To: 931566-close@bugs.debian.org

Subject: Bug#931566: fixed in apt 2.1.10

Date: Tue, 11 Aug 2020 12:48:30 +0000

Source: apt
Source-Version: 2.1.10
Done: Julian Andres Klode <jak@debian.org>

We believe that the bug you reported is fixed in the latest version of
apt, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 931566@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Julian Andres Klode <jak@debian.org> (supplier of updated apt package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.8
Date: Tue, 11 Aug 2020 14:28:07 +0200
Source: apt
Architecture: source
Version: 2.1.10
Distribution: unstable
Urgency: medium
Maintainer: APT Development Team <deity@lists.debian.org>
Changed-By: Julian Andres Klode <jak@debian.org>
Closes: 931566 968220
Changes:
 apt (2.1.10) unstable; urgency=medium
 .
   * Default Acquire::AllowReleaseInfoChange::Suite to "true" (Closes: #931566)
   * acquire: Do not hide _error messages in Fail()
   * Further improvements to HTTP method (Closes: #968220, verified against
     that server and the Canonical infra where it blocked buildds)
     - Do not use non-blocking local I/O - they don't do anything anyway,
       and we can't really use non-blocking I/O here because we need to be able
       to flush it.
     - Restore successful exits from Die() and rewrite Die() in a more
       comprehensible way, after careful code path analysis
     - http: Fully flush local file both before/after server read, avoiding
       both partial flush before sending requests to the server, as well as
       preventing leftover data before receiving from the server, which cause
       data left in the buffer.
Checksums-Sha1:
 6d7df5bb0e8cee789834e0fae8c436bc40f675f7 2760 apt_2.1.10.dsc
 164d6425ef9202be3220a943bbf6b1beb7ed22d5 2179772 apt_2.1.10.tar.xz
 456adef0976e1c4dc1c9a4161fe698d7b602ee10 7206 apt_2.1.10_source.buildinfo
Checksums-Sha256:
 2368cefda44f61bb73970781be04ae7947290606d929f1682fb3cfa6ddb6ec0a 2760 apt_2.1.10.dsc
 aa678d0fcd614a7707e77f3219097401141f5426cd1095c4aa50043920a2c04b 2179772 apt_2.1.10.tar.xz
 2854cf57f5bafb91d46a94bf4bba8683cf37f002a202485558f8e2859d5c9847 7206 apt_2.1.10_source.buildinfo
Files:
 2cdf3eee708f002afc59afaf0c54b6d4 2760 admin important apt_2.1.10.dsc
 66fe46f1df713881782b2f9eb60436d9 2179772 admin important apt_2.1.10.tar.xz
 7261ff07fda643324f2b6a09f73bf320 7206 admin important apt_2.1.10_source.buildinfo

-----BEGIN PGP SIGNATURE-----

iQJDBAEBCgAtFiEET7WIqEwt3nmnTHeHb6RY3R2wP3EFAl8ykGIPHGpha0BkZWJp
YW4ub3JnAAoJEG+kWN0dsD9xsLgP/i7XxXjoGjxn2P2p08f24skg6qAFNTx5uI8V
S3EREZi74pStWhQanolc1gtUxg+ZC1L4d/mfBmmVTbgmX0rSeP/7/uN+grOU/syW
x9xNHIXKanQoqU7G+qpWmp9zwAeO2JJakYGLUxS7Q9O7jtAxCPiKN1VPGuayclVk
jCMvgxSFk6EEAp20RwsGIdftqPV7xrk/bjgyxDBsEPggCn/sxouW8g9uQ2s34CL9
pa1Kyt1zGwRvXhrUwM1qYoLTHIi99IrLiJYfDwWN9COj2cygEGvGN1Ne1FtM4xSq
6yDQJBr43IY4wbQiUMN57F5MqrmWKicR7l7caKL4n9YzBR1ZER1p85V04j8ddgqf
0VzdS5EpLYHVPyKn5dte5bHDHxUpcjQ/4I/WeUSDppRKxDSxFkDYq33Qtw4WnctK
kQDSlxWtFZxaZC43aJClARzxHvz4Bygxlda7tQ1S6d5zO19vLZGoZ28GriH0HPMm
9JVj+EZ6ZRcaimHQS9ZURuvPykx0e5k3ra98jD9EjBpjbfbfURJNkuD2bxkLljmq
T3fDDAlXLLX1rHePR7ST1QxLnsRZxqJ549PxU4dpMPKJ92f2TeQjBUungxvxNbPV
9clrPrAYTqiEYRCN0XRQ2zs0XDIYiinGlfsB/MGNZ2u1/5HT1AXTWgQmZq6s8gyl
0sXCuVnH
=GrVk
-----END PGP SIGNATURE-----

Message #106 received at 931566@bugs.debian.org (full text, mbox, reply):

From: Paul Gevers <elbrus@debian.org>

To: 931566@bugs.debian.org, debian-release <debian-release@lists.debian.org>

Subject: Re: Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Date: Sun, 18 Apr 2021 22:20:19 +0200

[Message part 1 (text/plain, inline)]

Hi Julian, David, SRM,

On Mon, 8 Jul 2019 09:11:48 +0200 Christoph Berg <myon@debian.org> wrote:
> Fwiw, I think this needs fixing in buster, not just in unstable.

We're getting close to the release of bullseye and it has been brought
to my attention that this bug is still unfixed in buster. Once we
release bullseye, this bug is going to run havoc for our buster users.

Can we somehow come up with a plan on how to handle this? Can we have a
fix in the next point release? Are there faster options than just
waiting some time after the next point release before we can release
bullseye, e.g. could the SRM allow an update to stable for the change of
an apt default to have the change earlier than the next point release?

Paul

[OpenPGP_signature (application/pgp-signature, attachment)]

Message #111 received at 931566@bugs.debian.org (full text, mbox, reply):

From: Julian Andres Klode <jak@debian.org>

To: Paul Gevers <elbrus@debian.org>, 931566@bugs.debian.org

Cc: debian-release <debian-release@lists.debian.org>

Subject: Re: Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Date: Sun, 18 Apr 2021 22:31:25 +0200

[Message part 1 (text/plain, inline)]

On Sun, Apr 18, 2021 at 10:20:19PM +0200, Paul Gevers wrote:
> Hi Julian, David, SRM,
> 
> On Mon, 8 Jul 2019 09:11:48 +0200 Christoph Berg <myon@debian.org> wrote:
> > Fwiw, I think this needs fixing in buster, not just in unstable.
> 
> We're getting close to the release of bullseye and it has been brought
> to my attention that this bug is still unfixed in buster. Once we
> release bullseye, this bug is going to run havoc for our buster users.

That's not accurate. This is _only_ a problem for users of testing,
where the codename changes from time to time.

For stable users, this is not a problem at all, more the opposite. Those
poor souls who have stable in their sources.list won't suddenly get
upgraded to bullseye.


> 
> Can we somehow come up with a plan on how to handle this? Can we have a
> fix in the next point release? Are there faster options than just
> waiting some time after the next point release before we can release
> bullseye, e.g. could the SRM allow an update to stable for the change of
> an apt default to have the change earlier than the next point release?

I have no intention of issuing a stable update.

-- 
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer                              i speak de, en

[signature.asc (application/pgp-signature, inline)]

Message #114 received at 931566@bugs.debian.org (full text, mbox, reply):

From: Christoph Berg <myon@debian.org>

To: Julian Andres Klode <jak@debian.org>, 931566@bugs.debian.org

Cc: Paul Gevers <elbrus@debian.org>, debian-release <debian-release@lists.debian.org>

Subject: Re: Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Date: Mon, 19 Apr 2021 17:40:09 +0200

Re: Julian Andres Klode
> > We're getting close to the release of bullseye and it has been brought
> > to my attention that this bug is still unfixed in buster. Once we
> > release bullseye, this bug is going to run havoc for our buster users.
> 
> That's not accurate. This is _only_ a problem for users of testing,
> where the codename changes from time to time.

This *is* a problem for users of "buster" where the suite will be
changing from "stable" to "oldstable". (Yes, we do release buster
twice, once as stable and then as oldstable.)

Unless the fix that has closed #931566 is also applied to the apt
version in buster, things will explode horribly.

Changed-By: Julian Andres Klode <jak@debian.org>
Changes:
 apt (2.1.10) unstable; urgency=medium
 .
   * Default Acquire::AllowReleaseInfoChange::Suite to "true" (Closes: #931566)

Notably, that needs to happen well before the bullseye release or else
systems will not be able to "apt-get update" non-interactively to
actually see the updated package.

> For stable users, this is not a problem at all, more the opposite. Those
> poor souls who have stable in their sources.list won't suddenly get
> upgraded to bullseye.

Yes, this part of the change is the good one. Pinning suite for
"buster" users is not.

> > Can we somehow come up with a plan on how to handle this? Can we have a
> > fix in the next point release? Are there faster options than just
> > waiting some time after the next point release before we can release
> > bullseye, e.g. could the SRM allow an update to stable for the change of
> > an apt default to have the change earlier than the next point release?
> 
> I have no intention of issuing a stable update.

On 2020-08-10 you said:

17:04 <Myon> juliank: is #931566 going to be fixed in buster as well?
17:04 -zwiebelbot- Debian#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true") - https://bugs.debian.org/931566
17:04 <juliank> Myon: yes
17:04 <Myon> cool
17:04 <Myon> thanks

Christoph

Message #119 received at 931566@bugs.debian.org (full text, mbox, reply):

From: Julian Andres Klode <jak@debian.org>

To: Christoph Berg <myon@debian.org>, 931566@bugs.debian.org, Paul Gevers <elbrus@debian.org>, debian-release <debian-release@lists.debian.org>

Subject: Re: Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Date: Mon, 19 Apr 2021 18:01:18 +0200

[Message part 1 (text/plain, inline)]

On Mon, Apr 19, 2021 at 05:40:09PM +0200, Christoph Berg wrote:
> Re: Julian Andres Klode
> > > We're getting close to the release of bullseye and it has been brought
> > > to my attention that this bug is still unfixed in buster. Once we
> > > release bullseye, this bug is going to run havoc for our buster users.
> > 
> > That's not accurate. This is _only_ a problem for users of testing,
> > where the codename changes from time to time.
> 
> This *is* a problem for users of "buster" where the suite will be
> changing from "stable" to "oldstable". (Yes, we do release buster
> twice, once as stable and then as oldstable.)
> 
> Unless the fix that has closed #931566 is also applied to the apt
> version in buster, things will explode horribly.

Oh I see. That's tricky.

> 
> Changed-By: Julian Andres Klode <jak@debian.org>
> Changes:
>  apt (2.1.10) unstable; urgency=medium
>  .
>    * Default Acquire::AllowReleaseInfoChange::Suite to "true" (Closes: #931566)
> 
> Notably, that needs to happen well before the bullseye release or else
> systems will not be able to "apt-get update" non-interactively to
> actually see the updated package.
> 
> > For stable users, this is not a problem at all, more the opposite. Those
> > poor souls who have stable in their sources.list won't suddenly get
> > upgraded to bullseye.
> 
> Yes, this part of the change is the good one. Pinning suite for
> "buster" users is not.
> 
> > > Can we somehow come up with a plan on how to handle this? Can we have a
> > > fix in the next point release? Are there faster options than just
> > > waiting some time after the next point release before we can release
> > > bullseye, e.g. could the SRM allow an update to stable for the change of
> > > an apt default to have the change earlier than the next point release?
> > 
> > I have no intention of issuing a stable update.
> 
> On 2020-08-10 you said:
> 
> 17:04 <Myon> juliank: is #931566 going to be fixed in buster as well?
> 17:04 -zwiebelbot- Debian#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true") - https://bugs.debian.org/931566
> 17:04 <juliank> Myon: yes
> 17:04 <Myon> cool
> 17:04 <Myon> thanks

I see. Nobody pinged me since then, but it is indeed fixed in the
1.8.5 stable update that at least one release team member was
not exited about.

https://salsa.debian.org/apt-team/apt/-/compare/1.8.2.2...1.8.5

So it's up to the release team if they want 1.8.5 or whether we'll have
to cherry-pick a subset of it into a 1.8.2.3. I think my opinion on that
is clear - I don't want to maintain a 1.8.2.z branch, it's more work compared
to just following the normal stable apt updates, and there's a lot less
testing going on.

-- 
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer                              i speak de, en

[signature.asc (application/pgp-signature, inline)]

Message #124 received at 931566@bugs.debian.org (full text, mbox, reply):

From: Julien Cristau <jcristau@debian.org>

To: Julian Andres Klode <jak@debian.org>, Christoph Berg <myon@debian.org>, 931566@bugs.debian.org, Paul Gevers <elbrus@debian.org>, debian-release <debian-release@lists.debian.org>

Subject: Re: Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Date: Mon, 19 Apr 2021 18:08:23 +0200

On Mon, Apr 19, 2021 at 06:01:18PM +0200, Julian Andres Klode wrote:
> I see. Nobody pinged me since then, but it is indeed fixed in the
> 1.8.5 stable update that at least one release team member was
> not exited about.
> 
> https://salsa.debian.org/apt-team/apt/-/compare/1.8.2.2...1.8.5
> 
> So it's up to the release team if they want 1.8.5 or whether we'll have
> to cherry-pick a subset of it into a 1.8.2.3. I think my opinion on that
> is clear - I don't want to maintain a 1.8.2.z branch, it's more work compared
> to just following the normal stable apt updates, and there's a lot less
> testing going on.
> 
Please upload just that fix to buster; I don't care too much about the
version number you pick.

Thanks,
Julien

Message #129 received at 931566@bugs.debian.org (full text, mbox, reply):

From: Julian Andres Klode <jak@debian.org>

To: Julien Cristau <jcristau@debian.org>

Cc: Christoph Berg <myon@debian.org>, 931566@bugs.debian.org, Paul Gevers <elbrus@debian.org>, debian-release <debian-release@lists.debian.org>

Subject: Re: Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Date: Mon, 19 Apr 2021 18:28:05 +0200

On Mon, Apr 19, 2021 at 06:08:23PM +0200, Julien Cristau wrote:
> On Mon, Apr 19, 2021 at 06:01:18PM +0200, Julian Andres Klode wrote:
> > I see. Nobody pinged me since then, but it is indeed fixed in the
> > 1.8.5 stable update that at least one release team member was
> > not exited about.
> > 
> > https://salsa.debian.org/apt-team/apt/-/compare/1.8.2.2...1.8.5
> > 
> > So it's up to the release team if they want 1.8.5 or whether we'll have
> > to cherry-pick a subset of it into a 1.8.2.3. I think my opinion on that
> > is clear - I don't want to maintain a 1.8.2.z branch, it's more work compared
> > to just following the normal stable apt updates, and there's a lot less
> > testing going on.
> > 
> Please upload just that fix to buster; I don't care too much about the
> version number you pick.

Is there a buster point release before bullseye release, or should that
be in buster-updates?

Given that buster is going to security support only soon anyway, I don't
mind where I apply security updates to that much :D


But I really do want to cherry-pick at least two other code fixes, and one test
suite fix:

* https://salsa.debian.org/apt-team/apt/-/commit/cfee71c6f2d1478ce4d4ed74ef690ae1350ea403
  https://salsa.debian.org/apt-team/apt/-/commit/75f452c7309d66548c86a6526cbd65fc51a70039

  (really just one change, but it was split over two releases, first
  turned error to warning, next one ignores it completely; because it
  was in 2 releases in main so I kept it separate :D)

  too, they'll make immediate configuration errors non-fatal. Currently
  they are fatal in the sense that they are ignored and the upgrade runs
  and then it just exits with 1 at the end. So it does not change the
  outcome, it just makes the error reporting less silly. 

  It is very likely that some upgrades on multi-arch systems fail erronously
  without them. To be fair, the multi-arch aspect is also fixed by
  https://salsa.debian.org/apt-team/apt/-/commit/7f65fa3843abc476cbba65c808abc5dd77835815
  but that changes ordering results, and is not strictly necessary.

* And I want
  https://salsa.debian.org/apt-team/apt/-/commit/cfc6870e9b8ad119219ce5dc1871531006bb2d71

  to avoid people getting packages removed that stuff still depends on
  because their prerm script failed. This might happen during an upgrade
  to bullseye, and it's very very likely APT will not be able to recover from it 
  - I've never successfully recovered a distribution upgrade that had a failure in the
  middle (and fwiw, all of them had, but they were my faults, really).

* Also the testsuite-only change in
  https://salsa.debian.org/apt-team/apt/-/commit/730c5c861c32c9385dc862af8673984b12902343
  which makes things work reliably on debci armhf (no regression
  potential, wheee)

-- 
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer                              i speak de, en

Message #134 received at 931566@bugs.debian.org (full text, mbox, reply):

From: Julien Cristau <jcristau@debian.org>

To: Julian Andres Klode <jak@debian.org>, Christoph Berg <myon@debian.org>, 931566@bugs.debian.org, Paul Gevers <elbrus@debian.org>, debian-release <debian-release@lists.debian.org>

Subject: Re: Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Date: Mon, 19 Apr 2021 18:31:48 +0200

On Mon, Apr 19, 2021 at 06:28:05PM +0200, Julian Andres Klode wrote:
> On Mon, Apr 19, 2021 at 06:08:23PM +0200, Julien Cristau wrote:
> > On Mon, Apr 19, 2021 at 06:01:18PM +0200, Julian Andres Klode wrote:
> > > I see. Nobody pinged me since then, but it is indeed fixed in the
> > > 1.8.5 stable update that at least one release team member was
> > > not exited about.
> > > 
> > > https://salsa.debian.org/apt-team/apt/-/compare/1.8.2.2...1.8.5
> > > 
> > > So it's up to the release team if they want 1.8.5 or whether we'll have
> > > to cherry-pick a subset of it into a 1.8.2.3. I think my opinion on that
> > > is clear - I don't want to maintain a 1.8.2.z branch, it's more work compared
> > > to just following the normal stable apt updates, and there's a lot less
> > > testing going on.
> > > 
> > Please upload just that fix to buster; I don't care too much about the
> > version number you pick.
> 
> Is there a buster point release before bullseye release, or should that
> be in buster-updates?
> 
I don't know.  It needs to make its way to spu soon either way.

> Given that buster is going to security support only soon anyway, I don't
> mind where I apply security updates to that much :D
> 
> 
> But I really do want to cherry-pick at least two other code fixes, and one test
> suite fix:
> 
Can we defer these until after the AllowReleaseInfoChange change is
out, please?

Thanks,
Julien

> * https://salsa.debian.org/apt-team/apt/-/commit/cfee71c6f2d1478ce4d4ed74ef690ae1350ea403
>   https://salsa.debian.org/apt-team/apt/-/commit/75f452c7309d66548c86a6526cbd65fc51a70039
> 
>   (really just one change, but it was split over two releases, first
>   turned error to warning, next one ignores it completely; because it
>   was in 2 releases in main so I kept it separate :D)
> 
>   too, they'll make immediate configuration errors non-fatal. Currently
>   they are fatal in the sense that they are ignored and the upgrade runs
>   and then it just exits with 1 at the end. So it does not change the
>   outcome, it just makes the error reporting less silly. 
> 
>   It is very likely that some upgrades on multi-arch systems fail erronously
>   without them. To be fair, the multi-arch aspect is also fixed by
>   https://salsa.debian.org/apt-team/apt/-/commit/7f65fa3843abc476cbba65c808abc5dd77835815
>   but that changes ordering results, and is not strictly necessary.
> 
> * And I want
>   https://salsa.debian.org/apt-team/apt/-/commit/cfc6870e9b8ad119219ce5dc1871531006bb2d71
> 
>   to avoid people getting packages removed that stuff still depends on
>   because their prerm script failed. This might happen during an upgrade
>   to bullseye, and it's very very likely APT will not be able to recover from it 
>   - I've never successfully recovered a distribution upgrade that had a failure in the
>   middle (and fwiw, all of them had, but they were my faults, really).
> 
> * Also the testsuite-only change in
>   https://salsa.debian.org/apt-team/apt/-/commit/730c5c861c32c9385dc862af8673984b12902343
>   which makes things work reliably on debci armhf (no regression
>   potential, wheee)
> 
> -- 
> debian developer - deb.li/jak | jak-linux.org - free software dev
> ubuntu core developer                              i speak de, en
> 

Message #139 received at 931566@bugs.debian.org (full text, mbox, reply):

From: Julian Andres Klode <jak@debian.org>

To: Julien Cristau <jcristau@debian.org>

Cc: Christoph Berg <myon@debian.org>, 931566@bugs.debian.org, Paul Gevers <elbrus@debian.org>, debian-release <debian-release@lists.debian.org>

Subject: Re: Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")

Date: Mon, 19 Apr 2021 19:44:53 +0200

On Mon, Apr 19, 2021 at 06:31:48PM +0200, Julien Cristau wrote:
> Can we defer these until after the AllowReleaseInfoChange change is
> out, please?

Done. I uploaded 1.8.2.3 with that one change as requested, and created a
pu bug for it.

-- 
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer                              i speak de, en

Message #144 received at 931566-close@bugs.debian.org (full text, mbox, reply):

From: Debian FTP Masters <ftpmaster@ftp-master.debian.org>

To: 931566-close@bugs.debian.org

Subject: Bug#931566: fixed in apt 1.8.2.3

Date: Mon, 19 Apr 2021 19:17:07 +0000

Source: apt
Source-Version: 1.8.2.3
Done: Julian Andres Klode <jak@debian.org>

We believe that the bug you reported is fixed in the latest version of
apt, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 931566@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Julian Andres Klode <jak@debian.org> (supplier of updated apt package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.8
Date: Mon, 19 Apr 2021 18:41:13 +0200
Source: apt
Architecture: source
Version: 1.8.2.3
Distribution: buster
Urgency: medium
Maintainer: APT Development Team <deity@lists.debian.org>
Changed-By: Julian Andres Klode <jak@debian.org>
Closes: 931566
Changes:
 apt (1.8.2.3) buster; urgency=medium
 .
   * Default Acquire::AllowReleaseInfoChange::Suite to "true" (Closes: #931566)
Checksums-Sha1:
 c2bbcb247ef9e61c15ed414554fa5b90b0121afa 2774 apt_1.8.2.3.dsc
 ec43ec0c607aa9b416cd0ecdf7386a00b6e5c97b 2191868 apt_1.8.2.3.tar.xz
 498b2365058da17af932267d5bb7a5d35c060343 7411 apt_1.8.2.3_source.buildinfo
Checksums-Sha256:
 03ed672edefe4badbb2c7b32332293403bb03feb2ea0777c0846939a2fcb8bba 2774 apt_1.8.2.3.dsc
 c21c9b18c4a26bc183432cb49b919af073862954f1ae8a204096b0a68c946d3b 2191868 apt_1.8.2.3.tar.xz
 671086ebc7d91b76d0d6ca56743f7102113267c588f459e9c4002b87bbf3a0eb 7411 apt_1.8.2.3_source.buildinfo
Files:
 fd7470b16c69e0a8caf9d7db4628cf70 2774 admin important apt_1.8.2.3.dsc
 ade576aaaf6a37fae44abbb074fc01bb 2191868 admin important apt_1.8.2.3.tar.xz
 f4b137976b9cd5c46905c1db216ae27e 7411 admin important apt_1.8.2.3_source.buildinfo

-----BEGIN PGP SIGNATURE-----

iQJDBAEBCgAtFiEET7WIqEwt3nmnTHeHb6RY3R2wP3EFAmB9ujAPHGpha0BkZWJp
YW4ub3JnAAoJEG+kWN0dsD9xt2cQAI3LpRRaDtZu/CDq7hi5nvCcYs2dinvWoUSD
xrQLQ8GEQxKVu2N0WoON7KW290tgbMu3aS1NtSivz5SAHOKCSdxm2YBGytde9Xso
sPjIGpI1S/uEMuu0Zs8ooFSdTbv9+TVW38dNIYsNpAcO4McGBzVsZzQgAN2Kc4kZ
7oTAwelGvyKqzcscJgcFuA0D3UwslOrds/f9z3263p1aCSzyw/kTop4euGaqOIwr
VVgXjasfIKfQTND3l/NqrG8MgWhicCz/CkL2G3skFhmsKUwp+EysRSAre4Zsuk95
vtQB/zztn2sDphZ5gFgBsCjzu2u6hG02yoRttNoPHCI6DQhP5CQBsdrYbCyI+eJw
r2DavU3VXKvlnSR9Ne7RblE3fBRFk8Hnx6T8sfDXoLyjde78jdkVczbcdUS9rOoQ
fNAeBP3kcqvIWBOwVxS4Q30k2flnKX32mVFlpXO/LE/Dn6EIuxT2eWhrFi7XQdl4
Oa/aBn5B68SoMxWdStodu+pAzsJrOrZoxBNAFMog+5wUmBbpkZDwoY1vrs6MI3b9
peiNzuZuc1+oBbSpDadayG/rqsa/3rkBPRZhlAUxL8JvwsKzMJ2YXIsrxxOtVpDi
CuD62xf6382YCtbqXdN04Gmk+jOtP7Am01g2Xu35AZub8bg/lWSNOQFB1NgMqPzC
bDnyOoTg
=RAty
-----END PGP SIGNATURE-----

Message #147 received at 931566@bugs.debian.org (full text, mbox, reply):

From: Christoph Berg <myon@debian.org>

To: 931566@bugs.debian.org

Subject: Re: Bug#931566: marked as pending in apt

Date: Fri, 23 Apr 2021 13:07:26 +0200

Re: Julian Andres Klode
> Default Acquire::AllowReleaseInfoChange::Suite to "true"
> 
> Closes: #931566

Thanks!

Christoph

Send a report that this bug log contains spam.