Debian Bug report logs - #978636
move to merged-usr-only?

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Ansgar <ansgar@debian.org>

To: submit@bugs.debian.org

Subject: move to merged-usr-only?

Date: Tue, 29 Dec 2020 14:16:58 +0100

Package: tech-ctte
X-Debbugs-Cc: debian-devel@lists.debian.org

Hi,

as suggested in [1], I would like to see Debian to move to support
only the merged-usr filesystem layout.  This would simplfy things for
the future and also address the problem with installing files under
aliased trees that dpkg has to do for both variants to be supported.

merged-usr has been the default for new installations since the last
release and (as far as I am aware) no world-breaking bugs have
happened since; some environments such as buildd chroots still do not
use merged-usr.

I would like to ask the technical committee to decide whether we want
to move to merged-usr-only.  It seems to be a case of overlapping
jurisdiction (6.1.2 in the constitution).

I'm not asking the committee to decide on a concrete technical
implementation for this.  Obviously we would need to also implement a
migration path for legacy installations for a move to merged-usr-only
to be implemented.  This also isn't relevant for Debian 11 (bullseye),
but I would like to have enough time in the Debian 12 (bookworm)
cycle.

Ansgar

[1]: https://lists.debian.org/debian-devel/2020/11/#00232
     continued in December: https://lists.debian.org/debian-devel/2020/12/#00386

Message #10 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Marco d'Itri <md@Linux.IT>

To: debian-devel@lists.debian.org, 978636@bugs.debian.org

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Tue, 29 Dec 2020 17:35:46 +0100

[Message part 1 (text/plain, inline)]

On Dec 29, Ansgar <ansgar@debian.org> wrote:

> as suggested in [1], I would like to see Debian to move to support
> only the merged-usr filesystem layout.  This would simplfy things for
> the future and also address the problem with installing files under
> aliased trees that dpkg has to do for both variants to be supported.
As the architect of the Debian merged-usr transition, I agree: removing 
support for unmerged systems would simplify many things.
Thank you Ansgar for bringing this to the CTTE.

> I'm not asking the committee to decide on a concrete technical
> implementation for this.  Obviously we would need to also implement a
> migration path for legacy installations for a move to merged-usr-only
> to be implemented.  This also isn't relevant for Debian 11 (bullseye),
> but I would like to have enough time in the Debian 12 (bookworm)
> cycle.
Agreed.
FWIW: we have had for a long time a reliable migration method, the
usrmerge package.
Except that on a live system it requires a reboot mid-conversion (due to
bind mounts created by systemd): since this is inconvenient and mildly
scary I did not propose wider adoption for buster.
The best workaround would probably be to run it from the initramfs, but
I have not been able to actually make this[1] work: I expect that 
somebody who knows initramfs-tools better than I do can fix it quickly.

[1] https://salsa.debian.org/md/usrmerge/-/tree/initramfs

-- 
ciao,
Marco

[signature.asc (application/pgp-signature, inline)]

Message #15 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Matthias Klumpp <matthias@tenstral.net>

To: debian-devel <debian-devel@lists.debian.org>, 978636@bugs.debian.org

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Tue, 29 Dec 2020 17:51:34 +0100

Am Di., 29. Dez. 2020 um 17:39 Uhr schrieb Marco d'Itri <md@linux.it>:
>
> On Dec 29, Ansgar <ansgar@debian.org> wrote:
>
> > as suggested in [1], I would like to see Debian to move to support
> > only the merged-usr filesystem layout.  This would simplfy things for
> > the future and also address the problem with installing files under
> > aliased trees that dpkg has to do for both variants to be supported.
> As the architect of the Debian merged-usr transition, I agree: removing
> support for unmerged systems would simplify many things.
> Thank you Ansgar for bringing this to the CTTE.

Indeed!

> > I'm not asking the committee to decide on a concrete technical
> > implementation for this.  Obviously we would need to also implement a
> > migration path for legacy installations for a move to merged-usr-only
> > to be implemented.  This also isn't relevant for Debian 11 (bullseye),
> > but I would like to have enough time in the Debian 12 (bookworm)
> > cycle.
> Agreed.
> FWIW: we have had for a long time a reliable migration method, the
> usrmerge package.
> Except that on a live system it requires a reboot mid-conversion (due to
> bind mounts created by systemd): since this is inconvenient and mildly
> scary I did not propose wider adoption for buster.
> The best workaround would probably be to run it from the initramfs, but
> I have not been able to actually make this[1] work: I expect that
> somebody who knows initramfs-tools better than I do can fix it quickly.

For package upgrades, we can already perform so-called "offline
upgrades", where the system reboots into a smaller systemd target,
applies all updates and then reboots again into the updated system.
This is implemented in PackageKit as an option and used by GNOME.
Fwupd uses similar concepts for certain firmware updates as well.
Maybe hooking into these facilities is also an option for the usrmerge
upgrade? (unless of course systemd is still interfering even in a
minimal target, that would be a dealbreaker).
More info about offline-upgrades can be found at [1], could be
interesting to look into.

This discussion is probably OT through for the issue at hand (*if* we
make that switch, not the *how* we make it in particular).

Cheers,
    Matthias

[1]: https://www.freedesktop.org/software/systemd/man/systemd.offline-updates.html

-- 
I welcome VSRE emails. See http://vsre.info/

Message #20 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Sean Whitton <spwhitton@spwhitton.name>

To: 978636@bugs.debian.org

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Mon, 25 Jan 2021 11:45:55 -0700

[Message part 1 (text/plain, inline)]

Dear Technical Committee members,

I call for votes on the following ballot to resolve #978636.  The voting
period starts immediately and lasts for up to one week, or until the
outcome is no longer in doubt (§6.3.1).

===BEGIN
The Technical Committee resolves that Debian 'bookworm' should support
only the merged-usr root filesystem layout, dropping support for the
non-merged-usr layout.

Until after the release of 'bullseye', any implementation of this
resolution must be done in the 'experimental' distribution, or otherwise
kept out of the critical paths for the release of 'bullseye'.

We do not recommend any particular implementation of the migration.

Y: Yes, support only merged-usr in the 'bookworm' release.
N: No, continue to support both layouts in 'bookworm'.
F: Further Discussion
===END

-- 
Sean Whitton

[signature.asc (application/pgp-signature, inline)]

Message #25 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Sean Whitton <spwhitton@spwhitton.name>

To: 978636@bugs.debian.org

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Mon, 25 Jan 2021 11:46:35 -0700

[Message part 1 (text/plain, inline)]

Hello,

On Mon 25 Jan 2021 at 11:45AM -07, Sean Whitton wrote:

> I call for votes on the following ballot to resolve #978636.  The voting
> period starts immediately and lasts for up to one week, or until the
> outcome is no longer in doubt (§6.3.1).
>
> ===BEGIN
> The Technical Committee resolves that Debian 'bookworm' should support
> only the merged-usr root filesystem layout, dropping support for the
> non-merged-usr layout.
>
> Until after the release of 'bullseye', any implementation of this
> resolution must be done in the 'experimental' distribution, or otherwise
> kept out of the critical paths for the release of 'bullseye'.
>
> We do not recommend any particular implementation of the migration.
>
> Y: Yes, support only merged-usr in the 'bookworm' release.
> N: No, continue to support both layouts in 'bookworm'.
> F: Further Discussion
> ===END

I vote:

  Y > F > N

-- 
Sean Whitton

[signature.asc (application/pgp-signature, inline)]

Message #30 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: Sean Whitton <spwhitton@spwhitton.name>, 978636@bugs.debian.org

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Mon, 25 Jan 2021 21:22:29 +0000

On Mon, 25 Jan 2021 at 11:46:35 -0700, Sean Whitton wrote:
> > The Technical Committee resolves that Debian 'bookworm' should support
> > only the merged-usr root filesystem layout, dropping support for the
> > non-merged-usr layout.

Should we be more specific than this in what we vote on, to avoid
later having to adjudicate between developers who say that a particular
implementation is or isn't merged-usr?

Some developers seem to be using "merged /usr" to refer to multiple
concrete layouts:

1. an arrangement where all regular files that have traditionally been
   in /bin, /sbin, /lib and /lib64 are physically located in /usr,
   with symlinks /bin -> usr/bin, /sbin -> usr/sbin, /lib -> usr/lib,
   /lib64 -> usr/lib64
   (this is what usrmerge, debootstrap --merged-usr and Fedora do; dpkg
   maintainer Guillem Jover refers to this as "merged /usr via aliased
   directories" which seems like a good unambiguous term)

2. an arrangement where all regular files that have traditionally been
   in /bin, /sbin, /lib and /lib64 are physically located in /usr,
   with /bin etc. becoming "symlink farms" containing symlinks like
   /bin/sh -> /usr/bin/sh, /lib/ld-linux.so.2 -> /usr/lib/ld-linux.so.2
   and so on
   2a. in one version of this, only those files that traditionally
       existed in the rootfs have symlinks, so that /bin/sh -> /usr/bin/sh
       exists but /bin/perl -> /usr/bin/perl does not
   2b. in another version of this, every file in /usr/bin, /usr/sbin,
       /usr/lib is represented in the symlink farm, so that
       /bin/perl -> /usr/bin/perl exists

I think we should choose wording to vote on such that if we vote yes,
it is clear which of these layouts are consistent with that resolution,
and which are not.

Guillem considers layout 1 to be broken and a mess. I don't agree,
but I'm not a dpkg maintainer. We should be aware that mandating this
implementation is likely to require some overruling.

I don't consider 2a to be merged /usr: it only does half the job of making
the directories in the rootfs equivalent to the directories in /usr.
For example, we would be able to run (for example) /usr/bin/sh scripts
developed on distros that don't distinguish, but not /bin/perl scripts.
However, it does do half the job, and I think Guillem might be considering
2a to be an implementation of the general concept of merged /usr?

I'm not sure whether *I* consider 2b to be merged /usr or not. It makes
the directories on the rootfs exactly equivalent to those in /usr, but
retains a lot of complexity. I would personally be willing to tolerate
that as a stepping-stone to reach layout 1, if the people who do the
work consider it to be less bad than doing the equivalent of usrmerge,
but I don't like it as a post-bullseye end goal in its own right.

If we're voting for layout 1 / status quo / FD, then that could be worded as:

    The Technical Committee resolves that Debian 'bookworm' should support
    only the "merged-usr via aliased directories" root filesystem layout
    in which /bin, /sbin and /lib are symbolic links to the corresponding
    directories in /usr, dropping support for the non-merged-usr layout.

Or if we're saying "either 1 or 2b":

    The Technical Committee resolves that Debian 'bookworm' should support
    only a merged-usr root filesystem layout in which /bin, /sbin and
    /lib are either symbolic links to the corresponding files in /usr,
    or directories containing such symbolic links, with every file
    /usr/bin/foo also available via the path /bin/foo and so on.

Or if we're saying "either 1 or 2a or 2b":

    The Technical Committee resolves that Debian 'bookworm' should support
    only a merged-usr root filesystem layout in which /bin, /sbin and
    /lib are either symbolic links to the corresponding files in /usr,
    or directories containing such symbolic links.

(I'm assuming that nobody will try to argue that /lib64, /libx32 and other
libQUAL directories should be treated differently from /lib, even if I
don't explicitly enumerate them.)

Sorry to derail this but I think we should be clear about what we're
resolving.

I think this is consistent with not recommending any particular
implementation of the migration path, which I'm taking to mean we don't
mind how existing unmerged installations reach the desired state - e.g.
if we recommend layout 1, you could get there by installing usrmerge.deb,
or via a special-case in dpkg, or any other mechanism that arrives at
layout 1.

    smcv

Message #35 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Keith Packard <keithp@keithp.com>

To: Simon McVittie <smcv@debian.org>, 978636@bugs.debian.org, Sean Whitton <spwhitton@spwhitton.name>, 978636@bugs.debian.org

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Mon, 25 Jan 2021 14:47:56 -0800

[Message part 1 (text/plain, inline)]

Simon McVittie <smcv@debian.org> writes:

> Should we be more specific than this in what we vote on, to avoid
> later having to adjudicate between developers who say that a particular
> implementation is or isn't merged-usr?

I think that and a transition plan are both key to this project. I
recently installed Debian from scratch on my HiFive unmatched board and
it got merged / and /usr. When I built packages on this device, they
created invalid packages as the shared library dependencies all listed
/lib instead of /usr/lib. That seems like a non-starter to me.

Any plan where a transitioning system cannot be used for ongoing Debian
development seems unworkable to me -- it leaves all active Debian
developers working on un-transitioned systems, which greatly reduces
test coverage from people in the best position to help fix things.

I do use a separate cowbuilder when creating packages to upload, and
that could be configured in un-transitioned mode, but I also regularly
debug packages on my native system as that offers much faster
development times.

> Guillem considers layout 1 to be broken and a mess. I don't agree,
> but I'm not a dpkg maintainer. We should be aware that mandating this
> implementation is likely to require some overruling.

From an architectural purity perspective, layout 1 'looks nicer'. As
that also matches what other distros are doing, it would make us more
consistent across the Linux ecosystem (I believe that's a good thing).

However, I believe only layout 2a would make it possible to build Debian
packages on transitioned systems. That seems necessary to me. We could,
in future, then transition from layout 2a to layout 1 once /lib (and
/bin?)  were no longer in the default search paths to cause invalid
packages to be created.

I don't understand the value of 2b; it's functionally similar to layout
1, but makes for additional work to create the shadow links, along with
consuming space for them. It also doesn't resolve the problem of
building packages.

> Sorry to derail this but I think we should be clear about what we're
> resolving.

I think you've added helpful clarification to the conversation, thanks!

-- 
-keith

[signature.asc (application/pgp-signature, inline)]

Message #40 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Adam Borowski <kilobyte@angband.pl>

To: 978636@bugs.debian.org

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Mon, 25 Jan 2021 23:21:41 +0100

On Mon, Jan 25, 2021 at 11:45:55AM -0700, Sean Whitton wrote:
> Dear Technical Committee members,
> 
> I call for votes on the following ballot to resolve #978636.  The voting
> period starts immediately and lasts for up to one week, or until the
> outcome is no longer in doubt (§6.3.1).
> 
> ===BEGIN
> The Technical Committee resolves that Debian 'bookworm' should support
> only the merged-usr root filesystem layout, dropping support for the
> non-merged-usr layout.

You did not even address, or even mention, that this idea has been vetoed by
dpkg maintainers.

And as that, it's a complete non-starter.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ .--[ Makefile ]
⣾⠁⢠⠒⠀⣿⡁ # beware of races
⢿⡄⠘⠷⠚⠋⠀ all: pillage burn
⠈⠳⣄⠀⠀⠀⠀ `----

Message #45 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Marco d'Itri <md@Linux.IT>

To: Simon McVittie <smcv@debian.org>, 978636@bugs.debian.org

Cc: Sean Whitton <spwhitton@spwhitton.name>

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Tue, 26 Jan 2021 00:28:45 +0100

[Message part 1 (text/plain, inline)]

On Jan 25, Simon McVittie <smcv@debian.org> wrote:

> 2. an arrangement where all regular files that have traditionally been
>    in /bin, /sbin, /lib and /lib64 are physically located in /usr,
>    with /bin etc. becoming "symlink farms" containing symlinks like
>    /bin/sh -> /usr/bin/sh, /lib/ld-linux.so.2 -> /usr/lib/ld-linux.so.2
>    and so on
>    2a. in one version of this, only those files that traditionally
>        existed in the rootfs have symlinks, so that /bin/sh -> /usr/bin/sh
>        exists but /bin/perl -> /usr/bin/perl does not
>    2b. in another version of this, every file in /usr/bin, /usr/sbin,
>        /usr/lib is represented in the symlink farm, so that
>        /bin/perl -> /usr/bin/perl exists
This would severely reduce the usefulness of sharing /usr between 
different systems (or doing snapshots, etc...), so it would require 
a lot of work for no significant benefits.
Also, as is has been discussed, if the /usr/doc/ transition was 
representative then this would probably take many years.
Also, this would have to deal (in maintainer scripts?) with the fact 
that most systems installed in the last few years have alraedy 
implemented option 1. More complexity for little benefits.
Also, no other distribution considered this.

> I think we should choose wording to vote on such that if we vote yes,
> it is clear which of these layouts are consistent with that resolution,
> and which are not.
This is a good idea.

-- 
ciao,
Marco

[signature.asc (application/pgp-signature, inline)]

Message #50 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Bastian Blank <waldi@debian.org>

To: Simon McVittie <smcv@debian.org>, 978636@bugs.debian.org

Cc: Sean Whitton <spwhitton@spwhitton.name>

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Tue, 26 Jan 2021 08:02:06 +0100

On Mon, Jan 25, 2021 at 09:22:29PM +0000, Simon McVittie wrote:
> Some developers seem to be using "merged /usr" to refer to multiple
> concrete layouts:
> 1. an arrangement where all regular files that have traditionally been
>    in /bin, /sbin, /lib and /lib64 are physically located in /usr,
>    with symlinks /bin -> usr/bin, /sbin -> usr/sbin, /lib -> usr/lib,
>    /lib64 -> usr/lib64
>    (this is what usrmerge, debootstrap --merged-usr and Fedora do; dpkg
>    maintainer Guillem Jover refers to this as "merged /usr via aliased
>    directories" which seems like a good unambiguous term)

Aren't there two sub-solutions?

1a. with packages shipping files both in /bin und /usr/bin.
1b. with packages shipping files only in /usr/bin.

Bastian

-- 
Klingon phaser attack from front!!!!!
100% Damage to life support!!!!

Message #55 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: Bastian Blank <waldi@debian.org>

Cc: 978636@bugs.debian.org, Sean Whitton <spwhitton@spwhitton.name>

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Tue, 26 Jan 2021 09:01:12 +0000

On Tue, 26 Jan 2021 at 08:02:06 +0100, Bastian Blank wrote:
> On Mon, Jan 25, 2021 at 09:22:29PM +0000, Simon McVittie wrote:
> > Some developers seem to be using "merged /usr" to refer to multiple
> > concrete layouts:
> > 1. an arrangement where all regular files that have traditionally been
> >    in /bin, /sbin, /lib and /lib64 are physically located in /usr,
> >    with symlinks /bin -> usr/bin, /sbin -> usr/sbin, /lib -> usr/lib,
> >    /lib64 -> usr/lib64
> >    (this is what usrmerge, debootstrap --merged-usr and Fedora do; dpkg
> >    maintainer Guillem Jover refers to this as "merged /usr via aliased
> >    directories" which seems like a good unambiguous term)
> 
> Aren't there two sub-solutions?
> 
> 1a. with packages shipping files both in /bin und /usr/bin.
> 1b. with packages shipping files only in /usr/bin.

What precisely do you mean by "shipping files"?

If the content of the .deb (as in dpkg --fsys-tarfile) included both
/bin/bash and /usr/bin/bash, then the package would fail to install on
systems where the aliasing symlinks already exist (unless there was a
special case for this situation in dpkg, which would require at least one
more release cycle before we could do it). Given that Debian 10 systems
with the aliasing symlinks already exist, I think we can probably
rule that out as not a practically available solution for Debian 12.

If the content of the .deb only included /usr/bin/bash, relying on an
externally-constructed /bin -> usr/bin symlink to ensure that the path
/bin/bash is resolvable, then that's the logical conclusion of layout 1,
but is not something we can do until *after* there has been a release
in which layout 1 is the only thing we support (with systems not already
in that layout undergoing a migration step whose precise implementation
is outside the TC's scope - usrmerge.deb is one implementation of that
migration step, but perhaps someone will design something better).
I believe Ansgar's goal in opening this bug was to make Debian 12 the
first release in which layout 1 is the only thing supported, allowing
the transition to this form of .deb content to start (and maybe finish)
during the Debian 13 cycle.

If the content of the .deb only included /usr/bin/bash, with a maintainer
script to create /bin/bash if the aliasing symlink /bin -> usr/bin does
not already exist, then that's compatible with either 1, 2a or 2b - but
in the case of Essential packages that have traditionally installed files
to /bin, /sbin or /lib, it's incompatible with the traditional layout
with a distinct /bin and /usr/bin (etc.), due to the extra requirements
placed on Essential packages.

I think at least the Essential subset needs to continue to have files in
/bin, /sbin, /lib in their dpkg --fsys-tarfile until *after* a transition
to (some form of) merged /usr has been completed, otherwise Essential
packages that have merely been unpacked and not configured will not
be providing their Essential functionality at paths like /bin/bash and
/lib/ld-linux.so.2 that other packages rely on.

But perhaps I misunderstood you and you mean something else?

    smcv

Message #60 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Bastian Blank <waldi@debian.org>

To: Simon McVittie <smcv@debian.org>, 978636@bugs.debian.org

Cc: Sean Whitton <spwhitton@spwhitton.name>

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Tue, 26 Jan 2021 10:30:34 +0100

On Tue, Jan 26, 2021 at 09:01:12AM +0000, Simon McVittie wrote:
> On Tue, 26 Jan 2021 at 08:02:06 +0100, Bastian Blank wrote:
> > On Mon, Jan 25, 2021 at 09:22:29PM +0000, Simon McVittie wrote:
> > > Some developers seem to be using "merged /usr" to refer to multiple
> > > concrete layouts:
> > > 1. an arrangement where all regular files that have traditionally been
> > >    in /bin, /sbin, /lib and /lib64 are physically located in /usr,
> > >    with symlinks /bin -> usr/bin, /sbin -> usr/sbin, /lib -> usr/lib,
> > >    /lib64 -> usr/lib64
> > >    (this is what usrmerge, debootstrap --merged-usr and Fedora do; dpkg
> > >    maintainer Guillem Jover refers to this as "merged /usr via aliased
> > >    directories" which seems like a good unambiguous term)
> > 
> > Aren't there two sub-solutions?
> > 
> > 1a. with packages shipping files both in /bin und /usr/bin.
> > 1b. with packages shipping files only in /usr/bin.
> 
> What precisely do you mean by "shipping files"?

"We allow packages to ship files".  So either we force all packages to
only ship files in /usr/*, instead of e.g. /bin.  Or we continue with
status quo, where packes may use either /bin or /usr/bin, which is part
of the current mess.

Bastian

-- 
	"That unit is a woman."
	"A mass of conflicting impulses."
		-- Spock and Nomad, "The Changeling", stardate 3541.9

Message #65 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: Keith Packard <keithp@keithp.com>

Cc: 978636@bugs.debian.org, Sean Whitton <spwhitton@spwhitton.name>

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Tue, 26 Jan 2021 09:38:25 +0000

On Mon, 25 Jan 2021 at 14:47:56 -0800, Keith Packard wrote:
> I think that and a transition plan are both key to this project. I
> recently installed Debian from scratch on my HiFive unmatched board and
> it got merged / and /usr.

That ship has already sailed: on #914897 in 2019, before Debian 10, the
TC resolved (unanimously) not to overrule the debootstrap maintainers
after they made merged /usr via aliasing symlinks the default for
new installations. We also voted on what we considered the desired
situation for Debian 11 to be, with option M "middle" (merged /usr not
mandatory, packages can be built on either merged-/usr or non-merged-/usr)
narrowly winning over option H "hard" (packages should only be built in
a merged-/usr environment).

Option W "weak" (packages should only be built in a non-merged-/usr
environment) was defeated by both those options.

> When I built packages on this device, they
> created invalid packages as the shared library dependencies all listed
> /lib instead of /usr/lib.

That shouldn't normally be the case, because shared library dependencies
are done in terms of SONAMEs and package names rather than in terms of
absolute paths (unless you are using RPATHs or RUNPATHs). One of the
variations tested by reproducible-builds.org is that it builds a package
on merged and unmerged /usr, so for any package that is reported to be
reproducible on that infrastructure (there are a lot of them), it
doesn't matter.

This sometimes requires forcing a canonical path via something like
--with-dbus-daemon=/usr/bin/dbus-daemon if the package's configure scripts
would normally detect an absolute path via PATH search and hard-code the
result; in my experience it has normally been executables rather than
libraries that have this problem, because ld.so and shlibs/symbols files
already provide an abstraction layer over the precise physical location
of shared libraries. (#913729 is a typical bug of this class.)

If you are doing lower-level things with libraries, or if you are shipping
libtool .la files or something else that hard-codes the absolute path to
a library, then, yes, this could be a problem with that specific package.

The resolution of #914897 says this is currently considered a bug in
those packages, although if we complete a transition to all supported
Debian systems being merged /usr via aliased directories (layout 1)
then these bugs cease to be bugs.

> Any plan where a transitioning system cannot be used for ongoing Debian
> development seems unworkable to me -- it leaves all active Debian
> developers working on un-transitioned systems, which greatly reduces
> test coverage from people in the best position to help fix things.

The resolution of #914897 was that we consider both transitioned and
un-transitioned systems equally valid for ongoing Debian development, and
any package that builds differently in those circumstances has a bug.

I for one have been continuing to use un-transitioned systems for the
opposite of the reason you are: to give that configuration more test
coverage, because it is *more* likely to exhibit bugs!

The classes of bugs "a file is /usr/bin/x but another package refers to
/bin/x" and "a file is /bin/y but another package refers to /usr/bin/y"
cease to be visible with merged /usr via aliasing, and can only affect
systems where /bin and /usr/bin are distinct. One of the motivations for
merged /usr is to have those classes of avoidable bugs cease to be bugs
at all: if /bin literally *is* /usr/bin on every supported Debian system,
then it doesn't matter which name you use for it, because both paths
are equivalent anyway.

However, if we want that (which I think we do), we need to get there
from here.

> I don't understand the value of 2b; it's functionally similar to layout
> 1, but makes for additional work to create the shadow links, along with
> consuming space for them.

Right. I wanted to describe this layout so that we can be clear
about whether the TC considers it to be a valid implementation of our
recommendation. If we don't, then hopefully we can avoid anyone arguing
that we have mandated this additional work.

    smcv

Message #70 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: Bastian Blank <waldi@debian.org>

Cc: 978636@bugs.debian.org, Sean Whitton <spwhitton@spwhitton.name>

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Tue, 26 Jan 2021 09:56:46 +0000

On Tue, 26 Jan 2021 at 10:30:34 +0100, Bastian Blank wrote:
> On Tue, Jan 26, 2021 at 09:01:12AM +0000, Simon McVittie wrote:
> > On Tue, 26 Jan 2021 at 08:02:06 +0100, Bastian Blank wrote:
> > > Aren't there two sub-solutions?
> > > 
> > > 1a. with packages shipping files both in /bin und /usr/bin.
> > > 1b. with packages shipping files only in /usr/bin.
> > 
> > What precisely do you mean by "shipping files"?
> 
> "We allow packages to ship files".  So either we force all packages to
> only ship files in /usr/*, instead of e.g. /bin.  Or we continue with
> status quo, where packes may use either /bin or /usr/bin, which is part
> of the current mess.

Oh, I see. So when you say "both" in 1a, you're referring to the overall
system - like the fact that we have both /bin/bash and /usr/bin/perl.

I don't see how we can force all packages to only ship files in /usr/*
(your 1b), except by either *first* moving to merged-/usr-via-aliasing
(layout 1, which in this case would have to be your 1a because it has to
happen before we can have 1b), or introducing new functionality in dpkg
and waiting another release cycle or two to make it guaranteed-to-exist.

Rationale:

* bash and libc6 are Essential
  (so are other packages, but these two are enough to demonstrate my point)
* bash has traditionally shipped /bin/bash, and this is part of its
  Essential interface (other packages ship #!/bin/bash scripts)
* libc6 ships /lib/ld-linux.so.2 and other architectures' equivalents,
  and this is part of its Essential interface
  (other i386 packages have ELF interpreter /lib/ld-linux.so.2)
* Essential packages are required to be functional after being merely
  unpacked, not configured (otherwise debootstrap can't work)

So if we unpack bash and libc6:i386, but do not configure them, /bin/bash
and /lib/ld-linux.so.2 must exist in the resulting chroot.

If that chroot is *already* mandated to be merged-/usr-via-aliasing
(layout 1), then it would be fine for bash's filesystem tarball
to contain ./usr/bin/bash and libc6's filesystem tarball to contain
./usr/lib/ld-linux.so.2, because the /bin -> usr/bin and /lib -> usr/lib
symlinks take care of keeping those packages' Essential interfaces intact.

However, if that chroot is in layout 2a or 2b, and bash's filesystem tarball
contains ./usr/bin/bash, then its Essential interface is incomplete:
there will be no /bin/bash symlink until the package is configured
(maintainer scripts are run).

I agree that your 1b is preferable *eventually*, but I think your 1a is
necessary as a stepping-stone to get there.

The only other option that I can see is for dpkg to gain the ability to
populate what we might call the "mergeable" directories (/bin, /sbin,
/lib*) in a purely declarative way, during unpack. If that isn't introduced
until Debian 12, then the packages in Debian 13 would be the first that
can rely on that feature existing.

    smcv

Message #75 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: 978636@bugs.debian.org

Cc: Sean Whitton <spwhitton@spwhitton.name>

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Tue, 26 Jan 2021 10:03:14 +0000

On Mon, 25 Jan 2021 at 21:22:29 +0000, Simon McVittie wrote:
> On Mon, 25 Jan 2021 at 11:46:35 -0700, Sean Whitton wrote:
> > > The Technical Committee resolves that Debian 'bookworm' should support
> > > only the merged-usr root filesystem layout, dropping support for the
> > > non-merged-usr layout.
> 
> Should we be more specific than this in what we vote on, to avoid
> later having to adjudicate between developers who say that a particular
> implementation is or isn't merged-usr?

Sorry, I had missed that we have prior art for this. When we
resolved #914897 [1] we wrote:

    ## What is "merged `/usr`"

    "Merged `/usr`" describes a possible future standard directories
    scheme in which the `/{bin,sbin,lib*}/` directories have been made
    superfluous through replacing them by symlinks to their `/usr`
    equivalents (`/usr/{bin,sbin,lib*}`).

That's exactly what Guillem calls "merged /usr via aliasing", or the
"layout 1" from my previous mail.

I still think our resolution for #978636 should be clear on what we mean
by merged-usr (like the resolution for #914897 was), but this gives me
more confidence that we did indeed all intend to be voting on mandating
merged /usr via aliasing, vs. not mandating that, vs. further discussion.

    smcv

[1] https://lists.debian.org/debian-devel-announce/2019/03/msg00001.html

Message #80 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Bastian Blank <waldi@debian.org>

To: Simon McVittie <smcv@debian.org>, 978636@bugs.debian.org

Cc: Sean Whitton <spwhitton@spwhitton.name>

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Tue, 26 Jan 2021 11:27:07 +0100

On Tue, Jan 26, 2021 at 09:56:46AM +0000, Simon McVittie wrote:
> Oh, I see. So when you say "both" in 1a, you're referring to the overall
> system - like the fact that we have both /bin/bash and /usr/bin/perl.

Yes.

> I don't see how we can force all packages to only ship files in /usr/*
> (your 1b), except by either *first* moving to merged-/usr-via-aliasing

Which is the easy part.  We say: other systems are not longer supported
after date X.  Otherwise we will never get it done.

> * bash and libc6 are Essential
>   (so are other packages, but these two are enough to demonstrate my point)
> * bash has traditionally shipped /bin/bash, and this is part of its
>   Essential interface (other packages ship #!/bin/bash scripts)
> * libc6 ships /lib/ld-linux.so.2 and other architectures' equivalents,
>   and this is part of its Essential interface
>   (other i386 packages have ELF interpreter /lib/ld-linux.so.2)
> * Essential packages are required to be functional after being merely
>   unpacked, not configured (otherwise debootstrap can't work)
> 
> So if we unpack bash and libc6:i386, but do not configure them, /bin/bash
> and /lib/ld-linux.so.2 must exist in the resulting chroot.

Before unpacking those packages, both /bin and /lib symlinks must
already exist, because it's past the cutoff date of non-aliased support.

> However, if that chroot is in layout 2a or 2b, and bash's filesystem tarball
> contains ./usr/bin/bash, then its Essential interface is incomplete:
> there will be no /bin/bash symlink until the package is configured
> (maintainer scripts are run).

Option 2 doesn't provide us any benefit.  The world already implemented
option 1.

> I agree that your 1b is preferable *eventually*, but I think your 1a is
> necessary as a stepping-stone to get there.

We are already effectively at 1a.  So we need to decide if we want 1b to
fix the fallout.

> The only other option that I can see is for dpkg to gain the ability to
> populate what we might call the "mergeable" directories (/bin, /sbin,
> /lib*) in a purely declarative way, during unpack.

No, because we want to support /usr only systems or snapshots, where /
is populated on first boot.  That's where the world is going.

Bastian

-- 
Earth -- mother of the most beautiful women in the universe.
		-- Apollo, "Who Mourns for Adonais?" stardate 3468.1

Message #85 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: Bastian Blank <waldi@debian.org>

Cc: 978636@bugs.debian.org, Sean Whitton <spwhitton@spwhitton.name>

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Tue, 26 Jan 2021 10:49:14 +0000

On Tue, 26 Jan 2021 at 11:27:07 +0100, Bastian Blank wrote:
> Before unpacking those packages, both /bin and /lib symlinks must
> already exist, because it's past the cutoff date of non-aliased support.

I would like that to become true, but the cutoff date of non-aliased
support has not yet happened, and this bug is about making it
happen. *After* we have done that, we can consider mandating your 1b.

> Option 2 doesn't provide us any benefit.  The world already implemented
> option 1.

I personally agree, and I hope the technical committee as a whole will too,
but I wanted to describe option 2 so that we can be clear about what we're
resolving and whether option 2 is it.

> We are already effectively at 1a.  So we need to decide if we want 1b to
> fix the fallout.

We are *partially* at 1a: systems that were installed since the release
of buster are (unless steps were taken to avoid that), and systems that
have had usrmerge installed are, but other upgraded systems are not.

I'm typing this on a laptop that is still not merged-/usr, because I think
it's more likely that packages will work correctly on merged-/usr, and
if packages that I care about fail to work on a non-merged-/usr system,
I want it to happen to me, not to someone who is in a worse position to
debug it.

> > The only other option that I can see is for dpkg to gain the ability to
> > populate what we might call the "mergeable" directories (/bin, /sbin,
> > /lib*) in a purely declarative way, during unpack.
> 
> No, because we want to support /usr only systems or snapshots, where /
> is populated on first boot.

Sure, and I'm not saying that this is the option that we should take
(in fact I think we should not do this), but it's an option that exists
and is possible. I think the reasons to reject it outweigh the reasons
to take it, but I want to be aware of all the options that exist, not
just the ones that are convenient for my own plans.

    smcv

Message #90 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Philip Hands <phil@hands.com>

To: 978636@bugs.debian.org

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Tue, 26 Jan 2021 12:18:37 +0100

[Message part 1 (text/plain, inline)]

Adam Borowski <kilobyte@angband.pl> writes:

> On Mon, Jan 25, 2021 at 11:45:55AM -0700, Sean Whitton wrote:
>> Dear Technical Committee members,
>> 
>> I call for votes on the following ballot to resolve #978636.  The voting
>> period starts immediately and lasts for up to one week, or until the
>> outcome is no longer in doubt (§6.3.1).
>> 
>> ===BEGIN
>> The Technical Committee resolves that Debian 'bookworm' should support
>> only the merged-usr root filesystem layout, dropping support for the
>> non-merged-usr layout.
>
> You did not even address, or even mention, that this idea has been vetoed by
> dpkg maintainers.

Where did you get the idea that maintainers have a veto regarding TC decisions?

BTW if you were to express your opinion here:

> And as that, it's a complete non-starter.

in the form of an expression of opinion (e.g. by putting something like
"I think" in front of it) rather than presenting it as though it were a
fact, you would be much less provocative, which might lead to a more
constructive discussion overall.

Cheers, Phil.
-- 
|)|  Philip Hands  [+44 (0)20 8530 9560]  HANDS.COM Ltd.
|-|  http://www.hands.com/    http://ftp.uk.debian.org/
|(|  Hugo-Klemm-Strasse 34,   21075 Hamburg,    GERMANY

[signature.asc (application/pgp-signature, inline)]

Message #95 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Wouter Verhelst <wouter@debian.org>

To: Marco d'Itri <md@linux.it>, 978636@bugs.debian.org

Cc: Simon McVittie <smcv@debian.org>, Sean Whitton <spwhitton@spwhitton.name>

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Tue, 26 Jan 2021 13:17:38 +0200

On Tue, Jan 26, 2021 at 12:28:45AM +0100, Marco d'Itri wrote:
> Also, as is has been discussed, if the /usr/doc/ transition was 
> representative then this would probably take many years.

You keep using that as an argument. I think it's very disinginuous to
point to a problem Debian had over 20 years ago[1] and say "look, we
don't know how to do this quickly". It's not like things haven't changed
since.

We can (and should, IMO) declare *today* that for bookworm, shipping
files in / (as opposed to /usr) that are not compatibility symlinks will
be RC.

You can start writing a lintian check today for the same.

If we take both these steps, this will only take us one release cycle,
and the dpkg maintainers can come up with a plan to remove /bin and
friends and replace them by symlinks in ways that will not confuse dpkg.

[1] the /usr/doc transition was in progress when I first joined Debian,
20 years ago. I don't remember the start of it, but I do remember it
ending.

-- 
To the thief who stole my anti-depressants: I hope you're happy

  -- seen somewhere on the Internet on a photo of a billboard

Message #100 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Luca Boccassi <bluca@debian.org>

To: Wouter Verhelst <wouter@debian.org>, 978636@bugs.debian.org, Marco d'Itri <md@linux.it>

Cc: Simon McVittie <smcv@debian.org>, Sean Whitton <spwhitton@spwhitton.name>

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Tue, 26 Jan 2021 14:52:16 +0000

[Message part 1 (text/plain, inline)]

On Tue, 2021-01-26 at 13:17 +0200, Wouter Verhelst wrote:
> On Tue, Jan 26, 2021 at 12:28:45AM +0100, Marco d'Itri wrote:
> > Also, as is has been discussed, if the /usr/doc/ transition was 
> > representative then this would probably take many years.
> 
> You keep using that as an argument. I think it's very disinginuous to
> point to a problem Debian had over 20 years ago[1] and say "look, we
> don't know how to do this quickly". It's not like things haven't changed
> since.
> 
> We can (and should, IMO) declare *today* that for bookworm, shipping
> files in / (as opposed to /usr) that are not compatibility symlinks will
> be RC.
> 
> You can start writing a lintian check today for the same.
> 
> If we take both these steps, this will only take us one release cycle,
> and the dpkg maintainers can come up with a plan to remove /bin and
> friends and replace them by symlinks in ways that will not confuse dpkg.
> 
> [1] the /usr/doc transition was in progress when I first joined Debian,
> 20 years ago. I don't remember the start of it, but I do remember it
> ending.

I mentioned this on IRC earlier, but I think it warrants a citation on
the list/bug since I don't think it was referenced before.

AFAIK, SUSE tried the symlink-farm way (ie: some variation of option
2), and the current status is explained here:

https://en.opensuse.org/openSUSE:Usr_merge

Some quotes:

"A previous attempt of the UsrMerge from 2012 was never finished"

"The current state is an inconsistent mess"

And this in a distro with a strong governance model behind.

If I understand correctly, it looks like they are now attempting to go
the usrmerged-way (ie: option 1, or the Fedora-option).

All in all, we have 2 real-world case studies.

- Fedora tried the usrmerge method, and succeeded
- SUSE tried the symlink-farm method, and (it appears) failed

Aside from all theoreticals and hyphoteticals, this seems to me to be a
pretty important real-world data point to consider when deciding which
strategy to adopt.

-- 
Kind regards,
Luca Boccassi

[signature.asc (application/pgp-signature, inline)]

Message #103 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Ansgar <ansgar@debian.org>

To: 978636@bugs.debian.org

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Tue, 26 Jan 2021 15:57:33 +0100

Hi,

Simon McVittie writes:
> Should we be more specific than this in what we vote on, to avoid
> later having to adjudicate between developers who say that a particular
> implementation is or isn't merged-usr?
>
> Some developers seem to be using "merged /usr" to refer to multiple
> concrete layouts:
>
> 1. an arrangement where all regular files that have traditionally been
>    in /bin, /sbin, /lib and /lib64 are physically located in /usr,
>    with symlinks /bin -> usr/bin, /sbin -> usr/sbin, /lib -> usr/lib,
>    /lib64 -> usr/lib64

Having filed this bug, I only consider this "merged-/usr".

Symlink farms (/bin/sh -> /usr/bin/sh) are a different layout and should
be given a different name.

With that said, I consider the symlink farms also unrealistic for use as
an intermediate state in the transition.  This was tried by another
distribution[1] and it didn't work out; nobody said why Debian wouldn't
end up in the same unfortunate situation from which it is even harder to
migrate to merged-/usr.

From my point of view the only realistic proposal for migration so far
is usrmerge, possibly with modifications.

We might also want to be extra careful and use non-merged-/usr on
buildds for one more release and only move them to merged-/usr for the
release after merged-/usr was made mandatory (so any packages installed
as part of the upgrade to the release that makes merged-/usr mandatory
are still built on systems with non-merged-/usr for the reasons we do so
currently).

But I consider these implementations details; it's only useful to
discuss them when we know where we are going and most (or hopefully all)
detail questions probably don't need the technical committee either.

Regards,
Ansgar

  [1]: https://en.opensuse.org/openSUSE:Usr_merge

Message #108 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Gunnar Wolf <gwolf@debian.org>

To: Wouter Verhelst <wouter@debian.org>, 978636@bugs.debian.org

Cc: Marco d'Itri <md@linux.it>, Simon McVittie <smcv@debian.org>, Sean Whitton <spwhitton@spwhitton.name>

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Tue, 26 Jan 2021 12:17:37 -0600

[Message part 1 (text/plain, inline)]

Wouter Verhelst dijo [Tue, Jan 26, 2021 at 01:17:38PM +0200]:
> On Tue, Jan 26, 2021 at 12:28:45AM +0100, Marco d'Itri wrote:
> > Also, as is has been discussed, if the /usr/doc/ transition was 
> > representative then this would probably take many years.
> 
> You keep using that as an argument. I think it's very disinginuous to
> point to a problem Debian had over 20 years ago[1] and say "look, we
> don't know how to do this quickly". It's not like things haven't changed
> since.
> 
> We can (and should, IMO) declare *today* that for bookworm, shipping
> files in / (as opposed to /usr) that are not compatibility symlinks will
> be RC.
> 
> You can start writing a lintian check today for the same.

I agree with you, and I think that once this TC issue/vote is settled
(which I expect to lean towards "yes", with Simon's option #1 — But I
have been surprised before more than once ;-) ), we should take steps
such as a GR to make what you suggest be enforced for Bookworm.

Keep in mind, though, that we have _many_ packages that are not
rebuilt even once per stable cycle. Many of them are not even in the
general awareness of their maintainers (or are just RC). If they
become insta-buggy, we will have to take massive action on them
(which... well, can of course be positive if we can clean them up from
many other inherited bits of cruft!)

[signature.asc (application/pgp-signature, inline)]

Message #113 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Felix Lechner <felix.lechner@lease-up.com>

To: 978636@bugs.debian.org

Cc: "Marco d'Itri" <md@linux.it>, Simon McVittie <smcv@debian.org>, Sean Whitton <spwhitton@spwhitton.name>, Luca Boccassi <bluca@debian.org>, Gunnar Wolf <gwolf@debian.org>, Wouter Verhelst <wouter@debian.org>

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Tue, 26 Jan 2021 11:17:38 -0800

Hi,

On Tue, Jan 26, 2021 at 3:45 AM Wouter Verhelst <wouter@debian.org> wrote:
>
> You can start writing a lintian check today

Here is a Lintian check that follows Ansgar's specification in the
second d-d thead. Of course, it will not be merged until the project
works out a suitable consensus on this controversial topic:

    https://salsa.debian.org/lintian/lintian/-/merge_requests/349

Thank you!

Kind regards
Felix Lechner

Message #118 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: Gunnar Wolf <gwolf@debian.org>

Cc: Wouter Verhelst <wouter@debian.org>, 978636@bugs.debian.org, Marco d'Itri <md@linux.it>, Sean Whitton <spwhitton@spwhitton.name>

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Tue, 26 Jan 2021 20:53:56 +0000

On Tue, 26 Jan 2021 at 12:17:37 -0600, Gunnar Wolf wrote:
> Wouter Verhelst dijo [Tue, Jan 26, 2021 at 01:17:38PM +0200]:
> > We can (and should, IMO) declare *today* that for bookworm, shipping
> > files in / (as opposed to /usr) that are not compatibility symlinks will
> > be RC.
> 
> I agree with you

For at least the Essential packages like bash and libc6, I don't think
we can go in that order. Given that implementations of merged /usr via
aliasing symlinks already exist, I think Essential packages that ship
part of their Essential functionality in /bin, /sbin, /lib* must stay
as they are now, until after we have had a flag day (a release)
at which either:

a. those aliasing symlinks are guaranteed to be in place (Ansgar's request
   in this bug)
b. those aliasing symlinks are forbidden and must be rolled back (Guillem's
   preferred answer to this, but judging by #914897 and discussion of this
   bug, very unlikely to be the technical committee's preferred answer)

That's because these two axioms collide:

* Essential packages must provide their Essential functionality while
  unpacked but not configured
  - e.g. unpacking libc6:i386 creates /lib/ld-linux.so.2, which is Essential
    functionality that other packages rely on

* There are existing installations with merged /usr via aliasing symlinks
  - Therefore compatibility symlinks (in either direction!) must be created
    by a maintainer script rather than directly shipped in the
    dpkg --fsys-tarfile, otherwise they will fail to unpack on those existing
    systems
    - e.g. if unpacking libc6:i386 creates /usr/lib/ld-linux.so.2, then it
      cannot also create /lib/ld-linux.so.2; that would have to happen
      later, when it's configured

I don't see a way to have a compromise position in Essential packages,
other than the one we have right now (where the name in the
dpkg --fsys-tarfile must match other packages' expectations, whether
that means with or without /usr), without breaking the Essential
property.

When thinking about this transition (and any contentious issue, really),
please distinguish between:

- things that (in someone's opinion, maybe yours) we *shouldn't* do;
- things that (according to properties we take as axiomatic, like the
  Essential property and the requirement that upgrades work) we *can't* do

I have been trying hard to consider all the options that are available -
even the ones that I personally think we *shouldn't* do - but immediately
dismiss the ones that we *can't* do.

    smcv

Message #123 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Ansgar <ansgar@debian.org>

To: 978636@bugs.debian.org

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Tue, 26 Jan 2021 22:36:27 +0100

On Tue, 2021-01-26 at 13:17 +0200, Wouter Verhelst wrote:
> We can (and should, IMO) declare *today* that for bookworm, shipping
> files in / (as opposed to /usr) that are not compatibility symlinks
> will be RC.

I fear we are drifting away from just deciding to move to merged-/usr
to implementation details, but I originally[1] suggested to wait one
release between making merged-/usr mandatory (could happen in bookworm)
and moving installation paths in packages (could happen in trixie).

This seems to avoid several problems:
- partial upgrades to bookworm or backported packages from bookworm to
bullseye and from trixie to bookworm should still just work (backports
from then-testing trixie to then-oldstable bullseye, that is over two
releases, would need to take a bit more care)
- we need to touch packages for the move only once
- compat symlinks have various problems (cf. references to essential
packages[2] and OpenSuSE[3]).

Ansgar

  [1]: https://lists.debian.org/debian-devel/2020/11/msg00232.html
  [2]: https://lists.debian.org/debian-ctte/2021/01/msg00041.html
  [3]: https://lists.debian.org/debian-ctte/2021/01/msg00037.html

Message #128 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: Sean Whitton <spwhitton@spwhitton.name>, 978636@bugs.debian.org

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Sun, 31 Jan 2021 13:11:53 +0000

On Mon, 25 Jan 2021 at 11:45:55 -0700, Sean Whitton wrote:
> ===BEGIN
> The Technical Committee resolves that Debian 'bookworm' should support
> only the merged-usr root filesystem layout, dropping support for the
> non-merged-usr layout.
> 
> Until after the release of 'bullseye', any implementation of this
> resolution must be done in the 'experimental' distribution, or otherwise
> kept out of the critical paths for the release of 'bullseye'.
> 
> We do not recommend any particular implementation of the migration.
> 
> Y: Yes, support only merged-usr in the 'bookworm' release.
> N: No, continue to support both layouts in 'bookworm'.
> F: Further Discussion
> ===END

I vote Y > F > N.

This is on the assumption that we are defining merged /usr the
same way we did when we resolved #914897 in
<https://lists.debian.org/debian-devel-announce/2019/03/msg00001.html>
(what Guillem Jover calls "merged /usr via aliased directories", with
symlinks like /bin -> usr/bin).

    smcv

Message #133 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Niko Tyni <ntyni@debian.org>

To: 978636@bugs.debian.org

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Sun, 31 Jan 2021 22:29:19 +0200

[Message part 1 (text/plain, inline)]

On Mon, Jan 25, 2021 at 11:45:55AM -0700, Sean Whitton wrote:

> ===BEGIN
> The Technical Committee resolves that Debian 'bookworm' should support
> only the merged-usr root filesystem layout, dropping support for the
> non-merged-usr layout.
> 
> Until after the release of 'bullseye', any implementation of this
> resolution must be done in the 'experimental' distribution, or otherwise
> kept out of the critical paths for the release of 'bullseye'.
> 
> We do not recommend any particular implementation of the migration.
> 
> Y: Yes, support only merged-usr in the 'bookworm' release.
> N: No, continue to support both layouts in 'bookworm'.
> F: Further Discussion
> ===END

I vote:

Y > F > N

-- 
Niko Tyni   ntyni@debian.org

[signature.asc (application/pgp-signature, inline)]

Message #138 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Gunnar Wolf <gwolf@debian.org>

To: Sean Whitton <spwhitton@spwhitton.name>, 978636@bugs.debian.org

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Sun, 31 Jan 2021 15:32:01 -0600

[Message part 1 (text/plain, inline)]

Sean Whitton dijo [Mon, Jan 25, 2021 at 11:45:55AM -0700]:
> ===BEGIN
> The Technical Committee resolves that Debian 'bookworm' should support
> only the merged-usr root filesystem layout, dropping support for the
> non-merged-usr layout.
> 
> Until after the release of 'bullseye', any implementation of this
> resolution must be done in the 'experimental' distribution, or otherwise
> kept out of the critical paths for the release of 'bullseye'.
> 
> We do not recommend any particular implementation of the migration.
> 
> Y: Yes, support only merged-usr in the 'bookworm' release.
> N: No, continue to support both layouts in 'bookworm'.
> F: Further Discussion
> ===END

My vote is:

Y > F > N

[signature.asc (application/pgp-signature, inline)]

Message #143 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Gunnar Wolf <gwolf@debian.org>

To: Sean Whitton <spwhitton@spwhitton.name>, 978636@bugs.debian.org

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Sun, 31 Jan 2021 15:34:27 -0600

[Message part 1 (text/plain, inline)]

Gunnar Wolf dijo [Sun, Jan 31, 2021 at 03:32:02PM -0600]:
> My vote is:
> 
> Y > F > N


...And to be clear: We at the TC are *not* doing detailed design
work. But I want to state (and not as part of the vote, but just as
yet another DD) that the only way I feel makes sense to continue now
is via Simon's ① option: Symlinking /bin → /usr/bin, /lib → /usr/lib,
/sbin → /usr/sbin, etc.

That will allow us to take a step later on to mandate packages not
shipping files in the no-longer-root-level-directories, but that
should be at least one further release cycle down the lane.

[signature.asc (application/pgp-signature, inline)]

Message #148 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Elana Hashman <ehashman@debian.org>

To: Sean Whitton <spwhitton@spwhitton.name>, 978636@bugs.debian.org

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Sun, 31 Jan 2021 14:31:43 -0800

[Message part 1 (text/plain, inline)]

On Mon, Jan 25, 2021 at 11:45:55AM -0700, Sean Whitton wrote:
> Dear Technical Committee members,
> 
> I call for votes on the following ballot to resolve #978636.  The voting
> period starts immediately and lasts for up to one week, or until the
> outcome is no longer in doubt (§6.3.1).
> 
> ===BEGIN
> The Technical Committee resolves that Debian 'bookworm' should support
> only the merged-usr root filesystem layout, dropping support for the
> non-merged-usr layout.
> 
> Until after the release of 'bullseye', any implementation of this
> resolution must be done in the 'experimental' distribution, or otherwise
> kept out of the critical paths for the release of 'bullseye'.
> 
> We do not recommend any particular implementation of the migration.
> 
> Y: Yes, support only merged-usr in the 'bookworm' release.
> N: No, continue to support both layouts in 'bookworm'.
> F: Further Discussion
> ===END

I vote:

Y > F > N

- e

[signature.asc (application/pgp-signature, inline)]

Message #153 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Margarita Manterola <marga@debian.org>

To: 978636@bugs.debian.org

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Mon, 1 Feb 2021 10:53:13 +0100

[Message part 1 (text/plain, inline)]

Hola Sean Whitton!

> ===BEGIN
> The Technical Committee resolves that Debian 'bookworm' should support
> only the merged-usr root filesystem layout, dropping support for the
> non-merged-usr layout.
> 
> Until after the release of 'bullseye', any implementation of this
> resolution must be done in the 'experimental' distribution, or otherwise
> kept out of the critical paths for the release of 'bullseye'.
> 
> We do not recommend any particular implementation of the migration.
> 
> Y: Yes, support only merged-usr in the 'bookworm' release.
> N: No, continue to support both layouts in 'bookworm'.
> F: Further Discussion
> ===END

I vote:
Y > F > N

With similar caveats as have been raised by others regarding what merged-usr
means. It should mean the same that was described in the previous
resolution[1]. In other words, by symlinking `/{bin,sbin,lib*}/` to
`/usr/{bin,sbin,lib*}/`.

On top of this, the fact that we are not mandating a particular implementation
also means that we are not mandating any particular migration path. The
migration chosen by those doing the implementation might not be finished by the
time of the bookworm release. We are not mandating that it should be, only that
the previous format does not need to be supported anymore.

[1]: https://lists.debian.org/debian-devel-announce/2019/03/msg00001.html

-- 
Regards,
Marga

[signature.asc (application/pgp-signature, inline)]

Message #158 received at 978636@bugs.debian.org (full text, mbox, reply):

From: David Bremner <david@tethera.net>

To: 978636@bugs.debian.org

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Mon, 01 Feb 2021 07:08:51 -0400

[Message part 1 (text/plain, inline)]

Sean Whitton <spwhitton@spwhitton.name> writes:

> ===BEGIN
> The Technical Committee resolves that Debian 'bookworm' should support
> only the merged-usr root filesystem layout, dropping support for the
> non-merged-usr layout.
>
> Until after the release of 'bullseye', any implementation of this
> resolution must be done in the 'experimental' distribution, or otherwise
> kept out of the critical paths for the release of 'bullseye'.
>
> We do not recommend any particular implementation of the migration.
>
> Y: Yes, support only merged-usr in the 'bookworm' release.
> N: No, continue to support both layouts in 'bookworm'.
> F: Further Discussion
> ===END

I vote

  Y > F > N

[signature.asc (application/pgp-signature, inline)]

Message #163 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Adam Borowski <kilobyte@angband.pl>

To: 978636@bugs.debian.org

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Mon, 1 Feb 2021 13:35:28 +0100

On Sun, Jan 31, 2021 at 03:34:27PM -0600, Gunnar Wolf wrote:
> ...And to be clear: We at the TC are *not* doing detailed design
> work. But I want to state (and not as part of the vote, but just as
> yet another DD) that the only way I feel makes sense to continue now
> is via Simon's ① option: Symlinking /bin → /usr/bin, /lib → /usr/lib,
> /sbin → /usr/sbin, etc.
> 
> That will allow us to take a step later on to mandate packages not
> shipping files in the no-longer-root-level-directories, but that
> should be at least one further release cycle down the lane.

I'd strongly urge the opposite order: FIRST decree that no package may
ship any file to non-canonical path (ie, have dpkg extract anything over
a symlink), and only then flip the switch.

Contrary to talk about a "20 years transition", I estimate there's just
~100 sources that would have to be changed.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ .--[ Makefile ]
⣾⠁⢠⠒⠀⣿⡁ # beware of races
⢿⡄⠘⠷⠚⠋⠀ all: pillage burn
⠈⠳⣄⠀⠀⠀⠀ `----

Message #168 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Marco d'Itri <md@Linux.IT>

To: Adam Borowski <kilobyte@angband.pl>, 978636@bugs.debian.org

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Mon, 1 Feb 2021 13:48:29 +0100

[Message part 1 (text/plain, inline)]

On Feb 01, Adam Borowski <kilobyte@angband.pl> wrote:

> I'd strongly urge the opposite order: FIRST decree that no package may
> ship any file to non-canonical path (ie, have dpkg extract anything over
> a symlink), and only then flip the switch.
There is no "switch": this decision only certifies what we have been 
doing by default for two releases.
Almost all new Debian systems are already installed this way.

> Contrary to talk about a "20 years transition", I estimate there's just
> ~100 sources that would have to be changed.
Then if you believe that the current situation is troublesome you may 
start now and it will be easy!

-- 
ciao,
Marco

[signature.asc (application/pgp-signature, inline)]

Message #173 received at 978636@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: Adam Borowski <kilobyte@angband.pl>, 978636@bugs.debian.org

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Mon, 1 Feb 2021 12:55:27 +0000

On Mon, 01 Feb 2021 at 13:35:28 +0100, Adam Borowski wrote:
> On Sun, Jan 31, 2021 at 03:34:27PM -0600, Gunnar Wolf wrote:
> > I want to state (and not as part of the vote, but just as
> > yet another DD) that the only way I feel makes sense to continue now
> > is via Simon's ① option: Symlinking /bin → /usr/bin, /lib → /usr/lib,
> > /sbin → /usr/sbin, etc.
> > 
> > That will allow us to take a step later on to mandate packages not
> > shipping files in the no-longer-root-level-directories, but that
> > should be at least one further release cycle down the lane.
> 
> I'd strongly urge the opposite order: FIRST decree that no package may
> ship any file to non-canonical path (ie, have dpkg extract anything over
> a symlink), and only then flip the switch.

I don't want to repeat myself, so please see
<https://lists.debian.org/debian-ctte/2021/01/msg00031.html> for one
reason why that doesn't work (unless you have a solution that I've
missed).

    smcv

Message #178 received at 978636-done@bugs.debian.org (full text, mbox, reply):

From: Sean Whitton <spwhitton@spwhitton.name>

To: 978636-done@bugs.debian.org

Subject: Re: Bug#978636: move to merged-usr-only?

Date: Mon, 01 Feb 2021 10:13:11 -0700

[Message part 1 (text/plain, inline)]

Hello,

On Mon 25 Jan 2021 at 11:45AM -07, Sean Whitton wrote:

> I call for votes on the following ballot to resolve #978636.  The voting
> period starts immediately and lasts for up to one week, or until the
> outcome is no longer in doubt (§6.3.1).

The vote has concluded.

Marga, David, Niko, Gunnar, Simon, Elana and myself all voted: YFN.

Therefore:

    The Technical Committee resolves that Debian 'bookworm' should
    support only the merged-usr root filesystem layout, dropping support
    for the non-merged-usr layout.

    Until after the release of 'bullseye', any implementation of this
    resolution must be done in the 'experimental' distribution, or
    otherwise kept out of the critical paths for the release of
    'bullseye'.

-- 
Sean Whitton

[signature.asc (application/pgp-signature, inline)]

Send a report that this bug log contains spam.