Debian Bug report logs - #557730
/etc/{protocols,network,services} not schroot's to scribble over

Reply or subscribe to this bug.

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Evan Broder <broder@MIT.EDU>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: /etc/{protocols,network,services} not schroot's to scribble over

Date: Mon, 23 Nov 2009 19:19:46 -0500

Package: schroot
Version: 1.3.1-1
Severity: normal

/etc/schroot/setup.d/20nssdatabases unconditionally copies the
databases listed in /etc/schroot/nssdatabases-defaults into the
chroot. However, /etc/protocols, /etc/network, and /etc/services are
all owned by netbase, which isn't installed in build chroots.

This means that if a package build-deps on netbase, dpkg will present
a conffile conflict for those three files, which is highly undesirable
in a non-interactive builder.

Message #10 received at 557730@bugs.debian.org (full text, mbox, reply):

From: Roger Leigh <rleigh@codelibre.net>

To: Evan Broder <broder@MIT.EDU>, 557730@bugs.debian.org

Subject: Re: [buildd-tools-devel] Bug#557730: /etc/{protocols, network, services} not schroot's to scribble over

Date: Tue, 24 Nov 2009 01:09:27 +0000

[Message part 1 (text/plain, inline)]

On Mon, Nov 23, 2009 at 07:19:46PM -0500, Evan Broder wrote:
> /etc/schroot/setup.d/20nssdatabases unconditionally copies the
> databases listed in /etc/schroot/nssdatabases-defaults into the
> chroot. However, /etc/protocols, /etc/network, and /etc/services are
> all owned by netbase, which isn't installed in build chroots.

OK.  I would suggest removing these from nssdatabases-defaults for
the time being if this is causing problems, as a workaround.

> This means that if a package build-deps on netbase, dpkg will present
> a conffile conflict for those three files, which is highly undesirable
> in a non-interactive builder.

I wonder if (for buildd chroots) we should be running dpkg with
--force-confnew to unconditionally force replacement with new
conffiles?  If this is configurable via APT such as
Dpkg::Options::force=--force-confnew then maybe we should be
setting this by default (this applies more generally than just
this specific case).

We could remove these files from nssdatabases-defaults to avoid
this issue.  However, the idea is that all the NSS databases
are "slaved" to the host and automated updating is wanted in
almost all cases.

We were considering creating a separate set of buildd defaults,
which would still allow the current default behaviour on non-
buildd systems, while allowing the defaults to be restricted
on buildd setups.

Does anyone else have any further comments?


Thanks for reporting this!


Regards,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
   `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.

[signature.asc (application/pgp-signature, inline)]

Message #23 received at 557730@bugs.debian.org (full text, mbox, reply):

From: Wookey <wookey@debian.org>

To: Debian Bug Tracking System <557730@bugs.debian.org>

Subject: Re: /etc/{protocols,network,services} not schroot's to scribble over

Date: Tue, 14 Feb 2023 15:23:54 +0000

Source: schroot
Version: 1.6.13-3
Followup-For: Bug #557730

Just discovered this bug from 2009. This problem has been annoying me
regularly since about then, and I finally got round to working out what was
actually going on, which led me here.

The primary practical issue is that builds in the chroot are quite likely to bring in netbase, and that package contains /etc/protocols and /etc/services so when dpkg finds that they are already present in the chroot it stops with:
Setting up netbase (6.4) ...

Configuration file '/etc/protocols'
 ==> File on system created by you or by a script.
 ==> File also in package provided by package maintainer.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** protocols (Y/I/N/O/D/Z) [default=N] ?

So I often find that a build has just got stuck at the beginning
unless I inject y\ny\n.  This has caused endless hassle over the
years, and it's a bit sad to see it's not been fixed in 13 years. I
always assumed that someone would sort this out at some point and the
hassle would go away.

There are various ways we could deal with this. Sbuild-createchroot could set
1) Dpkg::Options::force=--force-confnew in its chroots.

Are there reasons you might not want to set this in general? People
would expect conf changes they made in their chroots to be preserved
just the way they are in a normal system. So this seems like it would
be intrusive.

2) netbase could be installed in base chroots then the problem would not arise (or only arise once).

The problem here is that chroots can be made by many tools, and the
usual tools (debootstrap and mmdebstrap) do not put netbase in by
default. It would be very easy to make sbuild-createchroot just add
--include=netbase to the invocation, and I'm not sure there is any
real downside to doing that? Documenting the reason for including this
package so that people using other tools to make chroots know it's a
good idea would also be easy and helpful.

3) schroot could just not copy those files over.

Whilst having the passwd database reflected in the chroot is
incredibly useful it's not clear how often the services and protocols
are needed, but I assume people do find that functionality useful.
  
The actual issue here is that schroot is copying them over even when
they don't exist in the chroot, then the thing that is supposed to be
installing them gets tripped up (correctly) by dpkg's checks.

So a simple fix that keeps the functionality when it's useful, but
stops this breakage, would be to only copy over the files in the
nssdatabases list _if they are already present in the
chroot_. Possibly that should apply to all the files in the list?

I'll see if I can come up with a patch to do that.

Message #28 received at 557730@bugs.debian.org (full text, mbox, reply):

From: Ian Jackson <ijackson@chiark.greenend.org.uk>

To: Wookey <wookey@debian.org>, 557730@bugs.debian.org

Subject: Re: Bug#557730: /etc/{protocols,network,services} not schroot's to scribble over

Date: Tue, 14 Feb 2023 16:02:11 +0000

Wookey writes ("Bug#557730: /etc/{protocols,network,services} not schroot's to scribble over"):
> 2) netbase could be installed in base chroots then the problem would not arise (or only arise once).
> 
> The problem here is that chroots can be made by many tools, and the
> usual tools (debootstrap and mmdebstrap) do not put netbase in by
> default. It would be very easy to make sbuild-createchroot just add
> --include=netbase to the invocation, and I'm not sure there is any
> real downside to doing that? Documenting the reason for including this
> package so that people using other tools to make chroots know it's a
> good idea would also be easy and helpful.

A downside is that missing build-dependencies on netbase would no
longer be detected.  One alternative would be to declare netbase
build-essential.  TBH I'm not sure why that hasn't been done already.

I have a 4th option:

scroot could create this file by installing and then removing (but not
purging) a fake netbase.deb ("Version: 0~~).  Then, when the new
netbase appears, the usual conffile mechanism would DTRT, since the
file would not have been "locally created" (or indeed "locally
modified").

The fake netbase.deb could be contained within schroot.deb, in
/usr/share, so schroot wouldn't need to gain runtime build-deps on
dpkg tooling.

> Whilst having the passwd database reflected in the chroot is
> incredibly useful it's not clear how often the services and protocols
> are needed, but I assume people do find that functionality useful.

I had a package that failed its build-time tests due to lack of
/etc/protocols.  The missing build-dep was detected in the buildds,
because my own local sid build chroot has netbase installed, precisely
because of this bug.

Ian.

-- 
Ian Jackson <ijackson@chiark.greenend.org.uk>   These opinions are my own.  

Pronouns: they/he.  If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.

Message #33 received at 557730@bugs.debian.org (full text, mbox, reply):

From: Wookey <wookey@wookware.org>

To: Ian Jackson <ijackson@chiark.greenend.org.uk>

Cc: 557730@bugs.debian.org

Subject: Re: Bug#557730: /etc/{protocols,network,services} not schroot's to scribble over

Date: Tue, 14 Feb 2023 16:56:53 +0000

[Message part 1 (text/plain, inline)]

On 2023-02-14 16:02 +0000, Ian Jackson wrote:
> Wookey writes ("Bug#557730: /etc/{protocols,network,services} not schroot's to scribble over"):
> > 2) netbase could be installed in base chroots then the problem would not arise (or only arise once).
> > 
> A downside is that missing build-dependencies on netbase would no
> longer be detected.  One alternative would be to declare netbase
> build-essential.  TBH I'm not sure why that hasn't been done already.

Good point. And related to recent debian-devel discussion about bare
build chroots actually being bare so that missing build-deps are
indeed discovered. I don't think that just adding more things to
'essential' is actually helpful here. netbase is not essential,
especially for builds that explicitly mustn't use the (external)
network.

To be fair schroot has a 'config=buildd' setup where /etc/protocols
and /etc/services are not copied over (and configs 'minimal', 'sbuild'
and 'mini-buildd'). The problematic situation is 'config=default' (and
'desktop') where they are. But I use the 'default' setup a lot because
it mounts /home as well and that's hugely helpful for doing various
sort of work, as opposed to a totally-clean sbuild build. And I think
it may be the default for sbuild-createchroot, but I could be wrong
about that.

For nearly everything I do a config without /etc{protocols|services}
but with /home would work great, but none of the supplied configs:
minimal, buildd, mini-buildd, sbuild, default, desktop do that. Do we
really need a 7th config (and if so what should it be
called?). Obviously I can just make one, but the fact that problem
affects people so often with the 'default' config seems to me to be a
problem we should try to fix.

> I have a 4th option:
> 
> scroot could create this file by installing and then removing (but not
> purging) a fake netbase.deb ("Version: 0~~).  Then, when the new
> netbase appears, the usual conffile mechanism would DTRT, since the
> file would not have been "locally created" (or indeed "locally
> modified").

That is a neat idea.

> The fake netbase.deb could be contained within schroot.deb, in
> /usr/share, so schroot wouldn't need to gain runtime build-deps on
> dpkg tooling.

Except that it has to build this package live in order to contain the
/etc/protocols and /etc/services files from the host
environment. Having these default files with standard contents in a
pre-built .deb is pointless, isn't it?

> > Whilst having the passwd database reflected in the chroot is
> > incredibly useful it's not clear how often the services and protocols
> > are needed, but I assume people do find that functionality useful.
> 
> I had a package that failed its build-time tests due to lack of
> /etc/protocols.  The missing build-dep was detected in the buildds,
> because my own local sid build chroot has netbase installed, precisely
> because of this bug.

Right, which gets back to having a proper minimal environment used by
sbuild to do a clean build. I have that (and it doesn't mount home,
using the 'sbuild' profile). I use it once things are working
reasonably well to get at least one clean build before uploading. This
bug is a problem in the 'less clean, but more useful' 'default' chroot
environment which is best for diagnostic work and builds of various
sorts where some file persistence (of user files) is needed.

Wookey
-- 
Principal hats:  Debian, Wookware, ARM
http://wookware.org/

[signature.asc (application/pgp-signature, inline)]

Message #38 received at 557730@bugs.debian.org (full text, mbox, reply):

From: Ian Jackson <ijackson@chiark.greenend.org.uk>

To: Wookey <wookey@wookware.org>

Cc: 557730@bugs.debian.org

Subject: Re: Bug#557730: /etc/{protocols,network,services} not schroot's to scribble over

Date: Tue, 14 Feb 2023 17:39:29 +0000

Wookey writes ("Re: Bug#557730: /etc/{protocols,network,services} not schroot's to scribble over"):
> On 2023-02-14 16:02 +0000, Ian Jackson wrote:
> > The fake netbase.deb could be contained within schroot.deb, in
> > /usr/share, so schroot wouldn't need to gain runtime build-deps on
> > dpkg tooling.
> 
> Except that it has to build this package live in order to contain the
> /etc/protocols and /etc/services files from the host
> environment. Having these default files with standard contents in a
> pre-built .deb is pointless, isn't it?

No, I thinko it's fine.  They mostly contain standardised constants.

schroot could build-depend on netbase and copy the files from there,
or just have fixed copies which would be updated manually once a
release or something.

These files, especially services, do change, but it is rare for
low-level basic things (or build systems) to actually depend on the
new entries.

> > > Whilst having the passwd database reflected in the chroot is
> > > incredibly useful it's not clear how often the services and protocols
> > > are needed, but I assume people do find that functionality useful.
> > 
> > I had a package that failed its build-time tests due to lack of
> > /etc/protocols.  The missing build-dep was detected in the buildds,
> > because my own local sid build chroot has netbase installed, precisely
> > because of this bug.
> 
> Right, which gets back to having a proper minimal environment used by
> sbuild to do a clean build. I have that (and it doesn't mount home,
> using the 'sbuild' profile). I use it once things are working
> reasonably well to get at least one clean build before uploading. This
> bug is a problem in the 'less clean, but more useful' 'default' chroot
> environment which is best for diagnostic work and builds of various
> sorts where some file persistence (of user files) is needed.

IME having a somewhat-dirty build environment is both convenient and
not a problem.

For example, my own sid chroot which *does* mount /home and import my
passwd and so on, reproducibily-builds the same .debs as the buildds
even for src:xen, whose build system is hardly straightforward.

Ian.

-- 
Ian Jackson <ijackson@chiark.greenend.org.uk>   These opinions are my own.  

Pronouns: they/he.  If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.

Send a report that this bug log contains spam.