Debian Bug report logs - #810018
procps: Please (re)consider shipping procps pidof

Reply or subscribe to this bug.

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Andreas Henriksson <andreas@fatal.se>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: procps: Please (re)consider shipping procps pidof

Date: Tue, 5 Jan 2016 17:27:51 +0100

[Message part 1 (text/plain, inline)]

Package: procps
Version: 2:3.3.11-3
Severity: wishlist
Tags: patch

Dear Maintainer,

I've seen that you're in the past considered and discussed shipping
the procps version of pidof at
https://lists.debian.org/debian-devel/2013/12/msg00145.html

I'm filing this bug because I think the previous reasons against
this no longer apply.

c.f.
"[...] moving pidof around certainly doesn't kill off sysvinit-utils for
us (there are still over a dozen other tools in that package in
Debian)."

The sysvinit-utils package has gone through a heavy diet and now
ships alot less tools (in favour of shipping them from other maintained
upstreams, notably util-linux).

I'm generally interested in shrinking the essential set, but users of
pidof seems to be too many to reasonably introduce dependencies for.

The currently remaining tools in sysvinit-utils (in stretch/sid):
pidof - suggestion to ship it from (new essential) procps(-base)
service - might soon be in init-system-helpers (see #805487).
killall5 - analysis of the 19 matches on codesearch seems to boil down
	to: openrc, util-vserver. Adding dependencies seems doable.
fstab-decode - codesearch analysis indicates open-iscsi, drbl.
	Adding dependencies seems doable.

Summary: making sysvinit-utils non-essential seems doable.

This would also make sysvinit-utils dependencies transitively
non-essential. eg. startpar. (Not sure why that dependency
exists to start with, but proposed approach seemed simpler
than researching that for the same result.)
Please also note that sysvinit-utils already have some
reverse dependencies despite being essential which works
in our favour.

I'm attaching a patch for your convenience. Please note that a
synchronized sysvinit upload is needed. If you think we should
proceed I'll volunteer to NMU sysvinit for this.
When doing so you'll need to update the attached patch
with the version number of the sysvinit upload and the Closes
bug number in the debian/changelog.
An announcement on debian-devel about introducing a new essential
package is also needed.

Regards,
Andreas Henriksson


-- System Information:
Debian Release: 8.2
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 3.16.0-4-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_DK.UTF-8, LC_CTYPE=en_DK.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages procps depends on:
ii  initscripts   2.88dsf-59.2
ii  libc6         2.19-18+deb8u1
ii  libncurses5   5.9+20140913-1+b1
ii  libncursesw5  5.9+20140913-1+b1
ii  libprocps3    2:3.3.9-9
ii  libtinfo5     5.9+20140913-1+b1
ii  lsb-base      4.1+Debian13+nmu1

Versions of packages procps recommends:
ii  psmisc  22.21-2

procps suggests no packages.

-- no debconf information

[0001-Add-an-essential-procps-base-package-containing-pido.patch (text/x-diff, attachment)]

Message #10 received at 810018@bugs.debian.org (full text, mbox, reply):

From: Craig Small <csmall@debian.org>

To: Andreas Henriksson <andreas@fatal.se>, 810018@bugs.debian.org

Subject: Re: Bug#810018: procps: Please (re)consider shipping procps pidof

Date: Wed, 6 Jan 2016 07:54:58 +1100

On Tue, Jan 05, 2016 at 05:27:51PM +0100, Andreas Henriksson wrote:
> The sysvinit-utils package has gone through a heavy diet and now
> ships alot less tools (in favour of shipping them from other maintained
> upstreams, notably util-linux).
I can re-visit making a procps-base with pidof enabled in it. Upstream
is actually maintained (by myself and a few others). It sounds like it
is the right time to be doing this. I think it needs a broader
discussion than just us two.

> The currently remaining tools in sysvinit-utils (in stretch/sid):
> pidof - suggestion to ship it from (new essential) procps(-base)
> service - might soon be in init-system-helpers (see #805487).
> killall5 - analysis of the 19 matches on codesearch seems to boil down
> 	to: openrc, util-vserver. Adding dependencies seems doable.
Is killall5 being actively maintained upstream?
If it is not then procps upstream (e.g. me) could recode this and
maintain it as part of procps upstream then eventually flow this
down to procps-base at a later date. It's basically pkill with different
options.

> An announcement on debian-devel about introducing a new essential
> package is also needed.
OK, then I'm happy for procps-base to exist with only pidof in the
package. I'd do it slightly differently to how you had it in the patch
but with largely the same result.

 - Craig
-- 
Craig Small (@smallsees)   http://enc.com.au/       csmall at : enc.com.au
Debian GNU/Linux           http://www.debian.org/   csmall at : debian.org
GPG fingerprint:        5D2F B320 B825 D939 04D2  0519 3938 F96B DF50 FEA5

Message #15 received at 810018@bugs.debian.org (full text, mbox, reply):

From: Andreas Henriksson <andreas@fatal.se>

To: Craig Small <csmall@debian.org>, 810018@bugs.debian.org

Subject: Re: Bug#810018: procps: Please (re)consider shipping procps pidof

Date: Wed, 6 Jan 2016 13:32:24 +0100

Hello Craig Small.

On Wed, Jan 06, 2016 at 07:54:58AM +1100, Craig Small wrote:
> On Tue, Jan 05, 2016 at 05:27:51PM +0100, Andreas Henriksson wrote:
> > The sysvinit-utils package has gone through a heavy diet and now
> > ships alot less tools (in favour of shipping them from other maintained
> > upstreams, notably util-linux).
> I can re-visit making a procps-base with pidof enabled in it.

Thanks for your quick reply and for considering.

> Upstream is actually maintained (by myself and a few others).

I'm aware (but wasn't aware that you had much help from others). :)

> It sounds like it is the right time to be doing this.

I think soo too, from multiple perspectives but most importantly the
Debian release schedule.

> I think it needs a broader discussion than just us two.

ACK. I'm personally not too keen on posting to debian-devel myself
though since just about anything seems to turn into a gigantic shit
throwing party. I'd be very happy if you'd be willing to take that on. :P
I'm happy to assist with technical matters and do grunt work, possibly
provide some background info where people have questions, but mostly
stay away from any wider debian discussions which just raises my
frustration levels too much.

> 
> > The currently remaining tools in sysvinit-utils (in stretch/sid):
> > pidof - suggestion to ship it from (new essential) procps(-base)
> > service - might soon be in init-system-helpers (see #805487).
> > killall5 - analysis of the 19 matches on codesearch seems to boil down
> > 	to: openrc, util-vserver. Adding dependencies seems doable.
> Is killall5 being actively maintained upstream?

In theory there's a sysvinit upstream, but in practise Debian maintenance
of sysvinit is dead (and I don't know the status of the supposed upstream
sysvinit project) so any upstream changes probably won't flow into
Debian anyway. I don't see killall5 as a blocker for this given
very few users (which if pkill provides the same functionality could
probably just be ported over to it now that I guess any distribution
can pretty much be assumed to have procps toolset... or alternatively
just have those users depend on sysvinit-utils directly).

> If it is not then procps upstream (e.g. me) could recode this and
> maintain it as part of procps upstream then eventually flow this
> down to procps-base at a later date. It's basically pkill with different
> options.

Feel free to try establish communications with the sysvinit upstream
to check if it's suitable for procps upstream to implement killall5,
but I think this is mostly off-topic for this bug report and should
be handled as a separate (upstream) discussion.

But as already mentioned, not sure it's worth the effort to provide
killall5 rather than just porting the few users over to pkill.

> 
> > An announcement on debian-devel about introducing a new essential
> > package is also needed.
> OK, then I'm happy for procps-base to exist with only pidof in the
> package. I'd do it slightly differently to how you had it in the patch
> but with largely the same result.

Feel free to implement it as you wish, but please keep the essential
package content/dependencies as small as possible. The patch was just
provided as convenience, example and proof-of-concept. I find it
sometimes easier to get the details discussed with a patch as example
given I'm not a native english speaker.

Regards,
Andreas Henriksson

Message #20 received at 810018@bugs.debian.org (full text, mbox, reply):

From: Craig Small <csmall@debian.org>

To: Andreas Henriksson <andreas@fatal.se>, 810018@bugs.debian.org

Subject: Re: Bug#810018: procps: Please (re)consider shipping procps pidof

Date: Thu, 7 Jan 2016 09:17:41 +1100

On Wed, Jan 06, 2016 at 01:32:24PM +0100, Andreas Henriksson wrote:
> On Wed, Jan 06, 2016 at 07:54:58AM +1100, Craig Small wrote:
> > Upstream is actually maintained (by myself and a few others).
> I'm aware (but wasn't aware that you had much help from others). :)
Mainly me and Jim for top. Jim usually sends me his patches.

> ACK. I'm personally not too keen on posting to debian-devel myself
> though since just about anything seems to turn into a gigantic shit
> throwing party. I'd be very happy if you'd be willing to take that on. :P
Oh you noticed that too? I'll send the email.

> I'm happy to assist with technical matters and do grunt work, possibly
> provide some background info where people have questions, but mostly
> stay away from any wider debian discussions which just raises my
> frustration levels too much.
I think it does that to everyone. Especially anything that has a sniff
of systemd about it (anything with init has that sniff, alas).

So, to the actual proposal. I need help especially around the why.

===================
What:
Create a new package procps-base. This uses the existing procps source
package and just enable building of pidof. procps-base will be an 
Essential package and only contain pidof.

Why:
The aim is to make the sysvinit-utils package non-essential. This
was discussed previously in 2013 [1], however there are some important
differences between 2013 and now.
 1) The previous change was due to possible upstream relocations of
 pidof. This is about Debian package contents.
 2) In 2013 there were a lot of other programs in sysvinit, in 2016
 we're down to 4 (pidof, service, fstab-decode, killall5)

(why change from one essential to the other?)

What about the other binaries in sysvinit-utils?
 pidof - moves to new Essential package procps-base
 service - moving to init-system-helpers [2]
 fstab-decode - 2 packages use this (open-iscsi, drbl)
 killall5 -  2 packages use this (openrc, util-vserver)
The idea would be those 4 packages would depend on sysvinit-utils.
pidof is used be far too many packages to have the same treatment.

What other packages will procps-base depend on?
libc6 and libprocps5. libprocps would be newly pulled in.


1: https://lists.debian.org/debian-devel/2013/12/msg00121.html
2: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=805487

===================
 - Craig
-- 
Craig Small (@smallsees)   http://enc.com.au/       csmall at : enc.com.au
Debian GNU/Linux           http://www.debian.org/   csmall at : debian.org
GPG fingerprint:        5D2F B320 B825 D939 04D2  0519 3938 F96B DF50 FEA5

Message #25 received at 810018@bugs.debian.org (full text, mbox, reply):

From: Andreas Henriksson <andreas@fatal.se>

To: Craig Small <csmall@debian.org>, 810018@bugs.debian.org

Subject: Re: Bug#810018: procps: Please (re)consider shipping procps pidof

Date: Thu, 7 Jan 2016 12:53:12 +0100

Hello Craig Small.

Thanks for taking this on and drafting a proposal... I'll throw
in some of my own ideas below in case they're helpful so feel
free to leave out or reword things as you see fit.

Fwiw, in case you want to avoid any potential people out there
who might think you're just beating a dead horse for your own
personal gain feel free to start out with "I've been asked to..."
which should hopefully kill off those ideas from the get-go.

On Thu, Jan 07, 2016 at 09:17:41AM +1100, Craig Small wrote:
[...]
> So, to the actual proposal. I need help especially around the why.
> 
> ===================
> What:
> Create a new package procps-base. This uses the existing procps source
> package and just enable building of pidof. procps-base will be an 
> Essential package and only contain pidof.

Short and clear. +1

> 
> Why:

I see what you wrote mainly as a long-term why.... I'd like to add
a short-term:

Make sure more of the essential utilities in the base of the operating
system comes from active and healthy upstreams.

Also in general I think it's good to make sure we don't /needlessly/
deviate. This means we can benefit from issues discovered in other
distributions and fixed upstream, while at the same time we can be good
free software citizens and bring our own fixes upstream for the benefit
of the entire free software echosystem.
Hopefully pidof is trivial enough that it won't need much fixes though.

> The aim is to make the sysvinit-utils package non-essential. This
> was discussed previously in 2013 [1], however there are some important
> differences between 2013 and now.
>  1) The previous change was due to possible upstream relocations of
>  pidof. This is about Debian package contents.
>  2) In 2013 there were a lot of other programs in sysvinit, in 2016
>  we're down to 4 (pidof, service, fstab-decode, killall5)
> 
> (why change from one essential to the other?)

Why? See above. Otherwise I think this part is good as is. Some minor
ideas:
 - maybe say "longer term plans/ideas..." since I'm not vouching we'll
   be able to actually pull off makintg sysvinit-utils non-essential
   before stretch release.
 - maybe tone down making sysvinit-utils non-essential since I think
   that should be a separate discussion (and more considerations than
   just pidof and service, etc, needs to be taken into consideration).
   Providing procps pidof is a useful idea on its own in my opinion.

> 
> What about the other binaries in sysvinit-utils?
>  pidof - moves to new Essential package procps-base
>  service - moving to init-system-helpers [2]
>  fstab-decode - 2 packages use this (open-iscsi, drbl)
>  killall5 -  2 packages use this (openrc, util-vserver)
> The idea would be those 4 packages would depend on sysvinit-utils.
> pidof is used be far too many packages to have the same treatment.

I've also spent some time on researching the pidof users and the main
user seemed to be direct invokations from init scripts. These could
benefit from being converted to LSB "pidofproc" helper function (which
falls back on executing pidof if available though). I'm not sure the
busy work of fixing all init scripts is worth it though. Personally I'd
rather spend my time on writing native systemd unit files (which will
probably be easier to reuse from other potential future init systems
than LSB init scripts), and I doubt the vocal people who shys away from
anything systemd related is willing to actually step up and help out
with the practical work either. So not sure if it's worth even
mentioning LSB pidofproc... maybe in a short paragraph somewhere.

> 
> What other packages will procps-base depend on?
> libc6 and libprocps5. libprocps would be newly pulled in.

Would likely be good to mention what sysvinit-utils pulls in
for comparison, eg.

In comparison sysvinit-utils pulls in:
libc6, libselinux1, startpar

(Maybe skip mentioning libc6 in both cases.)

> 
> 
> 1: https://lists.debian.org/debian-devel/2013/12/msg00121.html
> 2: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=805487
> 
> ===================
>  - Craig

HTH.

Regards,
Andreas Henriksson

Message #30 received at 810018@bugs.debian.org (full text, mbox, reply):

From: Andreas Henriksson <andreas@fatal.se>

To: Craig Small <csmall@debian.org>, 810018@bugs.debian.org

Subject: Re: Bug#810018: procps: Please (re)consider shipping procps pidof

Date: Sat, 9 Jan 2016 03:27:25 +0100

[Message part 1 (text/plain, inline)]

Hello Craig.

On Thu, Jan 07, 2016 at 09:17:41AM +1100, Craig Small wrote:
[...]
> What other packages will procps-base depend on?
> libc6 and libprocps5. libprocps would be newly pulled in.
[...]

I was thinking that maybe it might be a good idea to possibly avoid this
dependency. That would allow us to keep essential out of any upcoming
libprocps transition, keep essential minimal, etc....

The idea is simply to build a static version of pidof and strip out the
unused parts of libprocps.a, made possible by building the lib with
-ffunction-section -fdata-sections and linking with -Wl,--gc-sections.
The attached patch does this. (Then just install pidof.static instead of
pidof in the base package and discard pidof.)

The end result is 34896 bytes for pidof.static (ie. approx 20k growth
to avoid the libprocps5 dependency).
(While also growing the libprocps.a for the extra sections, but
I don't see how that would affect anything.)

Things would be even better if we could somehow also avoid libsystemd0
dependency. It's already a dependency of essential packages (via atleast
util-linux) but just for the sake of not having to discuss it...
Not sure if it's worth implementing a double build-pass (with/without
libsystemd enabled) just for pidof though. Maybe there's a simpler way
or maybe we just leave it as is....

(Fwiw, the upstream util-linux build system supports building static
versions for basically all utilities which has proven handy in a couple
of occations. You might want to have a look there if you're interested
in an example of a tidier build system example.)

Do you have any thoughts on this? HTH.

Regards,
Andreas Henriksson

[pidofstatic.patch (text/x-diff, attachment)]

Message #35 received at 810018@bugs.debian.org (full text, mbox, reply):

From: Andreas Henriksson <andreas@fatal.se>

To: 810018@bugs.debian.org

Subject: Additional info possibly relevant for procps-base/pidof ...

Date: Tue, 19 Jan 2016 17:13:29 +0100

Hello Craig Small.

Here's a small status update which might be relevant to consider
for this bug report regarding procps-base / pidof.

The policy-related service management tools moved into into
init-system-helpers yesterdays uploads of sysvinit[1] and
init-system-helpers[2]. This means service is no longer part of
sysvinit-utils and pidof is the only remaining widely used *binary* in
that package.

I also noticed there's a non-binary which also needs to
be considered if sysvinit-utils is to be made non-essential:
-> /lib/init/init-d-script
This script is sourced from 10 packages according to
codesearch.debian.net and random samplings says atleast some of them are
unconditional. Introducing dependencies is still doable in my view.

Regards,
Andreas Henriksson

[1]: https://tracker.debian.org/news/741657
[2]: https://tracker.debian.org/news/741508

Message #40 received at 810018@bugs.debian.org (full text, mbox, reply):

From: Craig Small <csmall@enc.com.au>

To: Andreas Henriksson <andreas@fatal.se>, 810018@bugs.debian.org

Subject: Re: Bug#810018: Additional info possibly relevant for procps-base/pidof ...

Date: Sun, 17 Apr 2016 07:00:09 +0000

[Message part 1 (text/plain, inline)]

Hi Andreas,
  One thing before we move to the procps pid, it doesn't match all
processes the pidof one does [1]. I have fixed it upstream but probably
worth waiting for that to appear otherwise things like kde and sshd
sessions "disappear".

 - Craig

1: https://gitlab.com/procps-ng/procps/issues/4

On Wed, Jan 20, 2016 at 3:15 AM Andreas Henriksson <andreas@fatal.se> wrote:

> Hello Craig Small.
>
> Here's a small status update which might be relevant to consider
> for this bug report regarding procps-base / pidof.
>
> The policy-related service management tools moved into into
> init-system-helpers yesterdays uploads of sysvinit[1] and
> init-system-helpers[2]. This means service is no longer part of
> sysvinit-utils and pidof is the only remaining widely used *binary* in
> that package.
>
> I also noticed there's a non-binary which also needs to
> be considered if sysvinit-utils is to be made non-essential:
> -> /lib/init/init-d-script
> This script is sourced from 10 packages according to
> codesearch.debian.net and random samplings says atleast some of them are
> unconditional. Introducing dependencies is still doable in my view.
>
> Regards,
> Andreas Henriksson
>
> [1]: https://tracker.debian.org/news/741657
> [2]: https://tracker.debian.org/news/741508
>
-- 
Craig Small (@smallsees)   http://enc.com.au/       csmall at : enc.com.au
Debian GNU/Linux           http://www.debian.org/   csmall at : debian.org
GPG fingerprint:        5D2F B320 B825 D939 04D2  0519 3938 F96B DF50 FEA5

[Message part 2 (text/html, inline)]

Message #45 received at 810018@bugs.debian.org (full text, mbox, reply):

From: Andreas Henriksson <andreas@fatal.se>

To: 810018@bugs.debian.org

Subject: procps-base: some additional init-d-script info in the quest...

Date: Sat, 21 May 2016 19:18:29 +0200

Hello!

This information is not directly related to procps, but it is
kind of related to the reason why we'd want to introduce procps-base
as there's no big point in doing it if we can't also make sysvinit-utils
non-essential. Also we don't really have a good tracking bug so posting
here to archive the information somewhere before I forget about it....

I've looked closer at the users of /lib/init/init-d-script which was
mentioned in a previous post that some packages init script unconditionally
sources. Apparently "new school" init scripts as generated by current
dh-make and as visible in /etc/init.d/skeleton is the reason for these
unconditional sourcing of the file which causes problems if we downgrade
sysvinit-utils (which ships /lib/init/init-d-script) to non-essential.

Given this style of init script is apparently the currently recommended
form, I assume we can see the number of packages using it rise over
time...
The problem should be (partially?) less problematic if the package
using "new school" init script also ships a native systemd unit
file, which should avoid the init script being executed at all...
(Though some people still manually execute init scripts even under
systemd out of old habits and this only works because systemd
ships an LSB hook that catches this and issues systemctl seamlessly.)



a) packages shipping an init script that uses /lib/init/init-d-script according to codesearch.d.n
b) if it's not a big practical issue, because the init script will not be used under systemd.

pkg 				has masking systemd unit file
========================================================================
sddm				yes
coquelicot			yes
open-iscsi			yes
graphite-api			yes
apache2				no! (only partial snippet to extend autoconverted script)
batmand				yes
netplan				no! (but carries own fallback copy of init-d-script!)
courier-authdaemon		yes
uwsgi-emperor			no!
opendnssec-signer		yes
kxd				yes
lvm2				yes
waagent				yes
prometheus			yes
dbab				yes (also carries own fallback copy of init-d-script!)
prometheus-pgbouncer-exporter	yes
shairport-sync			yes
freeipmi-ipmiseld		no!
knot-resolver			yes
prometheus-mysqld-exporter	yes
hhvm				yes
prometheus-pushgateway		no!
courier-mta			yes
prometheus-node-exporter	no!


The investigation was done using (so might very well be incomplete):

for pkg in .... ; do
	apt-file show $pkg | grep -e 'init.d' -e 'systemd/system'
done

TODO: Investigate what happens if executing such an init script directly
under systemd where normally systemd-installed LSB hooks would override
it and use systemctl...


Regards,
Andreas Henriksson

Send a report that this bug log contains spam.