Debian Bug report logs - #974705
New dash package breaks systemds timedated test (hangs in wait)

version graph

Package: dash; Maintainer for dash is Andrej Shadura <andrewsh@debian.org>; Source for dash is src:dash (PTS, buildd, popcon).

Affects: src:fence-agents, src:systemd

Reported by: Michael Biebl <biebl@debian.org>

Date: Sat, 14 Nov 2020 01:39:01 UTC

Severity: serious

Tags: upstream

Merged with 975010

Found in version dash/0.5.11+git20200708+dd9ef66-2

Fixed in version dash/0.5.11+git20200708+dd9ef66-3

Done: Andrej Shadura <andrewsh@debian.org>

Bug is archived. No further changes may be made.

Forwarded to https://www.mail-archive.com/dash@vger.kernel.org/msg01975.html

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, Andrej Shadura <andrewsh@debian.org>:
Bug#974705; Package dash. (Sat, 14 Nov 2020 01:39:03 GMT) (full text, mbox, link).


Acknowledgement sent to Michael Biebl <biebl@debian.org>:
New Bug report received and forwarded. Copy sent to Andrej Shadura <andrewsh@debian.org>. (Sat, 14 Nov 2020 01:39:03 GMT) (full text, mbox, link).


Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Michael Biebl <biebl@debian.org>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: New dash package breaks systemds timedated test (hangs in wait)
Date: Sat, 14 Nov 2020 02:37:31 +0100
Package: dash
Version: 0.5.11+git20200708+dd9ef66-2
Severity: serious

Hi,

the latest update of dash seems to have broken the systemd autopkgtest
suite, specifically
https://salsa.debian.org/systemd-team/systemd/-/blob/debian/master/debian/tests/timedated

Afaics, the sh script get's stuck at the "wait" call at
https://salsa.debian.org/systemd-team/systemd/-/blob/debian/master/debian/tests/timedated#L166

Downgrading dash to 0.5.10.2-7 makes the test pass again.

Thus filing with RC to hopefully prevent testing migration until the
root cause is found.

Regards,
Michael



-- System Information:
Debian Release: bullseye/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (200, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 5.9.0-2-amd64 (SMP w/4 CPU threads)
Kernel taint flags: TAINT_FIRMWARE_WORKAROUND
Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages dash depends on:
ii  debconf [debconf-2.0]  1.5.74
ii  debianutils            4.11.2
ii  dpkg                   1.20.5
ii  libc6                  2.31-4

dash recommends no packages.

dash suggests no packages.

-- debconf information excluded



Information forwarded to debian-bugs-dist@lists.debian.org, Andrej Shadura <andrewsh@debian.org>:
Bug#974705; Package dash. (Sat, 14 Nov 2020 14:27:03 GMT) (full text, mbox, link).


Acknowledgement sent to Michael Biebl <biebl@debian.org>:
Extra info received and forwarded to list. Copy sent to Andrej Shadura <andrewsh@debian.org>. (Sat, 14 Nov 2020 14:27:03 GMT) (full text, mbox, link).


Message #10 received at 974705@bugs.debian.org (full text, mbox, reply):

From: Michael Biebl <biebl@debian.org>
To: 974705@bugs.debian.org
Subject: Re: New dash package breaks systemds timedated test (hangs in wait)
Date: Sat, 14 Nov 2020 15:25:53 +0100
[Message part 1 (text/plain, inline)]
On Sat, 14 Nov 2020 02:37:31 +0100 Michael Biebl <biebl@debian.org>
wrote:

> Afaics, the sh script get's stuck at the "wait" call at
> 
https://salsa.debian.org/systemd-team/systemd/-/blob/debian/master/debian/tests/timedated#L166
> 
> Downgrading dash to 0.5.10.2-7 makes the test pass again.

Using /bin/bash is also sufficient, to make this test pass.
 
> Thus filing with RC to hopefully prevent testing migration until the
> root cause is found.

It seems I was not quick enough for that.

[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Andrej Shadura <andrewsh@debian.org>:
Bug#974705; Package dash. (Sat, 14 Nov 2020 17:33:02 GMT) (full text, mbox, link).


Acknowledgement sent to Valentin Vidic <vvidic@debian.org>:
Extra info received and forwarded to list. Copy sent to Andrej Shadura <andrewsh@debian.org>. (Sat, 14 Nov 2020 17:33:02 GMT) (full text, mbox, link).


Message #15 received at 974705@bugs.debian.org (full text, mbox, reply):

From: Valentin Vidic <vvidic@debian.org>
To: 974705@bugs.debian.org
Subject: fence-agents test hangs
Date: Sat, 14 Nov 2020 18:24:28 +0100
One of autopkgtests in fence-agents package seems to be broken
in the same way - just hangs in wait forever and using bash works:

https://salsa.debian.org/ha-team/fence-agents/-/blob/master/debian/tests/delay

-- 
Valentin



Information forwarded to debian-bugs-dist@lists.debian.org, Andrej Shadura <andrewsh@debian.org>:
Bug#974705; Package dash. (Sun, 15 Nov 2020 11:54:06 GMT) (full text, mbox, link).


Acknowledgement sent to Michael Biebl <biebl@debian.org>:
Extra info received and forwarded to list. Copy sent to Andrej Shadura <andrewsh@debian.org>. (Sun, 15 Nov 2020 11:54:06 GMT) (full text, mbox, link).


Message #20 received at 974705@bugs.debian.org (full text, mbox, reply):

From: Michael Biebl <biebl@debian.org>
To: 974705@bugs.debian.org
Subject: Re: Bug#974705: Acknowledgement (New dash package breaks systemds timedated test (hangs in wait))
Date: Sun, 15 Nov 2020 12:51:01 +0100
[Message part 1 (text/plain, inline)]
Hi Andrew,

I ran git bisect which found the following as the first faulty commit:

6c691b3e5099de380a9d2f97d5c72f4a978bb794 is the first bad commit
commit 6c691b3e5099de380a9d2f97d5c72f4a978bb794
Author: Herbert Xu <herbert@gondor.apana.org.au>
Date:   Sat May 19 02:39:41 2018 +0800

    jobs: Only clear gotsigchld when waiting for everything
    
    The gotsigchld flag is always cleared in dowait but not all callers
    of dowait will wait for everything.  In particular, when jp is set
    we only wait until the set job isn't running anymore.
    
    This patch fixes this by only clearing gotsigchld if jp is unset.
    It also changes the waitcmd to actually set jp which corresponds
    to the behaviour of bash/ksh93/mksh.
    
    The only other caller of dowait that doesn't wait for everything
    is the jobless reaper.  This is in fact redundant now that we wait
    after every simple command.  This patch removes it.
    
    Finally as every caller of dowait needs to wait until either the
    given job is not running, or until all terminated jobs have been
    processed, this patch moves the loop into dowait itself.
    
    Fixes: 03876c0743a5 ("eval: Reap zombies after built-in...")
    Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>


https://git.kernel.org/pub/scm/utils/dash/dash.git/commit/?id=6c691b3e5099de380a9d2f97d5c72f4a978bb794

As you asked on IRC, to reproduce the problem I used autopkgtest with
LXC like this (on sid)

# autopkgtest-build-lxc debian sid
# autopkgtest --test-name=timedated systemd -- lxc -s autopkgtest-sid

If you want to edit the timedated script, you can also use

# apt-get source systemd
# vi  systemd-246.6/debian/tests/timedated
# autopkgtest --test-name=timedated systemd-246.6/ -B -- lxc -s
autopkgtest-sid


Regards,
Michael

[signature.asc (application/pgp-signature, inline)]

Added indication that 974705 affects src:systemd Request was from Michael Biebl <biebl@debian.org> to control@bugs.debian.org. (Sun, 15 Nov 2020 12:33:02 GMT) (full text, mbox, link).


Added indication that 974705 affects src:fence-agents Request was from Michael Biebl <biebl@debian.org> to control@bugs.debian.org. (Sun, 15 Nov 2020 12:33:02 GMT) (full text, mbox, link).


Added tag(s) fixed-upstream and upstream. Request was from Balint Reczey <balint.reczey@canonical.com> to control@bugs.debian.org. (Mon, 16 Nov 2020 15:15:03 GMT) (full text, mbox, link).


Set Bug forwarded-to-address to 'https://www.mail-archive.com/dash@vger.kernel.org/msg01950.html'. Request was from Balint Reczey <balint.reczey@canonical.com> to control@bugs.debian.org. (Mon, 16 Nov 2020 15:15:04 GMT) (full text, mbox, link).


Changed Bug forwarded-to-address to 'https://www.mail-archive.com/dash@vger.kernel.org/msg01975.html' from 'https://www.mail-archive.com/dash@vger.kernel.org/msg01950.html'. Request was from Andrej Shadura <andrewsh@debian.org> to control@bugs.debian.org. (Mon, 16 Nov 2020 15:27:02 GMT) (full text, mbox, link).


Removed tag(s) fixed-upstream. Request was from Andrej Shadura <andrewsh@debian.org> to control@bugs.debian.org. (Mon, 16 Nov 2020 15:27:03 GMT) (full text, mbox, link).


Merged 974705 975010 Request was from Michael Biebl <biebl@debian.org> to control@bugs.debian.org. (Tue, 17 Nov 2020 22:36:08 GMT) (full text, mbox, link).


Information forwarded to debian-bugs-dist@lists.debian.org, Andrej Shadura <andrewsh@debian.org>:
Bug#974705; Package dash. (Tue, 01 Dec 2020 06:09:03 GMT) (full text, mbox, link).


Acknowledgement sent to Herbert Xu <herbert@gondor.apana.org.au>:
Extra info received and forwarded to list. Copy sent to Andrej Shadura <andrewsh@debian.org>. (Tue, 01 Dec 2020 06:09:03 GMT) (full text, mbox, link).


Message #39 received at 974705@bugs.debian.org (full text, mbox, reply):

From: Herbert Xu <herbert@gondor.apana.org.au>
To: Andrej Shadura <andrew.shadura@collabora.co.uk>
Cc: dash@vger.kernel.org, Michael Biebl <biebl@debian.org>, 974705@bugs.debian.org
Subject: Re: Changes to job handling cause hangs in wait
Date: Tue, 1 Dec 2020 17:06:18 +1100
On Tue, Dec 01, 2020 at 04:42:03PM +1100, Herbert Xu wrote:
>
> Nevermind, I see that the script has been modified to use bash.
> 
> I can reproduce the problem now so it's all good.

OK the problem is this:

	sh -c 'sleep 1d& exec $MYSHELL -c "sleep 1& wait"'

You can replace MYSHELL with whatever shell you want to use.

Essentially dash will now wait for all children, even ones that
were created prior to its existence, however, bash only waits for
children that it created directly.

FWIW ksh exhibits the same behaviour as dash and I think there
is nothing wrong with this.

So the problem is really in the parent of this shell, which appears
to be bash:

bash -c set -e; export USER=`id -nu`; . /etc/profile >/dev/null 2>&1 || true;  . ~/.profile >/dev/null 2>&1 || true; buildtree="/tmp/autopkgtest-lxc.is4n6xxr/downtmp/build.f2G/real-tree"; mkdir -p -m 1777 -- "/tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-artifacts"; export AUTOPKGTEST_ARTIFACTS="/tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-artifacts"; export ADT_ARTIFACTS="$AUTOPKGTEST_ARTIFACTS"; mkdir -p -m 755 "/tmp/autopkgtest-lxc.is4n6xxr/downtmp/autopkgtest_tmp"; export AUTOPKGTEST_TMP="/tmp/autopkgtest-lxc.is4n6xxr/downtmp/autopkgtest_tmp"; export ADTTMP="$AUTOPKGTEST_TMP"; export DEBIAN_FRONTEND=noninteractive; export LANG=C.UTF-8; export DEB_BUILD_OPTIONS=parallel=2; unset LANGUAGE LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE   LC_MONETARY LC_MESSAGES LC_PAPER LC_NAME LC_ADDRESS   LC_TELEPHONE LC_MEASUREMENT LC_IDENTIFICATION LC_ALL;rm -f /tmp/autopkgtest_script_pid; set -C; echo $$ > /tmp/autopkgtest_script_pid; set +C; trap "rm -f /tmp/autopkgtest_script_pid" EXIT INT QUIT PIPE; cd "$buildtree"; export AUTOPKGTEST_NORMAL_USER=; export ADT_NORMAL_USER=; chmod +x /tmp/autopkgtest-lxc.is4n6xxr/downtmp/build.f2G/real-tree/debian/tests/timedated; touch /tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-stdout /tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-stderr; /tmp/autopkgtest-lxc.is4n6xxr/downtmp/build.f2G/real-tree/debian/tests/timedated 2> >(tee -a /tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-stderr >&2) > >(tee -a /tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-stdout);

For some reason this is causing the final two tee's to be created
as children of debian/tests/timedated rather than the bash shell.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt



Information forwarded to debian-bugs-dist@lists.debian.org, Andrej Shadura <andrewsh@debian.org>:
Bug#974705; Package dash. (Tue, 01 Dec 2020 06:18:03 GMT) (full text, mbox, link).


Acknowledgement sent to Herbert Xu <herbert@gondor.apana.org.au>:
Extra info received and forwarded to list. Copy sent to Andrej Shadura <andrewsh@debian.org>. (Tue, 01 Dec 2020 06:18:03 GMT) (full text, mbox, link).


Message #44 received at 974705@bugs.debian.org (full text, mbox, reply):

From: Herbert Xu <herbert@gondor.apana.org.au>
To: Andrej Shadura <andrew.shadura@collabora.co.uk>
Cc: dash@vger.kernel.org, Michael Biebl <biebl@debian.org>, 974705@bugs.debian.org
Subject: Re: Changes to job handling cause hangs in wait
Date: Tue, 1 Dec 2020 16:38:37 +1100
On Mon, Nov 16, 2020 at 10:29:16AM +0000, Andrej Shadura wrote:
> 
> I’d like to forward a bug report I received. Michael, who reported it,
> was able to bisect it to 6c691b3, which was the first of the series of
> commits changing job handling.
> 
> I must note that I had troubles debugging it: I was only able to
> reproduce the issue in an autopkgtest virtual machine, but not on my
> normal system nor in a fresh Debian unstable Docker container.

FWIW I'm unable to reproduce it with autopkgtest.  This is what
I get:

root@test0:~# autopkgtest --test-name=timedated systemd-246.6/ -B -- lxc -s autopkgtest-sid
autopkgtest [16:32:45]: starting date: 2020-12-01
autopkgtest [16:32:45]: version 5.15
autopkgtest [16:32:45]: host test0; command line: /usr/bin/autopkgtest --test-name=timedated systemd-246.6/ -B -- lxc -s autopkgtest-sid
autopkgtest [16:33:13]: testbed dpkg architecture: amd64
autopkgtest [16:33:13]: testbed running kernel: Linux 5.9.0-rc1+ #18 SMP PREEMPT Sat Aug 29 23:45:30 AEST 2020
autopkgtest [16:33:13]: @@@@@@@@@@@@@@@@@@@@ unbuilt-tree systemd-246.6/
autopkgtest [16:33:18]: testing package systemd version 246.6-5
autopkgtest [16:33:18]: build not needed
autopkgtest [16:33:18]: test timedated: preparing testbed
Reading package lists... Done
Building dependency tree
Reading state information... Done
Correcting dependencies...Starting pkgProblemResolver with broken count: 0
Starting 2 pkgProblemResolver with broken count: 0
Done
 Done
Starting pkgProblemResolver with broken count: 0
Starting 2 pkgProblemResolver with broken count: 0
Done
The following additional packages will be installed:
  acl libnss-systemd
The following NEW packages will be installed:
  acl libnss-systemd
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
1 not fully installed or removed.
Need to get 255 kB of archives.
After this operation, 577 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian sid/main amd64 libnss-systemd amd64 246.6-5 [194 kB]
Get:2 http://deb.debian.org/debian sid/main amd64 acl amd64 2.2.53-8 [60.8 kB]
Fetched 255 kB in 1s (489 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libnss-systemd:amd64.
(Reading database ... 12641 files and directories currently installed.)
Preparing to unpack .../libnss-systemd_246.6-5_amd64.deb ...
Unpacking libnss-systemd:amd64 (246.6-5) ...
Selecting previously unselected package acl.
Preparing to unpack .../acl_2.2.53-8_amd64.deb ...
Unpacking acl (2.2.53-8) ...
Setting up libnss-systemd:amd64 (246.6-5) ...
First installation detected...
Checking NSS setup...
Setting up acl (2.2.53-8) ...
Setting up autopkgtest-satdep (0) ...
Processing triggers for libc-bin (2.31-4) ...
(Reading database ... 12667 files and directories currently installed.)
Removing autopkgtest-satdep (0) ...
autopkgtest [16:33:29]: test timedated: [-----------------------
original tz: Etc/UTC
timedatectl works
change timezone
reset timezone to original
no adjtime file
UTC set in adjtime file
non-zero values in adjtime file
fourth line adjtime file
no final newline in adjtime file
only one line in adjtime file
only one line in adjtime file, no final newline
only two lines in adjtime file
only two lines in adjtime file, no final newline
unknown value in 3rd line of adjtime file
disable NTP
enable NTP
re-disable NTP
autopkgtest [16:33:32]: test timedated: -----------------------]
autopkgtest [16:33:32]: test timedated:  - - - - - - - - - - results - - - - - - - - - -
timedated            PASS
autopkgtest [16:33:32]: @@@@@@@@@@@@@@@@@@@@ summary
timedated            PASS
root@test0:~# 

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt



Information forwarded to debian-bugs-dist@lists.debian.org, Andrej Shadura <andrewsh@debian.org>:
Bug#974705; Package dash. (Tue, 01 Dec 2020 06:18:04 GMT) (full text, mbox, link).


Acknowledgement sent to Herbert Xu <herbert@gondor.apana.org.au>:
Extra info received and forwarded to list. Copy sent to Andrej Shadura <andrewsh@debian.org>. (Tue, 01 Dec 2020 06:18:04 GMT) (full text, mbox, link).


Message #49 received at 974705@bugs.debian.org (full text, mbox, reply):

From: Herbert Xu <herbert@gondor.apana.org.au>
To: Andrej Shadura <andrew.shadura@collabora.co.uk>
Cc: dash@vger.kernel.org, Michael Biebl <biebl@debian.org>, 974705@bugs.debian.org
Subject: Re: Changes to job handling cause hangs in wait
Date: Tue, 1 Dec 2020 16:42:03 +1100
On Tue, Dec 01, 2020 at 04:38:37PM +1100, Herbert Xu wrote:
> 
> FWIW I'm unable to reproduce it with autopkgtest.  This is what
> I get:
> 
> root@test0:~# autopkgtest --test-name=timedated systemd-246.6/ -B -- lxc -s autopkgtest-sid
> autopkgtest [16:32:45]: starting date: 2020-12-01
> autopkgtest [16:32:45]: version 5.15
> autopkgtest [16:32:45]: host test0; command line: /usr/bin/autopkgtest --test-name=timedated systemd-246.6/ -B -- lxc -s autopkgtest-sid
> autopkgtest [16:33:13]: testbed dpkg architecture: amd64
> autopkgtest [16:33:13]: testbed running kernel: Linux 5.9.0-rc1+ #18 SMP PREEMPT Sat Aug 29 23:45:30 AEST 2020
> autopkgtest [16:33:13]: @@@@@@@@@@@@@@@@@@@@ unbuilt-tree systemd-246.6/
> autopkgtest [16:33:18]: testing package systemd version 246.6-5

Nevermind, I see that the script has been modified to use bash.

I can reproduce the problem now so it's all good.

Thanks,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt



Information forwarded to debian-bugs-dist@lists.debian.org, Andrej Shadura <andrewsh@debian.org>:
Bug#974705; Package dash. (Tue, 01 Dec 2020 07:00:02 GMT) (full text, mbox, link).


Acknowledgement sent to Herbert Xu <herbert@gondor.apana.org.au>:
Extra info received and forwarded to list. Copy sent to Andrej Shadura <andrewsh@debian.org>. (Tue, 01 Dec 2020 07:00:03 GMT) (full text, mbox, link).


Message #54 received at 974705@bugs.debian.org (full text, mbox, reply):

From: Herbert Xu <herbert@gondor.apana.org.au>
To: Andrej Shadura <andrew.shadura@collabora.co.uk>
Cc: dash@vger.kernel.org, Michael Biebl <biebl@debian.org>, 974705@bugs.debian.org
Subject: Re: Changes to job handling cause hangs in wait
Date: Tue, 1 Dec 2020 17:56:23 +1100
On Tue, Dec 01, 2020 at 05:06:18PM +1100, Herbert Xu wrote:
>
> For some reason this is causing the final two tee's to be created
> as children of debian/tests/timedated rather than the bash shell.

An alternative to changing the parent is of course to do

	wait $MONPID

instead of

	wait

I think this makes more sense anyway as otherwise someone could
easily introduce a hang if they unwittingly add a backgrounded
job to this script.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt



Information forwarded to debian-bugs-dist@lists.debian.org, Andrej Shadura <andrewsh@debian.org>:
Bug#974705; Package dash. (Tue, 01 Dec 2020 10:27:02 GMT) (full text, mbox, link).


Acknowledgement sent to Harald van Dijk <harald@gigawatt.nl>:
Extra info received and forwarded to list. Copy sent to Andrej Shadura <andrewsh@debian.org>. (Tue, 01 Dec 2020 10:27:02 GMT) (full text, mbox, link).


Message #59 received at 974705@bugs.debian.org (full text, mbox, reply):

From: Harald van Dijk <harald@gigawatt.nl>
To: Herbert Xu <herbert@gondor.apana.org.au>, Andrej Shadura <andrew.shadura@collabora.co.uk>
Cc: dash@vger.kernel.org, Michael Biebl <biebl@debian.org>, 974705@bugs.debian.org
Subject: Re: Changes to job handling cause hangs in wait
Date: Tue, 1 Dec 2020 10:14:04 +0000
On 01/12/2020 06:06, Herbert Xu wrote:
> On Tue, Dec 01, 2020 at 04:42:03PM +1100, Herbert Xu wrote:
>>
>> Nevermind, I see that the script has been modified to use bash.
>>
>> I can reproduce the problem now so it's all good.
> 
> OK the problem is this:
> 
> 	sh -c 'sleep 1d& exec $MYSHELL -c "sleep 1& wait"'
> 
> You can replace MYSHELL with whatever shell you want to use.
> 
> Essentially dash will now wait for all children, even ones that
> were created prior to its existence, however, bash only waits for
> children that it created directly.
> 
> FWIW ksh exhibits the same behaviour as dash and I think there
> is nothing wrong with this.

POSIX says:

  "If  the  wait utility is invoked with no operands, it shall wait 
until all process IDs known to the invoking shell have terminated and 
exit with a zero exit status."

I would say that child processes that were created before dash was 
started do not have process IDs known to dash.

> So the problem is really in the parent of this shell, which appears
> to be bash:
> 
> bash -c set -e; export USER=`id -nu`; . /etc/profile >/dev/null 2>&1 || true;  . ~/.profile >/dev/null 2>&1 || true; buildtree="/tmp/autopkgtest-lxc.is4n6xxr/downtmp/build.f2G/real-tree"; mkdir -p -m 1777 -- "/tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-artifacts"; export AUTOPKGTEST_ARTIFACTS="/tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-artifacts"; export ADT_ARTIFACTS="$AUTOPKGTEST_ARTIFACTS"; mkdir -p -m 755 "/tmp/autopkgtest-lxc.is4n6xxr/downtmp/autopkgtest_tmp"; export AUTOPKGTEST_TMP="/tmp/autopkgtest-lxc.is4n6xxr/downtmp/autopkgtest_tmp"; export ADTTMP="$AUTOPKGTEST_TMP"; export DEBIAN_FRONTEND=noninteractive; export LANG=C.UTF-8; export DEB_BUILD_OPTIONS=parallel=2; unset LANGUAGE LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE   LC_MONETARY LC_MESSAGES LC_PAPER LC_NAME LC_ADDRESS   LC_TELEPHONE LC_MEASUREMENT LC_IDENTIFICATION LC_ALL;rm -f /tmp/autopkgtest_script_pid; set -C; echo $$ > /tmp/autopkgtest_script_pid; set +C; trap "rm -f /tmp/autopkgtest_script_pid" EXIT INT QUIT PIPE; cd "$buildtree"; export AUTOPKGTEST_NORMAL_USER=; export ADT_NORMAL_USER=; chmod +x /tmp/autopkgtest-lxc.is4n6xxr/downtmp/build.f2G/real-tree/debian/tests/timedated; touch /tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-stdout /tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-stderr; /tmp/autopkgtest-lxc.is4n6xxr/downtmp/build.f2G/real-tree/debian/tests/timedated 2> >(tee -a /tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-stderr >&2) > >(tee -a /tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-stdout);
> 
> For some reason this is causing the final two tee's to be created
> as children of debian/tests/timedated rather than the bash shell.

This is because of the same optimisation that dash also has, where it 
tries to avoid creating a subshell for the last command in a list when 
it can just exec() without a fork() instead. A minimal example without 
an explicit exec is

  bash -c 'dash -c ": & wait" <(sleep 1d)'

Cheers,
Harald van Dijk

Information forwarded to debian-bugs-dist@lists.debian.org, Andrej Shadura <andrewsh@debian.org>:
Bug#974705; Package dash. (Tue, 01 Dec 2020 10:39:03 GMT) (full text, mbox, link).


Acknowledgement sent to Herbert Xu <herbert@gondor.apana.org.au>:
Extra info received and forwarded to list. Copy sent to Andrej Shadura <andrewsh@debian.org>. (Tue, 01 Dec 2020 10:39:03 GMT) (full text, mbox, link).


Message #64 received at 974705@bugs.debian.org (full text, mbox, reply):

From: Herbert Xu <herbert@gondor.apana.org.au>
To: Harald van Dijk <harald@gigawatt.nl>
Cc: Andrej Shadura <andrew.shadura@collabora.co.uk>, dash@vger.kernel.org, Michael Biebl <biebl@debian.org>, 974705@bugs.debian.org
Subject: Re: Changes to job handling cause hangs in wait
Date: Tue, 1 Dec 2020 21:34:48 +1100
On Tue, Dec 01, 2020 at 10:14:04AM +0000, Harald van Dijk wrote:
> 
> POSIX says:
> 
>   "If  the  wait utility is invoked with no operands, it shall wait until
> all process IDs known to the invoking shell have terminated and exit with a
> zero exit status."
> 
> I would say that child processes that were created before dash was started
> do not have process IDs known to dash.

Well I disagree and the fact that the original ksh does the same
thing is an important fact.

> > bash -c set -e; export USER=`id -nu`; . /etc/profile >/dev/null 2>&1 || true;  . ~/.profile >/dev/null 2>&1 || true; buildtree="/tmp/autopkgtest-lxc.is4n6xxr/downtmp/build.f2G/real-tree"; mkdir -p -m 1777 -- "/tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-artifacts"; export AUTOPKGTEST_ARTIFACTS="/tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-artifacts"; export ADT_ARTIFACTS="$AUTOPKGTEST_ARTIFACTS"; mkdir -p -m 755 "/tmp/autopkgtest-lxc.is4n6xxr/downtmp/autopkgtest_tmp"; export AUTOPKGTEST_TMP="/tmp/autopkgtest-lxc.is4n6xxr/downtmp/autopkgtest_tmp"; export ADTTMP="$AUTOPKGTEST_TMP"; export DEBIAN_FRONTEND=noninteractive; export LANG=C.UTF-8; export DEB_BUILD_OPTIONS=parallel=2; unset LANGUAGE LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE   LC_MONETARY LC_MESSAGES LC_PAPER LC_NAME LC_ADDRESS   LC_TELEPHONE LC_MEASUREMENT LC_IDENTIFICATION LC_ALL;rm -f /tmp/autopkgtest_script_pid; set -C; echo $$ > /tmp/autopkgtest_script_pid; set +C; trap "rm -f /tmp/autopkgtest_script_pid" EXIT INT QUIT PIPE; cd "$buildtree"; export AUTOPKGTEST_NORMAL_USER=; export ADT_NORMAL_USER=; chmod +x /tmp/autopkgtest-lxc.is4n6xxr/downtmp/build.f2G/real-tree/debian/tests/timedated; touch /tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-stdout /tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-stderr; /tmp/autopkgtest-lxc.is4n6xxr/downtmp/build.f2G/real-tree/debian/tests/timedated 2> >(tee -a /tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-stderr >&2) > >(tee -a /tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-stdout);
> > 
> > For some reason this is causing the final two tee's to be created
> > as children of debian/tests/timedated rather than the bash shell.
> 
> This is because of the same optimisation that dash also has, where it tries
> to avoid creating a subshell for the last command in a list when it can just
> exec() without a fork() instead. A minimal example without an explicit exec
> is
> 
>   bash -c 'dash -c ": & wait" <(sleep 1d)'

I'm not sure about that because bash itself is still hanging around,
if it were really the -c optimisation then bash should not appear in
the ps output at all.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt



Information forwarded to debian-bugs-dist@lists.debian.org, Andrej Shadura <andrewsh@debian.org>:
Bug#974705; Package dash. (Tue, 01 Dec 2020 10:54:03 GMT) (full text, mbox, link).


Acknowledgement sent to Harald van Dijk <harald@gigawatt.nl>:
Extra info received and forwarded to list. Copy sent to Andrej Shadura <andrewsh@debian.org>. (Tue, 01 Dec 2020 10:54:03 GMT) (full text, mbox, link).


Message #69 received at 974705@bugs.debian.org (full text, mbox, reply):

From: Harald van Dijk <harald@gigawatt.nl>
To: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Andrej Shadura <andrew.shadura@collabora.co.uk>, dash@vger.kernel.org, Michael Biebl <biebl@debian.org>, 974705@bugs.debian.org
Subject: Re: Changes to job handling cause hangs in wait
Date: Tue, 1 Dec 2020 10:50:19 +0000
On 01/12/2020 10:34, Herbert Xu wrote:
> On Tue, Dec 01, 2020 at 10:14:04AM +0000, Harald van Dijk wrote:
>>
>> POSIX says:
>>
>>    "If  the  wait utility is invoked with no operands, it shall wait until
>> all process IDs known to the invoking shell have terminated and exit with a
>> zero exit status."
>>
>> I would say that child processes that were created before dash was started
>> do not have process IDs known to dash.
> 
> Well I disagree

Then dash still has a bug regardless:

  bash -c 'dash -c "wait" <(sleep 1d)'

Here, dash does not wait for the child process.

The sleep process is either known to dash, in which case dash must wait 
in both examples, or not known to dash, in which case dash must wait in 
neither example.

>                 and the fact that the original ksh does the same
> thing is an important fact.

The original ksh is well-known not to conform to the current POSIX 
requirements, in some cases because POSIX changed requirements, in some 
cases because ksh's behaviour was clearly buggy and not worth standardising.

>>> bash -c set -e; export USER=`id -nu`; . /etc/profile >/dev/null 2>&1 || true;  . ~/.profile >/dev/null 2>&1 || true; buildtree="/tmp/autopkgtest-lxc.is4n6xxr/downtmp/build.f2G/real-tree"; mkdir -p -m 1777 -- "/tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-artifacts"; export AUTOPKGTEST_ARTIFACTS="/tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-artifacts"; export ADT_ARTIFACTS="$AUTOPKGTEST_ARTIFACTS"; mkdir -p -m 755 "/tmp/autopkgtest-lxc.is4n6xxr/downtmp/autopkgtest_tmp"; export AUTOPKGTEST_TMP="/tmp/autopkgtest-lxc.is4n6xxr/downtmp/autopkgtest_tmp"; export ADTTMP="$AUTOPKGTEST_TMP"; export DEBIAN_FRONTEND=noninteractive; export LANG=C.UTF-8; export DEB_BUILD_OPTIONS=parallel=2; unset LANGUAGE LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE   LC_MONETARY LC_MESSAGES LC_PAPER LC_NAME LC_ADDRESS   LC_TELEPHONE LC_MEASUREMENT LC_IDENTIFICATION LC_ALL;rm -f /tmp/autopkgtest_script_pid; set -C; echo $$ > /tmp/autopkgtest_script_pid; set +C; trap "rm -f /tmp/autopkgtest_script_pid" EXIT INT QUIT PIPE; cd "$buildtree"; export AUTOPKGTEST_NORMAL_USER=; export ADT_NORMAL_USER=; chmod +x /tmp/autopkgtest-lxc.is4n6xxr/downtmp/build.f2G/real-tree/debian/tests/timedated; touch /tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-stdout /tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-stderr; /tmp/autopkgtest-lxc.is4n6xxr/downtmp/build.f2G/real-tree/debian/tests/timedated 2> >(tee -a /tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-stderr >&2) > >(tee -a /tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-stdout);
>>>
>>> For some reason this is causing the final two tee's to be created
>>> as children of debian/tests/timedated rather than the bash shell.
>>
>> This is because of the same optimisation that dash also has, where it tries
>> to avoid creating a subshell for the last command in a list when it can just
>> exec() without a fork() instead. A minimal example without an explicit exec
>> is
>>
>>    bash -c 'dash -c ": & wait" <(sleep 1d)'
> 
> I'm not sure about that because bash itself is still hanging around,
> if it were really the -c optimisation then bash should not appear in
> the ps output at all.

Good point, it's a bit more complicated. Can take a more in-depth look 
at that example later.

Here's a simpler example without bash at all, then, where it is far 
clearer that the -c optimisation is responsible:

  dash -c 'sleep 1d & dash -c ": & wait"'

This used to exit immediately, leaving the sleep process running in the 
background without waiting for it. On the dash that's currently provided 
by Ubuntu, based on 0.5.10.2, it still behaves that way. With current 
dash from Git, it does not. This is clearly a change in behaviour in 
dash not in response to any bug (real or not) in bash.

Cheers,
Harald van Dijk

Information forwarded to debian-bugs-dist@lists.debian.org, Andrej Shadura <andrewsh@debian.org>:
Bug#974705; Package dash. (Tue, 01 Dec 2020 10:57:03 GMT) (full text, mbox, link).


Acknowledgement sent to Herbert Xu <herbert@gondor.apana.org.au>:
Extra info received and forwarded to list. Copy sent to Andrej Shadura <andrewsh@debian.org>. (Tue, 01 Dec 2020 10:57:03 GMT) (full text, mbox, link).


Message #74 received at 974705@bugs.debian.org (full text, mbox, reply):

From: Herbert Xu <herbert@gondor.apana.org.au>
To: Harald van Dijk <harald@gigawatt.nl>
Cc: Andrej Shadura <andrew.shadura@collabora.co.uk>, dash@vger.kernel.org, Michael Biebl <biebl@debian.org>, 974705@bugs.debian.org
Subject: Re: Changes to job handling cause hangs in wait
Date: Tue, 1 Dec 2020 21:53:28 +1100
On Tue, Dec 01, 2020 at 10:50:19AM +0000, Harald van Dijk wrote:
> 
> This used to exit immediately, leaving the sleep process running in the
> background without waiting for it. On the dash that's currently provided by
> Ubuntu, based on 0.5.10.2, it still behaves that way. With current dash from
> Git, it does not. This is clearly a change in behaviour in dash not in
> response to any bug (real or not) in bash.

I'm not suggesting it's a bug in bash.  If anything it's a bug
in the script.  You should never do a naked wait unless you are
sure that there are no other children around that you don't know of.

In the original bug, the proper solution is to wait on the PID
that the script just sent a kill signal to.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt



Information forwarded to debian-bugs-dist@lists.debian.org, Andrej Shadura <andrewsh@debian.org>:
Bug#974705; Package dash. (Tue, 01 Dec 2020 10:57:04 GMT) (full text, mbox, link).


Acknowledgement sent to Harald van Dijk <harald@gigawatt.nl>:
Extra info received and forwarded to list. Copy sent to Andrej Shadura <andrewsh@debian.org>. (Tue, 01 Dec 2020 10:57:04 GMT) (full text, mbox, link).


Message #79 received at 974705@bugs.debian.org (full text, mbox, reply):

From: Harald van Dijk <harald@gigawatt.nl>
To: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Andrej Shadura <andrew.shadura@collabora.co.uk>, dash@vger.kernel.org, Michael Biebl <biebl@debian.org>, 974705@bugs.debian.org
Subject: Re: Changes to job handling cause hangs in wait
Date: Tue, 1 Dec 2020 10:55:06 +0000
On 01/12/2020 10:53, Herbert Xu wrote:
> On Tue, Dec 01, 2020 at 10:50:19AM +0000, Harald van Dijk wrote:
>>
>> This used to exit immediately, leaving the sleep process running in the
>> background without waiting for it. On the dash that's currently provided by
>> Ubuntu, based on 0.5.10.2, it still behaves that way. With current dash from
>> Git, it does not. This is clearly a change in behaviour in dash not in
>> response to any bug (real or not) in bash.
> 
> I'm not suggesting it's a bug in bash.

You wrote: "So the problem is really in the parent of this shell, which 
appears to be bash:"

Cheers,
Harald van Dijk



Information forwarded to debian-bugs-dist@lists.debian.org, Andrej Shadura <andrewsh@debian.org>:
Bug#974705; Package dash. (Tue, 01 Dec 2020 10:57:06 GMT) (full text, mbox, link).


Acknowledgement sent to Herbert Xu <herbert@gondor.apana.org.au>:
Extra info received and forwarded to list. Copy sent to Andrej Shadura <andrewsh@debian.org>. (Tue, 01 Dec 2020 10:57:06 GMT) (full text, mbox, link).


Message #84 received at 974705@bugs.debian.org (full text, mbox, reply):

From: Herbert Xu <herbert@gondor.apana.org.au>
To: Harald van Dijk <harald@gigawatt.nl>
Cc: Andrej Shadura <andrew.shadura@collabora.co.uk>, dash@vger.kernel.org, Michael Biebl <biebl@debian.org>, 974705@bugs.debian.org
Subject: Re: Changes to job handling cause hangs in wait
Date: Tue, 1 Dec 2020 21:56:05 +1100
On Tue, Dec 01, 2020 at 10:55:06AM +0000, Harald van Dijk wrote:
> 
> You wrote: "So the problem is really in the parent of this shell, which
> appears to be bash:"

You should read my follow-up email too that suggested changing
the systemd script.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt



Information forwarded to debian-bugs-dist@lists.debian.org, Andrej Shadura <andrewsh@debian.org>:
Bug#974705; Package dash. (Tue, 01 Dec 2020 11:03:03 GMT) (full text, mbox, link).


Acknowledgement sent to Harald van Dijk <harald@gigawatt.nl>:
Extra info received and forwarded to list. Copy sent to Andrej Shadura <andrewsh@debian.org>. (Tue, 01 Dec 2020 11:03:03 GMT) (full text, mbox, link).


Message #89 received at 974705@bugs.debian.org (full text, mbox, reply):

From: Harald van Dijk <harald@gigawatt.nl>
To: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Andrej Shadura <andrew.shadura@collabora.co.uk>, dash@vger.kernel.org, Michael Biebl <biebl@debian.org>, 974705@bugs.debian.org
Subject: Re: Changes to job handling cause hangs in wait
Date: Tue, 1 Dec 2020 10:59:20 +0000
On 01/12/2020 10:56, Herbert Xu wrote:
> On Tue, Dec 01, 2020 at 10:55:06AM +0000, Harald van Dijk wrote:
>>
>> You wrote: "So the problem is really in the parent of this shell, which
>> appears to be bash:"
> 
> You should read my follow-up email too that suggested changing
> the systemd script.

I do not appreciate your false accusation that I did not read your 
follow-up email. You suggested that as an alternative, without 
retracting your claim that this is a problem in bash.

Cheers,
Harald van Dijk



Information forwarded to debian-bugs-dist@lists.debian.org, Andrej Shadura <andrewsh@debian.org>:
Bug#974705; Package dash. (Tue, 01 Dec 2020 11:03:04 GMT) (full text, mbox, link).


Acknowledgement sent to Herbert Xu <herbert@gondor.apana.org.au>:
Extra info received and forwarded to list. Copy sent to Andrej Shadura <andrewsh@debian.org>. (Tue, 01 Dec 2020 11:03:04 GMT) (full text, mbox, link).


Message #94 received at 974705@bugs.debian.org (full text, mbox, reply):

From: Herbert Xu <herbert@gondor.apana.org.au>
To: Harald van Dijk <harald@gigawatt.nl>
Cc: Andrej Shadura <andrew.shadura@collabora.co.uk>, dash@vger.kernel.org, Michael Biebl <biebl@debian.org>, 974705@bugs.debian.org
Subject: Re: Changes to job handling cause hangs in wait
Date: Tue, 1 Dec 2020 22:01:46 +1100
On Tue, Dec 01, 2020 at 10:59:20AM +0000, Harald van Dijk wrote:
> 
> I do not appreciate your false accusation that I did not read your follow-up
> email. You suggested that as an alternative, without retracting your claim
> that this is a problem in bash.

Well I'm not accusing you of anything, I was just trying to make
sure you are aware of my complete message.

In any case, I never said that this is a bug in bash, merely that
bash is involved in creating a situation where dash started out
with children in the background.

If anything this could just be an artefact of the bash script
rather than bash itself.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt



Information forwarded to debian-bugs-dist@lists.debian.org, Andrej Shadura <andrewsh@debian.org>:
Bug#974705; Package dash. (Tue, 01 Dec 2020 13:09:03 GMT) (full text, mbox, link).


Acknowledgement sent to Michael Biebl <biebl@debian.org>:
Extra info received and forwarded to list. Copy sent to Andrej Shadura <andrewsh@debian.org>. (Tue, 01 Dec 2020 13:09:03 GMT) (full text, mbox, link).


Message #99 received at 974705@bugs.debian.org (full text, mbox, reply):

From: Michael Biebl <biebl@debian.org>
To: Herbert Xu <herbert@gondor.apana.org.au>, Harald van Dijk <harald@gigawatt.nl>
Cc: Andrej Shadura <andrew.shadura@collabora.co.uk>, dash@vger.kernel.org, 974705@bugs.debian.org
Subject: Re: Changes to job handling cause hangs in wait
Date: Tue, 1 Dec 2020 14:04:17 +0100
Am 01.12.20 um 11:53 schrieb Herbert Xu:
> On Tue, Dec 01, 2020 at 10:50:19AM +0000, Harald van Dijk wrote:
>>
>> This used to exit immediately, leaving the sleep process running in the
>> background without waiting for it. On the dash that's currently provided by
>> Ubuntu, based on 0.5.10.2, it still behaves that way. With current dash from
>> Git, it does not. This is clearly a change in behaviour in dash not in
>> response to any bug (real or not) in bash.
> 
> I'm not suggesting it's a bug in bash.  If anything it's a bug
> in the script.

Keep in mind, that the timedated script is not the only affected script, 
there is at least src:fence-agents as well and there might be other 
packages affected as well as local scripts which expect /bin/sh to 
behave POSIX compliant in that regard.

If this change in behaviour is kept, I would at least expect a big fat 
NEWS entry in the Debian dash package which is shown on upgrades and it 
should probably also update the package description which currently 
still says that it is a POSIX-compliant shell.

Regards,
Michael



Information forwarded to debian-bugs-dist@lists.debian.org, Andrej Shadura <andrewsh@debian.org>:
Bug#974705; Package dash. (Tue, 01 Dec 2020 23:24:02 GMT) (full text, mbox, link).


Acknowledgement sent to Michael Biebl <biebl@debian.org>:
Extra info received and forwarded to list. Copy sent to Andrej Shadura <andrewsh@debian.org>. (Tue, 01 Dec 2020 23:24:03 GMT) (full text, mbox, link).


Message #104 received at 974705@bugs.debian.org (full text, mbox, reply):

From: Michael Biebl <biebl@debian.org>
To: Herbert Xu <herbert@gondor.apana.org.au>, Andrej Shadura <andrew.shadura@collabora.co.uk>
Cc: dash@vger.kernel.org, 974705@bugs.debian.org
Subject: Re: Changes to job handling cause hangs in wait
Date: Wed, 2 Dec 2020 00:21:52 +0100
Am 01.12.20 um 07:56 schrieb Herbert Xu:
> On Tue, Dec 01, 2020 at 05:06:18PM +1100, Herbert Xu wrote:
>>
>> For some reason this is causing the final two tee's to be created
>> as children of debian/tests/timedated rather than the bash shell.
> 
> An alternative to changing the parent is of course to do
> 
> 	wait $MONPID
> 
> instead of
> 
> 	wait
> 
> I think this makes more sense anyway as otherwise someone could
> easily introduce a hang if they unwittingly add a backgrounded
> job to this script.

I'll update the timedated test accordingly.
Incidentally we already have
https://salsa.debian.org/systemd-team/systemd/-/merge_requests/110



Information forwarded to debian-bugs-dist@lists.debian.org, Andrej Shadura <andrewsh@debian.org>:
Bug#974705; Package dash. (Tue, 01 Dec 2020 23:30:02 GMT) (full text, mbox, link).


Acknowledgement sent to Herbert Xu <herbert@gondor.apana.org.au>:
Extra info received and forwarded to list. Copy sent to Andrej Shadura <andrewsh@debian.org>. (Tue, 01 Dec 2020 23:30:02 GMT) (full text, mbox, link).


Message #109 received at 974705@bugs.debian.org (full text, mbox, reply):

From: Herbert Xu <herbert@gondor.apana.org.au>
To: Michael Biebl <biebl@debian.org>
Cc: Andrej Shadura <andrew.shadura@collabora.co.uk>, dash@vger.kernel.org, 974705@bugs.debian.org
Subject: Re: Changes to job handling cause hangs in wait
Date: Wed, 2 Dec 2020 10:26:06 +1100
On Wed, Dec 02, 2020 at 12:21:52AM +0100, Michael Biebl wrote:
>
> I'll update the timedated test accordingly.
> Incidentally we already have
> https://salsa.debian.org/systemd-team/systemd/-/merge_requests/110

Thanks.

I will make dash not wait for existing children as this is indeed
a change in behaviour which is undesirable.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt



Information forwarded to debian-bugs-dist@lists.debian.org, Andrej Shadura <andrewsh@debian.org>:
Bug#974705; Package dash. (Wed, 02 Dec 2020 05:33:03 GMT) (full text, mbox, link).


Acknowledgement sent to Herbert Xu <herbert@gondor.apana.org.au>:
Extra info received and forwarded to list. Copy sent to Andrej Shadura <andrewsh@debian.org>. (Wed, 02 Dec 2020 05:33:03 GMT) (full text, mbox, link).


Message #114 received at 974705@bugs.debian.org (full text, mbox, reply):

From: Herbert Xu <herbert@gondor.apana.org.au>
To: Michael Biebl <biebl@debian.org>
Cc: Andrej Shadura <andrew.shadura@collabora.co.uk>, dash@vger.kernel.org, 974705@bugs.debian.org
Subject: [PATCH] jobs: Only block in waitcmd on first run
Date: Wed, 2 Dec 2020 16:31:45 +1100
This patch ensures that waitcmd never blocks unless there are
outstanding jobs.  This could otherwise trigger a hang if children
were created prior to the shell coming into existence, or if
there are backgrounded children of other kinds (e.g., a here-
document).

Fixes: 6c691b3e5099 ("jobs: Only clear gotsigchld when waiting...")
Reported-by: Michael Biebl <biebl@debian.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

diff --git a/src/jobs.c b/src/jobs.c
index 3417633..516786f 100644
--- a/src/jobs.c
+++ b/src/jobs.c
@@ -81,6 +81,7 @@
 #define DOWAIT_NONBLOCK 0
 #define DOWAIT_BLOCK 1
 #define DOWAIT_WAITCMD 2
+#define DOWAIT_WAITCMD_ALL 4
 
 /* array of jobs */
 static struct job *jobtab;
@@ -615,7 +616,7 @@ waitcmd(int argc, char **argv)
 				jp->waited = 1;
 				jp = jp->prev_job;
 			}
-			if (!dowait(DOWAIT_WAITCMD, 0))
+			if (!dowait(DOWAIT_WAITCMD_ALL, 0))
 				goto sigout;
 		}
 	}
@@ -1138,6 +1139,7 @@ static int dowait(int block, struct job *jp)
 		pid = waitone(block, jp);
 		rpid &= !!pid;
 
+		block &= ~DOWAIT_WAITCMD_ALL;
 		if (!pid || (jp && jp->state != JOBRUNNING))
 			block = DOWAIT_NONBLOCK;
 	} while (pid >= 0);
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt



Information forwarded to debian-bugs-dist@lists.debian.org, Andrej Shadura <andrewsh@debian.org>:
Bug#974705; Package dash. (Thu, 03 Dec 2020 12:30:02 GMT) (full text, mbox, link).


Acknowledgement sent to Michael Biebl <biebl@debian.org>:
Extra info received and forwarded to list. Copy sent to Andrej Shadura <andrewsh@debian.org>. (Thu, 03 Dec 2020 12:30:02 GMT) (full text, mbox, link).


Message #119 received at 974705@bugs.debian.org (full text, mbox, reply):

From: Michael Biebl <biebl@debian.org>
To: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Andrej Shadura <andrew.shadura@collabora.co.uk>, dash@vger.kernel.org, 974705@bugs.debian.org
Subject: Re: Changes to job handling cause hangs in wait
Date: Thu, 3 Dec 2020 13:27:51 +0100
Am 02.12.20 um 00:26 schrieb Herbert Xu:
> On Wed, Dec 02, 2020 at 12:21:52AM +0100, Michael Biebl wrote:
>>
>> I'll update the timedated test accordingly.
>> Incidentally we already have
>> https://salsa.debian.org/systemd-team/systemd/-/merge_requests/110
> 
> Thanks.

Hm, so I applied this change and switched backed to dash, but now I get

autopkgtest systemd --test-name=timedated -s -- lxc -s autopkgtest-sid
...
disable NTP
enable NTP
Terminated
autopkgtest [13:21:42]: test timedated: -----------------------]
autopkgtest [13:21:42]: test timedated:  - - - - - - - - - - results - - 
- - - - - - - -
timedated            FAIL non-zero exit status 143


Running with sh -x
...
+ kill 825
+ wait 825
Terminated


Herbert, any idea what going wrong there?
(systemd 247.1-2 and dash 0.5.11+git20200708+dd9ef66-2)

Michael



Information forwarded to debian-bugs-dist@lists.debian.org, Andrej Shadura <andrewsh@debian.org>:
Bug#974705; Package dash. (Thu, 03 Dec 2020 12:51:02 GMT) (full text, mbox, link).


Acknowledgement sent to Michael Biebl <biebl@debian.org>:
Extra info received and forwarded to list. Copy sent to Andrej Shadura <andrewsh@debian.org>. (Thu, 03 Dec 2020 12:51:02 GMT) (full text, mbox, link).


Message #124 received at 974705@bugs.debian.org (full text, mbox, reply):

From: Michael Biebl <biebl@debian.org>
To: Herbert Xu <herbert@gondor.apana.org.au>, Andrej Shadura <andrew.shadura@collabora.co.uk>
Cc: dash@vger.kernel.org, 974705@bugs.debian.org
Subject: Re: Changes to job handling cause hangs in wait
Date: Thu, 3 Dec 2020 13:49:44 +0100
Am 01.12.20 um 07:56 schrieb Herbert Xu:
> On Tue, Dec 01, 2020 at 05:06:18PM +1100, Herbert Xu wrote:
>>
>> For some reason this is causing the final two tee's to be created
>> as children of debian/tests/timedated rather than the bash shell.
> 
> An alternative to changing the parent is of course to do
> 
> 	wait $MONPID
> 
> instead of
> 
> 	wait
> 
> I think this makes more sense anyway as otherwise someone could
> easily introduce a hang if they unwittingly add a backgrounded
> job to this script.

I applied this change, but now the script terminates with 143 [1]

I'm happy for any suggestions to make the script more correct/robust and 
make it pass successfully.

Michael


[1] 
https://ci.debian.net/data/autopkgtest/testing/amd64/s/systemd/8600208/log.gz




Information forwarded to debian-bugs-dist@lists.debian.org, Andrej Shadura <andrewsh@debian.org>:
Bug#974705; Package dash. (Mon, 07 Dec 2020 03:57:02 GMT) (full text, mbox, link).


Acknowledgement sent to Herbert Xu <herbert@gondor.apana.org.au>:
Extra info received and forwarded to list. Copy sent to Andrej Shadura <andrewsh@debian.org>. (Mon, 07 Dec 2020 03:57:02 GMT) (full text, mbox, link).


Message #129 received at 974705@bugs.debian.org (full text, mbox, reply):

From: Herbert Xu <herbert@gondor.apana.org.au>
To: Michael Biebl <biebl@debian.org>
Cc: Andrej Shadura <andrew.shadura@collabora.co.uk>, dash@vger.kernel.org, 974705@bugs.debian.org
Subject: Re: Changes to job handling cause hangs in wait
Date: Mon, 7 Dec 2020 14:55:06 +1100
On Thu, Dec 03, 2020 at 01:27:51PM +0100, Michael Biebl wrote:
>
> Hm, so I applied this change and switched backed to dash, but now I get
> 
> autopkgtest systemd --test-name=timedated -s -- lxc -s autopkgtest-sid
> ...
> disable NTP
> enable NTP
> Terminated
> autopkgtest [13:21:42]: test timedated: -----------------------]
> autopkgtest [13:21:42]: test timedated:  - - - - - - - - - - results - - - -
> - - - - - -
> timedated            FAIL non-zero exit status 143
> 
> 
> Running with sh -x
> ...
> + kill 825
> + wait 825
> Terminated
> 
> 
> Herbert, any idea what going wrong there?
> (systemd 247.1-2 and dash 0.5.11+git20200708+dd9ef66-2)

Sorry, my bad.  wait(1) with no arguments ignores the error status
of the child and always return zero.  wait(1) specifically on a
child obviously returns the error status of that child.  Since the
child was killed, we need to ignore that, so this works for me:

	wait $MONPID 2> /dev/null || :

Thanks,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt



Reply sent to Andrej Shadura <andrewsh@debian.org>:
You have taken responsibility. (Thu, 10 Dec 2020 09:21:03 GMT) (full text, mbox, link).


Notification sent to Michael Biebl <biebl@debian.org>:
Bug acknowledged by developer. (Thu, 10 Dec 2020 09:21:03 GMT) (full text, mbox, link).


Message #134 received at 974705-close@bugs.debian.org (full text, mbox, reply):

From: Debian FTP Masters <ftpmaster@ftp-master.debian.org>
To: 974705-close@bugs.debian.org
Subject: Bug#974705: fixed in dash 0.5.11+git20200708+dd9ef66-3
Date: Thu, 10 Dec 2020 09:18:57 +0000
Source: dash
Source-Version: 0.5.11+git20200708+dd9ef66-3
Done: Andrej Shadura <andrewsh@debian.org>

We believe that the bug you reported is fixed in the latest version of
dash, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 974705@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Andrej Shadura <andrewsh@debian.org> (supplier of updated dash package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Thu, 10 Dec 2020 10:07:56 +0100
Source: dash
Architecture: source
Version: 0.5.11+git20200708+dd9ef66-3
Distribution: unstable
Urgency: medium
Maintainer: Andrej Shadura <andrewsh@debian.org>
Changed-By: Andrej Shadura <andrewsh@debian.org>
Closes: 974705 974825 974900 975010 976685
Changes:
 dash (0.5.11+git20200708+dd9ef66-3) unstable; urgency=medium
 .
   [ Johannes 'josch' Schauer ]
   * Don’t rely on /usr/share/man/man1/ existing (Closes: #974825).
 .
   [ Andrej Shadura ]
   * Disable libc glob support (Closes: #974900, #976685).
   * Apply upstream patch:
     - jobs: Only block in waitcmd on first run (Closes: #974705, #975010).
Checksums-Sha1:
 0df9a64f9f6f7e050cbe03611a792f8fb0e59f24 1906 dash_0.5.11+git20200708+dd9ef66-3.dsc
 1edd2f22cc44f69ce9d6b5d900af6d37640caeac 43040 dash_0.5.11+git20200708+dd9ef66-3.debian.tar.xz
Checksums-Sha256:
 56ef65ce7170af72469b114a77525e4ef035d0e711d7a1457336cbf6488b6195 1906 dash_0.5.11+git20200708+dd9ef66-3.dsc
 fa420d19153b20d0a0f5fa5885599d7e5dc717753faec63533edb4e77aa21079 43040 dash_0.5.11+git20200708+dd9ef66-3.debian.tar.xz
Files:
 e939d7206ef105e285c015090c5ef509 1906 shells optional dash_0.5.11+git20200708+dd9ef66-3.dsc
 1435220b6b2026f2f1868bf7e28f3cde 43040 shells optional dash_0.5.11+git20200708+dd9ef66-3.debian.tar.xz

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEeuS9ZL8A0js0NGiOXkCM2RzYOdIFAl/R5bMACgkQXkCM2RzY
OdKmOgf/R2HcMGe2mn2qdL8tI0es6iwdg1S12Pq74p51Os9impdQ6Pv6Iy0v6qQZ
Wq+k28nqbUbby8+HrNHGV8QHP89teTimyirxemdQpORrETWQVFdNJ/bKcMC/3KYz
mlteVyU4F+ArOJbkEzkv7RO2el7TDErytPNyxMRkvwf8+pCRAF9Q5lg/uPDKoJkI
wG+xgz6LVjqk9codvKnqwuHstKn2HfAY2Y2hrj7JsSzG+7bb3M1VqlhtpepZBlYT
QQCrM03XzLZvTuoxfilmlVBZ19i8rAalorzoGEfY0IEIAGzB7FiVP7cV/dxBDgkO
gBP+BRRREBdPQ1rxRZWyGkVOiPt2BQ==
=1dPH
-----END PGP SIGNATURE-----




Reply sent to Andrej Shadura <andrewsh@debian.org>:
You have taken responsibility. (Thu, 10 Dec 2020 09:21:04 GMT) (full text, mbox, link).


Notification sent to Paul Gevers <elbrus@debian.org>:
Bug acknowledged by developer. (Thu, 10 Dec 2020 09:21:04 GMT) (full text, mbox, link).


Message #139 received at 975010-close@bugs.debian.org (full text, mbox, reply):

From: Debian FTP Masters <ftpmaster@ftp-master.debian.org>
To: 975010-close@bugs.debian.org
Subject: Bug#975010: fixed in dash 0.5.11+git20200708+dd9ef66-3
Date: Thu, 10 Dec 2020 09:18:57 +0000
Source: dash
Source-Version: 0.5.11+git20200708+dd9ef66-3
Done: Andrej Shadura <andrewsh@debian.org>

We believe that the bug you reported is fixed in the latest version of
dash, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 975010@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Andrej Shadura <andrewsh@debian.org> (supplier of updated dash package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Thu, 10 Dec 2020 10:07:56 +0100
Source: dash
Architecture: source
Version: 0.5.11+git20200708+dd9ef66-3
Distribution: unstable
Urgency: medium
Maintainer: Andrej Shadura <andrewsh@debian.org>
Changed-By: Andrej Shadura <andrewsh@debian.org>
Closes: 974705 974825 974900 975010 976685
Changes:
 dash (0.5.11+git20200708+dd9ef66-3) unstable; urgency=medium
 .
   [ Johannes 'josch' Schauer ]
   * Don’t rely on /usr/share/man/man1/ existing (Closes: #974825).
 .
   [ Andrej Shadura ]
   * Disable libc glob support (Closes: #974900, #976685).
   * Apply upstream patch:
     - jobs: Only block in waitcmd on first run (Closes: #974705, #975010).
Checksums-Sha1:
 0df9a64f9f6f7e050cbe03611a792f8fb0e59f24 1906 dash_0.5.11+git20200708+dd9ef66-3.dsc
 1edd2f22cc44f69ce9d6b5d900af6d37640caeac 43040 dash_0.5.11+git20200708+dd9ef66-3.debian.tar.xz
Checksums-Sha256:
 56ef65ce7170af72469b114a77525e4ef035d0e711d7a1457336cbf6488b6195 1906 dash_0.5.11+git20200708+dd9ef66-3.dsc
 fa420d19153b20d0a0f5fa5885599d7e5dc717753faec63533edb4e77aa21079 43040 dash_0.5.11+git20200708+dd9ef66-3.debian.tar.xz
Files:
 e939d7206ef105e285c015090c5ef509 1906 shells optional dash_0.5.11+git20200708+dd9ef66-3.dsc
 1435220b6b2026f2f1868bf7e28f3cde 43040 shells optional dash_0.5.11+git20200708+dd9ef66-3.debian.tar.xz

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEeuS9ZL8A0js0NGiOXkCM2RzYOdIFAl/R5bMACgkQXkCM2RzY
OdKmOgf/R2HcMGe2mn2qdL8tI0es6iwdg1S12Pq74p51Os9impdQ6Pv6Iy0v6qQZ
Wq+k28nqbUbby8+HrNHGV8QHP89teTimyirxemdQpORrETWQVFdNJ/bKcMC/3KYz
mlteVyU4F+ArOJbkEzkv7RO2el7TDErytPNyxMRkvwf8+pCRAF9Q5lg/uPDKoJkI
wG+xgz6LVjqk9codvKnqwuHstKn2HfAY2Y2hrj7JsSzG+7bb3M1VqlhtpepZBlYT
QQCrM03XzLZvTuoxfilmlVBZ19i8rAalorzoGEfY0IEIAGzB7FiVP7cV/dxBDgkO
gBP+BRRREBdPQ1rxRZWyGkVOiPt2BQ==
=1dPH
-----END PGP SIGNATURE-----




Information forwarded to debian-bugs-dist@lists.debian.org, Andrej Shadura <andrewsh@debian.org>:
Bug#974705; Package dash. (Fri, 11 Dec 2020 19:21:04 GMT) (full text, mbox, link).


Acknowledgement sent to Michael Biebl <biebl@debian.org>:
Extra info received and forwarded to list. Copy sent to Andrej Shadura <andrewsh@debian.org>. (Fri, 11 Dec 2020 19:21:05 GMT) (full text, mbox, link).


Message #144 received at 974705@bugs.debian.org (full text, mbox, reply):

From: Michael Biebl <biebl@debian.org>
To: 974705@bugs.debian.org, Herbert Xu <herbert@gondor.apana.org.au>
Subject: Re: Changes to job handling cause hangs in wait
Date: Fri, 11 Dec 2020 20:17:00 +0100
[Message part 1 (text/plain, inline)]
Hi Herbert

On Mon, 7 Dec 2020 14:55:06 +1100 Herbert Xu <herbert@gondor.apana.org.au> wrote:

> Sorry, my bad.  wait(1) with no arguments ignores the error status
> of the child and always return zero.  wait(1) specifically on a
> child obviously returns the error status of that child.  Since the
> child was killed, we need to ignore that, so this works for me:
> 
>       wait $MONPID 2> /dev/null || :

Hah, I basically came to the same result in
https://salsa.debian.org/systemd-team/systemd/-/commit/85438481ca690f627c58d86aad6193f0ad5df21c

I see that this has been addressed in dash now as well, so thanks for
that!
[signature.asc (application/pgp-signature, inline)]

Bug archived. Request was from Debbugs Internal Request <owner@bugs.debian.org> to internal_control@bugs.debian.org. (Sat, 09 Jan 2021 07:25:03 GMT) (full text, mbox, link).


Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Thu Aug 8 03:19:17 2024; Machine Name: buxtehude

Debian Bug tracking system

Debbugs is free software and licensed under the terms of the GNU Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.

Copyright © 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson, 2005-2017 Don Armstrong, and many other contributors.