Debian Bug report logs - #761423
busybox: using find in pipe with dd produce semi-random output

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Alex Andreotti <alex.andreotti@gmail.com>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: busybox: using find in pipe with dd produce semi-random output

Date: Sat, 13 Sep 2014 21:00:11 +0200

Package: busybox
Version: 1:1.22.0-8
Severity: normal

Dear Maintainer,

probably the problem is in the mainstream but it affect also Debian.

   * What led up to the situation?

I was trying to find a shell way to check if a directory is empty

   * What exactly did you do (or not do) that was effective (or
     ineffective)?

I created a chroot with only busybox and the libs needed (as shown by ldd) to be sure it is using only busybox commands.

 sudo chroot chrootdir /bin/busybox sh

created a shell function:

isempty() { [ "$(find "$1" -name "?*" | dd bs=$((${#1}+3)) count=1 2>/dev/null)" = "$1" ] ; }
mkdir /test && /test/.x
while [ 1 ] ; do if ! isempty /test ; then echo error ; break ; else echo not empty ; fi ; done

run it few times
run 1 to 3 exit immediatly with error
run 4 show a non emtpy then error
run 5 show 9 non empty then error
(nobody removed or added files in /test)

also running manually the command below show different results from time to time

find /test -name "?*" | dd count=1 2>/dev/null

   * What was the outcome of this action?

output truncated randomically, seem by line ending, when is wrong show only the first line.

   * What outcome did you expect instead?

always the same predictable full output of find.


-- System Information:
Debian Release: jessie/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (500, 'testing'), (500, 'stable'), (1, 'experimental')
Architecture: i386 (i686)

Kernel: Linux 3.14-1-686-pae (SMP w/4 CPU cores)
Locale: LANG=it_IT.UTF-8, LC_CTYPE=it_IT.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages busybox depends on:
ii  libc6  2.19-10

busybox recommends no packages.

busybox suggests no packages.

-- no debconf information

Message #10 received at 761423@bugs.debian.org (full text, mbox, reply):

From: Alex Andreotti <alex.andreotti@gmail.com>

To: 761423@bugs.debian.org

Subject: busybox: using find in pipe with dd produce semi-random output

Date: Sat, 13 Sep 2014 22:57:58 +0200

Sorry I see I've missed a `touch /test/.x` in the bugreport.

The test case is really simple:

    % busybox sh
    
    BusyBox v1.22.1 (Debian 1:1.22.0-8) built-in shell (ash)
    Enter 'help' for a list of built-in commands.
    
    ~ $ mkdir /tmp/emptytest
    ~ $ touch /tmp/emptytest/.x
    ~ $ find /tmp/emptytest/ -name "*?" | dd count=1 2>/dev/null
    /tmp/emptytest/
    ~ $ find /tmp/emptytest/ -name "*?" | dd count=1 2>/dev/null 
    /tmp/emptytest/
    ~ $ find /tmp/emptytest/ -name "*?" | dd count=1 2>/dev/null 
    /tmp/emptytest/
    /tmp/emptytest/.x
    ~ $ find /tmp/emptytest/ -name "*?" | dd count=1 2>/dev/null 
    /tmp/emptytest/
    ~ $ find /tmp/emptytest/ -name "*?" | dd count=1 2>/dev/null 
    /tmp/emptytest/
    /tmp/emptytest/.x
    ~ $ find /tmp/emptytest/ -name "*?" | dd count=1 2>/dev/null 
    /tmp/emptytest/
    ~ $ find /tmp/emptytest/ -name "*?" | dd count=1 2>/dev/null 
    /tmp/emptytest/
    /tmp/emptytest/.x
    
The output should be always:

    /tmp/emptytest/
    /tmp/emptytest/.x
    
Assuming dd block size default to 512.

Notice that omitting count or in pipe with cat it work properly.

I guess something wrong on how dd close the pipe or find handle
broken pipes.

Message #17 received at 761423-done@bugs.debian.org (full text, mbox, reply):

From: Michael Tokarev <mjt@tls.msk.ru>

To: Alex Andreotti <alex.andreotti@gmail.com>, 761423-done@bugs.debian.org

Subject: Re: Bug#761423: busybox: using find in pipe with dd produce semi-random output

Date: Mon, 10 Nov 2014 21:56:28 +0300

13.09.2014 23:00, Alex Andreotti wrote:
> isempty() { [ "$(find "$1" -name "?*" | dd bs=$((${#1}+3)) count=1 2>/dev/null)" = "$1" ] ; }
> mkdir /test && /test/.x
> while [ 1 ] ; do if ! isempty /test ; then echo error ; break ; else echo not empty ; fi ; done
> 
> run it few times
> run 1 to 3 exit immediatly with error
> run 4 show a non emtpy then error
> run 5 show 9 non empty then error
> (nobody removed or added files in /test)
> 
> also running manually the command below show different results from time to time
> 
> find /test -name "?*" | dd count=1 2>/dev/null

I'm not really sure this is a bug, and upstream thinks it is not a bug.

The thing is: when you tell dd to exit after first input block,
the next thing is entirely timing-dependent.

`find' applet outputs names as it finds them, using standard buffered
output routines.  These are usually configured to perform line-buffering.
So once `find' finds a first file matching the criteria, it prints its
name in one write(2) operation.  What happens next depends on timing.

If `find' finds another file and will manage to write it BEFORE `dd'
will be able to consume first file written, dd will read both, because
the pipe between the two is just a stream, each time you read from it
you can read all available data.

Or, dd is free to read first filename before `find' writes second, --
in this case, dd will write first filename and exit, because it is
asked to process just one data block.

>    * What was the outcome of this action?
> 
> output truncated randomically, seem by line ending, when is wrong show only the first line.
> 
>    * What outcome did you expect instead?
> 
> always the same predictable full output of find.

If you want predictable _full_ output, the only way to get it,
I think, is to use `iflag=fullblock' option.  Unfortunately it
is non-standard GNU dd extension, and is not implemented by
busybox.  Maybe a wishlist item for busybox dd is in order...

Meanwhile, I'm closing this bugreport, because, as described,
it is not a bug really.

Thanks,

/mjt

Send a report that this bug log contains spam.