Debian Bug report logs - #595063
dash: read() builtin doesn't work with /proc files containing only an integer value

version graph

Package: dash; Maintainer for dash is Gerrit Pape <pape@smarden.org>; Source for dash is src:dash.

Reported by: Steve Schnepp <steve.schnepp@gmail.com>

Date: Tue, 31 Aug 2010 19:18:01 UTC

Severity: normal

Tags: upstream

Found in version dash/0.5.4-12

Reply or subscribe to this bug.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, steve.schnepp@gmail.com, Gerrit Pape <pape@smarden.org>:
Bug#595063; Package dash. (Tue, 31 Aug 2010 19:18:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Steve Schnepp <steve.schnepp@gmail.com>:
New Bug report received and forwarded. Copy sent to steve.schnepp@gmail.com, Gerrit Pape <pape@smarden.org>. (Tue, 31 Aug 2010 19:18:04 GMT) Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: Steve Schnepp <steve.schnepp@gmail.com>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: dash: read() builtin doesn't work with /proc files containing only an integer value
Date: Tue, 31 Aug 2010 21:15:25 +0200
Package: dash
Version: 0.5.4-12
Severity: normal


dash's read() builtin seems to read the underlying file 1 char at a
time. This doesn't work with some files under /proc, since procfs isn't
fully POSIX compliant. 


With bash it works : 

$ bash -c 'read MAX < /proc/sys/kernel/pid_max; ec
32768

With dash it only reads the first character :

$ dash -c 'read MAX < /proc/sys/kernel/pid_max; ec
3

If we use the cat(1) external program it works :

$ dash -c 'MAX=$(cat /proc/sys/kernel/pid_max); echo $MAX'
32768

After a little digging, it only appears on files that contains just an
integer value. When asked to read with a non-null offset (*ppos != 0),
__do_proc_dointvec() just returns 0 (meaning an EOF) as shown on [1].

[1] http://lxr.linux.no/#linux+v2.6.32/kernel/sysctl.c#L2371

I'm aware that the issue isn't strictly a dash one, since it has the
right to read one character at a time. But since fixing procfs to be
conforming to POSIX isn't a realistic option, would it be possible to
have a workaround that doesn't involve an external tool like cat(1) ? 

-- System Information:
Debian Release: 5.0.5
  APT prefers proposed-updates
  APT policy: (500, 'proposed-updates'), (500, 'stable')
Architecture: i386 (i686)

Kernel: Linux 2.6.32-bpo.3-686 (SMP w/1 CPU core)
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages dash depends on:
ii  libc6                       2.7-18lenny4 GNU C Library: Shared libraries

dash recommends no packages.

dash suggests no packages.

-- debconf information:
* dash/sh: true




Information forwarded to debian-bugs-dist@lists.debian.org, Gerrit Pape <pape@smarden.org>:
Bug#595063; Package dash. (Tue, 31 Aug 2010 20:24:08 GMT) Full text and rfc822 format available.

Acknowledgement sent to Jonathan Nieder <jrnieder@gmail.com>:
Extra info received and forwarded to list. Copy sent to Gerrit Pape <pape@smarden.org>. (Tue, 31 Aug 2010 20:24:08 GMT) Full text and rfc822 format available.

Message #10 received at 595063@bugs.debian.org (full text, mbox):

From: Jonathan Nieder <jrnieder@gmail.com>
To: Steve Schnepp <steve.schnepp@gmail.com>
Cc: 595063@bugs.debian.org
Subject: Re: dash: read() builtin doesn't work with /proc files containing only an integer value
Date: Tue, 31 Aug 2010 15:19:04 -0500
tags 595063 + upstream
quit

Hi Steve,

Steve Schnepp wrote:

> With bash it works : 
> 
> $ bash -c 'read MAX < /proc/sys/kernel/pid_max; ec
> 32768
> 
> With dash it only reads the first character :
> 
> $ dash -c 'read MAX < /proc/sys/kernel/pid_max; ec
> 3

Could you report this upstream?  The address is dash@vger.kernel.org
(no need to subscribe; the convention is to always reply-to-all
there).

Realistically, the phenomenon will only get fixed if someone suggests
a patch. :)

It really is possible that the best place to fix this is in the
kernel.  It comes down to what is simplest.

Thanks for the report,
Jonathan




Added tag(s) upstream. Request was from Jonathan Nieder <jrnieder@gmail.com> to control@bugs.debian.org. (Tue, 31 Aug 2010 20:24:10 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Gerrit Pape <pape@smarden.org>:
Bug#595063; Package dash. (Wed, 01 Sep 2010 08:09:06 GMT) Full text and rfc822 format available.

Acknowledgement sent to Steve Schnepp <steve.schnepp@gmail.com>:
Extra info received and forwarded to list. Copy sent to Gerrit Pape <pape@smarden.org>. (Wed, 01 Sep 2010 08:09:06 GMT) Full text and rfc822 format available.

Message #17 received at 595063@bugs.debian.org (full text, mbox):

From: Steve Schnepp <steve.schnepp@gmail.com>
To: dash@vger.kernel.org
Cc: 595063@bugs.debian.org
Subject: read() builtin doesn't read integer value /proc files but bash's does
Date: Wed, 1 Sep 2010 10:06:15 +0200
[Message part 1 (text/plain, inline)]
Hi, I opened bug 595063 on the debian BTS [1] and I was suggested to resend
the email upstream.

So I copied the body of the bug below :

dash's read() builtin seems to read the underlying file 1 char at a
time. This doesn't work with some files under /proc, since procfs isn't
fully POSIX compliant.

With bash it works :

$ bash -c 'read MAX < /proc/sys/kernel/pid_max; echo $MAX'
32768

With dash it only reads the first character :

$ dash -c 'read MAX < /proc/sys/kernel/pid_max; echo $MAX'
3

If we use the cat(1) external program it works :

$ dash -c 'MAX=$(cat /proc/sys/kernel/pid_max); echo $MAX'
32768

After a little digging, it only appears on files that contains just an
integer value. When asked to read with a non-null offset (*ppos != 0),
__do_proc_dointvec() just returns 0 (meaning an EOF) as shown on [2].

I'm aware that the issue isn't strictly a dash one, since it has the
right to read one character at a time. But since fixing procfs to be
conforming to POSIX isn't a realistic option, would it be possible to
have a workaround that doesn't involve an external tool like cat(1) ?

[1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=595063
[2] http://lxr.linux.no/#linux+v2.6.32/kernel/sysctl.c#L2371

-- 
Steve Schnepp
http://blog.pwkf.org/
[Message part 2 (text/html, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Gerrit Pape <pape@smarden.org>:
Bug#595063; Package dash. (Wed, 01 Sep 2010 08:15:07 GMT) Full text and rfc822 format available.

Acknowledgement sent to Steve Schnepp <steve.schnepp@gmail.com>:
Extra info received and forwarded to list. Copy sent to Gerrit Pape <pape@smarden.org>. (Wed, 01 Sep 2010 08:15:07 GMT) Full text and rfc822 format available.

Message #22 received at 595063@bugs.debian.org (full text, mbox):

From: Steve Schnepp <steve.schnepp@gmail.com>
To: dash@vger.kernel.org
Cc: 595063@bugs.debian.org
Subject: read() builtin doesn't read integer value /proc files (but bash's does)
Date: Wed, 1 Sep 2010 10:10:11 +0200
Hi, I opened bug 595063 on the debian BTS [1] and I was suggested to
resend the email upstream.

So I copied the body of the bug below :

dash's read() builtin seems to read the underlying file 1 char at a
time. This doesn't work with some files under /proc, since procfs isn't
fully POSIX compliant.

With bash it works :

$ bash -c 'read MAX < /proc/sys/kernel/pid_max; echo $MAX'
32768

With dash it only reads the first character :

$ dash -c 'read MAX < /proc/sys/kernel/pid_max; echo $MAX'
3

If we use the cat(1) external program it works :

$ dash -c 'MAX=$(cat /proc/sys/kernel/pid_max); echo $MAX'
32768

After a little digging, it only appears on files that contains just an
integer value. When asked to read with a non-null offset (*ppos != 0),
__do_proc_dointvec() just returns 0 (meaning an EOF) as shown on [2].

I'm aware that the issue isn't strictly a dash one, since it has the
right to read one character at a time. But since fixing procfs to be
conforming to POSIX isn't a realistic option, would it be possible to
have a workaround that doesn't involve an external tool like cat(1) ?

[1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=595063
[2] http://lxr.linux.no/#linux+v2.6.32/kernel/sysctl.c#L2371
--
Steve Schnepp
http://blog.pwkf.org/




Information forwarded to debian-bugs-dist@lists.debian.org, Gerrit Pape <pape@smarden.org>:
Bug#595063; Package dash. (Thu, 02 Sep 2010 15:12:10 GMT) Full text and rfc822 format available.

Acknowledgement sent to Steve Schnepp <steve.schnepp@gmail.com>:
Extra info received and forwarded to list. Copy sent to Gerrit Pape <pape@smarden.org>. (Thu, 02 Sep 2010 15:12:10 GMT) Full text and rfc822 format available.

Message #27 received at 595063@bugs.debian.org (full text, mbox):

From: Steve Schnepp <steve.schnepp@gmail.com>
To: dash@vger.kernel.org
Cc: 595063@bugs.debian.org
Subject: Re: read() builtin doesn't read integer value /proc files (but bash's does)
Date: Thu, 2 Sep 2010 17:02:55 +0200
[Message part 1 (text/plain, inline)]
2010/9/1 Steve Schnepp <steve.schnepp@gmail.com>:
> conforming to POSIX isn't a realistic option, would it be possible to
> have a workaround that doesn't involve an external tool like cat(1) ?

Hi, I just hacked & attached a little patch away to be able to solve this case.
Feel free to reply with your comments.

NB: I just targeted dash-0.5.5.1, but it might apply to any version.

--
Steve Schnepp
http://blog.pwkf.org/
[chunked_bltin_read.diff (application/octet-stream, attachment)]

Added tag(s) patch. Request was from Steve Schnepp <steve.schnepp@gmail.com> to control@bugs.debian.org. (Thu, 02 Sep 2010 15:18:02 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Gerrit Pape <pape@smarden.org>:
Bug#595063; Package dash. (Thu, 02 Sep 2010 19:18:10 GMT) Full text and rfc822 format available.

Acknowledgement sent to Jilles Tjoelker <jilles@stack.nl>:
Extra info received and forwarded to list. Copy sent to Gerrit Pape <pape@smarden.org>. (Thu, 02 Sep 2010 19:18:10 GMT) Full text and rfc822 format available.

Message #34 received at 595063@bugs.debian.org (full text, mbox):

From: Jilles Tjoelker <jilles@stack.nl>
To: Steve Schnepp <steve.schnepp@gmail.com>
Cc: dash@vger.kernel.org, 595063@bugs.debian.org
Subject: Re: read() builtin doesn't read integer value /proc files (but bash's does)
Date: Thu, 2 Sep 2010 21:09:47 +0200
On Wed, Sep 01, 2010 at 10:10:11AM +0200, Steve Schnepp wrote:
> Hi, I opened bug 595063 on the debian BTS [1] and I was suggested to
> resend the email upstream.

> So I copied the body of the bug below :

> dash's read() builtin seems to read the underlying file 1 char at a
> time. This doesn't work with some files under /proc, since procfs isn't
> fully POSIX compliant.

> [snip]

> After a little digging, it only appears on files that contains just an
> integer value. When asked to read with a non-null offset (*ppos != 0),
> __do_proc_dointvec() just returns 0 (meaning an EOF) as shown on [2].

> I'm aware that the issue isn't strictly a dash one, since it has the
> right to read one character at a time. But since fixing procfs to be
> conforming to POSIX isn't a realistic option, would it be possible to
> have a workaround that doesn't involve an external tool like cat(1) ?

Given that other files in /proc do work, I don't see why the ones that
only contain an integer value cannot be fixed. All the necessary state
to produce the second and further bytes is available.

Choosing a powerful abstraction like a regular file has its
implications.

Note that a change in the file between the single-byte reads will cause
an inconsistent value to be read. This is also the case with regular
files on a filesystem, so it is acceptable.

If single-byte reads are really unacceptable, then the proper way to
read these files needs to be documented, and clear violations that will
not work properly should cause an error (in this case, this means that
reading one byte from offset 0 should fail like reading one byte from
offset 1 does).

-- 
Jilles Tjoelker




Information forwarded to debian-bugs-dist@lists.debian.org, Gerrit Pape <pape@smarden.org>:
Bug#595063; Package dash. (Fri, 03 Sep 2010 09:27:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Steve Schnepp <steve.schnepp@gmail.com>:
Extra info received and forwarded to list. Copy sent to Gerrit Pape <pape@smarden.org>. (Fri, 03 Sep 2010 09:27:03 GMT) Full text and rfc822 format available.

Message #39 received at 595063@bugs.debian.org (full text, mbox):

From: Steve Schnepp <steve.schnepp@gmail.com>
To: Jilles Tjoelker <jilles@stack.nl>
Cc: dash@vger.kernel.org, 595063@bugs.debian.org
Subject: Re: read() builtin doesn't read integer value /proc files (but bash's does)
Date: Fri, 3 Sep 2010 11:23:21 +0200
2010/9/2 Jilles Tjoelker <jilles@stack.nl>:

Thanks for your prompt reply.

> Note that a change in the file between the single-byte reads will cause
> an inconsistent value to be read. This is also the case with regular
> files on a filesystem, so it is acceptable.

Are you implying that:
- if the procfs is made to support char per char reads, dash reading
an inconsistent value is actually a feature ?
- buffering should, therefore, always be explicit ?

On a side note, the whole procfs seems to be designed around one
unique page read if possible (1x 4K).
I think it does so in order to be able to vastly simplify its
usage/implementation by kernel modules.

> If single-byte reads are really unacceptable, then the proper way to
> read these files needs to be documented, and clear violations that will
> not work properly should cause an error (in this case, this means that
> reading one byte from offset 0 should fail like reading one byte from
> offset 1 does).

+1 for "the proper way to read these files needs to be documented" and
I also think that emitting an error would be better than silently
returning erroneous data. [ EOVERFLOW is coming to my mind ]

--
Steve Schnepp
http://blog.pwkf.org/




Information forwarded to debian-bugs-dist@lists.debian.org, Gerrit Pape <pape@smarden.org>:
Bug#595063; Package dash. (Fri, 03 Sep 2010 21:27:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Jilles Tjoelker <jilles@stack.nl>:
Extra info received and forwarded to list. Copy sent to Gerrit Pape <pape@smarden.org>. (Fri, 03 Sep 2010 21:27:03 GMT) Full text and rfc822 format available.

Message #44 received at 595063@bugs.debian.org (full text, mbox):

From: Jilles Tjoelker <jilles@stack.nl>
To: Steve Schnepp <steve.schnepp@gmail.com>
Cc: dash@vger.kernel.org, 595063@bugs.debian.org
Subject: Re: read() builtin doesn't read integer value /proc files (but bash's does)
Date: Fri, 3 Sep 2010 23:25:05 +0200
On Thu, Sep 02, 2010 at 05:02:55PM +0200, Steve Schnepp wrote:
> 2010/9/1 Steve Schnepp <steve.schnepp@gmail.com>:
> > conforming to POSIX isn't a realistic option, would it be possible to
> > have a workaround that doesn't involve an external tool like cat(1) ?

> Hi, I just hacked & attached a little patch away to be able to solve
> this case.
> Feel free to reply with your comments.

> NB: I just targeted dash-0.5.5.1, but it might apply to any version.

This patch assumes that the file descriptor is discarded afterwards (its
position does not matter). Therefore the very common construct
  while read x; do
    ...
  done
stops working.

A possible fix is to check first if the input supports seeking. If it
does, use the buffering and at the end of the line seek backwards for
the number of bytes remaining in the buffer. If it does not, read one
byte at a time.

-- 
Jilles Tjoelker




Information forwarded to debian-bugs-dist@lists.debian.org, Gerrit Pape <pape@smarden.org>:
Bug#595063; Package dash. (Sat, 04 Sep 2010 18:24:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Steve Schnepp <steve.schnepp@gmail.com>:
Extra info received and forwarded to list. Copy sent to Gerrit Pape <pape@smarden.org>. (Sat, 04 Sep 2010 18:24:03 GMT) Full text and rfc822 format available.

Message #49 received at 595063@bugs.debian.org (full text, mbox):

From: Steve Schnepp <steve.schnepp@gmail.com>
To: Jilles Tjoelker <jilles@stack.nl>
Cc: dash@vger.kernel.org, 595063@bugs.debian.org
Subject: Re: read() builtin doesn't read integer value /proc files (but bash's does)
Date: Sat, 4 Sep 2010 20:20:33 +0200
[Message part 1 (text/plain, inline)]
2010/9/3 Jilles Tjoelker <jilles@stack.nl>:
> This patch assumes that the file descriptor is discarded afterwards (its
> position does not matter). Therefore the very common construct
>  while read x; do
>    ...
>  done
> stops working.

Ohh.. thanks for that, I didn't see it.

Actually "while read x" continues to work.
But "reopening the file" doesn't as in :

read a b < datafile
echo ${a} ${b}
read a b < datafile
echo ${a} ${b}

I attached an updated patch that corrects this pb by discarding the
buffer when opening a new file.
I also put everything in new files (bufreadcmd.c & .h), in order to
ease its understanding.

--
Steve Schnepp
http://blog.pwkf.org/


>
> A possible fix is to check first if the input supports seeking. If it
> does, use the buffering and at the end of the line seek backwards for
> the number of bytes remaining in the buffer. If it does not, read one
> byte at a time.
>
> --
> Jilles Tjoelker
>


--
Steve Schnepp
http://blog.pwkf.org/
[dash_readbuf.diff (text/x-patch, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, Gerrit Pape <pape@smarden.org>:
Bug#595063; Package dash. (Sat, 04 Sep 2010 19:36:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to Jilles Tjoelker <jilles@stack.nl>:
Extra info received and forwarded to list. Copy sent to Gerrit Pape <pape@smarden.org>. (Sat, 04 Sep 2010 19:36:05 GMT) Full text and rfc822 format available.

Message #54 received at 595063@bugs.debian.org (full text, mbox):

From: Jilles Tjoelker <jilles@stack.nl>
To: Steve Schnepp <steve.schnepp@gmail.com>
Cc: dash@vger.kernel.org, 595063@bugs.debian.org
Subject: Re: read() builtin doesn't read integer value /proc files (but bash's does)
Date: Sat, 4 Sep 2010 21:35:04 +0200
On Sat, Sep 04, 2010 at 08:20:33PM +0200, Steve Schnepp wrote:
> 2010/9/3 Jilles Tjoelker <jilles@stack.nl>:
> > This patch assumes that the file descriptor is discarded afterwards (its
> > position does not matter). Therefore the very common construct
> >  while read x; do
> >    ...
> >  done
> > stops working.

> Ohh.. thanks for that, I didn't see it.

> Actually "while read x" continues to work.
> But "reopening the file" doesn't as in :

> read a b < datafile
> echo ${a} ${b}
> read a b < datafile
> echo ${a} ${b}

You're right, it's even stranger than I expected.

> I attached an updated patch that corrects this pb by discarding the
> buffer when opening a new file.

This discarding is still bad as it throws away valid data if the open
file description is shared. This happens if stdin is redirected inside a
while read... loop.

Furthermore, I think constructions like
  read x; cat
and
  read x; (read y); read z
should keep working. This requires that the input's file position be
synced whenever another process may see it (fork/exit). Due to the
highly dynamic character of the shell and the common use of fd 0, this
probably means that you can't do better than syncing after each read
builtin. (For example, 'read' could be overridden with a function after
the third line.)

Another thought:
  exec 3<&0; read x; read y <&3
or even
  sh -c 'read x; read y <&3' 3<&0
Different file descriptors may refer to the same open file description
and the shell may not know this.

-- 
Jilles Tjoelker




Information forwarded to debian-bugs-dist@lists.debian.org, Gerrit Pape <pape@smarden.org>:
Bug#595063; Package dash. (Sun, 28 Nov 2010 09:27:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Herbert Xu <herbert@gondor.apana.org.au>:
Extra info received and forwarded to list. Copy sent to Gerrit Pape <pape@smarden.org>. (Sun, 28 Nov 2010 09:27:03 GMT) Full text and rfc822 format available.

Message #59 received at 595063@bugs.debian.org (full text, mbox):

From: Herbert Xu <herbert@gondor.apana.org.au>
To: Jilles Tjoelker <jilles@stack.nl>
Cc: Steve Schnepp <steve.schnepp@gmail.com>, dash@vger.kernel.org, 595063@bugs.debian.org
Subject: Re: read() builtin doesn't read integer value /proc files (but bash's does)
Date: Sun, 28 Nov 2010 16:42:19 +0800
On Sat, Sep 04, 2010 at 07:35:04PM +0000, Jilles Tjoelker wrote:
> 
> > I attached an updated patch that corrects this pb by discarding the
> > buffer when opening a new file.
> 
> This discarding is still bad as it throws away valid data if the open
> file description is shared. This happens if stdin is redirected inside a

I'm with Jilles on this.  I also don't particularly feel like
bloating dash just because of the borked /proc interface when
there is a perfectly adequate work-around in "cat".

	value=$(cat /proc/file)

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt




Information forwarded to debian-bugs-dist@lists.debian.org, Gerrit Pape <pape@smarden.org>:
Bug#595063; Package dash. (Wed, 15 Dec 2010 10:00:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Cristian Ionescu-Idbohrn <cristian.ionescu-idbohrn@axis.com>:
Extra info received and forwarded to list. Copy sent to Gerrit Pape <pape@smarden.org>. (Wed, 15 Dec 2010 10:00:02 GMT) Full text and rfc822 format available.

Message #64 received at 595063@bugs.debian.org (full text, mbox):

From: Cristian Ionescu-Idbohrn <cristian.ionescu-idbohrn@axis.com>
To: Herbert Xu <herbert@gondor.apana.org.au>
Cc: dash@vger.kernel.org, 595063@bugs.debian.org
Subject: Re: read() builtin doesnt read integer value /proc files (but bashs does)
Date: Wed, 15 Dec 2010 10:49:30 +0100 (CET)
On Sun, 28 Nov 2010, Herbert Xu wrote:
> On Sat, Sep 04, 2010 at 07:35:04PM +0000, Jilles Tjoelker wrote:
> >
> > > I attached an updated patch that corrects this pb by discarding the
> > > buffer when opening a new file.
> >
> > This discarding is still bad as it throws away valid data if the open
> > file description is shared. This happens if stdin is redirected inside a
>
> I'm with Jilles on this.  I also don't particularly feel like
> bloating dash just because of the borked /proc interface when
> there is a perfectly adequate work-around in "cat".
>
> 	value=$(cat /proc/file)

I wouldn't call that "a perfectly adequate work-around", but a painful and
unadequate work-around.  And this example will hopefully show why:

$ dash -c 'loops=10000; while [ $loops -gt 0 ];do read MAX
</proc/sys/kernel/pid_max; loops=$(($loops - 1)); done; times'
0m0.180000s 0m0.100000s
0m0.000000s 0m0.000000s

total: 0.28s

$ dash -c 'loops=10000; while [ $loops -gt 0 ];do MAX=$(cat
/proc/sys/kernel/pid_max); loops=$(($loops - 1)); done; times'
0m0.280000s 0m1.330000s
0m3.840000s 0m1.560000s

total: 7.01s

That is, the first example is 24x more efficient than the second.  And
that realy _matters_, I would say.


Cheers,

-- 
Cristian




Information forwarded to debian-bugs-dist@lists.debian.org, Gerrit Pape <pape@smarden.org>:
Bug#595063; Package dash. (Wed, 15 Dec 2010 19:00:06 GMT) Full text and rfc822 format available.

Acknowledgement sent to Jonathan Nieder <jrnieder@gmail.com>:
Extra info received and forwarded to list. Copy sent to Gerrit Pape <pape@smarden.org>. (Wed, 15 Dec 2010 19:00:06 GMT) Full text and rfc822 format available.

Message #69 received at 595063@bugs.debian.org (full text, mbox):

From: Jonathan Nieder <jrnieder@gmail.com>
To: Cristian Ionescu-Idbohrn <cristian.ionescu-idbohrn@axis.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>, dash@vger.kernel.org, 595063@bugs.debian.org
Subject: Re: read() builtin doesnt read integer value /proc files (but bashs does)
Date: Wed, 15 Dec 2010 12:55:51 -0600
Hi Cristian,

Cristian Ionescu-Idbohrn wrote:
> On Sun, 28 Nov 2010, Herbert Xu wrote:
> > On Sat, Sep 04, 2010 at 07:35:04PM +0000, Jilles Tjoelker wrote:

>>> This discarding is still bad as it throws away valid data if the open
>>> file description is shared. This happens if stdin is redirected inside a
>>
>> I'm with Jilles on this.  I also don't particularly feel like
>> bloating dash just because of the borked /proc interface when
>> there is a perfectly adequate work-around in "cat".
>>
>> 	value=$(cat /proc/file)
>
> I wouldn't call that "a perfectly adequate work-around", but a painful and
> unadequate work-around.

For what it's worth, here's what bash does (based on strace):

1. Determine whether the file is seekable.  That is, seek using
SEEK_CUR with offset 0.

2. If seekable, read a nice big chunk and then seek back to put the
file offset back in the right place.  If not seekable, read one byte
at a time.

This works in /proc because files in /proc are seekable.

That said, I don't think borked /proc is a great reason to do this
(it's a better reason to fix /proc).  Speeding up the read builtin
might be a good reason.

Regards,
Jonathan




Information forwarded to debian-bugs-dist@lists.debian.org, Gerrit Pape <pape@smarden.org>:
Bug#595063; Package dash. (Wed, 15 Dec 2010 19:15:06 GMT) Full text and rfc822 format available.

Acknowledgement sent to Cristian Ionescu-Idbohrn <cristian.ionescu-idbohrn@axis.com>:
Extra info received and forwarded to list. Copy sent to Gerrit Pape <pape@smarden.org>. (Wed, 15 Dec 2010 19:15:06 GMT) Full text and rfc822 format available.

Message #74 received at 595063@bugs.debian.org (full text, mbox):

From: Cristian Ionescu-Idbohrn <cristian.ionescu-idbohrn@axis.com>
To: dash@vger.kernel.org
Cc: 595063@bugs.debian.org
Subject: Re: read() builtin doesnt read integer value /proc files (but bashs does)
Date: Wed, 15 Dec 2010 20:12:35 +0100 (CET)
Jonathan,

On Wed, 15 Dec 2010, Jonathan Nieder wrote:
> Cristian Ionescu-Idbohrn wrote:
> > On Sun, 28 Nov 2010, Herbert Xu wrote:
> >>
> >> I'm with Jilles on this.  I also don't particularly feel like
> >> bloating dash just because of the borked /proc interface when
> >> there is a perfectly adequate work-around in "cat".
> >>
> >> 	value=$(cat /proc/file)
> >
> > I wouldn't call that "a perfectly adequate work-around", but a painful
> > and unadequate work-around.
>
> This works in /proc because files in /proc are seekable.
>
> That said, I don't think borked /proc is a great reason to do this
> (it's a better reason to fix /proc).  Speeding up the read builtin
> might be a good reason.

Right.  So, there are 2 options here.  One is to to make dash work like
bash on a proc filesystem, the other to "fix" the kernel.

How many linux distributions depend on a "working" dash?
Which alternative is the more realistic one?
What are the ETAs odds?
How do we proceed?


Cheers,

-- 
Cristian




Information forwarded to debian-bugs-dist@lists.debian.org, Gerrit Pape <pape@smarden.org>:
Bug#595063; Package dash. (Sat, 18 Dec 2010 22:33:06 GMT) Full text and rfc822 format available.

Acknowledgement sent to Jilles Tjoelker <jilles@stack.nl>:
Extra info received and forwarded to list. Copy sent to Gerrit Pape <pape@smarden.org>. (Sat, 18 Dec 2010 22:33:06 GMT) Full text and rfc822 format available.

Message #79 received at 595063@bugs.debian.org (full text, mbox):

From: Jilles Tjoelker <jilles@stack.nl>
To: Jonathan Nieder <jrnieder@gmail.com>
Cc: Cristian Ionescu-Idbohrn <cristian.ionescu-idbohrn@axis.com>, Herbert Xu <herbert@gondor.apana.org.au>, dash@vger.kernel.org, 595063@bugs.debian.org
Subject: Re: read() builtin doesnt read integer value /proc files (but bashs does)
Date: Sat, 18 Dec 2010 23:23:44 +0100
On Wed, Dec 15, 2010 at 12:55:51PM -0600, Jonathan Nieder wrote:
> Cristian Ionescu-Idbohrn wrote:
> > On Sun, 28 Nov 2010, Herbert Xu wrote:
> > > On Sat, Sep 04, 2010 at 07:35:04PM +0000, Jilles Tjoelker wrote:

> >>> This discarding is still bad as it throws away valid data if the open
> >>> file description is shared. This happens if stdin is redirected inside a

> >> I'm with Jilles on this.  I also don't particularly feel like
> >> bloating dash just because of the borked /proc interface when
> >> there is a perfectly adequate work-around in "cat".

> >> 	value=$(cat /proc/file)

> > I wouldn't call that "a perfectly adequate work-around", but a painful and
> > unadequate work-around.

> For what it's worth, here's what bash does (based on strace):

> 1. Determine whether the file is seekable.  That is, seek using
> SEEK_CUR with offset 0.

> 2. If seekable, read a nice big chunk and then seek back to put the
> file offset back in the right place.  If not seekable, read one byte
> at a time.

> This works in /proc because files in /proc are seekable.

> That said, I don't think borked /proc is a great reason to do this
> (it's a better reason to fix /proc).  Speeding up the read builtin
> might be a good reason.

The optimization is of limited benefit (still way more syscalls than
strictly necessary) and does not apply to the common use case of reading
from a pipe. Generally, if 'read' is too slow, it is better to spend a
fork on a tool like grep, sed or awk which processes large amounts of
text much more efficiently.

As for value=$(cat /proc/file), there are at least two ways to make this
faster. The traditional ksh way is the extension value=$(</proc/file)
which is permitted but not required by POSIX. I do not really like this
as it makes the scripts not POSIX compliant. In recent mksh I noticed
another way: by making cat(1) a builtin under certain circumstances.
These circumstances include the absence of options (other than "--") and
should probably also exclude foreground commands in interactive job
control shells (as builtins cannot be suspended). To avoid needing to
implement extensions like FreeBSD cat's ability to read from Unix domain
sockets, named files could also be excluded, requiring value=$(cat
</proc/file).

-- 
Jilles Tjoelker




Removed tag(s) patch. Request was from Jonathan Nieder <jrnieder@gmail.com> to control@bugs.debian.org. (Mon, 27 Dec 2010 01:09:03 GMT) Full text and rfc822 format available.

Bug 595063 cloned as bug 608082. Request was from Jonathan Nieder <jrnieder@gmail.com> to control@bugs.debian.org. (Mon, 27 Dec 2010 01:09:03 GMT) Full text and rfc822 format available.

Message sent on to Steve Schnepp <steve.schnepp@gmail.com>:
Bug#595063. (Mon, 27 Dec 2010 01:09:08 GMT) Full text and rfc822 format available.

Message #86 received at 595063-submitter@bugs.debian.org (full text, mbox):

From: Jonathan Nieder <jrnieder@gmail.com>
To: Jilles Tjoelker <jilles@stack.nl>
Cc: 595063-submitter@bugs.debian.org
Subject: Re: read() builtin doesnt read integer value /proc files (but bashs does)
Date: Sun, 26 Dec 2010 19:04:31 -0600
tags 595063 - patch
clone 595063 -1
retitle -1 dash: build in cat (no arguments case)
severity -1 wishlist
quit

Jilles Tjoelker wrote:

> As for value=$(cat /proc/file), there are at least two ways to make this
> faster.
[...]
> In recent mksh I noticed
> another way: by making cat(1) a builtin under certain circumstances.
> These circumstances include the absence of options (other than "--") and
> should probably also exclude foreground commands in interactive job
> control shells (as builtins cannot be suspended). To avoid needing to
> implement extensions like FreeBSD cat's ability to read from Unix domain
> sockets, named files could also be excluded, requiring value=$(cat
> </proc/file).

I like this idea a lot.  Cloning the bug so it isn't forgotten.




Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Fri Apr 25 01:57:44 2014; Machine Name: beach.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.