Debian Bug report logs - #429021
libc6: fputs can lose data in buffer on signal

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Dmitry Potapov <dpotapov@gmail.com>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: libc6: fputs can lose data in buffer on signal

Date: Fri, 15 Jun 2007 17:10:59 +0400

Package: libc6
Version: 2.3.6.ds1-13
Severity: important


fputs (and probably other FILE based output functions) can lose
previously written data that were accumulated in the user space
buffer when a signal arrives. Here is an example that demonstrates
the problem:

# cat t.c
////////////////////////////////////////
#include <stdio.h>
#include <signal.h>
#include <unistd.h>
#include <string.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <unistd.h>


void handler(int sig)
{
    char buffer[80];

    snprintf(buffer, sizeof(buffer), "signal handler called, sig=%d\n", sig);
    write(2, buffer, strlen(buffer));
}

static int main_pid;

#define MYSIGNUM SIGINT

int reader(int fd)
{
    char buffer[1024];
    ssize_t count;
    size_t num_bytes, num_lines;

    sleep (1);
    kill (main_pid, MYSIGNUM);
    sleep (1);

    num_bytes = num_lines = 0;
    while ((count = read(fd, buffer, sizeof(buffer))) > 0)
    {
        char* p;
        for (p=buffer; p!=buffer+count; p++)
            if (*p =='\n')
                num_lines++;
        num_bytes += count;
    }
    if (count < 0)
    {
        perror ("read");
        return 1;
    }
    printf ("reader: num_bytes=%d num_lines=%d\n", num_bytes, num_lines);
    if (num_bytes != 8*10000) {
    	printf ("reader: number of missing bytes: %d\n", 80000 - num_bytes);
	return 1;
    }
    close(fd);
    return 0;
}

int writer(int fd)
{
    int i;
    size_t num_bytes, num_lines;
    FILE* f = fdopen(fd, "w");
    if (f == NULL) {
	    perror ("fdopen");
	    return 1;
    }
    num_bytes = num_lines = 0;
    for (i=0; i<10001; i++)
        if (fputs ("test...\n", f) == EOF)
        {
            printf ("error at num_bytes=%d\n", num_bytes);
            perror("fputs");
        }
	else
	{
		num_bytes += 8;
		num_lines ++;
	}
    if (fclose(f) == EOF) {
	    perror("fclose");
	    return 1;
    }
    printf ("writer: num_bytes=%d num_lines=%d\n", num_bytes, num_lines);
    if (num_bytes != 8*10000) {
    	printf ("writer: expected num_bytes=80000 but was %d\n", num_bytes);
	return 1;
    }
    return 0;
}

int main() {
    struct sigaction sa;
    int filedes[2];
    pid_t pid;
    int ret;

    main_pid = getpid();

    sa.sa_handler = handler;
    sigemptyset(&sa.sa_mask);
    sa.sa_flags = 0;
    sigaction(MYSIGNUM, &sa, NULL);

    if (pipe(filedes))
    {
        perror("pipe");
	    return 1;
    }

    pid=fork();
    if (pid == -1)
    {
        perror("fork");
        return 1;
    }
    if (pid==0)
    {
        close(filedes[1]);
        return reader(filedes[0]);
    }
    else
    {
        int status;
        close(filedes[0]);
        ret = writer(filedes[1]);
        wait(&status);
        return (ret==0 && status==0) ? 0 : 1;
    }
}

////////////////////////////////////////

# gcc -W -Wall t.c -o t && ./t
signal handler called, sig=2
error at num_bytes=69632
fputs: Interrupted system call
writer: num_bytes=80000 num_lines=10000
reader: num_bytes=75904 num_lines=9488
reader: number of missing bytes: 4096


-- System Information:
Debian Release: 4.0
  APT prefers stable
  APT policy: (500, 'stable')
Architecture: i386 (i686)
Shell:  /bin/sh linked to /bin/bash
Kernel: Linux 2.6.18-4-k7
Locale: LANG=ru_RU.UTF-8, LC_CTYPE=ru_RU.UTF-8 (charmap=UTF-8)

Versions of packages libc6 depends on:
ii  tzdata                        2007b-1    Time Zone and Daylight Saving Time

libc6 recommends no packages.

-- no debconf information

Message #10 received at 429021-done@bugs.debian.org (full text, mbox, reply):

From: Pierre Habouzit <madcoder@debian.org>

To: Dmitry Potapov <dpotapov@gmail.com>, 429021-done@bugs.debian.org

Subject: Re: Bug#429021: libc6: fputs can lose data in buffer on signal

Date: Fri, 15 Jun 2007 18:09:42 +0200

[Message part 1 (text/plain, inline)]

On Fri, Jun 15, 2007 at 05:10:59PM +0400, Dmitry Potapov wrote:
> Package: libc6
> Version: 2.3.6.ds1-13
> Severity: important
> 
> 
> fputs (and probably other FILE based output functions) can lose
> previously written data that were accumulated in the user space
> buffer when a signal arrives. Here is an example that demonstrates
> the problem:
> 
> # cat t.c
> ////////////////////////////////////////
> #include <stdio.h>
> #include <signal.h>
> #include <unistd.h>
> #include <string.h>
> #include <errno.h>
> #include <sys/types.h>
> #include <sys/wait.h>
> #include <unistd.h>
> 
> 
> void handler(int sig)
> {
>     char buffer[80];
> 
>     snprintf(buffer, sizeof(buffer), "signal handler called, sig=%d\n", sig);
>     write(2, buffer, strlen(buffer));
> }
> 
> static int main_pid;
> 
> #define MYSIGNUM SIGINT
> 
> int reader(int fd)
> {
>     char buffer[1024];
>     ssize_t count;
>     size_t num_bytes, num_lines;
> 
>     sleep (1);
>     kill (main_pid, MYSIGNUM);
>     sleep (1);
> 
>     num_bytes = num_lines = 0;
>     while ((count = read(fd, buffer, sizeof(buffer))) > 0)
>     {
>         char* p;
>         for (p=buffer; p!=buffer+count; p++)
>             if (*p =='\n')
>                 num_lines++;
>         num_bytes += count;
>     }
>     if (count < 0)
>     {
>         perror ("read");
>         return 1;
>     }
>     printf ("reader: num_bytes=%d num_lines=%d\n", num_bytes, num_lines);
>     if (num_bytes != 8*10000) {
>     	printf ("reader: number of missing bytes: %d\n", 80000 - num_bytes);
> 	return 1;
>     }
>     close(fd);
>     return 0;
> }
> 
> int writer(int fd)
> {
>     int i;
>     size_t num_bytes, num_lines;
>     FILE* f = fdopen(fd, "w");
>     if (f == NULL) {
> 	    perror ("fdopen");
> 	    return 1;
>     }
>     num_bytes = num_lines = 0;
>     for (i=0; i<10001; i++)
>         if (fputs ("test...\n", f) == EOF)
>         {
>             printf ("error at num_bytes=%d\n", num_bytes);
>             perror("fputs");
>         }
> 	else
> 	{
> 		num_bytes += 8;
> 		num_lines ++;
> 	}
>     if (fclose(f) == EOF) {
> 	    perror("fclose");
> 	    return 1;
>     }
>     printf ("writer: num_bytes=%d num_lines=%d\n", num_bytes, num_lines);
>     if (num_bytes != 8*10000) {
>     	printf ("writer: expected num_bytes=80000 but was %d\n", num_bytes);
> 	return 1;
>     }
>     return 0;
> }
> 
> int main() {
>     struct sigaction sa;
>     int filedes[2];
>     pid_t pid;
>     int ret;
> 
>     main_pid = getpid();
> 
>     sa.sa_handler = handler;
>     sigemptyset(&sa.sa_mask);
>     sa.sa_flags = 0;

  The bug is here, you should set the flags to SA_RESTART so that
interupted syscalls are restarted. If you don't do so you'll break the
libio.

  you can check using sigaction to fetch default state that SA_RESTART
is the default state for sa_flags, so you are knowingly breaking your
application.

  THe porable way to ignore a signal is to :

  sigaction(SIGINT, NULL, &sa);
  sa.sa_handler = your_handler;
  sigempty(sa.sa_mask);
  sigaction(SIGINT, &sa, NULL);

  with that, when the kernel return -ESTARTSYS syscalls are restarted.


-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org

[Message part 2 (application/pgp-signature, inline)]

Message #17 received at 429021@bugs.debian.org (full text, mbox, reply):

From: Daniel Jacobowitz <drow@false.org>

To: Pierre Habouzit <madcoder@debian.org>

Cc: Dmitry Potapov <dpotapov@gmail.com>, 429021@bugs.debian.org

Subject: Re: Bug#429021: libc6: fputs can lose data in buffer on signal

Date: Fri, 15 Jun 2007 12:33:40 -0400

On Fri, Jun 15, 2007 at 06:09:42PM +0200, Pierre Habouzit wrote:
>   The bug is here, you should set the flags to SA_RESTART so that
> interupted syscalls are restarted. If you don't do so you'll break the
> libio.

Is this documented anywhere?  The manual says that EINTR is treated as
an ordinary error, which would be fine; they don't say EINTR will lose
your data.

This seems like a bug to me unless you can find language in POSIX to
the contrary.

-- 
Daniel Jacobowitz
CodeSourcery

Message #22 received at 429021@bugs.debian.org (full text, mbox, reply):

From: Pierre Habouzit <madcoder@debian.org>

To: Daniel Jacobowitz <drow@false.org>

Cc: Dmitry Potapov <dpotapov@gmail.com>, 429021@bugs.debian.org

Subject: Re: Bug#429021: libc6: fputs can lose data in buffer on signal

Date: Fri, 15 Jun 2007 18:47:55 +0200

[Message part 1 (text/plain, inline)]

reopen 429021
thanks

On Fri, Jun 15, 2007 at 12:33:40PM -0400, Daniel Jacobowitz wrote:
> On Fri, Jun 15, 2007 at 06:09:42PM +0200, Pierre Habouzit wrote:
> >   The bug is here, you should set the flags to SA_RESTART so that
> > interupted syscalls are restarted. If you don't do so you'll break the
> > libio.
> 
> Is this documented anywhere?  The manual says that EINTR is treated as
> an ordinary error, which would be fine; they don't say EINTR will lose
> your data.

  Nothing in POSIX says what happens wrt to fwrite and signals. At least
it's very unclear. But indeed the fact that SA_RESTART is the default
value for linux does not seem to be a POSIX requirement either.

> This seems like a bug to me unless you can find language in POSIX to
> the contrary.

  Maybe. I'm not sure.

  I'll open a bug on sourceware.org soon then. We'll see what Uli says,
if he's in a good mood enough...

-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org

[Message part 2 (application/pgp-signature, inline)]

Message #29 received at 429021@bugs.debian.org (full text, mbox, reply):

From: Dmitry Potapov <dpotapov@gmail.com>

To: Pierre Habouzit <madcoder@debian.org>

Cc: Daniel Jacobowitz <drow@false.org>, 429021@bugs.debian.org

Subject: Re: Bug#429021: libc6: fputs can lose data in buffer on signal

Date: Mon, 18 Jun 2007 12:44:02 +0400

On Fri, Jun 15, 2007 at 06:47:55PM +0200, Pierre Habouzit wrote:
>   Nothing in POSIX says what happens wrt to fwrite and signals. At least
> it's very unclear. But indeed the fact that SA_RESTART is the default
> value for linux does not seem to be a POSIX requirement either.

I have studied the source today, and it turned out that the problem
is not signal specific, but how glibc treats any error on writing.
Unfortunately, it loses all data stored in the buffer. Probably,
it does not matter for fatal errors, because you will not able to
write more anyway, but for recoverable errors like EINTR or ENOSPC,
it is not a smart thing to do. Here is a small patch that corrects
this problem:

 
--- glibc-2.3.6/libio/fileops.c.orig	2004-12-07 01:36:56.000000000 +0300
+++ glibc-2.3.6/libio/fileops.c	2007-06-18 12:29:50.739890036 +0400
@@ -516,10 +516,19 @@ new_do_write (fp, data, to_do)
     fp->_cur_column = INTUSE(_IO_adjust_column) (fp->_cur_column - 1, data,
 						 count) + 1;
   _IO_setg (fp, fp->_IO_buf_base, fp->_IO_buf_base, fp->_IO_buf_base);
-  fp->_IO_write_base = fp->_IO_write_ptr = fp->_IO_buf_base;
-  fp->_IO_write_end = (fp->_mode <= 0
+  if (__builtin_expect (count == to_do, 1))
+    {
+      fp->_IO_write_base = fp->_IO_write_ptr = fp->_IO_buf_base;
+      fp->_IO_write_end = (fp->_mode <= 0
 		       && (fp->_flags & (_IO_LINE_BUF+_IO_UNBUFFERED))
 		       ? fp->_IO_buf_base : fp->_IO_buf_end);
+    }
+  else if (count > 0 && fp->_IO_write_base == data)
+    {
+      memmove (data, data+count, 
+               fp->_IO_write_ptr - fp->_IO_write_base - count);
+      fp->_IO_write_ptr -= count;
+    }
   return count;
 }
 
 
The patch does not change the behavior of new_do_write in the case
of success, but when an error happens, I do not emptify the output
buffer. In fact, in the case when data were partly written (count>0)
and data were from the output buffer (fp->_IO_write_base == data),
I remove written data from the buffer and adjust the write pointer
accordingly.

Warning: I have not tested this patch, but it seems trivial, so
I hope it works.

Also, I should note that though this patch prevents losing buffer
on error, it does not prevent the possibility that fputs can return
an error, while a part of the given string was written, but it is
restriction of the interface. So using fputs with non-restartable
signals is not a good idea anyway, still fwrite can be used in
this way for writing bytes if this bug with losing data in buffer
is fixed.

Thanks,
Dmitry

Message #38 received at 429021-close@bugs.debian.org (full text, mbox, reply):

From: Clint Adams <schizo@debian.org>

To: 429021-close@bugs.debian.org

Subject: Bug#429021: fixed in glibc 2.6-1

Date: Sat, 07 Jul 2007 22:47:03 +0000

Source: glibc
Source-Version: 2.6-1

We believe that the bug you reported is fixed in the latest version of
glibc, which is due to be installed in the Debian FTP archive:

glibc-doc_2.6-1_all.deb
  to pool/main/g/glibc/glibc-doc_2.6-1_all.deb
glibc_2.6-1.diff.gz
  to pool/main/g/glibc/glibc_2.6-1.diff.gz
glibc_2.6-1.dsc
  to pool/main/g/glibc/glibc_2.6-1.dsc
locales_2.6-1_all.deb
  to pool/main/g/glibc/locales_2.6-1_all.deb



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 429021@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Clint Adams <schizo@debian.org> (supplier of updated glibc package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.7
Date: Sat, 07 Jul 2007 09:43:02 -0400
Source: glibc
Binary: libc0.1-prof libc6-dev-amd64 locales-all libc6-i686 libc6-dev-ppc64 libc0.3-pic glibc-doc libc0.3 libc6-dev-mipsn32 libc0.1-i686 libc0.1-i386 libc6-mips64 libc6.1-dev libc6-s390x libnss-files-udeb libc0.1-dev-i386 libc6-dev-sparc64 libc6-i386 libc0.3-dev libc6-udeb libc6-dbg libc6.1-pic libc6-dev libc0.3-prof libc6-sparcv9 libc0.1-udeb libc6-dev-i386 libc6.1-prof libc6-mipsn32 libc0.1-dev locales libc6-pic libc0.3-udeb libc6-dev-powerpc libc0.1-pic libc6-ppc64 libc0.3-dbg libc0.1-dbg libc6-amd64 libc0.1 libc6-prof libc6-xen libc6-dev-mips64 libc6-powerpc libc6 libc6-sparcv9b libc6.1-udeb libc6.1-dbg nscd libc6-sparc64 libnss-dns-udeb libc6.1 libc6-dev-s390x
Architecture: source all
Version: 2.6-1
Distribution: unstable
Urgency: low
Maintainer: GNU Libc Maintainers <debian-glibc@lists.debian.org>
Changed-By: Clint Adams <schizo@debian.org>
Description: 
 glibc-doc  - GNU C Library: Documentation
 libc6      - GNU C Library: Shared libraries
 libc6-dbg  - GNU C Library: Libraries with debugging symbols
 libc6-dev  - GNU C Library: Development Libraries and Header Files
 libc6-dev-i386 - GNU C Library: 32bit development libraries for AMD64
 libc6-i386 - GNU C Library: 32bit shared libraries for AMD64
 libc6-pic  - GNU C Library: PIC archive library
 libc6-prof - GNU C Library: Profiling Libraries
 libc6-udeb - GNU C Library: Shared libraries - udeb (udeb)
 libnss-dns-udeb - GNU C Library: NSS helper for DNS - udeb (udeb)
 libnss-files-udeb - GNU C Library: NSS helper for files - udeb (udeb)
 locales    - GNU C Library: National Language (locale) data [support]
 locales-all - GNU C Library: Precompiled locale data
 nscd       - GNU C Library: Name Service Cache Daemon
Closes: 428884 429021 431365
Changes: 
 glibc (2.6-1) unstable; urgency=low
 .
   [ Pierre Habouzit ]
   [ Clint Adams]
   * New upstream version.
    - Remove locale/iso3166-RS.diff (obsolete).
    - Remove locale/fix-exhausted-memory.diff (merged upstream).
    - Update locale/LC_COLLATE-keywords-ordering.diff.
    - Remove localedata/locale-hy_AM.diff (obsolete).
    - Remove localedata/locale-pl_PL.diff (merged upstream).
    - Remove localedata/locales-sr.diff (obsolete).
    - Update localedata/tailor-iso14651_t1.diff.
    - Update localedata/first_weekday.diff.
    - Remove alpha/cvs-cfi.diff (merged upstream).
    - Remove arm/cvs-check_pf.c (merged upstream).
    - Remove hppa/cvs-hppa-update.diff (obsolete).
    - Update hppa/submitted-nptl-carlos.diff from "upstream".
    - Remove hppa/submitted-nptl-carlos2.diff (merged upstream).
    - Remove hppa/local-r19use.diff (merged upstream).
    - Remove hurd-i386/cvs-futimes.diff (merged upstream).
    - Remove m68k/cvs-m68k-update.diff (obsolete).
    - Update m68k/local-mathinline_h.diff.
    - Remove mips/cvs-ldsodefs_h.diff (merged upstream).
    - Remove mips/submitted-msq.diff (merged upstream).
    - Remove all/cvs-iconv-E13B.diff (obsolete).
    - Remove all/submitted-new-brf-encoding.diff (merged upstream).
    - Remove any/cvs-2.5-branch-update.diff (obsolete).
    - Remove any/cvs-pow.diff (obsolete).
    - Remove any/cvs-printf_fp-c.diff (obsolete).
    - Remove any/cvs-ftw-c.diff (obsolete).
    - Remove any/cvs-bits_in_h-ipv6.diff (obsolete).
    - Remove any/cvs-itoa-c.diff (obsolete).
    - Remove any/cvs-lt-update.diff (obsolete).
    - Remove any/cvs-realpath.diff (obsolete).
    - Remove any/cvs-vfprintf-stack-smashing.diff (obsolete).
    - Remove any/cvs-zdump-64-bit.diff (obsolete).
    - Update any/local-ldso-disable-hwcap.diff.
    - Remove any/submitted-gethostbyname_r.diff (obsolete).
    - Remove any/submitted-iconv-colon.diff (merged upstream).
    - Update any/submitted-strfry.diff.
    - Remove any/submitted-unistd_XOPEN_VERSION.diff (obsolete).
    - Remove any/cvs-glob-c.diff (obsolete).
    - Remove any/cvs-scanf_hexfloat.diff (obsolete).
    - Remove alpha/submitted-sigsuspend.diff (merged upstream).
    - Remove arm/cvs-procinfo-eabi.diff (obsolete).
   * debian/sysdeps/depflags.pl: Clean out relationships for packages
     that do not exist in sarge or later.
   * debian/debhelper.in/libc.install: do not hardcode the glibc
     version number in the path to gai.conf.
   * debian/control.in/main, debian/sysdeps/depflags.pl:
     use linux-libc-dev on all linux architectures, and
     remove all references to linux-kernel-headers.
   * Bump shlibdeps version to 2.6-1 due to sync_file_range, futimens,
     utimensat, __sched_cpucount, sched_getcpu, strerror_l, and
     epoll_pwait symbols.
   * debian/rules.d/build.mk: pass --enable-profile to configure.
 .
   [ Aurelien Jarno ]
   * patches/sparc/local-undefined-registers.diff: new file to ignore
     global registers while looking for undefined symbols.
   * debian/script.in/kernelcheck.sh: add a warning for FreeBSD kernels
     5.X.
   * local/etc_init.d/glibc.sh, debhelper.in/libc.preinst: don't check for
     linux kernel, it is now done in script.in/kernelcheck.sh.
   * patches/any/cvs-malloc.diff: new patch from upstream to fix malloc ABI.
   * patches/any/local-linuxthreads-lowlevellock.diff: new patch to support
     low level locking on linuxthreads.
   * patches/any/local-linuxthreads-fatalprepare.diff: new patch to support
     FATAL_PREPARE on linuxthreads, by not using __libc_pthread_functions_init
     and PTHFCT_CALL.
   * patches/hppa/local-linuxthreads.diff: new patch to get glibc buildable on
     hppa with linuxthreads.
   * patches/arm/submitted-RTLD_SINGLE_THREAD_P.diff: fix a missing #defined
     on arm.
   * patches/localedata/submitted-as_IN.diff: new patch to fix a not anymore
     unassigned unicode code.
   * locales-depver: tighten locales dependencies.
   * debian/sysdeps/linux.mk, debian/script.in/kernelcheck.sh: bump minimum
     kernel requirement to 2.6.8.
   * debian/rules, debian/rules.d/build.mk, debian/sysdeps/*.mk: also defines
     CXX as g++ is used in the testsuite.
 .
   [ Petr Salinger]
   * kfreebsd/local-sysdeps.diff: update to revision 1949 (from glibc-bsd).
   * any/local-linuxthreads-defines.diff: new patch to restore a few defines
     in config.make.in still needed by linuxthreads.
   * patches/local-tst-mktime2.diff: fix time/tst-mktime2.c.
 .
   [ Pierre Habouzit ]
   * kernelchecks.sh: Make the warning about kernel 2.6 more explicit so that
     people can deal with it without adding yet-another critical bug on the
     glibc.
   * kernelchecks.sh: add some quotes to unconfuse syntax hilighting a bit.
   * nscd.init: ksh is confused if you call functions start or stop.
     Closes: 428884.
   * patches/locale/preprocessor-collate.diff: update it to work (with
     restrictions) with depth >= 2 copies. The patch is scurvy and make locale
     parsing completely non reentrant.
   * debian/control: ${Source-Version} -> ${binary:Version}.
   * debian/debhelper.in/nscd.lintian: yes, /var/db for lintian is ok.
   * patches/any/submitted-fileops-and-signals.diff: fixes libio file
     operations in presence of recoverable errors.
     Closes: 429021.
   * patches/localedata/fix-am_ET.diff: fix am_ET using our preprocessor
     extensions.
 .
   [ Samuel Thibault ]
   * Remove patches/hurd-i386/cvs-getsid.diff (merged upstream).
   * patches/hurd-i386/submitted-ioctl-unsigned-size_t.diff: new patch to fix
     some packages that use unsigned or size_t in ioctls.
     Closes: 431365.
Files: 
 346951849e66cca3a5914cf911c03ed2 2309 libs required glibc_2.6-1.dsc
 5bebc866c6c09210f397cd2d8c90ef86 659757 libs required glibc_2.6-1.diff.gz
 14173ca503a4640aa3bc47f76fdb721a 1628578 doc optional glibc-doc_2.6-1_all.deb
 93c21834fcf3363125fd363e35866a84 4374384 libs standard locales_2.6-1_all.deb
Package-Type: udeb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Debian!

iD8DBQFGkBUr5m0u66uWM3ARAml0AJsEoWwsfkv18AOTLYN+EuMdWwiTngCfcT0B
wb2Uo8fAFEiev5+5PwPBBiE=
=IlBp
-----END PGP SIGNATURE-----

Message #43 received at 429021@bugs.debian.org (full text, mbox, reply):

From: Stephane Chazelas <Stephane_Chazelas@yahoo.fr>

To: Andreas Schwab <schwab@suse.de>

Cc: Pierre-Philippe Coupard <pcoupard@skynet.be>, bug-bash@gnu.org, Dmitry Potapov <dpotapov@gmail.com>, 429021@bugs.debian.org

Subject: Re: builtin echo command redirection misbehaves in detached scripts when terminal is closed

Date: Sun, 9 Sep 2007 22:18:07 +0100

On Sun, Sep 09, 2007 at 10:08:14PM +0200, Andreas Schwab wrote:
> Stephane Chazelas <Stephane_Chazelas@yahoo.fr> writes:
> 
> > On Sun, Sep 09, 2007 at 07:10:59PM +0100, Stephane Chazelas wrote:
> > [...]
> >> What OS and version of glibc? I do get the error message but I
> >> get both a and b in the file.
> >> 
> >> That was on Linux, glibc 2.6.1.
> > [...]
> >
> > Actually,
> >
> > bash -c 'echo a; echo b > a' >&-
> >
> > is enough for me to reproduce the problem.
> 
> Guess you have a buggy libc, then.
[...]

I wouldn't be surprised if it has to do with the fix to debian
bug #429021. http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=429021
(I'm CCing Dmitry who is the author of that change according to
bugs.debian.org)

I was testing with debian package 2.6.1-2 that includes Dmitry's
fix for that bug. I don't know if that fix will is planned to be
included in the GNU tree, it doesn't seem it yet in the glibc
CVS repository.

Now, I'm not sure if we can say that the new glibc behavior
observed is bogus (other than it's different from the behavior
observed in all the libcs I tried with). It is not a harmless
change, for sure as it seems to have broken at least bash, zsh
and possibly ksh93.

Dmitry, you may find that whole thread at:
http://groups.google.com/group/gnu.bash.bug/browse_thread/thread/e311bdd4f945a21e/621b7189217760f1

Best regards,
Stéphane

Message #48 received at 429021@bugs.debian.org (full text, mbox, reply):

From: Pierre-Philippe Coupard <pcoupard@skynet.be>

To: Stephane Chazelas <Stephane_Chazelas@yahoo.fr>

Cc: Andreas Schwab <schwab@suse.de>, Dmitry Potapov <dpotapov@gmail.com>, bug-bash@gnu.org, 429021@bugs.debian.org

Subject: Re: builtin echo command redirection misbehaves in detached scripts when terminal is closed

Date: Sun, 09 Sep 2007 23:41:56 +0200

The change is far from trivial or harmless, if it was intended. I had to 
rebuild a custom server I run in a hurry because it was flooding an IRC 
channel with log lines a backend bash script sent to stderr. And I can 
think of plenty of ways to trash files with this bug.

Anyway, thanks a lot Stéphane and Andreas for testing this!

Stephane Chazelas wrote:
> Now, I'm not sure if we can say that the new glibc behavior
> observed is bogus (other than it's different from the behavior
> observed in all the libcs I tried with). It is not a harmless
> change, for sure as it seems to have broken at least bash, zsh
> and possibly ksh93.
>
> Dmitry, you may find that whole thread at:
> http://groups.google.com/group/gnu.bash.bug/browse_thread/thread/e311bdd4f945a21e/621b7189217760f1
>
> Best regards,
> Stéphane
>
>
>   

Message #53 received at 429021@bugs.debian.org (full text, mbox, reply):

From: Aurelien Jarno <aurelien@aurel32.net>

To: Stephane Chazelas <Stephane_Chazelas@yahoo.fr>, 429021@bugs.debian.org

Cc: Andreas Schwab <schwab@suse.de>, Pierre-Philippe Coupard <pcoupard@skynet.be>, bug-bash@gnu.org, Dmitry Potapov <dpotapov@gmail.com>

Subject: Re: Bug#429021: builtin echo command redirection misbehaves in detached scripts when terminal is closed

Date: Mon, 10 Sep 2007 00:05:57 +0200

Stephane Chazelas a écrit :
> On Sun, Sep 09, 2007 at 10:08:14PM +0200, Andreas Schwab wrote:
>> Stephane Chazelas <Stephane_Chazelas@yahoo.fr> writes:
>>
>>> On Sun, Sep 09, 2007 at 07:10:59PM +0100, Stephane Chazelas wrote:
>>> [...]
>>>> What OS and version of glibc? I do get the error message but I
>>>> get both a and b in the file.
>>>>
>>>> That was on Linux, glibc 2.6.1.
>>> [...]
>>>
>>> Actually,
>>>
>>> bash -c 'echo a; echo b > a' >&-
>>>
>>> is enough for me to reproduce the problem.
>> Guess you have a buggy libc, then.
> [...]
> 
> I wouldn't be surprised if it has to do with the fix to debian
> bug #429021. http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=429021
> (I'm CCing Dmitry who is the author of that change according to
> bugs.debian.org)
> 

I can reproduce the "bug" with glibc from etch, or even from sarge, so I
really doubt that it comes from this change.

-- 
  .''`.  Aurelien Jarno	            | GPG: 1024D/F1BCDB73
 : :' :  Debian developer           | Electrical Engineer
 `. `'   aurel32@debian.org         | aurelien@aurel32.net
   `-    people.debian.org/~aurel32 | www.aurel32.net

Message #58 received at 429021@bugs.debian.org (full text, mbox, reply):

From: Stephane Chazelas <Stephane_Chazelas@yahoo.fr>

To: Aurelien Jarno <aurelien@aurel32.net>

Cc: 429021@bugs.debian.org, Andreas Schwab <schwab@suse.de>, Pierre-Philippe Coupard <pcoupard@skynet.be>, bug-bash@gnu.org, Dmitry Potapov <dpotapov@gmail.com>

Subject: Re: Bug#429021: builtin echo command redirection misbehaves in detached scripts when terminal is closed

Date: Mon, 10 Sep 2007 08:41:51 +0100

On Mon, Sep 10, 2007 at 12:05:57AM +0200, Aurelien Jarno wrote:
[...]
> >>> bash -c 'echo a; echo b > a' >&-
> >>>
> >>> is enough for me to reproduce the problem.

[both "a" and "b" seen in file "a".]

> >> Guess you have a buggy libc, then.
> > [...]
> > 
> > I wouldn't be surprised if it has to do with the fix to debian
> > bug #429021. http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=429021
> > (I'm CCing Dmitry who is the author of that change according to
> > bugs.debian.org)
> > 
> 
> I can reproduce the "bug" with glibc from etch, or even from sarge, so I
> really doubt that it comes from this change.
[...]

Hi Aurelien.

The reason I suspected that is that Andreas with a glibc-2.6.1
was not seeing the problem so that it could be because it was a
debian issue. Also Pierre-Philippe says it is not in debian
unstable from 1st of May 2007 (glibc-2.5 based). And the only
diff on libio/fileops.c in glibc-2.6.1-2 is that fix for 429021,
and the log for that bug talks of something very related.

I could not reproduce the problem with a glibc-2.3.4 on an old
RedHat system. That version of glibc was inbetween sarge's
(2.3.2) and etch's (2.3.6).

Andreas, could you please confirm which distribution of Linux
you have and which version of the libc package?

All in all, it would suggest that the change was introduced by
debian if not in the fix for 429021. To sum up:

glibc's fflush seems to empty its buffer upon a unsuccessful
fflush() (a fflush(3) where the write(2) fails) on
  - debian unstable glibc 2.5 (according to Pierre-Philippe)
  - Andreas' glibc 2.6.1
  - Some RedHat glibc 2.3.4 (according to me)
  - Solaris 7 system libc (not glibc)
  - HPUX 11.11 system libc (not glibc)

And it seems not to empty it in
  - debian unstable 2.6.1-2 (according to me and
  Pierre-Philippe)
  - debian etch (2.3.6?) according to Aurelien
  - debian sarge (2.3.2?) according to Aurelien

Best regards,
Stéphane

Message #63 received at 429021@bugs.debian.org (full text, mbox, reply):

From: Dmitry Potapov <dpotapov@gmail.com>

To: Stephane Chazelas <Stephane_Chazelas@yahoo.fr>

Cc: Andreas Schwab <schwab@suse.de>, Pierre-Philippe Coupard <pcoupard@skynet.be>, bug-bash@gnu.org, 429021@bugs.debian.org

Subject: Re: builtin echo command redirection misbehaves in detached scripts when terminal is closed

Date: Mon, 10 Sep 2007 11:56:33 +0400

On Sun, Sep 09, 2007 at 10:18:07PM +0100, Stephane Chazelas wrote:
> Now, I'm not sure if we can say that the new glibc behavior
> observed is bogus (other than it's different from the behavior
> observed in all the libcs I tried with).

What libc have you tried?

To me, the new behavior makes much more sense, as dropping buffer on
error is really weird thing to do. I have looked at the source code of
newlib and dietlibc, none of them drops buffer on error, and I am not
aware about any other implementation of libc that does.

> It is not a harmless
> change, for sure as it seems to have broken at least bash, zsh
> and possibly ksh93.

Unfortunately, you are right. I did not foresee that some shells may use
"dup2(open("file.txt"), fileno(stdout))". It is a dirty hack, which may
cause some other problems. Frankly, I am a bit surprised that bash uses
printf instead of write(2).  BTW, you cannot use 'printf' in signal
handlers, so it seems that you cannot use 'echo' in trap commands too.

Perhaps, we should rollback my patch and give some time for developers
to fix their broken shells, but, in this case, what is actually broken
are those shells, not libc!

Regards,
Dmitry

Message #68 received at 429021@bugs.debian.org (full text, mbox, reply):

From: Dmitry Potapov <dpotapov@gmail.com>

To: Aurelien Jarno <aurelien@aurel32.net>

Cc: Stephane Chazelas <Stephane_Chazelas@yahoo.fr>, 429021@bugs.debian.org, Andreas Schwab <schwab@suse.de>, Pierre-Philippe Coupard <pcoupard@skynet.be>, bug-bash@gnu.org

Subject: Re: Bug#429021: builtin echo command redirection misbehaves in detached scripts when terminal is closed

Date: Mon, 10 Sep 2007 12:06:14 +0400

On Mon, Sep 10, 2007 at 12:05:57AM +0200, Aurelien Jarno wrote:
> > I wouldn't be surprised if it has to do with the fix to debian
> > bug #429021. http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=429021
> > (I'm CCing Dmitry who is the author of that change according to
> > bugs.debian.org)
> > 
> 
> I can reproduce the "bug" with glibc from etch, or even from sarge, so I
> really doubt that it comes from this change.

I can NOT reproduce the problem with glibc from etch, and I do believe
that my patch caused the aforementioned problem, though I do not think
that the patch was incorrect, as to the real bug lies inside of those
shells.

Dmitry

Message #73 received at 429021@bugs.debian.org (full text, mbox, reply):

From: Stephane Chazelas <Stephane_Chazelas@yahoo.fr>

To: Dmitry Potapov <dpotapov@gmail.com>

Cc: Andreas Schwab <schwab@suse.de>, Pierre-Philippe Coupard <pcoupard@skynet.be>, bug-bash@gnu.org, 429021@bugs.debian.org

Subject: Re: builtin echo command redirection misbehaves in detached scripts when terminal is closed

Date: Mon, 10 Sep 2007 09:08:33 +0100

On Mon, Sep 10, 2007 at 11:56:33AM +0400, Dmitry Potapov wrote:
> On Sun, Sep 09, 2007 at 10:18:07PM +0100, Stephane Chazelas wrote:
> > Now, I'm not sure if we can say that the new glibc behavior
> > observed is bogus (other than it's different from the behavior
> > observed in all the libcs I tried with).
> 
> What libc have you tried?
> 
> To me, the new behavior makes much more sense, as dropping buffer on
> error is really weird thing to do. I have looked at the source code of
> newlib and dietlibc, none of them drops buffer on error, and I am not
> aware about any other implementation of libc that does.

Hi Dmitry,

thanks for replying, I gave a list in another email. I tried on
Solaris 7 and HPUX and both seem to flush the buffer upon an
unsuccessful fflush()

> > It is not a harmless
> > change, for sure as it seems to have broken at least bash, zsh
> > and possibly ksh93.
> 
> Unfortunately, you are right. I did not foresee that some shells may use
> "dup2(open("file.txt"), fileno(stdout))". It is a dirty hack, which may
> cause some other problems. Frankly, I am a bit surprised that bash uses
> printf instead of write(2).  BTW, you cannot use 'printf' in signal
> handlers, so it seems that you cannot use 'echo' in trap commands too.
> 
> Perhaps, we should rollback my patch and give some time for developers
> to fix their broken shells, but, in this case, what is actually broken
> are those shells, not libc!
[...]

On the other end, how would you force the flush of the buffer?

And how would you redirect stdout? We can use freopen() instead
of the hack above for files, but not for pipes or arbitrary fds
(as in >&3). Erik Blake was suggesting to use freopen(NULL) (not
to fix that very problem but because of the fact that if you
reassign stdout to some resource of a different nature, you need
to tell stdio as stdio may need to operate differently), but
that's not very portable according to POSIX. Would freopen(NULL)
flush the output buffer?

You cannot simply assign stdout to some value returned by
fdopen() as that's not portable either...

-- 
Stéphane

Message #78 received at 429021@bugs.debian.org (full text, mbox, reply):

From: Aurelien Jarno <aurelien@aurel32.net>

To: Dmitry Potapov <dpotapov@gmail.com>, 429021@bugs.debian.org

Subject: Re: Bug#429021: builtin echo command redirection misbehaves in detached scripts when terminal is closed

Date: Mon, 10 Sep 2007 10:16:15 +0200

Dmitry Potapov a écrit :
> On Mon, Sep 10, 2007 at 12:05:57AM +0200, Aurelien Jarno wrote:
>>> I wouldn't be surprised if it has to do with the fix to debian
>>> bug #429021. http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=429021
>>> (I'm CCing Dmitry who is the author of that change according to
>>> bugs.debian.org)
>>>
>> I can reproduce the "bug" with glibc from etch, or even from sarge, so I
>> really doubt that it comes from this change.
> 
> I can NOT reproduce the problem with glibc from etch, and I do believe
> that my patch caused the aforementioned problem, though I do not think
> that the patch was incorrect, as to the real bug lies inside of those
> shells.
> 

What is the exact command to run? On a etch machine I get:

[anguille:~]$ bash -c 'echo a; echo b > a' >&-
bash: line 0: echo: write error: Bad file descriptor
[anguille:~]$

-- 
  .''`.  Aurelien Jarno	            | GPG: 1024D/F1BCDB73
 : :' :  Debian developer           | Electrical Engineer
 `. `'   aurel32@debian.org         | aurelien@aurel32.net
   `-    people.debian.org/~aurel32 | www.aurel32.net

Message #83 received at 429021@bugs.debian.org (full text, mbox, reply):

From: Dmitry Potapov <dpotapov@gmail.com>

To: Stephane Chazelas <Stephane_Chazelas@yahoo.fr>

Cc: Andreas Schwab <schwab@suse.de>, Pierre-Philippe Coupard <pcoupard@skynet.be>, bug-bash@gnu.org, 429021@bugs.debian.org

Subject: Re: builtin echo command redirection misbehaves in detached scripts when terminal is closed

Date: Mon, 10 Sep 2007 14:17:41 +0400

Hi Stephane,

On Mon, Sep 10, 2007 at 09:08:33AM +0100, Stephane Chazelas wrote:
> thanks for replying, I gave a list in another email. I tried on
> Solaris 7 and HPUX and both seem to flush the buffer upon an
> unsuccessful fflush()

I see... I wonder how they work in regard of my original problem
described in the Bug#429021, because it is possible to not discard data
when write failed, but still clean buffer in fflush(). So, functions
like fwrite, printf will not lose some previously written data on error,
but fflush() will always have a clean output buffer at return, so
it will not break existing software, which use dup2 trick.

> On the other end, how would you force the flush of the buffer?

The flush means to _deliver_ data, which is impossible in this case.

> And how would you redirect stdout? We can use freopen() instead
> of the hack above for files, but not for pipes or arbitrary fds
> (as in >&3). 

I see... POSIX has fdopen to create a stream based on the existing
file descriptor, but there is no function to change an existing
stream like 'stdout'. So, I don't know any other portable solution
except avoiding 'stdout'. For some implementations, you can just
assign any FILE pointer to stdout like this:

FILE* out = fdopen(fd, mode);
if (out != NULL)
  {
    fclose(stdout);
    stdout = out;
  }
else
  report_error();

but in general it does not work, because stdout is rvalue.

> Erik Blake was suggesting to use freopen(NULL) (not
> to fix that very problem but because of the fact that if you
> reassign stdout to some resource of a different nature, you need
> to tell stdio as stdio may need to operate differently), but
> that's not very portable according to POSIX. Would freopen(NULL)
> flush the output buffer?

In Glibc, freopen:

  if (filename == NULL && _IO_fileno (fp) >= 0) 
    { 
      fd = __dup (_IO_fileno (fp));
      if (fd != -1)
        filename = fd_to_filename (fd);
    }

Then it closes, the original stream and opens a new one in
the same place. So I believe it should work with glibc
provided you do that you called it after dup2 and that your
system have /proc, because fd_to_filename relies on it.

freopen in newlib does not do anything special about NULL,
so I believe it does not work with NULL.

Perhaps, freopen("/dev/stdout") is a more portable way to
do what you want.

Regards,
Dmitry

Message #88 received at 429021@bugs.debian.org (full text, mbox, reply):

From: Stephane Chazelas <Stephane_Chazelas@yahoo.fr>

To: Dmitry Potapov <dpotapov@gmail.com>

Cc: Andreas Schwab <schwab@suse.de>, Pierre-Philippe Coupard <pcoupard@skynet.be>, bug-bash@gnu.org, 429021@bugs.debian.org

Subject: Re: builtin echo command redirection misbehaves in detached scripts when terminal is closed

Date: Mon, 10 Sep 2007 12:13:12 +0100

On Mon, Sep 10, 2007 at 02:17:41PM +0400, Dmitry Potapov wrote:
[...]
> On Mon, Sep 10, 2007 at 09:08:33AM +0100, Stephane Chazelas wrote:
> > thanks for replying, I gave a list in another email. I tried on
> > Solaris 7 and HPUX and both seem to flush the buffer upon an
> > unsuccessful fflush()
> 
> I see... I wonder how they work in regard of my original problem
> described in the Bug#429021, because it is possible to not discard data
> when write failed, but still clean buffer in fflush(). So, functions
> like fwrite, printf will not lose some previously written data on error,
> but fflush() will always have a clean output buffer at return, so
> it will not break existing software, which use dup2 trick.

I'll investigate this evening (BTW, it wasn't Solaris 7, but
Solaris 8).

> > On the other end, how would you force the flush of the buffer?
> 
> The flush means to _deliver_ data, which is impossible in this case.

Sorry, I meant flush() as in emptying the buffer (wether
flushing it to the fd or down the drain (discard it)).

BTW, does anybody know why our emails don't seem to make it to
the bash mailing list anymore?

-- 
Stéphane

Message #93 received at 429021@bugs.debian.org (full text, mbox, reply):

From: Chet Ramey <chet.ramey@case.edu>

To: Dmitry Potapov <dpotapov@gmail.com>

Cc: Stephane Chazelas <Stephane_Chazelas@yahoo.fr>, Andreas Schwab <schwab@suse.de>, Pierre-Philippe Coupard <pcoupard@skynet.be>, bug-bash@gnu.org, 429021@bugs.debian.org, chet@case.edu

Subject: Re: builtin echo command redirection misbehaves in detached scripts when terminal is closed

Date: Mon, 10 Sep 2007 09:18:55 -0400

Dmitry Potapov wrote:

> 
> Unfortunately, you are right. I did not foresee that some shells may use
> "dup2(open("file.txt"), fileno(stdout))". It is a dirty hack, which may
> cause some other problems. Frankly, I am a bit surprised that bash uses
> printf instead of write(2).  BTW, you cannot use 'printf' in signal
> handlers, so it seems that you cannot use 'echo' in trap commands too.

Luckily, neither of these things is true.

What's needed is a portable interface like BSD's fpurge(3).

Chet

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
		       Live Strong.  No day but today.
Chet Ramey, ITS, CWRU    chet@case.edu    http://cnswww.cns.cwru.edu/~chet/

Message #98 received at 429021@bugs.debian.org (full text, mbox, reply):

From: Dmitry Potapov <dpotapov@gmail.com>

To: Stephane Chazelas <Stephane_Chazelas@yahoo.fr>

Cc: Andreas Schwab <schwab@suse.de>, Pierre-Philippe Coupard <pcoupard@skynet.be>, bug-bash@gnu.org, 429021@bugs.debian.org

Subject: Re: builtin echo command redirection misbehaves in detached scripts when terminal is closed

Date: Mon, 10 Sep 2007 17:36:46 +0400

Hello Stephane,

I was wrong about suggestion freopen("/dev/stdout") in my previous mail.
It cannot be used to redirect stdout.

Regards,
Dmitry

Message #103 received at 429021@bugs.debian.org (full text, mbox, reply):

From: Andreas Schwab <schwab@suse.de>

To: chet.ramey@case.edu

Cc: Dmitry Potapov <dpotapov@gmail.com>, Stephane Chazelas <Stephane_Chazelas@yahoo.fr>, Pierre-Philippe Coupard <pcoupard@skynet.be>, bug-bash@gnu.org, 429021@bugs.debian.org, chet@case.edu

Subject: Re: builtin echo command redirection misbehaves in detached scripts when terminal is closed

Date: Mon, 10 Sep 2007 15:53:41 +0200

Chet Ramey <chet.ramey@case.edu> writes:

> What's needed is a portable interface like BSD's fpurge(3).

This is also available from glibc as __fpurge (likewise on Solaris).

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Message #108 received at 429021@bugs.debian.org (full text, mbox, reply):

From: Chet Ramey <chet.ramey@case.edu>

To: Andreas Schwab <schwab@suse.de>

Cc: Dmitry Potapov <dpotapov@gmail.com>, Stephane Chazelas <Stephane_Chazelas@yahoo.fr>, Pierre-Philippe Coupard <pcoupard@skynet.be>, bug-bash@gnu.org, 429021@bugs.debian.org, chet@case.edu

Subject: Re: builtin echo command redirection misbehaves in detached scripts when terminal is closed

Date: Mon, 10 Sep 2007 11:57:34 -0400

Andreas Schwab wrote:
> Chet Ramey <chet.ramey@case.edu> writes:
> 
>> What's needed is a portable interface like BSD's fpurge(3).
> 
> This is also available from glibc as __fpurge (likewise on Solaris).

Yes, though I have an aversion to calling functions with a `__' prefix
from user application code.

However:

"These  functions  are  nonstandard  and  not  portable."

It would be nice to have something standardized.  I can certainly add
yet another configure test for this -- I just wish I didn't have to.

Chet

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
		       Live Strong.  No day but today.
Chet Ramey, ITS, CWRU    chet@case.edu    http://cnswww.cns.cwru.edu/~chet/

Message #113 received at 429021@bugs.debian.org (full text, mbox, reply):

From: Stephane Chazelas <Stephane_Chazelas@yahoo.fr>

To: Chet Ramey <chet.ramey@case.edu>

Cc: Andreas Schwab <schwab@suse.de>, Dmitry Potapov <dpotapov@gmail.com>, Pierre-Philippe Coupard <pcoupard@skynet.be>, bug-bash@gnu.org, 429021@bugs.debian.org, chet@case.edu

Subject: Re: builtin echo command redirection misbehaves in detached scripts when terminal is closed

Date: Mon, 10 Sep 2007 17:39:09 +0100

On Mon, Sep 10, 2007 at 11:57:34AM -0400, Chet Ramey wrote:
> Andreas Schwab wrote:
> > Chet Ramey <chet.ramey@case.edu> writes:
> > 
> >> What's needed is a portable interface like BSD's fpurge(3).
> > 
> > This is also available from glibc as __fpurge (likewise on Solaris).
> 
> Yes, though I have an aversion to calling functions with a `__' prefix
> from user application code.
> 
> However:
> 
> "These  functions  are  nonstandard  and  not  portable."
> 
> It would be nice to have something standardized.  I can certainly add
> yet another configure test for this -- I just wish I didn't have to.
[...]

Note that zsh seems to have the same problem as bash here
(except that it uses fwrite + fputc instead of printf).

The problem I saw with ksh93 seems to be unrelated as ksh93
doesn't seem to be using stdio.

Dmitry, your t.c in the debian report gives:

On Solaris 8:

$ ./t
signal handler called, sig=2
error at num_bytes=15352
fputs: Interrupted system call
writer: num_bytes=80000 num_lines=10000
reader: num_bytes=74888 num_lines=9361
reader: number of missing bytes: 5112

On HPUX 11.11:

$ ./t
signal handler called, sig=2
error at num_bytes=16376
fputs: Interrupted system call
fclose: Interrupted system call
reader: num_bytes=71816 num_lines=8977
reader: number of missing bytes: 8184

So they don't seem to care either to retry and send the data
if the first write() fails.

With dietlibc:

$ ./t
signal handler called, sig=2
writer: num_bytes=80008 num_lines=10001
writer: expected num_bytes=80000 but was 80008
reader: num_bytes=80007 num_lines=10000
reader: number of missing bytes: -7

And dietlibc behaves the same as glibc patched with your
(Dmitry's) change upon the fflush. That is bash would misbehave
the same if linked against dietlibc.

I've also verified that if I revert your change and recompile
the glibc, bash's (and zsh's) problem goes away, so that would
confirm if needed be that it was that fix that introduced the
change in behavior.

-- 
Stéphane

Message #118 received at 429021@bugs.debian.org (full text, mbox, reply):

From: Dmitry Potapov <dpotapov@gmail.com>

To: Stephane Chazelas <Stephane_Chazelas@yahoo.fr>

Cc: Chet Ramey <chet.ramey@case.edu>, Andreas Schwab <schwab@suse.de>, Pierre-Philippe Coupard <pcoupard@skynet.be>, bug-bash@gnu.org, 429021@bugs.debian.org, chet@case.edu

Subject: Re: builtin echo command redirection misbehaves in detached scripts when terminal is closed

Date: Mon, 10 Sep 2007 21:25:26 +0400

On Mon, Sep 10, 2007 at 05:39:09PM +0100, Stephane Chazelas wrote:
> Dmitry, your t.c in the debian report gives:
> 
> On Solaris 8:
[...]
> On HPUX 11.11:
[...]
>
> So they don't seem to care either to retry and send the data
> if the first write() fails.

Yes, it seems they purge all data in the IO buffer on error.

> With dietlibc:
> 
> $ ./t
> signal handler called, sig=2
> writer: num_bytes=80008 num_lines=10001
> writer: expected num_bytes=80000 but was 80008
> reader: num_bytes=80007 num_lines=10000
> reader: number of missing bytes: -7
> 
> And dietlibc behaves the same as glibc patched with your
> (Dmitry's) change upon the fflush.

No, glibc with my patch gives:

$ ./t 
signal handler called, sig=2
error at num_bytes=69632
fputs: Interrupted system call
writer: num_bytes=80000 num_lines=10000
reader: num_bytes=80000 num_lines=10000

-7 indicates an error in dietlibc. Somehow, dietlibc does not take into
account that write(2) can write only part of data, and it should not be
considered as an error.  But this bug in dietlibc is irrelevant to our
problem. Newlib should work as glibc with my patch, but I have not
tested it.

Dmitry

Message #123 received at 429021@bugs.debian.org (full text, mbox, reply):

From: Stephane Chazelas <Stephane_Chazelas@yahoo.fr>

To: Dmitry Potapov <dpotapov@gmail.com>

Cc: Chet Ramey <chet.ramey@case.edu>, Andreas Schwab <schwab@suse.de>, Pierre-Philippe Coupard <pcoupard@skynet.be>, bug-bash@gnu.org, 429021@bugs.debian.org, chet@case.edu

Subject: Re: builtin echo command redirection misbehaves in detached scripts when terminal is closed

Date: Mon, 10 Sep 2007 18:36:51 +0100

On Mon, Sep 10, 2007 at 09:25:26PM +0400, Dmitry Potapov wrote:
[...]
> > With dietlibc:
> > 
> > $ ./t
> > signal handler called, sig=2
> > writer: num_bytes=80008 num_lines=10001
> > writer: expected num_bytes=80000 but was 80008
> > reader: num_bytes=80007 num_lines=10000
> > reader: number of missing bytes: -7
> > 
> > And dietlibc behaves the same as glibc patched with your
> > (Dmitry's) change upon the fflush.
> 
> No, glibc with my patch gives:
[...]

Sorry for the misunderstanding, I meant "upon the fflush", as in
wrt the issue at stake, that is the fact that dietlibc doesn't
seem to empty the output buffer upon an unsuccessful fflush
either, which confirms what you suspected earlier through
reading the dietlibc code. I did not mean that "t" was behaving
the same in glibc and dietlibc. With the glibc, I obtain:

$ ~/t
signal handler called, sig=2
error at num_bytes=66560
fputs: Interrupted system call
reader: num_bytes=80000 num_lines=10000
writer: num_bytes=80000 num_lines=10000

And with your fix reverted:

.../glibc-2.6.1/build-tree/i386-libc$ LD_LIBRARY_PATH=$PWD ~/t
signal handler called, sig=2
error at num_bytes=66560
fputs: Interrupted system call
writer: num_bytes=80000 num_lines=10000
reader: num_bytes=78976 num_lines=9872
reader: number of missing bytes: 1024

as expected.

Best regards,
Stéphane

Message #128 received at 429021@bugs.debian.org (full text, mbox, reply):

From: Nix <nix@esperi.org.uk>

To: 429021@bugs.debian.org

Subject: this bug causes nfs-utils 1.1.0 to fail

Date: Tue, 23 Oct 2007 16:30:59 +0100

Have a look at
<http://www.ussg.iu.edu/hypermail/linux/kernel/0709.2/2524.html>, where
it becomes clear that this glibc patch causes communication failure
between rpc.mountd (of nfs-utils 1.1.0) and the kernel during kernel
upcalls to verify fh validity. This leads to the entire NFS server
seizing up.

It's been worked around in more recent nfs-utils releases, but still,
this is not good. This patch needs to go.

Send a report that this bug log contains spam.