Debian Bug report logs - #48108
rsync --unbuffered-stdout patch

version graph

Package: rsync; Maintainer for rsync is Paul Slootman <paul@debian.org>; Source for rsync is src:rsync.

Reported by: Ian Jackson <ian@davenant.greenend.org.uk>

Date: Sat, 23 Oct 1999 15:48:00 UTC

Severity: wishlist

Tags: patch

Found in version 2.3.1-2

Reply or subscribe to this bug.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, Philip Hands <phil@hands.com>:
Bug#48108; Package rsync. Full text and rfc822 format available.

Acknowledgement sent to Ian Jackson <ian@davenant.greenend.org.uk>:
New Bug report received and forwarded. Copy sent to Philip Hands <phil@hands.com>. Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: Ian Jackson <ian@davenant.greenend.org.uk>
To: submit@bugs.debian.org
Subject: rsync --unbuffered-stdout patch
Date: Sat, 23 Oct 1999 15:13:44 +0100 (BST)
Package: rsync
Version: 2.3.1-2
Severity: wishlist

Below is a patch to give rsync a new --unbuffered-stdout option, which
I find useful if rsync -v etc. is directed into a logfile or pipe.  It
would be nice if this could go into Debian rsync and also upstream.

( You may notice that setting add-log-mailing-address in the changlog
has caused my Emacs to lie in my changelog entry.  If this upsets you
then you should put your personal options in your personal Emacs
configuration :-). )

Thanks,
Ian.

Only in rsync-2.3.1: Makefile
Only in rsync-2.3.1: access.o
Only in rsync-2.3.1: authenticate.o
Only in rsync-2.3.1: build
Only in rsync-2.3.1: checksum.o
Only in rsync-2.3.1: cleanup.o
Only in rsync-2.3.1: clientserver.o
Only in rsync-2.3.1: compat.o
Only in rsync-2.3.1: config.cache
Only in rsync-2.3.1: config.h
Only in rsync-2.3.1: config.log
Only in rsync-2.3.1: config.status
Only in rsync-2.3.1: connection.o
diff -ru orig/rsync-2.3.1/debian/changelog rsync-2.3.1/debian/changelog
--- orig/rsync-2.3.1/debian/changelog	Sat Oct 23 15:06:49 1999
+++ rsync-2.3.1/debian/changelog	Sat Oct 23 15:07:15 1999
@@ -1,3 +1,9 @@
+rsync (2.3.1-2.0.1) unstable; urgency=low
+
+  * New --unbuffered-stdout option.
+
+ -- Ian Jackson <phil@hands.com>  Sat, 23 Oct 1999 15:07:15 +0100
+
 rsync (2.3.1-2) unstable; urgency=low
 
   * apply Patrik Rak's fix for directory atribute setting (closes: #36179)
Only in rsync-2.3.1/debian: changelog~
Only in rsync-2.3.1/debian: files
Only in rsync-2.3.1/debian: substvars
Only in rsync-2.3.1/debian: tmp
Only in rsync-2.3.1: exclude.o
Only in rsync-2.3.1: fileio.o
Only in rsync-2.3.1: flist.o
Only in rsync-2.3.1: generator.o
Only in rsync-2.3.1: hlink.o
Only in rsync-2.3.1: io.o
Only in rsync-2.3.1/lib: compat.o
Only in rsync-2.3.1/lib: dummy
Only in rsync-2.3.1/lib: fnmatch.o
Only in rsync-2.3.1/lib: getopt.o
Only in rsync-2.3.1/lib: mdfour.o
Only in rsync-2.3.1/lib: snprintf.o
Only in rsync-2.3.1: loadparm.o
Only in rsync-2.3.1: log.o
diff -ru orig/rsync-2.3.1/main.c rsync-2.3.1/main.c
--- orig/rsync-2.3.1/main.c	Tue Apr  6 16:04:16 1999
+++ rsync-2.3.1/main.c	Sat Oct 23 15:06:18 1999
@@ -26,6 +26,7 @@
 extern int csum_length;
 
 extern int verbose;
+extern int unbuffered_stdout;
 
 static void report(int f)
 {
@@ -612,6 +613,10 @@
 
 	if (!parse_arguments(argc, argv, 1)) {
 		exit_cleanup(RERR_SYNTAX);
+	}
+
+	if (unbuffered_stdout) {
+		setvbuf(stdout,0,_IONBF,0);
 	}
 
 	argc -= optind;
Only in rsync-2.3.1: main.c~
Only in rsync-2.3.1: main.o
Only in rsync-2.3.1: match.o
diff -ru orig/rsync-2.3.1/options.c rsync-2.3.1/options.c
--- orig/rsync-2.3.1/options.c	Mon Mar 15 21:23:11 1999
+++ rsync-2.3.1/options.c	Sat Oct 23 15:04:48 1999
@@ -62,6 +62,7 @@
 int copy_unsafe_links=0;
 int block_size=BLOCK_SIZE;
 int size_only=0;
+int unbuffered_stdout=0;
 
 char *backup_suffix = BACKUP_SUFFIX;
 char *tmpdir = NULL;
@@ -130,6 +131,7 @@
   rprintf(F,"     --timeout=TIME          set IO timeout in seconds\n");
   rprintf(F," -I, --ignore-times          don't exclude files that match length and time\n");
   rprintf(F,"     --size-only             only use file size when determining if a file should be transferred\n");
+  rprintf(F,"     --unbuffered-stdout     do not buffer stdout, even if it is not a terminal\n");
   rprintf(F," -T  --temp-dir=DIR          create temporary files in directory DIR\n");
   rprintf(F,"     --compare-dest=DIR      also compare destination files relative to DIR\n");
   rprintf(F," -z, --compress              compress file data\n");
@@ -158,7 +160,7 @@
       OPT_RSYNC_PATH, OPT_FORCE, OPT_TIMEOUT, OPT_DAEMON, OPT_CONFIG, OPT_PORT,
       OPT_INCLUDE, OPT_INCLUDE_FROM, OPT_STATS, OPT_PARTIAL, OPT_PROGRESS,
       OPT_COPY_UNSAFE_LINKS, OPT_SAFE_LINKS, OPT_COMPARE_DEST,
-      OPT_LOG_FORMAT, OPT_PASSWORD_FILE, OPT_SIZE_ONLY};
+      OPT_LOG_FORMAT, OPT_PASSWORD_FILE, OPT_SIZE_ONLY, OPT_UNBUFFERED_STDOUT};
 
 static char *short_options = "oblLWHpguDCtcahvqrRIxnSe:B:T:z";
 
@@ -179,6 +181,7 @@
   {"one-file-system",0,  0,    'x'},
   {"ignore-times",0,     0,    'I'},
   {"size-only",   0,     0,    OPT_SIZE_ONLY},
+  {"unbuffered-stdout", 0, 0,  OPT_UNBUFFERED_STDOUT},
   {"help",        0,     0,    'h'},
   {"dry-run",     0,     0,    'n'},
   {"sparse",      0,     0,    'S'},
@@ -303,6 +306,10 @@
 			size_only = 1;
 			break;
 
+		case OPT_UNBUFFERED_STDOUT:
+			unbuffered_stdout = 1;
+			break;
+
 		case 'x':
 			one_file_system=1;
 			break;
@@ -538,7 +545,7 @@
 	argstr[0] = '-';
 	for (i=0;i<verbose;i++)
 		argstr[x++] = 'v';
-	/* the -q option is intentionally left out */
+	/* the -q and --unbuffered-stdout options are intentionally left out */
 	if (make_backups)
 		argstr[x++] = 'b';
 	if (update_only)
Only in rsync-2.3.1: options.c~
Only in rsync-2.3.1: options.o
Only in rsync-2.3.1: params.o
Only in rsync-2.3.1: receiver.o
Only in rsync-2.3.1: rsync
Only in rsync-2.3.1: rsync.o
Only in rsync-2.3.1: sender.o
Only in rsync-2.3.1: socket.o
Only in rsync-2.3.1: syscall.o
Only in rsync-2.3.1: token.o
Only in rsync-2.3.1: uidlist.o
Only in rsync-2.3.1: util.o
Only in rsync-2.3.1/zlib: adler32.o
Only in rsync-2.3.1/zlib: deflate.o
Only in rsync-2.3.1/zlib: dummy
Only in rsync-2.3.1/zlib: infblock.o
Only in rsync-2.3.1/zlib: infcodes.o
Only in rsync-2.3.1/zlib: inffast.o
Only in rsync-2.3.1/zlib: inflate.o
Only in rsync-2.3.1/zlib: inftrees.o
Only in rsync-2.3.1/zlib: infutil.o
Only in rsync-2.3.1/zlib: trees.o
Only in rsync-2.3.1/zlib: zutil.o


Tags added: patch Request was from Colin Walters <walters@debian.org> to control@bugs.debian.org. Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Philip Hands <phil@hands.com>, rsync@packages.qa.debian.org:
Bug#48108; Package rsync. Full text and rfc822 format available.

Acknowledgement sent to Ian Jackson <ian@davenant.greenend.org.uk>:
Extra info received and forwarded to list. Copy sent to Philip Hands <phil@hands.com>, rsync@packages.qa.debian.org. Full text and rfc822 format available.

Message #12 received at 48108@bugs.debian.org (full text, mbox):

From: Ian Jackson <ian@davenant.greenend.org.uk>
To: 48108@bugs.debian.org (Debian Bug Tracking System)
Subject: Bug#48108: rsync --unbuffered-stdout patch
Date: Thu, 4 Apr 2002 16:01:34 +0100
I have updated my patch for 2.3.2-1.6.  Please apply it.

Ian.

diff -ru orig/rsync-2.3.2/debian/changelog rsync-2.3.2/debian/changelog
--- orig/rsync-2.3.2/debian/changelog	Thu Apr  4 15:56:07 2002
+++ rsync-2.3.2/debian/changelog	Thu Apr  4 15:58:14 2002
@@ -1,3 +1,9 @@
+rsync (2.3.1-1.6.0.1) unstable; urgency=low
+
+  * New --unbuffered-stdout option.
+
+ -- Ian Jackson <ian@davenant.greenend.org.uk>  Sat, 23 Oct 1999 15:07:15 +0100
+
 rsync (2.3.2-1.6) stable; urgency=high
 
   * SECURITY: fix zlib double-free
Only in rsync-2.3.2/debian: changelog.orig
Only in rsync-2.3.2/debian: changelog.rej
Only in rsync-2.3.2/debian: changelog~
diff -ru orig/rsync-2.3.2/main.c rsync-2.3.2/main.c
--- orig/rsync-2.3.2/main.c	Mon Nov  8 13:15:01 1999
+++ rsync-2.3.2/main.c	Thu Apr  4 15:57:09 2002
@@ -24,6 +24,7 @@
 struct stats stats;
 
 extern int verbose;
+extern int unbuffered_stdout;
 
 static void report(int f)
 {
@@ -610,6 +611,10 @@
 
 	if (!parse_arguments(argc, argv, 1)) {
 		exit_cleanup(RERR_SYNTAX);
+	}
+
+	if (unbuffered_stdout) {
+		setvbuf(stdout,0,_IONBF,0);
 	}
 
 	argc -= optind;
Only in rsync-2.3.2: main.c.orig
diff -ru orig/rsync-2.3.2/options.c rsync-2.3.2/options.c
--- orig/rsync-2.3.2/options.c	Mon Nov  8 13:15:01 1999
+++ rsync-2.3.2/options.c	Thu Apr  4 15:59:25 2002
@@ -63,6 +63,7 @@
 int block_size=BLOCK_SIZE;
 int size_only=0;
 int delete_after=0;
+int unbuffered_stdout=0;
 
 char *backup_suffix = BACKUP_SUFFIX;
 char *tmpdir = NULL;
@@ -134,6 +135,7 @@
   rprintf(F,"     --timeout=TIME          set IO timeout in seconds\n");
   rprintf(F," -I, --ignore-times          don't exclude files that match length and time\n");
   rprintf(F,"     --size-only             only use file size when determining if a file should be transferred\n");
+  rprintf(F,"     --unbuffered-stdout     do not buffer stdout, even if it is not a terminal\n");
   rprintf(F," -T  --temp-dir=DIR          create temporary files in directory DIR\n");
   rprintf(F,"     --compare-dest=DIR      also compare destination files relative to DIR\n");
   rprintf(F," -P                          equivalent to --partial --progress\n");
@@ -165,7 +167,7 @@
       OPT_INCLUDE, OPT_INCLUDE_FROM, OPT_STATS, OPT_PARTIAL, OPT_PROGRESS,
       OPT_COPY_UNSAFE_LINKS, OPT_SAFE_LINKS, OPT_COMPARE_DEST,
       OPT_LOG_FORMAT, OPT_PASSWORD_FILE, OPT_SIZE_ONLY, OPT_ADDRESS,
-      OPT_DELETE_AFTER};
+      OPT_DELETE_AFTER, OPT_UNBUFFERED_STDOUT };
 
 static char *short_options = "oblLWHpguDCtcahvqrRIxnSe:B:T:zP";
 
@@ -186,6 +188,7 @@
   {"one-file-system",0,  0,    'x'},
   {"ignore-times",0,     0,    'I'},
   {"size-only",   0,     0,    OPT_SIZE_ONLY},
+  {"unbuffered-stdout", 0, 0,  OPT_UNBUFFERED_STDOUT},
   {"help",        0,     0,    'h'},
   {"dry-run",     0,     0,    'n'},
   {"sparse",      0,     0,    'S'},
@@ -312,6 +315,10 @@
 			size_only = 1;
 			break;
 
+		case OPT_UNBUFFERED_STDOUT:
+			unbuffered_stdout = 1;
+			break;
+
 		case 'x':
 			one_file_system=1;
 			break;
@@ -565,7 +572,7 @@
 	argstr[0] = '-';
 	for (i=0;i<verbose;i++)
 		argstr[x++] = 'v';
-	/* the -q option is intentionally left out */
+	/* the -q and --unbuffered-stdout options are intentionally left out */
 	if (make_backups)
 		argstr[x++] = 'b';
 	if (update_only)
Only in rsync-2.3.2: options.c.orig
Only in rsync-2.3.2: options.c.rej
Only in rsync-2.3.2: options.c~



Information forwarded to debian-bugs-dist@lists.debian.org, Paul Slootman <paul@debian.org>:
Bug#48108; Package rsync. Full text and rfc822 format available.

Acknowledgement sent to Ian Jackson <ian@davenant.greenend.org.uk>:
Extra info received and forwarded to list. Copy sent to Paul Slootman <paul@debian.org>. Full text and rfc822 format available.

Message #17 received at 48108@bugs.debian.org (full text, mbox):

From: Ian Jackson <ian@davenant.greenend.org.uk>
To: 48108@bugs.debian.org (Debian Bug Tracking System)
Subject: Bug#48108: rsync --unbuffered-stdout patch
Date: Sun, 19 Oct 2003 18:03:57 +0100
I have updated my patch for 2.5.5-0.1.  Please apply it.

Ian.

Only in rsync-2.5.5: Makefile
Only in rsync-2.5.5: access.o
Only in rsync-2.5.5: authenticate.o
Only in rsync-2.5.5: backup.o
Only in rsync-2.5.5: batch.o
Only in rsync-2.5.5: build
Only in rsync-2.5.5: checksum.o
Only in rsync-2.5.5: cleanup.o
Only in rsync-2.5.5: clientname.o
Only in rsync-2.5.5: clientserver.o
Only in rsync-2.5.5: compat.o
Only in rsync-2.5.5: config.h
Only in rsync-2.5.5: config.log
Only in rsync-2.5.5: config.status
Only in rsync-2.5.5: connection.o
diff -ru orig/rsync-2.5.5/debian/changelog rsync-2.5.5/debian/changelog
--- orig/rsync-2.5.5/debian/changelog	Fri Oct 17 20:16:14 2003
+++ rsync-2.5.5/debian/changelog	Fri Oct 17 20:30:53 2003
@@ -1,3 +1,9 @@
+rsync (2.5.5-0.1.0.1) unstable; urgency=low
+
+  * New --unbuffered-stdout option (bug #48108).
+
+ -- Ian Jackson <ian@davenant.greenend.org.uk>  Fri, 17 Oct 2003 20:17:25 +0100
+
 rsync (2.5.5-0.1) unstable; urgency=high
 
   * NMU
Only in rsync-2.5.5/debian: changelog.orig
Only in rsync-2.5.5/debian: changelog.rej
Only in rsync-2.5.5/debian: files
Only in rsync-2.5.5/debian: substvars
Only in rsync-2.5.5/debian: tmp
Only in rsync-2.5.5: exclude.o
Only in rsync-2.5.5: fileio.o
Only in rsync-2.5.5: flist.o
Only in rsync-2.5.5: generator.o
Only in rsync-2.5.5: hlink.o
Only in rsync-2.5.5: io.o
Only in rsync-2.5.5/lib: compat.o
Only in rsync-2.5.5/lib: dummy
Only in rsync-2.5.5/lib: fnmatch.o
Only in rsync-2.5.5/lib: mdfour.o
Only in rsync-2.5.5/lib: permstring.o
Only in rsync-2.5.5/lib: snprintf.o
Only in rsync-2.5.5: loadparm.o
Only in rsync-2.5.5: log.o
diff -ru orig/rsync-2.5.5/main.c rsync-2.5.5/main.c
--- orig/rsync-2.5.5/main.c	Wed Mar 27 05:10:44 2002
+++ rsync-2.5.5/main.c	Fri Oct 17 20:16:35 2003
@@ -26,6 +26,7 @@
 struct stats stats;
 
 extern int verbose;
+extern int unbuffered_stdout;
 
 static void show_malloc_stats(void);
 
@@ -762,6 +763,10 @@
 	if (!am_sender && argc > 1) {
 		usage(FERROR);
 		exit_cleanup(RERR_SYNTAX);
+	}
+
+	if (unbuffered_stdout) {
+		setvbuf(stdout,0,_IONBF,0);
 	}
 
 	if (argc == 0 && !am_sender) {
Only in rsync-2.5.5: main.c.orig
Only in rsync-2.5.5: main.o
Only in rsync-2.5.5: match.o
diff -ru orig/rsync-2.5.5/options.c rsync-2.5.5/options.c
--- orig/rsync-2.5.5/options.c	Tue Mar 19 20:16:42 2002
+++ rsync-2.5.5/options.c	Fri Oct 17 20:21:52 2003
@@ -81,6 +81,7 @@
 int opt_ignore_existing=0;
 int max_delete=0;
 int ignore_errors=0;
+int unbuffered_stdout=0;
 #ifdef _WIN32
 int modify_window=2;
 #else
@@ -235,6 +236,7 @@
   rprintf(F,"     --delete-excluded       also delete excluded files on the receiving side\n");
   rprintf(F,"     --delete-after          delete after transferring, not before\n");
   rprintf(F,"     --ignore-errors         delete even if there are IO errors\n");
+  rprintf(F,"     --unbuffered-stdout     do not buffer stdout, even if it is not a terminal\n");
   rprintf(F,"     --max-delete=NUM        don't delete more than NUM files\n");
   rprintf(F,"     --partial               keep partially transferred files\n");
   rprintf(F,"     --force                 force deletion of directories even if not empty\n");
@@ -285,7 +287,7 @@
       OPT_COPY_UNSAFE_LINKS, OPT_SAFE_LINKS, OPT_COMPARE_DEST,
       OPT_LOG_FORMAT, OPT_PASSWORD_FILE, OPT_SIZE_ONLY, OPT_ADDRESS,
       OPT_DELETE_AFTER, OPT_EXISTING, OPT_MAX_DELETE, OPT_BACKUP_DIR, 
-      OPT_IGNORE_ERRORS, OPT_BWLIMIT, OPT_BLOCKING_IO,
+      OPT_IGNORE_ERRORS, OPT_UNBUFFERED_STDOUT, OPT_BWLIMIT, OPT_BLOCKING_IO,
       OPT_NO_BLOCKING_IO, OPT_WHOLE_FILE, OPT_NO_WHOLE_FILE,
       OPT_MODIFY_WINDOW, OPT_READ_BATCH, OPT_WRITE_BATCH, OPT_IGNORE_EXISTING};
 
@@ -349,6 +351,7 @@
   {"progress",         0,  POPT_ARG_NONE,   &do_progress , 0, 0, 0 },
   {"partial",          0,  POPT_ARG_NONE,   &keep_partial , 0, 0, 0 },
   {"ignore-errors",    0,  POPT_ARG_NONE,   &ignore_errors , 0, 0, 0 },
+  {"unbuffered-stdout",0,  POPT_ARG_NONE,   &unbuffered_stdout , 0, 0, 0 },
   {"blocking-io",      0,  POPT_ARG_NONE,   &blocking_io , 0, 0, 0 },
   {"no-blocking-io",   0,  POPT_ARG_NONE,   0, 		     OPT_NO_BLOCKING_IO, 0, 0 },
   {0,                 'P', POPT_ARG_NONE,   0,               'P', 0, 0 },
@@ -630,7 +633,7 @@
 	for (i=0;i<verbose;i++)
 		argstr[x++] = 'v';
 
-	/* the -q option is intentionally left out */
+	/* the -q and --unbuffered-stdout options are intentionally left out */
 	if (make_backups)
 		argstr[x++] = 'b';
 	if (update_only)
Only in rsync-2.5.5: options.c.orig
Only in rsync-2.5.5: options.c.rej
Only in rsync-2.5.5: options.o
Only in rsync-2.5.5: params.o
Only in rsync-2.5.5/popt: dummy
Only in rsync-2.5.5: receiver.o
Only in rsync-2.5.5: rsync
Only in rsync-2.5.5: rsync.o
Only in rsync-2.5.5: sender.o
Only in rsync-2.5.5: socket.o
Only in rsync-2.5.5: syscall.o
Only in rsync-2.5.5: token.o
Only in rsync-2.5.5: uidlist.o
Only in rsync-2.5.5: util.o
Only in rsync-2.5.5/zlib: adler32.o
Only in rsync-2.5.5/zlib: deflate.o
Only in rsync-2.5.5/zlib: dummy
Only in rsync-2.5.5/zlib: infblock.o
Only in rsync-2.5.5/zlib: infcodes.o
Only in rsync-2.5.5/zlib: inffast.o
Only in rsync-2.5.5/zlib: inflate.o
Only in rsync-2.5.5/zlib: inftrees.o
Only in rsync-2.5.5/zlib: infutil.o
Only in rsync-2.5.5/zlib: trees.o
Only in rsync-2.5.5/zlib: zutil.o



Information forwarded to debian-bugs-dist@lists.debian.org, Paul Slootman <paul@debian.org>:
Bug#48108; Package rsync. Full text and rfc822 format available.

Acknowledgement sent to Ian Jackson <ian@davenant.greenend.org.uk>:
Extra info received and forwarded to list. Copy sent to Paul Slootman <paul@debian.org>. Full text and rfc822 format available.

Message #22 received at 48108@bugs.debian.org (full text, mbox):

From: Ian Jackson <ian@davenant.greenend.org.uk>
To: 48108@bugs.debian.org (Debian Bug Tracking System)
Subject: Re: Bug#48108: rsync --unbuffered-stdout patch
Date: Thu, 20 May 2004 19:30:02 +0100
On the 19th of October 2003 I wrote:
> I have updated my patch for 2.5.5-0.1.  Please apply it.

Since then there have been several uploads but this patch has not been
included.  Why ?

If I don't hear from you I will consider an NMU.

Thanks,
Ian.



Information forwarded to debian-bugs-dist@lists.debian.org:
Bug#48108; Package rsync. Full text and rfc822 format available.

Acknowledgement sent to Paul Slootman <paul@debian.org>:
Extra info received and forwarded to list. Full text and rfc822 format available.

Message #27 received at 48108@bugs.debian.org (full text, mbox):

From: Paul Slootman <paul@debian.org>
To: Ian Jackson <ian@davenant.greenend.org.uk>, 48108@bugs.debian.org
Subject: Re: Bug#48108: rsync --unbuffered-stdout patch
Date: Fri, 21 May 2004 10:27:52 +0200
On Thu 20 May 2004, Ian Jackson wrote:
> On the 19th of October 2003 I wrote:
> > I have updated my patch for 2.5.5-0.1.  Please apply it.
> 
> Since then there have been several uploads but this patch has not been
> included.  Why ?

There's no real argumentation why this patch is necessary, "it's useful"
isn't enough. I suggest subscribing to the rsync mailing list and
discussing it there, perhaps thera's a better way to do it.
Besides, if you want to have the same unbuffered behaviour irrespective
of whether stdout is a tty or not, use _IOLBF, not _IONBF.

> If I don't hear from you I will consider an NMU.

For a random patch that you happen to find useful? That's not really a
reason for an NMU.


Paul Slootman



Information forwarded to debian-bugs-dist@lists.debian.org, Paul Slootman <paul@debian.org>:
Bug#48108; Package rsync. Full text and rfc822 format available.

Acknowledgement sent to Ian Jackson <ian@davenant.greenend.org.uk>:
Extra info received and forwarded to list. Copy sent to Paul Slootman <paul@debian.org>. Full text and rfc822 format available.

Message #32 received at 48108@bugs.debian.org (full text, mbox):

From: Ian Jackson <ian@davenant.greenend.org.uk>
To: Paul Slootman <paul@debian.org>
Cc: 48108@bugs.debian.org
Subject: Re: Bug#48108: rsync --unbuffered-stdout patch
Date: Fri, 21 May 2004 19:47:24 +0100
Paul Slootman writes ("Re: Bug#48108: rsync --unbuffered-stdout patch"):
> On Thu 20 May 2004, Ian Jackson wrote:
> > Since then there have been several uploads but this patch has not been
> > included.  Why ?
> 
> There's no real argumentation why this patch is necessary, "it's useful"
> isn't enough.

!

Let me try to explain in a bit more detail: if you feed the stdout of
rsync into something not a tty (because for example you want to have a
record of what it did) then you can't also use it to find out how the
transfer is progressing because it always buffers the stdout.

The patch I supplied is very straightforward.

>  I suggest subscribing to the rsync mailing list and
> discussing it there, perhaps thera's a better way to do it.

If you think there is a better way to do it then please do tell me
what it is.  If you think there isn't a better way then I think you
should apply my patch, or tell me how else to solve the problem.

> Besides, if you want to have the same unbuffered behaviour irrespective
> of whether stdout is a tty or not, use _IOLBF, not _IONBF.

But I don't.  I want the output immediately.  The performance costs of
_IONBF are neglible unless you turn the verbosity up to extreme, in
which case you probably wanted timely reporting.

Ian.



Information forwarded to debian-bugs-dist@lists.debian.org:
Bug#48108; Package rsync. Full text and rfc822 format available.

Acknowledgement sent to Paul Slootman <paul@debian.org>:
Extra info received and forwarded to list. Full text and rfc822 format available.

Message #37 received at 48108@bugs.debian.org (full text, mbox):

From: Paul Slootman <paul@debian.org>
To: Ian Jackson <ian@davenant.greenend.org.uk>
Cc: 48108@bugs.debian.org
Subject: Re: Bug#48108: rsync --unbuffered-stdout patch
Date: Sun, 23 May 2004 20:31:28 +0200
On Fri 21 May 2004, Ian Jackson wrote:
> Paul Slootman writes ("Re: Bug#48108: rsync --unbuffered-stdout patch"):
> > On Thu 20 May 2004, Ian Jackson wrote:
> > > Since then there have been several uploads but this patch has not been
> > > included.  Why ?
> > 
> > There's no real argumentation why this patch is necessary, "it's useful"
> > isn't enough.
> 
> !
> 
> Let me try to explain in a bit more detail: if you feed the stdout of
> rsync into something not a tty (because for example you want to have a
> record of what it did) then you can't also use it to find out how the
> transfer is progressing because it always buffers the stdout.

> > Besides, if you want to have the same unbuffered behaviour irrespective
> > of whether stdout is a tty or not, use _IOLBF, not _IONBF.
> 
> But I don't.  I want the output immediately.  The performance costs of

First you say that the patch is necessary because it behaves differently
when stdout is not a tty. Now you say you don't want the same behaviour
as when stdout goes to a tty (i.e., line buffered; stdout IS line
buffered when going to a tty). Please be clear and consistent!

> The patch I supplied is very straightforward.

Yes, duh. It's clear _what_ it does; not _why_ you want it. I wrote:

  : There's no real argumentation why this patch is necessary, "it's useful"
  : isn't enough.

I didn't write "It's not clear what the patch does".


> >  I suggest subscribing to the rsync mailing list and
> > discussing it there, perhaps thera's a better way to do it.
> 
> If you think there is a better way to do it then please do tell me
> what it is.  If you think there isn't a better way then I think you
> should apply my patch, or tell me how else to solve the problem.

As I wrote:

  :  I suggest subscribing to the rsync mailing list and
  : discussing it there, perhaps thera's a better way to do it.

That might help solving the problem...
There have been earlier discussions about how the logging and reporting
should evolve, your (specific) input can help. The upstream author is
very responsive on the list.



Paul Slootman



Information forwarded to debian-bugs-dist@lists.debian.org, Paul Slootman <paul@debian.org>:
Bug#48108; Package rsync. Full text and rfc822 format available.

Acknowledgement sent to Ian Jackson <ian@davenant.greenend.org.uk>:
Extra info received and forwarded to list. Copy sent to Paul Slootman <paul@debian.org>. Full text and rfc822 format available.

Message #42 received at 48108@bugs.debian.org (full text, mbox):

From: Ian Jackson <ian@davenant.greenend.org.uk>
To: Paul Slootman <paul@debian.org>
Cc: 48108@bugs.debian.org
Subject: Re: Bug#48108: rsync --unbuffered-stdout patch
Date: Tue, 25 May 2004 22:10:40 +0100
Paul Slootman writes ("Re: Bug#48108: rsync --unbuffered-stdout patch"):
> Yes, duh. It's clear _what_ it does; not _why_ you want it. I wrote:
> 
>   : There's no real argumentation why this patch is necessary, "it's useful"
>   : isn't enough.
> 
> I didn't write "It's not clear what the patch does".

Obviously I've not understood the question; perhaps it would be better
if I explained my specific situation:

I'm invoking rsync as part of a mirroring system.  The mirroring
system wants (when it's invoked interactively) to give the human user
some small indication of progress while rsync is running.

So, I have to invoke rsync with --progress so that it prints things as
it goes.  But the output is too verbose for me, and in any case I want
a standardised progress indication across various programs including
rsync.  So I want to massage (condense, in this case) the output - I
run rsync with stdout redirected into a pipe which my parent process
reads.  But this is defeated by the buffering in stdio.

There are a number of ways to solve this particular problem.  I could
use a pty; I could patch rsync to output the massaged output format
directly; I could ptrace rsync so that I can see if it's making
progress or even mess with its innards; I could use an LD_PRELOAD
hack; I could sniff the network.  But all of these are overcomplicated
(and many of them are insane) compared to simply patching rsync so
that it doesn't buffer stdout.

The latter solution has an additional advantage: other programs which
want timely output from rsync (whether --progress output or something
else) will be able to get it too.  I used _IONBF because it seemed
possible that rsync might at some point print partial lines of
progress or other output to stdout, and that in the case where you
want timely output you probably want that completely unbuffered,
rather than line-buffered.

An alternative option is to make rsync _always_ unbuffer stdout when
you say --progress, but I wasn't sure that was right.

> As I wrote:
>   :  I suggest subscribing to the rsync mailing list and
>   : discussing it there, perhaps thera's a better way to do it.
> That might help solving the problem...

It seems to me that one role of a Debian package maintainer is to
handle issues like this with upstream.  I don't think in general bug
reports with sensible patches should elicit a `take it upstream'
response.  If you think that the patch would be sensible upstream
(which may well be true) then I think you should pass it on.

Ian.



Information forwarded to debian-bugs-dist@lists.debian.org, Paul Slootman <paul@debian.org>:
Bug#48108; Package rsync. Full text and rfc822 format available.

Acknowledgement sent to Ian Jackson <ian@davenant.greenend.org.uk>:
Extra info received and forwarded to list. Copy sent to Paul Slootman <paul@debian.org>. Full text and rfc822 format available.

Message #47 received at 48108@bugs.debian.org (full text, mbox):

From: Ian Jackson <ian@davenant.greenend.org.uk>
To: 48108@bugs.debian.org
Subject: Updated patch for 2.6.3-2
Date: Tue, 1 Mar 2005 20:13:54 +0000
Only in rsync-2.6.3: Makefile
Only in rsync-2.6.3: access.o
Only in rsync-2.6.3: authenticate.o
Only in rsync-2.6.3: backup.o
Only in rsync-2.6.3: batch.o
Only in rsync-2.6.3: build
Only in rsync-2.6.3: checksum.o
Only in rsync-2.6.3: cleanup.o
Only in rsync-2.6.3: clientname.o
Only in rsync-2.6.3: clientserver.o
Only in rsync-2.6.3: compat.o
Only in rsync-2.6.3: config.h
Only in rsync-2.6.3: config.log
Only in rsync-2.6.3: config.status
Only in rsync-2.6.3: connection.o
diff -ru orig/rsync-2.6.3/debian/changelog rsync-2.6.3/debian/changelog
--- orig/rsync-2.6.3/debian/changelog	Tue Mar  1 20:12:12 2005
+++ rsync-2.6.3/debian/changelog	Tue Mar  1 20:04:52 2005
@@ -1,3 +1,9 @@
+rsync (2.6.3-2.0iwj.0.1) unstable; urgency=low
+
+  * New --unbuffered-stdout option (bug #48108).
+
+ -- Ian Jackson <ian@davenant.greenend.org.uk>  Tue,  1 Mar 2005 20:04:52 +0000
+
 rsync (2.6.3-2) unstable; urgency=low
 
   * Now supports alternate location for rsync daemon config file through
Only in rsync-2.6.3/debian: changelog.orig
Only in rsync-2.6.3/debian: changelog.rej
Only in rsync-2.6.3/debian: changelog~
Only in rsync-2.6.3/debian: files
Only in rsync-2.6.3/debian: substvars
Only in rsync-2.6.3/debian: tmp
Only in rsync-2.6.3: exclude.o
Only in rsync-2.6.3: fileio.o
Only in rsync-2.6.3: flist.o
Only in rsync-2.6.3: generator.o
Only in rsync-2.6.3: hlink.o
Only in rsync-2.6.3: io.o
Only in rsync-2.6.3/lib: compat.o
Only in rsync-2.6.3/lib: dummy
Only in rsync-2.6.3/lib: mdfour.o
Only in rsync-2.6.3/lib: permstring.o
Only in rsync-2.6.3/lib: pool_alloc.o
Only in rsync-2.6.3/lib: snprintf.o
Only in rsync-2.6.3/lib: wildmatch.o
Only in rsync-2.6.3: loadparm.o
Only in rsync-2.6.3: log.o
diff -ru orig/rsync-2.6.3/main.c rsync-2.6.3/main.c
--- orig/rsync-2.6.3/main.c	Wed Sep 29 18:58:07 2004
+++ rsync-2.6.3/main.c	Tue Mar  1 20:06:06 2005
@@ -30,6 +30,7 @@
 extern int am_generator;
 extern int am_daemon;
 extern int verbose;
+extern int unbuffered_stdout;
 extern int blocking_io;
 extern int cvs_exclude;
 extern int delete_mode;
@@ -931,6 +932,10 @@
 	if (!am_sender && argc > 1) {
 		usage(FERROR);
 		exit_cleanup(RERR_SYNTAX);
+	}
+
+	if (unbuffered_stdout) {
+		setvbuf(stdout,0,_IONBF,0);
 	}
 
 	/* ... or no dest at all */
Only in rsync-2.6.3: main.c.orig
Only in rsync-2.6.3: main.c.rej
Only in rsync-2.6.3: main.c~
Only in rsync-2.6.3: main.o
Only in rsync-2.6.3: match.o
diff -ru orig/rsync-2.6.3/options.c rsync-2.6.3/options.c
--- orig/rsync-2.6.3/options.c	Tue Mar  1 20:12:12 2005
+++ rsync-2.6.3/options.c	Tue Mar  1 20:08:05 2005
@@ -132,6 +132,7 @@
 int link_dest = 0;
 
 int verbose = 0;
+int unbuffered_stdout = 0;
 int quiet = 0;
 int always_checksum = 0;
 int list_only = 0;
@@ -268,6 +269,7 @@
   rprintf(F,"     --delete-excluded       also delete excluded files on the receiving side\n");
   rprintf(F,"     --delete-after          receiver deletes after transferring, not before\n");
   rprintf(F,"     --ignore-errors         delete even if there are I/O errors\n");
+  rprintf(F,"     --unbuffered-stdout     do not buffer stdout, even if it is not a terminal\n");
   rprintf(F,"     --max-delete=NUM        don't delete more than NUM files\n");
   rprintf(F,"     --partial               keep partially transferred files\n");
   rprintf(F,"     --partial-dir=DIR       put a partially transferred file into DIR\n");
@@ -388,6 +390,7 @@
   {"partial",          0,  POPT_ARG_NONE,   &keep_partial, 0, 0, 0 },
   {"partial-dir",      0,  POPT_ARG_STRING, &partial_dir, 0, 0, 0 },
   {"ignore-errors",    0,  POPT_ARG_NONE,   &ignore_errors, 0, 0, 0 },
+  {"unbuffered-stdout",0,  POPT_ARG_NONE,   &unbuffered_stdout, 0, 0, 0 },
   {"blocking-io",      0,  POPT_ARG_VAL,    &blocking_io, 1, 0, 0 },
   {"no-blocking-io",   0,  POPT_ARG_VAL,    &blocking_io, 0, 0, 0 },
   {0,                 'P', POPT_ARG_NONE,   0,              'P', 0, 0 },
@@ -926,7 +929,7 @@
 	for (i = 0; i < verbose; i++)
 		argstr[x++] = 'v';
 
-	/* the -q option is intentionally left out */
+	/* the -q and --unbuffered-stdout options are intentionally left out */
 	if (make_backups)
 		argstr[x++] = 'b';
 	if (update_only)
Only in rsync-2.6.3: options.c.orig
Only in rsync-2.6.3: options.c.rej
Only in rsync-2.6.3: options.c~
Only in rsync-2.6.3: options.o
Only in rsync-2.6.3: params.o
Only in rsync-2.6.3: pipe.o
Only in rsync-2.6.3/popt: dummy
Only in rsync-2.6.3: progress.o
Only in rsync-2.6.3: receiver.o
Only in rsync-2.6.3: rsync
Only in rsync-2.6.3: rsync.o
Only in rsync-2.6.3: sender.o
Only in rsync-2.6.3: shconfig
Only in rsync-2.6.3: socket.o
Only in rsync-2.6.3: syscall.o
Only in rsync-2.6.3: token.o
Only in rsync-2.6.3: uidlist.o
Only in rsync-2.6.3: util.o
Only in rsync-2.6.3/zlib: adler32.o
Only in rsync-2.6.3/zlib: deflate.o
Only in rsync-2.6.3/zlib: dummy
Only in rsync-2.6.3/zlib: infblock.o
Only in rsync-2.6.3/zlib: infcodes.o
Only in rsync-2.6.3/zlib: inffast.o
Only in rsync-2.6.3/zlib: inflate.o
Only in rsync-2.6.3/zlib: inftrees.o
Only in rsync-2.6.3/zlib: infutil.o
Only in rsync-2.6.3/zlib: trees.o
Only in rsync-2.6.3/zlib: zutil.o



Tags added: patch Request was from Ian Jackson <ian@davenant.greenend.org.uk> to control@bugs.debian.org. Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Paul Slootman <paul@debian.org>:
Bug#48108; Package rsync. Full text and rfc822 format available.

Acknowledgement sent to Ian Jackson <ian@davenant.greenend.org.uk>:
Extra info received and forwarded to list. Copy sent to Paul Slootman <paul@debian.org>. Full text and rfc822 format available.

Message #54 received at 48108@bugs.debian.org (full text, mbox):

From: Ian Jackson <ian@davenant.greenend.org.uk>
To: 48108@bugs.debian.org
Subject: Updated patch again
Date: Mon, 3 Apr 2006 23:07:03 +0100
Here is the diff updated to apply cleanly to 2.6.4-6.

I notice that you didn't reply to my explanation of the 25th of May
2004, nor to my updated patch of the 1st of March 2005.  Perhaps you
would like me to make an NMU ?

Thanks,
Ian.



Information forwarded to debian-bugs-dist@lists.debian.org, Paul Slootman <paul@debian.org>:
Bug#48108; Package rsync. Full text and rfc822 format available.

Acknowledgement sent to Ian Jackson <ian@davenant.greenend.org.uk>:
Extra info received and forwarded to list. Copy sent to Paul Slootman <paul@debian.org>. Full text and rfc822 format available.

Message #59 received at 48108@bugs.debian.org (full text, mbox):

From: Ian Jackson <ian@davenant.greenend.org.uk>
To: 48108@bugs.debian.org
Subject: Updated patch again - really this time
Date: Mon, 3 Apr 2006 23:10:50 +0100
[Message part 1 (text/plain, inline)]
Sorry, I forgot to attach the patch to my last message.

Ian.

[rsync-unbuf-stdout-2.6.4.patch (text/plain, inline)]
diff -ru orig/rsync-2.6.4/debian/changelog rsync-2.6.4/debian/changelog
--- orig/rsync-2.6.4/debian/changelog	2006-04-03 22:56:51.000000000 +0100
+++ rsync-2.6.4/debian/changelog	2006-04-03 22:53:46.000000000 +0100
@@ -1,3 +1,9 @@
+rsync (2.6.4-6.0iwj.0.1) unstable; urgency=low
+
+  * New --unbuffered-stdout option (bug #48108).
+
+ -- Ian Jackson <ian@davenant.greenend.org.uk>  Mon,  3 Apr 2006 22:53:46 +0100
+
 rsync (2.6.4-6) unstable; urgency=high
 
   * Fixed grave bug:
Only in rsync-2.6.4/debian: changelog.orig
Only in rsync-2.6.4/debian: changelog.rej~
Only in rsync-2.6.4/debian: changelog~
diff -ru orig/rsync-2.6.4/main.c rsync-2.6.4/main.c
--- orig/rsync-2.6.4/main.c	2005-03-30 17:55:11.000000000 +0100
+++ rsync-2.6.4/main.c	2006-04-03 22:54:24.000000000 +0100
@@ -22,6 +22,7 @@
 #include "rsync.h"
 
 extern int verbose;
+extern int unbuffered_stdout;
 extern int dry_run;
 extern int list_only;
 extern int am_root;
@@ -953,6 +954,10 @@
 		exit_cleanup(RERR_SYNTAX);
 	}
 
+	if (unbuffered_stdout) {
+		setvbuf(stdout,0,_IONBF,0);
+	}
+
 	/* ... or no dest at all */
 	if (!am_sender && argc == 0)
 		list_only |= 1;
Only in rsync-2.6.4: main.c.orig
Only in rsync-2.6.4: main.c.rej~
Only in rsync-2.6.4: main.c~
diff -ru orig/rsync-2.6.4/options.c rsync-2.6.4/options.c
--- orig/rsync-2.6.4/options.c	2006-04-03 22:56:51.000000000 +0100
+++ rsync-2.6.4/options.c	2006-04-03 22:55:50.000000000 +0100
@@ -150,6 +150,7 @@
 char *dest_option = NULL;
 
 int verbose = 0;
+int unbuffered_stdout = 0;
 int quiet = 0;
 int log_before_transfer = 0;
 int log_format_has_i = 0;
@@ -341,6 +342,7 @@
   rprintf(F,"     --progress              show progress during transfer\n");
   rprintf(F," -P                          same as --partial --progress\n");
   rprintf(F," -i, --itemize-changes       output a change-summary for all updates\n");
+  rprintf(F,"     --unbuffered-stdout     do not buffer stdout, even if it is not a terminal\n");
   rprintf(F,"     --log-format=FORMAT     log file-transfers using specified format\n");
   rprintf(F,"     --password-file=FILE    read password from FILE\n");
   rprintf(F,"     --list-only             list the files instead of copying them\n");
@@ -441,6 +443,7 @@
   {"partial-dir",      0,  POPT_ARG_STRING, &partial_dir, 0, 0, 0 },
   {"delay-updates",    0,  POPT_ARG_NONE,   &delay_updates, 0, 0, 0 },
   {"ignore-errors",    0,  POPT_ARG_NONE,   &ignore_errors, 0, 0, 0 },
+  {"unbuffered-stdout",0,  POPT_ARG_NONE,   &unbuffered_stdout, 0, 0, 0 },
   {"blocking-io",      0,  POPT_ARG_VAL,    &blocking_io, 1, 0, 0 },
   {"no-blocking-io",   0,  POPT_ARG_VAL,    &blocking_io, 0, 0, 0 },
   {0,                 'F', POPT_ARG_NONE,   0, 'F', 0, 0 },
@@ -1264,7 +1267,7 @@
 	for (i = 0; i < verbose; i++)
 		argstr[x++] = 'v';
 
-	/* the -q option is intentionally left out */
+	/* the -q and --unbuffered-stdout options are intentionally left out */
 	if (make_backups)
 		argstr[x++] = 'b';
 	if (update_only)
Only in rsync-2.6.4: options.c.orig
Only in rsync-2.6.4: options.c.rej~
Only in rsync-2.6.4: options.c~

Information forwarded to debian-bugs-dist@lists.debian.org:
Bug#48108; Package rsync. Full text and rfc822 format available.

Acknowledgement sent to Paul Slootman <paul@debian.org>:
Extra info received and forwarded to list. Full text and rfc822 format available.

Message #64 received at 48108@bugs.debian.org (full text, mbox):

From: Paul Slootman <paul@debian.org>
To: Ian Jackson <ian@davenant.greenend.org.uk>, 48108@bugs.debian.org
Subject: Re: Bug#48108: Updated patch again
Date: Tue, 4 Apr 2006 11:35:25 +0200
On Mon 03 Apr 2006, Ian Jackson wrote:

> Here is the diff updated to apply cleanly to 2.6.4-6.

Very nice, unfortunately the current version is 2.6.7-1.
2.6.4-6 is a year old (although I admit that's the version in stable at
this time).

> I notice that you didn't reply to my explanation of the 25th of May
> 2004, nor to my updated patch of the 1st of March 2005.  Perhaps you
> would like me to make an NMU ?

You want to make an NMU of 2.6.4-6? ie., 2.6.4-6.1 ? I don't see how I
can have anything against that :-)

From your response then:
: It seems to me that one role of a Debian package maintainer is to
: handle issues like this with upstream.  I don't think in general bug
: reports with sensible patches should elicit a `take it upstream'
: response.  If you think that the patch would be sensible upstream
: (which may well be true) then I think you should pass it on.

I thought it would have been clear from my reactions that I don't see
that this is useful to have in the mainstream. Hence, the qualification
"bug reports with sensible patches" doesn't apply in my view. As a
consequence, "the patch would be sensible upstream" qualification
doesn't apply either, so in my opinion I shouldn't be bothering the
upstream maintainer with it. However, you are of course free to join the
mailing list and pursue the matter if you think it is important.


: An alternative option is to make rsync _always_ unbuffer stdout when
: you say --progress, but I wasn't sure that was right.

Now that does make more sense to me, because with --progress you
indicate you want up to date info, and hence output buffering then
doesn't make much sense. If you could knock up a patch for that (against
2.6.7-1 please!) then I'd be much more prone to add that.


Paul Slootman



Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Wed Apr 16 11:31:18 2014; Machine Name: beach.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.