Debian Bug report logs - #31858
findutils: In xargs, -i and -n don't work together

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Adrian Bridgett <bridgett@hursley.ibm.com>

To: submit@bugs.debian.org

Subject: findutils: -i and -n don't work together

Date: Thu, 14 Jan 1999 11:41:27 +0000

Package: findutils
Version: 4.1-33

When -n and -i are both specified, the last one stops the first one working.

Looking at the source code, lots of options override other ones :-(

~/fixedc$ echo a b | xargs -i echo X \{\} Y
X a b Y
~/fixedc$ echo a b | xargs -i -n 1 echo X \{\} Y
X {} Y a
X {} Y b
~/fixedc$ echo a b | xargs -i\{\} echo X \{\} Y
X a b Y
~/fixedc$ echo a b | xargs -i\{\} -n 1 echo X \{\} Y
X {} Y a
X {} Y b
~/fixedc$ echo a b | xargs -n 1 -i\{\} echo X \{\} Y
X a b Y

Adrian

Adrian Bridgett <bridgett@hursley.ibm.com>
Internal: 7-245528  External: 01962-815528

Message #10 received at submit@bugs.debian.org (full text, mbox, reply):

From: Adrian Bridgett <adrian.bridgett@zetnet.co.uk>

To: bridgett@hursley.ibm.com, 31858@bugs.debian.org

Cc: submit@bugs.debian.org

Subject: Re: Bug#31858: findutils: -i and -n don't work together

Date: Thu, 14 Jan 1999 18:29:13 +0000

On Thu, Jan 14, 1999 at 11:41:27AM +0000, Adrian Bridgett wrote:
> Package: findutils
> Version: 4.1-33
> 
> When -n and -i are both specified, the last one stops the first one working.

Looking at the source, it is documented that -n -i -l all override each
other.  I couldn't see this at all in the manpage (and nor can I see why -n
and -i shouldn't be used together).  The source code is quite neat but all
the same I don't understand it :-)

Adrian

email: adrian.bridgett@zetnet.co.uk, http://www.poboxes.com/adrian.bridgett
Windows NT - Unix in beta-testing.   PGP key available on public key servers
Avoid tiresome goat sacrifices  -=-  use Debian Linux http://www.debian.org

Message #28 received at 31858@bugs.debian.org (full text, mbox, reply):

From: owner@bugs.debian.org (Debian Bug Tracking System)

To: 31858@bugs.debian.org

Cc: debian-bugs-forwarded@lists.debian.org (#31858), Kevin Dalley <kevind@rahul.net> (findutils #31858)

Subject: Processed: In xargs, -i and -n don't work together (fwd)

Date: Sun, 20 Aug 2000 21:34:27 -0500

Processing commands for control@bugs.debian.org:

> forwarded 31858 bug-findutils@gnu.org
Bug#31858: findutils: In xargs, -i and -n don't work together
Forwarded-to-address changed from findutils-bugs@prep.ai.mit.edu to bug-findutils@gnu.org.

> thanks
Stopping processing here.

Please contact me if you need assistance.

Darren Benham
(administrator, Debian Bugs database)

Message #37 received at 31858@bugs.debian.org (full text, mbox, reply):

From: James Youngman <noreply@savannah.gnu.org>

To: 31858@bugs.debian.org

Subject: [bugs #8211] using -i breaks xargs

Date: Fri, 19 Nov 2004 15:16:20 -0800

This mail is an automated notification from the bugs tracker
 of the project: findutils.

/**************************************************************************/
[bugs #8211] Latest Modifications:

Changes by: 
		James Youngman <jay@gnu.org>
'Date: 
		Fri 11/19/04 at 23:19 (GMT)

            What     | Removed                   | Added
---------------------------------------------------------------------------
         Assigned to | None                      | jay
       Fixed Release | None                      | 4.2.5







/**************************************************************************/
[bugs #8211] Full Item Snapshot:

URL: <http://savannah.gnu.org/bugs/?func=detailitem&item_id=8211>
Project: findutils
Submitted by: 0
On: Fri 03/19/04 at 15:27

Category:  xargs
Severity:  5 - Average
Item Group:  None
Resolution:  Fixed
Privacy:  Public
Assigned to:  jay
Originator Name:  
Originator Email:  
Status:  Open
Release:  4.2.4
Fixed Release:  4.2.5


Summary:  using -i breaks xargs

Original Submission:  Using -i to xargs makes it stop being useful:

$ echo * | xargs echo foo | wc -l
9
$ echo * | xargs -i echo {} foo | wc -l
xargs: argument line too long
0

I understand that the point of xargs is splitting one long command in multiple commands to avoid errors.  If -i is specified, it no longer does that, so using xargs has no advantage over executing the long command directly (and just adds some overhead).

I guess this is the same bug that was reported to Debian's bug tracking system in Jan 1999 (see <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=31858>), though the report there makes it seem like the bug is not as bad as it actually is.

Follow-up Comments
------------------


-------------------------------------------------------
Date: Mon 11/08/04 at 21:35         By: James Youngman <jay>
I have made the suggested change to the documentation (this will be included in findutils-4.2.5).   The patch is attached.

-------------------------------------------------------
Date: Sun 11/07/04 at 11:49         By: Andreas Metzler <ametzler>
aarghh, typo.
replace "only split on blanks:" with "only split on newline:"


-------------------------------------------------------
Date: Sun 11/07/04 at 11:47         By: Andreas Metzler <ametzler>
Jay writes:
> I believe that this is the behaviour of the "-i" option that > shold be expected. Certainly it seems consistent with the
> behaviour of xargs on Solaris.

I think you are right, POSIX/Susv3 (XSI extension) seems to support your interpretation that -i should change xargs from splitting at <space> to only split on blanks:

--------------------------
-I  replstr
    [XSI] [Option Start] Insert mode: utility is executed for each line from standard input, taking the entire line as a single argument,[...]
--------------------------

And I'd expect -I and -i to be the same.

More supporting data:
a9203835@login__~>uname -a
AIX login 2 5 00404A0A4C00
a9203835@login__~>echo a b | xargs -I'{}' `pwd`/a.out X {} Y
Arg 0: [/u/user5/a9203835/a.out]
Arg 1: [X]
Arg 2: [a b]
Arg 3: [Y]
a9203835@login__~>echo a b | xargs  `pwd`/a.out X {} Y
Arg 0: [/u/user5/a9203835/a.out]
Arg 1: [X]
Arg 2: [{}]
Arg 3: [Y]
Arg 4: [a]
Arg 5: [b]
a9203835@login__~>printf 'anbn' | xargs -I'{}' `pwd`/a.out X {} Y
Arg 0: [/u/user5/a9203835/a.out]
Arg 1: [X]
Arg 2: [a]
Arg 3: [Y]
Arg 0: [/u/user5/a9203835/a.out]
Arg 1: [X]
Arg 2: [b]
Arg 3: [Y]
---------------------

AIX's finds provides both -i and -I and both behave identically.

I suggest to update GNU findutil's documentation to _clearly_ note that '-i' stop xargs from splitting at blanks and makes it pass _the_ complete line to the command and /after/ that close this bug.

-------------------------------------------------------
Date: Mon 05/03/04 at 10:06         By: James Youngman <jay>
I believe that this is the behaviour of the "-i" option that shold be expected.  Certainly it seems consistent with the behaviour of xargs on Solaris.

-------------------------------------------------------
Date: Tue 03/23/04 at 05:20         By: 0 <None>
I did some further tests with the following program (C code):

main ( int argc, char **argv ) { int i; for (i = 0; i < argc; i ++) { printf("Arg %d: [%s]n", i, argv[i]); } }

It shows that -i changes the way xargs invocates its command:

$ echo 0 1 | xargs ./test
Arg 0: [./main]
Arg 1: [0]
Arg 2: [1]
$ echo 0 1 | xargs -i ./test {}
Arg 0: [./main]
Arg 1: [0 1]

strace also confirms this.  With -i xargs calls
execve("./main", ["./main", "0 1"], [/* 25 vars */])
whereas normally it calls
execve("./main", ["./main", "0", "1"], [/* 25 vars */]).

Very frustrating. :(

-------------------------------------------------------
Date: Tue 03/23/04 at 05:11         By: 0 <None>
Arg, my "nice workaround" does not really work.  It triggers some other bug in xargs' -i.

The following commands ilustrate the bug:

$ touch 0 1 ; echo * | xargs -i rm {}
rm: cannot remove `0 1': No such file or directory

Needless to say, ``xargs rm'' just works.  I am just using the -i parameter in this example to show what's broken.  I don't see any reason why the above command should fail.

There are some circunstances when the user really needs the -i option (such as to use my workaround for the first bug) and then this second bug bites him.

Is -i really as broken as it seems (and has it been that broken since 1999, when it seems its first bugs were reported) or am I just missing something?

-------------------------------------------------------
Date: Tue 03/23/04 at 04:27         By: 0 <None>
I found out a nice workaround:
$ echo * | xargs | xargs -i echo {} foo | wc -l 
9

Still, the bug should be fixed, really fixed.




CC List
-------

CC Address                          | Comment
------------------------------------+-----------------------------
31858 --AT-- bugs --DOT-- debian --DOT-- org | similar report in Debian BTS



File Attachments
-------------------

-------------------------------------------------------
Date: Mon 11/08/04 at 21:35  Name: xargs-i-doc.patch  Size: 2.39KB   By: jay

http://savannah.gnu.org/bugs/download.php?item_id=8211&amp;item_file_id=1853






For detailed info, follow this link:
<http://savannah.gnu.org/bugs/?func=detailitem&item_id=8211>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/

Message #42 received at 31858@bugs.debian.org (full text, mbox, reply):

From: James Youngman <noreply@savannah.gnu.org>

To: 31858@bugs.debian.org

Subject: [bugs #8211] using -i breaks xargs

Date: Fri, 19 Nov 2004 15:16:41 -0800

This mail is an automated notification from the bugs tracker
 of the project: findutils.

/**************************************************************************/
[bugs #8211] Latest Modifications:

Changes by: 
		James Youngman <jay@gnu.org>
'Date: 
		Fri 11/19/04 at 23:19 (GMT)

            What     | Removed                   | Added
---------------------------------------------------------------------------
              Status | Open                      | Closed







/**************************************************************************/
[bugs #8211] Full Item Snapshot:

URL: <http://savannah.gnu.org/bugs/?func=detailitem&item_id=8211>
Project: findutils
Submitted by: 0
On: Fri 03/19/04 at 15:27

Category:  xargs
Severity:  5 - Average
Item Group:  None
Resolution:  Fixed
Privacy:  Public
Assigned to:  jay
Originator Name:  
Originator Email:  
Status:  Closed
Release:  4.2.4
Fixed Release:  4.2.5


Summary:  using -i breaks xargs

Original Submission:  Using -i to xargs makes it stop being useful:

$ echo * | xargs echo foo | wc -l
9
$ echo * | xargs -i echo {} foo | wc -l
xargs: argument line too long
0

I understand that the point of xargs is splitting one long command in multiple commands to avoid errors.  If -i is specified, it no longer does that, so using xargs has no advantage over executing the long command directly (and just adds some overhead).

I guess this is the same bug that was reported to Debian's bug tracking system in Jan 1999 (see <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=31858>), though the report there makes it seem like the bug is not as bad as it actually is.

Follow-up Comments
------------------


-------------------------------------------------------
Date: Mon 11/08/04 at 21:35         By: James Youngman <jay>
I have made the suggested change to the documentation (this will be included in findutils-4.2.5).   The patch is attached.

-------------------------------------------------------
Date: Sun 11/07/04 at 11:49         By: Andreas Metzler <ametzler>
aarghh, typo.
replace "only split on blanks:" with "only split on newline:"


-------------------------------------------------------
Date: Sun 11/07/04 at 11:47         By: Andreas Metzler <ametzler>
Jay writes:
> I believe that this is the behaviour of the "-i" option that > shold be expected. Certainly it seems consistent with the
> behaviour of xargs on Solaris.

I think you are right, POSIX/Susv3 (XSI extension) seems to support your interpretation that -i should change xargs from splitting at <space> to only split on blanks:

--------------------------
-I  replstr
    [XSI] [Option Start] Insert mode: utility is executed for each line from standard input, taking the entire line as a single argument,[...]
--------------------------

And I'd expect -I and -i to be the same.

More supporting data:
a9203835@login__~>uname -a
AIX login 2 5 00404A0A4C00
a9203835@login__~>echo a b | xargs -I'{}' `pwd`/a.out X {} Y
Arg 0: [/u/user5/a9203835/a.out]
Arg 1: [X]
Arg 2: [a b]
Arg 3: [Y]
a9203835@login__~>echo a b | xargs  `pwd`/a.out X {} Y
Arg 0: [/u/user5/a9203835/a.out]
Arg 1: [X]
Arg 2: [{}]
Arg 3: [Y]
Arg 4: [a]
Arg 5: [b]
a9203835@login__~>printf 'anbn' | xargs -I'{}' `pwd`/a.out X {} Y
Arg 0: [/u/user5/a9203835/a.out]
Arg 1: [X]
Arg 2: [a]
Arg 3: [Y]
Arg 0: [/u/user5/a9203835/a.out]
Arg 1: [X]
Arg 2: [b]
Arg 3: [Y]
---------------------

AIX's finds provides both -i and -I and both behave identically.

I suggest to update GNU findutil's documentation to _clearly_ note that '-i' stop xargs from splitting at blanks and makes it pass _the_ complete line to the command and /after/ that close this bug.

-------------------------------------------------------
Date: Mon 05/03/04 at 10:06         By: James Youngman <jay>
I believe that this is the behaviour of the "-i" option that shold be expected.  Certainly it seems consistent with the behaviour of xargs on Solaris.

-------------------------------------------------------
Date: Tue 03/23/04 at 05:20         By: 0 <None>
I did some further tests with the following program (C code):

main ( int argc, char **argv ) { int i; for (i = 0; i < argc; i ++) { printf("Arg %d: [%s]n", i, argv[i]); } }

It shows that -i changes the way xargs invocates its command:

$ echo 0 1 | xargs ./test
Arg 0: [./main]
Arg 1: [0]
Arg 2: [1]
$ echo 0 1 | xargs -i ./test {}
Arg 0: [./main]
Arg 1: [0 1]

strace also confirms this.  With -i xargs calls
execve("./main", ["./main", "0 1"], [/* 25 vars */])
whereas normally it calls
execve("./main", ["./main", "0", "1"], [/* 25 vars */]).

Very frustrating. :(

-------------------------------------------------------
Date: Tue 03/23/04 at 05:11         By: 0 <None>
Arg, my "nice workaround" does not really work.  It triggers some other bug in xargs' -i.

The following commands ilustrate the bug:

$ touch 0 1 ; echo * | xargs -i rm {}
rm: cannot remove `0 1': No such file or directory

Needless to say, ``xargs rm'' just works.  I am just using the -i parameter in this example to show what's broken.  I don't see any reason why the above command should fail.

There are some circunstances when the user really needs the -i option (such as to use my workaround for the first bug) and then this second bug bites him.

Is -i really as broken as it seems (and has it been that broken since 1999, when it seems its first bugs were reported) or am I just missing something?

-------------------------------------------------------
Date: Tue 03/23/04 at 04:27         By: 0 <None>
I found out a nice workaround:
$ echo * | xargs | xargs -i echo {} foo | wc -l 
9

Still, the bug should be fixed, really fixed.




CC List
-------

CC Address                          | Comment
------------------------------------+-----------------------------
31858 --AT-- bugs --DOT-- debian --DOT-- org | similar report in Debian BTS



File Attachments
-------------------

-------------------------------------------------------
Date: Mon 11/08/04 at 21:35  Name: xargs-i-doc.patch  Size: 2.39KB   By: jay

http://savannah.gnu.org/bugs/download.php?item_id=8211&amp;item_file_id=1853






For detailed info, follow this link:
<http://savannah.gnu.org/bugs/?func=detailitem&item_id=8211>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/

Message #49 received at 31858-close@bugs.debian.org (full text, mbox, reply):

From: Andreas Metzler <ametzler@debian.org>

To: 31858-close@bugs.debian.org

Subject: Bug#31858: fixed in findutils 4.2.20-2

Date: Thu, 09 Jun 2005 14:32:18 -0400

Source: findutils
Source-Version: 4.2.20-2

We believe that the bug you reported is fixed in the latest version of
findutils, which is due to be installed in the Debian FTP archive:

findutils_4.2.20-2.diff.gz
  to pool/main/f/findutils/findutils_4.2.20-2.diff.gz
findutils_4.2.20-2.dsc
  to pool/main/f/findutils/findutils_4.2.20-2.dsc
findutils_4.2.20-2_i386.deb
  to pool/main/f/findutils/findutils_4.2.20-2_i386.deb



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 31858@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Andreas Metzler <ametzler@debian.org> (supplier of updated findutils package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.7
Date: Thu,  9 Jun 2005 20:07:37 +0200
Source: findutils
Binary: findutils
Architecture: source i386
Version: 4.2.20-2
Distribution: unstable
Urgency: low
Maintainer: Andreas Metzler <ametzler@debian.org>
Changed-By: Andreas Metzler <ametzler@debian.org>
Description: 
 findutils  - utilities for finding files--find, xargs, and locate
Closes: 5956 31858 67782 102914 111143 111578 159221 162838 169730 175372 176201 185202 185203 192330 202512 219855 244766 246040 254676 256367 260780 261598 262476 286977 295399 300323 300429 300631
Changes: 
 findutils (4.2.20-2) unstable; urgency=low
 .
   * Upload to unstable.
 .
 findutils (4.2.20-1) experimental; urgency=low
 .
   * [locate] Do not index cifs (Closes: #295399) lustre (Closes: #300323) and
     nfs4 (Closes: #300631) filesystems by default.
   * [locate] Ignore /media by default. (Closes: #300429)
   * New upstream version 4.2.20
     - includes up-to-date Italian translation. (Closes: #286977)
 .
 findutils (4.2.14-1) experimental; urgency=low
 .
   * New upstream version
     - includes patch to fix savannah #11495.
     - locate: New options  -L, --follow|-P, -H, --nofollow for ignoring
       dangling symlinks (or not). (Closes: #159221)
     - find supports -exec ... {} +.
     - locate support for FreeBSD options (Sclms, m and s are ignored, but
       undocumented, I've already opened upstream report #11730 on that.)
       (Closes: #102914)
     - find built with --enable-d_type-optimisation, for using
       readdir/getdents. (Closes: #202512)
 .
 findutils (4.2.11-2) experimental; urgency=low
 .
   * find -printf '%n' (hard link count) was broken (savannah #11495).
 .
 findutils (4.2.11-1) experimental; urgency=low
 .
   * New upstream version.
 .
 findutils (4.2.10.CVS20041219-1) experimental; urgency=low
 .
   * New upstream, CVS from CVS20041219 (pre- 4.2.11).
     * -printf '%P' works correctly again.
   * Run testsuite.
 .
 findutils (4.2.9-1) experimental; urgency=low
 .
   * New upstream source
 .
 findutils (4.2.8-1) experimental; urgency=low
 .
   * New upstream version
     - New option --arg-file=file and extended documentation about
       stdin-handling (short in manpage, more extensive in info).
       (Closes: #5956)
     - improved behavior on automounted directories.
 .
 findutils (4.2.5-1) experimental; urgency=low
 .
   * New upstream version
     - includes 25_destdir_localstatedir.dpatch, our last remaining patch, we
       are vanilla now.
   * Add minimal debian/README.source.
   * Chuan-kai Lin <cklin@debian.org> has offered to serve as backup
     maintainer. Thanks. Add him to Uploaders.
 .
 findutils (4.2.4-1) experimental; urgency=low
 .
   * New upstream version
     - Closes: #175372: findutils: man pages have L's on references
     - includes 10_updatedb-findopts patch.
     - does not print warning about listing options after non-options unless
       connected to a tty.
     - xargs now uses 128Kb instead of 20Kb of command line by default, as
       ARG_MAX is 131072 on linux this Closes: #261598.
 .
 findutils (4.2.3.CVS20041106-1) experimental; urgency=low
 .
   * New upstream version (pre-4.2.4, CVS 2004-11-06).
     - Closes: #176201: findutils: xargs enviroment size limited to 20k
     - Closes: #254676: xargs: environment is too large for exec
     - Closes: #219855: -printf: accepts C octal escapes, contrary to the
       documentation.
     - Closes: #246040: findutils: find -printf %H segfaults
     - Closes: #185202: findutils: extra ")" not caught
     - Closes: #185203: findutils: overflow causes -mtime to succeed
     - Closes: #244766: /usr/bin/locate: locate should have equivalent of
       -print0 option
     - documents type of supported regular expressions for -regex
       (Closes: #162838)
     - Closes: #256367: Deprecated usage of trap in locate/updatedb.sh
     - includes some examples in find.1 manpage. (Closes: #111578)
     - new option -ignore_readdir_race to selectively suppress the obnoxious
       race-condition caused errormessages. (Closes: #67782)
     - -print0/-0 issues explained in more detail. (Closes: #111143)
     - includes supplied patch for GNU/FreeBSD. (Closes: #192330)
     - updatedb invokes "cd /" before running commands as unprivileged user.
       (Closes: #262476)
     - 'xargs -n 1 -i' works. (Closes: #31858). Please note that it is no bug
       that -i changes xargs to pass a whole line of input as a single argument
       instead of splitting on any blank characters as without -i. - Other
       implementations behave the same and the -I XSI extension documented in
       SUSv3/POSIX works like this, too.
 .
    * Removed unnecessary patches: 10_missing_includes 20_missing_newlines
      30_locate_nologinsh 35_updatedb-location 36_savannah-8623 40_direntry
      50_install_info
   * Add shfs to PRUNFS (Closes: #260780)
   * New patch: 10_updatedb-findopts: Add --findoptions option to updatedb to
     pass global options to find.
   * Set FINDOPTIONS='-ignore_readdir_race' for updatedb.sh's cron-job.
     (Closes: #169730)
   * standards-version 3.6.1 (no changes required).
Files: 
 d8131ceadf2d1c15b52622c533021e9f 663 base required findutils_4.2.20-2.dsc
 3be936480966e5d4c165047efe0f0b4b 12033 base required findutils_4.2.20-2.diff.gz
 652a8b1a819cacd6fff1cc0033a11032 306194 base required findutils_4.2.20-2_i386.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFCqIguHTOcZYuNdmMRAiVzAJ9SXQUwkrX+XJpMXqozjAJW4b2/eQCfVmHn
9PgyJmTGtS+ww4AI2JuJi4o=
=fOpd
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.