Debian Bug report logs -
#737180
diffutils: diff exit status is 2 instead of 1 on binary files that differ
Reported by: Vincent Lefevre <vincent@vinc17.net>
Date: Fri, 31 Jan 2014 02:09:01 UTC
Severity: normal
Tags: fixed-upstream, upstream
Found in version diffutils/1:3.3-1
Fixed in version diffutils/1:3.5-1
Done: Santiago Vila <sanvila@debian.org>
Bug is archived. No further changes may be made.
Forwarded to bug-diffutils@gnu.org
Toggle useless messages
Report forwarded
to debian-bugs-dist@lists.debian.org, Santiago Vila <sanvila@debian.org>:
Bug#737180; Package diffutils.
(Fri, 31 Jan 2014 02:09:06 GMT) (full text, mbox, link).
Acknowledgement sent
to Vincent Lefevre <vincent@vinc17.net>:
New Bug report received and forwarded. Copy sent to Santiago Vila <sanvila@debian.org>.
(Fri, 31 Jan 2014 02:09:06 GMT) (full text, mbox, link).
Message #5 received at submit@bugs.debian.org (full text, mbox, reply):
Package: diffutils
Version: 1:3.3-1
Severity: normal
When diffing binary files that differ, e.g.
$ diff /usr/share/doc/diffutils/changelog.Debian.gz /usr/share/doc/diffutils/changelog.gz
Binary files /usr/share/doc/diffutils/changelog.Debian.gz and /usr/share/doc/diffutils/changelog.gz differ
zsh: exit 2 diff /usr/share/doc/diffutils/changelog.Debian.gz
I get exit status 2 instead of 1.
The diff(1) man page says:
Exit status is 0 if inputs are the same, 1 if different, 2 if trouble.
We are in the case where the inputs are different, so that the
exit status should be 1.
Because of this bug, "svn diff --force --diff-cmd diff" fails on
binary files (ditto on more useful cases in practice, where a diff
wrapper is used to recognize some binary files, using the default
diff behavior as a fallback).
-- System Information:
Debian Release: jessie/sid
APT prefers unstable
APT policy: (500, 'unstable'), (500, 'testing'), (500, 'stable'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 3.12-1-amd64 (SMP w/2 CPU cores)
Locale: LANG=POSIX, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Versions of packages diffutils depends on:
ii libc6 2.17-97
diffutils recommends no packages.
Versions of packages diffutils suggests:
ii diffutils-doc 1:3.3-1
ii wdiff 1.2.1-2
-- no debconf information
Reply sent
to Santiago Vila <sanvila@unex.es>:
You have marked Bug as forwarded.
(Fri, 31 Jan 2014 10:15:09 GMT) (full text, mbox, link).
Message #8 received at 737180-forwarded@bugs.debian.org (full text, mbox, reply):
Hello.
I received this report from the Debian bug system.
Thanks.
---------- Forwarded message ----------
From: Vincent Lefevre <vincent@vinc17.net>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Date: Fri, 31 Jan 2014 03:06:13 +0100
Subject: Bug#737180: diffutils: diff exit status is 2 instead of 1 on binary
files that differ
Package: diffutils
Version: 1:3.3-1
Severity: normal
When diffing binary files that differ, e.g.
$ diff /usr/share/doc/diffutils/changelog.Debian.gz /usr/share/doc/diffutils/changelog.gz
Binary files /usr/share/doc/diffutils/changelog.Debian.gz and /usr/share/doc/diffutils/changelog.gz differ
zsh: exit 2 diff /usr/share/doc/diffutils/changelog.Debian.gz
I get exit status 2 instead of 1.
The diff(1) man page says:
Exit status is 0 if inputs are the same, 1 if different, 2 if trouble.
We are in the case where the inputs are different, so that the
exit status should be 1.
Because of this bug, "svn diff --force --diff-cmd diff" fails on
binary files (ditto on more useful cases in practice, where a diff
wrapper is used to recognize some binary files, using the default
diff behavior as a fallback).
[...]
Message #9 received at 737180-forwarded@bugs.debian.org (full text, mbox, reply):
Santiago Vila wrote:
> Exit status is 0 if inputs are the same, 1 if different, 2 if trouble.
For Diffutils "trouble" means "the output doesn't correctly represent
the difference between the two input files". Perhaps that's not what is
wanted in this situation with a diff wrapper that supports some binary
files but not others, but I expect that there are other uses where the
current behavior is wanted. The users in question could change their
wrapper to use "diff -a", or to look for the message they don't think is
trouble and to ignore exit status 2 in that case.
In theory we could have different exit statuses for Diffutils, one for
each sort of trouble, but I'm not sure that's a road we want to head down.
Message #10 received at 737180-forwarded@bugs.debian.org (full text, mbox, reply):
On 2014-01-31 08:02:32 -0800, Paul Eggert wrote:
> Santiago Vila wrote:
> > Exit status is 0 if inputs are the same, 1 if different, 2 if trouble.
>
> For Diffutils "trouble" means "the output doesn't correctly represent the
> difference between the two input files". Perhaps that's not what is wanted
> in this situation with a diff wrapper that supports some binary files but
> not others, but I expect that there are other uses where the current
> behavior is wanted.
POSIX actually specifies the behavior:
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/diff.html
EXIT STATUS
The following exit values shall be returned:
0
No differences were found.
1
Differences were found.
>1
An error occurred.
and earlier:
Diff Binary Output Format
In the POSIX locale, if one or both of the files being compared
are not text files, it is implementation-defined whether diff uses
the binary file output format or the other formats as specified
below. The binary file output format shall contain the pathnames
of two files being compared and the string "differ".
[The formats specified below are those for text files.]
So, for the output when diffing binary files, POSIX gives the choice
between a specific format (binary output format) or one of the formats
defined for text files. It is nowhere regarded as an error.
Note that Solaris diff uses the binary output (like GNU diff), but
follows POSIX for the exit status, i.e. exits with 1.
> The users in question could change their wrapper to use "diff -a",
> or to look for the message they don't think is trouble and to ignore
> exit status 2 in that case.
"diff -a" is a bad choice because outputting binary data can corrupt
the terminal settings (or for big binary files, it can distract the
user). And there are several problems with the second solution:
1. If one wants to run "diff" only once, one needs to capture stdout
and stderr in shell variables. I'm not sure whether this is possible.
Otherwise one needs to redirect diff's stdout and stderr to temporary
files, and this is rather ugly (one needs to have some security checks
and also check whether there is an error due to the redirection itself,
but in this case, it is not possible to know whether the cause of the
error is diff itself or the redirection).
2. One can also run diff normally, and in case of exit status 2, run
it again, capturing the output, and check it. Something like:
diff "$@"
err=$?
if [[ $err -eq 2 ]] then
out=$(diff "$@" 2>/dev/null)
[[ $out = "Binary files "*" differ" ]] && err=1
fi
exit $err
But there is an obvious race condition: it can happen that for the
first invocation, there was a real error with diff, so that one wants
to exit with error code 2, but the second invocation may succeed
(because the state of the system changed), and on binary files, one
would exit with 1 instead of 2.
Moreover it is not always possible to run diff twice on the same
files, in case of special files (such as /proc/self/fd/... from
process substitution).
And it can be inefficient.
And whether (1) or (2) is chosen, in case of recursive diff, it's
completely impossible to know what is going on: there may be a
"Binary files ... differ" *and* a real error.
> In theory we could have different exit statuses for Diffutils, one
> for each sort of trouble, but I'm not sure that's a road we want to
> head down.
That would still not be acceptable w.r.t. POSIX. The default should
conform to POSIX, but an option to exit with 2 instead of 1 in case
of binary files, for those who would need this behavior. But I'm
wondering whether this is really useful: probably not when the output
is read by a human, and if used by another process, this process
should check whether its input has the expected format anyway.
--
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)
Message #11 received at 737180-forwarded@bugs.debian.org (full text, mbox, reply):
[Message part 1 (text/plain, inline)]
Vincent Lefevre wrote:
> POSIX actually specifies the behavior:
Ouch, right you are. I recall reading POSIX and coming to the opposite
conclusion, that the behavior was not specified. Perhaps this was long
ago and input files were required to be text files.
Anyway, since it's incompatible with other 'diff' implementations and
with POSIX, and since there's not a particularly strong reason to
differ, let's change to be compatible. I applied the attached patch to
the savannah master and am taking the liberty of marking this as done;
please give it a try.
[0001-diff-exit-with-status-1-not-2-when-binary-files-diff.patch (text/plain, attachment)]
Added tag(s) upstream and fixed-upstream.
Request was from Vincent Lefevre <vincent@vinc17.net>
to control@bugs.debian.org.
(Sat, 01 Feb 2014 03:51:05 GMT) (full text, mbox, link).
Message #14 received at 737180-forwarded@bugs.debian.org (full text, mbox, reply):
On 2014-01-31 17:21:42 -0800, Paul Eggert wrote:
> Anyway, since it's incompatible with other 'diff' implementations and with
> POSIX, and since there's not a particularly strong reason to differ, let's
> change to be compatible. I applied the attached patch to the savannah
> master and am taking the liberty of marking this as done; please give it a
> try.
Thanks, I've applied the patch (except the NEWS hunk) against the
Debian package, and it works.
--
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)
Reply sent
to Santiago Vila <sanvila@debian.org>:
You have taken responsibility.
(Wed, 31 Aug 2016 22:24:19 GMT) (full text, mbox, link).
Notification sent
to Vincent Lefevre <vincent@vinc17.net>:
Bug acknowledged by developer.
(Wed, 31 Aug 2016 22:24:19 GMT) (full text, mbox, link).
Message #19 received at 737180-close@bugs.debian.org (full text, mbox, reply):
Source: diffutils
Source-Version: 1:3.5-1
We believe that the bug you reported is fixed in the latest version of
diffutils, which is due to be installed in the Debian FTP archive.
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to 737180@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Santiago Vila <sanvila@debian.org> (supplier of updated diffutils package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Format: 1.8
Date: Wed, 31 Aug 2016 23:01:00 +0100
Source: diffutils
Binary: diffutils diffutils-doc
Architecture: source
Version: 1:3.5-1
Distribution: unstable
Urgency: low
Maintainer: Santiago Vila <sanvila@debian.org>
Changed-By: Santiago Vila <sanvila@debian.org>
Description:
diffutils - File comparison utilities
diffutils-doc - Documentation for GNU diffutils in HTML format
Closes: 396485 737180 804544
Changes:
diffutils (1:3.5-1) unstable; urgency=low
.
* New upstream release.
* Drop all patches, as none of them is required anymore.
* There are now --color and --palette options. Closes: #396485.
* Exit status of diff for binary files that differ is now 1, not 2,
as mandated by POSIX. Closes: #737180.
* Option --no-dereference is now documented in manpage. Closes: #804544.
Checksums-Sha1:
d52224f7949fd0f8a54703dbb07c15bfcc7f8ffd 1438 diffutils_3.5-1.dsc
1169cce8eaaf7290dc087d65db7ed75de0dceb93 1360996 diffutils_3.5.orig.tar.xz
142e7536103411d9fc77a0ae2c20003fe555f111 10436 diffutils_3.5-1.debian.tar.xz
Checksums-Sha256:
a7ae6d0ac3bd114211e0526784e99866072a08ee89b3984786d22b0f03cf83c4 1438 diffutils_3.5-1.dsc
dad398ccd5b9faca6b0ab219a036453f62a602a56203ac659b43e889bec35533 1360996 diffutils_3.5.orig.tar.xz
fd5115182ba8880ed8dcb5c62af9ef5f8b5410c2f35b026298cc405bac5b73e0 10436 diffutils_3.5-1.debian.tar.xz
Files:
72680c49b9187593471d8f67c65e15e1 1438 utils required diffutils_3.5-1.dsc
569354697ff1cfc9a9de3781361015fa 1360996 utils required diffutils_3.5.orig.tar.xz
ce964dc6b4462e208b7a781fbe6ac7de 10436 utils required diffutils_3.5-1.debian.tar.xz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQEcBAEBCAAGBQJXx1RjAAoJEEHOfwufG4sysOkH/RGY6x3mJyoKuK/pU4y6pTez
4PpkvJj8x9jmg2MViCEoA/KrqcPm4mP4UdS8HEwigyz431Z/M8xMnG22yVkZP53i
upgBr42/diQbuyNWRXRvN5VV+Gqb67rwHK7Ei8agOuTHPKcUSlp07hCLDtmWgLkB
HrNwshaKN+HeR5g+EXzkc1Rcg2rME/rsTBzq1dkzK5BQmM4+hsDitFhOCA+yS3+i
LJ9gYkx0c/T7kj3pmYUYXHl2ykvSCX09qwPGwkJ3vPU1JJpSFYyMltbXAgeaBhY3
6n0lKf8ksO94BuU9RwvsbzW9IFp7GpiS0OuPP5Awg13jvTX+V+BE/uHEhORsfvo=
=I4EX
-----END PGP SIGNATURE-----
Bug archived.
Request was from Debbugs Internal Request <owner@bugs.debian.org>
to internal_control@bugs.debian.org.
(Sun, 09 Oct 2016 07:30:38 GMT) (full text, mbox, link).
Send a report that this bug log contains spam.
Debian bug tracking system administrator <owner@bugs.debian.org>.
Last modified:
Sat Jan 6 14:47:50 2018;
Machine Name:
buxtehude
Debian Bug tracking system
Debbugs is free software and licensed under the terms of the GNU
Public License version 2. The current version can be obtained
from https://bugs.debian.org/debbugs-source/.
Copyright © 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson,
2005-2017 Don Armstrong, and many other contributors.