Debian Bug report logs - #903391
diffoscope fails on deb archives that contain non-compressed control.tar

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Roderich Schupp <roderich.schupp@gmail.com>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: diffoscope fails on deb archives that contain non-compressed control.tar

Date: Mon, 09 Jul 2018 13:24:03 +0200

Package: diffoscope
Version: 99
Severity: normal

How to reproduce: generate two debs with non-compressed control.tar members,
e.g. in the diffoscope source directory run:

top=$(pwd)
for i in 1 2 ; do
  t=$(mktemp -d)
  ( cd $t
    ar xv $top/tests/data/test$i.deb
    gunzip control.tar.gz
    ar rv $top/non-compressed-control$i.deb debian-binary control.tar
data.tar.gz
  )
  rm -rf $t
done

And then

$ diffoscope non-compressed-control*.deb
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/diffoscope/main.py", line 460, in main
    sys.exit(run_diffoscope(parsed_args))
  File "/usr/lib/python3/dist-packages/diffoscope/main.py", line 432, in
run_diffoscope
    difference = compare_root_paths(path1, path2)
  File "/usr/lib/python3/dist-
packages/diffoscope/comparators/utils/compare.py", line 68, in
compare_root_paths
    difference = compare_files(file1, file2)
  File "/usr/lib/python3/dist-
packages/diffoscope/comparators/utils/compare.py", line 118, in compare_files
    return file1.compare(file2, source)
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py",
line 366, in compare
    difference = self._compare_using_details(other, source)
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py",
line 321, in _compare_using_details
    other.as_container, no_recurse=no_recurse))
  File "/usr/lib/python3/dist-
packages/diffoscope/comparators/utils/container.py", line 177, in compare_pair
    file1, file2, source=None, diff_content_only=no_recurse)
  File "/usr/lib/python3/dist-
packages/diffoscope/comparators/utils/compare.py", line 118, in compare_files
    return file1.compare(file2, source)
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py",
line 366, in compare
    difference = self._compare_using_details(other, source)
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py",
line 321, in _compare_using_details
    other.as_container, no_recurse=no_recurse))
  File "/usr/lib/python3/dist-
packages/diffoscope/comparators/utils/container.py", line 177, in compare_pair
    file1, file2, source=None, diff_content_only=no_recurse)
  File "/usr/lib/python3/dist-
packages/diffoscope/comparators/utils/compare.py", line 118, in compare_files
    return file1.compare(file2, source)
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py",
line 366, in compare
    difference = self._compare_using_details(other, source)
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py",
line 321, in _compare_using_details
    other.as_container, no_recurse=no_recurse))
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/deb.py", line
180, in comparisons
    my_md5sums = self.source.container.source.container.source.md5sums
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/deb.py", line
114, in md5sums
    self._md5sums = md5sums_file.parse()
AttributeError: 'TextFile' object has no attribute 'parse'


I think the problem is that comparators/deb.py assumes that a "md5sums" file is
always two levels "down" in its container (control.tar.gz -> control.tar ->
md5sums) which is false when control.tar isn't compressed.

Similar problems ensue when data.tar isn't compressed.
Probably connected to #879217.

Cheers, Roderich



-- System Information:
Debian Release: buster/sid
  APT prefers unstable-debug
  APT policy: (500, 'unstable-debug'), (500, 'unstable'), (1, 'experimental-debug'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.17.4 (SMP w/4 CPU cores)
Locale: LANG=en_US.utf8, LC_CTYPE=en_US.utf8 (charmap=UTF-8), LANGUAGE=en_US.utf8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages diffoscope depends on:
ii  libpython3.6-stdlib    3.6.6-1
ii  python3                3.6.6-1
ii  python3-distro         1.0.1-2
ii  python3-distutils      3.6.6-1
ii  python3-libarchive-c   2.1-3.1
ii  python3-magic          2:0.4.15-1
ii  python3-pkg-resources  39.2.0-1

Versions of packages diffoscope recommends:
pn  abootimg                         <none>
ii  acl                              2.2.52-3+b1
pn  apktool                          <none>
pn  binutils-multiarch               <none>
ii  bzip2                            1.0.6-8.1
ii  caca-utils                       0.99.beta19-2+b3
ii  colord                           1.3.3-2
ii  db-util                          5.3.1
ii  default-jdk [java-sdk]           2:1.10-67
ii  default-jdk-headless             2:1.10-67
ii  device-tree-compiler             1.4.6-1
pn  docx2txt                         <none>
ii  e2fsprogs                        1.44.3~rc2-1
pn  enjarify                         <none>
pn  fontforge-extras                 <none>
pn  fp-utils                         <none>
ii  genisoimage                      9:1.1.11-3+b2
ii  gettext                          0.19.8.1-6+b1
ii  ghc                              8.2.2-4
ii  ghostscript                      9.22~dfsg-2.1
ii  giflib-tools                     5.1.4-3
pn  gnumeric                         <none>
ii  gnupg                            2.2.8-3
ii  imagemagick                      8:6.9.10.2+dfsg-2
ii  imagemagick-6.q16 [imagemagick]  8:6.9.10.2+dfsg-2
pn  jsbeautifier                     <none>
ii  libarchive-tools                 3.2.2-4
pn  llvm                             <none>
ii  lz4                              1.8.2-1
ii  mono-utils                       4.6.2.7+dfsg-2
pn  odt2txt                          <none>
pn  oggvideotools                    <none>
ii  openjdk-10-jdk [java-sdk]        10.0.1+10-4
ii  openjdk-11-jdk [java-sdk]        11~21-2
ii  openssh-client                   1:7.7p1-2
pn  pgpdump                          <none>
ii  poppler-utils                    0.63.0-2
pn  procyon-decompiler               <none>
ii  python3-argcomplete              1.8.1-1
pn  python3-binwalk                  <none>
ii  python3-debian                   0.1.32
ii  python3-defusedxml               0.5.0-1
pn  python3-guestfs                  <none>
pn  python3-jsondiff                 <none>
pn  python3-progressbar              <none>
pn  python3-pyxattr                  <none>
ii  python3-tlsh                     3.4.4+20151206-1+b4
pn  r-base-core                      <none>
ii  rpm2cpio                         4.14.1+dfsg1-3
pn  sng                              <none>
ii  sqlite3                          3.24.0-1
ii  squashfs-tools                   1:4.3-6
ii  tcpdump                          4.9.2-3
ii  unzip                            6.0-21
ii  vim-common                       2:8.1.0089-1
ii  xmlbeans                         2.6.0+dfsg-3
ii  xxd                              2:8.1.0089-1
ii  xz-utils                         5.2.2-1.3

Versions of packages diffoscope suggests:
ii  libjs-jquery  3.2.1-1

-- no debconf information

Message #10 received at 903391@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>

To: 903391@bugs.debian.org

Cc: Roderich Schupp <roderich.schupp@gmail.com>

Subject: Re: diffoscope fails on deb archives that contain non-compressed control.tar

Date: Mon, 09 Jul 2018 13:39:39 +0100

tags 903391 + pending
thanks

Fixed in Git, pending upload:

  https://salsa.debian.org/reproducible-builds/diffoscope/commit/dc9ee98bdfce7d1a9564fab6e35f57802d6854f4

  diffoscope/comparators/deb.py |   2 +-
  tests/comparators/test_deb.py |   6 ++++++
  tests/data/bug903391_1.deb    | Bin 0 -> 12058 bytes
  tests/data/bug903391_2.deb    | Bin 0 -> 12186 bytes
  4 files changed, 7 insertions(+), 1 deletion(-)


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

Message #17 received at 903391-close@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>

To: 903391-close@bugs.debian.org

Subject: Bug#903391: fixed in diffoscope 100

Date: Sat, 01 Sep 2018 08:34:14 +0000

Source: diffoscope
Source-Version: 100

We believe that the bug you reported is fixed in the latest version of
diffoscope, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 903391@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Chris Lamb <lamby@debian.org> (supplier of updated diffoscope package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Sat, 01 Sep 2018 09:09:48 +0100
Source: diffoscope
Binary: diffoscope
Built-For-Profiles: nocheck
Architecture: source all
Version: 100
Distribution: unstable
Urgency: medium
Maintainer: Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>
Changed-By: Chris Lamb <lamby@debian.org>
Description:
 diffoscope - in-depth comparison of files, archives, and directories
Closes: 903391 903401 903447 903449 903565 904685 905598 906967
Changes:
 diffoscope (100) unstable; urgency=medium
 .
   [ Alexis Murzeau ]
   * Correct matching of .deb archive members. (Closes: #903565)
 .
   [ Chris Lamb ]
   * Support .deb archives that contain an uncompressed control.tar and data.tar
     Thanks to Roderich Schupp <roderich.schupp@gmail.com>.
     (Closes: #903391, #903401)
   * Wrap jsondiff calls with try-except to prevent fatal errors.
     (Closes: #903447, #903449)
   * Don't include the filename in llvm-bcanalyzer results. (Closes: #905598)
   * Update generated debian/tests/control.
   * Bump Standards-Version to 4.2.1.
 .
   [ Daniel Kahn Gillmor ]
   * Avoid line eraser error on dumb terminals. (Closes: #906967)
   * Correct spelling of ereser to eraser.
 .
   [ Mattia Rizzolo ]
   * On Debian, do not require apktool on ppc64el and s390x, ogvideotools on
     s390x and fp-utils on ppc64el and s390x as they are not installable there.
   * Explicitly add `file` to the dependencies of autopkgtest to have the tests
     triggered whenever `file` changes
 .
   [ Ricardo Gaviria ]
   * Handle errors with encrypted archive files. (Closes: #904685)
Checksums-Sha1:
 ee418ed8178f58ab456f7f7f99330c9ea4b3da42 4050 diffoscope_100.dsc
 bcf9d36178a98c9aeb24457588ba0638152661ed 9251936 diffoscope_100.tar.xz
 a60721094ec8ed73f7658baed694bfc94e128ff2 122780 diffoscope_100_all.deb
 af050ca0c265c4d181bd322c9f65323bc1bc4a45 6725 diffoscope_100_amd64.buildinfo
Checksums-Sha256:
 61cbdfa21cfbe6a6a8c32be511a13fbed0d0d1e2788ef5cc859cd2dc54c2c092 4050 diffoscope_100.dsc
 98af4a69d2ded2de221d050a110d60beb7a8b737a3bba1ac40755682c7a45b4b 9251936 diffoscope_100.tar.xz
 9f56af77c7756c9265c615117f0704d0009b8b6de2d164d6afcc8ce78d208e6f 122780 diffoscope_100_all.deb
 6fca17529ee1260aa9576840b97f3dffb0e16476435a655986bf03bc280a1fad 6725 diffoscope_100_amd64.buildinfo
Files:
 30b68b4a6dbcba4bc3aa7d948bd3f4da 4050 devel optional diffoscope_100.dsc
 148c2d1675b1f3475fc6ab28bc39eb2a 9251936 devel optional diffoscope_100.tar.xz
 cd5e06d93db32e437ca32ff5e36cce78 122780 devel optional diffoscope_100_all.deb
 f87adefc92b27a4441634aba05448839 6725 devel optional diffoscope_100_amd64.buildinfo

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEwv5L0nHBObhsUz5GHpU+J9QxHlgFAluKSc8ACgkQHpU+J9Qx
Hlg1lxAAjgTqnKwdWr3cQYA7MO4VqB11p9185NkXrR2vkHcUNvN1xv95J7/A4BZx
3BXeK2dfi8Qu1gN2NDeg6+m5FTUMDHesmXMlqrDG5JLVRHXM8a/FqpHHCAGJKLR6
QALd1uE24VSdbxBc2hJvTWDu6t1DbKvhXp83D9kBc8APvOsaxWpFNE7dDHDKEICa
EC8Ne925EEU/8xwl+LTtAN9DQEw0Pmi521NK2pBhX4NfETm6OJODb2cH/DCCMB9x
65IdU0hF4tMDzZOQ2utgWQ5Cx7Se66tcqNmamyunBocfZ77d5EA+IKsiYD2QQ7Iy
G+fhMJYTiiYPfxZvGS8ln3u29p1CFrj5vJ+Cggf7Dl0w+a8WEXIOaKPZ4KIddT7X
k2bg+dtMzr/4VcTFAy6zmtyn86hMNNkF2rdkG2EkeHNEHuP+HwnIPnmMpS5o7BEW
S9jtd93khRD+/Jp/Zen6K8X2As2Gvd3s97+0hInTPezweny/DhapNxwRc/sYLfvv
iuvrkssM4hJ+yXFogvCbI+Q4MLZpUtvXLU/IKDhTVQuPIBDNrSDQChPGOEeWYEsz
2PW+ls6ipNPWUUkL556OiN7V0dbZIEMlSLHzWdzr6FrRZwWZmJAfulO+1GIZTMyu
xwzo6q+pne8fLX7ynE53uXVpzvdAMDPDRkj4cJMMVcKojoCih0E=
=Uc9v
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.