Debian Bug report logs - #870049
diffoscope: hangs on some LLVM bytecode files and maybe other files

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@debian.org>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: diffoscope: hangs on some LLVM bytecode files and maybe other files

Date: Sat, 29 Jul 2017 11:16:48 +0200

Package: diffoscope
Version: 84
Severity: important

Dear Maintainer,

When testing rustc reproducibility:

$ for i in {1..10}; do diffoscope core-fcc113aeec9d4b78.0.bytecode.0 core-fcc113aeec9d4b78.0.bytecode.1 | wc -l; done
# hangs first time

$ for i in {1..10}; do diffoscope --max-diff-input-lines 0 core-fcc113aeec9d4b78.0.bytecode.0 core-fcc113aeec9d4b78.0.bytecode.1 | wc -l; done
# hangs after a few (3-30) tries, seems random

$ for i in {1..10}; do diff -ru <(llvm-bcanalyzer -dump core-fcc113aeec9d4b78.0.bytecode.0) <(llvm-bcanalyzer -dump core-fcc113aeec9d4b78.0.bytecode.1) | wc -l; done
# succeeds every time

When running `pstree -alup` one can see either one or both llvm-bcanalyzer processes running. I guess there's a race condition to do with filtering command output.

X

-- System Information:
Debian Release: buster/sid
  APT prefers testing
  APT policy: (990, 'testing'), (500, 'unstable-debug'), (500, 'testing-debug'), (500, 'buildd-unstable'), (300, 'unstable'), (100, 'experimental'), (1, 'experimental-debug')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.11.0-1-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_GB.utf8, LC_CTYPE=en_GB.utf8 (charmap=UTF-8), LANGUAGE=en_GB:en (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages diffoscope depends on:
ii  python3                3.5.3-3
ii  python3-libarchive-c   2.1-3.1
ii  python3-magic          1:5.30-1
ii  python3-pkg-resources  36.0.1-1

Versions of packages diffoscope recommends:
ii  acl                              2.2.52-3+b1
ii  apktool                          2.2.3+dfsg-1
ii  binutils-multiarch               2.28-6
ii  bzip2                            1.0.6-8.1
ii  caca-utils                       0.99.beta19-2+b2
ii  colord                           1.3.3-2
ii  default-jdk [java-sdk]           2:1.8-59
ii  default-jdk-headless             2:1.8-59
ii  device-tree-compiler             1.4.2-1
ii  docx2txt                         1.4-0.2
ii  enjarify                         1:1.0.3-3
ii  fontforge-extras                 0.3-4
ii  fp-utils-3.0.2 [fp-utils]        3.0.2+dfsg-4
ii  genisoimage                      9:1.1.11-3+b2
ii  gettext                          0.19.8.1-2+b1
ii  ghc                              8.0.1-17+b1
ii  ghostscript                      9.21~dfsg-1
ii  giflib-tools                     5.1.4-0.4
ii  gnupg                            2.1.18-8
ii  imagemagick                      8:6.9.7.4+dfsg-12
ii  imagemagick-6.q16 [imagemagick]  8:6.9.7.4+dfsg-12
ii  jsbeautifier                     1.6.4-6
ii  libarchive-tools                 3.2.2-2
ii  llvm                             1:3.8-36
ii  mono-utils                       4.6.2.7+dfsg-1
ii  odt2txt                          0.5-1+b2
ii  oggvideotools                    0.9.1-4
ii  openjdk-8-jdk [java-sdk]         8u131-b11-2
ii  openssh-client                   1:7.5p1-5
ii  pdftk                            2.02-4+b2
ii  pgpdump                          0.31-0.2
ii  poppler-utils                    0.48.0-2
ii  python3-argcomplete              1.8.1-1
ii  python3-debian                   0.1.30
ii  python3-guestfs                  1:1.34.6-2+b1
ii  python3-progressbar              2.3-4
ii  python3-rpm                      4.12.0.2+dfsg1-2+b1
ii  python3-tlsh                     3.4.4+20151206-1+b3
ii  r-base-core                      3.3.3-1
ii  rpm2cpio                         4.12.0.2+dfsg1-2+b1
ii  sng                              1.1.0-1+b1
ii  sqlite3                          3.19.3-3
ii  squashfs-tools                   1:4.3-4
ii  tcpdump                          4.9.0-3
ii  unzip                            6.0-21
ii  vim-common                       2:8.0.0197-5
ii  xxd                              2:8.0.0197-5
ii  xz-utils                         5.2.2-1.3

Versions of packages diffoscope suggests:
ii  libjs-jquery  3.1.1-2

-- no debconf information

Message #10 received at 870049@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@debian.org>

To: 870049@bugs.debian.org, Reproducible Builds discussion list <reproducible-builds@lists.alioth.debian.org>

Subject: Re: Bug#870049: diffoscope: hangs on some LLVM bytecode files and maybe other files

Date: Sat, 29 Jul 2017 09:24:00 +0000

[Message part 1 (text/plain, inline)]

Ximin Luo:
> Package: diffoscope
> Version: 84
> Severity: important
> 
> Dear Maintainer,
> 
> When testing rustc reproducibility:
> 
> $ for i in {1..10}; do diffoscope core-fcc113aeec9d4b78.0.bytecode.0 core-fcc113aeec9d4b78.0.bytecode.1 | wc -l; done
> # hangs first time
> 
> $ for i in {1..10}; do diffoscope --max-diff-input-lines 0 core-fcc113aeec9d4b78.0.bytecode.0 core-fcc113aeec9d4b78.0.bytecode.1 | wc -l; done
> # hangs after a few (3-30) tries, seems random
> 
> $ for i in {1..10}; do diff -ru <(llvm-bcanalyzer -dump core-fcc113aeec9d4b78.0.bytecode.0) <(llvm-bcanalyzer -dump core-fcc113aeec9d4b78.0.bytecode.1) | wc -l; done
> # succeeds every time
> 
> When running `pstree -alup` one can see either one or both llvm-bcanalyzer processes running. I guess there's a race condition to do with filtering command output.
> 

Whoops, forgot to attach the files. Here they are.

X

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git

[core-fcc113aeec9d4b78.0.bytecode.0 (application/octet-stream, attachment)]

[core-fcc113aeec9d4b78.0.bytecode.1 (application/octet-stream, attachment)]

Message #15 received at 870049-close@bugs.debian.org (full text, mbox, reply):

From: Mattia Rizzolo <mattia@debian.org>

To: 870049-close@bugs.debian.org

Subject: Bug#870049: fixed in diffoscope 85

Date: Mon, 31 Jul 2017 21:05:11 +0000

Source: diffoscope
Source-Version: 85

We believe that the bug you reported is fixed in the latest version of
diffoscope, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 870049@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Mattia Rizzolo <mattia@debian.org> (supplier of updated diffoscope package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.8
Date: Mon, 31 Jul 2017 22:27:45 +0200
Source: diffoscope
Binary: diffoscope
Architecture: source
Version: 85
Distribution: unstable
Urgency: medium
Maintainer: Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>
Changed-By: Mattia Rizzolo <mattia@debian.org>
Description:
 diffoscope - in-depth comparison of files, archives, and directories
Closes: 866120 868486 868534 870049
Changes:
 diffoscope (85) unstable; urgency=medium
 .
   [ Mattia Rizzolo ]
   * tools:
     + move from the deprecated platform.linux_distribution() to the external
       python package "distro".  Add it as an optioanl dependency, as without it
       distribution detection (i.e. `diffoscope --list-tools`) won't work.
       Also add it as a Debian (hard) dependency, as it's a very lightway
       package, and the usefulness it brings are great.
     + add a get_package_provider() function, returning the package name
       that best matches the system.
   * exc:
     + in RequiredToolNotFound.get_package(), just call the new
       get_package_provider()
   * debian/rules:
     + recommends the defusedxml python package, to avoid using the python3's
       standard xml library and its security holes.
 .
   [ Chris Lamb ]
   * comparators:
     + sqlite:
       - Simplify file detection by rewriting manual `recognizes` call
         with a `Sqlite3Database.RE_FILE_TYPE` definition.
     + xml:
       - Fix EPUB "missing file" tests; they ship a META-INF/container.xml file.
 .
   [ Ximin Luo ]
   * comparators:
     + factor common logic from various comparators into File.recognizes.
     + more tidying up and making names consistent.
     + directory:
       - make stat(1) warning textually like the other warnings.
       - bump stat(1) warning into an error.
       - use getfacl(1) before lsattr(1) as it's more general.
     + apk:
       - less kludgy way of detecting APKs.  Closes: #868486
   * main, logging:
     + restore old logger settings to avoid pytest fail in certain situations.
   * debian/rules:
     + add a check to prevent additions of "DOS/MBR" file type.
   * feeder:
     + force a flush when writing output to diff.  Closes: #870049
   * tests/comparators:
     + directory:
       - be less strict about the expected test output, to cope with a missing
         `getfacl`.  Closes: #868534
 .
   [ Juliana Oliveira Rodrigues ]
   * comparators:
     + Add new XML comparator.  Closes: #866120
       The comparator will use defusedxml if this is installed, to avoid
       falling in known security holes coming from the XML specification.
   * tests/comparators:
     + apk:
       - fix the tests after the addition of the XML comparator.
     + image:
       - fix test_ico_image for identify >= 6.9.8.
 .
   [ Guangyuan Yang ]
   * tests/comparators:
     + device:
       - fix 2 cases for FreeBSD.
Checksums-Sha1:
 fbf4fc1689b2b1241a3567aa972e5a4891c31c01 3228 diffoscope_85.dsc
 a781bde07e4705a0a3e1b2eb675905bd9a74f911 653004 diffoscope_85.tar.xz
 98181b38cc107175c30e083a31d6ab6441bbde06 18385 diffoscope_85_amd64.buildinfo
Checksums-Sha256:
 3af990a704ba3a33ce5e3cc8ec477c1e940fa6068e8523f9c337c87206c32588 3228 diffoscope_85.dsc
 2a71703db5199a6de56e21f8d1c50eea668c147e05ee06fdeda1bfd1bba27016 653004 diffoscope_85.tar.xz
 7cf18e23618d2fe4efbd4995ff4803f0af51a4e45ad0051fe5381c3f1b221b28 18385 diffoscope_85_amd64.buildinfo
Files:
 8c6156508c5386d5f6d03621212d23dd 3228 devel optional diffoscope_85.dsc
 0a6ae3525f191bff4b30b32cfc5e9b7f 653004 devel optional diffoscope_85.tar.xz
 e31ff3dcfd2b84502b318a43c55d263f 18385 devel optional diffoscope_85_amd64.buildinfo

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEi3hoeGwz5cZMTQpICBa54Yx2K60FAll/lV8ACgkQCBa54Yx2
K63H/w/+JwR4jI33AcYD1jxsgRho34BQ20t/F/0Eh2Y05q8sK0nh/8YxyvsmxhS4
/80ZJkNEm0bFqx378bLe18vC+wYoOTZDb4rPTQOWoxJ8NU6l4fASoXplgwYVCYo8
KDHFqWjbUH5eGYZtiT6KvIp7jHkvi5gjc29kyywZv7EC75kxqddCkQ8lzeHg5zls
XrwyOpKTRMRPmxP9uZS8Z6HiYVCmYjCuCN5uvnePldtxx+Ogde4E3z8SYZWgvnRY
SGyj2LnKsxvi5KIpCA6jIDwiDTiv99cdYg6G/F8nGcCBo2iHK96iyoGFmbg2t3ps
9Ru3EfFo7uou+N1G+nHUoDXoPxeB/1Y0ifrLCzrou3JFKPBFSxjiaXyhv0FAqxUF
QCsvzmo4pl9RRYJ7VSfb++s4ttMwBn8Hp+LI8mhAEOkeIxLnEGUrEsF02eF6ewC5
c2v0v5OoXW2nsCfSxEVGsYZ7IKMFwH5kia3zLUvY+A/n42h+R7MiBQ3aMucs5QoM
z9zxIY2LPRGP5D0VoEJ/tY45uA3Nz0hJDaKO1OTC/LyZ0AP5Qs+twVHMtqgcNmwS
ijnV+DlNUI1/TLP0sfYPK1U8J4JLbxgsSv2yjiK86H7QWaXWezsDAlY/krNyU4p3
ObyZx9AAxZOiQyALNbjzcc/x7LzczVYCRkDybTtMBclSulyVubA=
=pkPO
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.