Debian Bug report logs - #911446
diffoscope: pdf comparator no longer diffing metadata

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Reiner Herrmann <reiner@reiner-h.de>

To: submit@bugs.debian.org

Subject: diffoscope: pdf comparator no longer diffing metadata

Date: Sat, 20 Oct 2018 11:16:14 +0200

[Message part 1 (text/plain, inline)]

Source: diffoscope
Version: 103

Hi,

I just noticed on some packages on tests.r-b.o with PDF differences that
metadata like /CreationDate and /ID are no longer shown as a plaintext
diff, but as a binary/hex diff instead.
An example is this diff:
 https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/diffoscope-results/bibtool.html

With the version in stretch (78) this is working and I get nice diffs like:

 -/ModDate (D:20180223070913-12'00')
 +/ModDate (D:20180224090913+14'00')
 -/CreationDate (D:20180223070913-12'00')
 +/CreationDate (D:20180224090913+14'00')
 -/ID [<549377c76efe92fafc6a5825704410a3> <549377c76efe92fafc6a5825704410a3>]
 +/ID [<fec1ddadfcc55ebe2c00f662d2b60888> <fec1ddadfcc55ebe2c00f662d2b60888>]

Regards,
  Reiner

[signature.asc (application/pgp-signature, inline)]

Message #10 received at 911446@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>

To: Reiner Herrmann <reiner@reiner-h.de>, 911446@bugs.debian.org

Subject: Re: Bug#911446: diffoscope: pdf comparator no longer diffing metadata

Date: Mon, 22 Oct 2018 22:04:25 -0400

Dear Reiner,

> An example is this diff:
>  
> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/diffoscope-results/bibtool.html

Pasting here for posterity:

│ │ │ ├── ./usr/share/doc/bibtool/bibtool.pdf
│ │ │ │┄ Format-specific differences are supported for this file format but none were detected (PDF document, version 1.5)
│ │ │ │ @@ -27559,18 +27559,18 @@
│ │ │ │  0006ba60: 4269 6254 6f6f 6c20 4d61 6e75 616c 292f  BibTool Manual)/
│ │ │ │  0006ba70: 5375 626a 6563 7428 5665 7273 696f 6e20  Subject(Version 
│ │ │ │  0006ba80: 322e 3637 292f 4372 6561 746f 7228 4c61  2.67)/Creator(La
│ │ │ │  0006ba90: 5465 5820 7769 7468 2068 7970 6572 7265  TeX with hyperre
│ │ │ │  0006baa0: 6629 2f50 726f 6475 6365 7228 4c75 6154  f)/Producer(LuaT
│ │ │ │  0006bab0: 6558 2d31 2e37 2e30 292f 4b65 7977 6f72  eX-1.7.0)/Keywor
│ │ │ │  0006bac0: 6473 2829 0a2f 4372 6561 7469 6f6e 4461  ds()./CreationDa
│ │ │ │ -0006bad0: 7465 2028 443a 3230 3138 3032 3233 3037  te (D:2018022307
│ │ │ │ -0006bae0: 3039 3133 2d31 3227 3030 2729 2f4d 6f64  0913-12'00')/Mod
│ │ │ │ -0006baf0: 4461 7465 2028 443a 3230 3138 3032 3233  Date (D:20180223
│ │ │ │ -0006bb00: 3037 3039 3133 2d31 3227 3030 2729 2f54  070913-12'00')/T
│ │ │ │ +0006bad0: 7465 2028 443a 3230 3138 3032 3234 3039  te (D:2018022409
│ │ │ │ +0006bae0: 3039 3133 2b31 3427 3030 2729 2f4d 6f64  0913+14'00')/Mod
│ │ │ │ +0006baf0: 4461 7465 2028 443a 3230 3138 3032 3234  Date (D:20180224
│ │ │ │ +0006bb00: 3039 3039 3133 2b31 3427 3030 2729 2f54  090913+14'00')/T


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

Message #15 received at 911446@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>

To: Reiner Herrmann <reiner@reiner-h.de>, 911446@bugs.debian.org

Cc: doko@debian.org

Subject: Re: Bug#911446: diffoscope: pdf comparator no longer diffing metadata

Date: Mon, 22 Oct 2018 23:09:11 -0400

tags 911446 + pending
thanks

Hi Reiner,

> I just noticed on some packages on tests.r-b.o with PDF differences that
> metadata like /CreationDate and /ID are no longer shown as a plaintext
> diff, but as a binary/hex diff instead.

This was dropped via #893702 in:

  https://salsa.debian.org/reproducible-builds/diffoscope/commit/5530623e9672c8afa4f186303bba29232b28ccd6

… due to the dependency on GCJ which is due to be be removed from
Debian.

In the aforementioned bug, Vagrant mentions that there is a pdftk
fork that does not require GCJ but packaging and maintaining this
would probably outside the scope of the Reproducible Builds team.

However, doko also mentioned PyPDF (thanks), which I've integrated
and implemented in:

  https://salsa.debian.org/reproducible-builds/diffoscope/commit/4e7ba71d40384c7bddc3365814cb91fd6e551790

  debian/control                        |  1 +
  debian/tests/control                  |  2 +-
  diffoscope/comparators/pdf.py         | 35 ++++++++++++++++++++++++++++++++++-
  setup.py                              |  1 +
  tests/comparators/test_pdf.py         | 13 ++++++++++++-
  tests/data/pdf_metadata_expected_diff | 13 +++++++++++++
  6 files changed, 62 insertions(+), 3 deletions(-)

Whilst I was doing this, I found & filed #911649.


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

Message #22 received at 911446-close@bugs.debian.org (full text, mbox, reply):

From: Mattia Rizzolo <mattia@debian.org>

To: 911446-close@bugs.debian.org

Subject: Bug#911446: fixed in diffoscope 104

Date: Thu, 25 Oct 2018 13:34:10 +0000

Source: diffoscope
Source-Version: 104

We believe that the bug you reported is fixed in the latest version of
diffoscope, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 911446@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Mattia Rizzolo <mattia@debian.org> (supplier of updated diffoscope package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.8
Date: Thu, 25 Oct 2018 14:55:41 +0200
Source: diffoscope
Binary: diffoscope
Architecture: source
Version: 104
Distribution: unstable
Urgency: medium
Maintainer: Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>
Changed-By: Mattia Rizzolo <mattia@debian.org>
Description:
 diffoscope - in-depth comparison of files, archives, and directories
Closes: 910540 910542 911446
Changes:
 diffoscope (104) unstable; urgency=medium
 .
   [ Chris Lamb ]
   * comparators:
     + macho: Prevent a traceback if the filename cannot be found on the line.
       Closes: #910540
     + ocaml New comparator OCaml files using ocamlobjinfo.  Closes: #910542
     + pdf: Add support for comparing metadata using PyPDF2.  Closes: #911446
   * debian/tests/control.in: Add note on how to regenerate this file.
 .
   [ Mattia Rizzolo ]
   * d/control:
     + Build-Depend on linux-image-amd64 [amd64] to allow squashfs tests to run.
     + Add a Build-Conflicts against graphicsmagick-imagemagick-compat.
       See: #910652
   * Move the list of optional python packages from d/rules into setup.py, and
     have dh_python3 check the generated requires.txt.
   * Also add the optional python modules to the autopkgtest dependencies.
   * Temporarily drop Build-Depends and Test-Depends on apktool and
     oggvideotools, as they are not available in buster at the moment.
   * comparators:
     + java:
       - Rename the tool procyon-decompiler to procyon.
       - Properly fall back from procyon to javap also when procyon exists
         but doesn't return any output.
   * Declare in setup.py that diffoscope is good to go with Python 3.7.
 .
   [ Marek Marczykowski-Górecki ]
   * comparators:
     + fsimage: Handle FAT filesystems.  MR: !13
Checksums-Sha1:
 2b0e55b21fcaa4627251ae0545cb259b245f4291 4449 diffoscope_104.dsc
 5d593513a823bfba8929b162ac1803b99dc488cd 9258804 diffoscope_104.tar.xz
 7fc0d0b4d225d76acce50039681c34d6d5885b9c 21083 diffoscope_104_amd64.buildinfo
Checksums-Sha256:
 78c86c0f458645acc9fb3e8a37dc8c7e3b090b575b6cefb814f7a93813510ed4 4449 diffoscope_104.dsc
 226429bb90cb4440fe469018fda56f1d1c41c17ee2d4b1940d8a5350a6d75028 9258804 diffoscope_104.tar.xz
 b395db6cb2718d50e6a434dcdf3c8588933f5e6862047b9415c0fc0cf2bc3f14 21083 diffoscope_104_amd64.buildinfo
Files:
 88aadc711427d627bab448b77a24d307 4449 devel optional diffoscope_104.dsc
 23bfc90fbd201d915b582e4f47faf4d2 9258804 devel optional diffoscope_104.tar.xz
 a91e133604e5c5c1b04681416ee8c997 21083 devel optional diffoscope_104_amd64.buildinfo

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEi3hoeGwz5cZMTQpICBa54Yx2K60FAlvRwHgACgkQCBa54Yx2
K62X2A//akxAjjXtmDJWUVkhA9rB/l+hcDfPMB72N3d0ZYXidY9HTZb1wJV5jmNR
B9ZzjouqR5oiS5lQ+ZRDSROQvwv/nCXfUKK9p/ml058dA6ZOAX6+KlGSqnzzTw6N
uTbASXwsE1thB9I4r/UUfH2nAwUP16ZPzUdiJZ6tgzjA4nmaYEVZCS4Zhs6O76yT
zlXcwvTTVe0wR/k8VdTUA7muNnFqJQu1sEo44eTHzmo3emNjrJStWWdp3Rp0kSAz
xBWObWBH4I6o550snTyxroE+4EdEc6PIfTp8JsAsEYltcH38Ymyzlb1ip6bHUwKR
F+bxSLDmWOeJRAxpx5nEyd1w9kI/9LeN0SR8CwlMTQS8Au/skbqwIuZDFmJ+/VBY
YMJUTOMgNWo7Ak8gHDw9UW+rycM2r6ejV3CPliIHZw+dOfwCDSinD1CZscNgQBBj
/qQQchswzqPkcqSykkDZdluAJmw+9KnVSLNuOoSwWTtG7sgJrQBQJWlMduMaM8Jx
FaLC3Mb2NRbyAR4/03ZxMhwhTcFnVYTgrljnEYgytCqvaLW46Gf8GquZinCQLmai
1v2k3LYc91cjq07yFhf5P8Is10G+oP3ALIVa4NMAGcicXzzctmiXsFZ635rRreOI
1QbgUCqfqlzhbdyw64GgNdRceXNfz6czkoKibtYZ9PwY/Gd04BU=
=6lIn
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.