Debian Bug report logs - #908072
diffoscope: `bin/diffoscope --list-debian-substvars` output depends on installed packages

Toggle useless messages

Message #3 received at submit@bugs.debian.org (full text, mbox, reply):

From: Mattia Rizzolo <mattia@debian.org>

To: submit@bugs.debian.org

Subject: diffoscope: `bin/diffoscope --list-debian-substvars` output depends on installed packages

Date: Wed, 5 Sep 2018 22:46:20 +0200

[Message part 1 (text/plain, inline)]

Package: diffoscope
Version: 99
Severity: important

`bin/diffoscope --list-debian-substvars` depends on installed packages.

In particular, it seems "gnupg" and "rpm2cpio" are not printed if they
are not installed.
This makes d/rules generate differing d/tests/control whether the
package is built with nocheck or not.

This accidentally caused the FTBFS of version 100 (about to be
reported).

-- 
regards,
                        Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540      .''`.
more about me:  https://mapreri.org                             : :'  :
Launchpad user: https://launchpad.net/~mapreri                  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-

[signature.asc (application/pgp-signature, inline)]

Message #8 received at submit@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>

To: Mattia Rizzolo <mattia@debian.org>, 908072@bugs.debian.org, submit@bugs.debian.org

Subject: Re: Bug#908072: diffoscope: `bin/diffoscope --list-debian-substvars` output depends on installed packages

Date: Thu, 06 Sep 2018 11:57:37 +0100

Hi Mattia,

> In particular, it seems "gnupg" and "rpm2cpio" are not printed if they
> are not installed.

This is due to:

 * The @tool_required decorator adds all tool names that were passed to
   it to a global variable at module import time (NB. not execution
   time; decorators are executed at import time)

 * Our substvar generation then loops over this global variable.

 * Therefore if a module is is never fully imported then the  
   @tool_required will not be encountered.

The following is from diffoscope/comparators/rpm.py. As the "import
rpm" will fail if python3-rpm is not installed, the @tool_required will
not be imported.

  […]

  import rpm

  […]

  class RpmContainer(Archive):
      @tool_required('rpm2cpio')
      def extract(self, member_name, dest_dir):
          […]

Ideas? Hard-code the additions of gnupg or rpm2cpio in
ListDebianSubstvarsAction? Perhaps it would be cleaner to add some
explicit call in the RPM comparator, for example:

    - import rpm
    + try:
    +    import rpm
    + except ImportError:
    +    tool_required.all.add('rpm2cpio') # lol hack
    + raise

… but this is not terribly nice in itself.


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

Message #18 received at 908072@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>

To: 908072@bugs.debian.org

Subject: Re: diffoscope: `bin/diffoscope --list-debian-substvars` output depends on installed packages

Date: Mon, 10 Sep 2018 19:09:04 +0100

tags 908072 + pending
thanks

Fixed in Git, pending upload:

  https://salsa.debian.org/reproducible-builds/diffoscope/commit/889e4bf7394d465ae2c9a17a0661bb401c30f54a

  diffoscope/main.py | 15 ++++++++++++---
  1 file changed, 12 insertions(+), 3 deletions(-)

I didn't go with the:

    - import rpm
    + try:
    +    import rpm
    + except ImportError:
    +    tool_required.all.add('rpm2cpio') # lol hack
    + raise

… solution as, for example, we were not importing changes.py (and thus
calling @tool_required('gpg')) but this was only due to not importing
the enhanced Debian comparator in debian.py so this would have been
somewhat too distant of an action.

This solution also allows for a big stonkin' comment regarding this
ugliness.


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

Message #25 received at 908072-close@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>

To: 908072-close@bugs.debian.org

Subject: Bug#908072: fixed in diffoscope 101

Date: Sat, 15 Sep 2018 15:49:07 +0000

Source: diffoscope
Source-Version: 101

We believe that the bug you reported is fixed in the latest version of
diffoscope, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 908072@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Chris Lamb <lamby@debian.org> (supplier of updated diffoscope package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Sat, 15 Sep 2018 16:37:45 +0100
Source: diffoscope
Binary: diffoscope
Built-For-Profiles: nocheck
Architecture: source all
Version: 101
Distribution: unstable
Urgency: medium
Maintainer: Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>
Changed-By: Chris Lamb <lamby@debian.org>
Description:
 diffoscope - in-depth comparison of files, archives, and directories
Closes: 903444 908072 908074
Changes:
 diffoscope (101) unstable; urgency=medium
 .
   * Fix testsuite under LLVM >= 7.0. (Closes: #908074)
   * Substvar generation:
     - Ensure Debian substvar generation is deterministic regardless of
       installed packages. (Closes: #908072)
     - Clarify distinction between tools (eg. gpg) and packages (eg. gnupg) when
       generating Debian substvars.
     - Revert updating generated debian/tests/control made in diffoscope 100.
   * Disable binwalk's own user configuration for predictable results and to
     ensure it does not create unnecessary directories. (Closes: #903444)
   * Ensure we return "bytes" objects from Command.filter to avoid a
     "TypeError: Unicode-objects must be encoded before hashing" traceback.
   * Don't print GPG output in diffoscope.changes.
Checksums-Sha1:
 707d3f5ad91d129b32531cd573218bceaab2a08a 4067 diffoscope_101.dsc
 0ff3833cdf03a3aa196acc4b92f730159012cad4 9251288 diffoscope_101.tar.xz
 e8c1b6899b1143c6fe82afc8db48f9ce7f404ab1 123352 diffoscope_101_all.deb
 a9f838e848394a50a32f7a24ab9789649fec50cc 6716 diffoscope_101_amd64.buildinfo
Checksums-Sha256:
 14aa454ce299cff4934e243f799574fc7bde40d271891bec7edfd73482f51350 4067 diffoscope_101.dsc
 ace9e5f99e26e3f7fdee93d66384eacb46c37f41036d591475287f4060ccb4d4 9251288 diffoscope_101.tar.xz
 a8e8c788e7a65db91c2d071d5576f58ff4ff76c24ed22ebfbff7546df6a0d31b 123352 diffoscope_101_all.deb
 4b3448eb2457ba4c1ea90229db2cb585b29f1ba399412dc886a7ee70a71f1996 6716 diffoscope_101_amd64.buildinfo
Files:
 e557e849d1d057420f5c04e424b61cbb 4067 devel optional diffoscope_101.dsc
 48e1b01bd839b36be11665729e8544e0 9251288 devel optional diffoscope_101.tar.xz
 b9a14fa1ae24999ad933a7e54247c4d4 123352 devel optional diffoscope_101_all.deb
 76ddcb1491f6c21ca8c80d8f71541771 6716 devel optional diffoscope_101_amd64.buildinfo

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEwv5L0nHBObhsUz5GHpU+J9QxHlgFAludKKwACgkQHpU+J9Qx
HlgTpA/9H3SJ45WzI3RO/wGJpIf3ElJzMypibsPPnIqxEeZM/8dl2RI5at+x//50
o4rzeEnNivgZ5CJdYRQjcXo9z+xZI2s+ovVfw4ODc865AI5YD6xe691tduMOI0FR
7O6QmbsHlrt5uSxmPt7kOsC4va9P7cCPfz/6+szctm5hSkPDYLSS6pvauGYyablg
a9N/ltTPZ80pWzyg0/meAGiqRkH+EPC/UJqxpJRXhK7bqlAPou49eCkx7fW0f8BZ
/PPAg19EKdh0JspDnhbd86bmGy0gKwM++LUl0ZVxajjSEY5KYw2RWSS9i8MsqmyZ
u5Bl6lqWbcVj+aDuJZtLVXBJLSnjkB+WkKvN8Vf28Saj3hmwCHs/tV3T3Y7Kq1Cl
a3z6bNhvAg+0aZLspr6i6/peeAynEzF3ez1999Dv27XYNQVoZm+HuSsszLPRF9RR
oUADuhM0MHvx2vdk/BplUUrf/re1jmXall+/47cVwqhjng1R80ftt6XZKG4uYyza
J1hOaPJBMpA1p48DLzLF9UC05meswWpWvNyLs9YpqRleIXkKoftF2Fp05rTG7ZH5
OCwFSIvsHi9OgY+44mMsf5iw+TwoRrzAeU+R9HF/1p5ab7W24DDAuSjPofUrQoR1
jdbRNO49B28qqW5/pB/h+QxMe/B3W9P24KglfGrVyPX2hdgM2tQ=
=UgNh
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.