Debian Bug report logs - #931962
diffoscope: Falls back to raw hexdump with "NOBITS eh_frame" in ELF binaries

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Mike Hommey <mh+reportbug@glandium.org>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: diffoscope: Doesn't behave nicely in the face of NOBITS eh_frame

Date: Sat, 13 Jul 2019 09:50:55 +0900

Package: diffoscope
Version: 117
Severity: important

Dear Maintainer,

When comparing Debian debug packages, diffoscope ends up showing hexdump
diffs, when it could, in fact, output much nicer diffs.

The two attached files are a small .debug file from two
jenkins.debian.net firefox debug package builds, that differ because of
missing -fdebug-prefix-map flags.

When comparing them, the output looks like:
--- /tmp/41aa9e49143d7d5d4fb5e05623976b644573b2.debug
+++ /tmp/b70f29bd619ba23d3879579b19209fda75ea3b.debug
│┄ Command `readelf --wide --debug-dump=frames /tmp/41aa9e49143d7d5d4fb5e05623976b644573b2.debug` exited with 1.
Output:
│┄ <none>
@@ -30,161 +30,161 @@
 000001d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 000001e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 000001f0: 0000 0000 0000 0000 1000 0000 0000 0000  ................
 00000200: 52e5 7464 0400 0000 d00d 0000 0000 0000  R.td............
 00000210: d03d 0000 0000 0000 d03d 0000 0000 0000  .=.......=......
 00000220: 0000 0000 0000 0000 3002 0000 0000 0000  ........0.......
 00000230: 0100 0000 0000 0000 0400 0000 1400 0000  ................
-00000240: 0300 0000 474e 5500 3341 aa9e 4914 3d7d  ....GNU.3A..I.=}
-00000250: 5d4f b5e0 5623 976b 6445 73b2 4743 433a  ]O..V#.kdEs.GCC:
+00000240: 0300 0000 474e 5500 a8b7 0f29 bd61 9ba2  ....GNU....).a..
+00000250: 3d38 7957 9b19 209f da75 ea3b 4743 433a  =8yW.. ..u.;GCC:
 00000260: 2028 4465 6269 616e 2038 2e33 2e30 2d31   (Debian 8.3.0-1
 00000270: 3929 2038 2e33 2e30 0001 0000 0000 0000  9) 8.3.0........
 00000280: 00f0 0000 0000 0000 0001 0000 0000 0000  ................
 00000290: 0078 9c7b c3c0 c0c0 c400 021c 6032 4000  .x.{........`2@.
 000002a0: 4c31 0842 2886 4434 7e11 1abf 198d 3f05  L1.B(.D4~.....?.
etc.

First, there's the problem that diffoscope only displays readelf's
stdout, and its stderr is lost, which it contains the most important
information:
  section '.eh_frame' has the NOBITS type - its contents are unreliable.

Second, this skips everything else that diffoscope would normally do,
instead of falling back to a hexdump diff of the concerned section only.

With a readelf wrapper that always exits 0, this is what the output
looks like:
--- /tmp/41aa9e49143d7d5d4fb5e05623976b644573b2.debug
+++ /tmp/b70f29bd619ba23d3879579b19209fda75ea3b.debug
├ readelf --wide --sections {}
│ @@ -23,19 +23,19 @@
│    [18] .dynamic          NOBITS          0000000000003de0 000dd0 000200 10  WA  4   0  8
│    [19] .got              NOBITS          0000000000003fe0 000dd0 000020 08  WA  0   0  8
│    [20] .got.plt          NOBITS          0000000000004000 000dd0 000020 08  WA  0   0  8
│    [21] .data             NOBITS          0000000000004020 000dd0 000008 00  WA  0   0  8
│    [22] .bss              NOBITS          0000000000004028 000dd0 000008 00  WA  0   0  1
│    [23] .comment          PROGBITS        0000000000000000 00025c 00001d 01  MS  0   0  1
│    [24] .debug_aranges    PROGBITS        0000000000000000 000279 00005a 00   C  0   0  1
│ -  [25] .debug_info       PROGBITS        0000000000000000 0002d3 000330 00   C  0   0  1
│ -  [26] .debug_abbrev     PROGBITS        0000000000000000 000603 0000da 00   C  0   0  1
│ -  [27] .debug_line       PROGBITS        0000000000000000 0006dd 000176 00   C  0   0  1
│ -  [28] .debug_str        PROGBITS        0000000000000000 000853 0002c3 01 MSC  0   0  1
│ -  [29] .debug_ranges     PROGBITS        0000000000000000 000b16 00004e 00   C  0   0  1
│ +  [25] .debug_info       PROGBITS        0000000000000000 0002d3 00032e 00   C  0   0  1
│ +  [26] .debug_abbrev     PROGBITS        0000000000000000 000601 0000da 00   C  0   0  1
│ +  [27] .debug_line       PROGBITS        0000000000000000 0006db 000176 00   C  0   0  1
│ +  [28] .debug_str        PROGBITS        0000000000000000 000851 0002c4 01 MSC  0   0  1
│ +  [29] .debug_ranges     PROGBITS        0000000000000000 000b15 00004e 00   C  0   0  1
│    [30] .symtab           SYMTAB          0000000000000000 000b68 000690 18     31  49  8
│    [31] .strtab           STRTAB          0000000000000000 0011f8 0002ca 00      0   0  1
│    [32] .shstrtab         STRTAB          0000000000000000 0014c2 000137 00      0   0  1
│  Key to Flags:
│    W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
│    L (link order), O (extra OS processing required), G (group), T (TLS),
│    C (compressed), x (unknown), o (OS specific), E (exclude),
├ readelf --wide --notes {}
│ @@ -1,4 +1,4 @@
│
│  Displaying notes found in: .note.gnu.build-id
│    Owner                 Data size    Description
│ -  GNU                  0x00000014    NT_GNU_BUILD_ID (unique build ID bitstring)         Build ID: 3341aa9e9143d7d5d4fb5e05623976b644573b2
│ +  GNU                  0x00000014    NT_GNU_BUILD_ID (unique build ID bitstring)         Build ID: a8b70f29d619ba23d3879579b19209fda75ea3b
├ readelf --wide --debug-dump=rawline {}
│ @@ -21,19 +21,19 @@
│    Opcode 8 has 0 args
│    Opcode 9 has 1 arg
│    Opcode 10 has 0 args
│    Opcode 11 has 0 args
│    Opcode 12 has 1 arg
│
│   The Directory Table (offset 0x1b):
│ -  1  /build/1st/firefox-68.0/widget/gtk/mozgtk
│ +  1  /build/firefox-68.0/2nd/widget/gtk/mozgtk
│    2  /usr/lib/gcc/x86_64-linux-gnu/8/include
│    3  /usr/include/x86_64-linux-gnu/bits

etc.

Chances are this is involved in the firefox diffs timing out
https://tests.reproducible-builds.org/debian/dbd/unstable/amd64/firefox_68.0-1.diffoscope.html

Mike

Message #10 received at 931962@bugs.debian.org (full text, mbox, reply):

From: "Chris Lamb" <lamby@debian.org>

To: 931962@bugs.debian.org

Cc: "Mike Hommey" <mh+reportbug@glandium.org>

Subject: diffoscope: Falls back to raw hexdump with "NOBITS eh_frame" in ELF binaries (was: "Re: diffoscope: Doesn't behave nicely in the face of NOBITS eh_frame")

Date: Fri, 12 Jul 2019 22:05:24 -0300

clone 931962 -1 
reassign -1 diffoscope
retitle -1 diffoscope: Please include stderr from readelf in output
severity -1 normal
retitle 931962 diffoscope: Falls back to raw hexdump with "NOBITS eh_frame" in ELF binaries
thanks

Hi Mike,

Thank you for filing this. Just as a bit of bug triage, I'm going to
split this in two:

> First, there's the problem that diffoscope only displays readelf's
> stdout, and its stderr is lost, which it contains the most important
> information:
>   section '.eh_frame' has the NOBITS type - its contents are unreliable.

(This will be bug -1, keeping the rest as #931962.)


Best wishes,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

Message #19 received at 931962@bugs.debian.org (full text, mbox, reply):

From: "Chris Lamb" <chris@chris-lamb.co.uk>

To: 931962@bugs.debian.org

Cc: "Mike Hommey" <mh+reportbug@glandium.org>

Subject: Re: diffoscope: Falls back to raw hexdump with "NOBITS eh_frame" in ELF binaries

Date: Fri, 12 Jul 2019 22:14:35 -0300

forwarded 931962 https://salsa.debian.org/reproducible-builds/diffoscope/issues/58
thanks

I've "forwarded this upstream" here:

  https://salsa.debian.org/reproducible-builds/diffoscope/issues/58


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

Message #26 received at 931962@bugs.debian.org (full text, mbox, reply):

From: Mike Hommey <mh@glandium.org>

To: 931962@bugs.debian.org

Subject: Re: Bug#931962: diffoscope: Doesn't behave nicely in the face of NOBITS eh_frame

Date: Sat, 13 Jul 2019 10:12:35 +0900

[Message part 1 (text/plain, inline)]

On Sat, Jul 13, 2019 at 09:50:55AM +0900, Mike Hommey wrote:
> Package: diffoscope
> Version: 117
> Severity: important
> 
> Dear Maintainer,
> 
> When comparing Debian debug packages, diffoscope ends up showing hexdump
> diffs, when it could, in fact, output much nicer diffs.
> 
> The two attached files are a small .debug file from two
> jenkins.debian.net firefox debug package builds, that differ because of
> missing -fdebug-prefix-map flags.

As usual when someone says "attached files"... I forgot to attach the
files...

Here they are.

Mike

[41aa9e49143d7d5d4fb5e05623976b644573b2.debug (application/octet-stream, attachment)]

[b70f29bd619ba23d3879579b19209fda75ea3b.debug (application/octet-stream, attachment)]

Message #31 received at 931962@bugs.debian.org (full text, mbox, reply):

From: "Chris Lamb" <lamby@debian.org>

To: "Mike Hommey" <mh+reportbug@glandium.org>

Cc: 931962@bugs.debian.org, 931963@bugs.debian.org

Subject: Re: Bug#931962: diffoscope: Doesn't behave nicely in the face of NOBITS eh_frame

Date: Fri, 12 Jul 2019 22:27:20 -0300

Hi Mike,

> The two attached files are […]

Did you, perhance, forget to attach these? :)


Best wishes,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org 🍥 chris-lamb.co.uk
       `-

Message #36 received at 931962-close@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>

To: 931962-close@bugs.debian.org

Subject: Bug#931962: fixed in diffoscope 121

Date: Fri, 16 Aug 2019 17:19:37 +0000

Source: diffoscope
Source-Version: 121

We believe that the bug you reported is fixed in the latest version of
diffoscope, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 931962@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Chris Lamb <lamby@debian.org> (supplier of updated diffoscope package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Fri, 16 Aug 2019 09:49:07 -0700
Source: diffoscope
Architecture: source
Version: 121
Distribution: unstable
Urgency: medium
Maintainer: Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>
Changed-By: Chris Lamb <lamby@debian.org>
Closes: 849407 931962
Changes:
 diffoscope (121) unstable; urgency=medium
 .
   * Don't fallback to a (useless) raw hexdump when readelf(1) reports an minor
     issue in a section in an ELF binary. For example, when the "frames" section
     is of the "NOBITS" type, its contents are apparently "unreliable" and thus
     readelf(1) exits with a returncode of 1. (Closes: #849407, #931962)
   * Add support to Difference.from_command_exc and friends to optionally ignore
     specified returncodes from the called program and treat them as "no"
     difference.
   * Simplify the parsing of the optional "command_args" argument to the
     from_command and from_command_exc methods in the Difference class.
Checksums-Sha1:
 13e8b951ab92da18f103c6b99e6566fc044add8a 4660 diffoscope_121.dsc
 c7a7a80b45a534e52ecca06517a6dd73a56e5c25 1137456 diffoscope_121.tar.xz
 1b5836db849e50e5d8000263554cca4868f32bd6 26423 diffoscope_121_amd64.buildinfo
Checksums-Sha256:
 3162009ac9a9a457fa4a1cb766a9266090c32a6bc6aff372b1faabb3836e5a18 4660 diffoscope_121.dsc
 02fe7bb965d723a07184ad2ab85eaa24b34bc6c6eaf5664bf1f8a3475db23db4 1137456 diffoscope_121.tar.xz
 9a8f4c811c3c1d0962b4517bd5e109721433f26290b00f4f5ed84a6ddf8d5d4f 26423 diffoscope_121_amd64.buildinfo
Files:
 31c3cfee4c8010c853a96acd034e3b34 4660 devel optional diffoscope_121.dsc
 ca9432d5c97742a8b0943a243094271a 1137456 devel optional diffoscope_121.tar.xz
 f4dcbac68a51f270b89c6c2f161b1f9d 26423 devel optional diffoscope_121_amd64.buildinfo

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEwv5L0nHBObhsUz5GHpU+J9QxHlgFAl1W4g4ACgkQHpU+J9Qx
Hlh/DA/+KeOdAaRleEuDbf4psFseJAdWNV6v9Gi8mLGkxGkSHgciBmWK8810kn1K
AzxJJpnTyQBhmqvfS5ih9xegV4m1ptlAT8VK8yrHsrhbDTGlFPi8OqjACeN/5+++
d0XMuyMSTfOyTnblRfdGr+etml2uvmFKYlPIL46Yj8szekpdqdgNtxbyL6uNPh1l
QUI8cUKVLf0ucUlF0JGZKucup46KE9Sc8cVmj5J0Qc2fOd89tklcW5wwynRRKNI4
BNh3HXS79NUwIULgLQ7jhnnHtn2bJJ6IyFpkRhVWNKE40Bxhkl8jmFVEFJOz9Ku/
uXE8xA8dXvqvLdTsnEYxOoDM47DukRc5BKTsh8hGHe0WMcqbSGMxDDz8rcVozmxe
5ndARD8wWozpdE/nJD0hSpYu3u9xhyBeunkAE5p28s5ciIrhx9VOV7aYtVFCHMM+
SVKAxSxv6VmSvKK7hn8MZRBs1mwmV3ABsK/J/OqHhtB+DOWrNl07g/wQ1jKI7Vgc
z5DKA+6/OVutkQIEEjP0ve7u8lJJ1SVfDR8nZwf8vgDetBPTZIE0a7q6cZtLMCjL
TgZnJO6VAKAYzs4n6tVPH3obCFJ19TBQhDDBgnPf4i76KPMhmkW86Zz847BNv0VH
U5wgQPdoCsF9FUlOk9e8yBW2gYXVmuj2VnfDcVFRanMvmF9cm0s=
=ekzw
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.