Debian Bug report logs - #1032056
dtrace -G generates unreproducible object files due to random temp file

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Gioele Barabucci <gioele@svario.it>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: dtrace -G generates unreproducible object files due to random temp file

Date: Mon, 27 Feb 2023 08:45:24 +0100

Package: systemtap-sdt-dev
Version: 4.8-1
User: reproducible-builds@lists.alioth.debian.org
Usertags: randomness filesystem
Control: block 1032055 by -1

`dtrace -G` creates temporary files with random filenames. The name of 
these temporary files gets embedded in the ELF `.symtab` of the final 
object files, making them always slightly different.

This behavior makes all packages that use `dtrace`-produced object files 
inherently non reproducible.

Instead of being fully random, `dtrace` should employ some sort of 
controlled reproducible randomness, for example based on the value of 
`SOURCE_DATE_EPOCH`. See also 
<https://reproducible-builds.org/docs/randomness/>.

To reproduce this issue:

```
$ git clone https://salsa.debian.org/sssd-team/sssd.git
$ cd sssd
$ mkdir -p build && cd build/

$ dtrace -C -G -s ../src/systemtap/sssd_probes.d -o stap_generated_probes.o
$ readelf --wide --symbols stap_generated_probes.o > sym1.txt

$ dtrace -C -G -s ../src/systemtap/sssd_probes.d -o stap_generated_probes.o
$ readelf --wide --symbols stap_generated_probes.o > sym2.txt

$ diff -u sym1.txt sym2.txt
--- sym1.txt    2023-02-27 08:38:48.955299234 +0100
+++ sym2.txt    2023-02-27 08:38:49.103303352 +0100
@@ -2,7 +2,7 @@
 Symbol table '.symtab' contains 59 entries:
 Num:    Value  Size Type    Bind   Vis      Ndx Name
   0: 0000000000   0 NOTYPE  LOCAL  DEFAULT  UND
-  1: 0000000000   0 FILE    LOCAL  DEFAULT  ABS .dtrace-temp.4f0bbdda.c
+  1: 0000000000   0 FILE    LOCAL  DEFAULT  ABS .dtrace-temp.d20e76c7.c
   2: 0000000000   0 SECTION LOCAL  DEFAULT    1 .text
   3: 0000000000   7 FUNC    LOCAL  DEFAULT    1 __dtrace
   4: 0000000000   0 SECTION LOCAL  DEFAULT    5 .debug_info
```

Regards,

-- 
Gioele Barabucci

Message #12 received at 1032056@bugs.debian.org (full text, mbox, reply):

From: Gioele Barabucci <gioele@svario.it>

To: 1032056@bugs.debian.org

Subject: Re: Bug#1032056: dtrace -G generates unreproducible object files due to random temp file

Date: Mon, 27 Feb 2023 09:16:17 +0100

On Mon, 27 Feb 2023 08:45:24 +0100 Gioele Barabucci <gioele@svario.it> 
wrote:
> `dtrace -G` creates temporary files with random filenames. The name of 
> these temporary files gets embedded in the ELF `.symtab` of the final 
> object files, making them always slightly different.

The root cause of this issue is that, although the name of the temporary 
file is created in a deterministic way (from the SHA256 of the source 
file), the name of the source file is overwritten with a random name 
then the `-C` option (`use_cpp`) is used:

```
if s_filename != "" and use_cpp:
    (ignore, fname) = mkstemp(suffix=".d")
    cpp = os.environ.get("CPP", "cpp")
    retcode = call(split(cpp) + [...] + [s_filename, '-o', fname])
    if retcode != 0:
        print("\"cpp includes s_filename\" failed")
        usage()
        return 1
    s_filename = fname

[...]

# for reproducible-builds purposes, use a predictable tmpfile path
sha = hashlib.sha256()
sha.update(s_filename.encode('utf-8'))
sha.update(filename.encode('utf-8'))
fname = ".dtrace-temp." + sha.hexdigest()[:8] + ".c"
```

Could you please replace the use of `mkstemp` with something similar to 
what is used to determine the final temporary filename?

-- 
Gioele Barabucci

Message #21 received at 1032056@bugs.debian.org (full text, mbox, reply):

From: Gioele Barabucci <gioele@svario.it>

To: 1032056@bugs.debian.org

Subject: Re: Bug#1032056: dtrace -G generates unreproducible object files due to random temp file

Date: Mon, 27 Feb 2023 16:02:56 +0100

Dear systemtap maintainers,

now that this bug has been fixed upstream [1], could it cherry-picked 
for bookworm?

[1] sourceware.org/git/?p=systemtap.git;a=commit;h=05c8fa6a54

Regards,

-- 
Gioele Barabucci

Message #28 received at 1032056-close@bugs.debian.org (full text, mbox, reply):

From: Debian FTP Masters <ftpmaster@ftp-master.debian.org>

To: 1032056-close@bugs.debian.org

Subject: Bug#1032056: fixed in systemtap 4.8-2

Date: Tue, 28 Feb 2023 23:18:49 +0000

Source: systemtap
Source-Version: 4.8-2
Done: Emanuele Rocca <ema@debian.org>

We believe that the bug you reported is fixed in the latest version of
systemtap, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 1032056@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Emanuele Rocca <ema@debian.org> (supplier of updated systemtap package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.8
Date: Tue, 28 Feb 2023 21:45:06 +0100
Source: systemtap
Architecture: source
Version: 4.8-2
Distribution: unstable
Urgency: medium
Maintainer: Emanuele Rocca <ema@debian.org>
Changed-By: Emanuele Rocca <ema@debian.org>
Closes: 1032056
Changes:
 systemtap (4.8-2) unstable; urgency=medium
 .
   * Add d/patches/dtrace-deterministic-tmpfile.diff by Gioele Barabucci to
     ensure dtrace produces reproducible object files. The patch has been merged
     upstream already. Thanks Gioele! (Closes: #1032056)
   * Add autopkgtest d/tests/dtrace-reproducible to verify that the above works
     as advertised.
   * Refresh d/patches.
Checksums-Sha1:
 bfb25b53e0c38b1318e13c5b25a734c98661b90f 3041 systemtap_4.8-2.dsc
 08e71a6173496f4c185ae3483c2ee5f88594cfa4 31340 systemtap_4.8-2.debian.tar.xz
 667defd081ce56722fee21b737c3143ec541263f 12445 systemtap_4.8-2_amd64.buildinfo
Checksums-Sha256:
 bebfe9ad148dc4aa029aacd5ec6843bf8827ec7af423bc0cd63827bbc84f0c8d 3041 systemtap_4.8-2.dsc
 d8e8c12d86fb6e51afca990d92e878e10d42c5937e90e58c95e417ff19e36556 31340 systemtap_4.8-2.debian.tar.xz
 ffcfd1e0df109b97ebefcf285f3f511acdcb74b22207b04923d4579a7fbeb5eb 12445 systemtap_4.8-2_amd64.buildinfo
Files:
 c367307d250edabaf3ec9cd0b3393a4b 3041 devel optional systemtap_4.8-2.dsc
 19a00dcfb117d53579d01a08b346c73b 31340 devel optional systemtap_4.8-2.debian.tar.xz
 633d88ba1a2cd457d908da6de318443a 12445 devel optional systemtap_4.8-2_amd64.buildinfo

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEElUWWlhkoHBf/rFiR1QhaB1D9t6MFAmP+gZoACgkQ1QhaB1D9
t6Mtbw/+Pnlpu0fpyO5bzoTw5aGAw8SWTLkDicARbHLHMQitNPSzqFIiT//p1w0E
ViLxCIUcuEMaf9MjQzuQt28wUa2Ib/pI0qfiCBJA3h+GY2GDKXZVO+0sQOQtmnSs
I1W76nUsUMR+fIIgVVQZFMcfMrW1lmoAsr03DRLhVGLbzegTsBT1qTiQih8CoieV
IWJyu3EoEP/uGMxEStCFvfvGSL3LioUX0O7H1Bk1G91VNz3FCnG978ypXgGIxWbs
Yc9nCALhHEDkqaVG7bgizGoEVqcP/wlnJrx0izWkeTCoQGNyCu5sp4EaMispZQbK
V3lrR2w7YM3ZjzpxjJUE3nnjKNPdYgF4gotytBEbkq48XCm9+bjPD+M7gJK8aLFu
Y4vzQNAFWxXRDhiNBJaHgFCmrseWsiXX/bkcTUgfr7VitgH82E0eFvZwrwXNmaul
02SI+01BRtxt3VrU9/AOA8eak9etF6kx2RSCJiQnJLN4H8lhReCV8EqHrfEvv2+Q
aqMY6LvIcS7QPyoPp5NHn6Pr+qnbuJSatUtfUOpKtgnO093C7ogoWWSp1IDW5ydb
ShxTDUTsK07WYrkO+AQlblyQNwqkucCpNj7J51gLSRSIkkyNJJ0iVOOwBn0R/Wtp
Fb+tdxnMWS802kZnhyPcAAvp9amLW4+du+yBV5dB85hlHULk6lg=
=Pwyk
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.