Debian Bug report logs - #974959
dynare: reproducible builds: autotools generated files contain variable paths and data

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Vagrant Cascadian <vagrant@reproducible-builds.org>

To: submit@bugs.debian.org

Subject: dynare: reproducible builds: autotools generated files contain variable paths and data

Date: Mon, 16 Nov 2020 23:20:10 -0800

[Message part 1 (text/plain, inline)]

Source: dynare
Severity: normal
Tags: patch
User: reproducible-builds@lists.alioth.debian.org
Usertags: usrmerge buildpath randomness
X-Debbugs-Cc: reproducible-bugs@lists.alioth.debian.org

In files shipped in dynare, build paths, binary paths and data in
arbitrary order are embedded in a shipped Makefile and data
autom4te.cache/requests file:

  https://tests.reproducible-builds.org/debian/rb-pkg/bullseye/amd64/diffoscope-results/dynare.html

/usr/src/matlab/dynare-matlab/mex/sources/Makefile contains build paths
(e.g. /build/1st/dynare-4.6.2/ vs. /build/2/dynare-4.6.2/2nd/) and when
built on a usrmerge system vs. a non-usermerge system, binary paths that
vary (e.g. /bin/grep vs. /usr/bin/grep).

/usr/src/matlab/dynare-matlab/mex/build/matlab/autom4te.cache/requests
contains what appears to be the same information but in an undefined
order varies between builds.

I *think* these files would need to be regenerated in order to be used
by the end-user, as the build paths or binary paths embedded in them may
not match the system on which they would be used. If that is correct,
the attached patch removes these files using a debian/rules dh_install
override.


live well,
  vagrant

[0002-debian-rules-Remove-Makefile-and-autom4te.cache-requ.patch (text/x-diff, inline)]

From 6eb2e4c4bd27441a0039eec4704f526de033045d Mon Sep 17 00:00:00 2001
From: Vagrant Cascadian <vagrant@reproducible-builds.org>
Date: Tue, 17 Nov 2020 03:03:57 +0000
Subject: [PATCH 2/2] debian/rules: Remove Makefile and autom4te.cache/requests
 in dh_install override.

These files contain build-specific information and need to be
regenerated when used anyways.
---
 debian/rules | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/debian/rules b/debian/rules
index c77220d..5d0839e 100755
--- a/debian/rules
+++ b/debian/rules
@@ -31,6 +31,13 @@ override_dh_auto_test:
 execute_before_dh_auto_clean:
 	-[ -f Makefile ] && echo -e "distclean:\n\t-rm -f Makefile" > mex/build/matlab/Makefile
 
+override_dh_install:
+	dh_install
+	# Remove auto-generated files which cause reproducibility
+	# issues and need to be regenerated to use.
+	rm -f debian/dynare-matlab/usr/src/matlab/dynare-matlab/mex/sources/Makefile
+	rm -f debian/dynare-matlab/usr/src/matlab/dynare-matlab/mex/build/matlab/autom4te.cache/requests 
+
 execute_after_dh_installdocs-indep:
 	cp -dR doc/manual/build/html/ debian/dynare-doc/usr/share/doc/dynare/dynare.html
 
-- 
2.29.2

[signature.asc (application/pgp-signature, inline)]

Message #8 received at 974959-submitter@bugs.debian.org (full text, mbox, reply):

From: Sébastien Villemot <sebastien@debian.org>

To: 974959-submitter@bugs.debian.org

Subject: Bug#974959 marked as pending in dynare

Date: Wed, 25 Nov 2020 09:28:52 +0000

Control: tag -1 pending

Hello,

Bug #974959 in dynare reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:

https://salsa.debian.org/pkg-octave-team/dynare/-/commit/acb40406c6a856608535e2d35bc4330c58d5c629

------------------------------------------------------------------------
Remove auto-generated files which cause reproducibility issues

Closes: #974959
Thanks: Vagrant Cascadian
------------------------------------------------------------------------

(this message was generated automatically)
-- 
Greetings

https://bugs.debian.org/974959

Message #15 received at 974959-close@bugs.debian.org (full text, mbox, reply):

From: Debian FTP Masters <ftpmaster@ftp-master.debian.org>

To: 974959-close@bugs.debian.org

Subject: Bug#974959: fixed in dynare 4.6.3-1

Date: Wed, 25 Nov 2020 09:48:25 +0000

Source: dynare
Source-Version: 4.6.3-1
Done: =?utf-8?q?S=C3=A9bastien_Villemot?= <sebastien@debian.org>

We believe that the bug you reported is fixed in the latest version of
dynare, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 974959@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Sébastien Villemot <sebastien@debian.org> (supplier of updated dynare package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.8
Date: Wed, 25 Nov 2020 09:44:32 +0100
Source: dynare
Architecture: source
Version: 4.6.3-1
Distribution: unstable
Urgency: medium
Maintainer: Debian Octave Group <team+pkg-octave-team@tracker.debian.org>
Changed-By: Sébastien Villemot <sebastien@debian.org>
Closes: 974957 974959
Changes:
 dynare (4.6.3-1) unstable; urgency=medium
 .
   * New upstream version 4.6.3
   * Bump S-V to 4.5.1
   * Make generated PDFs reproducible.
     Thanks to Vagrant Cascadian (Closes: #974957)
   * Remove auto-generated files which cause reproducibility issues.
     Thanks to Vagrant Cascadian (Closes: #974959)
Checksums-Sha1:
 a5dd1dd492631de48d9cae506bf9c0367e64e078 2680 dynare_4.6.3-1.dsc
 cfec58dc2d0e21fd549b8b405bda3fc1e9a84535 17286312 dynare_4.6.3.orig.tar.xz
 b8811126ea972784a5db6947843474c7add2cc5f 2292 dynare_4.6.3.orig.tar.xz.asc
 9d38ec57fa5385a413b701407d1e4489d8fa1216 57364 dynare_4.6.3-1.debian.tar.xz
 a77aefc65db49711da5dd1f1fa7d7aa13e9843ed 23966 dynare_4.6.3-1_amd64.buildinfo
Checksums-Sha256:
 e686fe94e6df23e6a588118c516e0e4d087f5b5f97d491bf1e9a6c065474bb8a 2680 dynare_4.6.3-1.dsc
 1e346fc70a8ab47cad115ecb7116d98c920b366069a2491170661c51664352fd 17286312 dynare_4.6.3.orig.tar.xz
 ff1968529adee0b1b6fcb1ff3310345a5e61d878d0b4552fe0cdb4ff05a6c445 2292 dynare_4.6.3.orig.tar.xz.asc
 b7b0b3477fd9b707efbae2a31a253d42c8a7e9cbe9c04fc356cee3395058c092 57364 dynare_4.6.3-1.debian.tar.xz
 10a50aca47962505b9805f5065a1ac60101e9f3213d0ee3481e8ab0aa19b6336 23966 dynare_4.6.3-1_amd64.buildinfo
Files:
 83a79d1e199d7af7545a3e4439a78189 2680 math optional dynare_4.6.3-1.dsc
 d683f9f15c8535f7124fdbbe0b0d6334 17286312 math optional dynare_4.6.3.orig.tar.xz
 930be29316c396f645b9ddcf82d146a0 2292 math optional dynare_4.6.3.orig.tar.xz.asc
 0e1c95c71f7187a7d943f53cd7a24648 57364 math optional dynare_4.6.3-1.debian.tar.xz
 12a18cd5de32ca16d9f27dbcd2e5b3b7 23966 math optional dynare_4.6.3-1_amd64.buildinfo

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEU5UdlScuDFuCvoxKLOzpNQ7OvkoFAl++I8UACgkQLOzpNQ7O
vkriWg//ZJFtPJ8vb8fBdDMdc+9V+4FxLv+lUbhJMXQ+3fTlk0eWL9kNeoGvoaRZ
Dsb09Vze18rLChfuyoHzu2aBf4c68ty5/hVym/1L6AUJr3xKQ4gsVYaGFZXLUKeD
UkIGV1OvytRgBjQd/MpxOte0uijs4/opJIag/o0fUIFsnKimDnhsLgov0XbEtLIo
NdI9m6Mz4YUMkGqJddw6PQag+5ojyatPY/dyXStlb3MMj0OtDkqx1BwM2WsZ0LpY
7/6/6WUlF8tuTaG6cSHXl7FStUnCC8HKiVR0B6oqA2gs/xdym0StEF6zibjJLnYQ
Gcu6QPEiNmQvA7NyXR2ZO9DdghdivyK+fktGP/K0I8QSD1MRdtNh/RGfO0p5DBcY
OunOJQx4AWKlPfJDyXJtFqs45dnbCwm+AGNq3AzNv0yePinD2RBMliP+vNTX3c1Q
Zmt5HQG6uMqqueriV7YsXNzrxFfHwsMhSz5V3bwiIjkNfdM3Xmm/M9CS0OdTg6pB
YV15jxQSIFqQKLkQRTnhWokjD1JethvX1grYKZMM79Mw3FDEXFETPI6cbVKaPT05
33iFlcGkq1eKiNqKeBLm9UAwC6GwDx+nf1NRKzmk+agwDoYTSfFWe6PZqOlNR8PX
ohDZOEdwR+X49EsBgWYDfLBuxFksOKX50V7TjJNCKjWZyiltApw=
=o3dS
-----END PGP SIGNATURE-----

Message #20 received at 974959@bugs.debian.org (full text, mbox, reply):

From: Vagrant Cascadian <vagrant@reproducible-builds.org>

To: 974959@bugs.debian.org

Subject: Re: dynare: reproducible builds: autotools generated files contain variable paths and data

Date: Wed, 25 Nov 2020 12:55:10 -0800

[Message part 1 (text/plain, inline)]

Thanks for the upload, it appears to have fixed *almost* all the
reproducibility issues!

Unfortunately there is still an issue with Makefile.in, looking at the
latest build:

  https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/diffoscope-results/dynare.html

Apparently Makefile.in is generated in an non-deterministic way, which
somehow I didn't catch in my earlier testing.

It might be good to remove Makefile.in as well, as those files can still
be regenerated from Makefile.am...


live well,
  vagrant

[signature.asc (application/pgp-signature, inline)]

Message #25 received at 974959@bugs.debian.org (full text, mbox, reply):

From: Sébastien Villemot <sebastien@debian.org>

To: Vagrant Cascadian <vagrant@reproducible-builds.org>, 974959@bugs.debian.org

Subject: Re: Bug#974959: dynare: reproducible builds: autotools generated files contain variable paths and data

Date: Fri, 27 Nov 2020 09:54:37 +0100

[Message part 1 (text/plain, inline)]

Le mercredi 25 novembre 2020 à 12:55 -0800, Vagrant Cascadian a écrit :
> Thanks for the upload, it appears to have fixed *almost* all the
> reproducibility issues!
> 
> Unfortunately there is still an issue with Makefile.in, looking at the
> latest build:
> 
>   
> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/diffoscope-results/dynare.html
> 
> 
> Apparently Makefile.in is generated in an non-deterministic way, which
> somehow I didn't catch in my earlier testing.
> 
> It might be good to remove Makefile.in as well, as those files can still
> be regenerated from Makefile.am...

Removing Makefile.in is a bit more complicated, because that means that
automake needs to be run again in the postinst (we’re talking about
compilation on the user machine), and a dependency on automake thus
needs to be added.

Alternatively, the problem could be fixed directly in automake. I don’t
see why it should behave non-deterministically.

-- 
⢀⣴⠾⠻⢶⣦⠀  Sébastien Villemot
⣾⠁⢠⠒⠀⣿⡁  Debian Developer
⢿⡄⠘⠷⠚⠋⠀  https://sebastien.villemot.name
⠈⠳⣄⠀⠀⠀⠀  https://www.debian.org

[signature.asc (application/pgp-signature, inline)]

Message #40 received at 974959-done@bugs.debian.org (full text, mbox, reply):

From: Vagrant Cascadian <vagrant@reproducible-builds.org>

To: Sébastien Villemot <sebastien@debian.org>, 974959-done@bugs.debian.org

Subject: Re: Bug#974959: dynare: reproducible builds: autotools generated files contain variable paths and data

Date: Wed, 11 May 2022 16:35:13 -0700

[Message part 1 (text/plain, inline)]

Version: 4.6.4-2

On 2020-11-27, Sébastien Villemot wrote:
> Le mercredi 25 novembre 2020 à 12:55 -0800, Vagrant Cascadian a écrit :
>> Thanks for the upload, it appears to have fixed *almost* all the
>> reproducibility issues!
>> 
>> Unfortunately there is still an issue with Makefile.in, looking at the
>> latest build:
>> 
>>   
>> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/diffoscope-results/dynare.html
>> 
>> 
>> Apparently Makefile.in is generated in an non-deterministic way, which
>> somehow I didn't catch in my earlier testing.
>> 
>> It might be good to remove Makefile.in as well, as those files can still
>> be regenerated from Makefile.am...
>
> Removing Makefile.in is a bit more complicated, because that means that
> automake needs to be run again in the postinst (we’re talking about
> compilation on the user machine), and a dependency on automake thus
> needs to be added.
>
> Alternatively, the problem could be fixed directly in automake. I don’t
> see why it should behave non-deterministically.

Well, dynare has been building reproducibly since 4.6.4-2 (other than
some 4.7~ versions in experimental):

  https://tests.reproducible-builds.org/debian/history/dynare.html

Marking as done.

live well,
  vagrant

[signature.asc (application/pgp-signature, inline)]

Send a report that this bug log contains spam.