Debian Bug report logs - #851809
mono: please make the output of dh_makeclilibs reproducible

[Message part 1 (text/plain, inline)]
Source: mono
Version: 4.6.2.7+dfsg-1
Severity: wishlist
Tags: patch
User: reproducible-builds@lists.alioth.debian.org
Usertags: randomness toolchain
X-Debbugs-Cc: reproducible-bugs@lists.alioth.debian.org

Hi,

Whilst working on the Reproducible Builds effort [0], we noticed
that dh_makeclilibs generates "clilibs" files with a non-deterministic
order.

This affects packages such as gtk-sharp-beans. For example:

│ -gtk-sharp-beans 2.14.0.0__97a95fb57b03c03a libgtk-sharp-beans-cil
│  gio-sharp 2.14.0.0__35e10195dab3c99f libgtk-sharp-beans-cil
│ +gtk-sharp-beans 2.14.0.0__97a95fb57b03c03a libgtk-sharp-beans-cil

Patch attached.

 [0] https://reproducible-builds.org/

Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

[mono.diff.txt (text/plain, attachment)]
> Would you consider applying this patch and uploading?

Friendly ping on this :)

Best wishes,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

[Message part 1 (text/plain, inline)]
Version: 4.6.2.7+dfsg-2

It looks like this was fixed some time ago:

  84aa977e3d2275a8cc43e21398dc90d01fb21b04
  Sort dh_makeclilibs output so it's deterministic. Thanks to Chris Lamb

Though a corresponding message and bug number didn't make it into
debian/changelog.

Marking as done.

live well,
  vagrant

On 2017-01-18, Chris Lamb wrote:
> Source: mono
> Version: 4.6.2.7+dfsg-1
> Severity: wishlist
> Tags: patch
> User: reproducible-builds@lists.alioth.debian.org
> Usertags: randomness toolchain
> X-Debbugs-Cc: reproducible-bugs@lists.alioth.debian.org
>
> Hi,
>
> Whilst working on the Reproducible Builds effort [0], we noticed
> that dh_makeclilibs generates "clilibs" files with a non-deterministic
> order.
>
> This affects packages such as gtk-sharp-beans. For example:
>
> │ -gtk-sharp-beans 2.14.0.0__97a95fb57b03c03a libgtk-sharp-beans-cil
> │  gio-sharp 2.14.0.0__35e10195dab3c99f libgtk-sharp-beans-cil
> │ +gtk-sharp-beans 2.14.0.0__97a95fb57b03c03a libgtk-sharp-beans-cil
>
> Patch attached.
>
>  [0] https://reproducible-builds.org/
>
>
> Regards,
>
> -- 
>       ,''`.
>      : :'  :     Chris Lamb
>      `. `'`      lamby@debian.org / chris-lamb.co.uk
>        `-
> diff --git a/debian/dh_makeclilibs b/debian/dh_makeclilibs
> index 976fd75..17cfa88 100755
> --- a/debian/dh_makeclilibs
> +++ b/debian/dh_makeclilibs
> @@ -297,6 +297,7 @@ foreach my $package (@{$dh{DOPACKAGES}}) {
>      if (-e "$tmp/DEBIAN/clilibs") {
>          doit("chmod", 644,   "$tmp/DEBIAN/clilibs");
>          doit("chown", "0:0", "$tmp/DEBIAN/clilibs");
> +        complex_doit("LC_ALL=C sort $tmp/DEBIAN/clilibs -o $tmp/DEBIAN/clilibs");
>      }
>  }
>  
> _______________________________________________
> Reproducible-bugs mailing list
> Reproducible-bugs@lists.alioth.debian.org
> https://lists.alioth.debian.org/mailman/listinfo/reproducible-bugs

[signature.asc (application/pgp-signature, inline)]
reopen 851809
thanks

Hi Vagrant,

> Version: 4.6.2.7+dfsg-2
>
> It looks like this was fixed some time ago:
>
>   84aa977e3d2275a8cc43e21398dc90d01fb21b04
>   Sort dh_makeclilibs output so it's deterministic. Thanks to Chris Lamb

Interesting. I don't see this commit SHA in the cli-common.git
repository, nor some associated change in dh_makeclilibs under a
different SHA.

But whatever the historical Git archaeology, the bug is still
affecting packages — de4dot, for example.

Hope we can get this resolved soon. If it helps, a simpler patch might
even be:

  @@ -187,7 +187,7 @@ foreach my $package (@{$dh{DOPACKAGES}}) {
           if (defined($dh{EXCLUDE_FIND}) && $dh{EXCLUDE_FIND} ne '') {
               $exclude = "! \\( $dh{EXCLUDE_FIND} \\) ";
           }
  -        open(FIND, "find $tmp -type f \\( -name '*.dll' \\) $exclude |");
  +        open(FIND, "find $tmp -type f \\( -name '*.dll' \\) $exclude | sort |");

           dll:
           while (<FIND>) {

Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org 🍥 chris-lamb.co.uk
       `-

[Message part 1 (text/plain, inline)]
On 2022-04-28, Chris Lamb wrote:
>> Version: 4.6.2.7+dfsg-2
>>
>> It looks like this was fixed some time ago:
>>
>>   84aa977e3d2275a8cc43e21398dc90d01fb21b04
>>   Sort dh_makeclilibs output so it's deterministic. Thanks to Chris Lamb
>
> Interesting. I don't see this commit SHA in the cli-common.git
> repository, nor some associated change in dh_makeclilibs under a
> different SHA.

https://salsa.debian.org/dotnet-team/mono/-/commit/84aa977e3d2275a8cc43e21398dc90d01fb21b04

Was this functionality moved to cli-common somehow?

> But whatever the historical Git archaeology, the bug is still
> affecting packages — de4dot, for example.

Looking at the history of de4dot, it seems consistantly reproducible on
amd64 and arm64, but consistantly unreproducible on i386 and armhf:

  https://tests.reproducible-builds.org/debian/history/amd64/de4dot.html
  https://tests.reproducible-builds.org/debian/history/i386/de4dot.html

So the obvious difference here is 32-bit vs. 64-bit... a non-obvious
difference is choice of locale for each architecture for
tests.reproducible-builds.org infrastructure differs.

Looking at the history for the originally referenced gtk-sharp-beans is
a little less consistant, but still more frequently unreproducible on
i386/armhf:

  https://tests.reproducible-builds.org/debian/history/amd64/gtk-sharp-beans.html
  https://tests.reproducible-builds.org/debian/history/i386/gtk-sharp-beans.html

> Hope we can get this resolved soon. If it helps, a simpler patch might
> even be:
>
>   @@ -187,7 +187,7 @@ foreach my $package (@{$dh{DOPACKAGES}}) {
>            if (defined($dh{EXCLUDE_FIND}) && $dh{EXCLUDE_FIND} ne '') {
>                $exclude = "! \\( $dh{EXCLUDE_FIND} \\) ";
>            }
>   -        open(FIND, "find $tmp -type f \\( -name '*.dll' \\) $exclude |");
>   +        open(FIND, "find $tmp -type f \\( -name '*.dll' \\) $exclude | sort |");
>            
>            dll:
>            while (<FIND>) {

I wonder if you wouldn't also want to specify the locale to sort here?

live well,
  vagrant

[signature.asc (application/pgp-signature, inline)]
Hey,

>> Interesting. I don't see this commit SHA in the cli-common.git
>> repository, nor some associated change in dh_makeclilibs under a
>> different SHA.
>
> https://salsa.debian.org/dotnet-team/mono/-/commit/84aa977e3d2275a8cc43e21398dc90d01fb21b04
>
> Was this functionality moved to cli-common somehow?

Ah yes, or the issue is the other way around — the dh_makeclilibs in
the Mono repository (linked by you above) could originally have been a
code copy of the version in cli-common and has now diverged. The
history of the version in the Mono repository suggests this
interpretation, or the use of "re-synced" suggests that direction
anyway:

  https://salsa.debian.org/dotnet-team/mono/-/commits/master/debian/dh_makeclilibs

The underlying direction doesn't really matter to me though, as the
version in cli-common does not sort. :)

>>   -        open(FIND, "find $tmp -type f \\( -name '*.dll' \\) $exclude |");
>>   +        open(FIND, "find $tmp -type f \\( -name '*.dll' \\) $exclude | sort |");

> I wonder if you wouldn't also want to specify the locale to sort here?

Ah, of course! More importantly, though, the change should be the same
in both versions, so I'd be minded to go with the explicit complex_doit
call. Hah, I think my "if it helps" extra patch didn't actually "help"
at all.

Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org 🍥 chris-lamb.co.uk
       `-

Debian Bug report logs - #851809 mono: please make the output of dh_makeclilibs reproducible

Debian Bug report logs - #851809
mono: please make the output of dh_makeclilibs reproducible