Debian Bug report logs - #935127
bash: please make the build reproducible

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: "Chris Lamb" <lamby@debian.org>

To: submit@bugs.debian.org

Subject: bash: please make the build reproducible

Date: Mon, 19 Aug 2019 12:07:38 -0700

[Message part 1 (text/plain, inline)]

Source: bash
Version: 5.0-4
Severity: wishlist
Tags: patch
User: reproducible-builds@lists.alioth.debian.org
Usertags: buildpath
X-Debbugs-Cc: reproducible-bugs@lists.alioth.debian.org

Hi,

A patch is attached that makes the Bash interpreter build
reproducibily for me in unstable.

This is because in various places it encoded the absolute build path
via the inclusion of output from CFLAGS or even quite-literally as
"BUILD_DIR".

I did have another patch that modified the source files "properly"
themselves but I was not only having problems with autoreconf-ing the
package (it worked, but the command was returning a non-zero exit code
regardless which I papered over with a "|| true"...?),  additionally
one of the Makefiles could not have it's BUILD_DIR patched to point to
some deterministic location as it would cause the package to FTBFS.
Thus, we would have required some post-processing anyway. (Another
alternative to this could be to simply not ship that particular file
as it is likely not going to work on end-user machines anyway).

 [0] https://reproducible-builds.org/


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

[bash.diff.txt (text/plain, attachment)]

Message #10 received at 935127@bugs.debian.org (full text, mbox, reply):

From: "Chris Lamb" <lamby@debian.org>

To: 935127@bugs.debian.org, "Reproducible builds folks" <reproducible-builds@lists.alioth.debian.org>

Subject: Re: Bug#935127: bash: please make the build reproducible

Date: Fri, 28 Aug 2020 11:56:51 +0100

Chris Lamb wrote:

> A patch is attached that makes the Bash interpreter build
> reproducibly for me in unstable.

Gentle ping on this?

Bash is one of the few remaining packages in the 'Essential' package
set that remains unreproducible, and this patch has been in the BTS
for over a year now.


Regards,

--
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org 🍥 chris-lamb.co.uk
       `-

Message #15 received at 935127@bugs.debian.org (full text, mbox, reply):

From: "Chris Lamb" <lamby@debian.org>

To: 935127@bugs.debian.org

Cc: reproducible-bugs@lists.alioth.debian.org

Subject: Re: bash: please make the build reproducible

Date: Thu, 15 Oct 2020 16:58:03 -0000

Hi Bash maintainers,

> Bash is one of the few remaining packages in the 'Essential' package
> set that remains unreproducible, and this patch has been in the BTS
> for over a year now.

So, there are now only two packages in the Essential set that are
unreproducible. I plan to work on the other package (Perl) shortly,
but having Bash fixed in the archive itself would be very welcome
and motivating withal.


Kind regards,

--
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

Message #20 received at 935127@bugs.debian.org (full text, mbox, reply):

From: Matthias Klose <doko@debian.org>

To: Chris Lamb <lamby@debian.org>, 935127@bugs.debian.org

Cc: reproducible-bugs@lists.alioth.debian.org

Subject: Re: Bug#935127: bash: please make the build reproducible

Date: Thu, 15 Oct 2020 19:16:46 +0200

On 10/15/20 6:58 PM, Chris Lamb wrote:
> Hi Bash maintainers,
> 
>> Bash is one of the few remaining packages in the 'Essential' package
>> set that remains unreproducible, and this patch has been in the BTS
>> for over a year now.
> 
> So, there are now only two packages in the Essential set that are
> unreproducible. I plan to work on the other package (Perl) shortly,
> but having Bash fixed in the archive itself would be very welcome
> and motivating withal.

really?  No libgcc in the essential set? yes, it would be motivating if you
would address the GCC issues upstream and not keeping a set of local patches.

Message #25 received at 935127-close@bugs.debian.org (full text, mbox, reply):

From: Debian FTP Masters <ftpmaster@ftp-master.debian.org>

To: 935127-close@bugs.debian.org

Subject: Bug#935127: fixed in bash 5.1~rc1-2

Date: Thu, 15 Oct 2020 17:33:27 +0000

Source: bash
Source-Version: 5.1~rc1-2
Done: Matthias Klose <doko@debian.org>

We believe that the bug you reported is fixed in the latest version of
bash, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 935127@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Matthias Klose <doko@debian.org> (supplier of updated bash package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Thu, 15 Oct 2020 19:10:56 +0200
Source: bash
Architecture: source
Version: 5.1~rc1-2
Distribution: unstable
Urgency: medium
Maintainer: Matthias Klose <doko@debian.org>
Changed-By: Matthias Klose <doko@debian.org>
Closes: 935127
Changes:
 bash (5.1~rc1-2) unstable; urgency=medium
 .
   * Make the build reproducible. Closes: #935127.
Checksums-Sha1:
 a43da30fb4774d1a2889540989dafa44ac46d169 2324 bash_5.1~rc1-2.dsc
 2c5fffb728ac40f2ea66a9433060e1a2a8cc14e3 88152 bash_5.1~rc1-2.debian.tar.xz
 9fc4312ccecb30e7d5d939f26e1e18a0b9fce8ad 6224 bash_5.1~rc1-2_source.buildinfo
Checksums-Sha256:
 00c2802dc6229a527f3aa741ed3934d0b4ff34e48d6bfa372e10026be36fa351 2324 bash_5.1~rc1-2.dsc
 3fe3e6964036cc535d8d5352874eb6ecced45dc808e27c88e38a77e18662a893 88152 bash_5.1~rc1-2.debian.tar.xz
 41841de4fc451ad87271e8d2251f802a00a531459e679916b1b475fbaf66432f 6224 bash_5.1~rc1-2_source.buildinfo
Files:
 5be0a056486d38e15be777ef7959eec5 2324 base required bash_5.1~rc1-2.dsc
 4d8e5d52d561bd9371d894bb2f2f2689 88152 base required bash_5.1~rc1-2.debian.tar.xz
 53df6099da63a739ba09fc2681240104 6224 base required bash_5.1~rc1-2_source.buildinfo

-----BEGIN PGP SIGNATURE-----

iQJEBAEBCAAuFiEE1WVxuIqLuvFAv2PWvX6qYHePpvUFAl+Ig4QQHGRva29AZGVi
aWFuLm9yZwAKCRC9fqpgd4+m9dDJD/90mAIgBWjwFTQnnZP63tAoijpZgguyDaAD
IyKd8R7Lzg4TXqFFYQNKJHnG7jTHlRBnUSEqHjhzQSRxvnNOy9AXfFJsf2GDeD1O
ty6BZdtGFtwXLi6KXk7GSbh4tWJ91UOqZnoGJwpfW+50hTUd72lbvRbsxoQ2pWHO
tawLxEYPGh4Gs7rNUGRzelILzPKcy/t5Pyr69aENqhCZqbsISaEVuerrXJqFJUHr
6cGKqzh6hmvZ13YXkrIXrEwT/LSzGrMcAiLZutB7YbrE5mw/Fsw/tA2a3Yk2beBU
JKynoRxjBeX+PB6kulpmcTgGAbKDmVOsTPGkTr61fWhtGSlHyjJ2H6ezLy3m8VwJ
tDbvLieXTX9Q7fTaV1crqymXiAJmxLraxOFZ9pzZFSmo45z/8HQnqrijnNRSbdQ2
G54NpTQY/ZIXSE1XXWauB/ma00fNjvwbOsANO8bAHok9WkZgmvFrWjdU5beraGt4
vhYwDXbISOajLZb/xbrPxfUYEgS9YBzDLlxfPqw12AMUxqLFFHAPML9ub3k/tt1y
5LeSW/KWezgvwtAuzajQQEtdPbcAny3C2aCzysivrvTBqdSd4i/rk5FfzCzlV5gq
ILgEhQeMCtIHSDTaTEvfJYYjtcoPKDqoXdocRmgnYQXT59y3UdO3W8JcSXOlf0lH
/njA8tvPlw==
=PZCD
-----END PGP SIGNATURE-----

Message #30 received at 935127@bugs.debian.org (full text, mbox, reply):

From: "Chris Lamb" <lamby@debian.org>

To: "Matthias Klose" <doko@debian.org>, 935127@bugs.debian.org

Cc: reproducible-bugs@lists.alioth.debian.org

Subject: Re: Bug#935127: bash: please make the build reproducible

Date: Thu, 15 Oct 2020 22:58:03 -0000

Hi Matthias Klose,

> >> Bash is one of the few remaining packages in the 'Essential' package
> >> set that remains unreproducible, and this patch has been in the BTS
> >> for over a year now.
> >
> > So, there are now only two packages in the Essential set that are
> > unreproducible. I plan to work on the other package (Perl) shortly,
> > but having Bash fixed in the archive itself would be very welcome
> > and motivating withal.
>
> really?  No libgcc in the essential set? yes, it would be motivating if you
> would address the GCC issues upstream and not keeping a set of local patches.

Unfortunately I don't understand the hostility of this reply or how
it is relevant to Bash. Perhaps we are talking at cross-purposes. I am
using this page:

  https://tests.reproducible-builds.org/debian/unstable/amd64/pkg_set_essential.html

… which does not list GCC. I was also not aware that I was keeping
a set of local patches.

However, thank you for resolving this bug in your latest upload; really
really appreciated.


Kind regards,

--
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org 🍥 chris-lamb.co.uk
       `-

Message #35 received at 935127@bugs.debian.org (full text, mbox, reply):

From: Matthias Klose <doko@debian.org>

To: Chris Lamb <lamby@debian.org>, 935127@bugs.debian.org

Cc: reproducible-bugs@lists.alioth.debian.org

Subject: Re: Bug#935127: bash: please make the build reproducible

Date: Fri, 16 Oct 2020 10:54:18 +0200

On 10/16/20 12:58 AM, Chris Lamb wrote:
> Hi Matthias Klose,
> 
>>>> Bash is one of the few remaining packages in the 'Essential' package
>>>> set that remains unreproducible, and this patch has been in the BTS
>>>> for over a year now.
>>>
>>> So, there are now only two packages in the Essential set that are
>>> unreproducible. I plan to work on the other package (Perl) shortly,
>>> but having Bash fixed in the archive itself would be very welcome
>>> and motivating withal.
>>
>> really?  No libgcc in the essential set? yes, it would be motivating if you
>> would address the GCC issues upstream and not keeping a set of local patches.
> 
> Unfortunately I don't understand the hostility of this reply or how
> it is relevant to Bash.

Hostility?  You started speaking about "motivation" here.  If you need that,
fine.  But then why demotivate others by keeping local patches and not working
on upstreaming those?  It's now years that the reproducible builds project
doesn't address some of it's more fundamental issue with compilers.

Demotivated, Matthias

Message #40 received at 935127@bugs.debian.org (full text, mbox, reply):

From: Holger Levsen <holger@layer-acht.org>

To: Matthias Klose <doko@debian.org>

Cc: 935127@bugs.debian.org, reproducible-bugs@lists.alioth.debian.org

Subject: Re: Bug#935127: bash: please make the build reproducible

Date: Tue, 20 Oct 2020 08:13:32 +0000

[Message part 1 (text/plain, inline)]

Hi Matthias,

first: many thanks for uploading a fixed bash package!

second: the rest of this mail is not about bash anymore, so still mailing the
bug is kind of wrong. I'm doing it anyway to keep references intact. (Feel
free to just reply to the r-b-bugs list if you think thats better. or clone
and reassign, dunno?)

so, about libgcc and the essential set...

On Fri, Oct 16, 2020 at 10:54:18AM +0200, Matthias Klose wrote:
> >>> So, there are now only two packages in the Essential set that are
> >>> unreproducible. I plan to work on the other package (Perl) shortly,
> >>> but having Bash fixed in the archive itself would be very welcome
> >>> and motivating withal.
> >> really?  No libgcc in the essential set? yes, it would be motivating if you
> >> would address the GCC issues upstream and not keeping a set of local patches.

about gcc:

a.) I don't think *we* (r-b) have local GCC/libgcc patches since 2018, instead
    we are just using the gcc packages from Debian. We certainly have nothing
    not in our repo (because it's been empty for a while, including all
    of 2020).
    If "we" have them in Debian, I'd very much appreciate a quick
    pointer which of the 
    https://sources.debian.org/src/gcc-10/10.2.0-15/debian/patches/
    contain patches from us we should upstream?

b.) or am I/we miss something else?

about essential:

maybe/probably we (well, you and Chris) have been talking about different
essential sets or definitions, because
https://tests.reproducible-builds.org/debian/bullseye/amd64/pkg_set_essential.html
does not contain GCC, even the build-essential pkg_set doesnt contain it,
only https://tests.reproducible-builds.org/debian/bullseye/amd64/pkg_set_build-essential-depends.html,
which might be a bug in how we calculate the pkg sets...

but then, I think the calculation is right, see the one line at
https://salsa.debian.org/qa/jenkins.debian.net/-/blob/master/bin/reproducible_create_meta_pkg_sets.sh#L155
which considers all binary packages which set "Essential: yes" (and then looks up 
the source package that binary is coming from.) Or am I wrong?

> > Unfortunately I don't understand the hostility of this reply or how
> > it is relevant to Bash.
> Hostility?  You started speaking about "motivation" here.  If you need that,
> fine.  But then why demotivate others...

I'm very sorry this discussion arrived here. And that's all I'm going to say
about those 4 lines, except that we surely don't want to demotivate anyone. I'm also
sure Chris feels this way and regrets that his words caused "harm" on you.

(and i'm not sure about the quotes around harm, demotivation is surely harm noone wants.)

> ... by keeping local patches and not working
> on upstreaming those?  It's now years that the reproducible builds project
> doesn't address some of it's more fundamental issue with compilers.

To repeat very clearly: we (*) are not aware which patches you are talking about
and we would really appreciate pointers. And we surely want to upstream
everything we do. And we very much appreciate help. (And reminders if needed.)

(*) I've asked around.. :)

And we, I, want everyone to be happy and motivated. And if we fail, we like to 
know.

Last and not least: thanks for your time, support and all the work you put into
maintaining these important packages! I very much hope together we can fix all
the bugs which annoy us! Seriosly.


-- 
cheers,
	Holger

-------------------------------------------------------------------------------
               holger@(debian|reproducible-builds|layer-acht).org
       PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C

"... the premise [is] that privacy is about hiding a wrong. It's not.
 Privacy is an inherent human right, and a requirement for maintaining
 the human condition with dignity and respect." (Bruce Schneier)

[signature.asc (application/pgp-signature, inline)]

Message #45 received at 935127@bugs.debian.org (full text, mbox, reply):

From: Matthias Klose <doko@debian.org>

To: Holger Levsen <holger@layer-acht.org>

Cc: 935127@bugs.debian.org, reproducible-bugs@lists.alioth.debian.org

Subject: Re: Bug#935127: bash: please make the build reproducible

Date: Wed, 21 Oct 2020 13:11:53 +0200

On 10/20/20 10:13 AM, Holger Levsen wrote:
> Hi Matthias,
> 
> first: many thanks for uploading a fixed bash package!
> 
> second: the rest of this mail is not about bash anymore, so still mailing the
> bug is kind of wrong. I'm doing it anyway to keep references intact. (Feel
> free to just reply to the r-b-bugs list if you think thats better. or clone
> and reassign, dunno?)
> 
> so, about libgcc and the essential set...
> 
> On Fri, Oct 16, 2020 at 10:54:18AM +0200, Matthias Klose wrote:
>>>>> So, there are now only two packages in the Essential set that are
>>>>> unreproducible. I plan to work on the other package (Perl) shortly,
>>>>> but having Bash fixed in the archive itself would be very welcome
>>>>> and motivating withal.
>>>> really?  No libgcc in the essential set? yes, it would be motivating if you
>>>> would address the GCC issues upstream and not keeping a set of local patches.
> 
> about gcc:
> 
> a.) I don't think *we* (r-b) have local GCC/libgcc patches since 2018, instead
>     we are just using the gcc packages from Debian. We certainly have nothing
>     not in our repo (because it's been empty for a while, including all
>     of 2020).
>     If "we" have them in Debian, I'd very much appreciate a quick
>     pointer which of the 
>     https://sources.debian.org/src/gcc-10/10.2.0-15/debian/patches/
>     contain patches from us we should upstream?
> 
> b.) or am I/we miss something else?

I'm talking about

https://gcc.gnu.org/pipermail/gcc-patches/2017-July/479571.html
https://gcc.gnu.org/pipermail/gcc-patches/2017-August/480573.html

submitted upstream by Ximin Luo, not accepted in this form, and not part of the
Debian package either.  In the past the r-b effort used to build their own
compiler with a bunch of patches.  Is this still the case?

> about essential:
> 
> maybe/probably we (well, you and Chris) have been talking about different
> essential sets or definitions, because
> https://tests.reproducible-builds.org/debian/bullseye/amd64/pkg_set_essential.html
> does not contain GCC, even the build-essential pkg_set doesnt contain it,
> only https://tests.reproducible-builds.org/debian/bullseye/amd64/pkg_set_build-essential-depends.html,
> which might be a bug in how we calculate the pkg sets...
> 
> but then, I think the calculation is right, see the one line at
> https://salsa.debian.org/qa/jenkins.debian.net/-/blob/master/bin/reproducible_create_meta_pkg_sets.sh#L155
> which considers all binary packages which set "Essential: yes" (and then looks up 
> the source package that binary is coming from.) Or am I wrong?
> 
>>> Unfortunately I don't understand the hostility of this reply or how
>>> it is relevant to Bash.
>> Hostility?  You started speaking about "motivation" here.  If you need that,
>> fine.  But then why demotivate others...
> 
> I'm very sorry this discussion arrived here. And that's all I'm going to say
> about those 4 lines, except that we surely don't want to demotivate anyone. I'm also
> sure Chris feels this way and regrets that his words caused "harm" on you.
> 
> (and i'm not sure about the quotes around harm, demotivation is surely harm noone wants.)
> 
>> ... by keeping local patches and not working
>> on upstreaming those?  It's now years that the reproducible builds project
>> doesn't address some of it's more fundamental issue with compilers.
> 
> To repeat very clearly: we (*) are not aware which patches you are talking about
> and we would really appreciate pointers. And we surely want to upstream
> everything we do. And we very much appreciate help. (And reminders if needed.)
> 
> (*) I've asked around.. :)
> 
> And we, I, want everyone to be happy and motivated. And if we fail, we like to 
> know.
> 
> Last and not least: thanks for your time, support and all the work you put into
> maintaining these important packages! I very much hope together we can fix all
> the bugs which annoy us! Seriosly.
> 
> 

Message #50 received at 935127@bugs.debian.org (full text, mbox, reply):

From: Holger Levsen <holger@layer-acht.org>

To: Matthias Klose <doko@debian.org>

Cc: 935127@bugs.debian.org, reproducible-bugs@lists.alioth.debian.org

Subject: Re: Bug#935127: bash: please make the build reproducible

Date: Thu, 22 Oct 2020 15:01:21 +0000

[Message part 1 (text/plain, inline)]

On Wed, Oct 21, 2020 at 01:11:53PM +0200, Matthias Klose wrote:
> > b.) or am I/we miss something else?
> 
> I'm talking about
> https://gcc.gnu.org/pipermail/gcc-patches/2017-July/479571.html
> https://gcc.gnu.org/pipermail/gcc-patches/2017-August/480573.html

ah, those patches! 
 
> submitted upstream by Ximin Luo, not accepted in this form, and not part of the
> Debian package either.  In the past the r-b effort used to build their own
> compiler with a bunch of patches.  Is this still the case?

no.

sadly these patches are basically orphaned right now, noone is feeling responsible for
getting them upstream. this is also because the workaround is so easy: just rebuild
in the same path. which is far from optimal, but that's where we are right now.


-- 
cheers,
	Holger

-------------------------------------------------------------------------------
               holger@(debian|reproducible-builds|layer-acht).org
       PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C

“If the fires of 2020 horrify you, consider that, no matter what we do, by
 2050, when the benefits of even fast climate action will only begin to arrive,
 the area burned annually in the (US) West is expected to at least double and
 perhaps quadruple.” (David Wallace-Wells)

[signature.asc (application/pgp-signature, inline)]

Send a report that this bug log contains spam.