Debian Bug report logs - #806945
bash: Please make bash build reproducibly

version graph

Package: src:bash; Maintainer for src:bash is Matthias Klose <doko@debian.org>;

Reported by: Maria Valentina Marin <marivalenm@gmail.com>

Date: Thu, 3 Dec 2015 10:33:01 UTC

Severity: wishlist

Tags: patch

Found in version bash/4.3-14

Fixed in version bash/4.4.18-1

Done: Matthias Klose <doko@debian.org>

Bug is archived. No further changes may be made.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, marivalenm@gmail.com, reproducible-builds@lists.alioth.debian.org, Matthias Klose <doko@debian.org>:
Bug#806945; Package src:bash. (Thu, 03 Dec 2015 10:33:05 GMT) (full text, mbox, link).


Acknowledgement sent to Maria Valentina Marin <marivalenm@gmail.com>:
New Bug report received and forwarded. Copy sent to marivalenm@gmail.com, reproducible-builds@lists.alioth.debian.org, Matthias Klose <doko@debian.org>. (Thu, 03 Dec 2015 10:33:05 GMT) (full text, mbox, link).


Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Maria Valentina Marin <marivalenm@gmail.com>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: bash: Please make bash build reproducibly
Date: Thu, 03 Dec 2015 11:32:36 +0100
Source: bash
Version: 4.3-14
Severity: wishlist
User: reproducible-builds@lists.alioth.debian.org
Usertags: timestamps

Hi,

While working on the “reproducible builds” effort [1], we have noticed
that bash could not be built reproducibly.

There are two problems:
1. Bash uses an embedded copy of man2html which produces html that
   contains timestamps, it is recommended to drop this internal copy and
   instead depend on the Debian man2html which contains a patch [4].

2. The pdf files created by dvipdfmx contain fonts with indeterministic
   order and naming [2]. This can be fixed by not generating the pdf in
   the first place as gnu codding standards makes pdf generation
   optional [3]. This has to be solved upstream.

Regards,
akira

 [1]: https://wiki.debian.org/ReproducibleBuilds
 [2]: https://reproducible.debian.net/issues/unstable/fonts_in_pdf_files_issue.html
 [3]: https://www.gnu.org/prep/standards/standards.html#Standard-Targets
 [4]: http://sources.debian.net/src/man2html/1.6g-8/debian/patches/035-source-date-epoch.patch/



Information forwarded to debian-bugs-dist@lists.debian.org, Matthias Klose <doko@debian.org>:
Bug#806945; Package src:bash. (Fri, 04 Dec 2015 09:27:04 GMT) (full text, mbox, link).


Acknowledgement sent to ludo@gnu.org (Ludovic Courtès):
Extra info received and forwarded to list. Copy sent to Matthias Klose <doko@debian.org>. (Fri, 04 Dec 2015 09:27:04 GMT) (full text, mbox, link).


Message #10 received at 806945@bugs.debian.org (full text, mbox, reply):

From: ludo@gnu.org (Ludovic Courtès)
To: Greg Wooledge <wooledg@eeg.ccf.org>
Cc: bug-bash@gnu.org, 806945@bugs.debian.org, Maria Valentina Marin <marivalenm@gmail.com>
Subject: Re: Installation of PDF/PS/DVI and HTML files
Date: Fri, 04 Dec 2015 11:24:23 +0200
Greg Wooledge <wooledg@eeg.ccf.org> skribis:

> On Thu, Dec 03, 2015 at 01:08:13PM +0200, Ludovic Courtès wrote:
>> Given that the GCS suggests installing only the Info version of the
>> manual by default (info "(standards) Standard Targets")
>
>> What do you think?
>
> I think that's a stupid suggestion.  The de facto standard for "make"
> followed "make install" on a Unix-like system is to install man pages.
> If there's an info page, I have no objection to installing that as well,
> but to omit the standard man pages by default is ridiculous.

Agreed; apologies for being unclear.

As Mathieu wrote, I am of course fine installing man and Info manuals by
default, like GNU packages generally do.

The suggestion I make is to not install PDF/PS/DVI and HTML files by
default.  This would comply with the GCS and user expectations, and also
sidestep the bit-for-bit reproducibility issues that generating those
PDF/PS/DVI/HTML files entails.

What do you think?

Thanks,
Ludo’.



Information forwarded to debian-bugs-dist@lists.debian.org, Matthias Klose <doko@debian.org>:
Bug#806945; Package src:bash. (Sat, 05 Dec 2015 22:24:03 GMT) (full text, mbox, link).


Acknowledgement sent to chet.ramey@case.edu:
Extra info received and forwarded to list. Copy sent to Matthias Klose <doko@debian.org>. (Sat, 05 Dec 2015 22:24:03 GMT) (full text, mbox, link).


Message #15 received at 806945@bugs.debian.org (full text, mbox, reply):

From: Chet Ramey <chet.ramey@case.edu>
To: 806945@bugs.debian.org
Subject: Fwd: Re: Installation of PDF/PS/DVI and HTML files
Date: Sat, 5 Dec 2015 17:15:31 -0500
[Message part 1 (text/plain, inline)]

[Re: Installation of PDF/PS/DVI and HTML files.eml (message/rfc822, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, Matthias Klose <doko@debian.org>:
Bug#806945; Package src:bash. (Sun, 06 Dec 2015 00:15:08 GMT) (full text, mbox, link).


Acknowledgement sent to chet.ramey@case.edu:
Extra info received and forwarded to list. Copy sent to Matthias Klose <doko@debian.org>. (Sun, 06 Dec 2015 00:15:08 GMT) (full text, mbox, link).


Message #20 received at 806945@bugs.debian.org (full text, mbox, reply):

From: Chet Ramey <chet.ramey@case.edu>
To: Ludovic Courtès <ludo@gnu.org>, Greg Wooledge <wooledg@eeg.ccf.org>
Cc: chet.ramey@case.edu, bug-bash@gnu.org, 806945@bugs.debian.org, Maria Valentina Marin <marivalenm@gmail.com>
Subject: Re: Installation of PDF/PS/DVI and HTML files
Date: Sat, 5 Dec 2015 19:10:25 -0500
On 12/4/15 4:24 AM, Ludovic Courtès wrote:
> Greg Wooledge <wooledg@eeg.ccf.org> skribis:
> 
>> On Thu, Dec 03, 2015 at 01:08:13PM +0200, Ludovic Courtès wrote:
>>> Given that the GCS suggests installing only the Info version of the
>>> manual by default (info "(standards) Standard Targets")
>>
>>> What do you think?
>>
>> I think that's a stupid suggestion.  The de facto standard for "make"
>> followed "make install" on a Unix-like system is to install man pages.
>> If there's an info page, I have no objection to installing that as well,
>> but to omit the standard man pages by default is ridiculous.
> 
> Agreed; apologies for being unclear.
> 
> As Mathieu wrote, I am of course fine installing man and Info manuals by
> default, like GNU packages generally do.
> 
> The suggestion I make is to not install PDF/PS/DVI and HTML files by
> default.  

Again, only the HTML files are installed by `make install'.  The sticking
point here appears to be installing the HTML files, which you can suppress
by running `make install' with htmldir set to the empty string.  Is that
what you're saying?

> This would comply with the GCS and user expectations, and also
> sidestep the bit-for-bit reproducibility issues that generating those
> PDF/PS/DVI/HTML files entails.

So the problem is once again the build and not the install?  Since the
build version appears in the version string, and that changes each time
the binary is rebuilt, bit-by-bit reproducibility is not going to be
generally possible.

However, if it's the build, if something changes when you run make, it
implies that one of the source files changed or that the target did not
exist.  bash-4.4, unlike bash-4.3, will ship with the generated
documentation (look at the bash-4.4-beta distribution, for example).
Given that, under what circumstances would the generated documentation
need to be rebuilt by this `reproducible builds' effort?

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
		 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRU    chet@case.edu    http://cnswww.cns.cwru.edu/~chet/



Information forwarded to debian-bugs-dist@lists.debian.org, Matthias Klose <doko@debian.org>:
Bug#806945; Package src:bash. (Sun, 06 Dec 2015 21:21:09 GMT) (full text, mbox, link).


Acknowledgement sent to ludo@gnu.org (Ludovic Courtès):
Extra info received and forwarded to list. Copy sent to Matthias Klose <doko@debian.org>. (Sun, 06 Dec 2015 21:21:09 GMT) (full text, mbox, link).


Message #25 received at 806945@bugs.debian.org (full text, mbox, reply):

From: ludo@gnu.org (Ludovic Courtès)
To: Chet Ramey <chet.ramey@case.edu>
Cc: Greg Wooledge <wooledg@eeg.ccf.org>, bug-bash@gnu.org, 806945@bugs.debian.org, Maria Valentina Marin <marivalenm@gmail.com>
Subject: Re: Installation of PDF/PS/DVI and HTML files
Date: Sun, 06 Dec 2015 22:18:20 +0100
Chet Ramey <chet.ramey@case.edu> skribis:

> On 12/4/15 4:24 AM, Ludovic Courtès wrote:
>> Greg Wooledge <wooledg@eeg.ccf.org> skribis:
>> 
>>> On Thu, Dec 03, 2015 at 01:08:13PM +0200, Ludovic Courtès wrote:
>>>> Given that the GCS suggests installing only the Info version of the
>>>> manual by default (info "(standards) Standard Targets")
>>>
>>>> What do you think?
>>>
>>> I think that's a stupid suggestion.  The de facto standard for "make"
>>> followed "make install" on a Unix-like system is to install man pages.
>>> If there's an info page, I have no objection to installing that as well,
>>> but to omit the standard man pages by default is ridiculous.
>> 
>> Agreed; apologies for being unclear.
>> 
>> As Mathieu wrote, I am of course fine installing man and Info manuals by
>> default, like GNU packages generally do.
>> 
>> The suggestion I make is to not install PDF/PS/DVI and HTML files by
>> default.  
>
> Again, only the HTML files are installed by `make install'.  The sticking
> point here appears to be installing the HTML files,

Right, HTML and PDF/PS/DVI.

> which you can suppress by running `make install' with htmldir set to
> the empty string.

I guess that would work, indeed.

>> This would comply with the GCS and user expectations, and also
>> sidestep the bit-for-bit reproducibility issues that generating those
>> PDF/PS/DVI/HTML files entails.
>
> So the problem is once again the build and not the install?  Since the
> build version appears in the version string, and that changes each time
> the binary is rebuilt, bit-by-bit reproducibility is not going to be
> generally possible.

The broader context is that Debian has a policy of rebootstrapping
packages; that is, even if there’s a ‘make dist’-produced tarball, they
will run ‘autoreconf’ et al.

In the case of Bash, that entails a rebuild of the HTML and PDF/PS/DVI
files.  There are two problems discussed at
<https://bugs.debian.org/806945>:

  1. The man2html copy that Bash provides produces non-deterministic
     output;

  2. DVI/PS/PDF generation is not deterministic; this is not a
     Bash-specific issue, but the fact that Bash tries to install these
     files by default make the issue visible to Debian.

While discussing it with Akira and others, it occurred to me that Bash
shouldn’t be installing HTML/PDF/PS/DVI by default, at least per my
understanding of the GCS and its implementation in Automake, hence my
proposal.

I hope this clarifies the context.

Thanks,
Ludo’.



Information forwarded to debian-bugs-dist@lists.debian.org, Matthias Klose <doko@debian.org>:
Bug#806945; Package src:bash. (Mon, 07 Dec 2015 00:51:20 GMT) (full text, mbox, link).


Acknowledgement sent to chet.ramey@case.edu:
Extra info received and forwarded to list. Copy sent to Matthias Klose <doko@debian.org>. (Mon, 07 Dec 2015 00:51:20 GMT) (full text, mbox, link).


Message #30 received at 806945@bugs.debian.org (full text, mbox, reply):

From: Chet Ramey <chet.ramey@case.edu>
To: Ludovic Courtès <ludo@gnu.org>
Cc: chet.ramey@case.edu, Greg Wooledge <wooledg@eeg.ccf.org>, bug-bash@gnu.org, 806945@bugs.debian.org, Maria Valentina Marin <marivalenm@gmail.com>
Subject: Re: Installation of PDF/PS/DVI and HTML files
Date: Sun, 6 Dec 2015 19:50:08 -0500
On 12/6/15 4:18 PM, Ludovic Courtès wrote:

>>> As Mathieu wrote, I am of course fine installing man and Info manuals by
>>> default, like GNU packages generally do.
>>>
>>> The suggestion I make is to not install PDF/PS/DVI and HTML files by
>>> default.  
>>
>> Again, only the HTML files are installed by `make install'.  The sticking
>> point here appears to be installing the HTML files,
> 
> Right, HTML and PDF/PS/DVI.

Please read what I wrote.  The PDF/PS/dvi files are not installed by any
target.

>> which you can suppress by running `make install' with htmldir set to
>> the empty string.
> 
> I guess that would work, indeed.
> 
>>> This would comply with the GCS and user expectations, and also
>>> sidestep the bit-for-bit reproducibility issues that generating those
>>> PDF/PS/DVI/HTML files entails.

And now we're back to generating the files.

>> So the problem is once again the build and not the install?  Since the
>> build version appears in the version string, and that changes each time
>> the binary is rebuilt, bit-by-bit reproducibility is not going to be
>> generally possible.
> 
> The broader context is that Debian has a policy of rebootstrapping
> packages; that is, even if there’s a ‘make dist’-produced tarball, they
> will run ‘autoreconf’ et al.
> 
> In the case of Bash, that entails a rebuild of the HTML and PDF/PS/DVI
> files.  

Well, it involves generating them, since up through bash-4.3, the standard
distribution does not include them.  Starting with bash-4.4, they'll be in
the distribution, so that will not be a problem.  (If history is any guide,
though, it will be 2-3 years before Debian has cycled to bash-4.4.)

I'm going to assume that even running autoreconf doesn't change enough
files to cause existing targets to be rebuilt when their source doesn't
change (i.e., it's not run with --force).

There are two problems discussed at
> <https://bugs.debian.org/806945>:
> 
>   1. The man2html copy that Bash provides produces non-deterministic
>      output;

Yes, because of a timestamp.  This will not be a problem if the files don't
have to be generated.

How about this:  I will make sure that shipping the generated
documentation will not cause files to be regenerated, and that this will
work whether the build is done in the source directory or not.

> 
>   2. DVI/PS/PDF generation is not deterministic; this is not a
>      Bash-specific issue, but the fact that Bash tries to install these
>      files by default make the issue visible to Debian.

This is simply false.  You seem to be using generate and install
interchangeably.

See above for the solution to generating these files with `make all'.

> While discussing it with Akira and others, it occurred to me that Bash
> shouldn’t be installing HTML/PDF/PS/DVI by default, at least per my
> understanding of the GCS and its implementation in Automake, hence my
> proposal.

So installing the html files is a separate issue, unrelated to the issues
raised by the `reproducible builds' folks.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
		 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRU    chet@case.edu    http://cnswww.cns.cwru.edu/~chet/



Information forwarded to debian-bugs-dist@lists.debian.org, Matthias Klose <doko@debian.org>:
Bug#806945; Package src:bash. (Sat, 28 May 2016 11:42:04 GMT) (full text, mbox, link).


Acknowledgement sent to Reiner Herrmann <reiner@reiner-h.de>:
Extra info received and forwarded to list. Copy sent to Matthias Klose <doko@debian.org>. (Sat, 28 May 2016 11:42:04 GMT) (full text, mbox, link).


Message #35 received at 806945@bugs.debian.org (full text, mbox, reply):

From: Reiner Herrmann <reiner@reiner-h.de>
To: 806945@bugs.debian.org
Subject: Re: bash: Please make bash build reproducibly
Date: Sat, 28 May 2016 13:38:35 +0200
[Message part 1 (text/plain, inline)]
Hi,

it looks like most of the documentation related issues are now solved
by fixed toolchain packages.  But it still uses the embedded and
outdated copy of man2html, which doesn't support SOURCE_DATE_EPOCH yet.
The attached patch uses the system man2html instead of the embedded one.

After that, the only remaining issue is that the included header file
/usr/include/bash/config.h varies depending on the kernel version used
during build [1] (with kernel <4, PGRP_PIPE is defined).

Regards,
  Reiner

[1]: https://sources.debian.net/src/bash/4.3-14/configure.ac/#L1088
[bash-man2html.patch (text/x-diff, attachment)]
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Matthias Klose <doko@debian.org>:
Bug#806945; Package src:bash. (Thu, 02 Jun 2016 19:24:03 GMT) (full text, mbox, link).


Acknowledgement sent to Ximin Luo <infinity0@debian.org>:
Extra info received and forwarded to list. Copy sent to Matthias Klose <doko@debian.org>. (Thu, 02 Jun 2016 19:24:04 GMT) (full text, mbox, link).


Message #40 received at 806945@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@debian.org>
To: 806945@bugs.debian.org
Cc: Reiner Herrmann <reiner@reiner-h.de>, Ludovic Courtès <ludo@gnu.org>, Chet Ramey <chet.ramey@case.edu>, Greg Wooledge <wooledg@eeg.ccf.org>, bug-bash@gnu.org, 806945@bugs.debian.org, Maria Valentina Marin <marivalenm@gmail.com>
Subject: Re: bash: Please make bash build reproducibly
Date: Thu, 2 Jun 2016 21:21:35 +0200
CC'ing everyone that was on the previous conversation; Debian's BTS doesn't do this automatically :(

We've made some progress on the Debian side; there's a few more stumbling blocks though:

On Sat, 28 May 2016 13:38:35 +0200 Reiner Herrmann <reiner@reiner-h.de> wrote:
> Hi,
> 
> it looks like most of the documentation related issues are now solved
> by fixed toolchain packages.  But it still uses the embedded and
> outdated copy of man2html, which doesn't support SOURCE_DATE_EPOCH yet.
> The attached patch uses the system man2html instead of the embedded one.
> 
> After that, the only remaining issue is that the included header file
> /usr/include/bash/config.h varies depending on the kernel version used
> during build [1] (with kernel <4, PGRP_PIPE is defined).
> 

For this particular example, we can just patch this out, i.e. remove it from the installed config.h. Debian already forces PGRP_PIPE 1 in config-bot.h, which config.h includes at the end.

In general however, installing config.h is a code smell and an anti-pattern - it takes test results from the *build machine*, and then forces my machine to assume those. The presence of this anti-pattern potentially can make future versions unreproducible again, and we'll have to think of new ways to fix those, since this PGRP_PIPE forcing is just a lucky coincidence.

The ideal solution from a software architecture viewpoint, would be (1) make the headers platform independent and not require a config.h, or if this is truly impossible then (2) instead of installing config.h, install some scripts to allow the user to generate their own config.h, with their own values. However, I don't know how much effort either of these options are. Only a few installed headers actually need config.h; perhaps these could be fixed to *not* require it:

/usr/include/bash$ grep -r '#include .config.h.'
builtins.h:#include "config.h"
lib/glob/strmatch.h:#include <config.h>
shell.h:#include "config.h"
shmbutil.h:#include <config.h>

X

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
git://github.com/infinity0/pubkeys.git



Information forwarded to debian-bugs-dist@lists.debian.org, Matthias Klose <doko@debian.org>:
Bug#806945; Package src:bash. (Mon, 06 Jun 2016 02:15:04 GMT) (full text, mbox, link).


Acknowledgement sent to Ximin Luo <infinity0@debian.org>:
Extra info received and forwarded to list. Copy sent to Matthias Klose <doko@debian.org>. (Mon, 06 Jun 2016 02:15:04 GMT) (full text, mbox, link).


Message #45 received at 806945@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@debian.org>
To: 806945@bugs.debian.org, Chet Ramey <chet.ramey@case.edu>
Cc: Reiner Herrmann <reiner@reiner-h.de>, Ludovic Courtès <ludo@gnu.org>, Greg Wooledge <wooledg@eeg.ccf.org>, bug-bash@gnu.org, Maria Valentina Marin <marivalenm@gmail.com>
Subject: Re: bash: Please make bash build reproducibly
Date: Mon, 6 Jun 2016 04:11:41 +0200
(Chet, your specific attention is required for this email, please)

Ximin Luo:
> On Sat, 28 May 2016 13:38:35 +0200 Reiner Herrmann <reiner@reiner-h.de> wrote:
>> After that, the only remaining issue is that the included header file
>> /usr/include/bash/config.h varies depending on the kernel version used
>> during build [1] (with kernel <4, PGRP_PIPE is defined).
> 
> For this particular example, we can just patch this out, i.e. remove it from the installed config.h. Debian already forces PGRP_PIPE 1 in config-bot.h, which config.h includes at the end.
> 

I dug into this a bit more and it looks like the cause of the difference is this snippet from configure.ac:

linux*)     LOCAL_LDFLAGS=-rdynamic      # allow dynamic loading
        case "`uname -r`" in
        2.[[456789]]*|3*)   AC_DEFINE(PGRP_PIPE) ;;
        esac ;;


This was added between bash-3.0.16 and bash-3.1, way before Linux 4 came out. So I wonder if this snippet should instead be:

linux*)     LOCAL_LDFLAGS=-rdynamic      # allow dynamic loading
        case "`uname -r`" in
        1.*|2.[[0123]]*) true ;;
        *) AC_DEFINE(PGRP_PIPE) ;;
        esac ;;

to set this for all future kernels? Then Debian (and probably other distros) could get rid of our patch, too.

However, the question still remains why config.h is installed into the end-user system, and if bash-built-with-linux-5 required PGRP_PIPE to be *undefined*, we would still have a reproducibility problem.

Ximin

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
git://github.com/infinity0/pubkeys.git



Information forwarded to debian-bugs-dist@lists.debian.org, Matthias Klose <doko@debian.org>:
Bug#806945; Package src:bash. (Tue, 07 Jun 2016 10:30:03 GMT) (full text, mbox, link).


Acknowledgement sent to Ximin Luo <infinity0@debian.org>:
Extra info received and forwarded to list. Copy sent to Matthias Klose <doko@debian.org>. (Tue, 07 Jun 2016 10:30:03 GMT) (full text, mbox, link).


Message #50 received at 806945@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@debian.org>
To: 806945@bugs.debian.org, Chet Ramey <chet.ramey@case.edu>
Cc: Reiner Herrmann <reiner@reiner-h.de>, Ludovic Courtès <ludo@gnu.org>, Greg Wooledge <wooledg@eeg.ccf.org>, bug-bash@gnu.org, Maria Valentina Marin <marivalenm@gmail.com>
Subject: Re: bash: Please make bash build reproducibly
Date: Tue, 7 Jun 2016 12:26:58 +0200
[Message part 1 (text/plain, inline)]
Control: tags -1 + patch

I've attached the full Debian patch to make bash 4.3-14 reproducible, that includes Reiner's from above.

It also includes *fixing a bug in upstream bash*, which currently does-not-appear-in-the-wild *only because* distros already happen to be working around it. But upstream really should fix it - i.e. to fix the PGRP_PIPE check in configure/configure.ac to account for Linux 4+.

I decided to just set PGRP_PIPE unconditionally in configure.ac and configure, because I figure nobody will ever again use linux 0, 1 or 2 so it's not worth the extra complexity (which only gives a minor performance gain anyway, as opposed to having *incorrect behaviour*). Upstream is free to choose whichever behaviour he wants - either this simpler version, or the more complex version from my previous email quoted below. Both are correct, if I understand right, and when either is applied to upstream, all distros (including Debian) can drop our specific patches for PGRP_PIPE.

To re-iterate again, this does not solve the longer-term issue of "installing config.h is bad".

Ximin

Ximin Luo:
> (Chet, your specific attention is required for this email, please)
> 
> Ximin Luo:
>> On Sat, 28 May 2016 13:38:35 +0200 Reiner Herrmann <reiner@reiner-h.de> wrote:
>>> After that, the only remaining issue is that the included header file
>>> /usr/include/bash/config.h varies depending on the kernel version used
>>> during build [1] (with kernel <4, PGRP_PIPE is defined).
>>
>> For this particular example, we can just patch this out, i.e. remove it from the installed config.h. Debian already forces PGRP_PIPE 1 in config-bot.h, which config.h includes at the end.
>>
> 
> I dug into this a bit more and it looks like the cause of the difference is this snippet from configure.ac:
> 
> linux*)     LOCAL_LDFLAGS=-rdynamic      # allow dynamic loading
>         case "`uname -r`" in
>         2.[[456789]]*|3*)   AC_DEFINE(PGRP_PIPE) ;;
>         esac ;;
> 
> 
> This was added between bash-3.0.16 and bash-3.1, way before Linux 4 came out. So I wonder if this snippet should instead be:
> 
> linux*)     LOCAL_LDFLAGS=-rdynamic      # allow dynamic loading
>         case "`uname -r`" in
>         1.*|2.[[0123]]*) true ;;
>         *) AC_DEFINE(PGRP_PIPE) ;;
>         esac ;;
> 
> to set this for all future kernels? Then Debian (and probably other distros) could get rid of our patch, too.
> 
> However, the question still remains why config.h is installed into the end-user system, and if bash-built-with-linux-5 required PGRP_PIPE to be *undefined*, we would still have a reproducibility problem.
> 
> Ximin
> 


-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
git://github.com/infinity0/pubkeys.git
[bash_4.3-14_4.3-14.0~reproducible1.debdiff (text/plain, attachment)]

Added tag(s) patch. Request was from Ximin Luo <infinity0@debian.org> to 806945-submit@bugs.debian.org. (Tue, 07 Jun 2016 10:30:03 GMT) (full text, mbox, link).


Information forwarded to debian-bugs-dist@lists.debian.org, Matthias Klose <doko@debian.org>:
Bug#806945; Package src:bash. (Mon, 26 Sep 2016 13:51:04 GMT) (full text, mbox, link).


Acknowledgement sent to Ximin Luo <infinity0@debian.org>:
Extra info received and forwarded to list. Copy sent to Matthias Klose <doko@debian.org>. (Mon, 26 Sep 2016 13:51:04 GMT) (full text, mbox, link).


Message #57 received at 806945@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@debian.org>
To: Debian Bug Tracking System <806945@bugs.debian.org>
Subject: Re: bash: Please make bash build reproducibly
Date: Mon, 26 Sep 2016 15:45:12 +0200
[Message part 1 (text/plain, inline)]
Package: src:bash
Followup-For: Bug #806945

Dear Maintainer,

Attached is the patch refreshed for bash version 4.4-1. I have confirmed that
it works the same way as advertised previously.

As an update, pgrp-pipe.diff is now optional; you can also just drop it to get
the same effect, since upstream have patched configure{,.ac} to correctly
handle Linux 4. However, I recommend keeping our patch as-is, in case upstream
forget to do this again for Linux 5.

Outside of this patch, we have also started to vary the build-path during our
tests. This has revealed that bash also embeds the build-path into the output;
see the attached diffoscope output for details. We don't have a patch for this
yet, but in case anyone wants to help here is how we *would* fix it:

CFLAGS - we eventually want to patch gcc to fix this; bash doesn't need to
  handle this specifically.
BUILD_DIR - this does seem bash-specific, but I haven't yet studied the
  Makefiles in enough detail to propose something concrete.
PDF ID - this is caused by [1] and we will probably patch texlive-bin to fix
  this; bash doesn't need to handle this specifically.

The other differences are due to the three mentioned above, and will disappear
once these are fixed.

It would still be good if you applied the attached patch, just to "get it in
there" so we don't have to keep refreshing it against newer bash versions.

Thanks!
Ximin

[1] https://sources.debian.net/src/texlive-bin/2016.20160513.41080-6/texk/web2c/pdftexdir/utils.c/#L731

-- System Information:
Debian Release: stretch/sid
  APT prefers testing
  APT policy: (990, 'testing'), (500, 'stable'), (300, 'unstable'), (200, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.6.0-1-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_GB.utf8, LC_CTYPE=en_GB.utf8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
[bash_4.4-1.0~reproducible1.diffoscope (text/plain, attachment)]
[bash_806945.patch (text/x-diff, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, Matthias Klose <doko@debian.org>:
Bug#806945; Package src:bash. (Fri, 11 Nov 2016 05:15:04 GMT) (full text, mbox, link).


Acknowledgement sent to Friedrich Mayrhofer <friedricmayrhofer6@gmail.com>:
Extra info received and forwarded to list. Copy sent to Matthias Klose <doko@debian.org>. (Fri, 11 Nov 2016 05:15:04 GMT) (full text, mbox, link).


Message #62 received at 806945@bugs.debian.org (full text, mbox, reply):

From: Friedrich Mayrhofer <dianacuadra@unach.cl>
To: undisclosed-recipients:;
Subject: Did You Get My Message This Time?
Date: Fri, 11 Nov 2016 01:59:12 -0300 (CLST)


This is the second time i am sending you this mail.I, Friedrich Mayrhofer Donate $ 1,000,000.00 to You, Email  Me personally for more details.

Regards.
Friedrich Mayrhofer



Information forwarded to debian-bugs-dist@lists.debian.org, Matthias Klose <doko@debian.org>:
Bug#806945; Package src:bash. (Sat, 12 Nov 2016 14:27:06 GMT) (full text, mbox, link).


Acknowledgement sent to friedricmayrhofer6@gmail.com:
Extra info received and forwarded to list. Copy sent to Matthias Klose <doko@debian.org>. (Sat, 12 Nov 2016 14:27:06 GMT) (full text, mbox, link).


Message #67 received at 806945@bugs.debian.org (full text, mbox, reply):

From: Friedrich Mayrhofer <emilio.porcu@usm.cl>
To: undisclosed-recipients:;
Subject: Did You Get My Message This Time?
Date: Sat, 12 Nov 2016 06:24:17 -0300
-- 
This is the second time i am sending you this mail.I, Friedrich  
Mayrhofer Donate $ 1,000,000.00 to You, Email  Me personally for more  
details.

Regards.
Friedrich Mayrhofer







Information forwarded to debian-bugs-dist@lists.debian.org, Matthias Klose <doko@debian.org>:
Bug#806945; Package src:bash. (Wed, 16 Nov 2016 06:09:03 GMT) (full text, mbox, link).


Acknowledgement sent to Friedrich Mayrhofer <friedricmayrhofer6@gmail.com>:
Extra info received and forwarded to list. Copy sent to Matthias Klose <doko@debian.org>. (Wed, 16 Nov 2016 06:09:03 GMT) (full text, mbox, link).


Message #72 received at 806945@bugs.debian.org (full text, mbox, reply):

From: Friedrich Mayrhofer <fernandopinto@unach.cl>
To: undisclosed-recipients:;
Subject: Did You Get My Message This Time?
Date: Wed, 16 Nov 2016 02:49:44 -0300 (CLST)

-- 
This is the second time i am sending you this mail.I, Friedrich Mayrhofer Donate $ 1,000,000.00 to You, Email  Me personally for more details.

Regards.
Friedrich Mayrhofer



Information forwarded to debian-bugs-dist@lists.debian.org, Matthias Klose <doko@debian.org>:
Bug#806945; Package src:bash. (Fri, 18 Nov 2016 07:09:02 GMT) (full text, mbox, link).


Acknowledgement sent to Friedrich Mayrhofer <friedricmayrhofer6@gmail.com>:
Extra info received and forwarded to list. Copy sent to Matthias Klose <doko@debian.org>. (Fri, 18 Nov 2016 07:09:03 GMT) (full text, mbox, link).


Message #77 received at 806945@bugs.debian.org (full text, mbox, reply):

From: Friedrich Mayrhofer <danielvera@unach.cl>
To: undisclosed-recipients:;
Subject: Did You Get My Message This Time?
Date: Fri, 18 Nov 2016 03:53:42 -0300 (CLST)

-- 

This is the second time i am sending you this mail.I, Friedrich Mayrhofer Donate $ 1,000,000.00 to You, Email  Me personally for more details.

Regards.
Friedrich Mayrhofer



Information forwarded to debian-bugs-dist@lists.debian.org, Matthias Klose <doko@debian.org>:
Bug#806945; Package src:bash. (Fri, 27 Jan 2017 12:20:02 GMT) (full text, mbox, link).


Acknowledgement sent to "USPS Ground Support" <jason.kidd@48timer.com>:
Extra info received and forwarded to list. Copy sent to Matthias Klose <doko@debian.org>. (Fri, 27 Jan 2017 12:20:02 GMT) (full text, mbox, link).


Message #82 received at 806945@bugs.debian.org (full text, mbox, reply):

From: "USPS Ground Support" <jason.kidd@48timer.com>
To: 806945@bugs.debian.org
Subject: Notification status of your delivery (USPS 03448983)
Date: Fri, 27 Jan 2017 11:51:18 +0000
[Message part 1 (text/plain, inline)]
Dear Customer,

Your parcel was successfully delivered January 26 to USPS Station, but our courier cound not contact you.

Postal label is enclosed to this e-mail. Please check the attachment!

Sincerely yours,
Jason Kidd,
USPS Senior Office Manager.

[Undelivered-Parcel-ID-03448983.zip (application/zip, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, Matthias Klose <doko@debian.org>:
Bug#806945; Package src:bash. (Thu, 09 Feb 2017 01:03:03 GMT) (full text, mbox, link).


Acknowledgement sent to da1u70hf@lhcp1053.webapps.net:
Extra info received and forwarded to list. Copy sent to Matthias Klose <doko@debian.org>. (Thu, 09 Feb 2017 01:03:03 GMT) (full text, mbox, link).


Message #87 received at 806945@bugs.debian.org (full text, mbox, reply):

From: da1u70hf@lhcp1053.webapps.net
To: 806945@bugs.debian.org
Subject: New notice to Appear in Court
Date: Thu, 9 Feb 2017 01:01:11 +0000
[Message part 1 (text/plain, inline)]
Dear Sir or Madam,

This is to inform you to appear in the Court on the February 14.
Please prepare all the documents relating to the case and bring them on the specified date.

Please review all the details in the attachment.

Sincerely yours,
 ,
Court Secretary.

[Notice_00921018.zip (application/zip, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, Matthias Klose <doko@debian.org>:
Bug#806945; Package src:bash. (Tue, 09 May 2017 21:15:03 GMT) (full text, mbox, link).


Acknowledgement sent to Ximin Luo <infinity0@debian.org>:
Extra info received and forwarded to list. Copy sent to Matthias Klose <doko@debian.org>. (Tue, 09 May 2017 21:15:03 GMT) (full text, mbox, link).


Message #92 received at 806945@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@debian.org>
To: Debian Bug Tracking System <806945@bugs.debian.org>, reproducible-bugs@lists.alioth.debian.org
Subject: Re: bash: Please make bash build reproducibly
Date: Tue, 09 May 2017 21:11:00 +0000
[Message part 1 (text/plain, inline)]
Ximin Luo:
> [..]
> 
> CFLAGS - we eventually want to patch gcc to fix this; bash doesn't need to
>   handle this specifically.
> BUILD_DIR - this does seem bash-specific, but I haven't yet studied the
>   Makefiles in enough detail to propose something concrete.
> PDF ID - this is caused by [1] and we will probably patch texlive-bin to fix
>   this; bash doesn't need to handle this specifically.
> 

Hi, here is an updated patch that fixes the BUILD_DIR issue. My patch to GCC (#862113) fixes the CFLAGS issue here, as expected. The only remaining issue is the PDF ID issue, which I'll work on in the near-to-mid future.

i.e. this patch is (hopefully) the complete set of changes that need to go into the bash package, for reproducibility purposes.

Together with the GCC patch and a future texlive package, these will make the bash package fully reproducible under all build environments.

X

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git
[bash_806945.patch (text/x-diff, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, Matthias Klose <doko@debian.org>:
Bug#806945; Package src:bash. (Fri, 04 Aug 2017 18:21:03 GMT) (full text, mbox, link).


Acknowledgement sent to Gaconnet Pierre <pgaconnet@seinesaintdenis.fr>:
Extra info received and forwarded to list. Copy sent to Matthias Klose <doko@debian.org>. (Fri, 04 Aug 2017 18:21:04 GMT) (full text, mbox, link).


Message #97 received at 806945@bugs.debian.org (full text, mbox, reply):

From: Gaconnet Pierre <pgaconnet@seinesaintdenis.fr>
Subject: Su buzón está lleno
Date: Fri, 4 Aug 2017 20:10:33 +0200 (CEST)
[Message part 1 (text/plain, inline)]
c 


Ce message et toutes les pièces jointes sont établis à l'intention exclusive de ses destinataires et sont confidentiels. Si vous recevez ce message par erreur, merci de le détruire et d'en avertir immédiatement l'expéditeur. Toute utilisation de ce message non conforme à sa destination, toute diffusion ou toute publication, totale ou partielle, est interdite sauf autorisation expresse.
L'internet ne permettant pas d'assurer l'intégrité de ce message, le Conseil Départemental de Seine-Saint-Denis décline toute responsabilité au titre de ce message, dans l'hypothèse où il aurait été modifié. D'autre part, le Conseil Départemental de Seine-Saint-Denis ne reconnait exclusivement que les délégations de signatures écrites par les personnes habilitées et ne peut donc être engagé par un message électronique
[Message part 2 (text/html, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Matthias Klose <doko@debian.org>:
Bug#806945; Package src:bash. (Fri, 04 Aug 2017 18:45:02 GMT) (full text, mbox, link).


Acknowledgement sent to delegadasmujer@buenosaires.gob.ar:
Extra info received and forwarded to list. Copy sent to Matthias Klose <doko@debian.org>. (Fri, 04 Aug 2017 18:45:02 GMT) (full text, mbox, link).


Message #102 received at 806945@bugs.debian.org (full text, mbox, reply):

From: delegadasmujer@buenosaires.gob.ar
To: undisclosed-recipients:;
Subject: Su cuota de webmail ha excedido la cuota
Date: Fri, 04 Aug 2017 15:29:22 -0300
Su cuota de webmail ha excedido la cuota, que es de 2 GB. Actualmente  
se sitúa en 2,3 GB.
Para reactivar y aumentar su cuota de webmail, haga clic en el  
siguiente enlace o copie el enlace y actualizar su cuenta webmail
Para activar.


https://www.supersimplesurvey.com/Survey/18441/admin-page/


Si no, puede resultar en la terminación de su cuenta de webmail.
Gracias y Disculpen las molestias
Admin/Webmaster/localhost





Reply sent to Matthias Klose <doko@debian.org>:
You have taken responsibility. (Tue, 06 Feb 2018 12:30:17 GMT) (full text, mbox, link).


Notification sent to Maria Valentina Marin <marivalenm@gmail.com>:
Bug acknowledged by developer. (Tue, 06 Feb 2018 12:30:17 GMT) (full text, mbox, link).


Message #107 received at 806945-close@bugs.debian.org (full text, mbox, reply):

From: Matthias Klose <doko@debian.org>
To: 806945-close@bugs.debian.org
Subject: Bug#806945: fixed in bash 4.4.18-1
Date: Tue, 06 Feb 2018 12:20:33 +0000
Source: bash
Source-Version: 4.4.18-1

We believe that the bug you reported is fixed in the latest version of
bash, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 806945@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Matthias Klose <doko@debian.org> (supplier of updated bash package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Tue, 06 Feb 2018 12:20:45 +0100
Source: bash
Binary: bash bash-static bash-builtins bash-doc
Architecture: source
Version: 4.4.18-1
Distribution: unstable
Urgency: medium
Maintainer: Matthias Klose <doko@debian.org>
Changed-By: Matthias Klose <doko@debian.org>
Description:
 bash       - GNU Bourne Again SHell
 bash-builtins - Bash loadable builtins - headers & examples
 bash-doc   - Documentation and examples for the GNU Bourne Again SHell
 bash-static - GNU Bourne Again SHell (static version)
Closes: 537913 789811 806945 831282 839155 859263 865599 882474
Changes:
 bash (4.4.18-1) unstable; urgency=medium
 .
   * bash 4.4.18 release (bash 4.4 patchlevel 18).
   * bash: Remove dependency on dash. Closes: #537913.
   * Update config.guess and config.sub. Closes: #882474.
   * skel.profile: Add $HOME/.local/bin if it exists. Closes: #839155.
   * Stop building with -no-pie. Closes: #865599, #859263.
   * /etc/bash.bashrc: Don't overwrite PS1 if SUDO_PS1 is set. Closes: #789811.
   * Make the build reproducible. Closes: #806945.
     - Use the system provided man2html to generate the htm docs.
     - Set PGRP_PIPE unconditionally on Linux.
   * Fix typo in German help (Carsten Leonhardt). Closes: #831282.
Checksums-Sha1:
 60397be2e36a634ca237dda93630618954d9f0ae 2321 bash_4.4.18-1.dsc
 033efd338b9d3958c75bf4fd3887ed362e2f9026 5036272 bash_4.4.18.orig.tar.xz
 ef08496dab4ec743b72add8a589414a7e1b51aa5 57652 bash_4.4.18-1.debian.tar.xz
 ddec9503ff61be44c63121922afd88240196efed 6708 bash_4.4.18-1_source.buildinfo
Checksums-Sha256:
 e0b2cb3d3e07a8788ee6ebd807ac93761a433adc4b3eb40202c28bcc10fe5982 2321 bash_4.4.18-1.dsc
 704143a7170041ac9f1025455d6d23ff0f353711a3dc557b47d6e6322f24cd02 5036272 bash_4.4.18.orig.tar.xz
 19a64bdd13e036901a84c2452b897b375c2f0237e8e56d312b485e0095fad0f4 57652 bash_4.4.18-1.debian.tar.xz
 5a5b8b34e85da38ea0cd96578b1caa1f200752218ad4239fdca1c12711f953d6 6708 bash_4.4.18-1_source.buildinfo
Files:
 20254e40d474485b498e58fefecb08ce 2321 base required bash_4.4.18-1.dsc
 e3f57de774124c7b01a63a408cd49e2e 5036272 base required bash_4.4.18.orig.tar.xz
 3fcca5e2013a13ec4a3561e0c4657bfb 57652 base required bash_4.4.18-1.debian.tar.xz
 b38c7207da0a09b91e94e599103c80a4 6708 base required bash_4.4.18-1_source.buildinfo

-----BEGIN PGP SIGNATURE-----

iQJEBAEBCAAuFiEE1WVxuIqLuvFAv2PWvX6qYHePpvUFAlp5mpIQHGRva29AZGVi
aWFuLm9yZwAKCRC9fqpgd4+m9RRaD/4ncFWiyydvZADQ7ChyZoNKpUPxSqJiUhUn
wRHq6LAB7TFPYImtG/D2Z2yS0F7oWvJubv/Ng3ifexVzXw/qyO/iqBV4hQ2pnABp
THA48U9TB1bIFSRt9GWXZJhs65GBrW5C0rGiPUfxUNhqdOVRLIfsJATSnQdh6V2w
5wJc13cV6L6rS5wsJslg084wA/HJBGzDiCbr2eONJwn+bkv7e7KrJN0ivjHykMyQ
gUlaWjX++c2y45KGCobVnNKLrmvLwhRhiCKhXMCnyzd4KoR5+YCwMZrcxhWFFRPO
cJypKHe5OaSFk9R7y9W2djFWm5ASRLFKBnzK0N4bXU9kuIpPSRMRSrIytK3a12QH
kGcUs4CDrSg0mm9UrL+I2rhwjgB1HX0PyPqYQAf8TKRqLqZx+zkkbCQsYzHAvoHN
SNLqjzrhRBYpxH5e38oEsMcOqIMGO5H1+e9UOpVdPwHhHOgA1lLjD/x/i3LB1lmC
60I79q+SY4x3PbDW/kmDTPiZ9JryqVUAfPWdjkb0YEVWUrPfH9i8Mn7CVhuXtGve
vxQc6VFCUktIuJzzn73Ey+ZsXH+vz5yor+Mnww62BhltJzmpJC5sX4ju8waSD2p0
K6jCPYM1lqEOrwwGRsPK7sz9TNGiR4n/a6JQ9ze3yGVfmX1StwvZyDzON299Q0Oq
lENse2Y4qQ==
=8wP2
-----END PGP SIGNATURE-----




Bug archived. Request was from Debbugs Internal Request <owner@bugs.debian.org> to internal_control@bugs.debian.org. (Thu, 15 Mar 2018 07:27:48 GMT) (full text, mbox, link).


Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Wed Dec 6 06:59:33 2023; Machine Name: buxtehude

Debian Bug tracking system

Debbugs is free software and licensed under the terms of the GNU Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.

Copyright © 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson, 2005-2017 Don Armstrong, and many other contributors.