Debian Bug report logs - #877470
bsh; please make the build reproducible (timestamps)

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: jathan <jathanblackred@openmailbox.org>

To: Debian BTS <submit@bugs.debian.org>

Subject: bsh; please make the build reproducible (timestamps)

Date: Sun, 1 Oct 2017 22:21:06 -0500

[Message part 1 (text/plain, inline)]

Source: bsh
Version: 2.0b4-18
Severity: wishlist
Tags: patch
User: reproducible-builds@lists.alioth.debian.org
Usertags: timestamps
X-Debbugs-Cc: reproducible-bugs@lists.alioth.debian.org

Hi!

While working on the “reproducible builds” effort [1], we have noticed
that bsh could not be built reproducibly.

The attached patch clamps the timestamps to the changelog timestamp when
creating the source archive. Once applied, bsh can be built reproducibly
in our current experimental framework. Best regards.

Jathan

 [1]: https://wiki.debian.org/ReproducibleBuilds

-- 
Por favor evita enviarme adjuntos en formato de word o powerpoint, si
quieres saber porque lee esto:
http://www.gnu.org/philosophy/no-word-attachments.es.html
¡Cámbiate a GNU/Linux! http://getgnulinux.org/es

[bsh_2.0b4-18_2.0b4-18.1.debdiff (text/plain, attachment)]

[signature.asc (application/pgp-signature, attachment)]

Message #10 received at 877470@bugs.debian.org (full text, mbox, reply):

From: Emmanuel Bourg <ebourg@apache.org>

To: jathan <jathanblackred@openmailbox.org>, 877470@bugs.debian.org

Subject: Re: Bug#877470: bsh; please make the build reproducible (timestamps)

Date: Mon, 2 Oct 2017 17:11:48 +0200

Thank you for the patch Jathan.

Aren't tar timestamps already normalized by strip-nondeterminism though?

Emmanuel Bourg

Message #15 received at 877470@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>

To: 877470@bugs.debian.org

Cc: jathan <jathanblackred@openmailbox.org>

Subject: Re: bsh; please make the build reproducible (timestamps)

Date: Wed, 04 Oct 2017 07:27:08 +0100

Hi,

> bsh; please make the build reproducible (timestamps)

+SOURCE_DATE_EPOCH       ?= $(shell dpkg-parsechangelog -STimestamp)

I'd actually use /usr/share/dpkg/pkg-info.mk for this :)

+	  --clamp-mtime --mtime="$(SOURCE_DATE_EPOCH)"

Shouldn't this be --mtime="@$(SOURCE_DATE_EPOCH)" (nb. with the @?)


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

Message #20 received at 877470@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>

To: jathan <jathanblackred@openmailbox.org>, reproducible-builds@lists.alioth.debian.org

Cc: 877470@bugs.debian.org

Subject: Re: Question about strip-nondeterminism in bsh

Date: Mon, 09 Oct 2017 16:47:55 +0100

[Adding 877470@bugs.debian.org to CC]

Hi jathan,

> Also I want to ask you what does it mean you actually use
> /usr/share/dpkg/pkg-info.mk for SOURCE_DATE_EPOCH

See, for example:

  https://github.com/lamby/pkg-python-daiquiri/blob/debian/sid/debian/rules#L3

If you take a look at the /usr/share/dpkg/pkg-info.mk file itself,
it should be fairly clear what it exports to the outside environment.

In the python-daiquiri example, I'm using DEB_VERSION_UPSTREAM for a
reproducibility-related reason, but you should be able to see how it
applies to SOURCE_DATE_EPOCH too :)

> Thanks a lot and sorry for my delay to reply,

No worries about the delay. Note that I've added 877470@bugs.debian.org to
the CC; generally when a question pertains to a specific bug, it's a great
idea to ensure that the conversation is archived there. Keeps everyone sane
too as it's easy to "load" context after a few days/weeks. :)


Best wishes,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

Message #25 received at 877470@bugs.debian.org (full text, mbox, reply):

From: jathan <jathanblackred@openmailbox.org>

To: reproducible-builds@lists.alioth.debian.org, 877470@bugs.debian.org

Subject: Re: Question about strip-nondeterminism in bsh

Date: Mon, 16 Oct 2017 13:56:53 -0500

[Message part 1 (text/plain, inline)]

On 09/10/17 10:47, Chris Lamb wrote:
> [Adding 877470@bugs.debian.org to CC]
> 
> Hi jathan,
> 
>> Also I want to ask you what does it mean you actually use
>> /usr/share/dpkg/pkg-info.mk for SOURCE_DATE_EPOCH
> 
> See, for example:
> 
>   https://github.com/lamby/pkg-python-daiquiri/blob/debian/sid/debian/rules#L3
> 
> If you take a look at the /usr/share/dpkg/pkg-info.mk file itself,
> it should be fairly clear what it exports to the outside environment.
> 
> In the python-daiquiri example, I'm using DEB_VERSION_UPSTREAM for a
> reproducibility-related reason, but you should be able to see how it
> applies to SOURCE_DATE_EPOCH too :)
> 
>> Thanks a lot and sorry for my delay to reply,
> 
> No worries about the delay. Note that I've added 877470@bugs.debian.org to
> the CC; generally when a question pertains to a specific bug, it's a great
> idea to ensure that the conversation is archived there. Keeps everyone sane
> too as it's easy to "load" context after a few days/weeks. :)
> 
> 
> Best wishes,
> 
Hi Lamby!,

Thanks a lot for sharing your pkg-python-daiquiri example. I have
executed "less /usr/share/dpkg/pkg-info.mk" on my local machine with
Debian Stretch and I have the next content:

# Makefile snippet defining the following variables:
#
# DEB_SOURCE: the source package name
# DEB_VERSION: the full version of the package (epoch + upstream vers. +
revision)
# DEB_VERSION_EPOCH_UPSTREAM: the package's version without the Debian
revision
# DEB_VERSION_UPSTREAM_REVISION: the package's version without the
Debian epoch
# DEB_VERSION_UPSTREAM: the package's upstream version
# DEB_DISTRIBUTION: the distribution(s) listed in the current entry of
debian/changelog
#
# SOURCE_DATE_EPOCH: the source release date as seconds since the epoch, as
#   specified by <https://reproducible-builds.org/specs/source-date-epoch/>

dpkg_late_eval ?= $(or $(value DPKG_CACHE_$(1)),$(eval DPKG_CACHE_$(1)
:= $(shell $(2)))$(value DPKG_CACHE_$(1)))

DEB_SOURCE = $(call dpkg_late_eval,DEB_SOURCE,dpkg-parsechangelog -SSource)
DEB_VERSION = $(call dpkg_late_eval,DEB_VERSION,dpkg-parsechangelog
-SVersion)
DEB_VERSION_EPOCH_UPSTREAM = $(call
dpkg_late_eval,DEB_VERSION_EPOCH_UPSTREAM,echo '$(DEB_VERSION)' | sed -e
's/-[^-]*$$//')
DEB_VERSION_UPSTREAM_REVISION = $(call
dpkg_late_eval,DEB_VERSION_UPSTREAM_REVISION,echo '$(DEB_VERSION)' | sed
-e 's/^[0-9]*://')
DEB_VERSION_UPSTREAM = $(call dpkg_late_eval,DEB_VERSION_UPSTREAM,echo
'$(DEB_VERSION_EPOCH_UPSTREAM)' | sed -e 's/^[0-9]*://')
DEB_DISTRIBUTION = $(call
dpkg_late_eval,DEB_DISTRIBUTION,dpkg-parsechangelog -SDistribution)

SOURCE_DATE_EPOCH ?= $(call
dpkg_late_eval,SOURCE_DATE_EPOCH,dpkg-parsechangelog -STimestamp)

export SOURCE_DATE_EPOCH



How can I see the /usr/share/dpkg/pkg-info.mk you have used for
pkg-python-daiquiri to make a comparison of contents and understand
deeper how you did it please? Thinking about these two lines of your code:

include /usr/share/dpkg/pkg-info.mk

export PBR_VERSION = $(DEB_VERSION_UPSTREAM)


And also considering the /usr/share/dpkg/pkg-info.mk file suggestion, I
would apply SOURCE_DATE_EPOCH in the next way:

include /usr/share/dpkg/pkg-info.mk

export SOURCE_DATE_EPOCH = $(call
dpkg_late_eval,SOURCE_DATE_EPOCH,dpkg-parsechangelog -STimestamp)
or
export SOURCE_DATE_EPOCH = $(shell dpkg-parsechangelog -STimestamp)
(based on
https://wiki.debian.org/ReproducibleBuilds/TimestampsProposal#Examples)

Would it be right for me to use either of these two options or am I
wrong in both? Best regards.

Jathan

-- 
Por favor evita enviarme adjuntos en formato de word o powerpoint, si
quieres saber porque lee esto:
http://www.gnu.org/philosophy/no-word-attachments.es.html
¡Cámbiate a GNU/Linux! http://getgnulinux.org/es

[signature.asc (application/pgp-signature, attachment)]

Message #30 received at 877470@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>

To: reproducible-builds@lists.alioth.debian.org, 877470@bugs.debian.org

Subject: Re: Question about strip-nondeterminism in bsh

Date: Mon, 16 Oct 2017 20:41:37 +0100

Jathan,

> include /usr/share/dpkg/pkg-info.mk
> 
> export SOURCE_DATE_EPOCH = $(call
> dpkg_late_eval,SOURCE_DATE_EPOCH,dpkg-parsechangelog -STimestamp)
> or
> export SOURCE_DATE_EPOCH = $(shell dpkg-parsechangelog -STimestamp)

I fear you are confused here — if you include include
/usr/share/dpkg/pkg-info.mk, SOURCE_DATE_EPOCH will be exported for
you. Indeed, that's the entire point of using this include :)

> How can I see the /usr/share/dpkg/pkg-info.mk you have used for
> pkg-python-daiquiri to make a comparison of contents

Hm? I don't understand why you would need to see mine? It's
almost certainly the same…


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

Message #35 received at 877470@bugs.debian.org (full text, mbox, reply):

From: jathan <jathanblackred@openmailbox.org>

To: 877470@bugs.debian.org

Subject: Re: Bug#877470: bsh; please make the build reproducible (timestamps)

Date: Tue, 17 Oct 2017 13:34:57 -0500

[Message part 1 (text/plain, inline)]

On Mon, 2 Oct 2017 17:11:48 +0200 Emmanuel Bourg <ebourg@apache.org> wrote:
> Thank you for the patch Jathan.
> 
> Aren't tar timestamps already normalized by strip-nondeterminism though?
> 
> Emmanuel Bourg
> 
> 
Hi Emmanuel,

Tar timestamps are not normalized by strip-nondeterminism. Best regards.

Jathan

-- 
Por favor evita enviarme adjuntos en formato de word o powerpoint, si
quieres saber porque lee esto:
http://www.gnu.org/philosophy/no-word-attachments.es.html
¡Cámbiate a GNU/Linux! http://getgnulinux.org/es

[signature.asc (application/pgp-signature, attachment)]

Message #40 received at 877470@bugs.debian.org (full text, mbox, reply):

From: jathan <jathanblackred@openmailbox.org>

To: 877470@bugs.debian.org

Subject: Re: bsh; please make the build reproducible (timestamps)

Date: Wed, 25 Oct 2017 00:10:04 -0500

[Message part 1 (text/plain, inline)]

On Wed, 04 Oct 2017 07:27:08 +0100 Chris Lamb <lamby@debian.org> wrote:
> Hi,
> 
> > bsh; please make the build reproducible (timestamps)
> 
> +SOURCE_DATE_EPOCH       ?= $(shell dpkg-parsechangelog -STimestamp)
> 
> I'd actually use /usr/share/dpkg/pkg-info.mk for this :)
> 
> +	  --clamp-mtime --mtime="$(SOURCE_DATE_EPOCH)"
> 
> Shouldn't this be --mtime="@$(SOURCE_DATE_EPOCH)" (nb. with the @?)
> 
> 
> Regards,
> 
> -- 
>       ,''`.
>      : :'  :     Chris Lamb
>      `. `'`      lamby@debian.org / chris-lamb.co.uk
>        `-
> 
> Hi,

I have rewrited the bsh patch based on the observations and suggestions
of Lamby, sending a new bsh_2.0b4-18_2.0b4-18.1.debdiff file. The
attached patch clamps the timestamps to the changelog timestamp when
creating the source archive using SOURCE_DATE_EPOCH variable. Once
applied, bsh can be built reproducibly in our current experimental
framework. Cheers.

Jathan

-- 
Por favor evita enviarme adjuntos en formato de word o powerpoint, si
quieres saber porque lee esto:
http://www.gnu.org/philosophy/no-word-attachments.es.html
¡Cámbiate a GNU/Linux! http://getgnulinux.org/es

[bsh_2.0b4-18_2.0b4-18.1.debdiff (text/plain, attachment)]

[signature.asc (application/pgp-signature, attachment)]

Message #43 received at 877470@bugs.debian.org (full text, mbox, reply):

From: pkg-java-maintainers@lists.alioth.debian.org

To: 877470@bugs.debian.org, 877470-submitter@bugs.debian.org

Subject: Pending fixes for bugs in the bsh package

Date: Wed, 29 Nov 2017 23:53:03 +0000

tag 877470 + pending
thanks

Some bugs in the bsh package are closed in revision
eda9f8fad7f4986309b79ebf6f34d50ca7fdab95 in branch 'master' by
Emmanuel Bourg

The full diff can be seen at
https://anonscm.debian.org/cgit/pkg-java/bsh.git/commit/?id=eda9f8f

Commit message:

    Applied Jonathan Bustillos' patch to make the build reproducible (Closes: #877470)

Message #53 received at 877470-close@bugs.debian.org (full text, mbox, reply):

From: Emmanuel Bourg <ebourg@apache.org>

To: 877470-close@bugs.debian.org

Subject: Bug#877470: fixed in bsh 2.0b4-19

Date: Thu, 30 Nov 2017 00:03:48 +0000

Source: bsh
Source-Version: 2.0b4-19

We believe that the bug you reported is fixed in the latest version of
bsh, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 877470@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Emmanuel Bourg <ebourg@apache.org> (supplier of updated bsh package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.8
Date: Thu, 30 Nov 2017 00:39:30 +0100
Source: bsh
Binary: bsh libbsh-java bsh-doc bsh-src
Architecture: source
Version: 2.0b4-19
Distribution: unstable
Urgency: medium
Maintainer: Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>
Changed-By: Emmanuel Bourg <ebourg@apache.org>
Description:
 bsh        - Java scripting environment (BeanShell) Version 2
 bsh-doc    - Documentation for bsh
 bsh-src    - Java scripting environment (BeanShell) Version 2 (source code)
 libbsh-java - Java scripting environment (BeanShell) Version 2 (library)
Closes: 875340 877470
Changes:
 bsh (2.0b4-19) unstable; urgency=medium
 .
   * Fixed the build failure with Java 9 (Closes: #875340)
   * Applied Jonathan Bustillos' patch to make the build reproducible
     (Closes: #877470)
   * Build with the DH sequencer instead of CDBS
   * Moved the package to Git
   * Standards-Version updated to 4.1.1
   * Switch to debhelper level 10
Checksums-Sha1:
 a8f8fd91f72af057371131ca8508565a0a5489f3 2168 bsh_2.0b4-19.dsc
 3426298335919ec04359657d287ecce58f6dd59e 9636 bsh_2.0b4-19.debian.tar.xz
 1856a1573f2231d8f1b24e305bab4fc43b70c319 11616 bsh_2.0b4-19_source.buildinfo
Checksums-Sha256:
 704bc2d34e06efd788f6959b00c507eb9e306a24c2c5e6ae7a57ff22046caa39 2168 bsh_2.0b4-19.dsc
 d373b7ed5d54d1572378f4b7b361791d06abad6d8233e14abe7ad9b0c9568022 9636 bsh_2.0b4-19.debian.tar.xz
 e52e78dd3444bc12264407bec7695f8bb01453b424bbd81e95f5fa7df82ffc34 11616 bsh_2.0b4-19_source.buildinfo
Files:
 042e464727b30bd615e356db49b560dd 2168 devel optional bsh_2.0b4-19.dsc
 4cac52e0817cf3cce5a22f98a2c9008c 9636 devel optional bsh_2.0b4-19.debian.tar.xz
 c55ab653a25c9ef124c9abd85194586a 11616 devel optional bsh_2.0b4-19_source.buildinfo

-----BEGIN PGP SIGNATURE-----

iQJGBAEBCgAwFiEEuM5N4hCA3PkD4WxA9RPEGeS50KwFAlofSFYSHGVib3VyZ0Bh
cGFjaGUub3JnAAoJEPUTxBnkudCse8QP/1z0vK6zO1Fm3xhYGfHgbgi3fNRubkVz
gC+PRz44IeGPggot2Ka78IjrEPKeQ3yHiYlR1x/ca/YPM0YQC5fmczvERXcoFI+o
l6dH+NEnlIbm5UQtEK1Vtv823xNHpORIxCsHTvdFrlFAsN84ZT2duA35YUdz9Rel
JKFYPYwP906uE8NZM6FOqU600YNEB5GJqbSZLkrp9QGJ9PflLfAqoeQRkRzSPg6t
HiLnaTHurwVYmak3aydj2/ls9yZbBhDoVjLiZdIu0cFr8vq8tQFtmW/Cp2sOdk30
3zkUdfvNBLdTfE3rB3GifviOZa8ZEqLwqnMDToievsrx4NH9IHQExsOjVk/9GYJS
mIGeHxkz3HzhCm/Hd4PFeKzHdBLuNbhCPobpCYV/N2/DV0s4XtsWd0qeburqs+Ul
NgmKXb4HTBp1wmT221H2YwdbAzHbfcU2xLbm0gKxiLo/JEJjVHh9oPh5Y0m73hpw
MjXU86m8ZwPXul+Zji6jE2he2qhZLU6pe807J//smGk4SOBp3vIa+5rJpHdhN0k+
ZvO8dAf7RbrvYuhPqcerMBAEEdRA5qhqEHu8lEyLLNfyV4GRsMGndbU+9nktMJLy
Y/uhGvoyPoGoRMRXoH0Kylf0jnOv9uDKzBRs9Rc4fyxb31wKfD1fnk6bQBfrQ9Jv
SLup7rG3ZsVl
=r5rx
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.