Debian Bug report logs - #929793
liblopsub: please make the build reproducible

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: "Chris Lamb" <lamby@debian.org>

To: submit@bugs.debian.org

Subject: liblopsub: please make the build reproducible

Date: Fri, 31 May 2019 09:44:44 +0100

[Message part 1 (text/plain, inline)]

Source: liblopsub
Version: 1.0.2-1
Severity: wishlist
Tags: patch
User: reproducible-builds@lists.alioth.debian.org
Usertags: timestamps
X-Debbugs-Cc: reproducible-bugs@lists.alioth.debian.org

Hi,

Whilst working on the Reproducible Builds effort [0], we noticed
that liblopsub could not be built reproducibly.

Patch attached.

  [0] https://reproducible-builds.org/


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

[liblopsub.diff.txt (text/plain, attachment)]

Message #10 received at submit@bugs.debian.org (full text, mbox, reply):

From: Andre Noll <maan@tuebingen.mpg.de>

To: Chris Lamb <lamby@debian.org>, 929793@bugs.debian.org

Cc: submit@bugs.debian.org

Subject: Re: Bug#929793: liblopsub: please make the build reproducible

Date: Fri, 31 May 2019 16:45:56 +0200

[Message part 1 (text/plain, inline)]

On Fri, May 31, 09:44, Chris Lamb wrote
> Whilst working on the Reproducible Builds effort [0], we noticed
> that liblopsub could not be built reproducibly.

No general objection from my side.  However,

> -DATE := $(shell date '+%B %Y')
> +DATE_FMT = '+%B %Y'
> +ifdef SOURCE_DATE_EPOCH
> +	DATE := $(shell LC_ALL=C date -u -d "@$(SOURCE_DATE_EPOCH)" "+$(DATE_FMT)" 2>/dev/null || LC_ALL=C date -u -r "$(SOURCE_DATE_EPOCH)" "+$(DATE_FMT)" 2>/dev/null || date LC_ALL=C -u "+$(DATE_FMT)")
> +else
> +	DATE := $(shell date "+$(DATE_FMT)")
> +endif

This does not work if SOURCE_DATE_EPOCH is set because the unnamed
argument has with two '+' characters (one from DATE_FMT and one
from the command line). Also DATE_FMT should be a singly expanded
make variable. Next, the quotes in DATE_FMT will be copied into the
command line, resulting in a syntax error. Finally, "date" and "LC_ALL=C"
in the third command of the first alternative are in the wrong order.

Also, it would be nice to get rid of the overlong line.  The patch
below fixes the syntax errors and is easier to read IMO.

Care to provide a commit message which explains why we try to
pass $(SOURCE_DATE_EPOCH) as an argument to both -d and -r, who
sets SOURCE_DATE_EPOCH, and what type its value is supposed to be
(filename or number of seconds)?

Thanks
Andre
---
diff --git a/Makefile b/Makefile
index 408e3a5..b72db24 100644
--- a/Makefile
+++ b/Makefile
@@ -21,7 +21,18 @@ INSTALL := install
 GZIP := gzip -f9
 ZCAT := zcat
 
-DATE := $(shell date '+%B %Y')
+DATE_FMT := +%B %Y
+ifdef SOURCE_DATE_EPOCH
+	DATE := $(shell \
+		LC_ALL=C date -u -d '@$(SOURCE_DATE_EPOCH)' '$(DATE_FMT)' \
+			2>/dev/null || \
+		LC_ALL=C date -u -r '$(SOURCE_DATE_EPOCH)' '$(DATE_FMT)' \
+			2>/dev/null || \
+		LC_ALL=C date -u '$(DATE_FMT)' \
+	)
+else
+	DATE := $(shell date "+$(DATE_FMT)")
+endif
 GIT_VERSION := $(shell ./version-gen.sh)
 PLAIN_VERSION := $(firstword $(subst -, , $(GIT_VERSION)))
 MAJOR_VERSION := $(firstword $(subst ., , $(PLAIN_VERSION)))

-- 
Max Planck Institute for Developmental Biology
Max-Planck-Ring 5, 72076 Tübingen, Germany. Phone: (+49) 7071 601 829
http://people.tuebingen.mpg.de/maan/

[signature.asc (application/pgp-signature, inline)]

Message #20 received at 929793@bugs.debian.org (full text, mbox, reply):

From: "Chris Lamb" <lamby@debian.org>

To: "Andre Noll" <maan@tuebingen.mpg.de>, 929793@bugs.debian.org

Subject: Re: Bug#929793: liblopsub: please make the build reproducible

Date: Fri, 31 May 2019 17:05:48 +0100

Hi Andr,

> This does not work if SOURCE_DATE_EPOCH is set because the unnamed
> argument has with two '+' characters (one from DATE_FMT and one
> from the command line).

Mea culpa; I must have copy-pasted too hastily from my terminal to my
email editor (also causing the quotes and LC_ALL confusion) from a
previous iteration of my patch.

> Also DATE_FMT should be a singly expanded make variable.

(Singly, as in ":=" vs "=" ?)

> Care to provide a commit message which explains why we try to
> pass $(SOURCE_DATE_EPOCH) as an argument to both -d and -r, who
> sets SOURCE_DATE_EPOCH, and what type its value is supposed to be
> (filename or number of seconds)?

In terms of a its value, probably best to canonically link to:

  https://reproducible-builds.org/specs/source-date-epoch/

… although as a spoiler, it is a UNIX timestamp. :) Thanks for 
your review.


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org 🍥 chris-lamb.co.uk
       `-

Message #25 received at 929793@bugs.debian.org (full text, mbox, reply):

From: Andre Noll <maan@tuebingen.mpg.de>

To: Chris Lamb <lamby@debian.org>

Cc: 929793@bugs.debian.org

Subject: Re: Bug#929793: liblopsub: please make the build reproducible

Date: Fri, 31 May 2019 20:20:26 +0200

[Message part 1 (text/plain, inline)]

On Fri, May 31, 17:05, Chris Lamb wrote

> > Also DATE_FMT should be a singly expanded make variable.
> 
> (Singly, as in ":=" vs "=" ?)

Yes.

> > Care to provide a commit message which explains why we try to
> > pass $(SOURCE_DATE_EPOCH) as an argument to both -d and -r, who
> > sets SOURCE_DATE_EPOCH, and what type its value is supposed to be
> > (filename or number of seconds)?
> 
> In terms of a its value, probably best to canonically link to:
> 
>   https://reproducible-builds.org/specs/source-date-epoch/
> 
> … although as a spoiler, it is a UNIX timestamp. :) Thanks for 
> your review.

Then I don't grok the purpose of the

	LC_ALL=C date -u -r '$(SOURCE_DATE_EPOCH)' '$(DATE_FMT)'

command. After all, at least GNU date(1) expects a filename as the
argument to -r.

So how about the simpler patch below?

Best
Andre
---
commit 439a6e427979dbe146319d15411c31f6ebe1f962
Author: Chris Lamb <lamby@debian.org>
Date:   Fri May 31 19:54:04 2019 +0200

    Don't embed compile-time timestamps into generated files.
    
    Currently the build is not reproducible because make(1) runs
    date(1) to provide the month and the year for the man page. Fix
    this by honouring SOURCE_DATE_EPOCH as described in
    
            https://reproducible-builds.org/specs/source-date-epoch/

diff --git a/Makefile b/Makefile
index 408e3a5..3c7bde2 100644
--- a/Makefile
+++ b/Makefile
@@ -21,7 +21,15 @@ INSTALL := install
 GZIP := gzip -f9
 ZCAT := zcat
 
-DATE := $(shell date '+%B %Y')
+DATE_FMT := +%B %Y
+# To get a reproducible build, we use $(SOURCE_DATE_EPOCH) instead of the
+# current time if this variable is set.
+ifdef SOURCE_DATE_EPOCH
+	DATE := $(shell LC_ALL=C date -u -d '@$(SOURCE_DATE_EPOCH)' \
+		'$(DATE_FMT)' 2>/dev/null || LC_ALL=C date -u '$(DATE_FMT)')
+else
+	DATE := $(shell date '+$(DATE_FMT)')
+endif
 GIT_VERSION := $(shell ./version-gen.sh)
 PLAIN_VERSION := $(firstword $(subst -, , $(GIT_VERSION)))
 MAJOR_VERSION := $(firstword $(subst ., , $(PLAIN_VERSION)))
-- 
Max Planck Institute for Developmental Biology
Max-Planck-Ring 5, 72076 Tübingen, Germany. Phone: (+49) 7071 601 829
http://people.tuebingen.mpg.de/maan/

[signature.asc (application/pgp-signature, inline)]

Message #30 received at 929793@bugs.debian.org (full text, mbox, reply):

From: "Chris Lamb" <lamby@debian.org>

To: "Andre Noll" <maan@tuebingen.mpg.de>

Cc: 929793@bugs.debian.org

Subject: Re: Bug#929793: liblopsub: please make the build reproducible

Date: Tue, 04 Jun 2019 18:16:37 +0100

Andre Noll wrote:

> Then I don't grok the purpose of the
> 
> 	LC_ALL=C date -u -r '$(SOURCE_DATE_EPOCH)' '$(DATE_FMT)'
> 
> command. After all, at least GNU date(1) expects a filename as the
> argument to -r.

I do not have access to such a system but I believe this is for BSD
variants of date(1), hence the waterfall of ||'s.

Your patch looks great, thanks. :)


Best wishes,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org 🍥 chris-lamb.co.uk
       `-

Message #35 received at 929793@bugs.debian.org (full text, mbox, reply):

From: Andre Noll <maan@tuebingen.mpg.de>

To: Chris Lamb <lamby@debian.org>

Cc: 929793@bugs.debian.org

Subject: Re: Bug#929793: liblopsub: please make the build reproducible

Date: Wed, 5 Jun 2019 14:27:09 +0200

[Message part 1 (text/plain, inline)]

On Tue, Jun 04, 18:16, Chris Lamb wrote
> Andre Noll wrote:
> 
> > Then I don't grok the purpose of the
> > 
> > 	LC_ALL=C date -u -r '$(SOURCE_DATE_EPOCH)' '$(DATE_FMT)'
> > 
> > command. After all, at least GNU date(1) expects a filename as the
> > argument to -r.
> 
> I do not have access to such a system but I believe this is for BSD
> variants of date(1), hence the waterfall of ||'s.

Indeed: FreeBSD-12's date(1) has

	-r seconds
	     Print the date and time represented by seconds, where seconds is
	     the number of seconds since the Epoch (00:00:00 UTC, January 1,
	     1970; see time(3)), and can be specified in decimal, octal, or
	     hex.

and NetBSD-8 also has -r which expects a "seconds" argument.

I guess this is not relevant to the case at hand since all Debian
flavours use GNU userland, so we can rely GNU date(1). Unless you
have strong feelings about it I'll apply the patch as is, i.e. without
the above alternative for BSD compatibility.

Best
Andre
-- 
Max Planck Institute for Developmental Biology
Max-Planck-Ring 5, 72076 Tübingen, Germany. Phone: (+49) 7071 601 829
http://people.tuebingen.mpg.de/maan/

[signature.asc (application/pgp-signature, inline)]

Message #40 received at 929793@bugs.debian.org (full text, mbox, reply):

From: "Chris Lamb" <lamby@debian.org>

To: "Andre Noll" <maan@tuebingen.mpg.de>

Cc: 929793@bugs.debian.org

Subject: Re: Bug#929793: liblopsub: please make the build reproducible

Date: Wed, 05 Jun 2019 15:04:45 +0100

Hi Andre,

> Indeed: FreeBSD-12's date(1) has
> 
> 	-r seconds
> 	     Print the date and time represented by seconds, where seconds is
> 	     the number of seconds since the Epoch (00:00:00 UTC, January 1,
> 	     1970; see time(3)), and can be specified in decimal, octal, or
> 	     hex.
> 
> and NetBSD-8 also has -r which expects a "seconds" argument.
> 
> I guess this is not relevant to the case at hand since all Debian
> flavours use GNU userland, so we can rely GNU date(1).

I provided it for completeness only and in case it was suitable for
upstream. If that is not relevant, please ignore. Thanks.


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org 🍥 chris-lamb.co.uk
       `-

Message #45 received at 929793@bugs.debian.org (full text, mbox, reply):

From: Andre Noll <maan@tuebingen.mpg.de>

To: Chris Lamb <lamby@debian.org>

Cc: 929793@bugs.debian.org

Subject: Re: Bug#929793: liblopsub: please make the build reproducible

Date: Wed, 5 Jun 2019 19:58:06 +0200

[Message part 1 (text/plain, inline)]

Hi Chris,

On Wed, Jun 05, 15:04, Chris Lamb wrote
> > Indeed: FreeBSD-12's date(1) has
> > 
> > 	-r seconds
> > 	     Print the date and time represented by seconds, where seconds is
> > 	     the number of seconds since the Epoch (00:00:00 UTC, January 1,
> > 	     1970; see time(3)), and can be specified in decimal, octal, or
> > 	     hex.
> > 
> > and NetBSD-8 also has -r which expects a "seconds" argument.
> > 
> > I guess this is not relevant to the case at hand since all Debian
> > flavours use GNU userland, so we can rely GNU date(1).
> 
> I provided it for completeness only and in case it was suitable for
> upstream. If that is not relevant, please ignore. Thanks.

I went for the simple command and pushed out the commit.

Thanks
Andre
-- 
Max Planck Institute for Developmental Biology
Max-Planck-Ring 5, 72076 Tübingen, Germany. Phone: (+49) 7071 601 829
http://people.tuebingen.mpg.de/maan/

[signature.asc (application/pgp-signature, inline)]

Message #50 received at 929793-done@bugs.debian.org (full text, mbox, reply):

From: Andre Noll <maan@tuebingen.mpg.de>

To: 929793-done@bugs.debian.org

Subject: Re: Bug#929793: Info received (Bug#929793: liblopsub: please make the build reproducible)

Date: Fri, 2 Aug 2019 08:35:37 +0200

[Message part 1 (text/plain, inline)]

Fixed in upstream commit bb3a974325f5967c38da61ff61b5f7c732fcfa59.
-- 
Max Planck Institute for Developmental Biology
Max-Planck-Ring 5, 72076 Tübingen, Germany. Phone: (+49) 7071 601 829
http://people.tuebingen.mpg.de/maan/

[signature.asc (application/pgp-signature, inline)]

Send a report that this bug log contains spam.