Debian Bug report logs - #931854
liblopsub: please make the output of lopsubgen reproducible

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: "Chris Lamb" <lamby@debian.org>

To: submit@bugs.debian.org

Subject: liblopsub: please make the output of lopsubgen reproducible

Date: Thu, 11 Jul 2019 10:43:06 -0300

[Message part 1 (text/plain, inline)]

Source: liblopsub
Version: 1.0.2-1
Severity: wishlist
Tags: patch
User: reproducible-builds@lists.alioth.debian.org
Usertags: toolchain timestamps
X-Debbugs-Cc: reproducible-bugs@lists.alioth.debian.org

Hi,

Whilst working on the Reproducible Builds effort [0], we noticed
that liblopsub generates output that is not reproducible. The
lopsubgen utility does not respect SOURCE_DATE_EPOCH [1] and thus
packages such as src:tfortune are rendered unreproducible as they
then encode the build date and time.

Patch attached.

  [0] https://reproducible-builds.org/
  [1] https://reproducible-builds.org/specs/source-date-epoch/


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

[liblopsub.diff.txt (text/plain, attachment)]

Message #10 received at submit@bugs.debian.org (full text, mbox, reply):

From: Andre Noll <maan@tuebingen.mpg.de>

To: Chris Lamb <lamby@debian.org>, 931854@bugs.debian.org

Cc: submit@bugs.debian.org

Subject: Re: Bug#931854: liblopsub: please make the output of lopsubgen reproducible

Date: Fri, 12 Jul 2019 11:09:13 +0200

[Message part 1 (text/plain, inline)]

On Thu, Jul 11, 10:43, Chris Lamb wrote

> Whilst working on the Reproducible Builds effort [0], we noticed
> that liblopsub generates output that is not reproducible. The
> lopsubgen utility does not respect SOURCE_DATE_EPOCH [1] and thus
> packages such as src:tfortune are rendered unreproducible as they
> then encode the build date and time.
> 
> Patch attached.

Good catch. I've applied your patch and made some minor edits to
improve code readability and to fix the typo in the new comment.

Are you OK with the commit below?

Thanks
Andre
---
commit 0a8299dd8da7dba7437550d12bf180d9e10d512b
Author: Chris Lamb <lamby@debian.org>
Date:   Fri Jul 12 07:59:50 2019 +0200

    lsg.c: Make the output of lopsubgen reproducible.
    
    Whilst working on the Reproducible Builds effort [0], we noticed that
    liblopsub generates output that is not reproducible. The lopsubgen
    utility does not respect SOURCE_DATE_EPOCH [1] and thus packages such
    as src:tfortune are rendered unreproducible as they then encode the
    build date and time.
    
    This patch makes lopsubgen honour SOURCE_DATE_EPOCH.
    
    [0] https://reproducible-builds.org/
    [1] https://reproducible-builds.org/specs/source-date-epoch/
    
    Signed-off-by: Andre Noll <maan@tuebingen.mpg.de>

diff --git a/lsg.c b/lsg.c
index 54b7816..83a72da 100644
--- a/lsg.c
+++ b/lsg.c
@@ -610,7 +610,7 @@ static char *get_output_path(const char *suffix, const char *arg,
 static void gen_man(struct lls_parse_result *lpr, const char *cmdline)
 {
 	int i;
-	time_t t;
+	time_t t = 0;
 	struct tm *tmp;
 	FILE *out;
 	char *outpath = get_output_path("man",
@@ -626,12 +626,22 @@ static void gen_man(struct lls_parse_result *lpr, const char *cmdline)
 	if (suite.commands[0].name.orig) {
 		char date[200];
 		const char *version_string;
-
 		if (!suite.date) {
-			t = time(NULL);
-			tmp = localtime(&t);
+			/*
+			 * If the SOURCE_DATE_EPOCH environment variable
+			 * contains a positive integer in the time_t range, use
+			 * that instead of the current time. See:
+			 * <https://reproducible-builds.org/specs/source-date-epoch/>
+			 * for more information.
+			 */
+			char *source_date_epoch = getenv("SOURCE_DATE_EPOCH");
+			if (source_date_epoch != NULL)
+				t = strtoll(source_date_epoch, NULL, 10);
+			if (t <= 0)
+				t = time(NULL);
+			tmp = gmtime(&t);
 			if (tmp == NULL) {
-				perror("localtime");
+				perror("gmtime");
 				exit(EXIT_FAILURE);
 			}
 			if (strftime(date, sizeof(date), "%B %Y", tmp) == 0) {
-- 
Max Planck Institute for Developmental Biology
Max-Planck-Ring 5, 72076 Tübingen, Germany. Phone: (+49) 7071 601 829
http://people.tuebingen.mpg.de/maan/

[signature.asc (application/pgp-signature, inline)]

Message #20 received at 931854@bugs.debian.org (full text, mbox, reply):

From: "Chris Lamb" <lamby@debian.org>

To: "Andre Noll" <maan@tuebingen.mpg.de>, 931854@bugs.debian.org

Cc: reproducible-bugs@lists.alioth.debian.org

Subject: Re: Bug#931854: liblopsub: please make the output of lopsubgen reproducible

Date: Fri, 12 Jul 2019 10:30:41 -0300

Herr Noll,

> > Whilst working on the Reproducible Builds effort [0], we noticed
> > that liblopsub generates output that is not reproducible.
[…]
> Good catch. I've applied your patch and made some minor edits to
> improve code readability and to fix the typo in the new comment.

Thanks again for your careful review of my haphazard and often-errant
programming style. I am perfectly OK with the quoted commit and I
look forward to it landing in Debian in short time.


Best wishes,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org 🍥 chris-lamb.co.uk
       `-

Message #25 received at 931854@bugs.debian.org (full text, mbox, reply):

From: Andre Noll <maan@tuebingen.mpg.de>

To: Chris Lamb <lamby@debian.org>

Cc: 931854@bugs.debian.org, reproducible-bugs@lists.alioth.debian.org, Adam Borowski <kilobyte@angband.pl>

Subject: Re: Bug#931854: liblopsub: please make the output of lopsubgen reproducible

Date: Sat, 13 Jul 2019 15:09:49 +0200

[Message part 1 (text/plain, inline)]

On Fri, Jul 12, 10:30, Chris Lamb wrote
> > > Whilst working on the Reproducible Builds effort [0], we noticed
> > > that liblopsub generates output that is not reproducible.
> […]
> > Good catch. I've applied your patch and made some minor edits to
> > improve code readability and to fix the typo in the new comment.
> 
> Thanks again for your careful review of my haphazard and often-errant
> programming style. I am perfectly OK with the quoted commit and I
> look forward to it landing in Debian in short time.

I've merged the commit and bumped the Debian version number. Adam:
would you please upload lopsub-1.0.3-2? This version corresponds to
the current master branch of the public repo.

Thanks
Andre
-- 
Max Planck Institute for Developmental Biology
Max-Planck-Ring 5, 72076 Tübingen, Germany. Phone: (+49) 7071 601 829
http://people.tuebingen.mpg.de/maan/

[signature.asc (application/pgp-signature, inline)]

Message #30 received at 931854@bugs.debian.org (full text, mbox, reply):

From: Adam Borowski <kilobyte@angband.pl>

To: Andre Noll <maan@tuebingen.mpg.de>, 931854@bugs.debian.org

Subject: Re: Bug#931854: liblopsub: please make the output of lopsubgen reproducible

Date: Sat, 13 Jul 2019 22:48:39 +0200

On Sat, Jul 13, 2019 at 03:09:49PM +0200, Andre Noll wrote:
> I've merged the commit and bumped the Debian version number. Adam:
> would you please upload lopsub-1.0.3-2? This version corresponds to
> the current master branch of the public repo.

I would gladly upload your updates, but I don't know what upstream tarball
to use.  There's no watch file (the usual automated way to fetch one), you
did not provide one by uploading to mentors.d.n or some site of your own,
and neither did you provide instructions for some custom workflow (there are
many of those).


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ According to recent spams, "all my email accounts are owned
⢿⡄⠘⠷⠚⠋  by a hacker".  So what's the problem?
⠈⠳⣄⠀⠀⠀⠀

Message #35 received at 931854@bugs.debian.org (full text, mbox, reply):

From: Andre Noll <maan@tuebingen.mpg.de>

To: Adam Borowski <kilobyte@angband.pl>

Cc: 931854@bugs.debian.org

Subject: Re: Bug#931854: liblopsub: please make the output of lopsubgen reproducible

Date: Sun, 14 Jul 2019 12:28:33 +0200

[Message part 1 (text/plain, inline)]

On Sat, Jul 13, 22:48, Adam Borowski wrote
> On Sat, Jul 13, 2019 at 03:09:49PM +0200, Andre Noll wrote:
> > I've merged the commit and bumped the Debian version number. Adam:
> > would you please upload lopsub-1.0.3-2? This version corresponds to
> > the current master branch of the public repo.
> 
> I would gladly upload your updates, but I don't know what upstream tarball
> to use.  There's no watch file (the usual automated way to fetch one), you
> did not provide one by uploading to mentors.d.n or some site of your own,
> and neither did you provide instructions for some custom workflow (there are
> many of those).

Sorry for not being clear. You can create the tarball with

	git fetch
	git archive --prefix liblopsub-1.0.3/ origin/master | xz > ../liblopsub_1.0.3.orig.tar.xz

But yes, that's tedious and error prone.

I could certainly create the tarball and upload it somewhere for the
tools to to pick up, but TBH I think that's a bit pointless because
everybody can create the tarball from the public git repo with a
command like the above. The only thing that needs to be communicated is
a git tree-ish, i.e. an identifier for the tree that should be tared,
origin/master in this case.

My preferred choice would be to create a signed tag each time the
version number in debian/changelog changes, to indicate that a new
version should be uploaded, but I have no strong opinion in this
regard.

Which mechanism do you prefer to get informed about pending updates?

Best
Andre
-- 
Max Planck Institute for Developmental Biology
Max-Planck-Ring 5, 72076 Tübingen, Germany. Phone: (+49) 7071 601 829
http://people.tuebingen.mpg.de/maan/

[signature.asc (application/pgp-signature, inline)]

Message #40 received at 931854@bugs.debian.org (full text, mbox, reply):

From: Adam Borowski <kilobyte@angband.pl>

To: Andre Noll <maan@tuebingen.mpg.de>

Cc: 931854@bugs.debian.org

Subject: Re: Bug#931854: liblopsub: please make the output of lopsubgen reproducible

Date: Sun, 14 Jul 2019 14:56:58 +0200

On Sun, Jul 14, 2019 at 12:28:33PM +0200, Andre Noll wrote:
> On Sat, Jul 13, 22:48, Adam Borowski wrote
> > I would gladly upload your updates, but I don't know what upstream tarball
> > to use.  There's no watch file (the usual automated way to fetch one), you
> > did not provide one by uploading to mentors.d.n or some site of your own,
> > and neither did you provide instructions for some custom workflow (there are
> > many of those).

> 	git fetch
> 	git archive --prefix liblopsub-1.0.3/ origin/master | xz > ../liblopsub_1.0.3.orig.tar.xz

This looks like the "git" mode of uscan; I haven't used it though as the
vast majority of my upstreams use github or something similar that offers
downloadable tarballs for tags.

> I could certainly create the tarball and upload it somewhere for the
> tools to to pick up, but TBH I think that's a bit pointless because
> everybody can create the tarball from the public git repo with a
> command like the above.

Yeah, but there's no commonly agreed way to do that.  Or rather, there's
_too many_ ways, with none being dominant.  And generating a bit-to-bit
identical tarball is not as easy as it sounds.

> The only thing that needs to be communicated is a git tree-ish, i.e. an
> identifier for the tree that should be tared, origin/master in this case.

My hopes are rather with dropping the tarball requirement as obsolete and
using a git-based setup directly, but this has been in development for more
than a decade, with no light at the end of the tunnel (so no train for now).

> My preferred choice would be to create a signed tag each time the
> version number in debian/changelog changes, to indicate that a new
> version should be uploaded, but I have no strong opinion in this
> regard.

That's a common and recommended workflow, although when the package can't be
uploaded immediately (eg. NEW or sponsoring), I'd recommend leaving not
pushing the tag until the upload is done.  Thus, a finalized changelog
version serves that purpose, yet is not immutable.

> Which mechanism do you prefer to get informed about pending updates?

Anything that's convenient for you; in general RFS bugs are best but some
folks prefer other arrangements.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Yo momma uses IPv4!
⢿⡄⠘⠷⠚⠋⠀ But why should you?
⠈⠳⣄⠀⠀⠀⠀ https://ipv4flagday.net/

Message #45 received at 931854@bugs.debian.org (full text, mbox, reply):

From: Andre Noll <maan@tuebingen.mpg.de>

To: Adam Borowski <kilobyte@angband.pl>

Cc: 931854@bugs.debian.org

Subject: Re: Bug#931854: liblopsub: please make the output of lopsubgen reproducible

Date: Mon, 15 Jul 2019 18:09:57 +0200

[Message part 1 (text/plain, inline)]

On Sun, Jul 14, 14:56, Adam Borowski wrote
> On Sun, Jul 14, 2019 at 12:28:33PM +0200, Andre Noll wrote:
> > On Sat, Jul 13, 22:48, Adam Borowski wrote
> > > I would gladly upload your updates, but I don't know what upstream tarball
> > > to use.  There's no watch file (the usual automated way to fetch one), you
> > > did not provide one by uploading to mentors.d.n or some site of your own,
> > > and neither did you provide instructions for some custom workflow (there are
> > > many of those).
> 
> > 	git fetch
> > 	git archive --prefix liblopsub-1.0.3/ origin/master | xz > ../liblopsub_1.0.3.orig.tar.xz
> 
> This looks like the "git" mode of uscan; I haven't used it though as the
> vast majority of my upstreams use github or something similar that offers
> downloadable tarballs for tags.

We also have a gitweb instance which has the same contents as the
public git repos:

	http://git.tuebingen.mpg.de/

The snapshot links of each project lets you download the tarball of
any commit. For example

	wget -O liblopsub_1.0.3.orig.tar.gz 'http://git.tuebingen.mpg.de/?p=lopsub.git;a=snapshot;h=master;sf=tgz'

However, you still need to specify the name of the output file, and
the tarball will extract into a directory named lopsub-master-$SHA1,
which is probably not optimal.

> > I could certainly create the tarball and upload it somewhere for the
> > tools to to pick up, but TBH I think that's a bit pointless because
> > everybody can create the tarball from the public git repo with a
> > command like the above.
> 
> Yeah, but there's no commonly agreed way to do that.  Or rather, there's
> _too many_ ways, with none being dominant.  And generating a bit-to-bit
> identical tarball is not as easy as it sounds.

Right. I don't know if git archive creates identical tarballs when
run from two different repos that share the same commit. The man page
does not mention anything about this.

> > My preferred choice would be to create a signed tag each time the
> > version number in debian/changelog changes, to indicate that a new
> > version should be uploaded, but I have no strong opinion in this
> > regard.
> 
> That's a common and recommended workflow, although when the package can't be
> uploaded immediately (eg. NEW or sponsoring), I'd recommend leaving not
> pushing the tag until the upload is done.  Thus, a finalized changelog
> version serves that purpose, yet is not immutable.
> 
> > Which mechanism do you prefer to get informed about pending updates?
> 
> Anything that's convenient for you; in general RFS bugs are best but some
> folks prefer other arrangements.

OK, then lets continue to use email for this. I'll always include a
suitable git archive command in the future.

Thanks
Andre
-- 
Max Planck Institute for Developmental Biology
Max-Planck-Ring 5, 72076 Tübingen, Germany. Phone: (+49) 7071 601 829
http://people.tuebingen.mpg.de/maan/

[signature.asc (application/pgp-signature, inline)]

Message #50 received at 931854-done@bugs.debian.org (full text, mbox, reply):

From: Andre Noll <maan@tuebingen.mpg.de>

To: 931854-done@bugs.debian.org

Subject: Re: Bug#931854: liblopsub: please make the output of lopsubgen reproducible

Date: Wed, 31 Jul 2019 19:04:36 +0200

[Message part 1 (text/plain, inline)]

Fixed in upstream commit 0a8299dd8da7dba7437550d12bf180d9e10d512b.
-- 
Max Planck Institute for Developmental Biology
Max-Planck-Ring 5, 72076 Tübingen, Germany. Phone: (+49) 7071 601 829
http://people.tuebingen.mpg.de/maan/

[signature.asc (application/pgp-signature, inline)]

Send a report that this bug log contains spam.