Debian Bug report logs - #787444
help2man: support externally-supplied --date for reproducibility

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Daniel Kahn Gillmor <dkg@fifthhorseman.net>

To: submit@bugs.debian.org

Subject: help2man: support externally-supplied --date for reproducibility

Date: Mon, 01 Jun 2015 14:49:06 -0400

[Message part 1 (text/plain, inline)]

Package: help2man
Version: 1.46.6
Tags: patch
User: reproducible-builds@lists.alioth.debian.org
Usertags: timestamp toolchain

Hi Brendan--

Please consider the attached patch as a step on the way toward allowing
packages that use help2man to build reproducibly.  For more info, see:

 https://reproducible.debian.net/issues/unstable/timestamps_in_manpage_generated_by_help2man_issue.html

Thanks for help2man,

       --dkg

[0001-add-date-option-to-enable-reproducible-builds.patch (text/x-diff, inline)]

From c0622fc09da334f4637eb983c8585de2e8128642 Mon Sep 17 00:00:00 2001
From: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
Date: Mon, 1 Jun 2015 14:21:57 -0400
Subject: [PATCH] add --date option to enable reproducible builds

This patch provides a -d or --date option to help2man, which allows
the invoker to avoid embedding the current timestamp in the generated
manpage.

This should help to address the issue described here:

 https://reproducible.debian.net/issues/unstable/timestamps_in_manpage_generated_by_help2man_issue.html
---
 debian/changelog |  8 ++++++++
 help2man.PL      | 11 +++++++++--
 help2man.texi    |  5 +++++
 3 files changed, 22 insertions(+), 2 deletions(-)

diff --git a/debian/changelog b/debian/changelog
index 80fa32f..4f1b980 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,11 @@
+help2man (1.46.6+nmu1) unstable; urgency=medium
+
+  * Non-Maintainer Upload.
+  * add -d or --date option to avoid embedding the current month+year in
+    the generated man page.
+
+ -- Daniel Kahn Gillmor <dkg@fifthhorseman.net>  Mon, 01 Jun 2015 14:21:08 -0400
+
 help2man (1.46.6) unstable; urgency=medium
 
   * Add Danish translation (thanks to Joe Hansen).
diff --git a/help2man.PL b/help2man.PL
index cefed06..97da3ea 100755
--- a/help2man.PL
+++ b/help2man.PL
@@ -16,7 +16,7 @@ use 5.008;
 use Config;
 use Getopt::Long;
 
-my ($program, $version) = ('help2man', '1.46.6');
+my ($program, $version) = ('help2man', '1.46.6+nmu1');
 
 my %opts;
 die "Usage: $0 [--quiet] [--stdout] [--with-gettext] [--name] [--version]\n"
@@ -207,7 +207,7 @@ my $help_option = '--help';
 my $version_option = '--version';
 my $discard_stderr = 1;
 my ($opt_name, @opt_include, $opt_output, $opt_info, $opt_no_info, $opt_libtool,
-    $version_text);
+    $version_text, $opt_date);
 
 my %opt_def = (
     'n|name=s'		 => \$opt_name,
@@ -219,6 +219,7 @@ my %opt_def = (
     'I|opt-include=s'	 => sub { push @opt_include, [ pop, 0 ] },
     'o|output=s'	 => \$opt_output,
     'p|info-page=s'	 => \$opt_info,
+    'd|date=s'	 	 => \$opt_date,
     'N|no-info'		 => \$opt_no_info,
     'l|libtool'		 => \$opt_libtool,
     'help'		 => sub { print $help_info; exit },
@@ -371,6 +372,12 @@ $version_text ||= get_option_value $ARGV[0], $version_option;
 # is used on the footer of the generated manual pages.  If in doubt, you may
 # just use %x as the value (which should be the full locale-specific date).
 my $date = enc strftime _("%B %Y"), localtime;
+if ($opt_date)
+{
+    # --date overrides local time.
+    $date = $opt_date;
+}
+
 my $program = program_basename $ARGV[0];
 my $package = $program;
 my $version;
diff --git a/help2man.texi b/help2man.texi
index f35067d..89df8a0 100644
--- a/help2man.texi
+++ b/help2man.texi
@@ -167,6 +167,11 @@ Send output to @var{file} rather than @code{stdout}.
 @itemx --info-page=@var{text}
 Name of Texinfo manual.
 
+@item -d @var{text}
+@itemx --date=@var{text}
+The string to use for the date of the man page.  By default, help2man
+will use the current month and year.
+
 @item -N
 @itemx --no-info
 Suppress inclusion of a @samp{SEE ALSO} paragraph directing the reader
-- 
2.1.4

Message #10 received at 787444@bugs.debian.org (full text, mbox, reply):

From: "Brendan O'Dea" <bod@debian.org>

To: Daniel Kahn Gillmor <dkg@fifthhorseman.net>, 787444@bugs.debian.org

Subject: Re: Bug#787444: help2man: support externally-supplied --date for reproducibility

Date: Wed, 3 Jun 2015 22:24:51 +1000

On 2 June 2015 at 04:49, Daniel Kahn Gillmor <dkg@fifthhorseman.net> wrote:
> Please consider the attached patch as a step on the way toward allowing
> packages that use help2man to build reproducibly.  For more info, see:

Hi Daniel,

I appreciate that you've provided a relatively generic solution to
your problem: adding an option to set the date, but it occurs to me
that most users don't actually care.  Additionally, this will require
changing the builds for anything which uses help2man to add a --date
option.

My inclination is instead to do something fairly specific to your
project: If the environment variable DEB_BUILD_CHANGELOG (or something
similar) is set, then the file to which it refers will be used to find
the latest revision date, and that date will be used on the manual
pages.  Presumably the build system could set this prior to
build/test?

Regards,
Brendan

Message #15 received at 787444@bugs.debian.org (full text, mbox, reply):

From: Daniel Kahn Gillmor <dkg@fifthhorseman.net>

To: Brendan O'Dea <bod@debian.org>, 787444@bugs.debian.org, Debian Reproducible Builds <reproducible-builds@lists.alioth.debian.org>

Subject: Re: Bug#787444: help2man: support externally-supplied --date for reproducibility

Date: Wed, 03 Jun 2015 10:19:15 -0400

Hi Brendan--

Thanks for the quick followup! (cc'ing the r-b list)

On Wed 2015-06-03 08:24:51 -0400, Brendan O'Dea wrote:
> On 2 June 2015 at 04:49, Daniel Kahn Gillmor <dkg@fifthhorseman.net> wrote:
>> Please consider the attached patch as a step on the way toward allowing
>> packages that use help2man to build reproducibly.  For more info, see:
>
> I appreciate that you've provided a relatively generic solution to
> your problem: adding an option to set the date, but it occurs to me
> that most users don't actually care.  Additionally, this will require
> changing the builds for anything which uses help2man to add a --date
> option.

You're right, and that might make the fix take longer to take effect.
But it's not a terrible cost, and we have other packages in that same
situation.
 
> My inclination is instead to do something fairly specific to your
> project: If the environment variable DEB_BUILD_CHANGELOG (or something
> similar) is set, then the file to which it refers will be used to find
> the latest revision date, and that date will be used on the manual
> pages.  Presumably the build system could set this prior to
> build/test?

i think this could work, but it might not apply particularly well to
anyone outside of debian who uses help2man.  I note that help2man is a
native package -- do you know if it's used outside of debian at all?

Making the help2man run dpkg-parsechangelog also seems a little hairier
than it ought to be, and introduces a dependency on dpkg-dev which seems
a little weird.

If the build system is going to set environment variables, maybe it
could set an environment variable with a parseable date string
(ISO-8601?) and help2man could look at that environment variable and
extract a string from it?  (ideally this extraction step would be
TZ-independent, too)

What do you think?

     --dkg

Message #20 received at 787444@bugs.debian.org (full text, mbox, reply):

From: "Brendan O'Dea" <bod@debian.org>

To: Daniel Kahn Gillmor <dkg@fifthhorseman.net>

Cc: 787444@bugs.debian.org, Debian Reproducible Builds <reproducible-builds@lists.alioth.debian.org>

Subject: Re: Bug#787444: help2man: support externally-supplied --date for reproducibility

Date: Thu, 4 Jun 2015 22:17:59 +1000

On 4 June 2015 at 00:19, Daniel Kahn Gillmor <dkg@fifthhorseman.net> wrote:
> On Wed 2015-06-03 08:24:51 -0400, Brendan O'Dea wrote:
>> My inclination is instead to do something fairly specific to your
>> project: If the environment variable DEB_BUILD_CHANGELOG (or something
>> similar) is set, then the file to which it refers will be used to find
>> the latest revision date, and that date will be used on the manual
>> pages.  Presumably the build system could set this prior to
>> build/test?
>
> i think this could work, but it might not apply particularly well to
> anyone outside of debian who uses help2man.  I note that help2man is a
> native package -- do you know if it's used outside of debian at all?

help2man is part of the GNU project, but since I'm the upstream and
the debian maintainer I build it as a native package, dupload the
results to Debian and upload the tarball to ftp.gnu.org at the same
time.

The idea was that unless that environment variable was set, the
behaviour would be unchanged: the current date would be used.  It
seems unlikely that DEB_BUILD_CHANGELOG would be set accidentally.

> Making the help2man run dpkg-parsechangelog also seems a little hairier
> than it ought to be, and introduces a dependency on dpkg-dev which seems
> a little weird.

I wasn't going to use dpkg-parsechangelog, it is a sufficiently
standard format that a simple capturing regex would suffice.

> If the build system is going to set environment variables, maybe it
> could set an environment variable with a parseable date string
> (ISO-8601?) and help2man could look at that environment variable and
> extract a string from it?  (ideally this extraction step would be
> TZ-independent, too)

That would also work, and if your reproducible build environment was
to pull the changelog date and stick it into an environment variable,
then you should only need to patch the tools rather than the Makefile
of every package which uses them.

At worst, for the tools you can't patch, you're in the same place you
are now--having to patch all the Makefiles, and even that is perhaps
easier as you already have the date:

  --date=${DEB_BUILD_CHANGELOG_DATE:-$(date --iso-8601=seconds)}

Regards,
Brendan

Message #25 received at 787444@bugs.debian.org (full text, mbox, reply):

From: Daniel Kahn Gillmor <dkg@fifthhorseman.net>

To: Brendan O'Dea <bod@debian.org>

Cc: 787444@bugs.debian.org, Debian Reproducible Builds <reproducible-builds@lists.alioth.debian.org>

Subject: Re: Bug#787444: help2man: support externally-supplied --date for reproducibility

Date: Thu, 04 Jun 2015 10:19:03 -0400

On Thu 2015-06-04 08:17:59 -0400, Brendan O'Dea wrote:
> The idea was that unless that environment variable was set, the
> behaviour would be unchanged: the current date would be used.  It
> seems unlikely that DEB_BUILD_CHANGELOG would be set accidentally.

right, but this wouldn't be terribly useful for folks like fedora, who
also ship help2man and might want to be reproducible:

 https://admin.fedoraproject.org/pkgdb/package/help2man/
 http://securityblog.redhat.com/2013/09/18/reproducible-builds-for-fedora/

>> If the build system is going to set environment variables, maybe it
>> could set an environment variable with a parseable date string
>> (ISO-8601?) and help2man could look at that environment variable and
>> extract a string from it?  (ideally this extraction step would be
>> TZ-independent, too)
>
> That would also work, and if your reproducible build environment was
> to pull the changelog date and stick it into an environment variable,
> then you should only need to patch the tools rather than the Makefile
> of every package which uses them.
>
> At worst, for the tools you can't patch, you're in the same place you
> are now--having to patch all the Makefiles, and even that is perhaps
> easier as you already have the date:
>
>   --date=${DEB_BUILD_CHANGELOG_DATE:-$(date --iso-8601=seconds)}

Yep, i think this makes sense; reproducible-build folks -- do we want to
take this tack with help2man?  If so, what would the environment
variable be named?  what would its contents be?

         --dkg

Message #30 received at 787444@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@pwned.gg>

To: Reproducible Builds discussion list <reproducible-builds@lists.alioth.debian.org>, Brendan O'Dea <bod@debian.org>

Cc: 787444@bugs.debian.org

Subject: Re: [Reproducible-builds] Bug#787444: help2man: support externally-supplied --date for reproducibility

Date: Thu, 04 Jun 2015 16:38:53 +0200

[Message part 1 (text/plain, inline)]

On 04/06/15 16:19, Daniel Kahn Gillmor wrote:
>>> If the build system is going to set environment variables, maybe it
>>> could set an environment variable with a parseable date string
>>> (ISO-8601?) and help2man could look at that environment variable and
>>> extract a string from it?  (ideally this extraction step would be
>>> TZ-independent, too)
>>
>> That would also work, and if your reproducible build environment was
>> to pull the changelog date and stick it into an environment variable,
>> then you should only need to patch the tools rather than the Makefile
>> of every package which uses them.
>>
>> At worst, for the tools you can't patch, you're in the same place you
>> are now--having to patch all the Makefiles, and even that is perhaps
>> easier as you already have the date:
>>
>>   --date=${DEB_BUILD_CHANGELOG_DATE:-$(date --iso-8601=seconds)}
> 
> Yep, i think this makes sense; reproducible-build folks -- do we want to
> take this tack with help2man?  If so, what would the environment
> variable be named?  what would its contents be?
> 

Hi,

A few months ago I had an idea for this that would be more generalisable. See

https://lists.alioth.debian.org/pipermail/reproducible-builds/Week-of-Mon-20150330/001317.html

for details. TL;DR is to have SOURCEDATE as an environment variable in ISO8601 format.

It would be awesome for help2man to support this. At some point, debhelper can even set this environment variable automatically.

X

-- 
GPG: 4096R/1318EFAC5FBBDBCE
git://github.com/infinity0/pubkeys.git

[signature.asc (application/pgp-signature, attachment)]

Message #35 received at 787444@bugs.debian.org (full text, mbox, reply):

From: Daniel Kahn Gillmor <dkg@fifthhorseman.net>

To: Ximin Luo <infinity0@pwned.gg>, Reproducible Builds discussion list <reproducible-builds@lists.alioth.debian.org>, Brendan O'Dea <bod@debian.org>

Cc: 787444@bugs.debian.org

Subject: Re: [Reproducible-builds] Bug#787444: help2man: support externally-supplied --date for reproducibility

Date: Thu, 04 Jun 2015 13:30:09 -0400

On Thu 2015-06-04 10:38:53 -0400, Ximin Luo wrote:
> A few months ago I had an idea for this that would be more generalisable. See
>
> https://lists.alioth.debian.org/pipermail/reproducible-builds/Week-of-Mon-20150330/001317.html
>
> for details. TL;DR is to have SOURCEDATE as an environment variable in ISO8601 format.

I'm fine with this part of your proposal.

Your proposal also includes a bunch of workarounds with faketime, which
i'm a little concerned about (i say this as the faketime maintainer in
debian, not wanting its particular flakiness in the build path for
everything).

Can we separate the $SOURCEDATE part of your proposal from the faketime
part and just work on $SOURCEDATE independently?

What TZ should SOURCEDATE have?  ISO8601 is capable of supplying a TZ as
well, so the current time could be written in a wide variety of ways
while meaning the same instant:

0 dkg@alice:~$ date '+%FT%T%z' && date -u '+%FT%T%z'
2015-06-04T13:25:25-0400
2015-06-04T17:25:25+0000
0 dkg@alice:~$

I feel like we should we always set it to UTC, so that the inbound
parsed offset would be +0000.  sound sensible?

> It would be awesome for help2man to support this.

help2man (and any other tool that accepts $SOURCEDATE) would also need
to ensure that it extracts the parts it wants in a TZ-independent
fashion as well.  (not parsing back to localtime)

> At some point, debhelper can even set this environment variable
> automatically.

We should open a bug with debhelper requesting this feature as soon as
we come to agreement on the name and semantics.

   --dkg

Message #40 received at 787444@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@pwned.gg>

To: Reproducible Builds discussion list <reproducible-builds@lists.alioth.debian.org>, Brendan O'Dea <bod@debian.org>

Cc: 787444@bugs.debian.org

Subject: Re: [Reproducible-builds] Bug#787444: help2man: support externally-supplied --date for reproducibility

Date: Thu, 04 Jun 2015 20:40:42 +0200

[Message part 1 (text/plain, inline)]

On 04/06/15 19:30, Daniel Kahn Gillmor wrote:
> What TZ should SOURCEDATE have?  ISO8601 is capable of supplying a TZ as
> well, so the current time could be written in a wide variety of ways
> while meaning the same instant:
> 
> 0 dkg@alice:~$ date '+%FT%T%z' && date -u '+%FT%T%z'
> 2015-06-04T13:25:25-0400
> 2015-06-04T17:25:25+0000
> 0 dkg@alice:~$
> 
> I feel like we should we always set it to UTC, so that the inbound
> parsed offset would be +0000.  sound sensible?
> 

I had thought about this a bit, and not yet decided the best way - hence why I haven't yet written this idea up on the Debian Wiki. FWIW, here are my thoughts:

- Always setting it to UTC would be "simplest", but then our format wouldn't be ISO8601 - we'd have to say "ISO8601 but omit the offset / ignore any offset". RFC 2822 and 3339, the other formats mentioned in `man date`, also include an offset.

- It's neater to keep the TZ-offset the same as in debian/changelog. Generated dates will then be consistent with debian/changelog. If debian/changelog says "Mar 2015", then the generated documentation will say "Mar 2015".

- However, it seems awkward to get date(1) to preserve the original offset. I suppose this is an artefact of using localtime(3) as you mentioned. In general the libc time stuff seems to have disastrous behaviour if you want to play with time zones other than local time or UTC, and this affects some other languages like python.

- One can maybe hack around it with the TZ envvar, but it has a very weird syntax [1], and the hack is dependent on the actual value:

$ date -d "2015-06-04T20:18:13+0800" --iso-8601=seconds
2015-06-04T14:18:13+0200 # nope, my local TZ is different

$ TZ="UTC+08" date -d "2015-06-04T20:18:13+0800" --iso-8601=seconds
2015-06-04T04:18:13-0800

$ TZ="UTC-08" date -d "2015-06-04T20:18:13+0800" --iso-8601=seconds
2015-06-04T20:18:13+0800 # argh, TZ expects the opposite sign

[1] http://www.gnu.org/software/libc/manual/html_node/TZ-Variable.html

- A possible workaround is to advise people to just extract the information directly out of SOURCEDATE using a regex (or strip off the last 5 chars of SOURCEDATE before giving it to localtime, maybe). But such advice is extra mental work for developers who may not bother reading the document that we put such advice on.

>> It would be awesome for help2man to support this.
> 
> help2man (and any other tool that accepts $SOURCEDATE) would also need
> to ensure that it extracts the parts it wants in a TZ-independent
> fashion as well.  (not parsing back to localtime)
> 

X

-- 
GPG: 4096R/1318EFAC5FBBDBCE
git://github.com/infinity0/pubkeys.git

[signature.asc (application/pgp-signature, attachment)]

Message #45 received at 787444@bugs.debian.org (full text, mbox, reply):

From: "Brendan O'Dea" <bod@debian.org>

To: Ximin Luo <infinity0@pwned.gg>

Cc: Reproducible Builds discussion list <reproducible-builds@lists.alioth.debian.org>, 787444@bugs.debian.org

Subject: Re: [Reproducible-builds] Bug#787444: help2man: support externally-supplied --date for reproducibility

Date: Fri, 5 Jun 2015 11:51:37 +1000

On 5 June 2015 at 04:40, Ximin Luo <infinity0@pwned.gg> wrote:
> On 04/06/15 19:30, Daniel Kahn Gillmor wrote:
>> What TZ should SOURCEDATE have?  ISO8601 is capable of supplying a TZ as
>> well, so the current time could be written in a wide variety of ways
>> while meaning the same instant:
>>
>> 0 dkg@alice:~$ date '+%FT%T%z' && date -u '+%FT%T%z'
>> 2015-06-04T13:25:25-0400
>> 2015-06-04T17:25:25+0000
>> 0 dkg@alice:~$
>>
>> I feel like we should we always set it to UTC, so that the inbound
>> parsed offset would be +0000.  sound sensible?
>>
>
> I had thought about this a bit, and not yet decided the best way - hence why I haven't yet written this idea up on the Debian Wiki. FWIW, here are my thoughts:
>
> - Always setting it to UTC would be "simplest", but then our format wouldn't be ISO8601 - we'd have to say "ISO8601 but omit the offset / ignore any offset". RFC 2822 and 3339, the other formats mentioned in `man date`, also include an offset.

Given that you're basically creating a standard here, you get to
mandate the format.

I would say that it must be in ISO 8601 format, in the UTC timezone,
preferably using the "Z" suffix, and if the program interprets that
date and emits it in some way then it should be also in UTC.

  2015-06-05T01:08:20Z
  2015-06-05T01:08:20+0000

> - It's neater to keep the TZ-offset the same as in debian/changelog. Generated dates will then be consistent with debian/changelog. If debian/changelog says "Mar 2015", then the generated documentation will say "Mar 2015".

Local times, and daylight savings are just too much of a PITA.  Just
use UTC and if builds on the first of the month are possibly different
to the changelog, so be it.

> - However, it seems awkward to get date(1) to preserve the original offset. I suppose this is an artefact of using localtime(3) as you mentioned. In general the libc time stuff seems to have disastrous behaviour if you want to play with time zones other than local time or UTC, and this affects some other languages like python.
>
> - One can maybe hack around it with the TZ envvar, but it has a very weird syntax [1], and the hack is dependent on the actual value:

The TZ syntax dates back to when all systems were in the Northern
Hemisphere, and pretty much all folks cared about was EST5EDT
(Eastern) and PST8PDT (Pacific).

> $ date -d "2015-06-04T20:18:13+0800" --iso-8601=seconds
> 2015-06-04T14:18:13+0200 # nope, my local TZ is different
>
> $ TZ="UTC+08" date -d "2015-06-04T20:18:13+0800" --iso-8601=seconds
> 2015-06-04T04:18:13-0800
>
> $ TZ="UTC-08" date -d "2015-06-04T20:18:13+0800" --iso-8601=seconds
> 2015-06-04T20:18:13+0800 # argh, TZ expects the opposite sign
>
> [1] http://www.gnu.org/software/libc/manual/html_node/TZ-Variable.html
>
> - A possible workaround is to advise people to just extract the information directly out of SOURCEDATE using a regex (or strip off the last 5 chars of SOURCEDATE before giving it to localtime, maybe). But such advice is extra mental work for developers who may not bother reading the document that we put such advice on.

Mandate input/output in UTC and give a few examples.

  export SOURCEDATE=2015-06-05T01:08:20Z

  Shell:
    $ date -u -d $SOURCEDATE;
    Fri Jun  5 01:08:20 UTC 2015

  Perl:

    use Time::Local 'timegm';
    my $sourcedate = $ENV{SOURCEDATE};
    my ($year, $mon, $mday, $hour, $min, $sec) =
        $sourcedate =~
/^(\d{4})-(\d\d)-(\d\d)T(\d\d):(\d\d):(\d\d)(?:Z|\+0000)$/;
    my $d = timegm $sec, $min, $hour, $mday, $mon-1, $year;
    print scalar gmtime $d;

  Python:

    etc.

>>> It would be awesome for help2man to support this.
>>
>> help2man (and any other tool that accepts $SOURCEDATE) would also need
>> to ensure that it extracts the parts it wants in a TZ-independent
>> fashion as well.  (not parsing back to localtime)

I'm happy to add this.

--bod

Message #50 received at 787444@bugs.debian.org (full text, mbox, reply):

From: Daniel Kahn Gillmor <dkg@fifthhorseman.net>

To: Brendan O'Dea <bod@debian.org>, Ximin Luo <infinity0@pwned.gg>, Lunar <lunar@debian.org>

Cc: 787444@bugs.debian.org, Reproducible Builds discussion list <reproducible-builds@lists.alioth.debian.org>

Subject: Re: [Reproducible-builds] Bug#787444: help2man: support externally-supplied --date for reproducibility

Date: Thu, 04 Jun 2015 22:19:48 -0400

On Thu 2015-06-04 21:51:37 -0400, Brendan O'Dea wrote:
> Local times, and daylight savings are just too much of a PITA.  Just
> use UTC and if builds on the first of the month are possibly different
> to the changelog, so be it.

I agree with Brendan here.

If there are no objections to the idea that we're sticking with ISO-8601
in UTC, preferably with the Z suffix (e.g. 2015-06-05T01:08:20Z) then we
just need to settle on what the right name is.

Lunar^, you'd mentioned that there had been discussions about a
preferred variable name that you might like better than SOURCEDATE.

Any preference?

Where should we document this?

      --dkg

Message #55 received at 787444@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@pwned.gg>

To: Reproducible Builds discussion list <reproducible-builds@lists.alioth.debian.org>, Brendan O'Dea <bod@debian.org>, Lunar <lunar@debian.org>

Cc: 787444@bugs.debian.org

Subject: Re: [Reproducible-builds] Bug#787444: help2man: support externally-supplied --date for reproducibility

Date: Fri, 05 Jun 2015 13:12:46 +0200

[Message part 1 (text/plain, inline)]

On 05/06/15 04:19, Daniel Kahn Gillmor wrote:
> On Thu 2015-06-04 21:51:37 -0400, Brendan O'Dea wrote:
>> Local times, and daylight savings are just too much of a PITA.  Just
>> use UTC and if builds on the first of the month are possibly different
>> to the changelog, so be it.
> 
> I agree with Brendan here.
> 
> If there are no objections to the idea that we're sticking with ISO-8601
> in UTC, preferably with the Z suffix (e.g. 2015-06-05T01:08:20Z) then we
> just need to settle on what the right name is.
> 

If we're going to mandate that it ends with Z, might I suggest that we add "UTC" or "_UTC" to the variable name? It leaves the option open in the future that we might allow TZ offsets.

Note that the TZ offsets mentioned in ISO8601 and the other RFC standards are not "time zones" or "local times" that are subject to DST. They are *fixed offsets*, and don't require extra context (such as the TZ database) to parse correctly. So it would be less of a PITA than you might think.

> Lunar^, you'd mentioned that there had been discussions about a
> preferred variable name that you might like better than SOURCEDATE.
> 
> Any preference?
> 
> Where should we document this?
> 

Holger had previously suggested to me to put it on a subpage in the reproducible builds section on the Debian Wiki, and I agree that would be a good place.

X

-- 
GPG: 4096R/1318EFAC5FBBDBCE
git://github.com/infinity0/pubkeys.git

[signature.asc (application/pgp-signature, attachment)]

Message #60 received at 787444@bugs.debian.org (full text, mbox, reply):

From: "Brendan O'Dea" <bod@debian.org>

To: Ximin Luo <infinity0@pwned.gg>, 787444@bugs.debian.org

Cc: Reproducible Builds discussion list <reproducible-builds@lists.alioth.debian.org>, Lunar <lunar@debian.org>

Subject: Re: Bug#787444: [Reproducible-builds] Bug#787444: help2man: support externally-supplied --date for reproducibility

Date: Sat, 6 Jun 2015 00:55:34 +1000

On 5 June 2015 at 21:12, Ximin Luo <infinity0@pwned.gg> wrote:
> If we're going to mandate that it ends with Z, might I suggest that we add "UTC" or "_UTC" to the variable name? It leaves the option open in the future that we might allow TZ offsets.
>
> Note that the TZ offsets mentioned in ISO8601 and the other RFC standards are not "time zones" or "local times" that are subject to DST. They are *fixed offsets*, and don't require extra context (such as the TZ database) to parse correctly. So it would be less of a PITA than you might think.

Agreed: it is a fixed offset which may be applied with little more
than some multiplications by 60 and the appropriate addition and
subtraction.  There are however three possible formats which need to
be handled outside of Z if you want to be thorough (±hh:mm, ±hhmm and
±hh), and given that this proposal as I understand it is to have the
build infrastructure set this variable, then pushing the complexity to
that single place and making the changes to the program which needs to
interpret this variable *as simple as possible* is likely to make
convincing the authors of such programs to add the feature somewhat
easier.

As far as the naming, given that only programs are going to be
setting/parsing this variable, it can be as descriptive as required
and should be chosen to reduce the possibility of an identically named
variable for another purpose being accidentally interpreted.  I'm
presuming that outside the scope of reproducible builds that the
variable would not be set, in which case programs should fall back to
the default behaviour such as using the current date or the
modification of a file--in which case an unrelated variable of the
same name in the environment could be a problem.

Any of UTC_SOURCE_DATE, SOURCE_DATE_UTC or
REPRODUCABLE_BUILD_SOURCE_DATE_UTC would work for me, but outside of
the concerns mentioned above I don't actually care.

--bod

Message #65 received at 787444@bugs.debian.org (full text, mbox, reply):

From: Daniel Kahn Gillmor <dkg@fifthhorseman.net>

To: Brendan O'Dea <bod@debian.org>, Ximin Luo <infinity0@pwned.gg>, 787444@bugs.debian.org

Cc: Lunar <lunar@debian.org>, Reproducible Builds discussion list <reproducible-builds@lists.alioth.debian.org>

Subject: Re: [Reproducible-builds] Bug#787444: Bug#787444: help2man: support externally-supplied --date for reproducibility

Date: Fri, 05 Jun 2015 12:49:34 -0400

On Fri 2015-06-05 10:55:34 -0400, Brendan O'Dea wrote:
> Any of UTC_SOURCE_DATE, SOURCE_DATE_UTC

My vote is for SOURCE_DATE_UTC, and i agree with Brendan that we should
take the opportunity to define this as strictly and narrowly as possible
(i.e. end in a 'Z', none of the other offsets), so that people relying
on it know they're getting a fixed thing, and don't have to implement
any fancy parsing/offsetting code if they're not already using an
ISO8601-compliant date-parsing library.

> REPRODUCABLE_BUILD_SOURCE_DATE_UTC would work for me

I don't like this one, because apparently it's too hard to spell :)

  --dkg

Message #70 received at 787444@bugs.debian.org (full text, mbox, reply):

From: Holger Levsen <holger@layer-acht.org>

To: reproducible-builds@lists.alioth.debian.org

Cc: "Brendan O'Dea" <bod@debian.org>, 787444@bugs.debian.org

Subject: Re: [Reproducible-builds] Bug#787444: Bug#787444: help2man: support externally-supplied --date for reproducibility

Date: Sat, 6 Jun 2015 16:39:41 +0200

[Message part 1 (text/plain, inline)]

Hi,

On Freitag, 5. Juni 2015, Daniel Kahn Gillmor wrote:
> My vote is for SOURCE_DATE_UTC, and i agree with Brendan that we should
> take the opportunity to define this as strictly and narrowly as possible
> (i.e. end in a 'Z', none of the other offsets), so that people relying
> on it know they're getting a fixed thing, and don't have to implement
> any fancy parsing/offsetting code if they're not already using an
> ISO8601-compliant date-parsing library.

sounds good to me too!


thanks & cheers,
	Holger

[signature.asc (application/pgp-signature, inline)]

Message #75 received at 787444@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@pwned.gg>

To: Reproducible Builds discussion list <reproducible-builds@lists.alioth.debian.org>

Cc: Brendan O'Dea <bod@debian.org>, 787444@bugs.debian.org

Subject: Re: [Reproducible-builds] Bug#787444: Bug#787444: help2man: support externally-supplied --date for reproducibility

Date: Tue, 09 Jun 2015 17:59:52 +0200

[Message part 1 (text/plain, inline)]

On 06/06/15 16:39, Holger Levsen wrote:
> Hi,
> 
> On Freitag, 5. Juni 2015, Daniel Kahn Gillmor wrote:
>> My vote is for SOURCE_DATE_UTC, and i agree with Brendan that we should
>> take the opportunity to define this as strictly and narrowly as possible
>> (i.e. end in a 'Z', none of the other offsets), so that people relying
>> on it know they're getting a fixed thing, and don't have to implement
>> any fancy parsing/offsetting code if they're not already using an
>> ISO8601-compliant date-parsing library.
> 
> sounds good to me too!
> 

Sorry to go back on this a little bit.

Going through the POSIX time functions[1], which unfortunately influences a lot of other naive language libraries such as Python[2], PHP[3], Perl[4], I suggest that we define SOURCE_DATE_EPOCH (name open to discussion) instead:

SOURCE_DATE_EPOCH = "$(date -d "$(dpkg-parsechangelog --count 1 -SDate)" +%s)" # unix timestamp

The reason is that most languages have the "gmtime()" POSIX function to convert a unix timestamp into a time-tuple. However, not every language has an easy way to convert from SOURCE_DATE_UTC into the other options - because the POSIX time functions don't. Often, one needs to do, e.g.:

>>> import os, time
>>> os.environ["TZ"]="UTC"; time.tzset(); time.mktime(time.strptime("2015-06-09T12:50:12Z", "%Y-%m-%dT%H:%M:%SZ"))
1433854212.0

contrast with:

>>> import time
>>> time.gmtime(1433854212)
time.struct_time(tm_year=2015, tm_mon=6, tm_mday=9, tm_hour=12, tm_min=50, tm_sec=12, tm_wday=1, tm_yday=160, tm_isdst=0)

Granted, PHP has gmmktime[5] but this is non-standard.

More examples of other languages, where it's more "basic" to work with unix timestamps:

- In Java, one does "new Date(timestamp) and then uses DateFormat with an explicit scope-restricted TimeZone (none of this tzset global variable business).
- In Javascript, one also does "new Date(timestamp)" then call getUTC{Hours,Seconds} etc on the resulting object.

Given the above, I think it would still be good to define SOURCE_DATE as I originally suggested:

SOURCE_DATE = "$(date -d "$(dpkg-parsechangelog --count 1 -SDate)" --iso-8601=seconds)" # includes the TZ offset

- if the language/tool already has/uses a ISO8601 parser in its standard library, this is as convenient as the previous SOURCE_DATE_UTC
- if the language/tool doesn't have/use one, then SOURCE_DATE_UTC doesn't actually give us any benefits:
  - it's far easier to use SOURCE_DATE_EPOCH, if you want to play with the date programmatically
  - OTOH if you're just going to take substrings/regex-match it, this works just as easily for SOURCE_DATE vs SOURCE_DATE_UTC, and the former contains more information

But I care less about this latter point; the main point of this email is to argue for SOURCE_DATE_EPOCH over SOURCE_DATE_UTC (iso8601 locked to "Z" timezone).

X

[1] http://pubs.opengroup.org/onlinepubs/007908799/xsh/time.h.html
[2] https://docs.python.org/2/library/time.html
[3] https://php.net/manual/en/ref.datetime.php
[4] http://perldoc.perl.org/index-functions-by-cat.html#Time-related-functions
[5] https://php.net/manual/en/function.gmmktime.php

-- 
GPG: 4096R/1318EFAC5FBBDBCE
git://github.com/infinity0/pubkeys.git

[signature.asc (application/pgp-signature, attachment)]

Message #80 received at 787444@bugs.debian.org (full text, mbox, reply):

From: "Brendan O'Dea" <bod@debian.org>

To: Ximin Luo <infinity0@pwned.gg>, 787444@bugs.debian.org

Cc: Reproducible Builds discussion list <reproducible-builds@lists.alioth.debian.org>

Subject: Re: Bug#787444: [Reproducible-builds] Bug#787444: Bug#787444: help2man: support externally-supplied --date for reproducibility

Date: Wed, 10 Jun 2015 20:41:16 +1000

On 10 June 2015 at 01:59, Ximin Luo <infinity0@pwned.gg> wrote:
> Given the above, I think it would still be good to define SOURCE_DATE as I originally suggested:
>
> SOURCE_DATE = "$(date -d "$(dpkg-parsechangelog --count 1 -SDate)" --iso-8601=seconds)" # includes the TZ offset
>
> - if the language/tool already has/uses a ISO8601 parser in its standard library, this is as convenient as the previous SOURCE_DATE_UTC
> - if the language/tool doesn't have/use one, then SOURCE_DATE_UTC doesn't actually give us any benefits:
>   - it's far easier to use SOURCE_DATE_EPOCH, if you want to play with the date programmatically
>   - OTOH if you're just going to take substrings/regex-match it, this works just as easily for SOURCE_DATE vs SOURCE_DATE_UTC, and the former contains more information
>
> But I care less about this latter point; the main point of this email is to argue for SOURCE_DATE_EPOCH over SOURCE_DATE_UTC (iso8601 locked to "Z" timezone).

I disagree that SOURCE_DATE_UTC provides no benefit over SOURCE_DATE:
at the very least, a program could choose to use it as an
uninterpreted string (similar to the proposed --date option at the
start of this bug, but from the environment rather than a flag).
SOURCE_DATE with an arbitrary offset not so much.

I'm happy to accept SOURCE_DATE, SOURCE_DATE_UTC, SOURCE_DATE_EPOCH or
even SOURCE_DATE_EXTRACTED_FROM_DEBIAN_CHANGELOG_WITH_NO_INTERPRETATION
however.  Pick one.

--bod

Message #85 received at 787444@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@pwned.gg>

To: Reproducible Builds discussion list <reproducible-builds@lists.alioth.debian.org>, 787444@bugs.debian.org

Subject: Re: [Reproducible-builds] Bug#787444: Bug#787444: Bug#787444: help2man: support externally-supplied --date for reproducibility

Date: Wed, 10 Jun 2015 12:54:07 +0200

[Message part 1 (text/plain, inline)]

On 10/06/15 12:41, Brendan O'Dea wrote:
> On 10 June 2015 at 01:59, Ximin Luo <infinity0@pwned.gg> wrote:
>> Given the above, I think it would still be good to define SOURCE_DATE as I originally suggested:
>>
>> SOURCE_DATE = "$(date -d "$(dpkg-parsechangelog --count 1 -SDate)" --iso-8601=seconds)" # includes the TZ offset
>>
>> - if the language/tool already has/uses a ISO8601 parser in its standard library, this is as convenient as the previous SOURCE_DATE_UTC
>> - if the language/tool doesn't have/use one, then SOURCE_DATE_UTC doesn't actually give us any benefits:
>>   - it's far easier to use SOURCE_DATE_EPOCH, if you want to play with the date programmatically
>>   - OTOH if you're just going to take substrings/regex-match it, this works just as easily for SOURCE_DATE vs SOURCE_DATE_UTC, and the former contains more information
>>
>> But I care less about this latter point; the main point of this email is to argue for SOURCE_DATE_EPOCH over SOURCE_DATE_UTC (iso8601 locked to "Z" timezone).
> 
> I disagree that SOURCE_DATE_UTC provides no benefit over SOURCE_DATE:
> at the very least, a program could choose to use it as an
> uninterpreted string (similar to the proposed --date option at the
> start of this bug, but from the environment rather than a flag).
> SOURCE_DATE with an arbitrary offset not so much.
> 

The TZ offset is given statically in debian/changelog, and that can also be part of the "uninterpreted string". It's no less arbitrary than the rest of the date itself.

> I'm happy to accept SOURCE_DATE, SOURCE_DATE_UTC, SOURCE_DATE_EPOCH or
> even SOURCE_DATE_EXTRACTED_FROM_DEBIAN_CHANGELOG_WITH_NO_INTERPRETATION
> however.  Pick one.
> 

OK, then let's go with SOURCE_DATE_EPOCH.

X

-- 
GPG: 4096R/1318EFAC5FBBDBCE
git://github.com/infinity0/pubkeys.git

[signature.asc (application/pgp-signature, attachment)]

Message #90 received at 787444@bugs.debian.org (full text, mbox, reply):

From: "Brendan O'Dea" <bod@c47.org>

To: Ximin Luo <infinity0@pwned.gg>, 787444@bugs.debian.org

Cc: Reproducible Builds discussion list <reproducible-builds@lists.alioth.debian.org>

Subject: Re: Bug#787444: [Reproducible-builds] Bug#787444: Bug#787444: Bug#787444: help2man: support externally-supplied --date for reproducibility

Date: Wed, 10 Jun 2015 20:58:24 +1000

On 10 June 2015 at 20:54, Ximin Luo <infinity0@pwned.gg> wrote:
> On 10/06/15 12:41, Brendan O'Dea wrote:
>> On 10 June 2015 at 01:59, Ximin Luo <infinity0@pwned.gg> wrote:
>>> SOURCE_DATE = "$(date -d "$(dpkg-parsechangelog --count 1 -SDate)" --iso-8601=seconds)" # includes the TZ offset

> The TZ offset is given statically in debian/changelog, and that can also be part of the "uninterpreted string". It's no less arbitrary than the rest of the date itself.

Your example above will emit the local timezone, and not the one from
the changelog.

> OK, then let's go with SOURCE_DATE_EPOCH.

Sold.  Adding it now.

--bod

Message #95 received at 787444@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@pwned.gg>

To: Reproducible Builds discussion list <reproducible-builds@lists.alioth.debian.org>, 787444@bugs.debian.org

Subject: Re: [Reproducible-builds] Bug#787444: Bug#787444: Bug#787444: Bug#787444: help2man: support externally-supplied --date for reproducibility

Date: Wed, 10 Jun 2015 13:11:20 +0200

[Message part 1 (text/plain, inline)]

On 10/06/15 12:58, Brendan O'Dea wrote:
> On 10 June 2015 at 20:54, Ximin Luo <infinity0@pwned.gg> wrote:
>> On 10/06/15 12:41, Brendan O'Dea wrote:
>>> On 10 June 2015 at 01:59, Ximin Luo <infinity0@pwned.gg> wrote:
>>>> SOURCE_DATE = "$(date -d "$(dpkg-parsechangelog --count 1 -SDate)" --iso-8601=seconds)" # includes the TZ offset
> 
>> The TZ offset is given statically in debian/changelog, and that can also be part of the "uninterpreted string". It's no less arbitrary than the rest of the date itself.
> 
> Your example above will emit the local timezone, and not the one from
> the changelog.
> 

You are right, my bad. I *meant* that it should be the same offset as in debian/changelog. But yes, annoyingly it is hard to get date(1) to do that, because of the same reasons I outlined earlier. But debhelper could do it correctly in one place, and then tools could use it.

I guess it's up to me to write a patch for that, and then see if people want it. I suppose we can leave this discussion for now.

X

-- 
GPG: 4096R/1318EFAC5FBBDBCE
git://github.com/infinity0/pubkeys.git

[signature.asc (application/pgp-signature, attachment)]

Message #100 received at 787444-close@bugs.debian.org (full text, mbox, reply):

From: Brendan O'Dea <bod@debian.org>

To: 787444-close@bugs.debian.org

Subject: Bug#787444: fixed in help2man 1.47.1

Date: Mon, 15 Jun 2015 16:23:41 +0000

Source: help2man
Source-Version: 1.47.1

We believe that the bug you reported is fixed in the latest version of
help2man, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 787444@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Brendan O'Dea <bod@debian.org> (supplier of updated help2man package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.8
Date: Mon, 15 Jun 2015 23:06:35 +1000
Source: help2man
Binary: help2man
Architecture: source i386
Version: 1.47.1
Distribution: unstable
Urgency: medium
Maintainer: Brendan O'Dea <bod@debian.org>
Changed-By: Brendan O'Dea <bod@debian.org>
Description:
 help2man   - Automatic manpage generator
Closes: 787444
Changes:
 help2man (1.47.1) unstable; urgency=medium
 .
   * Set document language for localised info pages.
   * Update German translations (thanks to Mario Blättermann).
   * Update Ukranian translations (thanks to Yuri Chornoivan).
   * Update French translations (thanks to David Prévot).
   * Update Vietnamese translation (thanks to Trần Ngọc Quân).
   * Update Norwegian Bokmaal translation (thanks to Johnny A. Solbu).
   * Update Russian translation (thanks to Yuri Kozlov).
   * Update Danish translation (thanks to Joe Hansen).
   * Update Polish translations (thanks to Jakub Bogusz).
   * Update Spanish translation and add translation of the info page
     (thanks to Antonio Ceballos).
   * Add support for reproducible builds by using $SOURCE_DATE_EPOCH as the
     date for the generated pages (closes: #787444).
Checksums-Sha1:
 3398bc2e12fdcbfa5e6360da5af142577480e994 1596 help2man_1.47.1.dsc
 ef8cdde2c8332db3ebb7d910f81c46cd44c2ec15 179164 help2man_1.47.1.tar.xz
 a8ac22334d64503dcec8e59d0f0b6bbe4ef7462b 150056 help2man_1.47.1_i386.deb
Checksums-Sha256:
 68de18928dee81ef692848398dfb7c4858f418b2f7cd6daa586e45e8d8b821c2 1596 help2man_1.47.1.dsc
 c59b26f60cb06e45b00e729dea721e7a17220e2c17d800eb428271a750382b06 179164 help2man_1.47.1.tar.xz
 70f536336fd5577cf03ddb3e90b208111a613c82b3e9003a9dc84cb8b947fd14 150056 help2man_1.47.1_i386.deb
Files:
 5b68e7a52e0c661027f6416dc55341be 1596 devel optional help2man_1.47.1.dsc
 cf7aaaeea40bff1176df825430694173 179164 devel optional help2man_1.47.1.tar.xz
 5374d0a1cd96e15a84dc36834759132d 150056 devel optional help2man_1.47.1_i386.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBAgAGBQJVftxmAAoJEBx8Qe3r3btgmocP/imYzk49BDPLdOlQjiFkc7r/
8mzq/en1wRRz7CCCkaZOVjF9uw02RwTG8+H9/pXQpdSFYfu40uuJhuXpg0MlRI7p
QYokC5SETd/E2NQCNG65p6oZu/bPnQG5jL4N3r234vR/NqcIhWm8z7wSaUughnyP
ZN3jHVhYEBTWLjrFhXLPRZ5D5qj6QO4tkRBwUpDanJ8zVqxgtu+y2rccxkpkGUxb
y1JNN0gAL1nAo5w++StvexCTyuNuBYcaCFWBMB5EzlWdO7LQFhbo9K+s4CwQanJc
A7IPH20K5NqkfQ8hsdX4GyuM+OwweJ8+6Jhz9h55akOyJXVbvBpfLEj2xLB5iA0+
1O1lnE/ZD5g48o2Jn/pGiC06ql+xa2DDaY68qOaxUcS79gmhz4FQC08NsXQGafcY
Xq7jSHZQdNMxFXm1B4bTGqYTYs2TTYzI4EGR0+e0j06sTp98iKTGbT4YcdG/H1NM
vpku1he5Klvqrx9n/kGR+44cHTgbAyUmIUPJoLaLmZ9T4ZxqzNRPfDnhPwsfT9hf
55KE0yOpdReRVNCnXCdiwF44TZ9usKDGrTYxEZp4qNqYi1xpBqJTdQKwFezVf8eM
20x59FcktpN86lBiK2+LRmxXiDyqHMe9OBN8el8kAhw50OonsR+BWXrWvRmQ0I5A
/Y0NLhMf3DiTx6vDdcVq
=bCv5
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.