Debian Bug report logs - #843773
sbuild should use build date as binnmu changelog date

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Ian Jackson <ijackson@chiark.greenend.org.uk>

To: submit@bugs.debian.org

Subject: sbuild should use build date as binnmu changelog date

Date: Wed, 9 Nov 2016 12:03:48 +0000

Package: sbuild
Version: 0.72.0-2

Currently, when adding a changelog stanza for a binnmu (or when
appending to the version number is requested for another reason),
sbuild uses the existing source changelog timestamp when inventing
the changelog entry for the binnmu itself:

http://sources.debian.net/src/sbuild/0.72.0-2/lib/Sbuild/Build.pm/#L2005

This causes problems because it means that (in the usual case) the
rebuilt package has files with the same timestamps as the previous
build, but different contents.  So on the end system, the timestamps
cani be misleading, causing malfunction of backup programs etc.  (Eg
an upgrade to a binnmu would be captured only partially in a backup,
leading to lossage.)

AIUI there were two reasons why this particular timestamp was (might
have been) chosen:

Firstly, part of an early attempt to assist multiarch by making all
the changelogs identical on different architectures.  But in fact, the
changelog is not identical in any case (because different
architectures may have differently version-numbered binnmus).  So the
binnmu changelog entry is nowadays put in a separate file, and need
not be the same on different architectures.

Secondly, an attempt to assist reproducible builds.  But the
reproducible build output necessarily includes the complete binnmu
changelog entry; therefore the complete binnmu changelog entry is an
input to a repro-build attempt.  It is indeed contained in the
Binary-Only-Changes field of the .buildinfo.

Subsequent binnmu builds of the same package should generate packages
containing increasing timestamps.  The best timestamp to use is the
timestamp of the build attempt.

So, sbuild should use `date -R`[1] instead of the date from the last
changelog entry in the source package, when generating the binnmu
changelog entry.

[1] Actually, sbuild seems to have a tweakable parameter
"Pkg Start Time" which looks like it would be appropriate, so
something like this:
   my $date = strftime_c "%FT%TZ", gmtime($self->get('Pkg Start Time'));

Ian.
   
-- 
Ian Jackson <ijackson@chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Message #10 received at 843773@bugs.debian.org (full text, mbox, reply):

From: Ian Jackson <ijackson@chiark.greenend.org.uk>

To: Guillem Jover <guillem@debian.org>

Cc: Sven Joachim <svenjoac@gmx.de>, debian-devel@lists.debian.org, Debian Wanna-Build Team <debian-wb-team@lists.debian.org>, reproducible-builds@lists.alioth.debian.org, 843773@bugs.debian.org

Subject: Re: misleading timestamps in binnmus

Date: Wed, 9 Nov 2016 12:59:12 +0000

Thanks to everyone who has provided information.  I have summarised
it in #843773, against sbuild.

What version of sbuild do buildds run ?  Ie, supposing that this is
fixed in sbuild in stretch, will this be fixed on the buildds ?  Or do
we need to update jessie, or what ?

Ian.

-- 
Ian Jackson <ijackson@chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Message #15 received at 843773@bugs.debian.org (full text, mbox, reply):

From: Cyril Brulebois <kibi@debian.org>

To: Ian Jackson <ijackson@chiark.greenend.org.uk>

Cc: Guillem Jover <guillem@debian.org>, Sven Joachim <svenjoac@gmx.de>, debian-devel@lists.debian.org, Debian Wanna-Build Team <debian-wb-team@lists.debian.org>, reproducible-builds@lists.alioth.debian.org, 843773@bugs.debian.org

Subject: Re: misleading timestamps in binnmus

Date: Thu, 10 Nov 2016 02:12:44 +0100

[Message part 1 (text/plain, inline)]

Ian Jackson <ijackson@chiark.greenend.org.uk> (2016-11-09):
> What version of sbuild do buildds run ?  Ie, supposing that this is
> fixed in sbuild in stretch, will this be fixed on the buildds ?  Or do
> we need to update jessie, or what ?

sbuild on buildds uses a specific version of sbuild, for reasons I'm not
going to summarize. The base version is close to what's in jessie (see the
first lines of any build log which has “sbuild (Debian sbuild) 0.65.2”).

dsa-puppet.git has:
,---[ modules/debian-org/files/apt.preferences ]---
| …
| Package: sbuild
| Pin: release o=buildd.debian.org
| Pin-Priority: 500
| 
| Package: buildd
| Pin: release o=buildd.debian.org
| Pin-Priority: 500
| 
| Package: libsbuild-perl
| Pin: release o=buildd.debian.org
| Pin-Priority: 500
`---

Repository seems to live under:
  https://apt.buildd.debian.org/


KiBi.

[signature.asc (application/pgp-signature, inline)]

Message #20 received at 843773@bugs.debian.org (full text, mbox, reply):

From: Johannes Schauer <josch@debian.org>

To: 843773@bugs.debian.org

Cc: ijackson@chiark.greenend.org.uk, reproducible-builds@lists.alioth.debian.org

Subject: Re: sbuild should use build date as binnmu changelog date

Date: Thu, 10 Nov 2016 03:49:18 -0200

[Message part 1 (text/plain, inline)]

Hi Ian and reproducible-builds folks,

On Wed, 9 Nov 2016 12:03:48 +0000 Ian Jackson <ijackson@chiark.greenend.org.uk> wrote:
> Currently, when adding a changelog stanza for a binnmu (or when appending to
> the version number is requested for another reason), sbuild uses the existing
> source changelog timestamp when inventing the changelog entry for the binnmu
> itself:
> 
> http://sources.debian.net/src/sbuild/0.72.0-2/lib/Sbuild/Build.pm/#L2005
> 
> This causes problems because it means that (in the usual case) the
> rebuilt package has files with the same timestamps as the previous
> build, but different contents.  So on the end system, the timestamps
> cani be misleading, causing malfunction of backup programs etc.  (Eg
> an upgrade to a binnmu would be captured only partially in a backup,
> leading to lossage.)
> 
> AIUI there were two reasons why this particular timestamp was (might
> have been) chosen:
> 
> Firstly, part of an early attempt to assist multiarch by making all
> the changelogs identical on different architectures.  But in fact, the
> changelog is not identical in any case (because different
> architectures may have differently version-numbered binnmus).  So the
> binnmu changelog entry is nowadays put in a separate file, and need
> not be the same on different architectures.
> 
> Secondly, an attempt to assist reproducible builds.  But the
> reproducible build output necessarily includes the complete binnmu
> changelog entry; therefore the complete binnmu changelog entry is an
> input to a repro-build attempt.  It is indeed contained in the
> Binary-Only-Changes field of the .buildinfo.
> 
> Subsequent binnmu builds of the same package should generate packages
> containing increasing timestamps.  The best timestamp to use is the
> timestamp of the build attempt.
> 
> So, sbuild should use `date -R`[1] instead of the date from the last
> changelog entry in the source package, when generating the binnmu
> changelog entry.
> 
> [1] Actually, sbuild seems to have a tweakable parameter
> "Pkg Start Time" which looks like it would be appropriate, so
> something like this:
>    my $date = strftime_c "%FT%TZ", gmtime($self->get('Pkg Start Time'));

thanks for putting all the relevant information from the original thread on
debian-devel into this bug report in an organized manner!

While "Pkg Start Time" might be a good default, I guess for to be able to
reproduce a binNMU it would be necessary to also allow the user to pass a
custom timestamp.

I propose to add the command line option --binNMU-date and use that or, if the
command line option is not given, the value from the environment variable
SOURCE_DATE_EPOCH.

The --binNMU-date option can then also be used by debrebuild.pl (see #774415).

Does that sound like an acceptable fix?

Thanks!

cheers, josch

[signature.asc (application/pgp-signature, inline)]

Message #25 received at 843773@bugs.debian.org (full text, mbox, reply):

From: Ansgar Burchardt <ansgar@debian.org>

To: debian-devel@lists.debian.org

Cc: 843773@bugs.debian.org

Subject: Re: misleading timestamps in binnmus

Date: Thu, 10 Nov 2016 10:00:21 +0100

On Wed, 2016-11-09 at 10:04 +0100, Sven Joachim wrote:
> On 2016-11-08 22:30 -0200, Johannes Schauer wrote:
> > It seems that sbuild indeed re-uses the timestamp from the last
> > debian/changelog entry in the binNMU changelog entry.
> 
> This has been done in an early attempt to make binNMUs co-installable 
> in a multiarch world:
> 
> ,----
> > sbuild (0.62.2-1) unstable; urgency=low
> > [...]
> >     - Improve binNMU handling to permit binNMUs for multiarch
> > packages
> >       (Closes: #620112).  Currently, binary NMUs use the current
> > date
> >       in the new changelog entry, but co-installable packages
> > require
> >       an identical changelog.  To avoid this, take the date from
> > the
> >       previous changelog entry to ensure the same date for all
> > binNMUs.
> > [...]
> >  -- Roger Leigh <rleigh@debian.org>  Tue, 05 Apr 2011 10:46:49
> > +0100
> 
> `----
> 
> Which did not help, because the changelog is not actually identical
> anyway.

The date from the last sourceful upload should probably still be used
for any date/time information included in generated files to ensure
they are identical on all architectures (or at least to try to do so).

If you change the date in the binNMU entry, SOURCE_DATE_EPOCH should
probably be set to the date of the last sourceful upload (instead of
just using the most recent changelog entry).

Ansgar

Message #30 received at 843773@bugs.debian.org (full text, mbox, reply):

From: Emilio Pozuelo Monfort <pochu@debian.org>

To: debian-devel@lists.debian.org

Cc: 843773@bugs.debian.org

Subject: Re: misleading timestamps in binnmus

Date: Thu, 10 Nov 2016 10:04:55 +0100

On 10/11/16 10:00, Ansgar Burchardt wrote:
> On Wed, 2016-11-09 at 10:04 +0100, Sven Joachim wrote:
>> On 2016-11-08 22:30 -0200, Johannes Schauer wrote:
>>> It seems that sbuild indeed re-uses the timestamp from the last
>>> debian/changelog entry in the binNMU changelog entry.
>>
>> This has been done in an early attempt to make binNMUs co-installable 
>> in a multiarch world:
>>
>> ,----
>>> sbuild (0.62.2-1) unstable; urgency=low
>>> [...]
>>>     - Improve binNMU handling to permit binNMUs for multiarch
>>> packages
>>>       (Closes: #620112).  Currently, binary NMUs use the current
>>> date
>>>       in the new changelog entry, but co-installable packages
>>> require
>>>       an identical changelog.  To avoid this, take the date from
>>> the
>>>       previous changelog entry to ensure the same date for all
>>> binNMUs.
>>> [...]
>>>  -- Roger Leigh <rleigh@debian.org>  Tue, 05 Apr 2011 10:46:49
>>> +0100
>>
>> `----
>>
>> Which did not help, because the changelog is not actually identical
>> anyway.
> 
> The date from the last sourceful upload should probably still be used
> for any date/time information included in generated files to ensure
> they are identical on all architectures (or at least to try to do so).
> 
> If you change the date in the binNMU entry, SOURCE_DATE_EPOCH should
> probably be set to the date of the last sourceful upload (instead of
> just using the most recent changelog entry).

There are many differences among different architectures, see e.g.:

lame (3.99.5+repack1-9+b1) sid; urgency=low, binary-only=yes

  * Binary-only non-maintainer upload for amd64; no source changes.
  * Rebuild against ncurses 6.0.

 -- amd64 / i386 Build Daemon (x86-csail-01)
<buildd_amd64-x86-csail-01@buildd.debian.org>  Thu, 11 Jun 2015 12:33:22 +0200

But that is not a problem as the entry is added to changelog.$arch for this reason.

Cheers,
Emilio

Message #35 received at 843773@bugs.debian.org (full text, mbox, reply):

From: Johannes Schauer <josch@debian.org>

To: 843773@bugs.debian.org, debian-devel@lists.debian.org

Cc: 843773@bugs.debian.org

Subject: Re: [buildd-tools-devel] Bug#843773: misleading timestamps in binnmus

Date: Thu, 10 Nov 2016 08:59:48 -0200

[Message part 1 (text/plain, inline)]

Hi,

Quoting Emilio Pozuelo Monfort (2016-11-10 07:04:55)
> On 10/11/16 10:00, Ansgar Burchardt wrote:
> > The date from the last sourceful upload should probably still be used
> > for any date/time information included in generated files to ensure
> > they are identical on all architectures (or at least to try to do so).
> > 
> > If you change the date in the binNMU entry, SOURCE_DATE_EPOCH should
> > probably be set to the date of the last sourceful upload (instead of
> > just using the most recent changelog entry).
> 
> There are many differences among different architectures, see e.g.:
> 
> lame (3.99.5+repack1-9+b1) sid; urgency=low, binary-only=yes
> 
>   * Binary-only non-maintainer upload for amd64; no source changes.
>   * Rebuild against ncurses 6.0.
> 
>  -- amd64 / i386 Build Daemon (x86-csail-01)
> <buildd_amd64-x86-csail-01@buildd.debian.org>  Thu, 11 Jun 2015 12:33:22 +0200
> 
> But that is not a problem as the entry is added to changelog.$arch for this reason.

ansgar is not just talking about the changelog which indeed lives in different
files, depending on the host architecture.

What if there are files that are shared by a M-A:same package but with content
that depends on SOURCE_DATE_EPOCH? If we just generate a new date for every
individual build, then these files will be different and the package not be
co-installable anymore.

Quoting Sven Joachim (2016-11-10 07:14:27)
> Wouldn't this cause dpkg-deb to clamp the mtime to that date, precisely
> the problem this thread is all about?

yes, which is why we probably shouldn't set SOURCE_DATE_EPOCH to the same value
again but find a different solution.

One solution would be to increase SOURCE_DATE_EPOCH by 1 second for every
binNMU to a package.

Any other ideas?

Thanks!

cheers, josch

[signature.asc (application/pgp-signature, inline)]

Message #40 received at 843773@bugs.debian.org (full text, mbox, reply):

From: Ian Jackson <ijackson@chiark.greenend.org.uk>

To: Johannes Schauer <josch@debian.org>

Cc: 843773@bugs.debian.org, reproducible-builds@lists.alioth.debian.org

Subject: Re: sbuild should use build date as binnmu changelog date

Date: Thu, 10 Nov 2016 11:29:32 +0000

Johannes Schauer writes ("Re: sbuild should use build date as binnmu changelog date"):
> While "Pkg Start Time" might be a good default, I guess for to be able to
> reproduce a binNMU it would be necessary to also allow the user to pass a
> custom timestamp.

Not only a custom timestamp (although I can see situations where that
might be useful), but a whole custom binnmu changelog entry.

Is there already a command line option to provide the binnmu changelog
entry ?  I think srebuild and/or derebuild wants to use the binnmu
changelog entry from the .buildinfo file and pass it to sbuild.

Imagine, for example, that a new version of sbuild changes some minor
detail of the text in its binnmu changelog entries.  That new version
of sbuild would generate different .debs - unless it used the one from
the .buildinfo.  (And it is no answer to say "use the old sbuild",
because sbuild runs outside the build chroot and there might be very
good reasons for wanting a new one.)

> I propose to add the command line option --binNMU-date and use that
> or, if the command line option is not given, the value from the
> environment variable SOURCE_DATE_EPOCH.

I think this is a good option to have, just for flexibility's sake,
but I don't think debrebuild.pl should use it.

Ian.

-- 
Ian Jackson <ijackson@chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Message #45 received at 843773@bugs.debian.org (full text, mbox, reply):

From: Ian Jackson <ijackson@chiark.greenend.org.uk>

To: Cyril Brulebois <kibi@debian.org>

Cc: Guillem Jover <guillem@debian.org>, Sven Joachim <svenjoac@gmx.de>, debian-devel@lists.debian.org, Debian Wanna-Build Team <debian-wb-team@lists.debian.org>, reproducible-builds@lists.alioth.debian.org, 843773@bugs.debian.org

Subject: Re: misleading timestamps in binnmus

Date: Thu, 10 Nov 2016 11:33:13 +0000

Cyril Brulebois writes ("Re: misleading timestamps in binnmus"):
> Ian Jackson <ijackson@chiark.greenend.org.uk> (2016-11-09):
> > What version of sbuild do buildds run ?  Ie, supposing that this is
> > fixed in sbuild in stretch, will this be fixed on the buildds ?  Or do
> > we need to update jessie, or what ?
> 
> sbuild on buildds uses a specific version of sbuild, for reasons I'm not
> going to summarize. The base version is close to what's in jessie (see the
> first lines of any build log which has “sbuild (Debian sbuild) 0.65.2”).
...
> Repository seems to live under:
>   https://apt.buildd.debian.org/

Thanks.  When Johannes has decided exactly what the sbuild patch looks
like in sid, I will take a look at the repo there and backport the
change.  In what form should I supply mhy update ?  As an source+all
upload of sbuild, as if I were being sponsored ?  As a
git-format-patch against a git import of what I find there (or a
dgitish git branch) ?

Ian.

-- 
Ian Jackson <ijackson@chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Message #50 received at 843773@bugs.debian.org (full text, mbox, reply):

From: Holger Levsen <holger@layer-acht.org>

To: Emilio Pozuelo Monfort <pochu@debian.org>, 843773@bugs.debian.org, debian-devel@lists.debian.org

Subject: Re: [buildd-tools-devel] Bug#843773: misleading timestamps in binnmus

Date: Thu, 10 Nov 2016 12:18:16 +0000

[Message part 1 (text/plain, inline)]

On Thu, Nov 10, 2016 at 08:59:48AM -0200, Johannes Schauer wrote:
> One solution would be to increase SOURCE_DATE_EPOCH by 1 second for every
> binNMU to a package.
> 
> Any other ideas?

set SOURCE_DATE_EPOCH to the creation time of that changelog.$arch
entry?


-- 
cheers,
	Holger

[signature.asc (application/pgp-signature, inline)]

Message #55 received at 843773@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@debian.org>

To: 843773@bugs.debian.org

Subject: Re: [buildd-tools-devel] Bug#843773: misleading timestamps in binnmus

Date: Thu, 10 Nov 2016 17:13:00 +0000

Ansgar Burchardt wrote:
> The date from the last sourceful upload should probably still be used
> for any date/time information included in generated files to ensure
> they are identical on all architectures (or at least to try to do so).
> 
> If you change the date in the binNMU entry, SOURCE_DATE_EPOCH should
> probably be set to the date of the last sourceful upload (instead of
> just using the most recent changelog entry).
> 

Holger Levsen wrote:
> On Thu, Nov 10, 2016 at 08:59:48AM -0200, Johannes Schauer wrote:
> > One solution would be to increase SOURCE_DATE_EPOCH by 1 second for every
> > binNMU to a package.
> > 
> > Any other ideas?
> 
> set SOURCE_DATE_EPOCH to the creation time of that changelog.$arch
> entry?
> 

I'm tending towards the latter suggestion because it's simpler. There's no need to stick to a +1 second scheme etc, and it might mislead people into thinking they can do calculations with this - such as reversing the original timestamp of the sourceful-upload.

Our naming of "SOURCE_DATE_EPOCH" did not really take into account the fact that a source package can be built with many different configurations to create many different build products that are each reproducible themselves. (Debian itself also doesn't do this too clearly, the "+bn" syntax "looks like" it's just a suffix but actually signals an entirely different namespace from source package versions.)

If it helps one sleep better, one can interpret the "SOURCE" in "SOURCE_DATE_EPOCH" to refer to "all implicit and explicit inputs of the build result, including the source code of the package being built but also the binary build dependencies".

(If you want to be super-accurate, you can take the max() of all of the changelogs of all of the transitive build-deps, but I think that's going a bit too far.)

X

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git

Message #60 received at 843773@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@debian.org>

To: 843773@bugs.debian.org, Emilio Pozuelo Monfort <pochu@debian.org>, debian-devel@lists.debian.org, Holger Levsen <holger@layer-acht.org>, Ansgar Burchardt <ansgar@debian.org>

Subject: Re: [buildd-tools-devel] Bug#843773: misleading timestamps in binnmus

Date: Thu, 10 Nov 2016 17:17:00 +0000

(resending again to the correct addresses; I could never get used to debbugs CC behaviour.)

Ximin Luo:
> Ansgar Burchardt wrote:
>> The date from the last sourceful upload should probably still be used
>> for any date/time information included in generated files to ensure
>> they are identical on all architectures (or at least to try to do so).
>>
>> If you change the date in the binNMU entry, SOURCE_DATE_EPOCH should
>> probably be set to the date of the last sourceful upload (instead of
>> just using the most recent changelog entry).
>>
> 
> Holger Levsen wrote:
>> On Thu, Nov 10, 2016 at 08:59:48AM -0200, Johannes Schauer wrote:
>>> One solution would be to increase SOURCE_DATE_EPOCH by 1 second for every
>>> binNMU to a package.
>>>
>>> Any other ideas?
>>
>> set SOURCE_DATE_EPOCH to the creation time of that changelog.$arch
>> entry?
>>
> 
> I'm tending towards the latter suggestion because it's simpler. There's no need to stick to a +1 second scheme etc, and it might mislead people into thinking they can do calculations with this - such as reversing the original timestamp of the sourceful-upload.
> 
> Our naming of "SOURCE_DATE_EPOCH" did not really take into account the fact that a source package can be built with many different configurations to create many different build products that are each reproducible themselves. (Debian itself also doesn't do this too clearly, the "+bn" syntax "looks like" it's just a suffix but actually signals an entirely different namespace from source package versions.)
> 
> If it helps one sleep better, one can interpret the "SOURCE" in "SOURCE_DATE_EPOCH" to refer to "all implicit and explicit inputs of the build result, including the source code of the package being built but also the binary build dependencies".
> 
> (If you want to be super-accurate, you can take the max() of all of the changelogs of all of the transitive build-deps, but I think that's going a bit too far.)
> 
> X
> 

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git

Message #65 received at 843773@bugs.debian.org (full text, mbox, reply):

From: Aurelien Jarno <aurelien@aurel32.net>

To: Ian Jackson <ijackson@chiark.greenend.org.uk>

Cc: Cyril Brulebois <kibi@debian.org>, Guillem Jover <guillem@debian.org>, Sven Joachim <svenjoac@gmx.de>, debian-devel@lists.debian.org, Debian Wanna-Build Team <debian-wb-team@lists.debian.org>, reproducible-builds@lists.alioth.debian.org, 843773@bugs.debian.org

Subject: Re: misleading timestamps in binnmus

Date: Thu, 10 Nov 2016 22:34:36 +0100

On 2016-11-10 11:33, Ian Jackson wrote:
> Cyril Brulebois writes ("Re: misleading timestamps in binnmus"):
> > Ian Jackson <ijackson@chiark.greenend.org.uk> (2016-11-09):
> > > What version of sbuild do buildds run ?  Ie, supposing that this is
> > > fixed in sbuild in stretch, will this be fixed on the buildds ?  Or do
> > > we need to update jessie, or what ?
> > 
> > sbuild on buildds uses a specific version of sbuild, for reasons I'm not
> > going to summarize. The base version is close to what's in jessie (see the
> > first lines of any build log which has “sbuild (Debian sbuild) 0.65.2”).
> ...
> > Repository seems to live under:
> >   https://apt.buildd.debian.org/
> 
> Thanks.  When Johannes has decided exactly what the sbuild patch looks
> like in sid, I will take a look at the repo there and backport the
> change.  In what form should I supply mhy update ?  As an source+all

When it's done, just ping us with the commit number, we will backport it
in our branch and we will deploy it on the build daemons.

Aurelien

-- 
Aurelien Jarno                          GPG: 4096R/1DDD8C9B
aurelien@aurel32.net                 http://www.aurel32.net

Message #70 received at 843773@bugs.debian.org (full text, mbox, reply):

From: Johannes Schauer <josch@debian.org>

To: 843773@bugs.debian.org

Cc: debian-devel@lists.debian.org

Subject: Re: [buildd-tools-devel] Bug#843773: Bug#843773: misleading timestamps in binnmus

Date: Sat, 12 Nov 2016 08:24:48 +0100

[Message part 1 (text/plain, inline)]

Hi,

Quoting Ximin Luo (2016-11-10 18:13:00)
> Holger Levsen wrote:
> > On Thu, Nov 10, 2016 at 08:59:48AM -0200, Johannes Schauer wrote:
> > > One solution would be to increase SOURCE_DATE_EPOCH by 1 second for every
> > > binNMU to a package.
> > > 
> > > Any other ideas?
> > set SOURCE_DATE_EPOCH to the creation time of that changelog.$arch
> > entry?
> I'm tending towards the latter suggestion because it's simpler. There's no
> need to stick to a +1 second scheme etc, and it might mislead people into
> thinking they can do calculations with this - such as reversing the original
> timestamp of the sourceful-upload.

maybe I'm misunderstanding the problem but for the sake getting a better
picture, let me write a summary of the conversation so far:

Currently, buildds (using sbuild) create binNMU changelog entries by copying
the date from the last changelog entry. See for example
libghc-yesod-auth-oauth-dev. 16 binNMUs and the same timestamp is used for all
of them. In his original post, Ian complains that despite the package version
being incremented, the file mtimes are not. This is because we now use the
timestamp from the latest changelog entry for the mtimes of the binary package
contents so that the binary packages become reproducible. If we want to have
later mtimes for the files in binNMU binary packages compared the ones in the
last version, then the question becomes: which mechanism should we use to set
this timestamp?

I do not think that reproducibility is an issue here. No matter which mechanism
we choose to determine the timestamp of the new changelog entry, that timestamp
will be recorded in the Binary-Only-Changes field of the generated .buildinfo
file. The content of that field can then be used by package builders to
generate the new debian/changelog which in turn means that at the time that
dpkg-buildpackage is run, SOURCE_DATE_EPOCH will be set to a deterministic
value: the value of the last changelog entry which we obtain from the
Binary-Only-Changes field.

Back to the question of how we should determine the initial timestamp for the
new binNMU. One way would be to just use the timestamp of the time at which the
binNMU changelog entry is created by sbuild on the buildds. The problem with
this approach is, that binNMUs will be done at slightly different times on each
architecture and thus a different changelog timestamp will be used per
architecture. This does not cause Multi-Arch:same file conflicts because of the
changelog itself because these are stored in architecture-specific paths for
binNMUs (if dh_installchangelogs is used). Instead, file conflicts might be
created from files with content that depends on SOURCE_DATE_EPOCH. Because each
architecture will generate their own timestamp, SOURCE_DATE_EPOCH will be
different on each. If the content of shared files depends on the set timestamp,
then these packages will not be co-installable anymore. One solultion to this
problem would be to mandate that files shared across Multi-Arch:same binary
packages must be independent of SOURCE_DATE_EPOCH but that might impose too
much of a burden of writing and then maintaining the appropriate patches for
upstreams that refuse to become independent of SOURCE_DATE_EPOCH. Moving the
architecture-invariant but SOURCE_DATE_EPOCH dependent files to
Architecture:all packages is not a solution either because we might want the
packages containing these files be able to transport their architecture to
their dependencies.

Thus I think it would be best if the changelog timestamp for the same binNMU
version would be equal across all architectures. One way to achieve this would
be by using a unified scheme to generate the timestamps, for example by
incrementing the original timestamp by one second for each binNMU. Another way
to achieve this would be to require a timestamp to be supplied (or generated)
every time a binNMU is scheduled. That timestamp would then be passed to all
buildds and be used by sbuild to generate the new entry in debian/changelog.

What do you think?

Thanks!

cheers, josch

[signature.asc (application/pgp-signature, inline)]

Message #75 received at 843773@bugs.debian.org (full text, mbox, reply):

From: Ian Jackson <ijackson@chiark.greenend.org.uk>

To: Johannes Schauer <josch@debian.org>

Cc: 843773@bugs.debian.org, debian-devel@lists.debian.org

Subject: Re: [buildd-tools-devel] Bug#843773: Bug#843773: misleading timestamps in binnmus

Date: Mon, 14 Nov 2016 13:52:18 +0000

Johannes Schauer writes ("Re: [buildd-tools-devel] Bug#843773: Bug#843773: misleading timestamps in binnmus"):
> Instead, file conflicts might be created from files with
> content that depends on SOURCE_DATE_EPOCH.

tl;dr:
   Analysis.  Revised proposal:
   Introduce BUILD_DATE_EPOCH (= time of sbuild binnmu invocation)
   and use it for file timestamps (only (for now))


We have a difficulty now.  There are, for a MA-Same package, two
relevant timestamps:

(i) The time of the original source package changelog.  I'm going to
   call this the `source timestamp'.

   This has to be used for the embedded timestamps of all shared
   files in MA-same packages.

   Note that it is technically not correct to use the source timestamp
   for embedded timestamps of these files.  Consider a package which
   is binnmu'd on all architectures because a documentation rebuild is
   needed.  In practice, though, such timestamps are generally either
   in hidden places and ignored by almost anything, or displayed to
   users in a kind of context where users are already often exposed to
   out-of-date timestamps (or where users might already expect the
   timestamp to relate to the content of the documentation, rather
   than the time it was last reformatted).

   It would be tolerable although technically not quite correct, for
   the reasons I have just discussed, to use the source timestamp for
   embedded timestamps of arch-specific files in MA-Same packages, and
   embedded timestamps of all files in non-MA packages.

(ii) The time of the binnmu build (or perhaps the time, when the
   binnmu request was generated and the request message chosen etc.)
   I will call this the `binnmu timestamp'.

   This should be used (at least) for all file timestamps of any
   arch-specific files (to avoid the bug I reported at the start of
   this thread).

   This timestamp can safely be used for _file timestamps_ of all
   files, without disturbing reproducibility.

   It would be correct to use this timestamp for embedded timestamps
   in arch-specific files, since (in the usual case), arch-specific
   files will change on such a rebuild.  There is no reason not to use
   the binnmu timestamp in these cases, except for the effort of
   causing this to happen.

   It would be correct to use this timestamp for embedded timestamps
   in MA-same shared files iff the shared files were (intentionally)
   changed by the rebuild.  We don't have a mechanism to say, right
   now, whether that is the case.

   I don't think it is possible to make the binnmu timestamp the same
   across architectures.  For example, a package might be rebuilt only
   on some architectures.  I don't think we want to change that.  In
   particular, even if we were prepared to change that in Debian, we
   should not impose always-binnmu-all-arches as a rule on all our
   downstreams.

Both of these timestamps are in principle available to dpkg, dh, et
al.

I suggest the following scheme:

 * The binnmu timestamp will be set to the current time when the
   binnmu changelog entry is generated.  (The whole binnmu changelog
   entry must in any case be reused when a build is to be reproduced,
   so there is no reproducibility hazard here.)

 * For file timestamps of generated files, we will use the binnmu
   timestamp.

 * For all embedded timestamps in shared files, we will use the
   source timestamp.

 * For embedded timestamps, we will initially use the source
   timestamp; but, we will treat the misleading timestamp as a minor
   bug and intend, eventually (ie some time next century) arrange for
   it to be set to the binnmu timestamp, where possible.

We would implement this as follows:

1. dpkg-buildpackage will be changed to set BUILD_DATE_EPOCH to the
   binnmu timestamp, if there is one.  It will set it to the source
   timestamp otherwise.  dpkg-buildpackage will conversely be changed
   to set SOURCE_DATE_EPOCH to the _source_ changelog timestamp, even
   if there is also a binnmu changelog entry.

2. dpkg-deb will be changed to honour BUILD_DATE_EPOCH instead of
   SOURCE_DATE_EPOCH, if it is set, when clamping file timestamps.

3. sbuild should be changed to set the binnmu changelog timestamp to
   the answer from time(2), by default.

4. Eventually, if anyone cares, we will teach tools that generate
   arch-specific files, and include timestamps, to honour
   BUILD_DATE_EPOCH (either by adjusting those tools directly, or by
   adjusting the way they are called by dh, or whatever).  We could
   also arrange for SOURCE_DATE_EPOCH to be set to the binnmu
   timestamp iff the package generates no MA-same .debs, or other
   crazy things.  None of these future changes are essential for our
   tools to function correct; they only fix the minor cosmetic issue
   of misleading embedded timestamps.

Ian.
;4~

-- 
Ian Jackson <ijackson@chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Message #80 received at 843773@bugs.debian.org (full text, mbox, reply):

From: Johannes Schauer <josch@debian.org>

To: 843773@bugs.debian.org, debian-devel@lists.debian.org

Subject: Re: [buildd-tools-devel] Bug#843773: Bug#843773: misleading timestamps in binnmus

Date: Mon, 14 Nov 2016 15:38:24 +0100

[Message part 1 (text/plain, inline)]

Hi,

Quoting Ian Jackson (2016-11-14 14:52:18)
>    I don't think it is possible to make the binnmu timestamp the same
>    across architectures.  For example, a package might be rebuilt only
>    on some architectures.  I don't think we want to change that.  In
>    particular, even if we were prepared to change that in Debian, we
>    should not impose always-binnmu-all-arches as a rule on all our
>    downstreams.

Thank you for your analysis and your proposal.

I want to understand why passing the same timestamp to all architectures is an
inferior solution to your proposal. I also don't see why it's a problem that a
package might only be rebuilt on some architectures. If only some architectures
of a M-A:same package get a binNMU, then they are not co-installable anyways.

Thanks!

cheers, josch

[signature.asc (application/pgp-signature, inline)]

Message #85 received at 843773@bugs.debian.org (full text, mbox, reply):

From: Ian Jackson <ijackson@chiark.greenend.org.uk>

To: Johannes Schauer <josch@debian.org>

Cc: 843773@bugs.debian.org, debian-devel@lists.debian.org

Subject: Re: [buildd-tools-devel] Bug#843773: Bug#843773: misleading timestamps in binnmus

Date: Mon, 14 Nov 2016 16:33:55 +0000

Johannes Schauer writes ("Re: [buildd-tools-devel] Bug#843773: Bug#843773: misleading timestamps in binnmus"):
> I want to understand why passing the same timestamp to all
> architectures is an inferior solution to your proposal.

This is a sensible question.  Thanks for helping to explore all the
issues.

TBH I'm not completely sure that it is, but:

Unless the timestamp is of the binnmu request, plumbing to try to get
the same timestamp will be difficult.

I'm not a fan of the idea of merely adding 1 second per binnmu.  That
would mean that making a second binnmu correctly would involve looking
in the archive to see what the previous binnmu timestamp was.  It
would also mean that the timestamps would be quite blatant lies: for
example, there would be files claiming to have been generated with
compiler X at time T, where compiler X did not exist at time T.

If the timestamp is of the binnmu request then I guess it will all
work, but the extra plumbing seems unnecessary.

> I also don't see why it's a problem that a package might only be
> rebuilt on some architectures. If only some architectures of a
> M-A:same package get a binNMU, then they are not co-installable
> anyways.

I think you're right that this isn't a problem.

Can I ask you the converse question: what same-timestamp proposal do
you think is best and why ?

Regards,
Ian.

-- 
Ian Jackson <ijackson@chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Message #90 received at 843773@bugs.debian.org (full text, mbox, reply):

From: Johannes Schauer <josch@debian.org>

To: 843773@bugs.debian.org, debian-devel@lists.debian.org

Subject: Re: [buildd-tools-devel] Bug#843773: Bug#843773: misleading timestamps in binnmus

Date: Mon, 14 Nov 2016 18:10:57 +0100

[Message part 1 (text/plain, inline)]

Hi,

Quoting Ian Jackson (2016-11-14 17:33:55)
> Unless the timestamp is of the binnmu request, plumbing to try to get
> the same timestamp will be difficult.
> 
> I'm not a fan of the idea of merely adding 1 second per binnmu.  That
> would mean that making a second binnmu correctly would involve looking
> in the archive to see what the previous binnmu timestamp was.  It
> would also mean that the timestamps would be quite blatant lies: for
> example, there would be files claiming to have been generated with
> compiler X at time T, where compiler X did not exist at time T.
> 
> If the timestamp is of the binnmu request then I guess it will all
> work, but the extra plumbing seems unnecessary.
> 
> > I also don't see why it's a problem that a package might only be
> > rebuilt on some architectures. If only some architectures of a
> > M-A:same package get a binNMU, then they are not co-installable
> > anyways.
> 
> I think you're right that this isn't a problem.
> 
> Can I ask you the converse question: what same-timestamp proposal do
> you think is best and why ?

I found Guillem's suggestion the most sensible and as far as I understand the
matter also the easiest to implement:

Quoting Guillem Jover (2016-11-09 00:18:25)
> So the actual problem is that the last timestamp gets reused for the
> binNMUs, which seems totally bogus to me. This needs to be fixed in
> whatever is injecting the binNMU entries on the buildds.

but that proposal did not get any attention so I somehow assumed that there's
probably a reason not to do so and thus I suggested an alternative: choose the
new timestamp by using the binNMU number and add that many number of seconds. A
+b17 binNMU would add 17 seconds to the original changelog timestamp. Thus, no
archive lookups would be required.

But maybe to talk about this option: what would speak against changing the
"nmu" command of wanna-build to also add an option that allows setting a
timestamp, or even let wanna-build generate that timestamp itself (from the
time it processes the "nmu" command) and then pass it to sbuild via a
not-yet-existing --binNMU-timestamp option?

With that solution we would not have to change anything other than wanna-build
and sbuild. And I would take care of the latter.

Thanks!

cheers, josch

[signature.asc (application/pgp-signature, inline)]

Message #95 received at 843773@bugs.debian.org (full text, mbox, reply):

From: Holger Levsen <holger@layer-acht.org>

To: debian-devel@lists.debian.org

Cc: 843773@bugs.debian.org

Subject: Re: [buildd-tools-devel] Bug#843773: Bug#843773: misleading timestamps in binnmus

Date: Mon, 14 Nov 2016 17:25:34 +0000

[Message part 1 (text/plain, inline)]

Hi,

thanks for having this discussion!

On Mon, Nov 14, 2016 at 06:10:57PM +0100, Johannes Schauer wrote:
> Quoting Ian Jackson (2016-11-14 17:33:55)
> > Can I ask you the converse question: what same-timestamp proposal do
> > you think is best and why ?
>
> I found Guillem's suggestion the most sensible and as far as I understand the
> matter also the easiest to implement:
> 
> Quoting Guillem Jover (2016-11-09 00:18:25)
> > So the actual problem is that the last timestamp gets reused for the
> > binNMUs, which seems totally bogus to me. This needs to be fixed in
> > whatever is injecting the binNMU entries on the buildds.
> 
> but that proposal did not get any attention…

I'm not sure what his actual proposal was.

To me it seems a binNMU should change SOURCE_DATE_EPOCH, as
debian/changelog gets modified by changelog.$arch, so it's actually a
different source which is being build.

What do you think?


-- 
cheers,
	Holger

[signature.asc (application/pgp-signature, inline)]

Message #100 received at 843773@bugs.debian.org (full text, mbox, reply):

From: Raphael Hertzog <hertzog@debian.org>

To: Ian Jackson <ijackson@chiark.greenend.org.uk>, 843773@bugs.debian.org, debian-devel@lists.debian.org

Subject: Re: [buildd-tools-devel] Bug#843773: misleading timestamps in binnmus

Date: Wed, 16 Nov 2016 11:17:19 +0100

Hi,

On Mon, 14 Nov 2016, Johannes Schauer wrote:
> > Can I ask you the converse question: what same-timestamp proposal do
> > you think is best and why ?
> 
> I found Guillem's suggestion the most sensible and as far as I understand the
> matter also the easiest to implement:
> 
> Quoting Guillem Jover (2016-11-09 00:18:25)
> > So the actual problem is that the last timestamp gets reused for the
> > binNMUs, which seems totally bogus to me. This needs to be fixed in
> > whatever is injecting the binNMU entries on the buildds.
> 
> but that proposal did not get any attention so I somehow assumed that there's
> probably a reason not to do so

I don't think so. Given the discussions we had, I agree that it would be
best if the bin-nmu timestamp could be set by wanna-build itself, at the
time the binnmus are requested. That would give a consistent (and real)
timestamp across all arches being rebuilt together.

> and thus I suggested an alternative: choose the
> new timestamp by using the binNMU number and add that many number of seconds. A
> +b17 binNMU would add 17 seconds to the original changelog timestamp. Thus, no
> archive lookups would be required.

This is assuming that the bin-nmu versioning space is linear. It no longer
is because the bin-nmu is no longer defined by the "+bX" added to the
version but by the "binary-only=yes" changelog attribute.

In fact, I expect that we will want bin-nmu within PPA and that we will
have bin-nmu versions like <version>+ppafoo1 in one repo and
<version>+ppabar1 in another repository. (Ubuntu could also benefit
from this when it rebuilds a single source package for multiple release
in a PPA.)

That's something that you could already implement in sbuild BTW. It
currently does not allow to pass such a binNMU version while dpkg
allows it. :-)

Maybe it can be smart and if the parameter to --binNMU-version contains
(or starts with?) non-digits, then it should assume that it's the full
bin-NMU suffix that is passed.

> But maybe to talk about this option: what would speak against changing the
> "nmu" command of wanna-build to also add an option that allows setting a
> timestamp, or even let wanna-build generate that timestamp itself (from the
> time it processes the "nmu" command) and then pass it to sbuild via a
> not-yet-existing --binNMU-timestamp option?
> 
> With that solution we would not have to change anything other than wanna-build
> and sbuild. And I would take care of the latter.

+1 from me on this solution. I don't see anything significant drawback.

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: http://www.freexian.com/services/debian-lts.html
Learn to master Debian: http://debian-handbook.info/get/

Message #105 received at 843773@bugs.debian.org (full text, mbox, reply):

From: Johannes Schauer <josch@debian.org>

To: 843773@bugs.debian.org, debian-devel@lists.debian.org

Cc: 843773@bugs.debian.org

Subject: Re: [buildd-tools-devel] Bug#843773: Bug#843773: Bug#843773: misleading timestamps in binnmus

Date: Wed, 16 Nov 2016 12:50:21 +0100

[Message part 1 (text/plain, inline)]

Hi,

Quoting Holger Levsen (2016-11-14 18:25:34)
> To me it seems a binNMU should change SOURCE_DATE_EPOCH, as debian/changelog
> gets modified by changelog.$arch, so it's actually a different source which
> is being build.

debian/changelog doesn't get modified by changelog.$arch. The latter is
generated by dh_installchangelogs from debian/changelog and gets installed into
the binary package. The order is (currently) the following:

 - "nmu" command is sent to wanna-build
 - sbuild is executed with the --make-binNMU on the buildds
 - sbuild modifies debian/changelog in the source package by adding a new
   changelog entry at the top, copying the timestamp of the last entry
 - sbuild executes dpkg-buildpackage
 - dpkg-buildpackage parses debian/changelog and calculates SOURCE_DATE_EPOCH
   from the top-most entry
 - debian/rules calls dh_installchangelogs which creates changelog.$arch

Maybe you were talking about reproducing the package build from the timestamp
in changelog.$arch?

Thanks!

cheers, josch

[signature.asc (application/pgp-signature, inline)]

Message #110 received at 843773@bugs.debian.org (full text, mbox, reply):

From: Wouter Verhelst <wouter@debian.org>

To: Ian Jackson <ijackson@chiark.greenend.org.uk>, 843773@bugs.debian.org, debian-devel@lists.debian.org

Subject: Re: [buildd-tools-devel] Bug#843773: Bug#843773: misleading timestamps in binnmus

Date: Thu, 1 Dec 2016 16:24:16 +0100

Hi,

(Sorry for piping in so late to the party here)

On Mon, Nov 14, 2016 at 06:10:57PM +0100, Johannes Schauer wrote:
> But maybe to talk about this option: what would speak against changing the
> "nmu" command of wanna-build to also add an option that allows setting a
> timestamp, or even let wanna-build generate that timestamp itself (from the
> time it processes the "nmu" command) and then pass it to sbuild via a
> not-yet-existing --binNMU-timestamp option?

Wanna-build has a "State-Change" date:

wouter@wuiet:~$ wanna-build -A powerpc --info nbd
nbd:
  Package             : nbd
  Version             : 1:3.14-4
  Builder             : buildd_powerpc-porpora
  State               : Installed
  Section             : admin
  Priority            : source
  Installed-Version   : 1:3.14-4
  Previous-State      : Uploaded
  State-Change        : 2016-11-21 23:13:18.744533
  Build-time          : 9255
  CalculatedPri       : 50
  component           : main
  Distribution        : sid
  Notes               : out-of-date
  Old-Failed          : -------------------- 1:2.9.23-1 --------------------
    fails test suite
  State-Days          : 9
  State-Time          : 835808
  Success-build-time  : 366

Why not use that?

-- 
< ron> I mean, the main *practical* problem with C++, is there's like a dozen
       people in the world who think they really understand all of its rules,
       and pretty much all of them are just lying to themselves too.
 -- #debian-devel, OFTC, 2016-02-12

Message #115 received at 843773@bugs.debian.org (full text, mbox, reply):

From: Johannes Schauer <josch@debian.org>

To: Wouter Verhelst <wouter@debian.org>, "Ian Jackson" <ijackson@chiark.greenend.org.uk>, 843773@bugs.debian.org, debian-devel@lists.debian.org

Subject: Re: [buildd-tools-devel] Bug#843773: Bug#843773: misleading timestamps in binnmus

Date: Thu, 01 Dec 2016 16:51:39 +0100

[Message part 1 (text/plain, inline)]

Hi,

Quoting Wouter Verhelst (2016-12-01 16:24:16)
> On Mon, Nov 14, 2016 at 06:10:57PM +0100, Johannes Schauer wrote:
> > But maybe to talk about this option: what would speak against changing the
> > "nmu" command of wanna-build to also add an option that allows setting a
> > timestamp, or even let wanna-build generate that timestamp itself (from the
> > time it processes the "nmu" command) and then pass it to sbuild via a
> > not-yet-existing --binNMU-timestamp option?
> 
> Wanna-build has a "State-Change" date:
> 
> wouter@wuiet:~$ wanna-build -A powerpc --info nbd
> nbd:
>   Package             : nbd
>   Version             : 1:3.14-4
>   Builder             : buildd_powerpc-porpora
>   State               : Installed
>   Section             : admin
>   Priority            : source
>   Installed-Version   : 1:3.14-4
>   Previous-State      : Uploaded
>   State-Change        : 2016-11-21 23:13:18.744533
>   Build-time          : 9255
>   CalculatedPri       : 50
>   component           : main
>   Distribution        : sid
>   Notes               : out-of-date
>   Old-Failed          : -------------------- 1:2.9.23-1 --------------------
>     fails test suite
>   State-Days          : 9
>   State-Time          : 835808
>   Success-build-time  : 366
> 
> Why not use that?

I don't know wanna-build but this timestamp seems to be architecture specific
(I see "powerpc" in your paste above)?

Instead, sbuild should be called with the same input timestamp on all
architectures when an nmu is to be built.

Thanks!

cheers, josch

[signature.asc (application/pgp-signature, inline)]

Message #120 received at 843773@bugs.debian.org (full text, mbox, reply):

From: Wouter Verhelst <w@uter.be>

To: Ian Jackson <ijackson@chiark.greenend.org.uk>, 843773@bugs.debian.org, debian-devel@lists.debian.org

Subject: Re: [buildd-tools-devel] Bug#843773: Bug#843773: misleading timestamps in binnmus

Date: Thu, 1 Dec 2016 19:12:47 +0100

On Thu, Dec 01, 2016 at 04:51:39PM +0100, Johannes Schauer wrote:
> Hi,
> 
> Quoting Wouter Verhelst (2016-12-01 16:24:16)
> > On Mon, Nov 14, 2016 at 06:10:57PM +0100, Johannes Schauer wrote:
> > > But maybe to talk about this option: what would speak against changing the
> > > "nmu" command of wanna-build to also add an option that allows setting a
> > > timestamp, or even let wanna-build generate that timestamp itself (from the
> > > time it processes the "nmu" command) and then pass it to sbuild via a
> > > not-yet-existing --binNMU-timestamp option?
> > 
> > Wanna-build has a "State-Change" date:
> > 
> > wouter@wuiet:~$ wanna-build -A powerpc --info nbd
> > nbd:
> >   Package             : nbd
> >   Version             : 1:3.14-4
> >   Builder             : buildd_powerpc-porpora
> >   State               : Installed
> >   Section             : admin
> >   Priority            : source
> >   Installed-Version   : 1:3.14-4
> >   Previous-State      : Uploaded
> >   State-Change        : 2016-11-21 23:13:18.744533
> >   Build-time          : 9255
> >   CalculatedPri       : 50
> >   component           : main
> >   Distribution        : sid
> >   Notes               : out-of-date
> >   Old-Failed          : -------------------- 1:2.9.23-1 --------------------
> >     fails test suite
> >   State-Days          : 9
> >   State-Time          : 835808
> >   Success-build-time  : 366
> > 
> > Why not use that?
> 
> I don't know wanna-build but this timestamp seems to be architecture specific
> (I see "powerpc" in your paste above)?
> 
> Instead, sbuild should be called with the same input timestamp on all
> architectures when an nmu is to be built.

Hmm, yes. That doesn't fit then.

-- 
< ron> I mean, the main *practical* problem with C++, is there's like a dozen
       people in the world who think they really understand all of its rules,
       and pretty much all of them are just lying to themselves too.
 -- #debian-devel, OFTC, 2016-02-12

Message #127 received at 843773-close@bugs.debian.org (full text, mbox, reply):

From: Johannes Schauer <josch@debian.org>

To: 843773-close@bugs.debian.org

Subject: Bug#843773: fixed in sbuild 0.73.0-1

Date: Sat, 24 Dec 2016 01:50:15 +0000

Source: sbuild
Source-Version: 0.73.0-1

We believe that the bug you reported is fixed in the latest version of
sbuild, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 843773@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Johannes Schauer <josch@debian.org> (supplier of updated sbuild package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Sat, 24 Dec 2016 02:28:48 +0100
Source: sbuild
Binary: libsbuild-perl sbuild buildd
Architecture: source
Version: 0.73.0-1
Distribution: unstable
Urgency: medium
Maintainer: Debian buildd-tools Developers <buildd-tools-devel@lists.alioth.debian.org>
Changed-By: Johannes Schauer <josch@debian.org>
Description:
 buildd     - Daemon for automatically building Debian binary packages from Deb
 libsbuild-perl - Library for building Debian binary packages from Debian sources
 sbuild     - Tool for building Debian binary packages from Debian sources
Closes: 820977 842057 842138 842281 842705 842947 843622 843692 843694 843773 843933 844457 844583
Changes:
 sbuild (0.73.0-1) unstable; urgency=medium
 .
   * New upstream release
   * man/sbuild.1.in: clarify docs for --append-to-version and --make-binNMU
     (closes: #844583)
   * Allow passing a directory to --extra-package which will add all .deb inside
     (closes: #844457)
   * Unconditionally update repositories given via --extra-repository
     (closes: #842281)
   * For binNMUs, instead of copying the timestamp of the last changelog entry,
     generate a new one (closes: #843773)
   * Output build type in buildlog in the format used by dpkg-buildpackage
     --build (closes: #820977)
   * Install extra packages into their own internal repository which is created
     at resolver creation time so that extra packages are considered during the
     upgrade step (closes: #843694)
   * Also output .buildinfo in buildlog (closes: #843933)
   * pass the absolute path of extra .deb package to dpkg-deb (closes: #843692)
   * Always pass the source package to lintian (closes: #843622)
   * Do not print negative used space but n/a instead (closes: #842947)
   * bin/sbuild-apt: pass --yes to apt operations as the apt operations are not
     interactive (closes: #842138)
   * Do not set 'Changes File' in copy_changes to avoid it being set to the
     source-only changes file (closes: #842705)
   * starting-build-command and finished-build-commands are now run as root.
   * The autopkgtest-virt backends are no longer supplied with the chroot name.
   * Add SBUILD_BUILD_ARCH percent escape.
   * Use Dpkg::Build::Info::get_build_env_whitelist() to generate the default of
     ENVIRONMENT_FILTER.
   * Add the --binNMU-timestamp command line option to allow one to pass a
     custom timestamp for the new binNMU changelog entry.
   * Add option --binNMU-changelog which allows one to pass a complete
     debian/changelog entry for binary-only builds.
   * The autopkgtest backend now runs into its own process group.
   * Bump sbuild Depends on libdpkg-perl to at least 1.18.14 because of its use
     of Dpkg::Build::Info::get_build_env_whitelist()
   * Move most Build-Depends to Build-Depends-Indep
   * Drop dependency and build dependency on libio-zlib-perl which is provided
     by perl-modules even in old-stable
   * Drop dependency on apt-utils because sbuild does not use apt-ftparchive
     anymore
   * Do not let libsbuild-perl depend on adduser, dctrl-tools and devscripts as
     it doesn't use them.
   * Make schroot and autopkgtest Recommends as they are both working backends
   * Run dh_installinit with --no-start and --no-restart-on-upgrade so that the
     buildd daemon does not get started on installation (closes: #842057)
   * Update autopkgtest
       - Don't sign internal dummy repository anymore
       - Download source package inside the build chroot because the host might
         not be having the requested release in its sources.list
       - Do not depend on procenv build depends as they should be installed
         inside the build chroot and not on the host
       - use "apt install" instead of "dpkg -i" to also install dependencies
       - Several tiny fixes to make the test suite succeed again
   * Make schroot a hard dependency of buildd (it requires
     /etc/schroot/chroot.d/ in its postinst script)
   * Add patch to remove DCMD config value which is not used and just uselessly
     making libsbuild-perl depend on devscripts
   * Add kmod to Suggests of sbuild because modprobe is optionally used by
     sbuild-createchroot
   * Add patch to use dpkg-query instead of grep-status so that dependency on
     grep-dctrl is not needed anymore
   * Add patch that suppresses schroot writing to stderr in sbuild-createchroot
   * Add patch that checks for modprobe in sbuild-createchroot
   * Remove xs-testsuite header from debian/control
   * Adapt libsbuild-perl short description to not say "Tool" anymore but
     "Library"
   * Replace git:// in Vcs-Git field by https://
   * Bump Standard-Version to 3.9.8 (no changes required)
   * Convert debian/copyright to DEP-5 (make machine readable)
Checksums-Sha1:
 dba1eb1215bb0bd4eaa71e14793ad015cd6cf364 2349 sbuild_0.73.0-1.dsc
 963491282ced901f05036b04a213fa7d231bc36f 194172 sbuild_0.73.0.orig.tar.xz
 833a704fbdf9ec3078366ef741c32815eacc3cbe 49720 sbuild_0.73.0-1.debian.tar.xz
Checksums-Sha256:
 08534b08d4fa74d712a48ad1c8b1221c0bf1db1cd62365606a02b296ed86450c 2349 sbuild_0.73.0-1.dsc
 b4875bc02307b74274d5d80bd5552136ff507bdfbf42556f42568d73b1564a37 194172 sbuild_0.73.0.orig.tar.xz
 ccfd6236794b714ae376ed5a30e91a3174e93afcfe7bae890df81974f0ab83a2 49720 sbuild_0.73.0-1.debian.tar.xz
Files:
 e6f1ecf6dfa228fb7d5c77d3b2bc63e6 2349 devel extra sbuild_0.73.0-1.dsc
 954b235ede41aa8df9cf46db88b0ff13 194172 devel extra sbuild_0.73.0.orig.tar.xz
 9dc878b6e2e70e2ea46be550b3d5f4fc 49720 devel extra sbuild_0.73.0-1.debian.tar.xz

-----BEGIN PGP SIGNATURE-----

iQIuBAEBCAAYBQJYXc+iERxqb3NjaEBkZWJpYW4ub3JnAAoJEPLLpcePvYPhB54P
/2EddOPnApDQFmrVB3FR9/QESqoccgxXztbI2iJd3ywbta5qVjngXz+IGC+OW/JO
p6dqV+r24qMDoWRlyoaS/9LTnVEZfPYwKth0xX/ue5mV85EaZrv1Y+uDiQD37Z1E
eafrpRMExnx/C2KgObCVuX4QnI9q9r/ig6ZQild0taHIcjCegKyj4J7eqxr8TJHm
esRvOXzXiUB/Y8DTpOgt+ec0so0wTvwbU3GzGiIHMsQzv/+GJo3NRB7H7s2UQBZN
8J9RTKSqLO/jBmTNuiObw73jz7vvhfXxHmTHpVvDaSj1ZZJjIClX+djNLEBDuYBP
giPnCauVy0htvy9DKhsLJmc03wT9UPdHeytYVNrCswpJhyfObGqdXfTQtzplzt7h
5tqEFUxBcpmJvxkHkwgaPL+XnCv1WI3EDyJoRg6Vgqmzu43+0G32+9JHywYvo8lw
GjTfETZZWYUsqfnj2asm4r0EK6tCep7IyF2cVmG3eQOPsKpefVsqH3oqTepatWce
9mU+QUQQzjNwtLXWRElxTFX3+N+E6uMML8Q4+5sm5CAE08HDSYt7Hk5SPjJ+vlXC
NIkF+z2xidC+P9UYMadNLj4ORfCczvSfDLbJyG+qOxP0YZhUxaFVDmIhlIc914Oe
HWK4T4/VLwZJ3Vp/LZPwGCKNX3Y+KlOwOporjlQTWsTX
=PvQd
-----END PGP SIGNATURE-----

Message #132 received at 843773@bugs.debian.org (full text, mbox, reply):

From: Guillem Jover <guillem@debian.org>

To: Ian Jackson <ijackson@chiark.greenend.org.uk>

Cc: Johannes Schauer <josch@debian.org>, 843773@bugs.debian.org, debian-devel@lists.debian.org

Subject: Re: [buildd-tools-devel] Bug#843773: Bug#843773: misleading timestamps in binnmus

Date: Sun, 1 Jan 2017 17:40:44 +0100

[ Had this half-drafted, but had not found the time to finish it up
  until now. ]

Hi!

On Mon, 2016-11-14 at 13:52:18 +0000, Ian Jackson wrote:
> Johannes Schauer writes ("Re: [buildd-tools-devel] Bug#843773: Bug#843773: misleading timestamps in binnmus"):
> > Instead, file conflicts might be created from files with
> > content that depends on SOURCE_DATE_EPOCH.
> 
> tl;dr:
>    Analysis.  Revised proposal:
>    Introduce BUILD_DATE_EPOCH (= time of sbuild binnmu invocation)
>    and use it for file timestamps (only (for now))

Thanks for the analysis. Although I see few problems with it.

* binNMUs are not co-installable anyway, and although I've got code
  to make them so on the dpkg side, other parties who have to deal
  with dependency resolution are less than enthused on the prospect
  of having to cover that new invariant.
* Even if we made binNMUs co-installable, they would contain files
  with different metadata, which is currently not considered when
  checking for the same-ness of the files, but I think it does
  make sense to consider in the future.
* Even if we didn't consider the metadata part of the criteria for
  same-ness, it still will need to be stored in the dpkg db, but the
  filesystem is a shared resource and only one of the instances can
  be present so the divergent metadata will make verification somewhat
  worthless, or way more complex to implement or reason about.
* Even if any of the above are ignored/solved somehow, the binNMUs
  will contain different metadata, which means you could end up
  with timestamps on disk going back and forth in time for the same
  content. Say you install one of each on several consecutive days
  libfoo_1+b2_arch-a, libfoo_1+b1_arch-b and libfoo_1+b3_arch-c,
  which might also upset some backup solutions.

And this would be yet another reason why allowing coexistence of
Multi-Arch:same and binNMUs that are not in lockstep for all
architectures would be a bad idea.

In any case the concept of binNMUs as being pure rebuilds is simply
an illusion. They are sourceful updates where the source is discarded,
and not only is the environment they are built against different, their
contents will be at least different just because of the different
version and different changelog entry.


So, yes, again IMO Multi-Arch:same plus ref-counted files are broken by
design, and they are pretty much incompatible with binNMUs. I stated
this in the monumental multiarch thread here some years ago. There
was rough consensus that this was either not a problem or one where
its benefits outweighted the potential problems. Fine, but then do not
expect miracles out of the current situation we got into. :)


So, I think this has been already implemented in sbuild and pbuilder,
but indeed, my proposal would be for now to ignore all this, and just
emit a changelog entry with the current build date for each binNMU.

When combined with the M-A:same and ref-counted files requirements, this
*is* still conceptually and practically wrong, because binNMUs are not
triggered for all architectures in lockstep, and are not triggered for
the same reasons (a b1_arch-y might not exist for the same reasons a
b1_arch-z).

The only correct "solution" I see while keeping the current mess, would
be to declare binNMU versions a globally shared resource across all
architectures (in and out of archive!), trigger them globally for all
architectures (or replay them for late comers), and use the binNMU
trigger date for the changelog entry (and ideally also the email address
of the person or role who triggered the binNMU, instead of the buildd
doing the build).

(Well the *only* correct solution would be IMO to ban ref-counted files,
but I don't think that will gain much favor this time around either,
although I think I might make it possible to disable it at dpkg build
time for those downstreams that do not want anything to do with this
insanity. :)

Thanks,
Guillem

Message #137 received at 843773@bugs.debian.org (full text, mbox, reply):

From: Adam Borowski <kilobyte@angband.pl>

To: 843773@bugs.debian.org, debian-devel@lists.debian.org

Subject: Re: [buildd-tools-devel] Bug#843773: Bug#843773: misleading timestamps in binnmus

Date: Sun, 1 Jan 2017 20:03:38 +0100

On Sun, Jan 01, 2017 at 05:40:44PM +0100, Guillem Jover wrote:
> The only correct "solution" I see while keeping the current mess, would
> be to declare binNMU versions a globally shared resource across all
> architectures (in and out of archive!), trigger them globally for all
> architectures (or replay them for late comers)

As someone who tried to make unofficial jessie for x32, I say: hell yeah,
oh so much please do this!  Or better yet, just drop that concept altogether
and instead make binNMUs automated _sourceful_ uploads.

For someone who's trying to simulate testing/stable at home, inconsistent
binNMU versions create such a mess that, despite jessie-x32 being mostly
done, you didn't hear it announced, I didn't bother to build security
updates as planned, it's a bitch to upgrade to unstable, and I'm not trying
that again for stretch.

So here's another reason for your idea, and why binNMUs are bad.


Meow!
-- 
Autotools hint: to do a zx-spectrum build on a pdp11 host, type:
  ./configure --host=zx-spectrum --build=pdp11

Send a report that this bug log contains spam.