Debian Bug report logs - #845194
amd64-microcode: please make the early initramfs image reproducible

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>

To: submit@bugs.debian.org

Subject: amd64-microcode: please make the early initramfs image reproducible

Date: Mon, 21 Nov 2016 12:18:06 +0100

[Message part 1 (text/plain, inline)]

Source: amd64-microcode
Version: 3.20160316.2
Severity: wishlist
Tags: patch
User: reproducible-builds@lists.alioth.debian.org
Usertags: timestamps fileordering toolchain
X-Debbugs-Cc: reproducible-bugs@lists.alioth.debian.org

Hi,

Whilst working on the Reproducible Builds effort [0] on behalf of the
Tails operating system [1], I noticed that amd64-microcode generates
a prepended initramfs image that is not reproducible.

Patch attached.

 [0] https://reproducible-builds.org/
 [1] https://tails.boum.org/


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

[amd64-microcode.diff.txt (text/plain, attachment)]

Message #10 received at 845194@bugs.debian.org (full text, mbox, reply):

From: Henrique de Moraes Holschuh <hmh@debian.org>

To: Chris Lamb <lamby@debian.org>, 845194@bugs.debian.org

Subject: Re: Bug#845194: amd64-microcode: please make the early initramfs image reproducible

Date: Mon, 21 Nov 2016 12:18:15 -0200

On Mon, Nov 21, 2016, at 09:18, Chris Lamb wrote:
> Whilst working on the Reproducible Builds effort [0] on behalf of the
> Tails operating system [1], I noticed that amd64-microcode generates
> a prepended initramfs image that is not reproducible.

So far so good, but...

> Patch attached.

It depends on SOURCE_DATE_EPOCH.  Is that going to be appropriately set
in every relevant situation?

Because the early initramfs is going to be (re?)generated at package
*install* / upgrade time.  So, it is not only at package build time...

Also, if the admin calls "update-initramfs", it will re-create the early
initramfs, and SOURCE_DATE_EPOCH is not going to be appropriately set at
that time.

For the intel-microcode package, I changed iucode-tool (v2.1 and later)
to set the dates to the latest microcode included in the image, which
makes it always reproducible as long as the same set of microcode is
used.  Unfortunately, this is not so trivial to do for amd64-microcode,
because we don't parse the microcode.

Maybe it it would be better to hard-code the value of SOURCE_DATE_EPOCH
at package build time into the hook script?  It would then *always* use
that date to create any early initramfs...

-- 
  Henrique de Moraes Holschuh <hmh@debian.org>

Message #15 received at 845194@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>

To: Henrique de Moraes Holschuh <hmh@debian.org>, 845194@bugs.debian.org

Subject: Re: Bug#845194: amd64-microcode: please make the early initramfs image reproducible

Date: Mon, 21 Nov 2016 16:03:02 +0100

Hi Henrique,

> Because the early initramfs is going to be (re?)generated at package
> *install* / upgrade time.  So, it is not only at package build time...

Yep that's fine; I am attempting to make a reproducible live system and
it will be exported during the live build system. :)

> Also, if the admin calls "update-initramfs", it will re-create the early
> initramfs, and SOURCE_DATE_EPOCH is not going to be appropriately set at
> that time.

Indeed, but then they have a running system so all bets are off.
 
> For the intel-microcode package, I changed iucode-tool (v2.1 and later)
> to set the dates to the latest microcode included in the image, which
> makes it always reproducible

(Curiously, whilst I see this code, for some reason my images. I'm sure
there is another cause of this however. Would followup in a different
bug if appropriate)


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

Message #20 received at 845194@bugs.debian.org (full text, mbox, reply):

From: Henrique de Moraes Holschuh <hmh@debian.org>

To: Chris Lamb <lamby@debian.org>

Cc: 845194@bugs.debian.org

Subject: Re: Bug#845194: amd64-microcode: please make the early initramfs image reproducible

Date: Thu, 24 Nov 2016 15:39:17 -0200

[Message part 1 (text/plain, inline)]

Chris,

Please test the attached patch.  Does it pass all the reproducibility
testing?

-- 
  Henrique Holschuh

[amd64-ucode_reproducible.patch (text/x-diff, attachment)]

Message #27 received at 845194@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>

To: Henrique de Moraes Holschuh <hmh@debian.org>

Cc: 845194@bugs.debian.org

Subject: Re: Bug#845194: amd64-microcode: please make the early initramfs image reproducible

Date: Thu, 24 Nov 2016 23:20:19 +0100

Henrique de Moraes Holschuh wrote:

> Please test the attached patch.  Does it pass all the reproducibility
> testing?

This is not actually tested in Debian's Reproducible Builds testing
framework — I discovered it when working directly on Tails. I thus
can't give you an answer straight away.

I will get back to this in a few days after I recreate my testcase.


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

Message #32 received at 845194@bugs.debian.org (full text, mbox, reply):

From: Holger Levsen <holger@layer-acht.org>

To: Henrique de Moraes Holschuh <hmh@debian.org>

Cc: 845194@bugs.debian.org

Subject: Re: Bug#845194: amd64-microcode: please make the early initramfs image reproducible

Date: Fri, 25 Nov 2016 14:52:03 +0000

[Message part 1 (text/plain, inline)]

On Thu, Nov 24, 2016 at 11:20:19PM +0100, Chris Lamb wrote:
> > Please test the attached patch.  Does it pass all the reproducibility
> > testing?
> This is not actually tested in Debian's Reproducible Builds testing
> framework — I discovered it when working directly on Tails.

to explain: we only test main, as using and or even building packages
from contrib or non-free might be problematic.

building only specifically whitelisted packages from contrib+non-free
would be doable though.

We do take patches however. (Even those with whitelists such simple as
WHITELIST_NON-FREE="package1 package2…" in the code… the hard part is
probably rather adding these extra packages/suites to the web ui…)


-- 
cheers,
	Holger

[signature.asc (application/pgp-signature, inline)]

Message #37 received at 845194@bugs.debian.org (full text, mbox, reply):

From: Henrique de Moraes Holschuh <hmh@debian.org>

To: Holger Levsen <holger@layer-acht.org>, 845194@bugs.debian.org

Subject: Re: Bug#845194: amd64-microcode: please make the early initramfs image reproducible

Date: Fri, 25 Nov 2016 14:56:24 -0200

On Fri, 25 Nov 2016, Holger Levsen wrote:
> On Thu, Nov 24, 2016 at 11:20:19PM +0100, Chris Lamb wrote:
> > > Please test the attached patch.  Does it pass all the reproducibility
> > > testing?
> > This is not actually tested in Debian's Reproducible Builds testing
> > framework — I discovered it when working directly on Tails.
> 
> to explain: we only test main, as using and or even building packages
> from contrib or non-free might be problematic.

Well, FWIW, intel-microcode and amd64-microcode have been manually
checked to be safe to autobuild (and tagged as such, XS-autobuild: yes).
In fact, they're autobuilt in unstable.

iucode-tool (a build-dep of intel-microcode) is FLOSS (GPL2+), and only
in contrib because it basically exists to manipulate non-free data.  It
is also autobuilt.  And I am its upstream.

> building only specifically whitelisted packages from contrib+non-free
> would be doable though.

I would be happy enough if someone would check the patch in this bug
report for amd64-microcode *once*.  It is not like there is much churn
in the build tooling of the microcode packages...

> We do take patches however. (Even those with whitelists such simple as
> WHITELIST_NON-FREE="package1 package2…" in the code… the hard part is
> probably rather adding these extra packages/suites to the web ui…)

Well, FWIW :-)

WHITELIST_CONTRIB="iucode-tool"
WHITELIST_NON-FREE="intel-microcode amd64-microcode"

pretty please?

-- 
  Henrique Holschuh

Message #42 received at 845194@bugs.debian.org (full text, mbox, reply):

From: Holger Levsen <holger@layer-acht.org>

To: Henrique de Moraes Holschuh <hmh@debian.org>, 845194@bugs.debian.org

Cc: Reproducible Builds discussion list <reproducible-builds@lists.alioth.debian.org>

Subject: Re: Bug#845194: amd64-microcode: please make the early initramfs image reproducible

Date: Sat, 26 Nov 2016 09:45:57 +0000

[Message part 1 (text/plain, inline)]

On Fri, Nov 25, 2016 at 02:56:24PM -0200, Henrique de Moraes Holschuh wrote:
> > to explain: we only test main, as using and or even building packages
> > from contrib or non-free might be problematic.
> Well, FWIW, intel-microcode and amd64-microcode have been manually
> checked to be safe to autobuild (and tagged as such, XS-autobuild: yes).
> In fact, they're autobuilt in unstable.

yes, sure, those three packages are fine, but we are only testing main
and not contrib and non-free because *in general* we cannot now whether
we can legally build them. that's why i said I would be fine with a
whitelisting solution… but we dont have that yet.
 
> I would be happy enough if someone would check the patch in this bug
> report for amd64-microcode *once*. 

I believe this is on lamby's agenda…


-- 
cheers,
	Holger

[signature.asc (application/pgp-signature, inline)]

Message #47 received at 845194@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>

To: Holger Levsen <holger@layer-acht.org>, Henrique de Moraes Holschuh <hmh@debian.org>, 845194@bugs.debian.org

Cc: Reproducible Builds discussion list <reproducible-builds@lists.alioth.debian.org>

Subject: Re: Bug#845194: amd64-microcode: please make the early initramfs image reproducible

Date: Sat, 26 Nov 2016 10:55:57 +0100

[Re-adding myself to CC]

Holger Levsen wrote:

> […]

I fear I may have confused the issue here; the source package "amd64-microcode"
is perfectly reproducible, so debating over whether it should be built from a
contrib/non-free/autobuilt point of view is — at the very least — not "on-topic"
for this bug.

> > I would be happy enough if someone would check the patch in this bug
> > report for amd64-microcode *once*. 
> 
> I believe this is on lamby's agenda…

Checking for whether the initramfs images generated *with* amd64-microcode
is indeed on my agenda (~72h).


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

Message #52 received at 845194@bugs.debian.org (full text, mbox, reply):

From: Holger Levsen <holger@layer-acht.org>

To: Chris Lamb <lamby@debian.org>

Cc: Henrique de Moraes Holschuh <hmh@debian.org>, 845194@bugs.debian.org, Reproducible Builds discussion list <reproducible-builds@lists.alioth.debian.org>

Subject: Re: Bug#845194: amd64-microcode: please make the early initramfs image reproducible

Date: Sat, 26 Nov 2016 10:10:13 +0000

[Message part 1 (text/plain, inline)]

On Sat, Nov 26, 2016 at 10:55:57AM +0100, Chris Lamb wrote:
> I fear I may have confused the issue here; the source package "amd64-microcode"
> is perfectly reproducible, so debating over whether it should be built from a
> contrib/non-free/autobuilt point of view is — at the very least — not "on-topic"
> for this bug.
 
ah! thanks for clarifying. (I actually noticed initially but forgot that
on the 2nd or 3rd reply…)


-- 
cheers,
	Holger

[signature.asc (application/pgp-signature, inline)]

Message #57 received at 845194@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>

To: Henrique de Moraes Holschuh <hmh@debian.org>

Cc: 845194@bugs.debian.org, reproducible-builds@lists.alioth.debian.org

Subject: Re: Bug#845194: amd64-microcode: please make the early initramfs image reproducible

Date: Sun, 27 Nov 2016 12:11:44 +0100

Henrique de Moraes Holschuh wrote:

> Please test the attached patch.  Does it pass all the reproducibility
> testing?

Not quite; I needed to make the following change to your patch:

 - CHANGELOG_TS :=$(shell date +%s
 + CHANGELOG_TS :=$(shell date --utc +%s

.. then it was reproducible. :)

FYI to check this yourself you will need to:

 a) Apply my patch in #845034 against initramfs-tools.

 b) Use cpio 2.12 or later, which is not in sid. However, I have backported
    the parts required in #804063.

 c) Export SOURCE_DATE_EPOCH to update-initramfs


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

Message #62 received at 845194@bugs.debian.org (full text, mbox, reply):

From: Henrique de Moraes Holschuh <hmh@debian.org>

To: Chris Lamb <lamby@debian.org>

Cc: 845194@bugs.debian.org, reproducible-builds@lists.alioth.debian.org

Subject: Re: Bug#845194: amd64-microcode: please make the early initramfs image reproducible

Date: Sun, 27 Nov 2016 17:45:02 -0200

On Sun, 27 Nov 2016, Chris Lamb wrote:
> Henrique de Moraes Holschuh wrote:
> > Please test the attached patch.  Does it pass all the reproducibility
> > testing?
> 
> Not quite; I needed to make the following change to your patch:
> 
>  - CHANGELOG_TS :=$(shell date +%s
>  + CHANGELOG_TS :=$(shell date --utc +%s
> 
> .. then it was reproducible. :)

Ah, ok.  Thanks for the review!

> FYI to check this yourself you will need to:
> 
>  a) Apply my patch in #845034 against initramfs-tools.
> 
>  b) Use cpio 2.12 or later, which is not in sid. However, I have backported
>     the parts required in #804063.
> 
>  c) Export SOURCE_DATE_EPOCH to update-initramfs

Good point.  Thanks!

-- 
  Henrique Holschuh

Message #71 received at 845194-close@bugs.debian.org (full text, mbox, reply):

From: Henrique de Moraes Holschuh <hmh@debian.org>

To: 845194-close@bugs.debian.org

Subject: Bug#845194: fixed in amd64-microcode 3.20160316.3

Date: Wed, 30 Nov 2016 19:21:05 +0000

Source: amd64-microcode
Source-Version: 3.20160316.3

We believe that the bug you reported is fixed in the latest version of
amd64-microcode, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 845194@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Henrique de Moraes Holschuh <hmh@debian.org> (supplier of updated amd64-microcode package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.8
Date: Tue, 29 Nov 2016 23:54:53 -0200
Source: amd64-microcode
Binary: amd64-microcode
Architecture: source amd64
Version: 3.20160316.3
Distribution: unstable
Urgency: medium
Maintainer: Henrique de Moraes Holschuh <hmh@debian.org>
Changed-By: Henrique de Moraes Holschuh <hmh@debian.org>
Description:
 amd64-microcode - Processor microcode firmware for AMD CPUs
Closes: 845194
Changes:
 amd64-microcode (3.20160316.3) unstable; urgency=medium
 .
   * initramfs: Make the early initramfs reproducible (closes: #845194)
   * rules: switch to simplified dh-based build (debhelper v9)
Checksums-Sha1:
 03ed905f3859d4c820d2f06fff672dffb0e254ed 1681 amd64-microcode_3.20160316.3.dsc
 495f315ecd044a4f33e3dbdcdd8cf479a3272d63 30360 amd64-microcode_3.20160316.3.tar.xz
 459895af5d51d6148952f93d27c80271aa33929d 4446 amd64-microcode_3.20160316.3_amd64.buildinfo
 f90a3027b515adf397afd3a5facf09c6afe61bab 31116 amd64-microcode_3.20160316.3_amd64.deb
Checksums-Sha256:
 c317333a9bc89ec111e7075b37dd6fc5e8de2ddfd233a223620df92c8ef3fddb 1681 amd64-microcode_3.20160316.3.dsc
 650693df889b95b83ebbc91f7e08cbbb983674d8718dc31fd1ebafd7a76b677f 30360 amd64-microcode_3.20160316.3.tar.xz
 e2199182323db479f98cd1f559eec8acf0d87206687d78cecdded873e3c745d1 4446 amd64-microcode_3.20160316.3_amd64.buildinfo
 f7bddaf712ffaa833ff65ef94bdd86720d55c2c56ae982c3db58181bbe70f147 31116 amd64-microcode_3.20160316.3_amd64.deb
Files:
 e5b438021aa70c0006856f6a0635dd73 1681 non-free/admin standard amd64-microcode_3.20160316.3.dsc
 0dc6fab6a7650cd5866a2ccea7f66b24 30360 non-free/admin standard amd64-microcode_3.20160316.3.tar.xz
 f910c5274abf248894dca37af24243a3 4446 non-free/admin standard amd64-microcode_3.20160316.3_amd64.buildinfo
 7056e449d8bac87d85a4e434379d0e6e 31116 non-free/admin standard amd64-microcode_3.20160316.3_amd64.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCgAGBQJYPjIFAAoJEP4Rv6aLFY6YglEP/3wbJdoqtysRjJCqxOaxqxni
eKNz6D8q9QqpWinOx3PxAf+eTQT6xP8vmTAPvHzOvpmb2fPtLjFeZLVf9yNHKlrG
HUnGLngWvS+FaK4YTuQvfVsDT0N03/t81W6vnRrXTn8GfGTkt/iXGh9nIIGsCEAT
0OQXPFK7wi7QRqjypD+T8gprGWetgC9J+QnwbmCbweLxh2ATwELX/6GExbimbzsx
9cdmKet9UextJoTe0EEL5RqwDRgWPPpD07QigRQiSjTe5AQXh/+rkLHZ6fW8pRRu
Kr09OcfBKmgmHI3uKKBHjAXNX6R/6u+XdPardk/JLNcfzcSbq0ulpNc1aHxHnuB7
efrPSdTYrmRIaZDENG2JTsZIOnZYLu4E5mMmdQS2IkR5PhkN3joSmd/UXkXxi4wK
ztnDJDcRDPTVT04iYMa+mBNd/78hoXRS82L0J1dZpqirL7LH0U2diPaRVm6so56t
9VIslkXhHYKO9sjh+0v+BYqKm7kc2cgQ/YQx0+7mmhuJTMHoRbu+xnc/jbr8zvET
d7sCq03mPXstjLSTjtDD43p+t7B2dMNmDk+xZhjyNyNSBOEMag0FaObXzJQ0fSoq
jddkUJCCqzK4UJE/gKmTOGq6eZtbqAzb2/G+Qp8cMUMTwaE1F0A1bmanJSTLV9ev
LFAIkTOVQcOL2Hpjlofz
=ScmT
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.