Debian Bug report logs - #978499
fop: reproducible builds: Support using SOURCE_DATE_EPOCH for timestamps in PDF files

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Vagrant Cascadian <vagrant@reproducible-builds.org>

To: submit@bugs.debian.org

Subject: fop: reproducible builds: Support using SOURCE_DATE_EPOCH for timestamps in PDF files

Date: Sun, 27 Dec 2020 19:17:22 -0800

[Message part 1 (text/plain, inline)]

Package: fop
Severity: normal
Tags: patch
User: reproducible-builds@lists.alioth.debian.org
Usertags: timestamps toolchain
X-Debbugs-Cc: reproducible-bugs@lists.alioth.debian.org

Several packages use fop to generate PDF files in Debian packages, but
the resulting PDF files have embedding timestamp information in the
CreationDate of the PDF:

  https://tests.reproducible-builds.org/debian/issues/unstable/timestamps_in_pdf_generated_by_apache_fop_issue.html


For example, in xorg-docs:

  https://tests.reproducible-builds.org/debian/rb-pkg/bullseye/amd64/diffoscope-results/xorg-docs.html

  /usr/share/doc/xorg-docs/xlfd/xlfd.pdf.gz
  
  CreationDate:·"D:20201225182038-12'00'"
  vs.
  CreationDate:·"D:20220129025203+14'00'"


The attached patch fixes this by adding support for the
SOURCE_DATE_EPOCH environment variable to fop, which embeds the
specified timestamp rather than the current time:

  https://reproducible-builds.org/docs/source-date-epoch/


Thanks for maintaining fop!


live well,
  vagrant

[0001-PDFInfo.java-Support-SOURCE_DATE_EPOCH-environment-v.patch (text/x-diff, inline)]

From 25826ea9c86d01a8392cf593b9aa93c72b469b19 Mon Sep 17 00:00:00 2001
From: Vagrant Cascadian <vagrant@reproducible-builds.org>
Date: Mon, 28 Dec 2020 02:48:21 +0000
Subject: [PATCH] PDFInfo.java: Support SOURCE_DATE_EPOCH environment variable.

https://reproducible-builds.org/docs/source-date-epoch/
---
 fop-core/src/main/java/org/apache/fop/pdf/PDFInfo.java | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/fop-core/src/main/java/org/apache/fop/pdf/PDFInfo.java b/fop-core/src/main/java/org/apache/fop/pdf/PDFInfo.java
index 3aa5d97..79f3f42 100644
--- a/fop-core/src/main/java/org/apache/fop/pdf/PDFInfo.java
+++ b/fop-core/src/main/java/org/apache/fop/pdf/PDFInfo.java
@@ -305,7 +305,14 @@ public class PDFInfo extends PDFObject {
      * @return the requested String representation
      */
     protected static String formatDateTime(final Date time) {
-        return formatDateTime(time, TimeZone.getDefault());
+        // https://reproducible-builds.org/docs/source-date-epoch/
+        String source_date_epoch = System.getenv("SOURCE_DATE_EPOCH");
+        if (source_date_epoch != null) {
+            Long sourcedate = (1000 * Long.parseLong(source_date_epoch));
+            return formatDateTime(new Date(sourcedate), TimeZone.getTimeZone("Etc/UTC"));
+        } else {
+            return formatDateTime(time, TimeZone.getDefault());
+        }
     }
 
     /**
-- 
2.20.1

[signature.asc (application/pgp-signature, inline)]

Message #12 received at 978499-close@bugs.debian.org (full text, mbox, reply):

From: Debian FTP Masters <ftpmaster@ftp-master.debian.org>

To: 978499-close@bugs.debian.org

Subject: Bug#978499: fixed in fop 1:2.5-2

Date: Tue, 29 Dec 2020 01:49:09 +0000

Source: fop
Source-Version: 1:2.5-2
Done: tony mancill <tmancill@debian.org>

We believe that the bug you reported is fixed in the latest version of
fop, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 978499@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
tony mancill <tmancill@debian.org> (supplier of updated fop package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.8
Date: Mon, 28 Dec 2020 11:42:20 -0800
Source: fop
Architecture: source
Version: 1:2.5-2
Distribution: unstable
Urgency: medium
Maintainer: Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>
Changed-By: tony mancill <tmancill@debian.org>
Closes: 978499
Changes:
 fop (1:2.5-2) unstable; urgency=medium
 .
   * Team upload
   * Use SOURCE_DATE_EPOCH from system environment (Closes: #978499)
     For reproducible builds of packages that use fop as a build-dep.
   * Remove deprecated get-orig-source target
   * Update debian/watch to version 4
   * Bump Standards-Version to 4.5.1
   * Set Rules-Requires-Root: no in debian/control
   * Use debhelper-compat 13
Checksums-Sha1:
 d968e1726e911fcf64f60cdcdff1b8bcfecc4c3d 2609 fop_2.5-2.dsc
 7fc9a28b24bfbc47b3d92323863fc1f3df12c41e 872680 fop_2.5-2.debian.tar.xz
 8711780cff10e8078d33b386bfce8da9602beaca 13562 fop_2.5-2_amd64.buildinfo
Checksums-Sha256:
 5c9b8cbfe28d2f6e4862b85283ecac81b49f076687e3361ee940948ca55bcb66 2609 fop_2.5-2.dsc
 0e832d792bb51e985730893e713638cd1bc7855c55607224724c4573b6f6f0a0 872680 fop_2.5-2.debian.tar.xz
 69943af7d972a6e9333ca3716e114023eef4e94cec24dc9526acc22e1c89915c 13562 fop_2.5-2_amd64.buildinfo
Files:
 ef9d9bb1e163758ce076c62ab5b68e9c 2609 text optional fop_2.5-2.dsc
 438823ef60f8149e1013cc13e7f0a6b4 872680 text optional fop_2.5-2.debian.tar.xz
 7e2f15de7898622a36ac6ca0b3aa3d55 13562 text optional fop_2.5-2_amd64.buildinfo

-----BEGIN PGP SIGNATURE-----

iQJIBAEBCgAyFiEE5Qr9Va3SequXFjqLIdIFiZdLPpYFAl/qhpcUHHRtYW5jaWxs
QGRlYmlhbi5vcmcACgkQIdIFiZdLPpbPpA//dYtYIvugoLzh0zpYshDKjvVpLB0V
4aPJAOGOZ85hbPVsPaY7CnmNtA3BRi0AjEXqggORKsS4K4XJiBJyVT1A+5rlxcU1
XGGuT2CNMrfrdFMdR8AjmG8dPuquapni4rA583FXad/TfbXwS8eqkb4FL1aiyJyi
S20XpPMs3IMqeriiVesxGnLAUp0rmwuKsj51ZLW+STHb1Bk07cqxXkAUWpu5O0aQ
WPipX+ErjgjqXdAXegqoln3LjjaxSjwx0nhg5icryOTSKXmp2ho8iwNi3ue1VR90
vBtMK/oZApVxkC5I4kZtjoaemnPrevHGHz7DO5RogYS5OJKHIUNA0Lkl0xkpKXwe
36xJzFS30cSGnHYj21nKRFGbiufv868su5yRAis/DOt8ok/ZBlj6vvYSdhNMbvLe
aNrzCHZaETDEBVMPrYByYpnvLZc23iksJbYOjQBmBigcgcc2aAxSTCataMvYZoRM
CpVCNnHpBCGtDyAYoPlMsx4K0nkQ4Qmm3ea6wr6T0Xx4E+/qDlL+1geBt5K6FXRE
4UZfBMeLixWQnd6MXGH/a6UsQ8Dn9deX7whne7wBRwSJtOP6RIX3ptwaFBZR939V
TfvQ+cMNKduliiuYcakx48uJe/fo+owgQCQVPx2xE3Iy3OcAXTV0xANWucJGPvv/
JVrGNyX3BkB4DHw=
=PHv/
-----END PGP SIGNATURE-----

Message #17 received at 978499@bugs.debian.org (full text, mbox, reply):

From: Vagrant Cascadian <vagrant@reproducible-builds.org>

To: 978499@bugs.debian.org

Cc: reproducible-bugs@lists.alioth.debian.org

Subject: #978499: fop: reproducible builds: Support using SOURCE_DATE_EPOCH for timestamps in PDF files

Date: Tue, 29 Dec 2020 11:13:48 -0800

[Message part 1 (text/plain, inline)]

Control: notfixed 978499 1:2.5-2

On 2020-12-27, Vagrant Cascadian wrote:
> Several packages use fop to generate PDF files in Debian packages, but
> the resulting PDF files have embedding timestamp information in the
> CreationDate of the PDF:
>
>   https://tests.reproducible-builds.org/debian/issues/unstable/timestamps_in_pdf_generated_by_apache_fop_issue.html

Thanks for the quick upload! unfortunately...


> For example, in xorg-docs:
>
>   https://tests.reproducible-builds.org/debian/rb-pkg/bullseye/amd64/diffoscope-results/xorg-docs.html
>
>   /usr/share/doc/xorg-docs/xlfd/xlfd.pdf.gz
>   
>   CreationDate:·"D:20201225182038-12'00'"
>   vs.
>   CreationDate:·"D:20220129025203+14'00'"

I rescheduled various builds after fop landed in unstable, and it
appears to not fully fix the issue...

It clearly fixed the issue for me when building xorg-docs with reprotest
locally, which does test time and timezone variations... but it uses
faketime, which often behaves differently than a system with an adjusted
running clock such as the tests.reproducible-builds.org infrastructure.

hrm.


> The attached patch fixes this by adding support for the
> SOURCE_DATE_EPOCH environment variable to fop, which embeds the
> specified timestamp rather than the current time:
>
>   https://reproducible-builds.org/docs/source-date-epoch/
>
>
> Thanks for maintaining fop!
>
>
> live well,
>   vagrant
> From 25826ea9c86d01a8392cf593b9aa93c72b469b19 Mon Sep 17 00:00:00 2001
> From: Vagrant Cascadian <vagrant@reproducible-builds.org>
> Date: Mon, 28 Dec 2020 02:48:21 +0000
> Subject: [PATCH] PDFInfo.java: Support SOURCE_DATE_EPOCH environment variable.
>
> https://reproducible-builds.org/docs/source-date-epoch/
> ---
>  fop-core/src/main/java/org/apache/fop/pdf/PDFInfo.java | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
>
> diff --git a/fop-core/src/main/java/org/apache/fop/pdf/PDFInfo.java b/fop-core/src/main/java/org/apache/fop/pdf/PDFInfo.java
> index 3aa5d97..79f3f42 100644
> --- a/fop-core/src/main/java/org/apache/fop/pdf/PDFInfo.java
> +++ b/fop-core/src/main/java/org/apache/fop/pdf/PDFInfo.java
> @@ -305,7 +305,14 @@ public class PDFInfo extends PDFObject {
>       * @return the requested String representation
>       */
>      protected static String formatDateTime(final Date time) {
> -        return formatDateTime(time, TimeZone.getDefault());
> +        // https://reproducible-builds.org/docs/source-date-epoch/
> +        String source_date_epoch = System.getenv("SOURCE_DATE_EPOCH");
> +        if (source_date_epoch != null) {
> +            Long sourcedate = (1000 * Long.parseLong(source_date_epoch));
> +            return formatDateTime(new Date(sourcedate), TimeZone.getTimeZone("Etc/UTC"));
> +        } else {
> +            return formatDateTime(time, TimeZone.getDefault());
> +        }
>      }
>  
>      /**
> -- 
> 2.20.1


Ah well, if anyone has a suggestion for how to really fix it, that would
be nice, since it would fix several packages...


live well,
  vagrant

[signature.asc (application/pgp-signature, inline)]

Message #24 received at 978499@bugs.debian.org (full text, mbox, reply):

From: tony mancill <tmancill@debian.org>

To: Vagrant Cascadian <vagrant@reproducible-builds.org>, 978499@bugs.debian.org

Cc: reproducible-bugs@lists.alioth.debian.org

Subject: Re: Bug#978499: #978499: fop: reproducible builds: Support using SOURCE_DATE_EPOCH for timestamps in PDF files

Date: Wed, 30 Dec 2020 17:45:44 -0800

[Message part 1 (text/plain, inline)]

On Tue, Dec 29, 2020 at 11:13:48AM -0800, Vagrant Cascadian wrote:
> Thanks for the quick upload! unfortunately...
> 
> > For example, in xorg-docs:
> >
> >   https://tests.reproducible-builds.org/debian/rb-pkg/bullseye/amd64/diffoscope-results/xorg-docs.html
> >
> >   /usr/share/doc/xorg-docs/xlfd/xlfd.pdf.gz
> >   
> >   CreationDate:·"D:20201225182038-12'00'"
> >   vs.
> >   CreationDate:·"D:20220129025203+14'00'"
> 
> I rescheduled various builds after fop landed in unstable, and it
> appears to not fully fix the issue...
> 
> It clearly fixed the issue for me when building xorg-docs with reprotest
> locally, which does test time and timezone variations... but it uses
> faketime, which often behaves differently than a system with an adjusted
> running clock such as the tests.reproducible-builds.org infrastructure.

Hrm indeed...

For what it's worth, the diffoscope for bullseye (which doesn't have the
fix for fop in there yet) and unstable look different to me.  In
bullseye, the "CreationDate" in the differs, as expected.  But in
unstable, the difference is in CreateDate in the XML metadata about the
file.

I think it's possible that we are falling into this block of
PDFMetadata.java [1]:

        //Set creation date if not available, yet
        if (info.getCreationDate() == null) {
            Date d = new Date();
            info.setCreationDate(d);
        }

That would fit the symptoms.  In any event, in for a penny, in for a pound.  I think we can fix this by checking for null creationDate in PDFInfo.java [2] and once again using SOURCE_DATE_EPOCH if set.

[1] https://salsa.debian.org/java-team/fop/-/blob/master/fop-core/src/main/java/org/apache/fop/pdf/PDFMetadata.java#L135-139
[2] https://salsa.debian.org/java-team/fop/-/blob/master/fop-core/src/main/java/org/apache/fop/pdf/PDFInfo.java#L190-195

I have pushed patch to wrap the original modification to PDFInfo.java in
a try/catch but haven't yet uploaded.  I'll amend that and I do a little
reprotesting before uploading again.  

> Ah well, if anyone has a suggestion for how to really fix it, that would
> be nice, since it would fix several packages...

I should have something up tomorrow. 

Cheers,
tony

[signature.asc (application/pgp-signature, inline)]

Message #29 received at 978499@bugs.debian.org (full text, mbox, reply):

From: Vagrant Cascadian <vagrant@reproducible-builds.org>

To: tony mancill <tmancill@debian.org>, 978499@bugs.debian.org

Subject: Re: Bug#978499: fop: reproducible builds: Support using SOURCE_DATE_EPOCH for timestamps in PDF files

Date: Wed, 30 Dec 2020 18:23:29 -0800

[Message part 1 (text/plain, inline)]

On 2020-12-30, tony mancill wrote:
> On Tue, Dec 29, 2020 at 11:13:48AM -0800, Vagrant Cascadian wrote:
>> Thanks for the quick upload! unfortunately...
>> 
>> > For example, in xorg-docs:
>> >
>> >   https://tests.reproducible-builds.org/debian/rb-pkg/bullseye/amd64/diffoscope-results/xorg-docs.html
>> >
>> >   /usr/share/doc/xorg-docs/xlfd/xlfd.pdf.gz
>> >   
>> >   CreationDate:·"D:20201225182038-12'00'"
>> >   vs.
>> >   CreationDate:·"D:20220129025203+14'00'"
>> 
>> I rescheduled various builds after fop landed in unstable, and it
>> appears to not fully fix the issue...
>> 
>> It clearly fixed the issue for me when building xorg-docs with reprotest
>> locally, which does test time and timezone variations... but it uses
>> faketime, which often behaves differently than a system with an adjusted
>> running clock such as the tests.reproducible-builds.org infrastructure.
>
> Hrm indeed...
>
> For what it's worth, the diffoscope for bullseye (which doesn't have the
> fix for fop in there yet) and unstable look different to me.  In
> bullseye, the "CreationDate" in the differs, as expected.  But in
> unstable, the difference is in CreateDate in the XML metadata about the
> file.
>
> I think it's possible that we are falling into this block of
> PDFMetadata.java [1]:
>
>         //Set creation date if not available, yet
>         if (info.getCreationDate() == null) {
>             Date d = new Date();
>             info.setCreationDate(d);
>         }
>
> That would fit the symptoms.  In any event, in for a penny, in for a pound.  I think we can fix this by checking for null creationDate in PDFInfo.java [2] and once again using SOURCE_DATE_EPOCH if set.
>
> [1] https://salsa.debian.org/java-team/fop/-/blob/master/fop-core/src/main/java/org/apache/fop/pdf/PDFMetadata.java#L135-139
> [2] https://salsa.debian.org/java-team/fop/-/blob/master/fop-core/src/main/java/org/apache/fop/pdf/PDFInfo.java#L190-195
>
> I have pushed patch to wrap the original modification to PDFInfo.java in
> a try/catch but haven't yet uploaded.  I'll amend that and I do a little
> reprotesting before uploading again.  

Thanks for continuing to dive into this one! :)


Maybe this is a red herring, but I also noticed that in PDFInfo.java
there are two definitions of the modified function with the same name...

    /**
     * Formats a date/time according to the PDF specification
    (D:YYYYMMDDHHmmSSOHH'mm').
     * @param time date/time value to format
     * @param tz the time zone
     * @return the requested String representation
     */
    protected static String formatDateTime(final Date time, TimeZone tz)
    {
        return DateFormatUtil.formatPDFDate(time, tz);
    }

    /**
     * Formats a date/time according to the PDF
    specification. (D:YYYYMMDDHHmmSSOHH'mm').
     * @param time date/time value to format
     * @return the requested String representation
     */
    protected static String formatDateTime(final Date time) {
        return formatDateTime(time, TimeZone.getDefault());
    }


Or is there some java thing to handle functions with the same names?



live well,
  vagrant

[signature.asc (application/pgp-signature, inline)]

Message #34 received at 978499@bugs.debian.org (full text, mbox, reply):

From: tony mancill <tmancill@debian.org>

To: Vagrant Cascadian <vagrant@reproducible-builds.org>

Cc: 978499@bugs.debian.org

Subject: Re: Bug#978499: fop: reproducible builds: Support using SOURCE_DATE_EPOCH for timestamps in PDF files

Date: Wed, 30 Dec 2020 23:02:28 -0800

On Wed, Dec 30, 2020 at 06:23:29PM -0800, Vagrant Cascadian wrote:
> On 2020-12-30, tony mancill wrote:
> > On Tue, Dec 29, 2020 at 11:13:48AM -0800, Vagrant Cascadian wrote:
> >> Thanks for the quick upload! unfortunately...
> >> 
> >> > For example, in xorg-docs:
> >> >
> >> >   https://tests.reproducible-builds.org/debian/rb-pkg/bullseye/amd64/diffoscope-results/xorg-docs.html
> >> >
> >> >   /usr/share/doc/xorg-docs/xlfd/xlfd.pdf.gz
> >> >   
> >> >   CreationDate:·"D:20201225182038-12'00'"
> >> >   vs.
> >> >   CreationDate:·"D:20220129025203+14'00'"
> >> 
> >> I rescheduled various builds after fop landed in unstable, and it
> >> appears to not fully fix the issue...
> >> 
> >> It clearly fixed the issue for me when building xorg-docs with reprotest
> >> locally, which does test time and timezone variations... but it uses
> >> faketime, which often behaves differently than a system with an adjusted
> >> running clock such as the tests.reproducible-builds.org infrastructure.
> >
> > Hrm indeed...
> >
> > For what it's worth, the diffoscope for bullseye (which doesn't have the
> > fix for fop in there yet) and unstable look different to me.  In
> > bullseye, the "CreationDate" in the differs, as expected.  But in
> > unstable, the difference is in CreateDate in the XML metadata about the
> > file.
> >
> > I think it's possible that we are falling into this block of
> > PDFMetadata.java [1]:
> >
> >         //Set creation date if not available, yet
> >         if (info.getCreationDate() == null) {
> >             Date d = new Date();
> >             info.setCreationDate(d);
> >         }
> >
> > That would fit the symptoms.  In any event, in for a penny, in for a pound.  I think we can fix this by checking for null creationDate in PDFInfo.java [2] and once again using SOURCE_DATE_EPOCH if set.
> >
> > [1] https://salsa.debian.org/java-team/fop/-/blob/master/fop-core/src/main/java/org/apache/fop/pdf/PDFMetadata.java#L135-139
> > [2] https://salsa.debian.org/java-team/fop/-/blob/master/fop-core/src/main/java/org/apache/fop/pdf/PDFInfo.java#L190-195
> >
> > I have pushed patch to wrap the original modification to PDFInfo.java in
> > a try/catch but haven't yet uploaded.  I'll amend that and I do a little
> > reprotesting before uploading again.  
> 
> Thanks for continuing to dive into this one! :)
> 
> Maybe this is a red herring, but I also noticed that in PDFInfo.java
> there are two definitions of the modified function with the same name...
> 
> (snip)
> 
> Or is there some java thing to handle functions with the same names?

Yes, it's a common pattern in Java.  The methods vary in their arguments
and so are distinct signatures.  In this case, the method that takes a
TimeZone as an argument is called by the other method of the same name
in PDFInfo *and* in PDFEmbeddedFile.  So... I went looking for all of
the invocations of new Date() in the fop code and found several other
methods where SOURCE_DATE_EPOCH should be checked.

I have an updated patch for fop that addresses the issue with xorg-docs
and probably a few others too.  I'm going to let ratt chew on the build
r-deps before uploading, but expect to upload tomorrow.

Cheers,
tony

Message #39 received at 978499-close@bugs.debian.org (full text, mbox, reply):

From: Debian FTP Masters <ftpmaster@ftp-master.debian.org>

To: 978499-close@bugs.debian.org

Subject: Bug#978499: fixed in fop 1:2.5-3

Date: Thu, 31 Dec 2020 20:33:44 +0000

Source: fop
Source-Version: 1:2.5-3
Done: tony mancill <tmancill@debian.org>

We believe that the bug you reported is fixed in the latest version of
fop, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 978499@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
tony mancill <tmancill@debian.org> (supplier of updated fop package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.8
Date: Thu, 31 Dec 2020 11:24:15 -0800
Source: fop
Architecture: source
Version: 1:2.5-3
Distribution: unstable
Urgency: medium
Maintainer: Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>
Changed-By: tony mancill <tmancill@debian.org>
Closes: 978499
Changes:
 fop (1:2.5-3) unstable; urgency=medium
 .
   * Team upload
   * Update SOURCE_DATE_EPOCH patch (Closes: #978499)
     - Conditionally use SOURCE_DATA_EPOCH in PDFInfo, PDFMetadata,
       PDFRenderingUtil, and FileIDGenerator classes.
     - Add try/catch logic for unparsable SOURCE_DATE_EPOCH
Checksums-Sha1:
 6c0d7b2a53925ae258cb52b6f7bd93440df4386b 2609 fop_2.5-3.dsc
 70db744862904cbea3b06bb6b0a4f2373b8a89db 873596 fop_2.5-3.debian.tar.xz
 5835a83c20010debb44e333aba90943581837811 13663 fop_2.5-3_amd64.buildinfo
Checksums-Sha256:
 7efd4de14cd7fadc44fd3c1af50df90511264886a24b5e7fb1d3142053c42fc6 2609 fop_2.5-3.dsc
 afcb36fb9ccd573d2bcfc326297b856e6020dc8b2d889c5e470fa81f8a604c05 873596 fop_2.5-3.debian.tar.xz
 03b8dfe873c297ad0c79800a6d9a08aef4d56daa37a3da999351bc03dbae54e9 13663 fop_2.5-3_amd64.buildinfo
Files:
 6794819efc8fbcbd1ca78d8fcbb9078a 2609 text optional fop_2.5-3.dsc
 de3f6fd4d44b0f9fad92f5b48238ca53 873596 text optional fop_2.5-3.debian.tar.xz
 391237548a556c657c413cf1cd47d4eb 13663 text optional fop_2.5-3_amd64.buildinfo

-----BEGIN PGP SIGNATURE-----

iQJIBAEBCgAyFiEE5Qr9Va3SequXFjqLIdIFiZdLPpYFAl/uMR8UHHRtYW5jaWxs
QGRlYmlhbi5vcmcACgkQIdIFiZdLPpZzUBAAlBfyxJLp/Lvx7yUzEeH0h3NCp7ip
u80RL/3Vwf8ADubGLofPGsT97yoPLENkwqDJFGRUnUR6Ijo2knvOO/QLIq5CTIPU
RH/r0ALVJ6VVvLrO2E5t7B+J151RIS9TqeVGjBB9drAEEvbFdSM4MoqW+sfJsJHB
7WEBCk9GnCTVcLaKLs+uih7mNL66c77rUXZRX8dXiyWrlliSssOcMdSqX/x3k4Hh
jBunIugAJK6JHuR//Zcm8pV95pmgdgEuCpSL7YaY/tPZFUHlvUv63OZASaWfp18E
cje4aOzcBPhqKvSLethqlgdnoeeg2XdmRA03RewcKVSA5LrtWFx1G50mLT+SDfvB
rRH3/ikpFfuScVHn/nfWGjNsjLd7osjUuzT+rylVVj3+F8HNuOgydDJj1wMFs0LI
uVAU3Quh7GVMcEyNgEk5bnBAuH6D24cjVecW5YH8OX0UpUHLD89oMEmCI8XAY89d
fbkKmECJBRNpc3RCARkgqotp4qjqje3IrfVxoOJHKnWhXFHuTurq/yQFQdBZPzE1
d2xC4CWWZpejnOn+2QQNvL2blfvp5nL2PbZiFnLOeW8Q/WP44oPhDe1n1zrWOHAP
7G84EsO/lLUK2yrwDFn7O7M9cD5BeuUZ0y87nn54y/BMpWlJZyWh2HR9oFf91G9g
3X4ceMuEnu8ZzIg=
=FReh
-----END PGP SIGNATURE-----

Message #44 received at 978499@bugs.debian.org (full text, mbox, reply):

From: Vagrant Cascadian <vagrant@reproducible-builds.org>

To: 978499@bugs.debian.org

Subject: Re: #978499: fop: reproducible builds: Support using SOURCE_DATE_EPOCH for timestamps in PDF files

Date: Fri, 01 Jan 2021 11:17:46 -0800

[Message part 1 (text/plain, inline)]

On 2020-12-31, Debian Bug Tracking System wrote:
>  fop (1:2.5-3) unstable; urgency=medium
>  .
>    * Team upload
>    * Update SOURCE_DATE_EPOCH patch (Closes: #978499)
>      - Conditionally use SOURCE_DATA_EPOCH in PDFInfo, PDFMetadata,
>        PDFRenderingUtil, and FileIDGenerator classes.
>      - Add try/catch logic for unparsable SOURCE_DATE_EPOCH

Thanks for the follow-up!

It seems so very, very close, xorg-docs now is only varying on the
timezone, but otherwise respecting SOURCE_DATE_EPOCH:

  https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/diffoscope-results/xorg-docs.html


But what really confuses me is that "treeview" is still ignoring
SOURCE_DATE_EPOCH entirely (e.g. timestamps showing up from 2022):

  https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/diffoscope-results/treeview.html


From the output, it looks like both packages are embedding the same type
of values...

I confirmed that both builds actually were using fop version 2.5-3,
according to the build logs. Almost makes me wonder if treeview somehow
has an invalid date in the changelog entry...


live well,
  vagrant

[signature.asc (application/pgp-signature, inline)]

Message #49 received at 978499@bugs.debian.org (full text, mbox, reply):

From: tony mancill <tmancill@debian.org>

To: Vagrant Cascadian <vagrant@reproducible-builds.org>, 978499@bugs.debian.org

Subject: Re: Bug#978499: #978499: fop: reproducible builds: Support using SOURCE_DATE_EPOCH for timestamps in PDF files

Date: Fri, 1 Jan 2021 11:43:10 -0800

[Message part 1 (text/plain, inline)]

On Fri, Jan 01, 2021 at 11:17:46AM -0800, Vagrant Cascadian wrote:
> On 2020-12-31, Debian Bug Tracking System wrote:
> >  fop (1:2.5-3) unstable; urgency=medium
> >  .
> >    * Team upload
> >    * Update SOURCE_DATE_EPOCH patch (Closes: #978499)
> >      - Conditionally use SOURCE_DATA_EPOCH in PDFInfo, PDFMetadata,
> >        PDFRenderingUtil, and FileIDGenerator classes.
> >      - Add try/catch logic for unparsable SOURCE_DATE_EPOCH
> 
> Thanks for the follow-up!
> 
> It seems so very, very close, xorg-docs now is only varying on the
> timezone, but otherwise respecting SOURCE_DATE_EPOCH:
> 
>   https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/diffoscope-results/xorg-docs.html
> 
> 
> But what really confuses me is that "treeview" is still ignoring
> SOURCE_DATE_EPOCH entirely (e.g. timestamps showing up from 2022):
> 
>   https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/diffoscope-results/treeview.html
> 
> 
> From the output, it looks like both packages are embedding the same type
> of values...
> 
> I confirmed that both builds actually were using fop version 2.5-3,
> according to the build logs. Almost makes me wonder if treeview somehow
> has an invalid date in the changelog entry...

In case it helps point out a hole in my testing strategy, I did my local
testing by extracting the xorg-docs source package into 4 different
directories and then building with sbuild, either against "pure" sid or
as below to test fop 2.5-3 before the upload, and then running
diffoscope against the resulting changes files:

sbuild --chroot=sid-amd64-sbuild --extra-package=/path/to/fop_2.5-3_all.deb --extra-package=/path/to/libfop-java_2.5-3_all.deb

For my pre-2.5-3 builds and when comparing 2.5-2 and 2.5-3 builds, I saw
differences like those on reproducible-builds.org.  The 'c' and 'd'
builds are both with fop 2.5-3, but several minutes apart, and only
show:

diffoscope c/xorg-docs_1.7.1-1.2_amd64.changes d/xorg-docs_1.7.1-1.2_amd64.changes
--- c/xorg-docs_1.7.1-1.2_amd64.changes
+++ d/xorg-docs_1.7.1-1.2_amd64.changes
├── Files
│ @@ -1,6 +1,6 @@
│  
│   c3c9468c8de1825668386eb1c8131e4f 1132 doc optional xorg-docs_1.7.1-1.2.dsc
│   f6a6ecca98d411d73492303db3190bca 13250 doc optional xorg-docs_1.7.1-1.2.diff.gz
│   6939769b47ecad2875ae10674ed4db03 84224 doc optional xorg-docs-core_1.7.1-1.2_all.deb
│   9baa141ec6258704be08b8873f8892c9 1160544 doc optional xorg-docs_1.7.1-1.2_all.deb
│ - a09f88c06107005935ff6664eccfe306 7625 doc optional xorg-docs_1.7.1-1.2_amd64.buildinfo
│ + 6664227d398e6954e41b5c4fc468448c 7625 doc optional xorg-docs_1.7.1-1.2_amd64.buildinfo
├── xorg-docs_1.7.1-1.2_amd64.buildinfo
│ ├── Build-Date
│ │ @@ -1 +1 @@
│ │ -Thu, 31 Dec 2020 06:40:14 +0000
│ │ +Thu, 31 Dec 2020 06:42:37 +0000
│ ├── Build-Path
│ │ @@ -1 +1 @@
│ │ -/build/xorg-docs-By7U5v/xorg-docs-1.7.1
│ │ +/build/xorg-docs-kFh9qq/xorg-docs-1.7.1
 

By comparing two separate 2.5-2 builds, I was able to confirm that 2.5-2
only partially addressed the embedded timestamps in the PDFs, but the
diff above looks like what we want, right?

Cheers,
tony

[signature.asc (application/pgp-signature, inline)]

Message #54 received at 978499@bugs.debian.org (full text, mbox, reply):

From: Vagrant Cascadian <vagrant@reproducible-builds.org>

To: tony mancill <tmancill@debian.org>, 978499@bugs.debian.org

Subject: Re: Bug#978499: fop: reproducible builds: Support using SOURCE_DATE_EPOCH for timestamps in PDF files

Date: Fri, 01 Jan 2021 12:17:45 -0800

[Message part 1 (text/plain, inline)]

On 2021-01-01, tony mancill wrote:
> On Fri, Jan 01, 2021 at 11:17:46AM -0800, Vagrant Cascadian wrote:
>> It seems so very, very close, xorg-docs now is only varying on the
>> timezone, but otherwise respecting SOURCE_DATE_EPOCH:
>> 
>>   https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/diffoscope-results/xorg-docs.html
>> 
>> 
>> But what really confuses me is that "treeview" is still ignoring
>> SOURCE_DATE_EPOCH entirely (e.g. timestamps showing up from 2022):
>> 
>>   https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/diffoscope-results/treeview.html
>> 
>> 
>> From the output, it looks like both packages are embedding the same type
>> of values...
>> 
>> I confirmed that both builds actually were using fop version 2.5-3,
>> according to the build logs. Almost makes me wonder if treeview somehow
>> has an invalid date in the changelog entry...
>
> In case it helps point out a hole in my testing strategy, I did my local
> testing by extracting the xorg-docs source package into 4 different
> directories and then building with sbuild, either against "pure" sid or
> as below to test fop 2.5-3 before the upload, and then running
> diffoscope against the resulting changes files:

> sbuild --chroot=sid-amd64-sbuild --extra-package=/path/to/fop_2.5-3_all.deb --extra-package=/path/to/libfop-java_2.5-3_all.deb

You might want to use two different chroots, one with timezone set to
UTC+14 and one with the timezone set to UTC-12. These are the widest
range of timezones actually in use in the real world, although in theory
you could do something crazy like UTC+0 and UTC+26.

Another thing to try would be to do one build in a virtual machine with
the clock adjusted ... e.g. "qemu -rtc 2022-04-01" as well as using a
very different timezone.


> diffoscope c/xorg-docs_1.7.1-1.2_amd64.changes d/xorg-docs_1.7.1-1.2_amd64.changes
> --- c/xorg-docs_1.7.1-1.2_amd64.changes
> +++ d/xorg-docs_1.7.1-1.2_amd64.changes
> ├── Files
> │ @@ -1,6 +1,6 @@
> │  
> │   c3c9468c8de1825668386eb1c8131e4f 1132 doc optional xorg-docs_1.7.1-1.2.dsc
> │   f6a6ecca98d411d73492303db3190bca 13250 doc optional xorg-docs_1.7.1-1.2.diff.gz
> │   6939769b47ecad2875ae10674ed4db03 84224 doc optional xorg-docs-core_1.7.1-1.2_all.deb
> │   9baa141ec6258704be08b8873f8892c9 1160544 doc optional xorg-docs_1.7.1-1.2_all.deb
> │ - a09f88c06107005935ff6664eccfe306 7625 doc optional xorg-docs_1.7.1-1.2_amd64.buildinfo
> │ + 6664227d398e6954e41b5c4fc468448c 7625 doc optional xorg-docs_1.7.1-1.2_amd64.buildinfo
> ├── xorg-docs_1.7.1-1.2_amd64.buildinfo
> │ ├── Build-Date
> │ │ @@ -1 +1 @@
> │ │ -Thu, 31 Dec 2020 06:40:14 +0000
> │ │ +Thu, 31 Dec 2020 06:42:37 +0000
> │ ├── Build-Path
> │ │ @@ -1 +1 @@
> │ │ -/build/xorg-docs-By7U5v/xorg-docs-1.7.1
> │ │ +/build/xorg-docs-kFh9qq/xorg-docs-1.7.1
>  
>
> By comparing two separate 2.5-2 builds, I was able to confirm that 2.5-2
> only partially addressed the embedded timestamps in the PDFs, but the
> diff above looks like what we want, right?

Yes, a diff like that is what we're hoping for!


From what I've been reading, java date objects are internally encoded in
UTC, but when you use the functions to output the date, it applies the
local timezone unless explicitly asked for a different timezone. /o\

Thanks for sharing the pain!


live well,
  vagrant

[signature.asc (application/pgp-signature, inline)]

Message #59 received at 978499@bugs.debian.org (full text, mbox, reply):

From: Vagrant Cascadian <vagrant@reproducible-builds.org>

To: tony mancill <tmancill@debian.org>, 978499@bugs.debian.org

Subject: Re: Bug#978499: fop: reproducible builds: Support using SOURCE_DATE_EPOCH for timestamps in PDF files

Date: Fri, 01 Jan 2021 20:23:20 -0800

[Message part 1 (text/plain, inline)]

On 2021-01-01, Vagrant Cascadian wrote:
> On 2021-01-01, tony mancill wrote:
>> On Fri, Jan 01, 2021 at 11:17:46AM -0800, Vagrant Cascadian wrote:
>>> It seems so very, very close, xorg-docs now is only varying on the
>>> timezone, but otherwise respecting SOURCE_DATE_EPOCH:
>>> 
>>>   https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/diffoscope-results/xorg-docs.html
>>> 
>>> 
>>> But what really confuses me is that "treeview" is still ignoring
>>> SOURCE_DATE_EPOCH entirely (e.g. timestamps showing up from 2022):
>>> 
>>>   https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/diffoscope-results/treeview.html

But it at least fixed one package!

  https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/gpsbabel.html


So that's three different packages with the whole range of ... not
fixed, partly fixed (e.g. timezone), and wholly fixed... :)

Most that are known appear to fall into the former two categories,
unfortunately.


live well,
  vagrant

[signature.asc (application/pgp-signature, inline)]

Send a report that this bug log contains spam.