Debian Bug report logs - #858431
strip-nondeterminism does not normalize Unix ownership from zip archives or .epub files

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Mike Swanson <mikeonthecomputer@gmail.com>

To: submit@bugs.debian.org

Subject: strip-nondeterminism does not normalize Unix ownership from zip archives

Date: Wed, 22 Mar 2017 03:48:02 -0700

Package: strip-nondeterminism
Version: 0.031-1

Zip archives may contain Unix metadata about its member files, including
ownership, mode, and so forth.

strip-nondeterminism fails to correct for ownership, allowing for
archives to be created and maintained with basically arbitrary and
unpredictable UIDs/GIDs, normally the UIDs/GIDs of the user the archive
is being created under.  Example run (starting out with an empty
directory):

chungy@turanga:sn$ fakeroot
root@turanga:sn# mkdir 1 2
root@turanga:sn# touch {1,2}/{root,user}
root@turanga:sn# chown 1000:1001 1/user && chown 1001:1002 2/user
root@turanga:sn# chmod 700 1/root 2/root
root@turanga:sn# zip -qj 1.zip 1/root 1/user && zip -qj 2.zip 2/user
2/root
root@turanga:sn# bsdtar -tvf 1.zip
-rwx------  0 0      0           0 Mar 22 03:44 root
-rw-r--r--  0 1000   1001        0 Mar 22 03:44 user
root@turanga:sn# bsdtar -tvf 2.zip
-rw-r--r--  0 1001   1002        0 Mar 22 03:44 user
-rwx------  0 0      0           0 Mar 22 03:44 root
root@turanga:sn# strip-nondeterminism ?.zip
root@turanga:sn# bsdtar -tvf 1.zip
-rwxr-xr-x  0 0      0           0 Mar 22 03:44 root
-rw-r--r--  0 1000   1001        0 Mar 22 03:44 user
root@turanga:sn# bsdtar -tvf 2.zip
-rwxr-xr-x  0 0      0           0 Mar 22 03:44 root
-rw-r--r--  0 1001   1002        0 Mar 22 03:44 user

What I expect to see, and believe should happen, is all UIDs and GIDs in
the zip archive become 0, owned by root.

Message #10 received at 858431@bugs.debian.org (full text, mbox, reply):

From: Santiago Vila <sanvila@unex.es>

To: Mike Swanson <mikeonthecomputer@gmail.com>, 858431@bugs.debian.org

Subject: Re: Bug#858431: strip-nondeterminism does not normalize Unix ownership from zip archives

Date: Wed, 22 Mar 2017 18:48:29 +0100

On Wed, Mar 22, 2017 at 03:48:02AM -0700, Mike Swanson wrote:

> root@turanga:sn# strip-nondeterminism ?.zip
> root@turanga:sn# bsdtar -tvf 1.zip
> -rwxr-xr-x  0 0      0           0 Mar 22 03:44 root
> -rw-r--r--  0 1000   1001        0 Mar 22 03:44 user
> root@turanga:sn# bsdtar -tvf 2.zip
> -rwxr-xr-x  0 0      0           0 Mar 22 03:44 root
> -rw-r--r--  0 1001   1002        0 Mar 22 03:44 user
> 
> What I expect to see, and believe should happen, is all UIDs and GIDs in
> the zip archive become 0, owned by root.

That would be inconsistent with the current behaviour with tarballs,
which also contain UIDs and GIDs and AFAIK are kept untouched by
strip-nondeterminism.

If those zipfiles are created in the build target of debian/rules,
why not just use "fakeroot tar czvf tarball.tar.gz file1 file2" or
"fakeroot zip zipfile.zip file1 file2"?

Also: What if the zipfile or the tarball comes from the orig.tar.gz
and we don't want to alter it in any way?

Thanks.

Message #15 received at 858431@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>

To: Santiago Vila <sanvila@unex.es>, 858431@bugs.debian.org, Mike Swanson <mikeonthecomputer@gmail.com>

Subject: Re: Bug#858431: strip-nondeterminism does not normalize Unix ownership from zip archives

Date: Wed, 22 Mar 2017 19:29:03 +0000

tags 858431 + wontfix
thanks

Santiago Vila wrote:

> > What I expect to see, and believe should happen, is all UIDs and GIDs in
> > the zip archive become 0, owned by root.
> 
> That would be inconsistent with the current behaviour with tarballs,
> which also contain UIDs and GIDs and AFAIK are kept untouched by
> strip-nondeterminism.

Indeed, and given that we would want the behaviour to be consistent across
archive formats and I think this goes beyond what strip-nondetermism should
do, I am marking this as wontfix.


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

Message #22 received at 858431@bugs.debian.org (full text, mbox, reply):

From: Reiner Herrmann <reiner@reiner-h.de>

To: Chris Lamb <lamby@debian.org>, 858431@bugs.debian.org

Cc: Santiago Vila <sanvila@unex.es>, Mike Swanson <mikeonthecomputer@gmail.com>

Subject: Re: Bug#858431: strip-nondeterminism does not normalize Unix ownership from zip archives

Date: Thu, 30 Mar 2017 19:10:14 +0200

[Message part 1 (text/plain, inline)]

On Wed, Mar 22, 2017 at 07:29:03PM +0000, Chris Lamb wrote:
> > > What I expect to see, and believe should happen, is all UIDs and GIDs in
> > > the zip archive become 0, owned by root.
> > 
> > That would be inconsistent with the current behaviour with tarballs,
> > which also contain UIDs and GIDs and AFAIK are kept untouched by
> > strip-nondeterminism.

tarballs are currently not touched/supported at all by strip-nondeterminism.

> Indeed, and given that we would want the behaviour to be consistent across
> archive formats and I think this goes beyond what strip-nondetermism should
> do, I am marking this as wontfix.

I think it would be more consistent to also normalize UID/GID in zip files,
as this is some non-determinism (that should be stripped).
And strip-nondeterminism currently also normalizes permissions to 755/644,
which is a bit related to UID/GID.

[signature.asc (application/pgp-signature, inline)]

Message #27 received at 858431@bugs.debian.org (full text, mbox, reply):

From: "Chris Lamb" <lamby@debian.org>

To: 858431@bugs.debian.org

Cc: "Mike Swanson" <mikeonthecomputer@gmail.com>

Subject: Re: strip-nondeterminism does not normalize Unix ownership from zip archives

Date: Tue, 12 Mar 2019 12:56:31 -0400

forwarded 858431 https://salsa.debian.org/reproducible-builds/strip-nondeterminism/issues/4
thanks

I've forwarded this upstream here:

  https://salsa.debian.org/reproducible-builds/strip-nondeterminism/issues/4


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

Message #34 received at 858431@bugs.debian.org (full text, mbox, reply):

From: "Chris Lamb" <lamby@debian.org>

To: 858431@bugs.debian.org, 920732@bugs.debian.org, "Mike Swanson" <mikeonthecomputer@gmail.com>

Subject: Re: strip-nondeterminism does not normalize Unix ownership from zip archives or .epub files

Date: Wed, 24 Apr 2019 11:38:14 -0400

forcemerge 858431 920732
retitle 858431 strip-nondeterminism does not normalize Unix ownership from zip archives or .epub files 
tags 858431 - wontfix
thanks

Two changes here:

 * .epub files are "just" .zip files, so merging and retitling to
    match.

 * Unmarking as wontfix; re-reading https://bugs.debian.org/858431#22,
   I believe we should indeed be normalising these.


Best wishes,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

Message #47 received at 858431-submitter@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>

To: 858431-submitter@bugs.debian.org

Subject: Bug #858431 in strip-nondeterminism marked as pending

Date: Sun, 28 Apr 2019 09:56:01 +0000

Control: tag -1 pending

Hello,

Bug #858431 in strip-nondeterminism reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:

https://salsa.debian.org/reproducible-builds/strip-nondeterminism/commit/f40f555085eeb086bfd4ee1fca1012550790a12d

------------------------------------------------------------------------
Workaround Archive::Zip's incorrect handling of the localExtraField field by monkey-patching the accessor methods to always return normalised values. This fixes the normalisation of Unix ownership (uid/gid) within .zip archives, .epub files, etc. (Closes: #858431, reproducible-builds/strip-nondeterminism#4)
------------------------------------------------------------------------

(this message was generated automatically)
-- 
Greetings

https://bugs.debian.org/858431

Message #52 received at 858431@bugs.debian.org (full text, mbox, reply):

From: "Chris Lamb" <lamby@debian.org>

To: 858431@bugs.debian.org, 920732@bugs.debian.org, "Holger Levsen" <holger@debian.org>, "Reiner Herrmann" <reiner@reiner-h.de>, "Santiago Vila" <sanvila@unex.es>

Cc: "Mike Swanson" <mikeonthecomputer@gmail.com>

Subject: Re: strip-nondeterminism does not normalize Unix ownership from zip archives or .epub files

Date: Sun, 28 Apr 2019 05:58:14 -0400

tags 858431 + pending
thanks

This is fixed in Git, pending upload:

  https://salsa.debian.org/reproducible-builds/strip-nondeterminism/commit/f40f555085eeb086bfd4ee1fca1012550790a12d

  Makefile.PL                                  |  1 +
  lib/File/StripNondeterminism/handlers/zip.pm | 31 +++++++++++++++++++++++-----
  2 files changed, 27 insertions(+), 5 deletions(-)


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

Message #55 received at 858431-submitter@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>

To: 858431-submitter@bugs.debian.org

Subject: Bug #858431 in strip-nondeterminism marked as pending

Date: Sun, 28 Apr 2019 16:31:23 +0000

Control: tag -1 pending

Hello,

Bug #858431 in strip-nondeterminism reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:

https://salsa.debian.org/reproducible-builds/strip-nondeterminism/commit/f40f555085eeb086bfd4ee1fca1012550790a12d

------------------------------------------------------------------------
Workaround Archive::Zip's incorrect handling of the localExtraField field by monkey-patching the accessor methods to always return normalised values. This fixes the normalisation of Unix ownership (uid/gid) within .zip archives, .epub files, etc. (Closes: #858431, reproducible-builds/strip-nondeterminism#4)
------------------------------------------------------------------------

(this message was generated automatically)
-- 
Greetings

https://bugs.debian.org/858431

Message #58 received at 858431-submitter@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>

To: 858431-submitter@bugs.debian.org

Subject: Bug #858431 in strip-nondeterminism marked as pending

Date: Sun, 28 Apr 2019 16:31:23 +0000

Control: tag -1 pending

Hello,

Bug #858431 in strip-nondeterminism reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:

https://salsa.debian.org/reproducible-builds/strip-nondeterminism/commit/48be1d062aee46957a9c57d2040f266be954b5c6

------------------------------------------------------------------------
Merge tag '1.1.3' into debian

Release version 1.1.3

* tag '1.1.3':
  Release version 1.1.3
  Workaround Archive::Zip's incorrect handling of the localExtraField field by monkey-patching the accessor methods to always return normalised values. This fixes the normalisation of Unix ownership (uid/gid) within .zip archives, .epub files, etc. (Closes: #858431, reproducible-builds/strip-nondeterminism#4)
  Check the return status from Archive::Zip when writing file to disk.
  Catch an edge-case where we can't even parse the provided length of an invalid field within zip files.
------------------------------------------------------------------------

(this message was generated automatically)
-- 
Greetings

https://bugs.debian.org/858431

Message #63 received at 858431-close@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>

To: 858431-close@bugs.debian.org

Subject: Bug#858431: fixed in strip-nondeterminism 1.1.3-1

Date: Sun, 28 Apr 2019 16:51:07 +0000

Source: strip-nondeterminism
Source-Version: 1.1.3-1

We believe that the bug you reported is fixed in the latest version of
strip-nondeterminism, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 858431@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Chris Lamb <lamby@debian.org> (supplier of updated strip-nondeterminism package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Sun, 28 Apr 2019 17:30:27 +0100
Source: strip-nondeterminism
Binary: dh-strip-nondeterminism libfile-stripnondeterminism-perl strip-nondeterminism
Architecture: source all
Version: 1.1.3-1
Distribution: experimental
Urgency: medium
Maintainer: Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>
Changed-By: Chris Lamb <lamby@debian.org>
Description:
 dh-strip-nondeterminism - file non-deterministic information stripper — Debhelper add-on
 libfile-stripnondeterminism-perl - file non-deterministic information stripper — Perl module
 strip-nondeterminism - file non-deterministic information stripper — stand-alone tool
Closes: 858431
Changes:
 strip-nondeterminism (1.1.3-1) experimental; urgency=medium
 .
   * Workaround Archive::Zip's incorrect handling of the localExtraField field
     by monkey-patching the accessor methods to always return normalised values.
     This fixes the normalisation of Unix ownership (uid/gid) within .zip
     archives, .epub files, etc.
     (Closes: #858431, reproducible-builds/strip-nondeterminism#4)
   * Check the return status from Archive::Zip when writing file to disk.
   * Catch an edgecase where/if we can't parse the provided length of an
     invalid field within .zip files.
Checksums-Sha1:
 6bc7b28c612301c2f1b65b7dd49aaaa0f096db1d 2555 strip-nondeterminism_1.1.3-1.dsc
 5f6f26842246ff130cba64152f6f091371035186 221786 strip-nondeterminism_1.1.3.orig.tar.bz2
 0cb0d7b371c3e8494058e7c3df1075a51d531175 31404 strip-nondeterminism_1.1.3-1.debian.tar.xz
 147b304fefd0b7e0011fb69d15205dafa458f177 13256 dh-strip-nondeterminism_1.1.3-1_all.deb
 15ae1f2565a48470121b7bf95edf736114034bdf 20424 libfile-stripnondeterminism-perl_1.1.3-1_all.deb
 d083865dac3523a7910c5c46df4630074b9d751c 13376 strip-nondeterminism_1.1.3-1_all.deb
 d5e40a525b2dcfbd4beda2e505b709bf8db0aee7 6362 strip-nondeterminism_1.1.3-1_amd64.buildinfo
Checksums-Sha256:
 f1c71f268d14960a170f0d4657e9b323fcb0a0f9370f0887804b7111905c8179 2555 strip-nondeterminism_1.1.3-1.dsc
 aade3e2855761824fb069ae773f344216b0285fc4cb118edea3eb32e888c955b 221786 strip-nondeterminism_1.1.3.orig.tar.bz2
 ad87b5d34ce0ba1463cfa5624fc4a29e0c52d4a241cb7018ab7f9dda9055cba5 31404 strip-nondeterminism_1.1.3-1.debian.tar.xz
 1411bba9541e0af554406d9512f9a5813f60a2280535c850a55a9fb707ada954 13256 dh-strip-nondeterminism_1.1.3-1_all.deb
 a7c9b3ea512cf7d2eb9928233edbac81c25f679e2a3a3e65aeb02411a4daec15 20424 libfile-stripnondeterminism-perl_1.1.3-1_all.deb
 70cd232e3b5e49e79d124a1a481443850a9f9bc3e2f7c1df09583f3cbf56312e 13376 strip-nondeterminism_1.1.3-1_all.deb
 30daaf0a6534b6e8a8843e76683e5c070aa11f83bbec06dcaeea661c77ef6f93 6362 strip-nondeterminism_1.1.3-1_amd64.buildinfo
Files:
 ec00328f67673602accf86dabdba9991 2555 devel optional strip-nondeterminism_1.1.3-1.dsc
 db196b77910287bfc2ec430168c9ad70 221786 devel optional strip-nondeterminism_1.1.3.orig.tar.bz2
 fd5422104cdde964cb532f0dedfbadd8 31404 devel optional strip-nondeterminism_1.1.3-1.debian.tar.xz
 0bf28ba05165008e21ef9193c510329e 13256 devel optional dh-strip-nondeterminism_1.1.3-1_all.deb
 71230b68b6badd1628861e7691256459 20424 perl optional libfile-stripnondeterminism-perl_1.1.3-1_all.deb
 cd86d4d82d1116317d15651e352915fb 13376 devel optional strip-nondeterminism_1.1.3-1_all.deb
 17a276ba3530462f61b365a91bb97853 6362 devel optional strip-nondeterminism_1.1.3-1_amd64.buildinfo

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEwv5L0nHBObhsUz5GHpU+J9QxHlgFAlzF1ZUACgkQHpU+J9Qx
HliAMw/+Ono4OGc2DHmAOnWBneXmScZ12PnVhcJPFSny/QRB7lA+ko7RVHm8KSPR
Qy2UoJEsw7itpWt+cizxz/kJELdvTEA1pq7/lBNgYq2eSszi+bQHP++Wd51u3ILr
LHpZG3i8i0lAdsruhAl//J/7Z/1oKn5qcXzBiu6iQz1oYLaMOdQdeLXRw9IsACIR
6boZmv3Mag8yomO5D1sxWRPNWToNkMa4Nkr0AcksfvOlPPkZPkvnO5ipRoQEU5aJ
3jLjhsq3EnyaQ1vUws7/Fji7OtpBZKUXkDFFUjeEyxigpDmbA3lOzGHYsXgQhk1t
IV8WnweX4mCwX428yjVvvg50+k1ZlKRPbH9f80D8CFrAbKYzXo3Xch+sywLFC1Zr
LlJLiR95xQA6DSnEJQt5BWL/1RPo99wb9EpQZDqPf6/zDVva+RzH/dN0UMQS1EqA
ltauil3xuP9f0b4cZZC/ksn9kk8Gb9vK2fxwnpPPGg5yVSjtQNUIyPwrkXmjzE9I
x8Bp9cZP2IKrOpCdjUyPsApm+KGs3bHxuFJMICiUas6A+xQ9NWvU061W3QJ4+dEr
cuMzjIIEhwYHe3g45hstjcqShn2SRwUsEUp1Gks2HB6JJZ6eXAzh0UW8vy1+IPVi
qGgXOs1GwRbKDa3as2Mxjg1SxxhqENSqdS0GLj8cId90m4Gr284=
=lenf
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.