Debian Bug report logs - #904685
diffoscope: RuntimeError when trying to extract an encrypted file within .zip

version graph

Package: diffoscope; Maintainer for diffoscope is Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>; Source for diffoscope is src:diffoscope (PTS, buildd, popcon).

Reported by: Ricardo Gaviria <ricardo@unitelabs.ch>

Date: Thu, 26 Jul 2018 16:15:02 UTC

Severity: normal

Tags: moreinfo, patch

Found in version diffoscope/51

Fixed in version diffoscope/100

Done: Chris Lamb <lamby@debian.org>

Bug is archived. No further changes may be made.

Forwarded to https://salsa.debian.org/reproducible-builds/diffoscope/merge_requests/10

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>:
Bug#904685; Package diffoscope. (Thu, 26 Jul 2018 16:15:04 GMT) (full text, mbox, link).


Acknowledgement sent to Ricardo Gaviria <ricardo@unitelabs.ch>:
New Bug report received and forwarded. Copy sent to Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>. (Thu, 26 Jul 2018 16:15:04 GMT) (full text, mbox, link).


Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Ricardo Gaviria <ricardo@unitelabs.ch>
To: submit@bugs.debian.org
Subject: diffoscope: RuntimeError when trying to extract an encrypted file (.bmp)
Date: Thu, 26 Jul 2018 18:10:15 +0200
[Message part 1 (text/plain, inline)]
Package: diffoscope
Version: 51
Severity: normal

Dear Maintainer,

I was trying to diff two versions of a windows installer directory, and
encountered a RuntimeError due to diffoscope not being able to extract
an encrypted file.

I would have expected for the system to maybe report that such files
could not be extracted, and proceed with the rest of the diffing but
rather was faced with the stack trace and incomplete diff:

```
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/diffoscope/__main__.py", line 177,
in main
    sys.exit(run_diffoscope(parsed_args))
  File "/usr/lib/python3/dist-packages/diffoscope/__main__.py", line 148,
in run_diffoscope
    parsed_args.file1, parsed_args.file2)
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/__init__.py",
line 92, in compare_root_paths
    return compare_directories(path1, path2)
  File
"/usr/lib/python3/dist-packages/diffoscope/comparators/directory.py", line
104, in compare_directories
    return FilesystemDirectory(path1).compare(FilesystemDirectory(path2))
  File
"/usr/lib/python3/dist-packages/diffoscope/comparators/directory.py", line
157, in compare
    my_file, other_file, source=name)
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/__init__.py",
line 113, in compare_files
    return file1.compare(file2, source)
  File
"/usr/lib/python3/dist-packages/diffoscope/comparators/directory.py", line
157, in compare
    my_file, other_file, source=name)
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/__init__.py",
line 113, in compare_files
    return file1.compare(file2, source)
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/binary.py",
line 199, in compare
    difference = self._compare_using_details(other, source)
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/binary.py",
line 174, in _compare_using_details
    details.extend(filter(None,
self.as_container.compare(other.as_container)))
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/__init__.py",
line 116, in compare_commented_files
    difference = compare_files(file1, file2, source=source)
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/__init__.py",
line 102, in compare_files
    if file1.has_same_content_as(file2):
  File "/usr/lib/python3/dist-packages/diffoscope/__init__.py", line 143,
in tool_check
    return original_function(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/binary.py",
line 185, in has_same_content_as
    my_size = os.path.getsize(self.path)
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils.py",
line 242, in path
    self._path = self.container.extract(self._name, self._temp_dir.name)
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/zip.py", line
90, in extract
    with self.archive.open(member_name) as source, open(targetpath, 'wb')
as target:
  File "/usr/lib/python3.5/zipfile.py", line 1299, in open
    "required for extraction" % name)
RuntimeError: File acrobat.bmp is encrypted, password required for
extraction
```

Is this expected behaviour or is there a cmd line parameter which I
could use to prevent this?

Thanks,
Ricardo

-- System Information:
Debian Release: stretch/sid
  APT prefers xenial-security
  APT policy: (500, 'xenial-security'), (500, 'xenial'), (100,
'xenial-backports')
Architecture: amd64 (x86_64)
Foreign Architectures: i386, arm64

Kernel: Linux 4.13.0-45-generic (SMP w/8 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages diffoscope depends on:
ii  python3-libarchive-c   2.1-3
ii  python3-magic          1:5.25-2ubuntu1.1
ii  python3-pkg-resources  20.7.0-1
pn  python3:any            <none>

Versions of packages diffoscope recommends:
ii  acl                                2.2.52-3
ii  binutils-multiarch                 2.26.1-1ubuntu1~16.04.6
ii  bzip2                              1.0.6-8
ii  caca-utils                         0.99.beta19-2build2~gcc5.2
ii  colord                             1.2.12-1ubuntu1
ii  cpio                               2.11+dfsg-5ubuntu1
ii  default-jdk [java-sdk]             2:1.8-56ubuntu2
ii  enjarify                           20151118-1
ii  fontforge-extras                   0.3-4ubuntu1
ii  fp-utils                           3.0.0+dfsg-2
ii  fp-utils-3.0.0 [fp-utils]          3.0.0+dfsg-2
ii  genisoimage                        9:1.1.11-3ubuntu1
ii  gettext                            0.19.7-2ubuntu3
ii  ghc                                7.10.3-7
ii  ghostscript                        9.18~dfsg~0-0ubuntu2.8
ii  gnupg                              1.4.20-1ubuntu3.2
ii  mono-utils                         4.2.1.102+dfsg2-7ubuntu4
ii  openjdk-8-jdk [java-sdk]           8u171-b11-0ubuntu0.16.04.1
ii  oracle-java8-installer [java-sdk]  8u171-1~webupd8~0
ii  pdftk                              2.02-4
ii  poppler-utils                      0.41.0-0ubuntu1.7
ii  python3-debian                     0.1.27ubuntu2
ii  python3-guestfs                    1:1.32.2-4ubuntu2
ii  python3-rpm                        4.12.0.1+dfsg1-3build3
ii  python3-tlsh                       3.4.4+20151206-1build1
ii  rpm2cpio                           4.12.0.1+dfsg1-3build3
ii  sng                                1.1.0-1
ii  sqlite3                            3.11.0-1ubuntu1
ii  squashfs-tools                     1:4.3-3ubuntu2.16.04.1
ii  unzip                              6.0-20ubuntu1
ii  vim-common                         2:7.4.1689-3ubuntu1.2
ii  xz-utils                           5.1.1alpha+20120614-2ubuntu2

Versions of packages diffoscope suggests:
ii  libjs-jquery  1.11.3+dfsg-4

-- no debconf information

-- 
Regards,
Ricardo Gaviria
Software Engineer, UniteLabs
*M: *+41 77 956 2376
*W: *http://unitelabs.ch
*In: *https://www.linkedin.com/in/ricardogaviria/
[Message part 2 (text/html, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>:
Bug#904685; Package diffoscope. (Fri, 27 Jul 2018 04:45:03 GMT) (full text, mbox, link).


Acknowledgement sent to Chris Lamb <lamby@debian.org>:
Extra info received and forwarded to list. Copy sent to Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>. (Fri, 27 Jul 2018 04:45:03 GMT) (full text, mbox, link).


Message #10 received at 904685@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>
To: Ricardo Gaviria <ricardo@unitelabs.ch>, 904685@bugs.debian.org
Subject: Re: Bug#904685: diffoscope: RuntimeError when trying to extract an encrypted file (.bmp)
Date: Fri, 27 Jul 2018 05:41:07 +0100
retitle 904685 diffoscope: RuntimeError when trying to extract an encrypted file within .zip
tags 904685 + moreinfo
thanks

Hi Ricardo,

> Is this expected behaviour or is there a cmd line parameter which I
> could use to prevent this?

Oh, not at all the expected behaviour. diffoscope should "never"
traceback on you.

>   File "/usr/lib/python3.5/zipfile.py", line 1299, in open
>     "required for extraction" % name)
> RuntimeError: File acrobat.bmp is encrypted, password required for
> extraction

Actually looks zipfile related. Could you reply, attaching the file in
question so we can debug & fix this?


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-



Changed Bug title to 'diffoscope: RuntimeError when trying to extract an encrypted file within .zip' from 'diffoscope: RuntimeError when trying to extract an encrypted file (.bmp)'. Request was from Chris Lamb <lamby@debian.org> to control@bugs.debian.org. (Fri, 27 Jul 2018 04:45:04 GMT) (full text, mbox, link).


Added tag(s) moreinfo. Request was from Chris Lamb <lamby@debian.org> to control@bugs.debian.org. (Fri, 27 Jul 2018 04:45:05 GMT) (full text, mbox, link).


Information forwarded to debian-bugs-dist@lists.debian.org, Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>:
Bug#904685; Package diffoscope. (Fri, 27 Jul 2018 15:18:02 GMT) (full text, mbox, link).


Acknowledgement sent to Ricardo Gaviria <ricardo@unitelabs.ch>:
Extra info received and forwarded to list. Copy sent to Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>. (Fri, 27 Jul 2018 15:18:02 GMT) (full text, mbox, link).


Message #19 received at 904685@bugs.debian.org (full text, mbox, reply):

From: Ricardo Gaviria <ricardo@unitelabs.ch>
To: Chris Lamb <lamby@debian.org>
Cc: 904685@bugs.debian.org
Subject: Re: Bug#904685: diffoscope: RuntimeError when trying to extract an encrypted file (.bmp)
Date: Fri, 27 Jul 2018 17:14:27 +0200
[Message part 1 (text/plain, inline)]
Hi Chris,

Thanks for your response. Please find attached the culprit. I believe the
issue may lie with the fact that I am trying to diff an archive which
happens to be encrypted and requires a password.

I'm guessing encrypted archives could be ignored when diff'ing and flagged,
and possibly just diffing the fingerprint of the archive blob? Or what do
you suggest?

Thanks again,
Ricardo

On Fri, Jul 27, 2018 at 6:41 AM Chris Lamb <lamby@debian.org> wrote:

> retitle 904685 diffoscope: RuntimeError when trying to extract an
> encrypted file within .zip
> tags 904685 + moreinfo
> thanks
>
> Hi Ricardo,
>
> > Is this expected behaviour or is there a cmd line parameter which I
> > could use to prevent this?
>
> Oh, not at all the expected behaviour. diffoscope should "never"
> traceback on you.
>
> >   File "/usr/lib/python3.5/zipfile.py", line 1299, in open
> >     "required for extraction" % name)
> > RuntimeError: File acrobat.bmp is encrypted, password required for
> > extraction
>
> Actually looks zipfile related. Could you reply, attaching the file in
> question so we can debug & fix this?
>
>
> Regards,
>
> --
>       ,''`.
>      : :'  :     Chris Lamb
>      `. `'`      lamby@debian.org / chris-lamb.co.uk
>        `-
>
-- 
Regards,
Ricardo Gaviria
Software Engineer, UniteLabs
*M: *+41 77 956 2376
*W: *http://unitelabs.ch
*In: *https://www.linkedin.com/in/ricardogaviria/
[Message part 2 (text/html, inline)]
[autorun.cdd (application/octet-stream, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>:
Bug#904685; Package diffoscope. (Fri, 27 Jul 2018 15:27:06 GMT) (full text, mbox, link).


Acknowledgement sent to Ricardo Gaviria <ricardo@unitelabs.ch>:
Extra info received and forwarded to list. Copy sent to Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>. (Fri, 27 Jul 2018 15:27:06 GMT) (full text, mbox, link).


Message #24 received at 904685@bugs.debian.org (full text, mbox, reply):

From: Ricardo Gaviria <ricardo@unitelabs.ch>
To: Chris Lamb <lamby@debian.org>
Cc: 904685@bugs.debian.org
Subject: Re: Bug#904685: diffoscope: RuntimeError when trying to extract an encrypted file (.bmp)
Date: Fri, 27 Jul 2018 17:23:12 +0200
[Message part 1 (text/plain, inline)]
By the way, if this is a lower priority bug than others. I will gladly have
a look into the issue and try and resolve it, provided that we agree on the
expected behaviour of the tool under such a scenario.

Ricardo

On Fri, Jul 27, 2018 at 5:14 PM Ricardo Gaviria <ricardo@unitelabs.ch>
wrote:

> Hi Chris,
>
> Thanks for your response. Please find attached the culprit. I believe the
> issue may lie with the fact that I am trying to diff an archive which
> happens to be encrypted and requires a password.
>
> I'm guessing encrypted archives could be ignored when diff'ing and
> flagged, and possibly just diffing the fingerprint of the archive blob? Or
> what do you suggest?
>
> Thanks again,
> Ricardo
>
> On Fri, Jul 27, 2018 at 6:41 AM Chris Lamb <lamby@debian.org> wrote:
>
>> retitle 904685 diffoscope: RuntimeError when trying to extract an
>> encrypted file within .zip
>> tags 904685 + moreinfo
>> thanks
>>
>> Hi Ricardo,
>>
>> > Is this expected behaviour or is there a cmd line parameter which I
>> > could use to prevent this?
>>
>> Oh, not at all the expected behaviour. diffoscope should "never"
>> traceback on you.
>>
>> >   File "/usr/lib/python3.5/zipfile.py", line 1299, in open
>> >     "required for extraction" % name)
>> > RuntimeError: File acrobat.bmp is encrypted, password required for
>> > extraction
>>
>> Actually looks zipfile related. Could you reply, attaching the file in
>> question so we can debug & fix this?
>>
>>
>> Regards,
>>
>> --
>>       ,''`.
>>      : :'  :     Chris Lamb
>>      `. `'`      lamby@debian.org / chris-lamb.co.uk
>>        `-
>>
> --
> Regards,
> Ricardo Gaviria
> Software Engineer, UniteLabs
> *M: *+41 77 956 2376 <+41%2077%20956%2023%2076>
> *W: *http://unitelabs.ch
> *In: *https://www.linkedin.com/in/ricardogaviria/
>
> --
Regards,
Ricardo Gaviria
Software Engineer, UniteLabs
*M: *+41 77 956 2376
*W: *http://unitelabs.ch
*In: *https://www.linkedin.com/in/ricardogaviria/
[Message part 2 (text/html, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>:
Bug#904685; Package diffoscope. (Fri, 27 Jul 2018 15:27:08 GMT) (full text, mbox, link).


Acknowledgement sent to Chris Lamb <lamby@debian.org>:
Extra info received and forwarded to list. Copy sent to Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>. (Fri, 27 Jul 2018 15:27:08 GMT) (full text, mbox, link).


Message #29 received at 904685@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>
To: Ricardo Gaviria <ricardo@unitelabs.ch>
Cc: 904685@bugs.debian.org
Subject: Re: Bug#904685: diffoscope: RuntimeError when trying to extract an encrypted file (.bmp)
Date: Fri, 27 Jul 2018 16:26:27 +0100
Hi Ricardo,

> By the way, if this is a lower priority bug than others. I will gladly have
> a look into the issue and try and resolve it, provided that we agree on the
> expected behaviour of the tool under such a scenario.

AIUI the *ideal* behaviour would be that encrypted files would:

  a) Not cause a traceback
  b) Be marked as such in the diff (see `add_comment` in the code)
  c) Be compared in a fallback (ie. binary) fashion

Hope that helps. :)


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-



Information forwarded to debian-bugs-dist@lists.debian.org, Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>:
Bug#904685; Package diffoscope. (Fri, 27 Jul 2018 15:30:03 GMT) (full text, mbox, link).


Acknowledgement sent to Ricardo Gaviria <ricardo@unitelabs.ch>:
Extra info received and forwarded to list. Copy sent to Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>. (Fri, 27 Jul 2018 15:30:03 GMT) (full text, mbox, link).


Message #34 received at 904685@bugs.debian.org (full text, mbox, reply):

From: Ricardo Gaviria <ricardo@unitelabs.ch>
To: Chris Lamb <lamby@debian.org>
Cc: 904685@bugs.debian.org
Subject: Re: Bug#904685: diffoscope: RuntimeError when trying to extract an encrypted file (.bmp)
Date: Fri, 27 Jul 2018 17:28:10 +0200
[Message part 1 (text/plain, inline)]
Perfect! thanks for the clarification. Will look into it so ;)

P.S. I love diffoscope!

On Fri, Jul 27, 2018 at 5:26 PM Chris Lamb <lamby@debian.org> wrote:

> Hi Ricardo,
>
> > By the way, if this is a lower priority bug than others. I will gladly
> have
> > a look into the issue and try and resolve it, provided that we agree on
> the
> > expected behaviour of the tool under such a scenario.
>
> AIUI the *ideal* behaviour would be that encrypted files would:
>
>   a) Not cause a traceback
>   b) Be marked as such in the diff (see `add_comment` in the code)
>   c) Be compared in a fallback (ie. binary) fashion
>
> Hope that helps. :)
>
>
> Regards,
>
> --
>       ,''`.
>      : :'  :     Chris Lamb
>      `. `'`      lamby@debian.org / chris-lamb.co.uk
>        `-
>
-- 
Regards,
Ricardo Gaviria
Software Engineer, UniteLabs
*M: *+41 77 956 2376
*W: *http://unitelabs.ch
*In: *https://www.linkedin.com/in/ricardogaviria/
[Message part 2 (text/html, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>:
Bug#904685; Package diffoscope. (Sat, 04 Aug 2018 21:09:03 GMT) (full text, mbox, link).


Acknowledgement sent to Ricardo Gaviria <ricardo@unitelabs.ch>:
Extra info received and forwarded to list. Copy sent to Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>. (Sat, 04 Aug 2018 21:09:03 GMT) (full text, mbox, link).


Message #39 received at 904685@bugs.debian.org (full text, mbox, reply):

From: Ricardo Gaviria <ricardo@unitelabs.ch>
To: Chris Lamb <lamby@debian.org>
Cc: 904685@bugs.debian.org
Subject: Re: Bug#904685: diffoscope: RuntimeError when trying to extract an encrypted file (.bmp)
Date: Sat, 4 Aug 2018 23:04:25 +0200
[Message part 1 (text/plain, inline)]
Hi Chris,

Based on this bug, please find attached a proposed patch for handling this
error gracefully by catching any exceptions caused by trying to open an
encrypted file in an archive. and forwarding it on as a
*ContainerExtractionError*.

I would gladly appreciate some feedback. I tried to update the changelog as
best as I understood here
<https://reproducible-builds.org/contribute/#Fixing_issues>.

Additionally, I see that I could have also just submitted a merge request
via salsa.debian.org. What is the usual workflow, email patches or merge
requests?

Regards,
Ricardo

On Fri, Jul 27, 2018 at 5:28 PM Ricardo Gaviria <ricardo@unitelabs.ch>
wrote:

> Perfect! thanks for the clarification. Will look into it so ;)
>
> P.S. I love diffoscope!
>
> On Fri, Jul 27, 2018 at 5:26 PM Chris Lamb <lamby@debian.org> wrote:
>
>> Hi Ricardo,
>>
>> > By the way, if this is a lower priority bug than others. I will gladly
>> have
>> > a look into the issue and try and resolve it, provided that we agree on
>> the
>> > expected behaviour of the tool under such a scenario.
>>
>> AIUI the *ideal* behaviour would be that encrypted files would:
>>
>>   a) Not cause a traceback
>>   b) Be marked as such in the diff (see `add_comment` in the code)
>>   c) Be compared in a fallback (ie. binary) fashion
>>
>> Hope that helps. :)
>>
>>
>> Regards,
>>
>> --
>>       ,''`.
>>      : :'  :     Chris Lamb
>>      `. `'`      lamby@debian.org / chris-lamb.co.uk
>>        `-
>>
> --
> Regards,
> Ricardo Gaviria
> Software Engineer, UniteLabs
> *M: *+41 77 956 2376 <+41%2077%20956%2023%2076>
> *W: *http://unitelabs.ch
> *In: *https://www.linkedin.com/in/ricardogaviria/
>
> --
Regards,
Ricardo Gaviria
Software Engineer, UniteLabs
*M: *+41 77 956 2376
*W: *http://unitelabs.ch
*In: *https://www.linkedin.com/in/ricardogaviria/
[Message part 2 (text/html, inline)]
[0001-Handle-error-when-encrypted-file-is-exctracted-insid.patch (text/x-patch, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>:
Bug#904685; Package diffoscope. (Sun, 05 Aug 2018 02:03:03 GMT) (full text, mbox, link).


Acknowledgement sent to Chris Lamb <lamby@debian.org>:
Extra info received and forwarded to list. Copy sent to Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>. (Sun, 05 Aug 2018 02:03:03 GMT) (full text, mbox, link).


Message #44 received at 904685@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>
To: Ricardo Gaviria <ricardo@unitelabs.ch>
Cc: 904685@bugs.debian.org
Subject: Re: Bug#904685: diffoscope: RuntimeError when trying to extract an encrypted file (.bmp)
Date: Sun, 05 Aug 2018 03:00:03 +0100
tags 904685 + patch
thanks

Dear Ricardo,

> Based on this bug, please find attached a proposed patch for handling
> this error gracefully [..]

Wow, thank you so much for this :)

> Additionally, I see that I could have also just submitted a merge request
> via salsa.debian.org. What is the usual workflow, email patches or merge
> requests?

It might depend more on the size of the patch and whether you were
thinking of making more in the future. Using salsa also has the
advantages of running the testsuite too, which may be useful here (see
below). Please do join our group on salsa ...

> I would gladly appreciate some feedback. I tried to update the changelog as
> best as I understood here
> <https://reproducible-builds.org/contribute/#Fixing_issues>.

Sure thing.

So, the debian/changelog entries for diffoscope are generated
automatically upon release so updating the changelog is not required.
In your commit message please suffix with "(Closes: #904685)" so the
versions/state is handled properly though.

(Furthermore, the "~reproducibleX" suffix as outlined on that page is
typically used when we fork packages from /elsewhere/ in Debian and, as
this project is part of the reproducible builds effort, there is no
naturally need to fork..)


           targetpath = os.path.join(dest_dir, os.path.basename(member_name)).encode(
               sys.getfilesystemencoding(), errors='replace')
  -        with self.archive.open(member_name) as source, open(targetpath, 'wb') as target:
  -            shutil.copyfileobj(source, target)
  -        return targetpath.decode(sys.getfilesystemencoding())
  +        try:
  +            with self.archive.open(member_name) as source, open(targetpath, 'wb') as target:
  +                shutil.copyfileobj(source, target)
  +            return targetpath.decode(sys.getfilesystemencoding())
  +        except Exception as exc:
                  ^^^^^^^^^

Isn't this a bit too "wide" an exception class to catch? How narrow can
we safely make it?

  +            raise ContainerExtractionError(member_name, exc)

I think I would also like to see:

 * A comment in the except block explaining why we might be seeing an
   exception in the first place.

 * A test :)


Best wishes,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-



Added tag(s) patch. Request was from Chris Lamb <lamby@debian.org> to control@bugs.debian.org. (Sun, 05 Aug 2018 02:03:05 GMT) (full text, mbox, link).


Information forwarded to debian-bugs-dist@lists.debian.org, Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>:
Bug#904685; Package diffoscope. (Sun, 05 Aug 2018 21:18:03 GMT) (full text, mbox, link).


Acknowledgement sent to Ricardo Gaviria <ricardo@unitelabs.ch>:
Extra info received and forwarded to list. Copy sent to Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>. (Sun, 05 Aug 2018 21:18:03 GMT) (full text, mbox, link).


Message #51 received at 904685@bugs.debian.org (full text, mbox, reply):

From: Ricardo Gaviria <ricardo@unitelabs.ch>
To: Chris Lamb <lamby@debian.org>
Cc: 904685@bugs.debian.org
Subject: Re: Bug#904685: diffoscope: RuntimeError when trying to extract an encrypted file (.bmp)
Date: Sun, 5 Aug 2018 23:14:06 +0200
[Message part 1 (text/plain, inline)]
Hi Chris,

Thanks for the swift feedback, and for sharing. I have issued a merge
request which can be found here
<https://salsa.debian.org/reproducible-builds/diffoscope/merge_requests/10>.
Have addressed your comments, including narrower exception, accompanied by
a comment, and one test case trying to compare 2 encrypted zipfiles.



*Isn't this a bit too "wide" an exception class to catch? How narrow canwe
safely make it?*

You are right that is too wide, the narrowest I think we can go is catching
a RuntimeError (which is not that narrow IMO) but that is what is thrown in
this case by the open() method in zipfile.py
<https://github.com/python/cpython/blob/a2fe1e52eb94c41d9ebce1ab284180d7b1faa2a4/Lib/zipfile.py#L1495>

If we want to be more explicit we could just explicitly check if the
archive is encrypted and raise a diffoscope exception accordingly:

+        try:
+            # Wrapped in a try block as exception may be raised due to
archive
+            # being encrypted, already closed, or opened incorrectly see
library
+            # zipfile.py line 1292
+            with self.archive.open(member_name) as source,
open(targetpath, 'wb') as target:
+                shutil.copyfileobj(source, target)
+            return targetpath.decode(sys.getfilesystemencoding())
+        except RuntimeError as exc:
+            raise ContainerExtractionError(member_name, exc)

Not sure what is the best approach here.

We can move this discussion over to the MR if that suits better. Just let
me know.

Thanks again,
Ricardo






On Sun, Aug 5, 2018 at 4:00 AM Chris Lamb <lamby@debian.org> wrote:

> tags 904685 + patch
> thanks
>
> Dear Ricardo,
>
> > Based on this bug, please find attached a proposed patch for handling
> > this error gracefully [..]
>
> Wow, thank you so much for this :)
>
> > Additionally, I see that I could have also just submitted a merge request
> > via salsa.debian.org. What is the usual workflow, email patches or merge
> > requests?
>
> It might depend more on the size of the patch and whether you were
> thinking of making more in the future. Using salsa also has the
> advantages of running the testsuite too, which may be useful here (see
> below). Please do join our group on salsa ...
>
> > I would gladly appreciate some feedback. I tried to update the changelog
> as
> > best as I understood here
> > <https://reproducible-builds.org/contribute/#Fixing_issues>.
>
> Sure thing.
>
> So, the debian/changelog entries for diffoscope are generated
> automatically upon release so updating the changelog is not required.
> In your commit message please suffix with "(Closes: #904685)" so the
> versions/state is handled properly though.
>
> (Furthermore, the "~reproducibleX" suffix as outlined on that page is
> typically used when we fork packages from /elsewhere/ in Debian and, as
> this project is part of the reproducible builds effort, there is no
> naturally need to fork..)
>
>
>            targetpath = os.path.join(dest_dir,
> os.path.basename(member_name)).encode(
>                sys.getfilesystemencoding(), errors='replace')
>   -        with self.archive.open(member_name) as source, open(targetpath,
> 'wb') as target:
>   -            shutil.copyfileobj(source, target)
>   -        return targetpath.decode(sys.getfilesystemencoding())
>   +        try:
>   +            with self.archive.open(member_name) as source,
> open(targetpath, 'wb') as target:
>   +                shutil.copyfileobj(source, target)
>   +            return targetpath.decode(sys.getfilesystemencoding())
>   +        except Exception as exc:
>                   ^^^^^^^^^
>
> Isn't this a bit too "wide" an exception class to catch? How narrow can
> we safely make it?
>
>   +            raise ContainerExtractionError(member_name, exc)
>
> I think I would also like to see:
>
>  * A comment in the except block explaining why we might be seeing an
>    exception in the first place.
>
>  * A test :)
>
>
> Best wishes,
>
> --
>       ,''`.
>      : :'  :     Chris Lamb
>      `. `'`      lamby@debian.org / chris-lamb.co.uk
>        `-
>
-- 
Regards,
Ricardo Gaviria
Software Engineer, UniteLabs
*M: *+41 77 956 2376
*W: *http://unitelabs.ch
*In: *https://www.linkedin.com/in/ricardogaviria/
[Message part 2 (text/html, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>:
Bug#904685; Package diffoscope. (Mon, 06 Aug 2018 03:45:04 GMT) (full text, mbox, link).


Acknowledgement sent to Chris Lamb <lamby@debian.org>:
Extra info received and forwarded to list. Copy sent to Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>. (Mon, 06 Aug 2018 03:45:04 GMT) (full text, mbox, link).


Message #56 received at 904685@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>
To: Ricardo Gaviria <ricardo@unitelabs.ch>
Cc: 904685@bugs.debian.org
Subject: Re: Bug#904685: diffoscope: RuntimeError when trying to extract an encrypted file (.bmp)
Date: Mon, 06 Aug 2018 04:40:09 +0100
fowarded 904685 https://salsa.debian.org/reproducible-builds/diffoscope/merge_requests/10
thanks

Hi Ricardo,

> We can move this discussion over to the MR if that suits better. Just let
> me know.

Have responded on the merge request & let's move over there.


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-



Set Bug forwarded-to-address to 'https://salsa.debian.org/reproducible-builds/diffoscope/merge_requests/10'. Request was from Chris Lamb <lamby@debian.org> to control@bugs.debian.org. (Mon, 06 Aug 2018 03:48:02 GMT) (full text, mbox, link).


Information forwarded to debian-bugs-dist@lists.debian.org, Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>:
Bug#904685; Package diffoscope. (Tue, 07 Aug 2018 14:09:02 GMT) (full text, mbox, link).


Acknowledgement sent to Chris Lamb <lamby@debian.org>:
Extra info received and forwarded to list. Copy sent to Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>. (Tue, 07 Aug 2018 14:09:02 GMT) (full text, mbox, link).


Message #63 received at 904685@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>
To: 904685@bugs.debian.org
Cc: Ricardo Gaviria <ricardo@unitelabs.ch>
Subject: Re: diffoscope: RuntimeError when trying to extract an encrypted file within .zip
Date: Tue, 07 Aug 2018 15:05:53 +0100
tags 904685 + pending
thanks

Applied in Git, pending upload.. Many thanks Ricardo!

  https://salsa.debian.org/reproducible-builds/diffoscope/commit/a6beb04368bce3f56993a7fd29313e2442db7b3b

  diffoscope/comparators/zip.py |  16 +++++++++++++---
  tests/comparators/test_zip.py |   7 +++++++
  tests/data/encrypted1.zip     | Bin 0 -> 599 bytes
  tests/data/encrypted2.zip     | Bin 0 -> 742 bytes
  4 files changed, 20 insertions(+), 3 deletions(-)


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-



Added tag(s) pending. Request was from Chris Lamb <lamby@debian.org> to control@bugs.debian.org. (Tue, 07 Aug 2018 14:09:04 GMT) (full text, mbox, link).


Reply sent to Chris Lamb <lamby@debian.org>:
You have taken responsibility. (Sat, 01 Sep 2018 08:39:20 GMT) (full text, mbox, link).


Notification sent to Ricardo Gaviria <ricardo@unitelabs.ch>:
Bug acknowledged by developer. (Sat, 01 Sep 2018 08:39:20 GMT) (full text, mbox, link).


Message #70 received at 904685-close@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>
To: 904685-close@bugs.debian.org
Subject: Bug#904685: fixed in diffoscope 100
Date: Sat, 01 Sep 2018 08:34:14 +0000
Source: diffoscope
Source-Version: 100

We believe that the bug you reported is fixed in the latest version of
diffoscope, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 904685@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Chris Lamb <lamby@debian.org> (supplier of updated diffoscope package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Sat, 01 Sep 2018 09:09:48 +0100
Source: diffoscope
Binary: diffoscope
Built-For-Profiles: nocheck
Architecture: source all
Version: 100
Distribution: unstable
Urgency: medium
Maintainer: Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>
Changed-By: Chris Lamb <lamby@debian.org>
Description:
 diffoscope - in-depth comparison of files, archives, and directories
Closes: 903391 903401 903447 903449 903565 904685 905598 906967
Changes:
 diffoscope (100) unstable; urgency=medium
 .
   [ Alexis Murzeau ]
   * Correct matching of .deb archive members. (Closes: #903565)
 .
   [ Chris Lamb ]
   * Support .deb archives that contain an uncompressed control.tar and data.tar
     Thanks to Roderich Schupp <roderich.schupp@gmail.com>.
     (Closes: #903391, #903401)
   * Wrap jsondiff calls with try-except to prevent fatal errors.
     (Closes: #903447, #903449)
   * Don't include the filename in llvm-bcanalyzer results. (Closes: #905598)
   * Update generated debian/tests/control.
   * Bump Standards-Version to 4.2.1.
 .
   [ Daniel Kahn Gillmor ]
   * Avoid line eraser error on dumb terminals. (Closes: #906967)
   * Correct spelling of ereser to eraser.
 .
   [ Mattia Rizzolo ]
   * On Debian, do not require apktool on ppc64el and s390x, ogvideotools on
     s390x and fp-utils on ppc64el and s390x as they are not installable there.
   * Explicitly add `file` to the dependencies of autopkgtest to have the tests
     triggered whenever `file` changes
 .
   [ Ricardo Gaviria ]
   * Handle errors with encrypted archive files. (Closes: #904685)
Checksums-Sha1:
 ee418ed8178f58ab456f7f7f99330c9ea4b3da42 4050 diffoscope_100.dsc
 bcf9d36178a98c9aeb24457588ba0638152661ed 9251936 diffoscope_100.tar.xz
 a60721094ec8ed73f7658baed694bfc94e128ff2 122780 diffoscope_100_all.deb
 af050ca0c265c4d181bd322c9f65323bc1bc4a45 6725 diffoscope_100_amd64.buildinfo
Checksums-Sha256:
 61cbdfa21cfbe6a6a8c32be511a13fbed0d0d1e2788ef5cc859cd2dc54c2c092 4050 diffoscope_100.dsc
 98af4a69d2ded2de221d050a110d60beb7a8b737a3bba1ac40755682c7a45b4b 9251936 diffoscope_100.tar.xz
 9f56af77c7756c9265c615117f0704d0009b8b6de2d164d6afcc8ce78d208e6f 122780 diffoscope_100_all.deb
 6fca17529ee1260aa9576840b97f3dffb0e16476435a655986bf03bc280a1fad 6725 diffoscope_100_amd64.buildinfo
Files:
 30b68b4a6dbcba4bc3aa7d948bd3f4da 4050 devel optional diffoscope_100.dsc
 148c2d1675b1f3475fc6ab28bc39eb2a 9251936 devel optional diffoscope_100.tar.xz
 cd5e06d93db32e437ca32ff5e36cce78 122780 devel optional diffoscope_100_all.deb
 f87adefc92b27a4441634aba05448839 6725 devel optional diffoscope_100_amd64.buildinfo

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEwv5L0nHBObhsUz5GHpU+J9QxHlgFAluKSc8ACgkQHpU+J9Qx
Hlg1lxAAjgTqnKwdWr3cQYA7MO4VqB11p9185NkXrR2vkHcUNvN1xv95J7/A4BZx
3BXeK2dfi8Qu1gN2NDeg6+m5FTUMDHesmXMlqrDG5JLVRHXM8a/FqpHHCAGJKLR6
QALd1uE24VSdbxBc2hJvTWDu6t1DbKvhXp83D9kBc8APvOsaxWpFNE7dDHDKEICa
EC8Ne925EEU/8xwl+LTtAN9DQEw0Pmi521NK2pBhX4NfETm6OJODb2cH/DCCMB9x
65IdU0hF4tMDzZOQ2utgWQ5Cx7Se66tcqNmamyunBocfZ77d5EA+IKsiYD2QQ7Iy
G+fhMJYTiiYPfxZvGS8ln3u29p1CFrj5vJ+Cggf7Dl0w+a8WEXIOaKPZ4KIddT7X
k2bg+dtMzr/4VcTFAy6zmtyn86hMNNkF2rdkG2EkeHNEHuP+HwnIPnmMpS5o7BEW
S9jtd93khRD+/Jp/Zen6K8X2As2Gvd3s97+0hInTPezweny/DhapNxwRc/sYLfvv
iuvrkssM4hJ+yXFogvCbI+Q4MLZpUtvXLU/IKDhTVQuPIBDNrSDQChPGOEeWYEsz
2PW+ls6ipNPWUUkL556OiN7V0dbZIEMlSLHzWdzr6FrRZwWZmJAfulO+1GIZTMyu
xwzo6q+pne8fLX7ynE53uXVpzvdAMDPDRkj4cJMMVcKojoCih0E=
=Uc9v
-----END PGP SIGNATURE-----




Bug archived. Request was from Debbugs Internal Request <owner@bugs.debian.org> to internal_control@bugs.debian.org. (Sun, 30 Sep 2018 07:28:59 GMT) (full text, mbox, link).


Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Wed May 17 10:23:24 2023; Machine Name: buxtehude

Debian Bug tracking system

Debbugs is free software and licensed under the terms of the GNU Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.

Copyright © 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson, 2005-2017 Don Armstrong, and many other contributors.