Debian Bug report logs - #981123
diffoscope suddenly needs a lot more /tmp space

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Holger Levsen <holger@debian.org>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: diffoscope suddenly needs a lot more /tmp space

Date: Tue, 26 Jan 2021 18:46:50 +0100

[Message part 1 (text/plain, inline)]

Package: diffoscope
Version: 164
Severity: important

diffoscope 164 was released on the 8th of January 2021 and approx.
since the 10th of January /tmp on jenkins.debian.net has regularily
seen filling up as can be seen on 
https://jenkins.debian.net/munin/debian.net/jenkins.debian.net/df.html
and 
https://jenkins.debian.net/munin/static/dynazoom.html?cgiurl_graph=/munin-cgi/munin-cgi-graph&plugin_name=debian.net/jenkins.debian.net/df&size_x=800&size_y=400&start_epoch=1608831726&stop_epoch=1611682926
even better and I do think the timing is relevant as there were no
other relevant changes to jenkins.debian.net those days.

I'm filing this bug as a first starting point in debugging this issue.


-- 
cheers,
	Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁       holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀ PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C
 ⠈⠳⣄

Make racists afraid again.

[signature.asc (application/pgp-signature, inline)]

Message #10 received at 981123@bugs.debian.org (full text, mbox, reply):

From: Holger Levsen <holger@layer-acht.org>

To: 981123@bugs.debian.org

Subject: Re: Bug#981123: diffoscope suddenly needs a lot more /tmp space

Date: Tue, 26 Jan 2021 18:12:26 +0000

[Message part 1 (text/plain, inline)]

On Tue, Jan 26, 2021 at 06:46:50PM +0100, Holger Levsen wrote:
> diffoscope 164 was released on the 8th of January 2021 and approx.
> since the 10th of January...

I forgot to explicitly mention that this timespan very well matches the
delay between a new diffoscope uploaded to unstable and becoming available
in those chroots where we are running it...


-- 
cheers,
	Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁       holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀ PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C
 ⠈⠳⣄

[signature.asc (application/pgp-signature, inline)]

Message #15 received at 981123@bugs.debian.org (full text, mbox, reply):

From: "Chris Lamb" <lamby@debian.org>

To: 981123@bugs.debian.org

Cc: "Holger Levsen" <holger@debian.org>

Subject: Re: diffoscope suddenly needs a lot more /tmp space

Date: Wed, 27 Jan 2021 11:46:39 +0000

forwarded 981123 https://salsa.debian.org/reproducible-builds/diffoscope/-/issues/234
thanks

Thanks Holger. Sorry for the slight whiplash here but I'm going to
move this to Salsa so I can more-easily bring in Dimitrios - I think
it is their changes that are causing this. (I hope you don't mind me
pinging you there too, but probably easiest to keep it all together.)


Regards,

--
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

Message #22 received at 981123@bugs.debian.org (full text, mbox, reply):

From: Holger Levsen <holger@layer-acht.org>

To: 981123@bugs.debian.org

Subject: Re: Bug#981123: diffoscope suddenly needs a lot more /tmp space

Date: Wed, 27 Jan 2021 11:57:53 +0000

[Message part 1 (text/plain, inline)]

On Wed, Jan 27, 2021 at 11:46:39AM +0000, Chris Lamb wrote:
> Thanks Holger. Sorry for the slight whiplash here but I'm going to
> move this to Salsa so I can more-easily bring in Dimitrios - I think
> it is their changes that are causing this. (I hope you don't mind me
> pinging you there too, but probably easiest to keep it all together.)

sure, i'm subscribed to diffoscope to salsa.

and thanks for forwarding it there! it's just easier & less mental work
for me to file a bug in the bts, that's why I prefer to go this way...


-- 
cheers,
	Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁       holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀ PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C
 ⠈⠳⣄

Stop saying that we are all in the same boat.
We’re all in the same storm.
But we’re not all in the same boat.

[signature.asc (application/pgp-signature, inline)]

Message #27 received at 981123-close@bugs.debian.org (full text, mbox, reply):

From: Debian FTP Masters <ftpmaster@ftp-master.debian.org>

To: 981123-close@bugs.debian.org

Subject: Bug#981123: fixed in diffoscope 166

Date: Sat, 30 Jan 2021 12:48:24 +0000

Source: diffoscope
Source-Version: 166
Done: Chris Lamb <lamby@debian.org>

We believe that the bug you reported is fixed in the latest version of
diffoscope, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 981123@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Chris Lamb <lamby@debian.org> (supplier of updated diffoscope package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Sat, 30 Jan 2021 12:31:22 +0000
Source: diffoscope
Architecture: source
Version: 166
Distribution: unstable
Urgency: medium
Maintainer: Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>
Changed-By: Chris Lamb <lamby@debian.org>
Closes: 981123
Changes:
 diffoscope (166) unstable; urgency=medium
 .
   [ Chris Lamb ]
   * New features and bugfixes:
     - Explicitly remove our top-level temporary directory.
       (Closes: #981123, reproducible-builds/diffoscope#234)
     - Increase fuzzy matching threshold to 130 ensure that we show more
       differences. (Closes: reproducible-builds/diffoscope#232)
     - Save our sys.argv in our top-level temporary directory in case it
       helps debug current/errant temporary directories.
     - Prefer to use "magic.Magic" over the "magic.open" compatibility
       interface. (Closes: reproducible-builds/diffoscope#236)
     - Reduce fuzzy threshold to 110 to prevent some test failures.
       (Closes: reproducible-builds/diffoscope#233)
 .
   * Output improvements:
     - Show fuzzyness amount in percentage terms, not out of the
       rather-arbitrary "400".
     - Improve the logging of fuzzy matching.
     - Print the free space in our temporary directory when we create it, not
       from within diffoscope.main.
 .
   * Codebase improvements:
     - Tidy the diffoscopecomparators.utils.fuzzy module.
     - Update my copyright years.
     - Clarify the grammar of a comment.
     - Clarify in a comment that __del__ is not always called, so temporary
       directories are not neccessary removed the *moment* they go out of scope.
 .
   [ Conrad Ratschan ]
   * Fix U-Boot Flattened Image Tree ("FIT") image detection for larger image
     files. (MR: reproducible-builds/diffoscope!75)
Checksums-Sha1:
 37ea31a6cfcdf9389128975b3b51d63b23f90838 4889 diffoscope_166.dsc
 7e3e6a34e5372f6356b453ba8829846c78b655bd 993456 diffoscope_166.tar.xz
 e5465107e89da9c655e6176f28482a753e0c00a9 26992 diffoscope_166_amd64.buildinfo
Checksums-Sha256:
 600aa2521dc0fc9eda3444d26e0b06e4151702f454bdf5c5f8fbb21f6e8e56a3 4889 diffoscope_166.dsc
 dcdd90ca78aa57efa2cdf1577d55098355bca47ecabbccf44d1f813f3c8800d2 993456 diffoscope_166.tar.xz
 7be97018acd3f2c7b3768d71c571c32db1b4feec4caa81a64aea267d2019159f 26992 diffoscope_166_amd64.buildinfo
Files:
 0ed37fae2404c5286e5cbdd6d55d4006 4889 devel optional diffoscope_166.dsc
 1d0b0cff9d2adc6c96dde6277990dcb7 993456 devel optional diffoscope_166.tar.xz
 e7cdeab503d2253157e11b57e4333c30 26992 devel optional diffoscope_166_amd64.buildinfo

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEwv5L0nHBObhsUz5GHpU+J9QxHlgFAmAVU58ACgkQHpU+J9Qx
Hljajg//dsgQ9j3QR9gPjj8Wruk8DgdCcmlR96F//zhprqAxkhFwofx8mLRY3/vy
Aq/U8texmTQM8aSZPcImnwKG8bCXQ9OLDysXs6VsDkm7AnDoIcZfbfFF/6dqRSjW
27Zw09UoxoYtT5GpSQejxS/RRzSBxDsk5O9N580b2XQtHPnFnM2O0DcDAWd2tStC
nlC0Jfs/LN/EVA1KcvX7t3XMfHZt6NPtCMsl22ztMeqsX/uqRh30vVH9K0lRUEx8
FLkhUZZ+ZnqQXkTaDABAx2b/7mGlOwapbxJZqAsoB/T+EcjPnmNQUhqmT7u5mT75
5R65SLSkd7494ioTxlgVtexrpchCrkxjlRUOOiCdOgSmPFedJlTtLOX/KgBZSKN9
5ikgf2qJs6UFwWLEYxGuKvkADmflqVY8pLzNhP6ZA83fvY2bmZF8ztfTRj3pWa7h
7wHsRsYJJ9NJOWOQo9/LiLQZawhPXCMc3FmmFAlNQwOcQNWQkDWXGdwraTwTvg/l
+dygcK3zIvP5oLIasi3UViRWe61IjEMexIDo6bjHMa/htZmTm8Yvl0x5dMIEJBCN
wjDKKBfmAntubBl7h1aIAqUMvHMNvLV4VCpvxYAfDTsSgHz6qvFBApT3uyWAljA2
X4ZCMqqiCtIvAbKBmt1oAu0ajgX6dYTCrBJaRnPlb0aB9AHbYMw=
=Nwqs
-----END PGP SIGNATURE-----

Message #32 received at 981123@bugs.debian.org (full text, mbox, reply):

From: Holger Levsen <holger@layer-acht.org>

To: 981123@bugs.debian.org

Subject: Re: [diffoscope] diffoscope 166 released 💠

Date: Sun, 7 Feb 2021 14:05:36 +0000

[Message part 1 (text/plain, inline)]

control: reopen -1
thanks

hi,

On Sat, Jan 30, 2021 at 01:12:03PM +0000, Chris Lamb wrote:
>   * New features and bugfixes:
>     - Explicitly remove our top-level temporary directory.
>       (Closes: #981123, reproducible-builds/diffoscope#234)

we are still seeing temporary directories not being cleaned up on jenkins.d.n
running diffoscope 166:

root@jenkins:/tmp# du -sch diffo*|egrep 'M|G' 
780M	diffoscope_04wln4t_
2.8G	diffoscope_18ihaolp
2.8G	diffoscope_5dyt60ak
359M	diffoscope__f4aa83s
325M	diffoscope_hltt6x9m
358M	diffoscope_hz3ag_4i
546M	diffoscope_i0oab24m
367M	diffoscope_ibi6bh1t
578M	diffoscope_jc9j5xsc
346M	diffoscope_k4qa3m_q
566M	diffoscope_m_8jvb35
479M	diffoscope_mfmqsycl
780M	diffoscope_n9umadhr
345M	diffoscope_t1o1b776
566M	diffoscope_urum_u22
780M	diffoscope_ws4s20x0
512M	diffoscope_xfvdyxh2
565M	diffoscope_zshqf8n0
341M	diffoscope_zsj1xipc
14G	total
root@jenkins:/tmp# ls -lartd $(du -sch diffo*|egrep 'M|G'|grep -v total |cut -b 5-)
drwx------  9566 jenkins jenkins 191340 Feb  3 15:36 diffoscope_ibi6bh1t
drwx------    10 jenkins jenkins    220 Feb  3 18:06 diffoscope_urum_u22
drwx------  9540 jenkins jenkins 190820 Feb  4 07:10 diffoscope_t1o1b776
drwx------  9551 jenkins jenkins 191040 Feb  4 07:27 diffoscope_hz3ag_4i
drwx------    10 jenkins jenkins    220 Feb  4 07:32 diffoscope_i0oab24m
drwx------    10 jenkins jenkins    220 Feb  4 09:11 diffoscope_m_8jvb35
drwx------  2758 jenkins jenkins  55180 Feb  4 22:18 diffoscope_xfvdyxh2
drwx------     8 jenkins jenkins    180 Feb  5 01:39 diffoscope_ws4s20x0
drwx------    10 jenkins jenkins    220 Feb  5 03:07 diffoscope_zshqf8n0
drwx------    10 jenkins jenkins    220 Feb  5 03:34 diffoscope_jc9j5xsc
drwx------  8519 jenkins jenkins 170400 Feb  5 07:09 diffoscope_hltt6x9m
drwx------ 10795 jenkins jenkins 215920 Feb  6 00:10 diffoscope_zsj1xipc
drwx------    10 jenkins jenkins    220 Feb  6 01:18 diffoscope_mfmqsycl
drwx------   167 jenkins jenkins   3360 Feb  6 20:20 diffoscope_5dyt60ak
drwx------   167 jenkins jenkins   3360 Feb  6 23:20 diffoscope_18ihaolp
drwx------ 10616 jenkins jenkins 212340 Feb  7 01:21 diffoscope_k4qa3m_q
drwx------ 10622 jenkins jenkins 212460 Feb  7 01:23 diffoscope__f4aa83s
drwx------     8 jenkins jenkins    180 Feb  7 01:29 diffoscope_04wln4t_
drwx------     8 jenkins jenkins    180 Feb  7 01:31 diffoscope_n9umadhr
root@jenkins:/tmp# ls -1 diffoscope_* |wc -l
72829
root@jenkins:/tmp# ls -1d diffoscope_* |wc -l
161


-- 
cheers,
	Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁       holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀ PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C
 ⠈⠳⣄

Everyone is entitled to their own opinion, but not their own facts.

[signature.asc (application/pgp-signature, inline)]

Message #41 received at 981123@bugs.debian.org (full text, mbox, reply):

From: "Chris Lamb" <lamby@debian.org>

To: "Holger Levsen" <holger@layer-acht.org>, 981123@bugs.debian.org

Subject: Re: Bug#981123: [diffoscope] diffoscope 166 released 💠

Date: Sun, 07 Feb 2021 17:58:03 -0000

Hi Holger,

> we are still seeing temporary directories not being cleaned up on jenkins.d.n
> running diffoscope 166:

Thanks. Can you pastebin (or similar) some of the new "argv" files
that get generated directly underneath these temporary directories?

I'd like to be able to reproduce this, and these files should be
the first step to working out why these directories are not being
cleaned up anymore.

Getting the actual filenames being saved (with "find -ls"?) might also
be useful too.


Regards,

-- 

    ⢀⣴⠾⠻⢶⣦⠀   
    ⣾⠁⢠⠒⠀⣿⡁   Chris Lamb
    ⢿⡄⠘⠷⠚⠋    lamby@debian.org / chris-lamb.co.uk
    ⠈⠳⣄⠀⠀⠀⠀

Message #46 received at 981123@bugs.debian.org (full text, mbox, reply):

From: Holger Levsen <holger@layer-acht.org>

To: 981123@bugs.debian.org

Subject: Re: Bug#981123: [diffoscope] diffoscope 166 released 💠

Date: Mon, 8 Feb 2021 14:50:25 +0000

[Message part 1 (text/plain, inline)]

Hi Chris,

On Sun, Feb 07, 2021 at 05:58:03PM -0000, Chris Lamb wrote:
> > we are still seeing temporary directories not being cleaned up on jenkins.d.n
> > running diffoscope 166:
> Thanks. Can you pastebin (or similar) some of the new "argv" files
> that get generated directly underneath these temporary directories?

see attached. I hope this is what you want. its the combined output
of these commands:

cd /tmp
ls -lartd $(du -sch diffo*|egrep 'M|G'|grep -v total |cut -b 5-) > tmpfiles
ls -R diffoscope_jc9j5xsc >> tmpfiles

> I'd like to be able to reproduce this, and these files should be
> the first step to working out why these directories are not being
> cleaned up anymore.

I'll be glad to help...!


-- 
cheers,
	Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁       holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀ PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C
 ⠈⠳⣄

That morning, the young barista woman told me that a customer came in with a
mask, but not wearing it. When she asked the customer to put on her mask
please, the woman said: "Why? There's no-one in here."

[tmpfiles (text/plain, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #53 received at 981123@bugs.debian.org (full text, mbox, reply):

From: "Chris Lamb" <lamby@debian.org>

To: "Holger Levsen" <holger@layer-acht.org>, 981123@bugs.debian.org

Subject: Re: Bug#981123: [diffoscope] diffoscope 166 released 💠

Date: Mon, 08 Feb 2021 17:55:04 -0000

Hi Holger,

> > Thanks. Can you pastebin (or similar) some of the new "argv" files
> > that get generated directly underneath these temporary directories?
>
> see attached. I hope this is what you want. its the combined output
> of these commands:
>
> cd /tmp
> ls -lartd $(du -sch diffo*|egrep 'M|G'|grep -v total |cut -b 5-) > tmpfiles
> ls -R diffoscope_jc9j5xsc >> tmpfiles

That provides some of the contents of the directories; thanks. Can you
also provide some of the contents of the "argv" files? These should
contain the actual command-line passed to diffoscope and, as part of
that, the crucial package name so that we can reproduce this locally.


Regards,

--
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org 🍥 chris-lamb.co.uk
       `-

Message #58 received at 981123@bugs.debian.org (full text, mbox, reply):

From: Holger Levsen <holger@layer-acht.org>

To: 981123@bugs.debian.org

Subject: Re: Bug#981123: [diffoscope] diffoscope 166 released 💠

Date: Mon, 8 Feb 2021 19:11:38 +0000

[Message part 1 (text/plain, inline)]

On Mon, Feb 08, 2021 at 05:55:04PM -0000, Chris Lamb wrote:
> That provides some of the contents of the directories; thanks. Can you
> also provide some of the contents of the "argv" files? These should
> contain the actual command-line passed to diffoscope and, as part of
> that, the crucial package name so that we can reproduce this locally.

see attached, that was generated by running this:

cd /tmp ; for i in $(du -sch diffo*|egrep 'M|G'|grep -v total |cut -b 5-) ; 
 do cat $i/argv > ${i}.argv_981123 ; done

IOW, the argv files of the biggest offenders :) All archlinux packages!


-- 
cheers,
	Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁       holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀ PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C
 ⠈⠳⣄

"... the premise [is] that privacy is about hiding a wrong. It's not.
 Privacy is an inherent human right, and a requirement for maintaining
 the human condition with dignity and respect." (Bruce Schneier)

[argv.tgz (application/x-gtar-compressed, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #61 received at 981123-submitter@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>

To: 981123-submitter@bugs.debian.org

Subject: Bug#981123 marked as pending in diffoscope

Date: Tue, 09 Feb 2021 11:53:55 +0000

Control: tag -1 pending

Hello,

Bug #981123 in diffoscope reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:

https://salsa.debian.org/reproducible-builds/diffoscope/-/commit/51469a1a9a8090c5428dbb044681c238786c93d0

------------------------------------------------------------------------
Ensure we cleanup our temporary directory by avoiding confusion between the TemporaryDirectory instance and underlying directory. (Closes: #981123)

We were calling shutil.rmtree on the TemporaryDirectory instance, rather
than its `.name` attribute. Also add a comment about why we can't store
the directory name and we need to keep the instance around.
------------------------------------------------------------------------

(this message was generated automatically)
-- 
Greetings

https://bugs.debian.org/981123

Message #68 received at 981123-close@bugs.debian.org (full text, mbox, reply):

From: Debian FTP Masters <ftpmaster@ftp-master.debian.org>

To: 981123-close@bugs.debian.org

Subject: Bug#981123: fixed in diffoscope 167

Date: Fri, 19 Feb 2021 09:18:32 +0000

Source: diffoscope
Source-Version: 167
Done: Chris Lamb <lamby@debian.org>

We believe that the bug you reported is fixed in the latest version of
diffoscope, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 981123@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Chris Lamb <lamby@debian.org> (supplier of updated diffoscope package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Fri, 19 Feb 2021 09:06:42 +0000
Source: diffoscope
Built-For-Profiles: nocheck
Architecture: source
Version: 167
Distribution: unstable
Urgency: medium
Maintainer: Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>
Changed-By: Chris Lamb <lamby@debian.org>
Closes: 981123
Changes:
 diffoscope (167) unstable; urgency=medium
 .
   * Temporary directory handling:
     - Ensure we cleanup our temporary directory by avoiding confusion between
       the TemporaryDirectory instance and the underlying directory.
       (Closes: #981123)
     - Use a potentially-useful suffix to our temporary directory based on the
       command-line passed to diffoscope.
     - Fix some tempfile/weakref interaction in Python 3.7 (ie. Debian buster).
       (Closes: reproducible-builds/diffoscope#239)
     - If our temporary directory does not exist anymore (eg. it has been
       cleaned up in tests, signal handling or reference counting),  make sure
       we recreate it.
 .
   * Bug fixes:
     - Don't rely on magic.Magic(...) to have an identical API between file's
       magic.py and PyPI's "python-magic" library.
       (Closes: reproducible-builds/diffoscope#238)
     - Don't rely on dumpimage returning an appropriate exit code; check that
       the file actually exists after we call it.
 .
   * Codebase changes:
     - Set a default Config.extended_filesystem_attributes.
     - Drop unused Config.acl and Config.xattr attributes.
     - Tidy imports in diffoscope/comparators/fit.py.
 .
   * Tests:
     - Add u-boot-tools to test dependencies so that salsa.debian.org pipelines
       actually test the new FIT comparator.
     - Strip newlines when determining Black version to avoid "requires black
       >= 20.8b1 (18.9b0\n detected)" in test output (NB. embedded newline).
     - Gnumeric is back in testing so re-add to test dependencies.
     - Use assert_diff (over get_data, etc.) in the FIT and APK comparators.
     - Mark test_apk.py::test_android_manifest as being allowed to fail for now.
     - Fix the FIT tests in buster and unstable.
Checksums-Sha1:
 30acce7d5d3b8372c03a368824403f016446f065 4935 diffoscope_167.dsc
 c49f55dc53d09b67cb466d14872810deed81a5d5 994344 diffoscope_167.tar.xz
 154c144c39a92620675fe9a53d1ce340460eb87a 6827 diffoscope_167_amd64.buildinfo
Checksums-Sha256:
 3ddb0350223fd908368262a609a5beb77f9cbad74bb7e853befece5a993e3369 4935 diffoscope_167.dsc
 d3a7c606fb1d0f38225c1fdc0a3d3a4a7b2147752aa451f56ff263f07bda1711 994344 diffoscope_167.tar.xz
 dc7fdf20c82fd3e63f60b11793dd01fb9850396dc46e82dc16592bdb3798e482 6827 diffoscope_167_amd64.buildinfo
Files:
 def519c1998462ff060c71ec2418285d 4935 devel optional diffoscope_167.dsc
 731e49ba144af3d0c84a63410b1930c8 994344 devel optional diffoscope_167.tar.xz
 e93432f6031ee3da6a5b043d8e7b93a6 6827 devel optional diffoscope_167_amd64.buildinfo

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEwv5L0nHBObhsUz5GHpU+J9QxHlgFAmAvf9QACgkQHpU+J9Qx
Hli/HQ//Ueb3eFQSkm+xCCCxt4V2VUp2EnGgGh3D24uW670v0FbhssAypURzDgWo
OCONnylQkJMHKgAx8rCtp62SyCIkY0TryqQAZYkgaMaGUPnui5feGf3JaCul03jY
AvpubLONwHslnFQGJMSJOktIM8TTP9CkxFbwhh7KAvvIyTSHG3uDWBpSUxIkRPzB
B2l3tFnGLDk1LVkXiOp31IWI+yRscU0/zYW88a1F3ciSEwQOpcMwYRpbVio7SePQ
HX0XxED8UO13HLHTVNoBbR/h141+IghUY+d8SDrovlnf0X+dpcvgYqQ8dfZhOvXS
0PBLYFasRca8b3BG65q8VKYY7FONEBkTXGh+RDCUtg/VnqINLQqMtnhkLC+dgxbr
K0TQnT7T167g/K5JilZ1Izejhsm0bbhht0mwQNDoKKrBwWJX3Va99aCiE0sdyM2X
MdLXBPfORvrTSd3NwXkQDKlN42QeOTkEPHWMQLSyrA+TtuXGL+zrlsnQ7kefDWS1
oAC8S/dKvk5iiK5BvaEQNXtnQiVcYonw3JLyEvBSlL5mJyNQaqF5wKbPDhuo+K0f
JxGf7ToDXTIedoHzxnkGj6C4uBg/8NGKfBD3Kb6MfPOTJTqZC6qk2ZjpHazkNeRQ
1g8hqaLcD+gCiYPqUNSA/LkrohSOySL06ictGtmIhDYcO66XYRI=
=+jpp
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.