Debian Bug report logs - #880709
zfsutils-linux 0.7.3 has an unlisted dependency on libuutil1linux >= 0.7.3

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Rich Ercolani <rincebrain@gmail.com>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: zfsutils-linux 0.7.3 has an unlisted dependency on libuutil1linux >= 0.7.3

Date: Sat, 04 Nov 2017 00:17:48 -0400

Package: zfsutils-linux
Version: 0.7.3-1
Severity: important

Dear Maintainer,

As the subject says, if you just install {zfsutils-linux,spl-dkms,zfs-dkms}/unstable, you can end up with libuutil1linux from stable or testing, and get back:
zpool: symbol lookup error: /lib/libzpool.so.2: undefined symbol: spl_pagesize
Explicitly installing libuutil1linux >= 0.7.3 resolves this, so the package should probably have an explicit version dependency listed.

(As you can see, I commonly configure stable > testing > unstable pinning preferences, so booting a new stretch machine and installing the above with those pinnings will result in this.)

- Rich

-- System Information:
Debian Release: 9.2
  APT prefers stable
  APT policy: (1000, 'stable'), (900, 'testing'), (800, 'unstable'), (500, 'stable-updates')
Architecture: amd64 (x86_64)

Kernel: Linux 4.9.0-3-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages zfsutils-linux depends on:
ii  libblkid1        2.29.2-1
ii  libc6            2.24-11+deb9u1
ii  libnvpair1linux  0.7.3-1
ii  libuuid1         2.29.2-1
ii  libuutil1linux   0.6.5.9-5
ii  libzfs2linux     0.7.3-1
ii  libzpool2linux   0.7.3-1
ii  python3          3.5.3-1
ii  zlib1g           1:1.2.8.dfsg-5

Versions of packages zfsutils-linux recommends:
ii  lsb-base                9.20161125
ii  zfs-dkms [zfs-modules]  0.7.3-1
ii  zfs-zed                 0.7.3-1

Versions of packages zfsutils-linux suggests:
pn  nfs-kernel-server           <none>
pn  samba-common-bin            <none>
pn  zfs-initramfs | zfs-dracut  <none>

-- no debconf information

Message #10 received at 880709@bugs.debian.org (full text, mbox, reply):

From: Nathaniel W Filardo <nwf@cs.jhu.edu>

To: 880709@bugs.debian.org

Subject: zfsutils-linux 0.7.3 has an unlisted dependency on libuutil1linux >= 0.7.3

Date: Tue, 7 Nov 2017 19:17:40 -0500

[Message part 1 (text/plain, inline)]

It appears that libzpool2linux is also an unlisted dependency.  Merely
having libuutil1linux installed was enough to clear the error of this bug
but left "/sbin/zfs: symbol lookup error: /lib/libzfs.so.2: undefined
symbol: SHA2Update" behind until libzpool2linux got added to the system.

Cheers,
--nwf;

[signature.asc (application/pgp-signature, inline)]

Message #15 received at 880709@bugs.debian.org (full text, mbox, reply):

From: Fabian Grünbichler <f.gruenbichler@proxmox.com>

To: Rich Ercolani <rincebrain@gmail.com>, 880709@bugs.debian.org

Subject: Re: Bug#880709: zfsutils-linux 0.7.3 has an unlisted dependency on libuutil1linux >= 0.7.3

Date: Thu, 16 Nov 2017 15:40:12 +0100

On Sat, Nov 04, 2017 at 12:17:48AM -0400, Rich Ercolani wrote:
> Package: zfsutils-linux
> Version: 0.7.3-1
> Severity: important
> 
> Dear Maintainer,
> 
> As the subject says, if you just install {zfsutils-linux,spl-dkms,zfs-dkms}/unstable, you can end up with libuutil1linux from stable or testing, and get back:
> zpool: symbol lookup error: /lib/libzpool.so.2: undefined symbol: spl_pagesize
> Explicitly installing libuutil1linux >= 0.7.3 resolves this, so the package should probably have an explicit version dependency listed.
> 
> (As you can see, I commonly configure stable > testing > unstable pinning preferences, so booting a new stretch machine and installing the above with those pinnings will result in this.)
> 
> - Rich

I see two ways to go forward here:
- add symbols files for the libraries and keep them uptodate
- require libraries and utilities to have the exact same version
  (upstream does not guarantuee ABI/API stability yet AFAIK, only
  on-disk-format stability).

I'll evaluate the symbols route tomorrow to see how much work it is
(starting from 0.6.5.11-1).

furthermore, the question of how to handle the userspace -> module
dependency needs to be solved somehow. again, there are two basic
possibilities:

- add a versioned dependency to the same major version now, and bump it
  whenever incompatibilities are known
- always depend on an identically versioned module

thoughts?

Message #20 received at 880709@bugs.debian.org (full text, mbox, reply):

From: Rich <rincebrain@gmail.com>

To: Fabian Grünbichler <f.gruenbichler@proxmox.com>

Cc: 880709@bugs.debian.org

Subject: Re: Bug#880709: zfsutils-linux 0.7.3 has an unlisted dependency on libuutil1linux >= 0.7.3

Date: Thu, 16 Nov 2017 10:09:45 -0500

[Message part 1 (text/plain, inline)]

I'm not sure I understand what you mean by symbol files for the libraries,
here - I know what symbols are, and usually I'd expect symbol files to
refer to something like external debug symbols, but it sounds like you're
suggesting something else?

I'd pretty strongly suggest depending on identically versioned module,
since AFAIK the guarantee for what happens if you replace the userland and
not the kernel module even across point versions is "nothing".

Of course, then you still have the problem of needing to reload the module
after replacing the userland, which you don't always get to do if you're
doing ZFS root...

Maybe it'd be a reasonable thing to do to try and get a patch upstream to
notice when userland and module version disagree and print a notice that
all bets are off. But then you'd need text processing to understand the
module versioning in each for anything but exact matching e.g. 0.7.3-1
versus 0.7.3.

At a minimum, you'd want to depend on the same point version (e.g. 0.7.3-1
and 0.7.3-2 should be fine to mix, unless we start integrating really
disruptive changes that aren't planned to go into the mainline version),
though we might want to just opt for exact version so we don't have to
worry about overlooking a change that would break this, but trying to do
that string processing wouldn't necessarily be portable or accepted
upstream (e.g. consider the versions emitted by git, IIRC of the form
0.X.Y-g[git shorthash], and you _definitely_ don't want to mix across
those).

Maybe just exact version matching for now (for both module/userland and
userland/dependent libraries) and an upstream enhancement request to notify
people they might set their house on fire if they mix differing
module/userland versions?

- Rich

On Thu, Nov 16, 2017 at 9:40 AM, Fabian Grünbichler <
f.gruenbichler@proxmox.com> wrote:

> On Sat, Nov 04, 2017 at 12:17:48AM -0400, Rich Ercolani wrote:
> > Package: zfsutils-linux
> > Version: 0.7.3-1
> > Severity: important
> >
> > Dear Maintainer,
> >
> > As the subject says, if you just install {zfsutils-linux,spl-dkms,zfs-dkms}/unstable,
> you can end up with libuutil1linux from stable or testing, and get back:
> > zpool: symbol lookup error: /lib/libzpool.so.2: undefined symbol:
> spl_pagesize
> > Explicitly installing libuutil1linux >= 0.7.3 resolves this, so the
> package should probably have an explicit version dependency listed.
> >
> > (As you can see, I commonly configure stable > testing > unstable
> pinning preferences, so booting a new stretch machine and installing the
> above with those pinnings will result in this.)
> >
> > - Rich
>
> I see two ways to go forward here:
> - add symbols files for the libraries and keep them uptodate
> - require libraries and utilities to have the exact same version
>   (upstream does not guarantuee ABI/API stability yet AFAIK, only
>   on-disk-format stability).
>
> I'll evaluate the symbols route tomorrow to see how much work it is
> (starting from 0.6.5.11-1).
>
> furthermore, the question of how to handle the userspace -> module
> dependency needs to be solved somehow. again, there are two basic
> possibilities:
>
> - add a versioned dependency to the same major version now, and bump it
>   whenever incompatibilities are known
> - always depend on an identically versioned module
>
> thoughts?
>
>

[Message part 2 (text/html, inline)]

Message #25 received at 880709@bugs.debian.org (full text, mbox, reply):

From: Fabian Grünbichler <f.gruenbichler@proxmox.com>

To: Rich <rincebrain@gmail.com>, 880709@bugs.debian.org

Subject: Re: Bug#880709: zfsutils-linux 0.7.3 has an unlisted dependency on libuutil1linux >= 0.7.3

Date: Fri, 17 Nov 2017 09:34:36 +0100

On Thu, Nov 16, 2017 at 10:09:45AM -0500, Rich wrote:
> I'm not sure I understand what you mean by symbol files for the libraries,
> here - I know what symbols are, and usually I'd expect symbol files to
> refer to something like external debug symbols, but it sounds like you're
> suggesting something else?

(usual disclaimer: I am NOT a/the maintainer of ZoL in Debian, just a
(downstream) user making suggestions / contributing patches)

symbols refers to symbols exported by the libraries and used by programs
linking with them. a symbols file is a way to track changes (it
basically lists the library version where a symbol was last changed in a
breaking manner, or introduced if no such change has happened since
then). it does mean a bit of additional overhead after importing
upstream versions, to add new symbols and check existing ones for
breaking changes (especially with an upstream like ZoL, where there are
no versioned symbols / stable public ABI / ...).

using a symbols file generated with 0.6.5.11-1 and updated using
0.7.3-1, this would generate the following versioned dependencies:

zfsutils-linux
  -> libnvpair1linux, libuutil1linux >= 0.6.5.11-1
  -> libzpool2linux, libzfs2linux >= 0.7.3-1
zfs-zed
  -> libnvpair1linux, libuutil1linux, libzpool2linux >= 0.6.5.11-1
  -> libzfs2linux >= 0.7.3-1
zfs-test
  -> libnvpair1linux, libuutil1linux, libzfs2linux >= 0.6.5.11-1
  -> libzpool2linux >= 0.7.3-1

libnvpair1linux
  -> libuutil1linux >= 0.6.5.11-1
libuutil1linux
  -> no new versioned dependencies
libzfs2linux
  -> libnvpair1linux >= 0.6.5.11-1
  -> libuutil1linux, libzpool2linux >= 0.7.3-1
libzpool2linux
  -> libnvpair1linux >= 0.6.5.11-1
  -> libuutil1linux >= 0.7.3-1

which would have pulled in libuutil1linux on upgrades (on a system with
zfsutils-linux installed, only libnvpair1linux would have stayed at
0.6.5.11-1 in a partial upgrade scenario).

> 
> I'd pretty strongly suggest depending on identically versioned module,
> since AFAIK the guarantee for what happens if you replace the userland and
> not the kernel module even across point versions is "nothing".
> 
> Of course, then you still have the problem of needing to reload the module
> after replacing the userland, which you don't always get to do if you're
> doing ZFS root...

and verifying that the DKMS build was successful.

in my experience, mixing user space and kernel modules within a release
series works good enough to not force this dependency, but bumping it
for major releases or known incompatibilities is a good idea for sure.
the delta between minor releases is usually in the range of 10-20
commits, so checking those for (potential) issues is not that hard.

> 
> Maybe it'd be a reasonable thing to do to try and get a patch upstream to
> notice when userland and module version disagree and print a notice that
> all bets are off. But then you'd need text processing to understand the
> module versioning in each for anything but exact matching e.g. 0.7.3-1
> versus 0.7.3.

also, I am not sure if there is a notion of 'version' for the userspace
besides library sonames, which are not bumped often enough in ZoL to be
reliable. I doubt upstream wants to check package versions ;)

> 
> At a minimum, you'd want to depend on the same point version (e.g. 0.7.3-1
> and 0.7.3-2 should be fine to mix, unless we start integrating really
> disruptive changes that aren't planned to go into the mainline version),
> though we might want to just opt for exact version so we don't have to
> worry about overlooking a change that would break this, but trying to do
> that string processing wouldn't necessarily be portable or accepted
> upstream (e.g. consider the versions emitted by git, IIRC of the form
> 0.X.Y-g[git shorthash], and you _definitely_ don't want to mix across
> those).

see above

> 
> Maybe just exact version matching for now (for both module/userland and
> userland/dependent libraries) and an upstream enhancement request to notify
> people they might set their house on fire if they mix differing
> module/userland versions?

I'll send a patch with symbols files for the library packages later
today or on Monday, IMHO the rest is up to the maintainers ;)

Message #30 received at 880709@bugs.debian.org (full text, mbox, reply):

From: Antonio Russo <antonio.e.russo@gmail.com>

To: 880709@bugs.debian.org

Subject: Library versions

Date: Sun, 19 Nov 2017 23:23:30 -0500

All upstream testing and development is done with matching library and utility versions.
I really don't see any advantage to making fine-grained dependencies that expose Debian
users to unconventional configurations with some possibly non-matching library versions.
Just my 2 cents.

Antonio

Message #35 received at 880709@bugs.debian.org (full text, mbox, reply):

From: Fabian Grünbichler <f.gruenbichler@proxmox.com>

To: Antonio Russo <antonio.e.russo@gmail.com>, 880709@bugs.debian.org

Subject: Re: Bug#880709: Library versions

Date: Mon, 20 Nov 2017 09:53:20 +0100

[Message part 1 (text/plain, inline)]

On Sun, Nov 19, 2017 at 11:23:30PM -0500, Antonio Russo wrote:
> All upstream testing and development is done with matching library and utility versions.
> I really don't see any advantage to making fine-grained dependencies that expose Debian
> users to unconventional configurations with some possibly non-matching library versions.
> Just my 2 cents.

After a bit more experiments, I tend to agree with you (especially given
the huge number of symbols exported by libzpool!). Attached is a patch
that simply emulates upstreams "break with every release" (tested with
sid and stretch-backport builds). I prefer this approach to hard-coding
the exact versions in d/control, as it also works for other packages
linking with ZFS libraries using the shlibs mechanism.

I think I will start a discussion with upstream regarding their plans
for future ABI guarantees and proper exporting and versioning of public
library symbols. Switching to proper symbols based versioned
dependencies in Debian should be possible without much pain anyway, if
and when upstream decides to declare the library interface (somewhat)
stable.

[0001-build-add-implicit-version-to-dh_makeshlibs.patch (text/x-diff, attachment)]

Message #42 received at 880709-close@bugs.debian.org (full text, mbox, reply):

From: Aron Xu <aron@debian.org>

To: 880709-close@bugs.debian.org

Subject: Bug#880709: fixed in zfs-linux 0.7.3-2

Date: Tue, 28 Nov 2017 09:11:28 +0000

Source: zfs-linux
Source-Version: 0.7.3-2

We believe that the bug you reported is fixed in the latest version of
zfs-linux, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 880709@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Aron Xu <aron@debian.org> (supplier of updated zfs-linux package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Tue, 28 Nov 2017 16:16:34 +0800
Source: zfs-linux
Binary: libnvpair1linux libuutil1linux libzfslinux-dev libzfs2linux libzpool2linux zfs-dkms zfs-initramfs zfs-dracut zfsutils-linux zfs-zed zfs-test zfs-dbg
Architecture: source amd64 all
Version: 0.7.3-2
Distribution: unstable
Urgency: medium
Maintainer: Debian ZFS on Linux maintainers <pkg-zfsonlinux-devel@lists.alioth.debian.org>
Changed-By: Aron Xu <aron@debian.org>
Description:
 libnvpair1linux - Solaris name-value library for Linux
 libuutil1linux - Solaris userland utility library for Linux
 libzfs2linux - OpenZFS filesystem library for Linux
 libzfslinux-dev - OpenZFS filesystem development files for Linux
 libzpool2linux - OpenZFS pool library for Linux
 zfs-dbg    - Debugging symbols for OpenZFS userland libraries and tools
 zfs-dkms   - OpenZFS filesystem kernel modules for Linux
 zfs-dracut - OpenZFS root filesystem capabilities for Linux - dracut
 zfs-initramfs - OpenZFS root filesystem capabilities for Linux - initramfs
 zfs-test   - OpenZFS test infrastructure an support scripts
 zfs-zed    - OpenZFS Event Daemon
 zfsutils-linux - command-line tools to manage OpenZFS filesystems
Closes: 880709 880902 882209
Changes:
 zfs-linux (0.7.3-2) unstable; urgency=medium
 .
   [ Fabian Grünbichler ]
   * d/rules: remove obsolete calls to dpkg-architecture (Closes: #882209)
   * zfs-test: add proper Breaks+Replaces (Closes: #880902)
   * build: add implicit version to dh_makeshlibs (Closes: #880709)
 .
   [ Aron Xu ]
   * Depend on matching version of spl-dkms (Closes: ##881013)
Checksums-Sha1:
 54225dbb393856a6a8a471196baf7ec93ea3b201 2663 zfs-linux_0.7.3-2.dsc
 3470ce564ee978304174f6763efc565f9e816a6f 41540 zfs-linux_0.7.3-2.debian.tar.xz
 a6f5ee09508de84dcbbd5c9da959563ea0a408d6 45680 libnvpair1linux_0.7.3-2_amd64.deb
 68c6511fb1a922c3287f1f9c11ef166388600a4a 48116 libuutil1linux_0.7.3-2_amd64.deb
 5b00361b08d45f30565cd446c3a906ba09a9271a 138076 libzfs2linux_0.7.3-2_amd64.deb
 9eb27c5a2f1df342036d35380191d6a322c6af63 1024008 libzfslinux-dev_0.7.3-2_amd64.deb
 c636c71ba8215a162d7844b13ccb1208934bbd94 562920 libzpool2linux_0.7.3-2_amd64.deb
 b4d969233c9a9336050cede7b1347c8a0018c7a9 3719392 zfs-dbg_0.7.3-2_amd64.deb
 330df1340fed6e35775b8fc1b1159ab16aa0f454 1381936 zfs-dkms_0.7.3-2_all.deb
 7c472c2be4e0de5d69f0f46cf9651f00a9f84396 20868 zfs-dracut_0.7.3-2_all.deb
 5a7bf9990a4d8735e7173172903b19289527780f 24412 zfs-initramfs_0.7.3-2_all.deb
 e6bcd65340b5ea987860fab6191b98b8aa0c9b24 8736 zfs-linux_0.7.3-2_amd64.buildinfo
 258e73e322d1e7a4bfe93f3a124a2b81fb99587b 2463732 zfs-test_0.7.3-2_amd64.deb
 a95dfb8a4eb27611ee6ebbb97ec9c4ab61d7ab79 60232 zfs-zed_0.7.3-2_amd64.deb
 ad4a07dd3ce3fd034eb718841722ad73d4a65297 349556 zfsutils-linux_0.7.3-2_amd64.deb
Checksums-Sha256:
 53407931f098eb27ef8bcd528e363b1c6a5ace3f8d4054f459b35e8dedf8854e 2663 zfs-linux_0.7.3-2.dsc
 d8d74841faeca3747d52e9365f3a25e32390744c8abf8e4af2543e005d1064e0 41540 zfs-linux_0.7.3-2.debian.tar.xz
 155edd92365ffab23153a8b9e96c1156d0245bcd321c446703cc9934ce04b5da 45680 libnvpair1linux_0.7.3-2_amd64.deb
 b12f449af004a9aa8b85194ca56287aaba5231f8396fcf6f34a5e401af772349 48116 libuutil1linux_0.7.3-2_amd64.deb
 4e4a881bfbb1e41bf993fe26b71c0bacec98c19a1b39eb73d76edadc6c24ecd5 138076 libzfs2linux_0.7.3-2_amd64.deb
 cdb623c01318235bb949cddcc4e31bedea66062c2bef4a13a71de81fe7c53202 1024008 libzfslinux-dev_0.7.3-2_amd64.deb
 16e45b1b979415fd3839d4c31cf1e2783edab582bb2207ac0e0084df07de6846 562920 libzpool2linux_0.7.3-2_amd64.deb
 ad150a6914e426f45e3d8e0c2fd33cf036c0803ac7f21bf9a4f4ba189afdae64 3719392 zfs-dbg_0.7.3-2_amd64.deb
 377a0b3b09330787560724287d7bea31763f4e4783363e0b38806f31a2e3524f 1381936 zfs-dkms_0.7.3-2_all.deb
 66bced87d9ab6c2ca75dfba5f4e4cc4fbba9fe8ec7bfb5c0aa0387242c2576f9 20868 zfs-dracut_0.7.3-2_all.deb
 dcadc59dc22e9ac38a9b6b9f33891b4ef04e1af100c48e619b90a9c308a1b7a8 24412 zfs-initramfs_0.7.3-2_all.deb
 a73ce75dbbb6b429c02153c463f9f19c8798eac09b8d36728f1784a59aee2f5f 8736 zfs-linux_0.7.3-2_amd64.buildinfo
 dce291ff8d2a2a255ae29e999c29da0dafd382a2106e152aaf69e8e44450f32c 2463732 zfs-test_0.7.3-2_amd64.deb
 3c8e8849dea235b9a86e7da47ba4b599a507484c82865e23bc01db46ca105f84 60232 zfs-zed_0.7.3-2_amd64.deb
 85decf2f2011fb8ed14521294a03cea7290b60c8fdb8a20678aada5d3a25d9f2 349556 zfsutils-linux_0.7.3-2_amd64.deb
Files:
 0d0e8c4f7722c5f6b0c4a6e9c67f0751 2663 contrib/kernel optional zfs-linux_0.7.3-2.dsc
 a700e29685b227dad062d05052b2b0e9 41540 contrib/kernel optional zfs-linux_0.7.3-2.debian.tar.xz
 111ff44d1868341b94640988ee499f9c 45680 contrib/libs optional libnvpair1linux_0.7.3-2_amd64.deb
 b3d17cf66ff953ff0361d8c50ac946a5 48116 contrib/libs optional libuutil1linux_0.7.3-2_amd64.deb
 7c55ff8a02c0f3139b0f7ab0567ec018 138076 contrib/libs optional libzfs2linux_0.7.3-2_amd64.deb
 2b54b0e5707e03717a91a707a438c10f 1024008 contrib/libdevel optional libzfslinux-dev_0.7.3-2_amd64.deb
 82d02296c7f629728d869e58d5dee510 562920 contrib/libs optional libzpool2linux_0.7.3-2_amd64.deb
 c6cf2aab4b6e844ccc51e6c031f915ec 3719392 contrib/debug extra zfs-dbg_0.7.3-2_amd64.deb
 99f2e221461667530d2bd453a843d3ad 1381936 contrib/kernel optional zfs-dkms_0.7.3-2_all.deb
 b1019d9aaf38a95cf504a5e827f3df81 20868 contrib/kernel optional zfs-dracut_0.7.3-2_all.deb
 486d1669fb22114c7c55b61b8c791504 24412 contrib/kernel optional zfs-initramfs_0.7.3-2_all.deb
 88d2af4c8c83efa71c155196747857f2 8736 contrib/kernel optional zfs-linux_0.7.3-2_amd64.buildinfo
 d742b19d636c1aae14a42034043126de 2463732 contrib/admin optional zfs-test_0.7.3-2_amd64.deb
 9933aaaf0b12f19f8590fd5d5e8d33bd 60232 contrib/admin optional zfs-zed_0.7.3-2_amd64.deb
 6a34fc9f739c14d36d79be2af512007e 349556 contrib/admin optional zfsutils-linux_0.7.3-2_amd64.deb

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEsW3g9zWrjdJ64EeNev3ArdrqwV4FAlodHSsACgkQev3Ardrq
wV6phQgAtIghYCz1mJi6S+tcPJrSvFLjqd0e8LIyTSBOZtbiE92j34XgpTR4qyzI
OHb/5pYanGzqQMvAncNYv026r6JkzthyXMLSE7HMZkYQ6HnVBRbp5Z7AZxnN0Lz9
6qFMXl0asA78iJIev3SjS+IOw9s7T3cpXQrL2CuWGDu5JR6IV02T2iHFBIAxw1Bf
+XXlplwCRajxyEDSRu/MjjsPQt5/8VUBfeKmACpiaAnkBrHB350d5qkgi/GyOGt5
wuNt6V18aLEFZryzxfKXUoZgYB6v9acoAyaTm5iCaB0oReLmev966ZsCS5GJElVh
VOAfEn3DqkdHhVYiCNpN7RC65w7DwA==
=csPW
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.