Debian Bug report logs - #808041
s390-zfcp: Install Debian on FC-attached SCSI devices on s390

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Hendrik Brueckner <brueckner@linux.vnet.ibm.com>

To: submit@bugs.debian.org

Subject: s390-zfcp: Install Debian on FC-attached SCSI devices on s390

Date: Tue, 15 Dec 2015 15:05:15 +0100

[Message part 1 (text/plain, inline)]

Package: d-i.debian.org
Severity: normal
Tags: d-i

Dear Debian Installer Team and Maintainers,

I would like to contribute a new package for the Debian Installer
to configure FCP devices and install Debian on SCSI devices which
are available on Linux on z Systems.

FCP device provide access to SAN storage, for example, SCSI devices.
SCSI devices and DASD devices are the main storage devices on a mainframe.

With the FCP configuration utility, users can enable FCP devices.
Depending on the hardware configuration of the FCP devices, LUNs will
be automatically detected and attached;  alternatively, users are guided
to manually add LUNs.  The FCP configuration utility can be preseeded
to perform an unattended configuration of FCP devices.

I will attach the patches for this new utility, as well as, a README
with more details in another mail.

Feedback for the FCP configuration utility is very welcome!
I am looking forward to discuss and work with you on the steps to make
it available with the Debian Installer.

Thank you very much!

Kind regards,
  Hendrik

-- 
Hendrik Brueckner
brueckner@linux.vnet.ibm.com      | IBM Deutschland Research & Development GmbH
Linux on z Systems Development    | Schoenaicher Str. 220, 71032 Boeblingen

[Message part 2 (application/pgp-signature, inline)]

Message #10 received at 808041@bugs.debian.org (full text, mbox, reply):

From: Hendrik Brueckner <brueckner@linux.vnet.ibm.com>

To: 808041@bugs.debian.org

Subject: Re: Bug#808041: s390-zfcp: Install Debian on FC-attached SCSI devices on s390 (patches)

Date: Tue, 15 Dec 2015 15:15:42 +0100

[Message part 1 (text/plain, inline)]

Control: tags -1 + patch

Below you can find the patches for the FCP configuration utility.
The patch set consists of five patches:

1. A README file describing how to use the FCP configuration utility.
2. Generic interfaces and functions to manage CCW devices on s390.
3. Functions to manage FCP devices.
4. The FCP configuration utility.
5. Debian packaging files to build the Debian Installer udeb.

Initial tests to configure FCP devices ran successful on z/VM and LPAR
using FCP devices with and without N_Port ID virtualization (NPIV).

Review and feedback of the patches is very welcome!  As a beginner in
Debian packaging, please review the Debian packaging files.

Thank you in advance!

Kind regards,
  Hendrik

[0001-s390-zfcp-README.patch (text/plain, attachment)]

[0002-s390-dev-Add-functions-to-manage-CCW-devices-on-s390.patch (text/plain, attachment)]

[0003-s390-zfcp-Add-FCP-device-specific-functions.patch (text/plain, attachment)]

[0004-zfcp-config-Add-utility-to-configure-FCP-devices-on-.patch (text/plain, attachment)]

[0005-debian-Add-Debian-packaging-files.patch (text/plain, attachment)]

[Message part 7 (application/pgp-signature, inline)]

Message #17 received at 808041@bugs.debian.org (full text, mbox, reply):

From: Cyril Brulebois <kibi@debian.org>

To: Hendrik Brueckner <brueckner@linux.vnet.ibm.com>, 808041@bugs.debian.org

Cc: Philipp Kern <pkern@debian.org>, Debian S/390 Team <debian-s390@lists.debian.org>

Subject: Re: Bug#808041: s390-zfcp: Install Debian on FC-attached SCSI devices on s390

Date: Tue, 15 Dec 2015 18:07:23 +0100

[Message part 1 (text/plain, inline)]

Hi!

Hendrik Brueckner <brueckner@linux.vnet.ibm.com> (2015-12-15):
> Package: d-i.debian.org
> Severity: normal
> Tags: d-i
> 
> Dear Debian Installer Team and Maintainers,
> 
> I would like to contribute a new package for the Debian Installer
> to configure FCP devices and install Debian on SCSI devices which
> are available on Linux on z Systems.
> 
> FCP device provide access to SAN storage, for example, SCSI devices.
> SCSI devices and DASD devices are the main storage devices on a mainframe.
> 
> With the FCP configuration utility, users can enable FCP devices.
> Depending on the hardware configuration of the FCP devices, LUNs will
> be automatically detected and attached;  alternatively, users are guided
> to manually add LUNs.  The FCP configuration utility can be preseeded
> to perform an unattended configuration of FCP devices.
> 
> I will attach the patches for this new utility, as well as, a README
> with more details in another mail.
> 
> Feedback for the FCP configuration utility is very welcome!
> I am looking forward to discuss and work with you on the steps to make
> it available with the Debian Installer.

Thanks! I'm adding Philipp and S/390 people in copy so that we get as
many eyes as possible on your proposed patches. I'm afraid I don't know
much about this platform for one thing, and lack time to learn more
about it right now for another; sorry about that. :/

Mraw,
KiBi.

[signature.asc (application/pgp-signature, inline)]

Message #22 received at 808041@bugs.debian.org (full text, mbox, reply):

From: Dimitri John Ledkov <xnox@debian.org>

To: Cyril Brulebois <kibi@debian.org>

Cc: Hendrik Brueckner <brueckner@linux.vnet.ibm.com>, 808041@bugs.debian.org, Philipp Kern <pkern@debian.org>, "Debian S/390 Team" <debian-s390@lists.debian.org>

Subject: Re: Bug#808041: s390-zfcp: Install Debian on FC-attached SCSI devices on s390

Date: Tue, 15 Dec 2015 19:33:56 +0000

Hi!

On 15 December 2015 at 17:07, Cyril Brulebois <kibi@debian.org> wrote:
> Hi!
>
> Hendrik Brueckner <brueckner@linux.vnet.ibm.com> (2015-12-15):
>> Package: d-i.debian.org
>> Severity: normal
>> Tags: d-i
>>
>> Dear Debian Installer Team and Maintainers,
>>
>> I would like to contribute a new package for the Debian Installer
>> to configure FCP devices and install Debian on SCSI devices which
>> are available on Linux on z Systems.
>>
>> FCP device provide access to SAN storage, for example, SCSI devices.
>> SCSI devices and DASD devices are the main storage devices on a mainframe.
>>
>> With the FCP configuration utility, users can enable FCP devices.
>> Depending on the hardware configuration of the FCP devices, LUNs will
>> be automatically detected and attached;  alternatively, users are guided
>> to manually add LUNs.  The FCP configuration utility can be preseeded
>> to perform an unattended configuration of FCP devices.
>>
>> I will attach the patches for this new utility, as well as, a README
>> with more details in another mail.
>>
>> Feedback for the FCP configuration utility is very welcome!
>> I am looking forward to discuss and work with you on the steps to make
>> it available with the Debian Installer.
>
> Thanks! I'm adding Philipp and S/390 people in copy so that we get as
> many eyes as possible on your proposed patches. I'm afraid I don't know
> much about this platform for one thing, and lack time to learn more
> about it right now for another; sorry about that. :/


I'm part of debian-installer team, and have been mostly uploading
partman-* et.al. packages. So I don't know all of the d-i, but do have
a fairly good working knowledge of it. I certainly can review /
sponsor things for you.

I also have access to s390x hardware to test things out. But my s390x
specific skills are extremely basic at the moment. I can just about
ipl things to get me into installer.

-- 
Regards,

Dimitri.

Message #27 received at 808041@bugs.debian.org (full text, mbox, reply):

From: Hendrik Brueckner <brueckner@linux.vnet.ibm.com>

To: Cyril Brulebois <kibi@debian.org>

Cc: Hendrik Brueckner <brueckner@linux.vnet.ibm.com>, 808041@bugs.debian.org, Philipp Kern <pkern@debian.org>, Debian S/390 Team <debian-s390@lists.debian.org>

Subject: Re: Bug#808041: s390-zfcp: Install Debian on FC-attached SCSI devices on s390

Date: Wed, 16 Dec 2015 14:31:46 +0100

[Message part 1 (text/plain, inline)]

Hi!

On Tue, Dec 15, 2015 at 06:07:23PM +0100, Cyril Brulebois wrote:
> Hendrik Brueckner <brueckner@linux.vnet.ibm.com> (2015-12-15):
> > Package: d-i.debian.org
> > Severity: normal
> > Tags: d-i
> > 
> > Dear Debian Installer Team and Maintainers,
> > 
> > I would like to contribute a new package for the Debian Installer
> > to configure FCP devices and install Debian on SCSI devices which
> > are available on Linux on z Systems.
> > 
> > FCP device provide access to SAN storage, for example, SCSI devices.
> > SCSI devices and DASD devices are the main storage devices on a mainframe.
> > 
> > With the FCP configuration utility, users can enable FCP devices.
> > Depending on the hardware configuration of the FCP devices, LUNs will
> > be automatically detected and attached;  alternatively, users are guided
> > to manually add LUNs.  The FCP configuration utility can be preseeded
> > to perform an unattended configuration of FCP devices.
> > 
> > I will attach the patches for this new utility, as well as, a README
> > with more details in another mail.
> > 
> > Feedback for the FCP configuration utility is very welcome!
> > I am looking forward to discuss and work with you on the steps to make
> > it available with the Debian Installer.
> 
> Thanks! I'm adding Philipp and S/390 people in copy so that we get as
> many eyes as possible on your proposed patches.
>

Thank you... CCing is more clear.  I initially added Philipp and the s390
mailing using X-Debbugs-CC.

> I'm afraid I don't know
> much about this platform for one thing, and lack time to learn more
> about it right now for another; sorry about that. :/

No problem.   Getting access to s390 hardware is a bit difficult.
But any help and feedback regarding the integration into debian
and the debian installer is very helpful.

Thanks and kind regards,
  Hendrik

[Message part 2 (application/pgp-signature, inline)]

Message #32 received at 808041@bugs.debian.org (full text, mbox, reply):

From: Hendrik Brueckner <brueckner@linux.vnet.ibm.com>

To: Dimitri John Ledkov <xnox@debian.org>, 808041@bugs.debian.org

Cc: Cyril Brulebois <kibi@debian.org>, Hendrik Brueckner <brueckner@linux.vnet.ibm.com>, Philipp Kern <pkern@debian.org>, Debian S/390 Team <debian-s390@lists.debian.org>

Subject: Re: Bug#808041: s390-zfcp: Install Debian on FC-attached SCSI devices on s390

Date: Wed, 16 Dec 2015 16:36:19 +0100

Hi Dimitri,

On Tue, Dec 15, 2015 at 07:33:56PM +0000, Dimitri John Ledkov wrote:
> On 15 December 2015 at 17:07, Cyril Brulebois <kibi@debian.org> wrote:
> > Hendrik Brueckner <brueckner@linux.vnet.ibm.com> (2015-12-15):
> >> Package: d-i.debian.org
> >> Severity: normal
> >> Tags: d-i
> >>
> >> Dear Debian Installer Team and Maintainers,
> >>
> >> I would like to contribute a new package for the Debian Installer
> >> to configure FCP devices and install Debian on SCSI devices which
> >> are available on Linux on z Systems.
> >>
> >> FCP device provide access to SAN storage, for example, SCSI devices.
> >> SCSI devices and DASD devices are the main storage devices on a mainframe.
> >>
> >> With the FCP configuration utility, users can enable FCP devices.
> >> Depending on the hardware configuration of the FCP devices, LUNs will
> >> be automatically detected and attached;  alternatively, users are guided
> >> to manually add LUNs.  The FCP configuration utility can be preseeded
> >> to perform an unattended configuration of FCP devices.
> >>
> >> I will attach the patches for this new utility, as well as, a README
> >> with more details in another mail.
> >>
> >> Feedback for the FCP configuration utility is very welcome!
> >> I am looking forward to discuss and work with you on the steps to make
> >> it available with the Debian Installer.
> >
> > Thanks! I'm adding Philipp and S/390 people in copy so that we get as
> > many eyes as possible on your proposed patches. I'm afraid I don't know
> > much about this platform for one thing, and lack time to learn more
> > about it right now for another; sorry about that. :/
> 
> 
> I'm part of debian-installer team, and have been mostly uploading
> partman-* et.al. packages. So I don't know all of the d-i, but do have
> a fairly good working knowledge of it. I certainly can review /
> sponsor things for you.

Great!  Thank you very much for helping me and it would be amazing if you
could act as sponsor.

> I also have access to s390x hardware to test things out. But my s390x
> specific skills are extremely basic at the moment. I can just about
> ipl things to get me into installer.
> 

That's good to know ;-)  First of all, if you questions regarding s390x,
do not hesitate to ask me or the debian-s390 mailing list.  If you can
IPL the system and get into the installer, you have already master lots
of things.  Within the Linux instance, then, things looks similar to Linux
on other platforms.... maybe except the way how devices are managed.

Let me provide some more insights into device management on the mainframe
because that's one of the fundamental differences to other platforms.

A mainframe is a large system and there can be numerous devices.  These
devices managed through an I/O subsystem, called the channel subsystem.
To handle devices on the channel subsystem, the Linux kernel provides a
dedicated layer, the common I/O layer (CIO).  This layer represents the
maninframe (s390) devices to Linux.  A device on the channel subsystem
is represented by a bus-ID in Linux, for example, 0.0.1224.  The "1234"
portion is the device number ranging from 0000 to FFFF.  Managing this
range of possible devices can be complex.

Due to the complexity, devices are disabled by default. Hence, devices
that the Linux instances require must be enabled first, i.e., the devices
must be set online.  This step can include some device configuration
changes.
After the devices are enabled, they appear to the Linux instance, for
example, as DASDs or FCP devices.  Some device configuration can or must
be performed at enablement.

This particular enablement step is unique to Linux on the mainframe.
After enablement, device management is similar to other Linux platforms.

So that's why there the DASD and the FCP device configuration module: to
enable DASD and FCP devices and block devices and SCSI devices available
to the Linux instance.  Then, they can be partitioned like any other
SCSI disk.


Thanks and kind regards,
  Hendrik

-- 
Hendrik Brueckner
brueckner@linux.vnet.ibm.com      | IBM Deutschland Research & Development GmbH
Linux on z Systems Development    | Schoenaicher Str. 220, 71032 Boeblingen


IBM Deutschland Research & Development GmbH
Vorsitzender des Aufsichtsrats: Martina Koederitz
Geschaeftsfuehrung: Dirk Wittkopp
Sitz der Gesellschaft: Boeblingen
Registergericht: Amtsgericht Stuttgart, HRB 243294

Message #35 received at 808041@bugs.debian.org (full text, mbox, reply):

From: Philipp Kern <pkern@debian.org>

To: Hendrik Brueckner <brueckner@linux.vnet.ibm.com>, 808041@bugs.debian.org

Subject: Re: Bug#808041: s390-zfcp: Install Debian on FC-attached SCSI devices on s390 (patches)

Date: Sat, 19 Dec 2015 00:03:37 +0100

[Message part 1 (text/plain, inline)]

Hi,

On Tue, Dec 15, 2015 at 03:15:42PM +0100, Hendrik Brueckner wrote:
> Below you can find the patches for the FCP configuration utility.
> The patch set consists of five patches:
> 
> 1. A README file describing how to use the FCP configuration utility.
> 2. Generic interfaces and functions to manage CCW devices on s390.
> 3. Functions to manage FCP devices.
> 4. The FCP configuration utility.
> 5. Debian packaging files to build the Debian Installer udeb.
> 
> Initial tests to configure FCP devices ran successful on z/VM and LPAR
> using FCP devices with and without N_Port ID virtualization (NPIV).
> 
> Review and feedback of the patches is very welcome!  As a beginner in
> Debian packaging, please review the Debian packaging files.

I took the liberty to push the patchset to [1]. This will still require
review, but any potential fix-up commits can just be based on your work.

(I hope to have some more time over the holidays to review, but as said
I can't test it.)

Kind regards and thanks
Philipp Kern

[1] http://anonscm.debian.org/cgit/d-i/s390-zfcp.git

[signature.asc (application/pgp-signature, inline)]

Message #38 received at 808041@bugs.debian.org (full text, mbox, reply):

From: Philipp Kern <pkern@debian.org>

To: Hendrik Brueckner <brueckner@linux.vnet.ibm.com>

Cc: Dimitri John Ledkov <xnox@debian.org>, 808041@bugs.debian.org, Cyril Brulebois <kibi@debian.org>, Debian S/390 Team <debian-s390@lists.debian.org>

Subject: Re: Bug#808041: s390-zfcp: Install Debian on FC-attached SCSI devices on s390

Date: Fri, 18 Dec 2015 23:59:21 +0100

Hendrik,

On Wed, Dec 16, 2015 at 04:36:19PM +0100, Hendrik Brueckner wrote:
> So that's why there the DASD and the FCP device configuration module: to
> enable DASD and FCP devices and block devices and SCSI devices available
> to the Linux instance.  Then, they can be partitioned like any other
> SCSI disk.

(I have installed Debian on zFCP in the past, but I moved it from zVM.)

Is there a way I could test FCP installation? As far as I understand
there is no emulation of a zFCP-like setup anywhere right, and you'd
need to have access to a zFCP controller?

(Which are harder to partition/segment than DASD-based storage…)

Kind regards
Philipp Kern

Message #43 received at 808041@bugs.debian.org (full text, mbox, reply):

From: Viktor Mihajlovski <mihajlov@linux.vnet.ibm.com>

To: 808041@bugs.debian.org

Subject: Re: Bug#808041: s390-zfcp: Install Debian on FC-attached SCSI devices on s390

Date: Mon, 21 Dec 2015 14:26:51 +0100

On Fri, 18 Dec 2015 23:59:21 +0100 Philipp Kern <pkern@debian.org> wrote:
...
> 
> Is there a way I could test FCP installation? As far as I understand
> there is no emulation of a zFCP-like setup anywhere right, and you'd
> need to have access to a zFCP controller?
> 
> (Which are harder to partition/segment than DASD-based storage…)
...

Thanks for pushing the patches!

I don't know of FCP emulation either. To test the module, you would
indeed have to have a zFCP HBA defined plus the necessary storage
network setup (with zoning/LUN masking) configured.

As I already have tested Hendrik's original patchset, building off of
the d-i repository will probably not lead to new insights. I could
however offer to re-run tests once a udeb is built for s390-fcp.
If I can provide additional info (logs of an install, screenshots, ...),
I'd be more than happy to do so.

-- 

Mit freundlichen Grüßen/Kind Regards
   Viktor Mihajlovski

IBM Deutschland Research & Development GmbH
Vorsitzender des Aufsichtsrats: Martina Köderitz
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen
Registergericht: Amtsgericht Stuttgart, HRB 243294

Message #46 received at 808041@bugs.debian.org (full text, mbox, reply):

From: Philipp Kern <pkern@debian.org>

To: Viktor Mihajlovski <mihajlov@linux.vnet.ibm.com>, 808041@bugs.debian.org

Cc: debian-s390@lists.debian.org, Hendrik Brueckner <brueckner@linux.vnet.ibm.com>

Subject: Re: Bug#808041: s390-zfcp: Install Debian on FC-attached SCSI devices on s390

Date: Wed, 6 Jan 2016 11:56:04 +0100

[Message part 1 (text/plain, inline)]

Control: tag -1 + pending

On Mon, Dec 21, 2015 at 02:26:51PM +0100, Viktor Mihajlovski wrote:
> On Fri, 18 Dec 2015 23:59:21 +0100 Philipp Kern <pkern@debian.org> wrote:
> ...
> > 
> > Is there a way I could test FCP installation? As far as I understand
> > there is no emulation of a zFCP-like setup anywhere right, and you'd
> > need to have access to a zFCP controller?
> > 
> > (Which are harder to partition/segment than DASD-based storage…)
> ...
> 
> Thanks for pushing the patches!
> 
> I don't know of FCP emulation either. To test the module, you would
> indeed have to have a zFCP HBA defined plus the necessary storage
> network setup (with zoning/LUN masking) configured.
> 
> As I already have tested Hendrik's original patchset, building off of
> the d-i repository will probably not lead to new insights. I could
> however offer to re-run tests once a udeb is built for s390-fcp.
> If I can provide additional info (logs of an install, screenshots, ...),
> I'd be more than happy to do so.

Going forward I am concerned with any possibility to regression test
this. I'd be helpful to arrange access for a crash-and-burn VM with a
zFCP HBA attached somewhere. (I know that this is difficult because it
also requires more elaborate storage zoning.)

The other case I'm wondering about is how well the installer will handle
the non-presence of DASDs and the co-existence of both.

In other news I just pushed the package to Debian NEW. There might be
concerns about the proper labeling of the license, in which case we need
to iterate a bit. But the package itself looked ready to upload.

Kind regards and thanks
Philipp Kern

[signature.asc (application/pgp-signature, inline)]

Message #53 received at 808041-close@bugs.debian.org (full text, mbox, reply):

From: Philipp Kern <pkern@debian.org>

To: 808041-close@bugs.debian.org

Subject: Bug#808041: fixed in s390-zfcp 1.0.0

Date: Mon, 08 Feb 2016 02:00:51 +0000

Source: s390-zfcp
Source-Version: 1.0.0

We believe that the bug you reported is fixed in the latest version of
s390-zfcp, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 808041@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Philipp Kern <pkern@debian.org> (supplier of updated s390-zfcp package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Wed, 06 Jan 2016 11:36:46 +0100
Source: s390-zfcp
Binary: s390-zfcp
Architecture: source s390x
Version: 1.0.0
Distribution: unstable
Urgency: medium
Maintainer: Debian Install System Team <debian-boot@lists.debian.org>
Changed-By: Philipp Kern <pkern@debian.org>
Description:
 s390-zfcp  - Activate and configure FCP devices for installation (udeb)
Closes: 808041
Changes:
 s390-zfcp (1.0.0) unstable; urgency=medium
 .
   [ Hendrik Brueckner ]
   * Initial release (Closes: #808041)
 .
   [ Philipp Kern ]
   * Minor fixups for Debian policy compliance.
Checksums-Sha1:
 a0e7480de929d7270fa04cf77e44cde34bbe33fa 1253 s390-zfcp_1.0.0.dsc
 8d32f0a00199adf4096b87233049cc5c2d7f29c4 23884 s390-zfcp_1.0.0.tar.xz
 fe2b345e689c3514a883d1957c2bb7fa0cf847bf 12470 s390-zfcp_1.0.0_s390x.udeb
Checksums-Sha256:
 1e974e14e1fa4d504f9ed3cc6793fe25b5a0e8b886a6f5360737dd3c0ef2b123 1253 s390-zfcp_1.0.0.dsc
 5a7617f3f7ef6507a9d35a6162146224232656c18c7c2be1603560ba3b3205d4 23884 s390-zfcp_1.0.0.tar.xz
 5f066a67317ccb0a658a0d7f78f3fc5a199173d4618131680f1a21c1c42c42de 12470 s390-zfcp_1.0.0_s390x.udeb
Files:
 d4d55a981d4f854de39a95b6801e8902 1253 debian-installer standard s390-zfcp_1.0.0.dsc
 8e281aefd62c249394741513dc8e9a00 23884 debian-installer standard s390-zfcp_1.0.0.tar.xz
 a479aaa11769323ebf75d4db41391379 12470 debian-installer standard s390-zfcp_1.0.0_s390x.udeb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBCAAGBQJWjO98AAoJEERuJUU10FbspTUIALDBEnLEjLom88HhRx2oSqQy
chVh6Ode9stecID39XLxoEji67MrBaUH67hvCG302Q4n3F8biJCL55U0VfkMukJ9
SAATIhNCJzGlmpejDh3Ypvfs177NWxStbbBJHaRJMjkO45BIjmV/47fUyYILN1cF
iI0EVd3aNgopj1jqELTyaAE6PkAHOj4b6Mz2uGmNmI18U9dU5jOhu8LHhxbVM1zH
mRXah/iK++CGIsR/cxcMJh79BgfGofiEbOogvZmyyxud7aDqcXcDSaosOIj5qV1q
Syz+vPqwzchWLVyXvessSo4bBL6MbmFw9vggRN7BYlhh02c7qE6DdxyIFPMEQ8Y=
=Nbp0
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.