Debian Bug report logs - #704594
ITP: python-arpy -- library for accessing the archive files and reading the contents

version graph

Package: wnpp; Maintainer for wnpp is wnpp@debian.org;

Reported by: Christoph Egger <christoph@debian.org>

Date: Wed, 3 Apr 2013 10:36:02 UTC

Owned by: Christoph Egger <christoph@debian.org>

Severity: wishlist

Fixed in version python-arpy/1.1.1-1

Done: Christoph Egger <christoph@debian.org>

Bug is archived. No further changes may be made.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, debian-devel@lists.debian.org, debian-python@lists.debian.org, wnpp@debian.org:
Bug#704594; Package wnpp. (Wed, 03 Apr 2013 10:36:06 GMT) Full text and rfc822 format available.

Acknowledgement sent to Christoph Egger <christoph@debian.org>:
New Bug report received and forwarded. Copy sent to debian-devel@lists.debian.org, debian-python@lists.debian.org, wnpp@debian.org. (Wed, 03 Apr 2013 10:36:06 GMT) Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: Christoph Egger <christoph@debian.org>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: ITP: python-arpy -- library for accessing the archive files and reading the contents
Date: Wed, 03 Apr 2013 12:34:12 +0200
Package: wnpp
Severity: wishlist
Owner: Christoph Egger <christoph@debian.org>

* Package name    : python-arpy
  Version         : 1.0
  Upstream Author : Stanisław Pitucha <viraptor@gmail.com>
* URL             : http://bitbucket.org/viraptor/arpy
* License         : Simplified BSD
  Programming Lang: python
  Description     : library for accessing the "ar"

arpy is a library for accessing the archive files and reading the
contents. It supports extended long filenames in both GNU and BSD
format. Right now it does not support the symbol tables, but can
ignore them gracefully.

==========

Note: Right now I plan to put this into python-mudules SVN and set the
team as `Maintainer:' if there are no objections even though I'm not
really active there. I do plan to take care of the package and not
move the work to active team members.

    Christoph



Information forwarded to debian-bugs-dist@lists.debian.org, wnpp@debian.org, Christoph Egger <christoph@debian.org>:
Bug#704594; Package wnpp. (Wed, 03 Apr 2013 13:21:08 GMT) Full text and rfc822 format available.

Acknowledgement sent to Helmut Grohne <helmut@subdivi.de>:
Extra info received and forwarded to list. Copy sent to wnpp@debian.org, Christoph Egger <christoph@debian.org>. (Wed, 03 Apr 2013 13:21:08 GMT) Full text and rfc822 format available.

Message #10 received at 704594@bugs.debian.org (full text, mbox):

From: Helmut Grohne <helmut@subdivi.de>
To: Christoph Egger <christoph@debian.org>, 704594@bugs.debian.org
Subject: Re: Bug#704594: ITP: python-arpy -- library for accessing the archive files and reading the contents
Date: Wed, 3 Apr 2013 15:15:43 +0200
On Wed, Apr 03, 2013 at 12:34:12PM +0200, Christoph Egger wrote:
> arpy is a library for accessing the archive files and reading the
> contents. It supports extended long filenames in both GNU and BSD
> format. Right now it does not support the symbol tables, but can
> ignore them gracefully.

Can you briefly explain the benefit of this library to you or the Debian
project? As far as I can see this package mostly provides a very small
subset of python-debian. There is one feature I missed in
python-debian's arfile implementation, but this isn't present in arpy
either: Being able to interpret an archive read from a pipe. For
instance the tarfile standard Python module does this for tar files. I
see no reason for why a library for ar archives should fail at such a
basic task. Maybe give upstream some more time to fix things on their
end and get the API right before including it into Debian?

Helmut



Information forwarded to debian-bugs-dist@lists.debian.org, wnpp@debian.org:
Bug#704594; Package wnpp. (Wed, 03 Apr 2013 13:24:07 GMT) Full text and rfc822 format available.

Acknowledgement sent to Christoph Egger <christoph@debian.org>:
Extra info received and forwarded to list. Copy sent to wnpp@debian.org. (Wed, 03 Apr 2013 13:24:07 GMT) Full text and rfc822 format available.

Message #15 received at 704594@bugs.debian.org (full text, mbox):

From: Christoph Egger <christoph@debian.org>
To: Helmut Grohne <helmut@subdivi.de>
Cc: 704594@bugs.debian.org
Subject: Re: Bug#704594: ITP: python-arpy -- library for accessing the archive files and reading the contents
Date: Wed, 03 Apr 2013 15:22:17 +0200
Hi!

Helmut Grohne <helmut@subdivi.de> writes:
> On Wed, Apr 03, 2013 at 12:34:12PM +0200, Christoph Egger wrote:
>> arpy is a library for accessing the archive files and reading the
>> contents. It supports extended long filenames in both GNU and BSD
>> format. Right now it does not support the symbol tables, but can
>> ignore them gracefully.
>
> Can you briefly explain the benefit of this library to you or the Debian
> project? As far as I can see this package mostly provides a very small
> subset of python-debian.

  Well I need a arfile implementation independant of Debian (for
non-debian-specific software). arpy provides exactly that. Asking
Fedora or even Windows users to install python-debian for not-at-all
Debian related software seems a bit strange.

> There is one feature I missed in python-debian's arfile
> implementation, but this isn't present in arpy either: Being able to
> interpret an archive read from a pipe.

  The 1.0.0 I intent to use can read ar from any file handle and should
therefore be able to provide exactly that. I have successfully used it
to handle bzip2 compressed ar-archives using python's bzip2 module.

> For instance the tarfile standard Python module does this for tar
> files. I see no reason for why a library for ar archives should fail
> at such a basic task. Maybe give upstream some more time to fix things
> on their end and get the API right before including it into Debian?

Regards

    Christoph



Information forwarded to debian-bugs-dist@lists.debian.org, wnpp@debian.org, Christoph Egger <christoph@debian.org>:
Bug#704594; Package wnpp. (Wed, 03 Apr 2013 13:48:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Helmut Grohne <helmut@subdivi.de>:
Extra info received and forwarded to list. Copy sent to wnpp@debian.org, Christoph Egger <christoph@debian.org>. (Wed, 03 Apr 2013 13:48:04 GMT) Full text and rfc822 format available.

Message #20 received at 704594@bugs.debian.org (full text, mbox):

From: Helmut Grohne <helmut@subdivi.de>
To: Christoph Egger <christoph@debian.org>
Cc: 704594@bugs.debian.org
Subject: Re: Bug#704594: ITP: python-arpy -- library for accessing the archive files and reading the contents
Date: Wed, 3 Apr 2013 15:44:33 +0200
On Wed, Apr 03, 2013 at 03:22:17PM +0200, Christoph Egger wrote:
>   The 1.0.0 I intent to use can read ar from any file handle and should
> therefore be able to provide exactly that. I have successfully used it
> to handle bzip2 compressed ar-archives using python's bzip2 module.

This is because the bz2 module supports seeking. A pipe usually does not
support seeking. Try reading from sys.stdin. I had to write my own ar
implementation for dedup.debian.net[1], because all the ones around
including arpy suck at streams. It really is not that hard to get this
right. So I ask you to delay uploading arpy to Debian until upstream has
fixed their API to support streaming reads.

Here is a test case to see the brokenness:

cat test/bsd_mixed.ar | python -c 'import arpy, sys; print(arpy.Archive(fileobj=sys.stdin).read_next_header())' 

Helmut

[1] You can find the source in the git repo mentioned on the site and I
    can relicense it for inclusion into arpy, but it is fairly minimal
    to just support the bits that I needed. But then again arpy is
    pretty minimal too.



Information forwarded to debian-bugs-dist@lists.debian.org, wnpp@debian.org:
Bug#704594; Package wnpp. (Wed, 03 Apr 2013 14:09:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Christoph Egger <christoph@debian.org>:
Extra info received and forwarded to list. Copy sent to wnpp@debian.org. (Wed, 03 Apr 2013 14:09:04 GMT) Full text and rfc822 format available.

Message #25 received at 704594@bugs.debian.org (full text, mbox):

From: Christoph Egger <christoph@debian.org>
To: Helmut Grohne <helmut@subdivi.de>
Cc: 704594@bugs.debian.org
Subject: Re: Bug#704594: ITP: python-arpy -- library for accessing the archive files and reading the contents
Date: Wed, 03 Apr 2013 16:04:24 +0200
Hi!

Helmut Grohne <helmut@subdivi.de> writes:
> So I ask you to delay uploading arpy to Debian until upstream has
> fixed their API to support streaming reads.

  So what do you suggest as a solution for an debian-independent ar
module *right* *now*? I don't care specifically about arpy or whatever
else as long as it can a) handle compressed ar-s and b) is
vendor-neutral. arpy seems to provide both and I am currently not aware
of any (reasonable) alternatives.

Regards

    Christoph



Information forwarded to debian-bugs-dist@lists.debian.org, wnpp@debian.org, Christoph Egger <christoph@debian.org>:
Bug#704594; Package wnpp. (Wed, 03 Apr 2013 14:30:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Helmut Grohne <helmut@subdivi.de>:
Extra info received and forwarded to list. Copy sent to wnpp@debian.org, Christoph Egger <christoph@debian.org>. (Wed, 03 Apr 2013 14:30:04 GMT) Full text and rfc822 format available.

Message #30 received at 704594@bugs.debian.org (full text, mbox):

From: Helmut Grohne <helmut@subdivi.de>
To: Christoph Egger <christoph@debian.org>
Cc: 704594@bugs.debian.org
Subject: Re: Bug#704594: ITP: python-arpy -- library for accessing the archive files and reading the contents
Date: Wed, 3 Apr 2013 16:27:43 +0200
On Wed, Apr 03, 2013 at 04:04:24PM +0200, Christoph Egger wrote:
>   So what do you suggest as a solution for an debian-independent ar
> module *right* *now*? I don't care specifically about arpy or whatever
> else as long as it can a) handle compressed ar-s and b) is
> vendor-neutral. arpy seems to provide both and I am currently not aware
> of any (reasonable) alternatives.

Work with upstream to address the remaining issue. There already are way
too many incomplete implementations. Similarly I had to put packaging
python-ssdeep on hold, because the upstream software has too many severe
issues. Not every software on the planet is worth including in Debian as
is and arpy needlessly fails at a very basic task.

The main problem here is that as far as I can see the API will have to
change in order to address this legitimate use case. Once a library has
larger adoption, changing it is harder and this issue will never be
fixed. Better do it now before exposing in Debian. I even provided a
proof of concept for how to do it.

Failure to insist on getting this right will just result in multiple ar
parsing libraries. I imagine that one version can read all the variants,
another one can read from stdin, a third one can write. This is a
disservice to our users and wastes resources (especially in package
lists).

So maybe we can work on a solution here? I'll try to patch arpy to
support this use case. Give me a week?

Helmut



Information forwarded to debian-bugs-dist@lists.debian.org, wnpp@debian.org, Christoph Egger <christoph@debian.org>:
Bug#704594; Package wnpp. (Wed, 03 Apr 2013 15:45:09 GMT) Full text and rfc822 format available.

Acknowledgement sent to Helmut Grohne <helmut@subdivi.de>:
Extra info received and forwarded to list. Copy sent to wnpp@debian.org, Christoph Egger <christoph@debian.org>. (Wed, 03 Apr 2013 15:45:09 GMT) Full text and rfc822 format available.

Message #35 received at 704594@bugs.debian.org (full text, mbox):

From: Helmut Grohne <helmut@subdivi.de>
To: Christoph Egger <christoph@debian.org>, 704594@bugs.debian.org
Subject: Re: Bug#704594: ITP: python-arpy -- library for accessing the archive files and reading the contents
Date: Wed, 3 Apr 2013 17:42:10 +0200
[Message part 1 (text/plain, inline)]
On Wed, Apr 03, 2013 at 04:27:43PM +0200, Helmut Grohne wrote:
> So maybe we can work on a solution here? I'll try to patch arpy to
> support this use case. Give me a week?

Turned out to be easier than expected. Maybe less arguing and more
fixing would have helped here. The attached patch makes reading from
stdin possible. It has one major drawback the moment. When your backing
fileobj does support seeking it still emulates forward seeks using
reads, because there is no way to reliably detect the availability of
seek. So accessing random members of an Archive could result in very bad
performance (especially if combined with bz2). In order to fix that the
user has to provide some indication whether seek is desired. When using
Archive without seek only the current ArchiveFileData may be read from
start till end once. To ease working with the Archive class I turned it
into an iterator yielding ArchiveFileDatas. When using this iterator
forward seeks are almost never necessary. You can omit it, but I still
think it to be useful.

Helmut
[arpy.py.patch (text/x-diff, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, wnpp@debian.org, Christoph Egger <christoph@debian.org>:
Bug#704594; Package wnpp. (Sun, 30 Jun 2013 21:39:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Stanisław Pitucha <viraptor@gmail.com>:
Extra info received and forwarded to list. Copy sent to wnpp@debian.org, Christoph Egger <christoph@debian.org>. (Sun, 30 Jun 2013 21:39:04 GMT) Full text and rfc822 format available.

Message #40 received at 704594@bugs.debian.org (full text, mbox):

From: Stanisław Pitucha <viraptor@gmail.com>
To: 704594@bugs.debian.org
Subject: Re: Bug#704594: ITP: python-arpy -- library for accessing the archive files and reading the contents
Date: Sun, 30 Jun 2013 22:36:48 +0100
[Message part 1 (text/plain, inline)]
Hi,
The patch has been included in the upstream release. I modified the
behaviour afterwards though.
Changes:
- no checking for negative offsets (seek() already does that)
- seeking ability is detected at initialisation, so that read-as-seek is
only used on non-seekable streams
- there's an explicit exception when seek is requested but can't be used

(also project has been moved to https://github.com/viraptor/arpy to get
travis-ci integration, old bitbucket page redirects there - current version
1.1.1)

-- 
KTHXBYE,

Stanisław Pitucha
[Message part 2 (text/html, inline)]

Added tag(s) pending. Request was from Anibal Monsalve Salazar <anibal@debian.org> to control@bugs.debian.org. (Sat, 06 Jul 2013 20:06:04 GMT) Full text and rfc822 format available.

Added indication that bug 704594 blocks 718498 Request was from Helmut Grohne <helmut@subdivi.de> to submit@bugs.debian.org. (Thu, 01 Aug 2013 14:51:07 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, wnpp@debian.org, Christoph Egger <christoph@debian.org>:
Bug#704594; Package wnpp. (Thu, 01 Aug 2013 15:18:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Steve Langasek <vorlon@debian.org>:
Extra info received and forwarded to list. Copy sent to wnpp@debian.org, Christoph Egger <christoph@debian.org>. (Thu, 01 Aug 2013 15:18:04 GMT) Full text and rfc822 format available.

Message #49 received at 704594@bugs.debian.org (full text, mbox):

From: Steve Langasek <vorlon@debian.org>
To: debian-devel@lists.debian.org, 718498@bugs.debian.org, 704594@bugs.debian.org
Subject: Re: python-debian: switch ar implementation to python-arpy
Date: Thu, 1 Aug 2013 16:15:52 +0100
[Message part 1 (text/plain, inline)]
On Thu, Aug 01, 2013 at 04:47:45PM +0200, Helmut Grohne wrote:
> Package: python-debian
> Version: 0.1.21+nmu2
> Severity: wishlist
> Control: block -1 by 704594

> On Thu, Aug 01, 2013 at 04:19:42PM +0200, Stefano Zacchiroli wrote:
> > Can you please file a bug report about that (ideally marking the pending
> > ITPs as blockers for it)? Regarding changing interfaces it would be
> > better not to do that, of course, but even more so because it's not
> > clear to me how many users of the debfile module are out there in the
> > wild. We can do a call for it or something, but let's move the
> > discussion to an appropriate bug report first.

> Sure. Here is your bug. So let me give a summary of advantages of the
> move:
>  * Less code.
>  * Works with non-seekable fileobjs.
>  * #642020 is fixed for free (but there also is a patch).
>  * #689313 would be obsoleted (but there also is a patch).
>  * python-arpy has a responsive upstream and maintainer.

The python-arpy package in the NEW queue doesn't have a python3 version, so
this would regress python-debian.  You might want to fix this first.

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek@ubuntu.com                                     vorlon@debian.org
[signature.asc (application/pgp-signature, inline)]

Reply sent to Christoph Egger <christoph@debian.org>:
You have taken responsibility. (Fri, 09 Aug 2013 00:03:08 GMT) Full text and rfc822 format available.

Notification sent to Christoph Egger <christoph@debian.org>:
Bug acknowledged by developer. (Fri, 09 Aug 2013 00:03:08 GMT) Full text and rfc822 format available.

Message #54 received at 704594-close@bugs.debian.org (full text, mbox):

From: Christoph Egger <christoph@debian.org>
To: 704594-close@bugs.debian.org
Subject: Bug#704594: fixed in python-arpy 1.1.1-1
Date: Fri, 09 Aug 2013 00:00:11 +0000
Source: python-arpy
Source-Version: 1.1.1-1

We believe that the bug you reported is fixed in the latest version of
python-arpy, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 704594@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Christoph Egger <christoph@debian.org> (supplier of updated python-arpy package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.8
Date: Mon, 18 Mar 2013 17:15:12 +0100
Source: python-arpy
Binary: python-arpy
Architecture: source all
Version: 1.1.1-1
Distribution: unstable
Urgency: low
Maintainer: Debian Python Modules Team <python-modules-team@lists.alioth.debian.org>
Changed-By: Christoph Egger <christoph@debian.org>
Description: 
 python-arpy - library for accessing "ar" archives
Closes: 704594
Changes: 
 python-arpy (1.1.1-1) unstable; urgency=low
 .
   * Initial release. (Closes: #704594)
Checksums-Sha1: 
 ee1ab4bb14bb44a8aeea33b1808246a2d22f90da 1848 python-arpy_1.1.1-1.dsc
 dc782f49726037b4abbf7e9135e8113c938fc535 7171 python-arpy_1.1.1.orig.tar.gz
 0a5a241e0324cd22821e910ad20fe320253170af 1810 python-arpy_1.1.1-1.debian.tar.gz
 6d800d4ec1c7d5e59bfe95ac2318657e1888464f 6412 python-arpy_1.1.1-1_all.deb
Checksums-Sha256: 
 40cd5f67da82b44dbc08aef297838b99036b380a78c5e4257e4a27e5c2cd495a 1848 python-arpy_1.1.1-1.dsc
 a3aba7bcd7948b5632d2eb8ce0e8196314c55281adef42dc52409e695f5ca15a 7171 python-arpy_1.1.1.orig.tar.gz
 83eb5ea81b0cc31ce30eb5996ffbfe0158ab6df1b31ac3b99c38a6c25f8d787e 1810 python-arpy_1.1.1-1.debian.tar.gz
 60c106d09d4ba3550debaa6d84941ff668e77d8cf0fd57075a87c8460faf4e9b 6412 python-arpy_1.1.1-1_all.deb
Files: 
 f5ced9d2053936b80655e35c1f8d57de 1848 python optional python-arpy_1.1.1-1.dsc
 e2ada3acac83e87b5b7385591da16340 7171 python optional python-arpy_1.1.1.orig.tar.gz
 a9ae55d5dc0b52388457c22d609b4559 1810 python optional python-arpy_1.1.1-1.debian.tar.gz
 8e1ec3131867b67dadf5423c39fc2c91 6412 python optional python-arpy_1.1.1-1_all.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iQIcBAEBCgAGBQJR2BWDAAoJEKv/7bJACMb5abgP/1+tAO74KRB23AhHhPJy9Tjx
MrRYHadeXmR1Uo6IZw4GLdO7kRwM1ZF4wV10OUpSgGn2MZmPwWSudx8Wp2fwbYT+
mU3+OAWtodDzvZxgAYbTgRFCOn2IO3kZ03R0dJtuLNNKR8Q2n5isO8JqHD5cO+z5
RCiOaH29xP/AYrlYF9TQFay8/SqQEJ9q1133DZBht75+Y+XyRIQCUNl+7yBWPhno
FCRr50pqnLdRQ/kNrUfljBSeKiEHxXppd8zcSkRagg8cKsbevnkik7t/NeRX4cUO
MtNaObnDmhjnBn2VPwkg0Gdi2xhqO2xzk/tc/yObuqE1otcI6wBfooXevVxR9FkG
45dMIdQ93kSdxIJ1TFH+QFm8OTpvfIE3l9m3WN0oJVtj2ziNhGCPA+nfpXMJPQgA
mUjHHOXLjL1gjntQtTqPuYJBFdfo8Ff4RPK8t1DLb5hmwUsFswfQoRpnY2hAlRob
jUVNK4RkJbMWvucYZfmuOIDJIxcxIpgTxtt3WXv/ERHDUxODA8L0L+2LqSFKutne
tAm6VXba60Gadg1Ap7URHN/o6vw1Z7sQrLqRLsQfm1EI5FH+MYufXP3+JRXpXyYB
9T++ZEwByKgvMTGvKEadE2ln5hs2TzLSd4NDv4C9FYhV7s5q3SGiYBuzkrJ+G7A9
AUbsvOZ9+tynhchtKEqR
=Poeo
-----END PGP SIGNATURE-----




Bug archived. Request was from Debbugs Internal Request <owner@bugs.debian.org> to internal_control@bugs.debian.org. (Fri, 06 Sep 2013 07:37:03 GMT) Full text and rfc822 format available.

Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Sat Apr 19 07:16:41 2014; Machine Name: beach.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.