Debian Bug report logs - #577580
python-distutils fails installing multiple arch-all packages since 0.4.72

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Yaroslav Halchenko <debian@onerussian.com>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: use of cdbs_python_build_versions forbids both arch and indep Python packages from the same source

Date: Mon, 12 Apr 2010 16:37:11 -0400

Package: cdbs
Version: 0.4.77
Severity: normal


per Jonas request, filing a fresh bug  report...  two packages of mine FTBFS
using recent cdbs version while they were building just fine using previous
versions.

the reason actually is following code:

/usr/share/cdbs/1/class/python-distutils.mk-$(patsubst %,install/%,$(cdbs_python_indep_packages)) :: install/%: python-install-py
/usr/share/cdbs/1/class/python-distutils.mk-    cd $(DEB_SRCDIR) && python$(cdbs_python_nondefault_version) $(DEB_PYTHON_SETUP_CMD) install --root=$(cdbs_python_destdir) \
/usr/share/cdbs/1/class/python-distutils.mk:        --install-purelib=/usr/lib/python$(cdbs_python_build_versions)/site-packages/ $(DEB_PYTHON_INSTALL_ARGS_ALL)

and that one defined as
                                                                                                             
/usr/share/cdbs/1/class/python-vars.mk-# arch(+indep): all; indep: current if supported, else first supported
/usr/share/cdbs/1/class/python-vars.mk:cdbs_python_build_versions = $(or $(if $(cdbs_python_arch_packages),$(cdbs_python_supported_versions)),$(filter $(cdbs_python_current_version),$(cdbs_python_supported_versions)),$(cdbs_python_first_supported_version))

so now cdbs_python_build_versions (despite having "s" in suffix) is supposed to
have all versions if there is any arch package as well (-lib in my case), for
building indep packages (e.g. python-mvpa), which fails with what you
can observe in original bugreports:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=577312
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=577308

so, if before it was working correctly, now it doesn't (backward
incompatibility).

Yes, someone could call my packaging "dirty" since it got a bit elaborate in an
attempt to allow arch-dependent -lib packages using cdbs.  I am aware of the
fact that it is calling python setup.py install twice (for all supported
versions for arch -lib package, and once for main package for main version),
and such approach worked fine for quite a while.  But I do consider that
the cdbs either needs explicitely to allow for automagic handling of -lib arch
packages or not to rely on the colliding $(cdbs_python_build_versions).

P.S. keeping severity at normal to be raised at the maintainers discretion

Cheers,

-- System Information:
Debian Release: squeeze/sid
  APT prefers unstable
  APT policy: (901, 'unstable'), (900, 'testing'), (300, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.31-1-amd64 (SMP w/2 CPU cores)
Locale: LANG=en_US, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash

Versions of packages cdbs depends on:
ii  debhelper                     7.4.15     helper programs for debian/rules

Versions of packages cdbs recommends:
ii  autotools-dev                 20090611.1 Update infrastructure for config.{

Versions of packages cdbs suggests:
ii  devscripts                    2.10.61    scripts to make the life of a Debi
ii  doc-base                      0.9.4      utilities to manage online documen

-- no debconf information

Message #10 received at 577580@bugs.debian.org (full text, mbox, reply):

From: Jonas Smedegaard <dr@jones.dk>

To: 577580@bugs.debian.org

Cc: control@bugs.debian.org

Subject: Re: Bug#577580: use of cdbs_python_build_versions forbids both arch and indep Python packages from the same source

Date: Tue, 13 Apr 2010 01:26:24 +0200

[Message part 1 (text/plain, inline)]

severity 577580 wishlist
retitle 577580 Please have python-distutils support multiple libs packages
thanks


Hi Yaroslav,

On Mon, Apr 12, 2010 at 04:37:11PM -0400, Yaroslav Halchenko wrote:

>per Jonas request, filing a fresh bug report...  two packages of mine 
>FTBFS using recent cdbs version while they were building just fine 
>using previous versions.

Thanks.

Just the subject is enlightening: I now understand that this is an issue 
of arch-all and arch-any from same source.  Thanks for clarifying!



>Yes, someone could call my packaging "dirty" since it got a bit 
>elaborate in an attempt to allow arch-dependent -lib packages using 
>cdbs.  I am aware of the fact that it is calling python setup.py 
>install twice (for all supported versions for arch -lib package, and 
>once for main package for main version),

It is not that (in itself) that you call setup-py twice, but more 
generally that you seem to twist python-distutils.mk into doing 
something not yet supported: Handle multiple libraries.

DEB_PYTHON_MODULE_PACKAGES was introduced in cdbs 0.4.54, resolving only 
to a single package by default, and with the following comment in the 
NEWS entry:

> As its name indicates, the new variable may support multiple Python 
> packages in the future.


>and such approach worked fine for quite a while.  But I do consider 
>that the cdbs either needs explicitely to allow for automagic handling 
>of -lib arch packages or not to rely on the colliding 
>$(cdbs_python_build_versions).

I suspect that it only worked in the past because a single version was 
supported (due to your limiting to at least 2.5 and Debian only 
relatively recently introducing 2.6).


I'd be happy to work on improving cdbs to support packages like these, 
but don't expect this to be solved fast.

Until cdbs python-distutils.mk gains multilib support, I suggest to only 
use it for the arch-any lib (as that is the trickiest to do right) and 
avoid cdbs variables for the arch-all one.

Oh, and please only declare DEB_PYTHON_MODULE_PACKAGES (not 
DEB_PYTHON_MODULE_PACKAGE too, and even with a different content - that 
is certainly asking for trouble!).


Kind regards,

 - Jonas

-- 
* Jonas Smedegaard - idealist & Internet-arkitekt
* Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[signature.asc (application/pgp-signature, inline)]

Message #19 received at 577580@bugs.debian.org (full text, mbox, reply):

From: Jonas Smedegaard <dr@jones.dk>

To: 577580@bugs.debian.org

Cc: control@bugs.debian.org

Subject: Re: Bug#577580: use of cdbs_python_build_versions forbids both arch and indep Python packages from the same source

Date: Tue, 13 Apr 2010 02:16:27 +0200

[Message part 1 (text/plain, inline)]

severity 577580 important
retitle 577580 python-distutils fails installing multiple arch-all packages since 0.4.72
tags 577580 pending
thanks

On Tue, Apr 13, 2010 at 01:26:24AM +0200, Jonas Smedegaard wrote:
>On Mon, Apr 12, 2010 at 04:37:11PM -0400, Yaroslav Halchenko wrote:
>
>>per Jonas request, filing a fresh bug report...  two packages of 
>>mine FTBFS using recent cdbs version while they were building just 
>>fine using previous versions.

>I suspect that it only worked in the past because a single version 
>was supported (due to your limiting to at least 2.5 and Debian only 
>relatively recently introducing 2.6).

I now noticed that you only recently tightened dependency to Python 2.5+ 
and looked closer on the cdbs brakage.

Indeed this was handled properly in the past, and I broke it with a 
cleanup I did at 0.4.72.

Fix now applied to Git.


Regards,

 - Jonas

-- 
* Jonas Smedegaard - idealist & Internet-arkitekt
* Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[signature.asc (application/pgp-signature, inline)]

Message #30 received at 577580-close@bugs.debian.org (full text, mbox, reply):

From: Jonas Smedegaard <dr@jones.dk>

To: 577580-close@bugs.debian.org

Subject: Bug#577580: fixed in cdbs 0.4.81

Date: Tue, 13 Apr 2010 00:47:17 +0000

Source: cdbs
Source-Version: 0.4.81

We believe that the bug you reported is fixed in the latest version of
cdbs, which is due to be installed in the Debian FTP archive:

cdbs_0.4.81.dsc
  to main/c/cdbs/cdbs_0.4.81.dsc
cdbs_0.4.81.tar.gz
  to main/c/cdbs/cdbs_0.4.81.tar.gz
cdbs_0.4.81_all.deb
  to main/c/cdbs/cdbs_0.4.81_all.deb



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 577580@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Jonas Smedegaard <dr@jones.dk> (supplier of updated cdbs package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160

Format: 1.8
Date: Tue, 13 Apr 2010 02:16:58 +0200
Source: cdbs
Binary: cdbs
Architecture: source all
Version: 0.4.81
Distribution: unstable
Urgency: low
Maintainer: CDBS Hackers <build-common-hackers@lists.alioth.debian.org>
Changed-By: Jonas Smedegaard <dr@jones.dk>
Description: 
 cdbs       - common build system for Debian packages
Closes: 577580
Changes: 
 cdbs (0.4.81) unstable; urgency=low
 .
   * Fix install arch-all packages part of arch-all+arch-any multilib
     packaging in python-distutils.mk (broken since 0.4.72). Closes:
     bug#577580, thanks to Yaroslav Halchenko.
Checksums-Sha1: 
 1f4032c56bea41e8ac456786a630eb49ea01c37a 1285 cdbs_0.4.81.dsc
 871347ca8601db9b38bbbf76035a96f3e101f658 251459 cdbs_0.4.81.tar.gz
 4a39b43271d4ed0d099c3c1a55e34b1b2b2f54ce 1240760 cdbs_0.4.81_all.deb
Checksums-Sha256: 
 9126955a9afc42aa0048079539e6a77a973eb85f9d9d0c78724f34fb7c72491b 1285 cdbs_0.4.81.dsc
 88745d454bf489eb4712edb7cd36758d695d13964f076326005fa51bfb0a6178 251459 cdbs_0.4.81.tar.gz
 fb83ef4d4e9696e52356300087b9a2e5d3e7b83d51dc83137b28bec3f3921ac8 1240760 cdbs_0.4.81_all.deb
Files: 
 98af6858ffef0da80715290410495aa6 1285 devel optional cdbs_0.4.81.dsc
 33f7c17dd148c72eafc7c7bda57bb14b 251459 devel optional cdbs_0.4.81.tar.gz
 5ed0505521a91591c49e666ad8e45bf2 1240760 devel optional cdbs_0.4.81_all.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEAREDAAYFAkvDuv4ACgkQn7DbMsAkQLgXtgCggTp3HgfxEn1B9ag+9Qc7tZWB
MG0AmwSrtExXeflVi087h4IGg/Itg7A1
=xxgJ
-----END PGP SIGNATURE-----

Message #35 received at 577580@bugs.debian.org (full text, mbox, reply):

From: Yaroslav Halchenko <debian@onerussian.com>

To: Jonas Smedegaard <dr@jones.dk>

Cc: 577580@bugs.debian.org

Subject: Re: Bug#577580 closed by Jonas Smedegaard <dr@jones.dk> (Bug#577580: fixed in cdbs 0.4.81)

Date: Wed, 5 May 2010 12:45:43 -0400

[Message part 1 (text/plain, inline)]

Hi Jonas,

Since I wasn't in addressee of your replies I did not receive any of them
unfortunately.  Just now found your replies on the web -- thanks.  But
could you address one of them for me?

> Oh, and please only declare DEB_PYTHON_MODULE_PACKAGES (not 
> DEB_PYTHON_MODULE_PACKAGE too, and even with a different content - that 
> is certainly asking for trouble!).

well -- the reason for all this is simply to make it work with cdbs
present in lenny, for which we would have easy ways to backport without
any patching the packaging (although I am yet to check if I can easily
backport nipy in particular but this approach worked fine for other
packages ;) forgotten what stopped me before)

why should it be seeking for a trouble if nothing in cdbs seems to rely
on that variable as far as I can see?

what would be a way to conditionally define one or another (i.e. is
there a CDBS_VERSION variable exposed may be suitable for comparison?)

Thanks in advance
Cheers
-- 
                                  .-.
=------------------------------   /v\  ----------------------------=
Keep in touch                    // \\     (yoh@|www.)onerussian.com
Yaroslav Halchenko              /(   )\               ICQ#: 60653192
                   Linux User    ^^-^^    [175555]

[signature.asc (application/pgp-signature, inline)]

Message #40 received at 577580@bugs.debian.org (full text, mbox, reply):

From: Yaroslav Halchenko <debian@onerussian.com>

To: Jonas Smedegaard <dr@jones.dk>

Cc: 577580@bugs.debian.org

Subject: Re: Bug#577580 closed by Jonas Smedegaard <dr@jones.dk> (Bug#577580: fixed in cdbs 0.4.81)

Date: Wed, 5 May 2010 15:17:49 -0400

[Message part 1 (text/plain, inline)]

ho ho -- now I am getting better comprehension of what you were
delivering to me in:

> I suspect that it only worked in the past because a single version was 
> supported (due to your limiting to at least 2.5 and Debian only 
> relatively recently introducing 2.6).

I was building fresh nipy and found that only 2.5 libraries were generated
within -lib* packages.  Indeed, in nipy we didn't have previously any libraries
for 2.6 BUT in mvpa we got everything just fine with previous versions of cdbs
and similar setup:

$> acpolicy python-mvpa-lib 
python-mvpa-lib:
  Installed: 0.4.4-1
  Candidate: 0.4.4-1
  Version table:
 *** 0.4.4-1 0
        900 http://debian.lcs.mit.edu squeeze/main Packages
        901 http://debian.lcs.mit.edu sid/main Packages
        100 /var/lib/dpkg/status
     0.4.4-1~squeeze.nd1 0
        500 http://neuro.debian.net squeeze/main Packages
     0.4.3-1~sid.nd1 0
        500 http://neuro.debian.net sid/main Packages

$> dpkg -L python-mvpa-lib | grep smlrc.so
/usr/lib/pyshared/python2.5/mvpa/clfs/libsmlrc/smlrc.so
/usr/lib/pyshared/python2.6/mvpa/clfs/libsmlrc/smlrc.so


But with current CDBS, building nipy I get only 2.5 ...

As to me it is a really serious concern -- there is a plentiful of packages
(and ever growing number with cython becoming more popular) where having both
"any" (-lib) and "all" Architectures is the way to go to don't pollute
architecture-dependent binary packages with arch-independent .py files.

I've tried to come up with a blunt workaround having following line after
include /usr/share/cdbs/1/class/python-distutils.mk

# Ensure building for all Python versions despite having both arch and
# indep binary packages at ones
cdbs_python_build_versions := $(cdbs_python_supported_versions)

but that had no effect (probably decision is done somewhere prior to that
point)...

in any case, for me all this sounds like a regression... not sure what I could
do... if you think it would be too long to fix it up, I guess I would need to
consider alternatives.

Thanks in advance for reply.

Cheers,
Yarik

On Wed, 05 May 2010, Yaroslav Halchenko wrote:

> Hi Jonas,

> Since I wasn't in addressee of your replies I did not receive any of them
> unfortunately.  Just now found your replies on the web -- thanks.  But
> could you address one of them for me?

> > Oh, and please only declare DEB_PYTHON_MODULE_PACKAGES (not 
> > DEB_PYTHON_MODULE_PACKAGE too, and even with a different content - that 
> > is certainly asking for trouble!).

> well -- the reason for all this is simply to make it work with cdbs
> present in lenny, for which we would have easy ways to backport without
> any patching the packaging (although I am yet to check if I can easily
> backport nipy in particular but this approach worked fine for other
> packages ;) forgotten what stopped me before)

> why should it be seeking for a trouble if nothing in cdbs seems to rely
> on that variable as far as I can see?

> what would be a way to conditionally define one or another (i.e. is
> there a CDBS_VERSION variable exposed may be suitable for comparison?)

> Thanks in advance
> Cheers
-- 
                                  .-.
=------------------------------   /v\  ----------------------------=
Keep in touch                    // \\     (yoh@|www.)onerussian.com
Yaroslav Halchenko              /(   )\               ICQ#: 60653192
                   Linux User    ^^-^^    [175555]

[signature.asc (application/pgp-signature, inline)]

Send a report that this bug log contains spam.