Debian Bug report logs - #538978
please add support for multiple python version to more build classes

version graph

Package: debhelper; Maintainer for debhelper is Debhelper Maintainers <debhelper-devel@lists.alioth.debian.org>; Source for debhelper is src:debhelper.

Reported by: Michal Čihař <nijel@debian.org>

Date: Tue, 28 Jul 2009 10:15:01 UTC

Severity: wishlist

Merged with 596807

Found in versions debhelper/7.3.8, debhelper/8.0.0

Reply or subscribe to this bug.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, Joey Hess <joeyh@debian.org>:
Bug#538978; Package debhelper. (Tue, 28 Jul 2009 10:15:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Michal Čihař <nijel@debian.org>:
New Bug report received and forwarded. Copy sent to Joey Hess <joeyh@debian.org>. (Tue, 28 Jul 2009 10:15:04 GMT) Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: Michal Čihař <nijel@debian.org>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: please add support for multiple python version to more build classes
Date: Tue, 28 Jul 2009 12:12:48 +0200
Package: debhelper
Version: 7.3.8
Severity: wishlist

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi

currently only distutils build class does support building for multiple
python versions. However python modules are also shipped in various
other package which use different build systems (rpm with autoconf, 
Gammu with CMake or pyexiv2 with SCons, when looking at mine packages).

It would be great if this functionality could be generic, so that I do
not need to duplicate all the code to build using all python version in
every package.

I know this can be quite tricky, because each package might take
different parameters to achieve this, so this is something what needs to
be configurable by parameter, something like:

    dh_auto_configure --build-system=cmake --multi-python=-DBUILD_PYTHON=

What would expand to invoking cmake several times passing it
- -DBUILD_PYTHON=/usr/bin/pythonX.Y for all Python versions it should use.

If you want to see examples look at above mentioned packages or just
catch me at DebConf.

- -- 
	Michal Čihař | http://cihar.com | http://blog.cihar.com

- -- System Information:
Debian Release: squeeze/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (1, 'experimental')
Architecture: i386 (i686)

Kernel: Linux 2.6.30-1-686 (SMP w/1 CPU core)
Locale: LANG=cs_CZ.UTF-8, LC_CTYPE=cs_CZ.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages debhelper depends on:
ii  binutils              2.19.51.20090723-1 The GNU assembler, linker and bina
ii  dpkg-dev              1.15.3.1           Debian package development tools
ii  file                  5.03-1             Determines file type using "magic"
ii  html2text             1.3.2a-14          advanced HTML to text converter
ii  man-db                2.5.5-3            on-line manual pager
ii  perl                  5.10.0-24          Larry Wall's Practical Extraction 
ii  perl-base             5.10.0-24          minimal Perl system
ii  po-debconf            1.0.16             tool for managing templates file t

debhelper recommends no packages.

Versions of packages debhelper suggests:
pn  dh-make                       <none>     (no description available)

- -- no debconf information

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkpuzyAACgkQ3DVS6DbnVgQb8wCg0GtJaeVF9Xlo8R2crBKDbuXr
WwMAnRTGhlU9Y6dUmaemB1d2sTU8NVkT
=EsHj
-----END PGP SIGNATURE-----




Information forwarded to debian-bugs-dist@lists.debian.org:
Bug#538978; Package debhelper. (Sat, 01 Aug 2009 16:45:31 GMT) Full text and rfc822 format available.

Acknowledgement sent to Joey Hess <joeyh@debian.org>:
Extra info received and forwarded to list. (Sat, 01 Aug 2009 16:45:31 GMT) Full text and rfc822 format available.

Message #10 received at submit@bugs.debian.org (full text, mbox):

From: Joey Hess <joeyh@debian.org>
To: Michal Čihař <nijel@debian.org>, 538978@bugs.debian.org
Cc: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: Re: Bug#538978: please add support for multiple python version to more build classes
Date: Fri, 31 Jul 2009 11:23:42 -0400
[Message part 1 (text/plain, inline)]
Michal Čihař wrote:
> currently only distutils build class does support building for multiple
> python versions. However python modules are also shipped in various
> other package which use different build systems (rpm with autoconf, 
> Gammu with CMake or pyexiv2 with SCons, when looking at mine packages).
> 
> It would be great if this functionality could be generic, so that I do
> not need to duplicate all the code to build using all python version in
> every package.
> 
> I know this can be quite tricky, because each package might take
> different parameters to achieve this, so this is something what needs to
> be configurable by parameter, something like:
> 
>     dh_auto_configure --build-system=cmake --multi-python=-DBUILD_PYTHON=
> 
> What would expand to invoking cmake several times passing it
> - -DBUILD_PYTHON=/usr/bin/pythonX.Y for all Python versions it should use.

This seems far too much an edge case and complicated (and unlikely to be
usable for everything anyway) to do.
 
-- 
see shy jo
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org:
Bug#538978; Package debhelper. (Sat, 01 Aug 2009 16:45:34 GMT) Full text and rfc822 format available.

Acknowledgement sent to Joey Hess <joeyh@debian.org>:
Extra info received and forwarded to list. (Sat, 01 Aug 2009 16:45:34 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Joey Hess <joeyh@debian.org>:
Bug#538978; Package debhelper. (Wed, 05 Aug 2009 08:45:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to Michal Čihař <nijel@debian.org>:
Extra info received and forwarded to list. Copy sent to Joey Hess <joeyh@debian.org>. (Wed, 05 Aug 2009 08:45:05 GMT) Full text and rfc822 format available.

Message #20 received at 538978@bugs.debian.org (full text, mbox):

From: Michal Čihař <nijel@debian.org>
To: Joey Hess <joeyh@debian.org>
Cc: 538978@bugs.debian.org
Subject: Re: Bug#538978: please add support for multiple python version to more build classes
Date: Wed, 5 Aug 2009 10:41:10 +0200
[Message part 1 (text/plain, inline)]
Hi

Dne Fri, 31 Jul 2009 11:23:42 -0400
Joey Hess <joeyh@debian.org> napsal(a):

> This seems far too much an edge case and complicated (and unlikely to be
> usable for everything anyway) to do.

I know it would be hard to find some solution which would help in most
cases. But I hate to put the logic to compile for all python versions
to all debian/rules. It would be great if this logic can be somehow
separated and reused independently of build system.

-- 
	Michal Čihař | http://cihar.com | http://blog.cihar.com
[signature.asc (application/pgp-signature, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org:
Bug#538978; Package debhelper. (Wed, 05 Aug 2009 14:00:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Joey Hess <joeyh@debian.org>:
Extra info received and forwarded to list. (Wed, 05 Aug 2009 14:00:07 GMT) Full text and rfc822 format available.

Message #25 received at 538978@bugs.debian.org (full text, mbox):

From: Joey Hess <joeyh@debian.org>
To: Michal Čihař <nijel@debian.org>
Cc: 538978@bugs.debian.org
Subject: Re: Bug#538978: please add support for multiple python version to more build classes
Date: Wed, 5 Aug 2009 09:55:50 -0400
[Message part 1 (text/plain, inline)]
Michal Čihař wrote:
> I know it would be hard to find some solution which would help in most
> cases. But I hate to put the logic to compile for all python versions
> to all debian/rules. It would be great if this logic can be somehow
> separated and reused independently of build system.

That logic follows below. 

If this were say, rewritten in python :) and factored out into a standalone
program with a name like python-multibuild, and included in some python
development package, then debhelper and rules files could just call it,
and pass it the setup.py to run and the action to do.

I'd love to remove this code from debhelper, if we can find someplace
else to put it.

sub dbg_build_needed {
        my $this=shift;
        my $act=shift;

        # Return a list of python-dbg package which are listed
        # in the build-dependencies. This is kinda ugly, but building
        # dbg extensions without checking if they're supposed to be
        # built may result in various FTBFS if the package is not
        # built in a clean chroot.

        my @dbg;
        open (CONTROL, 'debian/control') ||
                error("cannot read debian/control: $!\n");
        foreach my $builddeps (join('', <CONTROL>) =~
                        /^Build-Depends[^:]*:.*\n(?:^[^\w\n].*\n)*/gmi) {
                while ($builddeps =~ /(python[^, ]*-dbg)/g) {
                        push @dbg, $1;
                }
        }

        close CONTROL;
        return @dbg;

}

sub setup_py {
        my $this=shift;
        my $act=shift;

        # We need to to run setup.py with the default python first
        # as distutils/setuptools modifies the shebang lines of scripts.
        # This ensures that #!/usr/bin/python is used and not pythonX.Y
        # Take into account that the default Python must not be in
        # the requested Python versions.
        # Then, run setup.py with each available python, to build
        # extensions for each.
        
        my $python_default = `pyversions -d`;
        $python_default =~ s/^\s+//;
        $python_default =~ s/\s+$//;
        my @python_requested = split ' ', `pyversions -r 2>/dev/null`;
        if (grep /^\Q$python_default\E/, @python_requested) {
                @python_requested = ("python", grep(!/^\Q$python_default\E/,
                                        @python_requested));
        }
                
        my @python_dbg;                 
        my @dbg_build_needed = $this->dbg_build_needed();
        foreach my $python (map { $_."-dbg" } @python_requested) {
                if (grep /^(python-all-dbg|\Q$python\E)/, @dbg_build_needed) {
                        push @python_dbg, $python;
                }
                elsif (($python eq "python-dbg")
                       and (grep /^\Q$python_default\E/, @dbg_build_needed)) {
                        push @python_dbg, $python_default."-dbg";
                }
        }              

        foreach my $python (@python_requested, @python_dbg) {
                if (-x "/usr/bin/".$python) {
			# effectively this runs system($python, "setup.py", $act, @_);
                        $this->doit_in_sourcedir($python, "setup.py", $act, @_);
                }
        }
}


-- 
see shy jo
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Joey Hess <joeyh@debian.org>:
Bug#538978; Package debhelper. (Tue, 18 Aug 2009 15:24:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Michal Čihař <nijel@debian.org>:
Extra info received and forwarded to list. Copy sent to Joey Hess <joeyh@debian.org>. (Tue, 18 Aug 2009 15:24:03 GMT) Full text and rfc822 format available.

Message #30 received at 538978@bugs.debian.org (full text, mbox):

From: Michal Čihař <nijel@debian.org>
To: Joey Hess <joeyh@debian.org>
Cc: 538978@bugs.debian.org
Subject: Re: Bug#538978: please add support for multiple python version to more build classes
Date: Tue, 18 Aug 2009 17:21:15 +0200
[Message part 1 (text/plain, inline)]
Hi

Dne Wed, 5 Aug 2009 09:55:50 -0400
Joey Hess <joeyh@debian.org> napsal(a):

> Michal Čihař wrote:
> > I know it would be hard to find some solution which would help in most
> > cases. But I hate to put the logic to compile for all python versions
> > to all debian/rules. It would be great if this logic can be somehow
> > separated and reused independently of build system.
> 
> That logic follows below. 
> 
> If this were say, rewritten in python :) and factored out into a standalone
> program with a name like python-multibuild, and included in some python
> development package, then debhelper and rules files could just call it,
> and pass it the setup.py to run and the action to do.

Hmm, that sounds like a way to go if we really want to build binaries
for more Python versions. I will try to look more deeply into this
topic, putting it to my todo list to investigate later.

-- 
	Michal Čihař | http://cihar.com | http://blog.cihar.com
[signature.asc (application/pgp-signature, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, Joey Hess <joeyh@debian.org>:
Bug#538978; Package debhelper. (Thu, 24 Feb 2011 08:57:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Piotr Ozarowski <ozarow@gmail.com>:
Extra info received and forwarded to list. Copy sent to Joey Hess <joeyh@debian.org>. (Thu, 24 Feb 2011 08:57:03 GMT) Full text and rfc822 format available.

Message #35 received at 538978@bugs.debian.org (full text, mbox):

From: Piotr Ozarowski <ozarow@gmail.com>
To: 538978@bugs.debian.org
Subject: Re: Bug#538978: please add support for multiple python version to more build classes
Date: Thu, 24 Feb 2011 09:55:07 +0100
[Joey Hess, 2011-02-24]
> If this were say, rewritten in python :) and factored out into a standalone
> program with a name like python-multibuild, and included in some python
> development package, then debhelper and rules files could just call it,
> and pass it the setup.py to run and the action to do.

I'm interested in writing such tool and shipping it in python package,
could you elaborate a little bit more about API? What arguments do you
want to pass? Only something like this:

 $ python-multibuild build
 $ python-multibuild install
 $ python-multibuild test
 $ python-multibuild clean

or will you also pass more options, like build dir (the one with
setup.py file) for example?
-- 
Piotr Ożarowski                         Debian GNU/Linux Developer
www.ozarowski.pl          www.griffith.cc           www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645




Information forwarded to debian-bugs-dist@lists.debian.org:
Bug#538978; Package debhelper. (Thu, 24 Feb 2011 17:03:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to Joey Hess <joeyh@debian.org>:
Extra info received and forwarded to list. (Thu, 24 Feb 2011 17:03:05 GMT) Full text and rfc822 format available.

Message #40 received at 538978@bugs.debian.org (full text, mbox):

From: Joey Hess <joeyh@debian.org>
To: Piotr Ozarowski <ozarow@gmail.com>, 538978@bugs.debian.org
Subject: Re: Bug#538978: please add support for multiple python version to more build classes
Date: Thu, 24 Feb 2011 13:01:17 -0400
[Message part 1 (text/plain, inline)]
Piotr Ozarowski wrote:
> [Joey Hess, 2011-02-24]
> > If this were say, rewritten in python :) and factored out into a standalone
> > program with a name like python-multibuild, and included in some python
> > development package, then debhelper and rules files could just call it,
> > and pass it the setup.py to run and the action to do.
> 
> I'm interested in writing such tool and shipping it in python package,
> could you elaborate a little bit more about API? What arguments do you
> want to pass? Only something like this:

Anything that makes sense if the tool is used manually in a rules file
should be usable by debhelper, I guess.

-- 
see shy jo
[signature.asc (application/pgp-signature, inline)]

Merged 538978 596807. Request was from Joey Hess <joeyh@debian.org> to control@bugs.debian.org. (Thu, 08 Dec 2011 22:12:08 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Joey Hess <joeyh@debian.org>:
Bug#538978; Package debhelper. (Sat, 19 Jan 2013 17:15:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Piotr Ożarowski <piotr@debian.org>:
Extra info received and forwarded to list. Copy sent to Joey Hess <joeyh@debian.org>. (Sat, 19 Jan 2013 17:15:03 GMT) Full text and rfc822 format available.

Message #47 received at 538978@bugs.debian.org (full text, mbox):

From: Piotr Ożarowski <piotr@debian.org>
To: debian-python@lists.debian.org
Subject: pybuild - one to rule^W build them all
Date: Sat, 19 Jan 2013 18:14:22 +0100
[Message part 1 (text/plain, inline)]
[debhelper's #538978 and #597105 bugs BCCed]

Hi,

FYI: python3 package in experimental (3.3.0-2) contains pybuild tool
with dh command sequencer integration.

You can build modules/extensions for Python 2.X *and* Python 3.X with
this debian/rules file:

 #!/usr/bin/make -f
 %:
 	dh $@ --with python2,python3 --buildsystem=pybuild

pybuild.pm will check Build-Depends* and build Python (python,
python-dev, python-dbg, python-all, python-all-dev, python-all-dbg),
Python 3 (python3, python3-dev, python3-dbg, python3-all,
python3-all-dev, python3-all-dbg) and PyPy (pypy) modules/extensions.

It can be configured via command line or CDBS-like - by exporting
PYBUILD_OPTION / PYBUILD_OPTION_interpreter / 
PYBUILD_OPTION_interpreterversion env. variables in debian/rules, f.e.

* if you don't want to use debian/*.install files,
  you can set DESTDIR for each interpreter (note that you can use "python2" for
  all python2.X interpreters and "python3" for all python3.X ones):

 export PYBUILD_DESTDIR_python2=debian/python-foo/
 export PYBUILD_DESTDIR_python2-dbg=debian/python-foo-dbg/
 export PYBUILD_DESTDIR_python3=debian/python3-foo/
 export PYBUILD_DESTDIR_python3-dbg=debian/python3-foo-dbg/
 export PYBUILD_DESTDIR_python3.1-dbg=debian/spam-pkg-with-python3.1-only/
 export PYBUILD_DESTDIR_pypy=debian/pypy-foo/

* if you want to add some custom arguments to Python 3's install command:

 export PYBUILD_INSTALL_ARGS_python3=--install-scripts=/usr/share/foo/

* you can disable (almost) any action via --disable / DISABLE env. var:

 export PYBUILD_DISABLE=configure
 export PYBUILD_DISABLE_python3.1=test

* if you want to disable Build-Depends detection:

 export PYBUILD_INTERPRETERS=python{version} python{version}-dbg
 export PYBUILD_VERSIONS=2.7 3.3

* if auto detection didn't detect the right build system:

 export PYBUILD_SYSTEM=distutils

* if you want to add a custom command before or after each iteration:

 export PYBUILD_BEFORE_BUILD=echo {version} >> '{dir}/enabled'
 export PYBUILD_AFTER_INSTALL=rm -rf '{destdir}/{install_dir}/foo/tests'


Right now it supports distutils (i.e. setup.py) and custom build systems only
(I tried to add cmake one that uses dh_auto_* tools, but failed to teach cmake
which interpreter should be used).

Anyone wants to contribute more plugins? Let me know if you want some guidance
or if I can make writing plugins easier.

If your build system is not yet supported, you can use custom build system:

 export PYBUILD_SYSTEM=custom
 export PYBUILD_CLEAN_ARGS=rm -rf {build_dir}/python{version}/
 export PYBUILD_CONFIGURE_ARGS=./configure --python={version}
 export PYBUILD_BUILD_ARGS=make build --dir={build_dir} --python={version}
 export PYBUILD_INSTALL_ARGS=make --destdir={destdir} --python={version} --install-dir={install_dir}
 export PYBUILD_TEST_ARGS_python2=cd {build_dir}; nosetests
 export PYBUILD_TEST_ARGS_python3=cd {build_dir}; python{version} -m discover


PS pybuild manpage is available also here: http://deb.li/pybuild
-- 
Piotr Ożarowski                         Debian GNU/Linux Developer
www.ozarowski.pl          www.griffith.cc           www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645
[signature.asc (application/pgp-signature, inline)]

Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Wed Apr 16 11:17:15 2014; Machine Name: beach.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.