Debian Bug report logs - #604397
debian-policy: build-arch and build-indep targets are required

version graph

Package: debian-policy; Maintainer for debian-policy is Debian Policy List <debian-policy@lists.debian.org>; Source for debian-policy is src:debian-policy.

Reported by: Roger Leigh <rleigh@debian.org>

Date: Sun, 21 Nov 2010 21:42:02 UTC

Severity: wishlist

Merged with 345619, 374029, 619284

Found in versions debian-policy/3.6.2.1, debian-policy/3.9.1.0

Fixed in version debian-policy/3.9.4.0

Done: Russ Allbery <rra@debian.org>

Bug is archived. No further changes may be made.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Sun, 21 Nov 2010 21:42:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Roger Leigh <rleigh@debian.org>:
New Bug report received and forwarded. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Sun, 21 Nov 2010 21:42:05 GMT) Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: Roger Leigh <rleigh@debian.org>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: debian-policy: build-arch and build-indep targets are required
Date: Sun, 21 Nov 2010 21:38:31 +0000
[Message part 1 (text/plain, inline)]
Package: debian-policy
Version: 3.9.1.0
Severity: normal
Tags: patch

Hi,

As a goal for wheezy, I'd like the Build-Depends and
Build-Depends-Indep fields to be fully functional and usable for
autobuilding.  Currently, Build-Depends-Indep isn't useful, and
only Build-Depends is used in practice on our autobuilder
infrastructure (despite it being perfectly capable of handling
and using B-D-I internally).

The reason for this is that we do our building using dpkg-buildpackage,
and while binary-arch and binary-indep are required targets used by
dpkg-buildpackage, the corresponding build-arch and build-indep targets
are not required and are not used by dpkg-buildpackage at present.

There's some history to this in #374029.  Most of the information
about sbuild/buildds in there is outdated and incorrect.  The only
sticking point to using Build-Depends-Indep and autobuilding
arch-all packages is the lack of support for build-arch and build-all
in dpkg-buildpackage.

Simply enabling this in dpkg-buildpackage would cause widespread
breakage.  build-arch and build-indep have been suggested in Policy
for some time, and they are used by a low percentage of the archive
at present (<400 packages), while 50% of the archive uses dh or
cdbs rules which would allow trivial support in all those packages
once the tools are updated.

I'd like to propose that build-arch and build-indep be changed in
Policy from "may be provided" to "must be provided" in preparation
for enabling their use.  We've wanted to fix the root problem for
at least half a decade, and I'd like to get it done for wheezy.

This should probably also be accompanied by a new lintian check which
can warn if these rules are missing.

Note, this patch obviates the need for the patch in #601839.


Regards,
Roger

-- System Information:
Debian Release: 5.0.6
  APT prefers stable
  APT policy: (500, 'stable')
Architecture: powerpc (ppc)

Kernel: Linux 2.6.26-2-powerpc
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash

debian-policy depends on no packages.

debian-policy recommends no packages.

Versions of packages debian-policy suggests:
ii  doc-base                      0.8.20     utilities to manage online documen

-- no debconf information
[policy-build-rules.patch (text/x-diff, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Sun, 21 Nov 2010 21:57:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to Roger Leigh <rleigh@codelibre.net>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Sun, 21 Nov 2010 21:57:05 GMT) Full text and rfc822 format available.

Message #10 received at 604397@bugs.debian.org (full text, mbox):

From: Roger Leigh <rleigh@codelibre.net>
To: 604397@bugs.debian.org
Subject: Re: Bug#604397: debian-policy: build-arch and build-indep targets are required
Date: Sun, 21 Nov 2010 21:54:33 +0000
[Message part 1 (text/plain, inline)]
On Sun, Nov 21, 2010 at 09:38:31PM +0000, Roger Leigh wrote:
> There's some history to this in #374029.  Most of the information
> about sbuild/buildds in there is outdated and incorrect.  The only
> sticking point to using Build-Depends-Indep and autobuilding
> arch-all packages is the lack of support for build-arch and build-all
> in dpkg-buildpackage.

s/build-all/build-indep/


-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
   `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 22 Nov 2010 01:33:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Jakub Wilk <jwilk@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 22 Nov 2010 01:33:03 GMT) Full text and rfc822 format available.

Message #15 received at 604397@bugs.debian.org (full text, mbox):

From: Jakub Wilk <jwilk@debian.org>
To: Roger Leigh <rleigh@debian.org>, 604397@bugs.debian.org
Subject: Re: Bug#604397: debian-policy: build-arch and build-indep targets are required
Date: Mon, 22 Nov 2010 00:45:28 +0100
* Roger Leigh <rleigh@debian.org>, 2010-11-21, 21:38:
>Currently, Build-Depends-Indep isn't useful,

I've heard this many times, but this is not true. B-D-I is useful and 
many packages use it successfully.

>I'd like to propose that build-arch and build-indep be changed in
>Policy from "may be provided" to "must be provided" in preparation
>for enabling their use.  We've wanted to fix the root problem for
>at least half a decade, and I'd like to get it done for wheezy.

The only source packages that could possibly benefit from
build/build-arch/build-indep separation are those which build at least 
one arch:all package and at least one non-arch:all package. How about 
making the additional target obligatory only for them? If I calculated 
correctly that would reduce number of source packages affected by this 
transition from ~15K to ~2K.

>This should probably also be accompanied by a new lintian check which
>can warn if these rules are missing.

I think we should be changing policy only after the lintian check is 
implement and the majority of packages are fixed.

-- 
Jakub Wilk




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 22 Nov 2010 23:21:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Roger Leigh <rleigh@codelibre.net>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 22 Nov 2010 23:21:03 GMT) Full text and rfc822 format available.

Message #20 received at 604397@bugs.debian.org (full text, mbox):

From: Roger Leigh <rleigh@codelibre.net>
To: Jakub Wilk <jwilk@debian.org>, 604397@bugs.debian.org
Cc: Roger Leigh <rleigh@debian.org>
Subject: Re: Bug#604397: debian-policy: build-arch and build-indep targets are required
Date: Mon, 22 Nov 2010 23:18:43 +0000
[Message part 1 (text/plain, inline)]
On Mon, Nov 22, 2010 at 12:45:28AM +0100, Jakub Wilk wrote:
> * Roger Leigh <rleigh@debian.org>, 2010-11-21, 21:38:
>> I'd like to propose that build-arch and build-indep be changed in
>> Policy from "may be provided" to "must be provided" in preparation
>> for enabling their use.  We've wanted to fix the root problem for
>> at least half a decade, and I'd like to get it done for wheezy.
>
> The only source packages that could possibly benefit from
> build/build-arch/build-indep separation are those which build at least  
> one arch:all package and at least one non-arch:all package. How about  
> making the additional target obligatory only for them? If I calculated  
> correctly that would reduce number of source packages affected by this  
> transition from ~15K to ~2K.

That would make sense.  However, it should be noted that current
Policy requires binary-arch and binary-indep irrespective of if
any arch or indep packages are built, and it would be consistent
to require the same of the corresponding build targets.

>> This should probably also be accompanied by a new lintian check which
>> can warn if these rules are missing.
>
> I think we should be changing policy only after the lintian check is  
> implement and the majority of packages are fixed.

OK.

Just for the record, I've implemented support in debhelper's dh
command in #604563.  Once applied, this will automatically add support
to the huge chunk of the archive using "tiny" rules files.  cdbs will
be next on my list.  These two combined are, by a crude estimate,
grepping the lintian lab, approximately 50% of the total.


Regards,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
   `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Wed, 02 Mar 2011 03:03:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to Jonathan Nieder <jrnieder@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Wed, 02 Mar 2011 03:03:05 GMT) Full text and rfc822 format available.

Message #25 received at 604397@bugs.debian.org (full text, mbox):

From: Jonathan Nieder <jrnieder@gmail.com>
To: Roger Leigh <rleigh@codelibre.net>
Cc: Jakub Wilk <jwilk@debian.org>, 604397@bugs.debian.org, Roger Leigh <rleigh@debian.org>, 229357@bugs.debian.org
Subject: Re: debian-policy: require build-arch and build-indep targets
Date: Tue, 1 Mar 2011 21:01:31 -0600
user debian-policy@packages.debian.org
usertags 604397 + normative discussion
quit

(please consider dropping policy Bug#604397 or dpkg-buildpackage Bug#229357
from replies)
Hi,

Roger Leigh wrote:
[out of order for convenience]

> Just for the record, I've implemented support in debhelper's dh
> command in #604563.  Once applied, this will automatically add support
> to the huge chunk of the archive using "tiny" rules files.  cdbs will
> be next on my list.

debhelper 8.1.0 has such support now.  Thanks!

>> * Roger Leigh <rleigh@debian.org>, 2010-11-21, 21:38:

>>> I'd like to propose that build-arch and build-indep be changed in
>>> Policy from "may be provided" to "must be provided" in preparation
>>> for enabling their use.

Personally, I'm all for it; ideally it would happen in the following
order:

 1. Providing build-arch and build-indep becomes a best practice.
    lintian gains a check.  devref encourages the practice.

 2. Becomes a policy "should".

 3. Becomes a policy "must".

That sounds slow, no?  Yes, that's the point.  I'd like to propose
that we not make most of the packages in the archive instantly RC
buggy, today.

>>> We've wanted to fix the root problem for
>>> at least half a decade, and I'd like to get it done for wheezy.

I just noticed this gem in policy §4.9:

	If one or both of the targets build-arch and build-indep are
	not provided, then invoking debian/rules with one of the
	not-provided targets as arguments should produce a exit status
	code of 2.  Usually this is provided automatically by make if
	the target is missing.

So it seems to me that "dpkg-buildpackage -B" ought to be taught to
run the equivalent of

	debian/rules build-arch
	if test "$?" = 2
	then
		debian/rules build
	fi

.  In http://bugs.debian.org/72335 the only argument against that I
see is

	(and notwithstanding that we're going to require both or neither), it
	should say that "debian/rules -q with one of the not-provided targets
	...", because the programs which will want to test this are likely to
	do something cheap like:

	debian/rules -q build-arch
	if [ $? -eq 2 ]; then
	    debian/rules build
	else
	    debian/rules build-arch
	fi

	To try a full build only to receive an exit status of 2 would not say
	whether the build failed or the target was not found.

That doesn't seem so compelling to me; in the failure case, the
autobuilder would just try to pick up where it left off and fail
again, which doesn't sound so expensive.  What am I missing?

Thanks for moving this forward.

Regards,
Jonathan




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Wed, 02 Mar 2011 09:03:07 GMT) Full text and rfc822 format available.

Acknowledgement sent to "Bernhard R. Link" <brlink@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Wed, 02 Mar 2011 09:03:07 GMT) Full text and rfc822 format available.

Message #30 received at 604397@bugs.debian.org (full text, mbox):

From: "Bernhard R. Link" <brlink@debian.org>
To: 604397@bugs.debian.org
Subject: Re: Bug#604397: debian-policy: require build-arch and build-indep targets
Date: Wed, 2 Mar 2011 09:59:40 +0100
* Jonathan Nieder <jrnieder@gmail.com> [110302 04:03]:
> I just noticed this gem in policy §4.9:
>
> 	If one or both of the targets build-arch and build-indep are
> 	not provided, then invoking debian/rules with one of the
> 	not-provided targets as arguments should produce a exit status
> 	code of 2.  Usually this is provided automatically by make if
> 	the target is missing.

That was one of the many tries to get somewherewhere without making
non-caring packages buggy. Those attempts have cost us more than 10 years
with no result at all.

> That doesn't seem so compelling to me; in the failure case, the
> autobuilder would just try to pick up where it left off and fail
> again, which doesn't sound so expensive.  What am I missing?

If build is simply a call to a well-behaved make-file, you are right.
Sadly well-behaved upstream build systems are not the norm, and the
more expensive a package get, the bigger it usually is and thus the
more likely something fails in this regard.
In some cases that might even be the fault of debian/rules: one
usually has to do some preparations before calling make. If those
are properly guarded by dependencies on some files (and nothing
making those rules phony, as commonly misused patch rules often do),
this problem can be worked around. But that does not help if upstream's
build system does not behave well.

	Bernhard R. Link




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Wed, 02 Mar 2011 09:45:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to Jakub Wilk <jwilk@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Wed, 02 Mar 2011 09:45:05 GMT) Full text and rfc822 format available.

Message #35 received at 604397@bugs.debian.org (full text, mbox):

From: Jakub Wilk <jwilk@debian.org>
To: debian-policy@lists.debian.org, 604397@bugs.debian.org, 229357@bugs.debian.org
Subject: Re: Bug#604397: debian-policy: require build-arch and build-indep targets
Date: Wed, 2 Mar 2011 10:43:24 +0100
* Jonathan Nieder <jrnieder@gmail.com>, 2011-03-01, 21:01:
>So it seems to me that "dpkg-buildpackage -B" ought to be taught to
>run the equivalent of
>
>	debian/rules build-arch
>	if test "$?" = 2
>	then
>		debian/rules build
>	fi

make exits with code 2 "if any errors were encountered". So no, this 
approach cannot be used.

-- 
Jakub Wilk




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Wed, 02 Mar 2011 10:03:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Jonathan Nieder <jrnieder@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Wed, 02 Mar 2011 10:03:03 GMT) Full text and rfc822 format available.

Message #40 received at 604397@bugs.debian.org (full text, mbox):

From: Jonathan Nieder <jrnieder@gmail.com>
To: 604397@bugs.debian.org
Subject: Re: debian-policy: require build-arch and build-indep targets
Date: Wed, 2 Mar 2011 03:59:43 -0600
(-cc: Bug#229357)
Jakub Wilk wrote:
> * Jonathan Nieder <jrnieder@gmail.com>, 2011-03-01, 21:01:

>> So it seems to me that "dpkg-buildpackage -B" ought to be taught to
>> run the equivalent of
>>
>>	debian/rules build-arch
>>	if test "$?" = 2
>>	then
>>		debian/rules build
>>	fi
>
> make exits with code 2 "if any errors were encountered". So no, this
> approach cannot be used.

Did you read the rest of the message?

But okay, I am willing to accept that this is an approach we do
not want to use.  Which still leaves us with a number of options.

To help some existing packages today (and break others):

 1. Find an active "make" maintainer.
 2. Polish the patch in Bug#598534 and apply it.
 3. Use
	make-first-existing-target build-arch build -- -f debian/rules

To help no existing packages today but make it easy for packages
to opt in (and not break the others):

 1. Introduce a Build-Options facility for packages to advertise
    capabilities like this.
 2. Teach dpkg-buildpackage to honor it.

To move towards a world in which all packages support build-arch and
build-indep:

 1. Teach dpkg-buildpackage a new switch to use those targets.
    The operator can easily figure out when they are available if
    building by hand.
 2. Introduce a lintian test for build-arch/build-indep presence.
 3. Contribute patches.
 4. When there are not so many packages left without the feature,
    propose a mass bug-filing/release goal.
 5. Finally, update policy and make autobuilders use the switch
    to use those targets when building unstable and experimental.

Just my two cents,
Jonathan




Forcibly Merged 604397 619284. Request was from Jonathan Nieder <jrnieder@gmail.com> to control@bugs.debian.org. (Fri, 25 Mar 2011 20:54:04 GMT) Full text and rfc822 format available.

Forcibly Merged 374029 604397 619284. Request was from Russ Allbery <rra@debian.org> to control@bugs.debian.org. (Mon, 04 Apr 2011 02:54:03 GMT) Full text and rfc822 format available.

Added indication that bug 604397 blocks 572750 Request was from Stéphane Glondu <glondu@debian.org> to control@bugs.debian.org. (Sun, 24 Apr 2011 12:42:03 GMT) Full text and rfc822 format available.

Forcibly Merged 345619 374029 604397 619284. Request was from Raphaël Hertzog <hertzog@debian.org> to control@bugs.debian.org. (Sun, 15 May 2011 08:27:05 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Fri, 03 Jun 2011 16:12:07 GMT) Full text and rfc822 format available.

Acknowledgement sent to Roger Leigh <rleigh@codelibre.net>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Fri, 03 Jun 2011 16:12:07 GMT) Full text and rfc822 format available.

Message #53 received at 604397@bugs.debian.org (full text, mbox):

From: Roger Leigh <rleigh@codelibre.net>
To: 604397@bugs.debian.org
Subject: Re: Bug#604397: debian-policy: require build-arch and build-indep targets
Date: Fri, 3 Jun 2011 17:09:33 +0100
[Message part 1 (text/plain, inline)]
On Tue, Mar 01, 2011 at 09:01:31PM -0600, Jonathan Nieder wrote:
> Roger Leigh wrote:
> [out of order for convenience]
> 
> > Just for the record, I've implemented support in debhelper's dh
> > command in #604563.  Once applied, this will automatically add support
> > to the huge chunk of the archive using "tiny" rules files.  cdbs will
> > be next on my list.
> 
> debhelper 8.1.0 has such support now.  Thanks!

With dh and cdbs both supporting build-arch and build-indep automatically,
we are now in the situation that >50% of the archive supports these rules.

Is there any reason we can't now make build-arch and build-indep a "should"
in policy?

> >> * Roger Leigh <rleigh@debian.org>, 2010-11-21, 21:38:
> 
> >>> I'd like to propose that build-arch and build-indep be changed in
> >>> Policy from "may be provided" to "must be provided" in preparation
> >>> for enabling their use.
> 
> Personally, I'm all for it; ideally it would happen in the following
> order:
> 
>  1. Providing build-arch and build-indep becomes a best practice.
>     lintian gains a check.  devref encourages the practice.

I wrote a lintian check which is currently in a patch proposed as lintian
bug #605012.  I'm not sure if it needs to be in Policy before or after
it's implemented in lintian?  I thought lintian reflected policy for the
most part.

It probably needs a little more polish (testsuite support) before it can
be applied, but the core checks are done.

>  2. Becomes a policy "should".
> 
>  3. Becomes a policy "must".
> 
> That sounds slow, no?  Yes, that's the point.  I'd like to propose
> that we not make most of the packages in the archive instantly RC
> buggy, today.

Agreed.  Having lintian point out all the packages that are buggy would
be very useful.  We can wait until the majority of packages implement
the new rules before making it an RC-buggy "must".

Is there any recent work on the rules checking in make which would allow
dpkg-buildpackage to use the rules if present, but fall back to build
if absent?  This would be the most pragmatic approach, because it will
both provide backwards compatibility with all older source packages, and
use the rules if present in new ones.


Regards,
Roger
-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
   `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Sat, 04 Jun 2011 03:27:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to Steve Langasek <vorlon@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Sat, 04 Jun 2011 03:27:05 GMT) Full text and rfc822 format available.

Message #58 received at 604397@bugs.debian.org (full text, mbox):

From: Steve Langasek <vorlon@debian.org>
To: Roger Leigh <rleigh@codelibre.net>, 604397@bugs.debian.org
Subject: Re: Bug#604397: debian-policy: require build-arch and build-indep targets
Date: Fri, 3 Jun 2011 20:25:11 -0700
[Message part 1 (text/plain, inline)]
On Fri, Jun 03, 2011 at 05:09:33PM +0100, Roger Leigh wrote:
> > > Just for the record, I've implemented support in debhelper's dh
> > > command in #604563.  Once applied, this will automatically add support
> > > to the huge chunk of the archive using "tiny" rules files.  cdbs will
> > > be next on my list.

> > debhelper 8.1.0 has such support now.  Thanks!

> With dh and cdbs both supporting build-arch and build-indep automatically,
> we are now in the situation that >50% of the archive supports these rules.

> Is there any reason we can't now make build-arch and build-indep a "should"
> in policy?

How does adding this to policy help us get to the point where we're willing
to turn on use of build-arch on the buildds?

I don't think a policy "should" actually moves us down that road, because
there's no actual penalty for not complying.  The issue is *not* that
maintainers don't, in general, implement this target (in fact, it's been
around forever in the dh_make templates), the issue is that we have no way
of making use of it without a painful transition where lots of packages will
FTBFS, and it's hard even to know which packages those are without trying to
build them.

If people want to track those packages down and NMU them today, they don't
need a policy "should" to do so.  Nor is a policy "should" likely to make
the set of packages in need of an NMU for this any smaller.

If we're willing to flip the switch on the autobuilders and force
maintainers to deal with the breakage, we don't need a policy "should"
either... we can go straight to a policy "must" as soon as the switch is
flipped (and we should flip that switch *ASAP*, not let this question drag
on any further into the release cycle).

And if we want to provide a smooth transition instead, using something like
Joey's proposed make-first-existing-target interface in bug #598534 (as
discussed on debian-devel in
http://lists.debian.org/debian-devel/2010/09/msg00704.html), we don't need
this to be a policy "should" or "must" at all, because the autobuilders can
then DTRT for any package whether or not it implements the build-arch target
and the presence of the target merely lets us optimize build times and
reduce build dependencies - so it could remain a policy "may" indefinitely
(with some tightening of the language about how not having build-arch
requires different bits to be in Build-Depends than if you do have it).

> >  1. Providing build-arch and build-indep becomes a best practice.
> >     lintian gains a check.  devref encourages the practice.

> I wrote a lintian check which is currently in a patch proposed as lintian
> bug #605012.  I'm not sure if it needs to be in Policy before or after
> it's implemented in lintian?  I thought lintian reflected policy for the
> most part.

> It probably needs a little more polish (testsuite support) before it can
> be applied, but the core checks are done.

Unfortunately I see the same problem with this lintian check as with all the
rest - if we can actually check for the existence of the target *reliably*,
then we don't need to enforce its presence at all.

> Is there any recent work on the rules checking in make which would allow
> dpkg-buildpackage to use the rules if present, but fall back to build
> if absent?  This would be the most pragmatic approach, because it will
> both provide backwards compatibility with all older source packages, and
> use the rules if present in new ones.

The patch is outstanding; the make maintainer is TTBOMK currently
unavailable for Debian work.  If there's a consensus on
debian-policy/devel/ctte that we should go the make-first-existing-target
route, I would be more than happy to do an NMU of make to facilitate this.

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek@ubuntu.com                                     vorlon@debian.org
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Sat, 04 Jun 2011 04:33:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Jonathan Nieder <jrnieder@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Sat, 04 Jun 2011 04:33:02 GMT) Full text and rfc822 format available.

Message #63 received at 604397@bugs.debian.org (full text, mbox):

From: Jonathan Nieder <jrnieder@gmail.com>
To: Steve Langasek <vorlon@debian.org>, 604397@bugs.debian.org
Cc: Roger Leigh <rleigh@codelibre.net>
Subject: Re: debian-policy: require build-arch and build-indep targets
Date: Fri, 3 Jun 2011 23:32:13 -0500
Steve Langasek wrote:

> I don't think a policy "should" actually moves us down that road, because
> there's no actual penalty for not complying.  The issue is *not* that
> maintainers don't, in general, implement this target (in fact, it's been
> around forever in the dh_make templates),

As a counterexample, I did not implement the target in any packages I
worked on unless they actually could benefit from the
build-arch/build-indep distinction.  There was simply no obvious
reason to do it, and the documentation gave no hint that it was a good
idea.  The devref would be a better place to put that suggestion given
the current state of things, but to say that documenting it somewhere
would accomplish nothing seems like overstating things.

> the issue is that we have no way
> of making use of it without a painful transition where lots of packages will
> FTBFS, and it's hard even to know which packages those are without trying to
> build them.

Agreed, of course.  The obvious naïve conclusions are:

 1. dpkg-buildpackage could benefit from a new commandline option for
    people to use to test the build-arch/build-indep targets *without*
    changing the behavior on autobuilders, and

 2. autobuilders could benefit from a Build-Options for maintainers to
    declare that it is safe to use the build-arch/build-indep targets
    today.

But see below.

> If we're willing to flip the switch on the autobuilders and force
> maintainers to deal with the breakage, we don't need a policy "should"
> either... we can go straight to a policy "must" as soon as the switch is
> flipped (and we should flip that switch *ASAP*, not let this question drag
> on any further into the release cycle).

Flipping that switch would imho be a bad idea in the current release
cycle anyway.  It's a huge change.

> And if we want to provide a smooth transition instead, using something like
> Joey's proposed make-first-existing-target interface in bug #598534 (as
> discussed on debian-devel in
> http://lists.debian.org/debian-devel/2010/09/msg00704.html)

I was about to say: without a full archive rebuild there is no
guarantee that that would actually provide a smooth transition,
either, since the build-arch targets are not well tested in isolation.
But that was misguided of me:

Assuming that the "build" target depends on "build-arch" and
"build-indep" and does nothing for itself, the most frightening
obvious case is a package that does too little in build-arch, relying
on build-indep to generate some other files (e.g., manpages) that are
needed for binary-arch.  Typically binary-arch would depend on build
in that case, so while breakage of this kind is possible, I don't
imagine it would be widespread.

So I'm all for this.  A test rebuild would still be comforting, of
course.

> The patch is outstanding; the make maintainer is TTBOMK currently
> unavailable for Debian work.  If there's a consensus on
> debian-policy/devel/ctte that we should go the make-first-existing-target
> route, I would be more than happy to do an NMU of make to facilitate this.

Thanks much!  If you'd like, I can try out the two patches from
Bug#598534 and send a comparison there.

By the way, shouldn't the make package be orphaned to reflect the
current reality?  Totally unrelated to the above, I would like to work
with the maintainer to see the latest upstream version of "make"
packaged in experimental, but for a while now the maintainer has been
hard to reach.




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Sat, 04 Jun 2011 18:48:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Russ Allbery <rra@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Sat, 04 Jun 2011 18:48:02 GMT) Full text and rfc822 format available.

Message #68 received at 604397@bugs.debian.org (full text, mbox):

From: Russ Allbery <rra@debian.org>
To: debian-policy@lists.debian.org, 604397@bugs.debian.org
Subject: Re: Bug#604397: debian-policy: require build-arch and build-indep targets
Date: Sat, 04 Jun 2011 11:45:17 -0700
"Bernhard R. Link" <brlink@debian.org> writes:
> * Steve Langasek <vorlon@debian.org> [110604 05:27]:

>> If we're willing to flip the switch on the autobuilders and force
>> maintainers to deal with the breakage, we don't need a policy "should"
>> either... we can go straight to a policy "must" as soon as the switch
>> is flipped (and we should flip that switch *ASAP*, not let this
>> question drag on any further into the release cycle).

> Having seen this discussion now for almost a decade with people claiming
> smooth transitions coming really soon, I think this is the only way
> unless someone can actually show something for the buildds working now
> (and not in some months, because it has been months for years now).

Agreed.  I really don't think it would be that big of a deal, particularly
right now early in the release cycle.  We could pick some point where
there aren't a lot of ongoing transitions, give people a few weeks of
advance warning, and throw an NMU party, and we'd be done.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Sat, 04 Jun 2011 20:48:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Steve Langasek <vorlon@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Sat, 04 Jun 2011 20:48:03 GMT) Full text and rfc822 format available.

Message #73 received at 604397@bugs.debian.org (full text, mbox):

From: Steve Langasek <vorlon@debian.org>
To: Jonathan Nieder <jrnieder@gmail.com>, 604397@bugs.debian.org
Cc: Roger Leigh <rleigh@codelibre.net>
Subject: Re: Bug#604397: debian-policy: require build-arch and build-indep targets
Date: Sat, 4 Jun 2011 13:45:43 -0700
[Message part 1 (text/plain, inline)]
On Fri, Jun 03, 2011 at 11:32:13PM -0500, Jonathan Nieder wrote:
> > I don't think a policy "should" actually moves us down that road, because
> > there's no actual penalty for not complying.  The issue is *not* that
> > maintainers don't, in general, implement this target (in fact, it's been
> > around forever in the dh_make templates),

> As a counterexample, I did not implement the target in any packages I
> worked on unless they actually could benefit from the
> build-arch/build-indep distinction.  There was simply no obvious
> reason to do it, and the documentation gave no hint that it was a good
> idea.  The devref would be a better place to put that suggestion given
> the current state of things, but to say that documenting it somewhere
> would accomplish nothing seems like overstating things.

I'm saying it won't accomplish enough to be a worthwhile intermediate step,
and want us to get back to figuring out a roadmap that would enable us to
turn on build-arch handling on the buildds this cycle.

> > And if we want to provide a smooth transition instead, using something like
> > Joey's proposed make-first-existing-target interface in bug #598534 (as
> > discussed on debian-devel in
> > http://lists.debian.org/debian-devel/2010/09/msg00704.html)

> I was about to say: without a full archive rebuild there is no
> guarantee that that would actually provide a smooth transition,
> either, since the build-arch targets are not well tested in isolation.
> But that was misguided of me:

> Assuming that the "build" target depends on "build-arch" and
> "build-indep" and does nothing for itself, the most frightening
> obvious case is a package that does too little in build-arch, relying
> on build-indep to generate some other files (e.g., manpages) that are
> needed for binary-arch.  Typically binary-arch would depend on build
> in that case, so while breakage of this kind is possible, I don't
> imagine it would be widespread.

> So I'm all for this.  A test rebuild would still be comforting, of
> course.

Given that you seem to have argued in this same mail for providing both an
intermediate dpkg-buildpackage switch, and introducing a Build-Options field
that would have to be populated manually, I'm a little unclear: do you think
make-first-existing-target is a sufficient solution for the buildds, or not?
If you don't think it is, can you point to any problems with the
implementation?

> > The patch is outstanding; the make maintainer is TTBOMK currently
> > unavailable for Debian work.  If there's a consensus on
> > debian-policy/devel/ctte that we should go the make-first-existing-target
> > route, I would be more than happy to do an NMU of make to facilitate this.

> Thanks much!  If you'd like, I can try out the two patches from
> Bug#598534 and send a comparison there.

Thanks for the offer.  How do you plan to try them out?  Are you proposing a
full-archive rebuild?

> By the way, shouldn't the make package be orphaned to reflect the
> current reality?  Totally unrelated to the above, I would like to work
> with the maintainer to see the latest upstream version of "make"
> packaged in experimental, but for a while now the maintainer has been
> hard to reach.

I think it would be reasonable to let the MIA team know about Manoj's
protracted absence (DevRef 7.4).

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek@ubuntu.com                                     vorlon@debian.org
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Sat, 04 Jun 2011 21:27:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Jonathan Nieder <jrnieder@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Sat, 04 Jun 2011 21:27:03 GMT) Full text and rfc822 format available.

Message #78 received at 604397@bugs.debian.org (full text, mbox):

From: Jonathan Nieder <jrnieder@gmail.com>
To: Steve Langasek <vorlon@debian.org>
Cc: 604397@bugs.debian.org, Roger Leigh <rleigh@codelibre.net>
Subject: Re: debian-policy: require build-arch and build-indep targets
Date: Sat, 4 Jun 2011 16:22:43 -0500
Steve Langasek wrote:

> Given that you seem to have argued in this same mail for providing both an
> intermediate dpkg-buildpackage switch, and introducing a Build-Options field
> that would have to be populated manually, I'm a little unclear: do you think
> make-first-existing-target is a sufficient solution for the buildds, or not?

Sorry, I switched opinions mid-message.  I think make-first-existing-target
is sufficient.

More precisely, what I meant to say was that before I had thought
carefully about it, approaches like Build-Options and so on were
appealing, but after your advice, make-first-existing-target seems like a
much better idea (because simpler and sufficient).

> On Fri, Jun 03, 2011 at 11:32:13PM -0500, Jonathan Nieder wrote:

>> Thanks much!  If you'd like, I can try out the two patches from
>> Bug#598534 and send a comparison there.
>
> Thanks for the offer.  How do you plan to try them out?  Are you proposing a
> full-archive rebuild?

I am just going to try to break them.  Cases like these:

 A.
	%:
		dh $@

 B.
	build clean install binary-arch binary-indep binary:
		dh $@
	.PHONY: build-arch build-indep

 C. something using cdbs

 E.
	... typical debian/rules, plus:

	build-indep:
		false

Meanwhile I would be happy to see progress on the dpkg-buildpackage
side.  Once the pieces are together it should be possible to beg someone
to do a full archive rebuild before and after hitting the switch and list
packages that failed to build or whose binary packages changed in size
substantially (though as mentioned before, because "debian/rules
binary-arch" is suppposed to work on its own already, I'm not too worried
about it).

> I think it would be reasonable to let the MIA team know about Manoj's
> protracted absence (DevRef 7.4).

Good idea.  Will do.




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Sat, 04 Jun 2011 23:42:08 GMT) Full text and rfc822 format available.

Acknowledgement sent to Steve Langasek <vorlon@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Sat, 04 Jun 2011 23:42:08 GMT) Full text and rfc822 format available.

Message #83 received at 604397@bugs.debian.org (full text, mbox):

From: Steve Langasek <vorlon@debian.org>
To: Jonathan Nieder <jrnieder@gmail.com>
Cc: 604397@bugs.debian.org, Roger Leigh <rleigh@codelibre.net>
Subject: Re: debian-policy: require build-arch and build-indep targets
Date: Sat, 4 Jun 2011 16:39:19 -0700
On Sat, Jun 04, 2011 at 04:22:43PM -0500, Jonathan Nieder wrote:

> >> Thanks much!  If you'd like, I can try out the two patches from
> >> Bug#598534 and send a comparison there.

> > Thanks for the offer.  How do you plan to try them out?  Are you proposing a
> > full-archive rebuild?

> I am just going to try to break them.  Cases like these:

>  A.
> 	%:
> 		dh $@

>  B.
> 	build clean install binary-arch binary-indep binary:
> 		dh $@
> 	.PHONY: build-arch build-indep

>  C. something using cdbs

>  E.
> 	... typical debian/rules, plus:
> 
> 	build-indep:
> 		false

Ok.  In that case, some of the past discussions may be informative:

  http://lists.debian.org/debian-devel/2007/07/msg00048.html
  http://lists.debian.org/debian-devel/2007/07/msg00113.html

The main difference between the robustness of the implementations should be
when faced with debian/rules that is not a policy-compliant makefile.

> Meanwhile I would be happy to see progress on the dpkg-buildpackage
> side.  Once the pieces are together it should be possible to beg someone
> to do a full archive rebuild before and after hitting the switch and list
> packages that failed to build or whose binary packages changed in size
> substantially (though as mentioned before, because "debian/rules
> binary-arch" is suppposed to work on its own already, I'm not too worried
> about it).

That part is apparently trivial, as I seem to have written a patch for it 4
years ago :-)

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek@ubuntu.com                                     vorlon@debian.org




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 07:57:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Guillem Jover <guillem@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 07:57:04 GMT) Full text and rfc822 format available.

Message #88 received at 604397@bugs.debian.org (full text, mbox):

From: Guillem Jover <guillem@debian.org>
To: Jonathan Nieder <jrnieder@gmail.com>, 604397@bugs.debian.org
Subject: Re: Bug#604397: debian-policy: require build-arch and build-indep targets
Date: Mon, 6 Jun 2011 09:55:25 +0200
Hi!

On Wed, 2011-03-02 at 03:59:43 -0600, Jonathan Nieder wrote:
> Did you read the rest of the message?
> 
> But okay, I am willing to accept that this is an approach we do
> not want to use.  Which still leaves us with a number of options.
> 
> To help some existing packages today (and break others):
> 
>  1. Find an active "make" maintainer.
>  2. Polish the patch in Bug#598534 and apply it.
>  3. Use
> 	make-first-existing-target build-arch build -- -f debian/rules

Well, I don't think make-first-existing-target belongs in the make
package (but then I'm not the make maintainer). It's clearly a hack,
but if we'd decide this is good enough, then I'd rather see this
supported natively by make itself, which would avoid side-effects
when running make twice (or more), otherwise it might make slightly
more sense to be bolted into dpkg-buildpackage instead.

> To help no existing packages today but make it easy for packages
> to opt in (and not break the others):
> 
>  1. Introduce a Build-Options facility for packages to advertise
>     capabilities like this.
>  2. Teach dpkg-buildpackage to honor it.

I don't think Build-Options/Build-Features is a good idea, as stated
previously on these bug reports. For example most of my packages
already support build-arch/build-indep, now I have to also state that
explicitly? Buh. :)

> To move towards a world in which all packages support build-arch and
> build-indep:
> 
>  1. Teach dpkg-buildpackage a new switch to use those targets.
>     The operator can easily figure out when they are available if
>     building by hand.
>  2. Introduce a lintian test for build-arch/build-indep presence.
>  3. Contribute patches.
>  4. When there are not so many packages left without the feature,
>     propose a mass bug-filing/release goal.
>  5. Finally, update policy and make autobuilders use the switch
>     to use those targets when building unstable and experimental.

And I think we should just go ahead with a flag day and declare
build-arch/build-indep mandatory (at least for the cases were it makes
sense, see below). I'd even go further and combine that with
dpkg-buildpackage stopping to set compilation flags on the environment,
so we only have to deal once with possible mass FTBFS on the archive.

Also not all packages really need them:

  * Source packages that only provide arch:any packages should not have
    Build-Depends-Indep nor Build-Conflicts-Indep, so it should not
    matter if build/binary targets are called directly (instead of
    build-arch/binary-arch).
  * Source packages that only provide arch:all packages might have
    Build-Depends-Indep and Build-Conflicts-Indep, and because they
    have to be satisfied anyway, it does not matter if build/binary
    targets are called directly (instead of build-indep/binary-indep).
  * They really only makes sense for source packages providing both
    arch:any and arch:all packages.

This should reduce the affected packages.

regards,
guillem




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 09:18:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Steve Langasek <vorlon@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 09:18:08 GMT) Full text and rfc822 format available.

Message #93 received at 604397@bugs.debian.org (full text, mbox):

From: Steve Langasek <vorlon@debian.org>
To: 604397@bugs.debian.org, debian-ctte@lists.debian.org
Subject: Request for TC to rule on a course of action for supporting build-arch
Date: Mon, 6 Jun 2011 02:15:37 -0700
[Message part 1 (text/plain, inline)]
unmerge 604397
clone 604397 -1
reassign -1 tech-ctte
retitle -1 Please rule on how to implement debian/rules build-arch
merge 345619 604397
thanks

Fellow Committee members,

I am requesting your assistance in helping the project come to a conclusion
about how we can support the use of the 'build-arch' debian/rules target on
autobuilders, letting us at long last split Build-Depends and
Build-Depends-Indep the way they're meant to be split.

As you can see from the history of the source bug, this question has
lingered for some time - the first policy bug having been opened over 5
years ago.  I believe it has remained unresolved principally because we have
several different ways that the problem could be solved, each of which is
better than not solving it but none of which is overwhelmingly superior to
the others.  As a result, it has taken longer to decide how to implement
this than it would have taken us to actually implement it, even using one of
the methods that require touching every affected package in the archive.

I think that makes this one of the rare cases where the TC can usefully move
Debian forward by voting rather than by seeking consensus, because the root
problem here is the *lack of a decision* rather than any real technical
dispute about the right course of action.

The Technical Committee has sufficient authority to address this question
under any of §6.1.{1,2,4,5}.  If you prefer, we could also ask for a
referral from the policy editors or the dpkg maintainers, to eliminate any
question of supermajority requirements.

If this were to be put to a vote today, I would propose the following ballot
options:

 1) Implement support for calling 'debian/rules build-arch' in place of
    'debian/rules build' by checking for the presence of the target using
    'make -qn'.[1]

 2) Implement support for calling 'debian/rules build-arch' with a fallback
    to 'debian/rules build' by checking whether the output of the build-arch
    target matches that of a dummy target.[2]

 3) Implement support for calling 'debian/rules build-arch' in place of
    'debian/rules build' if a Build-Options field is set in debian/control
    of the source package specifying that this target is supported.[3]

 4) Turn on direct use of 'debian/rules build-arch' on the autobuilders for
    all packages in unstable and experimental immediately, with no fallback
    if the target does not exist; requires a corresponding update to Policy
    and mass updates to fix packages that fail to build as a result.

 5) Further Discussion

None of these options require extensive design work on the part of the TC;
where patches don't already exist, the implementation should be trivial.

I'm happy to expand on each of these options and present the pros and cons
as I see them, or you can each peruse the extensive historical record at
your leisure... :)

My own vote would likely be: 1, 2, 3, 5, 4.  (I could be persuaded to rank 4
above FD if this were the only way to move forward; but that's indisputably
the most disruptive to the archive, so I would hope we could reach agreement
that some or all of the other options are better.)

Cheers,
-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek@ubuntu.com                                     vorlon@debian.org

[1] http://lists.debian.org/debian-devel/2007/07/msg00048.html
[2] http://bugs.debian.org/598534
[3] http://bugs.debian.org/229357
[signature.asc (application/pgp-signature, inline)]

Disconnected #604397 from all other report(s). Request was from Steve Langasek <vorlon@debian.org> to control@bugs.debian.org. (Mon, 06 Jun 2011 09:18:16 GMT) Full text and rfc822 format available.

Bug 604397 cloned as bug 629385. Request was from Steve Langasek <vorlon@debian.org> to control@bugs.debian.org. (Mon, 06 Jun 2011 09:18:20 GMT) Full text and rfc822 format available.

Merged 345619 374029 604397 619284. Request was from Steve Langasek <vorlon@debian.org> to control@bugs.debian.org. (Mon, 06 Jun 2011 09:18:24 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 10:46:09 GMT) Full text and rfc822 format available.

Acknowledgement sent to Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 10:46:12 GMT) Full text and rfc822 format available.

Message #104 received at 604397@bugs.debian.org (full text, mbox):

From: Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>
To: 604397@bugs.debian.org, debian-ctte@lists.debian.org
Subject: Re: Bug#604397: Request for TC to rule on a course of action for supporting build-arch
Date: Mon, 6 Jun 2011 12:09:34 +0200
On Mon, Jun 06, 2011 at 02:15:37AM -0700, Steve Langasek wrote:
>  1) Implement support for calling 'debian/rules build-arch' in place of
>     'debian/rules build' by checking for the presence of the target using
>     'make -qn'.[1]
> 
>  2) Implement support for calling 'debian/rules build-arch' with a fallback
>     to 'debian/rules build' by checking whether the output of the build-arch
>     target matches that of a dummy target.[2]
> 
>  3) Implement support for calling 'debian/rules build-arch' in place of
>     'debian/rules build' if a Build-Options field is set in debian/control
>     of the source package specifying that this target is supported.[3]

For the record, I have been working to the patch that implement 3) to get accepted,
see bug #229357.

Cheers,
-- 
Bill. <ballombe@debian.org>

Imagine a large red swirl here. 




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 10:46:19 GMT) Full text and rfc822 format available.

Acknowledgement sent to Jakub Wilk <jwilk@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 10:46:22 GMT) Full text and rfc822 format available.

Message #109 received at 604397@bugs.debian.org (full text, mbox):

From: Jakub Wilk <jwilk@debian.org>
To: 604397@bugs.debian.org
Subject: Re: Bug#604397: debian-policy: require build-arch and build-indep targets
Date: Mon, 6 Jun 2011 12:29:20 +0200
* Guillem Jover <guillem@debian.org>, 2011-06-06, 09:55:
>And I think we should just go ahead with a flag day and declare 
>build-arch/build-indep mandatory (at least for the cases were it makes 
>sense, see below).

Agreed.

>I'd even go further and combine that with dpkg-buildpackage stopping to 
>set compilation flags on the environment, so we only have to deal once 
>with possible mass FTBFS on the archive.

I don't think this is a good idea. To enable build-arch, we'll probably 
need quite a few NMUing volunteers, and it IME is very demotivating when 
you have to fix a seemingly simple bug (e.g. missing build-arch target) 
only to discover tha the package would mysteriously FTBFS anyway (e.g. 
because it relied on *FLAGS exported by dpkg-buildpackage).

>Also not all packages really need them:
[...]
>They really only makes sense for source packages providing both 
>arch:any and arch:all packages.
>
>This should reduce the affected packages.

Just to provide some statistics:
- There are 16491 source packages in unstable[0].
- Out of these, 2206 source package build both arch:all and arch:any 
binaries[1].
- Out of these, 214 already FTBFS in unstable[2].
- Out of remaining 1992 packages:
  + 486 use so called "tiny" debhelper rules.
  + 664 use cdbs.
  + At least 175 defines both build-arch and build-indep explicitly.

So that leaves us with at most 667 packages to fix (plus an unknown 
number of packages that define build-{arch,indep}, but do so 
incorrectly). This is still a big number, we'd roughly double the number 
of currently observed FTBFS bugs if we announced flag day *today*.

However:
- Lintian doesn't warn about missing build-* targets yet, so many 
maintainers are not aware that their package are affected by this issue.
- These FTBFS would be trivial to fix. We'd just need an small army of 
volunteers (say, 10 people) to fix most of them in a single weekend.


[0] UDD query:
select distinct source from packages
where distribution='debian' and release='sid';

[1] UDD query:
select * from (
     select source, count(package) as arch_all from packages
     where distribution='debian' and release='sid' and architecture='all'
     group by source
) pkg_all
natural join
(
     select source, count(package) as arch_any from packages
     where distribution='debian' and release='sid' and architecture!='all'
     group by source
) pkg_any;

[2] UDD query (for all FTBFS bugs):
select source from bugs
where affects_unstable and severity>='serious' and title ilike '%ftbfs%' and status!='done';

-- 
Jakub Wilk




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 11:39:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 11:39:08 GMT) Full text and rfc822 format available.

Message #114 received at 604397@bugs.debian.org (full text, mbox):

From: Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>
To: Guillem Jover <guillem@debian.org>, 604397@bugs.debian.org
Cc: Jonathan Nieder <jrnieder@gmail.com>
Subject: Re: Bug#604397: debian-policy: require build-arch and build-indep targets
Date: Mon, 6 Jun 2011 13:37:22 +0200
On Mon, Jun 06, 2011 at 09:55:25AM +0200, Guillem Jover wrote:
> > To help no existing packages today but make it easy for packages
> > to opt in (and not break the others):
> > 
> >  1. Introduce a Build-Options facility for packages to advertise
> >     capabilities like this.
> >  2. Teach dpkg-buildpackage to honor it.
> 
> I don't think Build-Options/Build-Features is a good idea, as stated
> previously on these bug reports. For example most of my packages
> already support build-arch/build-indep, now I have to also state that
> explicitly? Buh. :)

Only for packages that would benefit from it, i.e. packages that generates
both arch-indep and arch-all packages and have large Build-Depends-Indep dependencies. 
This stays an optional feature. No need to add it everywhere.

Cheers,
-- 
Bill. <ballombe@debian.org>

Imagine a large red swirl here. 




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 13:36:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Roger Leigh <rleigh@codelibre.net>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 13:36:05 GMT) Full text and rfc822 format available.

Message #119 received at 604397@bugs.debian.org (full text, mbox):

From: Roger Leigh <rleigh@codelibre.net>
To: 604397@bugs.debian.org
Subject: Re: Bug#604397: debian-policy: require build-arch and build-indep targets
Date: Mon, 6 Jun 2011 14:33:22 +0100
[Message part 1 (text/plain, inline)]
On Fri, Jun 03, 2011 at 08:25:11PM -0700, Steve Langasek wrote:
> On Fri, Jun 03, 2011 at 05:09:33PM +0100, Roger Leigh wrote:
> > > > Just for the record, I've implemented support in debhelper's dh
> > > > command in #604563.  Once applied, this will automatically add support
> > > > to the huge chunk of the archive using "tiny" rules files.  cdbs will
> > > > be next on my list.
> 
> > > debhelper 8.1.0 has such support now.  Thanks!
> 
> > With dh and cdbs both supporting build-arch and build-indep automatically,
> > we are now in the situation that >50% of the archive supports these rules.
> 
> > Is there any reason we can't now make build-arch and build-indep a "should"
> > in policy?
> 
> How does adding this to policy help us get to the point where we're willing
> to turn on use of build-arch on the buildds?
> 
> I don't think a policy "should" actually moves us down that road, because
> there's no actual penalty for not complying.
[…]
> If people want to track those packages down and NMU them today, they don't
> need a policy "should" to do so.  Nor is a policy "should" likely to make
> the set of packages in need of an NMU for this any smaller.

My main objective to make it a "should", and implementing the lintian
check in addition, is that we would have a transitional phase where all
developers would be

- made aware of the changing requirement for the targets
- be told about the deficiency when they run lintian
- have time to transition their packages before it becomes a "must".

While neither of these changes actively "enforces" the addition of
these targets, neither do any harm, and by actively encouraging
adoption of the targets it will reduce the total number of NMUs
required should we go for the approach of a straight change in
dpkg-buildpackage.

Neither of these dictate /how/ the transition should proceed, which is
a separate issue which you brought to the CTTE, but both will be needed
irrespective of the choice picked by the technical committee.

> If we're willing to flip the switch on the autobuilders and force
> maintainers to deal with the breakage, we don't need a policy "should"
> either... we can go straight to a policy "must" as soon as the switch is
> flipped (and we should flip that switch *ASAP*, not let this question drag
> on any further into the release cycle).

This would make sense if we do want to go the "break the world" route;
the above does not preclude that, but does allow for a (potentially
very brief) period to allow for maintainers to pre-emptively fix their
packages before the change is made.  I'm certainly not opposed to this
approach if the decision is made to go this way.

> And if we want to provide a smooth transition instead, using something like
> Joey's proposed make-first-existing-target interface in bug #598534 (as
> discussed on debian-devel in
> http://lists.debian.org/debian-devel/2010/09/msg00704.html), we don't need
> this to be a policy "should" or "must" at all, because the autobuilders can
> then DTRT for any package whether or not it implements the build-arch target
> and the presence of the target merely lets us optimize build times and
> reduce build dependencies - so it could remain a policy "may" indefinitely
> (with some tightening of the language about how not having build-arch
> requires different bits to be in Build-Depends than if you do have it).

I would see make-first-existing-target as a good method for easing the
transition, and we could leave it a "may" indefinitely.  But would it
not be better (long-term) to aim for complete transition to requiring
build-arch/build-indep for all packages (as for binary-arch/
binary-indep) and use make-first-existing-target as a useful strategy
for doing the transition, but not as a long-term feature?  Especially
since it's a bit of a hack.

> > >  1. Providing build-arch and build-indep becomes a best practice.
> > >     lintian gains a check.  devref encourages the practice.
> 
> > I wrote a lintian check which is currently in a patch proposed as lintian
> > bug #605012.  I'm not sure if it needs to be in Policy before or after
> > it's implemented in lintian?  I thought lintian reflected policy for the
> > most part.

(This patch is now complete including the testsuite and ready to go.)

> Unfortunately I see the same problem with this lintian check as with all the
> rest - if we can actually check for the existence of the target *reliably*,
> then we don't need to enforce its presence at all.

It isn't 100% reliable (it can't be given pattern rules and includes); it
gives up for dh/cdbs-using packages, but both of these handily already
support the missing targets, so it's reliable enough to be useful.  If
the maintainer chose to use odd pattern rules, these could potentially
cause a false positive.

> > Is there any recent work on the rules checking in make which would allow
> > dpkg-buildpackage to use the rules if present, but fall back to build
> > if absent?  This would be the most pragmatic approach, because it will
> > both provide backwards compatibility with all older source packages, and
> > use the rules if present in new ones.
> 
> The patch is outstanding; the make maintainer is TTBOMK currently
> unavailable for Debian work.  If there's a consensus on
> debian-policy/devel/ctte that we should go the make-first-existing-target
> route, I would be more than happy to do an NMU of make to facilitate this.

As a medium-term solution it would be great if GNU make could itself
support querying whether or not a makefile supports a given target.
Since make needs to parse all the includes etc., only make itself can
do this 100% reliably.  If it's not already done, a feature request
upstream would be good.


Regards,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
   `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 14:03:06 GMT) Full text and rfc822 format available.

Acknowledgement sent to Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 14:03:06 GMT) Full text and rfc822 format available.

Message #124 received at 604397@bugs.debian.org (full text, mbox):

From: Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>
To: 604397@bugs.debian.org, debian-ctte@lists.debian.org
Cc: Steve Langasek <vorlon@debian.org>, Roger Leigh <rleigh@debian.org>
Subject: Re: Bug#604397: Request for TC to rule on a course of action for supporting build-arch
Date: Mon, 6 Jun 2011 15:59:15 +0200
On Mon, Jun 06, 2011 at 12:09:34PM +0200, Bill Allombert wrote:
> On Mon, Jun 06, 2011 at 02:15:37AM -0700, Steve Langasek wrote:
> >  1) Implement support for calling 'debian/rules build-arch' in place of
> >     'debian/rules build' by checking for the presence of the target using
> >     'make -qn'.[1]
> > 
> >  2) Implement support for calling 'debian/rules build-arch' with a fallback
> >     to 'debian/rules build' by checking whether the output of the build-arch
> >     target matches that of a dummy target.[2]
> > 
> >  3) Implement support for calling 'debian/rules build-arch' in place of
> >     'debian/rules build' if a Build-Options field is set in debian/control
> >     of the source package specifying that this target is supported.[3]
> 
> For the record, I have been working to the patch that implement 3) to get accepted,
> see bug #229357.

Still for the record this patch has been accepted in dpkg as of today:
    http://git.debian.org/?p=dpkg/dpkg.git;a=commitdiff;h=14d48ef
Does that make this bug moot for all concerned ?

I plan to repropose #218893 with updated wording to match this dpkg change.

Cheers,
-- 
Bill. <ballombe@debian.org>

Imagine a large red swirl here. 




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 14:03:08 GMT) Full text and rfc822 format available.

Acknowledgement sent to Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 14:03:08 GMT) Full text and rfc822 format available.

Message #129 received at 604397@bugs.debian.org (full text, mbox):

From: Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>
To: Roger Leigh <rleigh@debian.org>, 604397@bugs.debian.org
Subject: Re: Bug#604397: debian-policy: build-arch and build-indep targets are required
Date: Mon, 6 Jun 2011 16:02:05 +0200
On Sun, Nov 21, 2010 at 09:38:31PM +0000, Roger Leigh wrote:
> Package: debian-policy
> Version: 3.9.1.0
> Severity: normal
> Tags: patch
> 
> Hi,
> 
> The reason for this is that we do our building using dpkg-buildpackage,
> and while binary-arch and binary-indep are required targets used by
> dpkg-buildpackage, the corresponding build-arch and build-indep targets
> are not required and are not used by dpkg-buildpackage at present.

For the record, the patch to dpkg to implement Build-Features: build-arch
has been accepted. 
So we only need to add Build-Features: build-arch to the package that will
benefit from it (mainly packages that build both arch-indep and arch-all 
targets).

Cheers,
-- 
Bill. <ballombe@debian.org>

Imagine a large red swirl here. 




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 14:39:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Roger Leigh <rleigh@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 14:39:03 GMT) Full text and rfc822 format available.

Message #134 received at 604397@bugs.debian.org (full text, mbox):

From: Roger Leigh <rleigh@debian.org>
To: Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>, 604397@bugs.debian.org
Cc: debian-ctte@lists.debian.org, Steve Langasek <vorlon@debian.org>, Roger Leigh <rleigh@debian.org>
Subject: Re: Bug#604397: Request for TC to rule on a course of action for supporting build-arch
Date: Mon, 6 Jun 2011 15:37:07 +0100
[Message part 1 (text/plain, inline)]
On Mon, Jun 06, 2011 at 03:59:15PM +0200, Bill Allombert wrote:
> On Mon, Jun 06, 2011 at 12:09:34PM +0200, Bill Allombert wrote:
> > On Mon, Jun 06, 2011 at 02:15:37AM -0700, Steve Langasek wrote:
> > >  1) Implement support for calling 'debian/rules build-arch' in place of
> > >     'debian/rules build' by checking for the presence of the target using
> > >     'make -qn'.[1]
> > > 
> > >  2) Implement support for calling 'debian/rules build-arch' with a fallback
> > >     to 'debian/rules build' by checking whether the output of the build-arch
> > >     target matches that of a dummy target.[2]
> > > 
> > >  3) Implement support for calling 'debian/rules build-arch' in place of
> > >     'debian/rules build' if a Build-Options field is set in debian/control
> > >     of the source package specifying that this target is supported.[3]
> > 
> > For the record, I have been working to the patch that implement 3) to get accepted,
> > see bug #229357.
> 
> Still for the record this patch has been accepted in dpkg as of today:
>     http://git.debian.org/?p=dpkg/dpkg.git;a=commitdiff;h=14d48ef
> Does that make this bug moot for all concerned ?
> 
> I plan to repropose #218893 with updated wording to match this dpkg change.

This is great, thanks for doing this!

Has the following been considered:
- adding a command-line option for dpkg-buildpackage to explicitly
  enable particular build-features (overriding the feature in the
  source package).

This would allow us to test build the whole archive with the
build-arch and build-indep targets enforced.  And it would also
permit the buildds to switch to using them by default in the future.
And it would also allow the dpkg-buildpackage defaults for that
particular setting to be altered in the future as well.

The question to consider here is whether or not this particular build
feature is "transitional" or if it will remain indefinitely.

Needing to explicitly enable a target that should be the default is
something I'm not that happy with.  While it's fine to do this as a
means of transitioning with minimal breakage, and I'm totally OK with
this as a means of getting the transition done, I would prefer it not
be a requirement in perpetuity.  Once the archive has sufficient
coverage to make mandating the targets a realistic possibility, I
would be all for making it mandatory.


Now that dpkg-buildpackage can safely utilise the build-arch and
build-indep targets with the build feature enabled, we can make any
required changes to sbuild's build dependency handling to get it
used on the buildds.  Currently we
- always install Build-Depends and remove Build-Conflicts
- always build binary-arch packages
- build binary-indep packages in addition on request
- install Build-Depends-Indep and remove Build-Conflicts-Indep
  if building binary-indep packages

We don't currently have an option to build /only/ binary-indep
packages, but this might be useful for arch-all autobuilding if we
don't want to build them at the same time as for binary-arch.
In this case, would we still want to install Build-Depends and
Build-Depends-Indep?  (I would think so.)

We don't currently have a Build-Depends-Arch and Build-Conflicts-Arch;
but this might be useful.  This would allow Build-Depends to be used
for "generic" dependencies required for both build-arch and
build-indep, e.g. debhelper, cdbs etc.  And it would mean that when
doing a binary-indep-only build, the buildd won't need to install
masses of -dev packages which won't be used, since they would be in
Build-Depends-Arch.

(Enabling Build-Depends-Arch/Build-Conflicts-Arch in sbuild is a
trivial change which could be done today.)  This would be nice
because there's then a 1:1 correspondence between build, build-arch,
build-indep and the Build-Depends, Build-Depends-Arch and
Build-Depends-Indep fields, respectively (and the same for conflicts).


Regards,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
   `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 15:15:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 15:15:03 GMT) Full text and rfc822 format available.

Message #139 received at 604397@bugs.debian.org (full text, mbox):

From: Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>
To: Roger Leigh <rleigh@debian.org>
Cc: 604397@bugs.debian.org, debian-ctte@lists.debian.org, Steve Langasek <vorlon@debian.org>
Subject: Re: Bug#604397: Request for TC to rule on a course of action for supporting build-arch
Date: Mon, 6 Jun 2011 17:10:48 +0200
On Mon, Jun 06, 2011 at 03:37:07PM +0100, Roger Leigh wrote:
> On Mon, Jun 06, 2011 at 03:59:15PM +0200, Bill Allombert wrote:
> > On Mon, Jun 06, 2011 at 12:09:34PM +0200, Bill Allombert wrote:
> > > On Mon, Jun 06, 2011 at 02:15:37AM -0700, Steve Langasek wrote:
> > > >  1) Implement support for calling 'debian/rules build-arch' in place of
> > > >     'debian/rules build' by checking for the presence of the target using
> > > >     'make -qn'.[1]
> > > > 
> > > >  2) Implement support for calling 'debian/rules build-arch' with a fallback
> > > >     to 'debian/rules build' by checking whether the output of the build-arch
> > > >     target matches that of a dummy target.[2]
> > > > 
> > > >  3) Implement support for calling 'debian/rules build-arch' in place of
> > > >     'debian/rules build' if a Build-Options field is set in debian/control
> > > >     of the source package specifying that this target is supported.[3]
> > > 
> > > For the record, I have been working to the patch that implement 3) to get accepted,
> > > see bug #229357.
> > 
> > Still for the record this patch has been accepted in dpkg as of today:
> >     http://git.debian.org/?p=dpkg/dpkg.git;a=commitdiff;h=14d48ef
> > Does that make this bug moot for all concerned ?
> > 
> > I plan to repropose #218893 with updated wording to match this dpkg change.
> 
> This is great, thanks for doing this!

Thanks for your support. I have been fighting for ten years to get that issue 
fixed.

> The question to consider here is whether or not this particular build
> feature is "transitional" or if it will remain indefinitely.
> 
> Needing to explicitly enable a target that should be the default is
> something I'm not that happy with.  While it's fine to do this as a
> means of transitioning with minimal breakage, and I'm totally OK with
> this as a means of getting the transition done, I would prefer it not
> be a requirement in perpetuity.  Once the archive has sufficient
> coverage to make mandating the targets a realistic possibility, I
> would be all for making it mandatory.

If you read the original discussion in bug #218893, a lot of developers were
unhappy with having to add a build-arch target to packages that only build a
single kind of package (all arch-all or all arch-indep) or which did not have a
Build-Depends-Indep field, because this was useless, so this leads to the
proposal for an optional build feature. The dpkg developers at the time were 
also very much concerned with preserving backward compatibility. So this is 
a compromise solution. As such I would suggest we deploy it and then we can
discuss whether there is a consensus to make build-arch mandatory.

> We don't currently have an option to build /only/ binary-indep
> packages, but this might be useful for arch-all autobuilding if we
> don't want to build them at the same time as for binary-arch.
> In this case, would we still want to install Build-Depends and
> Build-Depends-Indep?  (I would think so.)

There is such option now: dpkg-buildpackage -A.

Cheers,
-- 
Bill. <ballombe@debian.org>

Imagine a large red swirl here. 




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 15:27:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Raphael Hertzog <hertzog@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 15:27:03 GMT) Full text and rfc822 format available.

Message #144 received at 604397@bugs.debian.org (full text, mbox):

From: Raphael Hertzog <hertzog@debian.org>
To: Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>, 604397@bugs.debian.org
Cc: debian-ctte@lists.debian.org, Steve Langasek <vorlon@debian.org>, Roger Leigh <rleigh@debian.org>
Subject: Re: Bug#604397: Request for TC to rule on a course of action for supporting build-arch
Date: Mon, 6 Jun 2011 17:22:22 +0200
Hi,

On Mon, 06 Jun 2011, Roger Leigh wrote:
> Has the following been considered:
> - adding a command-line option for dpkg-buildpackage to explicitly
>   enable particular build-features (overriding the feature in the
>   source package).

This has not been suggested yet, I'm not opposed to the idea and would
probably merge a patch for it.

> The question to consider here is whether or not this particular build
> feature is "transitional" or if it will remain indefinitely.

That really depends on the outcome of the policy process and/or the
tech-ctte decision.

I'm not opposed to anything although I do believe that the split
between build-arch and build-indep is only useful for a minority of
packages.

> We don't currently have an option to build /only/ binary-indep
> packages, but this might be useful for arch-all autobuilding if we
> don't want to build them at the same time as for binary-arch.
> In this case, would we still want to install Build-Depends and
> Build-Depends-Indep?  (I would think so.)

Yes, Build-Depends is required for the clean target.

> We don't currently have a Build-Depends-Arch and Build-Conflicts-Arch;
> but this might be useful.  This would allow Build-Depends to be used
> for "generic" dependencies required for both build-arch and
> build-indep, e.g. debhelper, cdbs etc.  And it would mean that when
> doing a binary-indep-only build, the buildd won't need to install
> masses of -dev packages which won't be used, since they would be in
> Build-Depends-Arch.

I don't think that we need to optimize the build process for the
arch-all buildd.

It's not a big deal if we install too many packages... the added
complexity on the other hand is worrying. But it would probably be cleaner
and maybe clearer (the lack of symetry between Build-Depends and
Build-Depends-Indep has confused many people).

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
                      ▶ http://RaphaelHertzog.fr (Français)




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 15:51:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Guillem Jover <guillem@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 15:51:03 GMT) Full text and rfc822 format available.

Message #149 received at 604397@bugs.debian.org (full text, mbox):

From: Guillem Jover <guillem@debian.org>
To: 604397@bugs.debian.org
Subject: Re: Bug#604397: debian-policy: require build-arch and build-indep targets
Date: Mon, 6 Jun 2011 17:48:26 +0200
On Mon, 2011-06-06 at 12:29:20 +0200, Jakub Wilk wrote:
> * Guillem Jover <guillem@debian.org>, 2011-06-06, 09:55:
> >I'd even go further and combine that with dpkg-buildpackage
> >stopping to set compilation flags on the environment, so we only
> >have to deal once with possible mass FTBFS on the archive.
> 
> I don't think this is a good idea. To enable build-arch, we'll
> probably need quite a few NMUing volunteers, and it IME is very
> demotivating when you have to fix a seemingly simple bug (e.g.
> missing build-arch target) only to discover tha the package would
> mysteriously FTBFS anyway (e.g. because it relied on *FLAGS exported
> by dpkg-buildpackage).

We could certainly do it in another slot, if it feels like going to
be more painful than needed.

> Just to provide some statistics:
> - There are 16491 source packages in unstable[0].
> - Out of these, 2206 source package build both arch:all and arch:any
> binaries[1].
> - Out of these, 214 already FTBFS in unstable[2].
> - Out of remaining 1992 packages:
>   + 486 use so called "tiny" debhelper rules.
>   + 664 use cdbs.
>   + At least 175 defines both build-arch and build-indep explicitly.
> 
> So that leaves us with at most 667 packages to fix (plus an unknown
> number of packages that define build-{arch,indep}, but do so
> incorrectly). This is still a big number, we'd roughly double the
> number of currently observed FTBFS bugs if we announced flag day
> *today*.

I think those could be reduced even more by splitting into two stages,
by first only activating build-arch for packages w/ Build-Depends-Indep,
and then the ones w/o.

The second round would benefit only from possible split building, the
first from reduced build dependencies.

regards,
guillem




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 16:03:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Guillem Jover <guillem@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 16:03:03 GMT) Full text and rfc822 format available.

Message #154 received at 604397@bugs.debian.org (full text, mbox):

From: Guillem Jover <guillem@debian.org>
To: Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>, 604397@bugs.debian.org
Subject: Re: Bug#604397: debian-policy: require build-arch and build-indep targets
Date: Mon, 6 Jun 2011 18:01:10 +0200
On Mon, 2011-06-06 at 13:37:22 +0200, Bill Allombert wrote:
> On Mon, Jun 06, 2011 at 09:55:25AM +0200, Guillem Jover wrote:
> > > To help no existing packages today but make it easy for packages
> > > to opt in (and not break the others):
> > > 
> > >  1. Introduce a Build-Options facility for packages to advertise
> > >     capabilities like this.
> > >  2. Teach dpkg-buildpackage to honor it.
> > 
> > I don't think Build-Options/Build-Features is a good idea, as stated
> > previously on these bug reports. For example most of my packages
> > already support build-arch/build-indep, now I have to also state that
> > explicitly? Buh. :)
> 
> Only for packages that would benefit from it, i.e. packages that generates
> both arch-indep and arch-all packages and have large Build-Depends-Indep
> dependencies. 

On my previous mail I already listed the only cases where it really
makes sense, and dpkg-dev already has the information to decide when
that's the case, and it could activate build-arch/build-indep
unconditionally on these cases.

> This stays an optional feature. No need to add it everywhere.

And to me that's one of the problems with Build-Options/Features,
another being the duplicated information. If we consider
build-arch/build-indep something useful enough to be widely usable
on all packages producing arch:any + arch:all packages, then making
this optional is close to keeping status quo, I expect a multi-year
period to make a dent on packages adding support for this, if at all.
(There's still packages using Source-Version substvar and a substantial
difference is that's known to break binNMUs, it was introduced around
2007...)

If this is considered only useful to a handful of packages, then
really, what's the point of all the discussions? All that energy
could have been used in NMUs instead.

regards,
guillem




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 16:15:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Guillem Jover <guillem@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 16:15:03 GMT) Full text and rfc822 format available.

Message #159 received at 604397@bugs.debian.org (full text, mbox):

From: Guillem Jover <guillem@debian.org>
To: Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>, 604397@bugs.debian.org
Cc: debian-ctte@lists.debian.org, Steve Langasek <vorlon@debian.org>, Roger Leigh <rleigh@debian.org>
Subject: Re: Bug#604397: Request for TC to rule on a course of action for supporting build-arch
Date: Mon, 6 Jun 2011 18:10:57 +0200
On Mon, 2011-06-06 at 15:59:15 +0200, Bill Allombert wrote:
> On Mon, Jun 06, 2011 at 12:09:34PM +0200, Bill Allombert wrote:
> > On Mon, Jun 06, 2011 at 02:15:37AM -0700, Steve Langasek wrote:
> > >  1) Implement support for calling 'debian/rules build-arch' in place of
> > >     'debian/rules build' by checking for the presence of the target using
> > >     'make -qn'.[1]
> > > 
> > >  2) Implement support for calling 'debian/rules build-arch' with a fallback
> > >     to 'debian/rules build' by checking whether the output of the build-arch
> > >     target matches that of a dummy target.[2]
> > > 
> > >  3) Implement support for calling 'debian/rules build-arch' in place of
> > >     'debian/rules build' if a Build-Options field is set in debian/control
> > >     of the source package specifying that this target is supported.[3]
> > 
> > For the record, I have been working to the patch that implement 3) to get accepted,
> > see bug #229357.
> 
> Still for the record this patch has been accepted in dpkg as of today:
>     http://git.debian.org/?p=dpkg/dpkg.git;a=commitdiff;h=14d48ef
> Does that make this bug moot for all concerned ?

FWIW, I don't agree with that change. If we'd decide to go with
non-mandatory support, I'd rather use one of the autodetection
options, preferably the “make -qn” one which would allow us to warn
the user for the cases where it makes sense to add the targets.

regards,
guillem




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 17:15:09 GMT) Full text and rfc822 format available.

Acknowledgement sent to Russ Allbery <rra@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 17:15:09 GMT) Full text and rfc822 format available.

Message #164 received at 604397@bugs.debian.org (full text, mbox):

From: Russ Allbery <rra@debian.org>
To: 604397@bugs.debian.org
Cc: debian-ctte@lists.debian.org
Subject: Re: Request for TC to rule on a course of action for supporting build-arch
Date: Mon, 06 Jun 2011 10:12:13 -0700
Steve Langasek <vorlon@debian.org> writes:

> The Technical Committee has sufficient authority to address this
> question under any of §6.1.{1,2,4,5}.  If you prefer, we could also ask
> for a referral from the policy editors or the dpkg maintainers, to
> eliminate any question of supermajority requirements.

I'm happy to provide a referral from Policy.  I think resolving this in
the tech-ctte is a great idea.

> If this were to be put to a vote today, I would propose the following
> ballot options:

>  1) Implement support for calling 'debian/rules build-arch' in place of
>     'debian/rules build' by checking for the presence of the target using
>     'make -qn'.[1]

>  2) Implement support for calling 'debian/rules build-arch' with a fallback
>     to 'debian/rules build' by checking whether the output of the build-arch
>     target matches that of a dummy target.[2]

>  3) Implement support for calling 'debian/rules build-arch' in place of
>     'debian/rules build' if a Build-Options field is set in debian/control
>     of the source package specifying that this target is supported.[3]

>  4) Turn on direct use of 'debian/rules build-arch' on the autobuilders for
>     all packages in unstable and experimental immediately, with no fallback
>     if the target does not exist; requires a corresponding update to Policy
>     and mass updates to fix packages that fail to build as a result.

>  5) Further Discussion

Those look like a good set of solutions to vote on to me.

> My own vote would likely be: 1, 2, 3, 5, 4.  (I could be persuaded to
> rank 4 above FD if this were the only way to move forward; but that's
> indisputably the most disruptive to the archive, so I would hope we
> could reach agreement that some or all of the other options are better.)

So that people know, my vote would probably be something like:

    2, 4, 1, 3, 5

I'm worried that make -qn is going to be too fragile.  That method has
been tried before in Lintian checks IIRC and didn't work well.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 19:12:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Bdale Garbee <bdale@gag.com>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 19:12:03 GMT) Full text and rfc822 format available.

Message #169 received at 604397@bugs.debian.org (full text, mbox):

From: Bdale Garbee <bdale@gag.com>
To: Steve Langasek <vorlon@debian.org>, 604397@bugs.debian.org, debian-ctte@lists.debian.org
Subject: Re: Request for TC to rule on a course of action for supporting build-arch
Date: Mon, 06 Jun 2011 12:58:42 -0600
[Message part 1 (text/plain, inline)]
On Mon, 6 Jun 2011 02:15:37 -0700, Steve Langasek <vorlon@debian.org> wrote:
> If this were to be put to a vote today, I would propose the following ballot
> options:
> 
>  1) Implement support for calling 'debian/rules build-arch' in place of
>     'debian/rules build' by checking for the presence of the target using
>     'make -qn'.[1]
> 
>  2) Implement support for calling 'debian/rules build-arch' with a fallback
>     to 'debian/rules build' by checking whether the output of the build-arch
>     target matches that of a dummy target.[2]
> 
>  3) Implement support for calling 'debian/rules build-arch' in place of
>     'debian/rules build' if a Build-Options field is set in debian/control
>     of the source package specifying that this target is supported.[3]
> 
>  4) Turn on direct use of 'debian/rules build-arch' on the autobuilders for
>     all packages in unstable and experimental immediately, with no fallback
>     if the target does not exist; requires a corresponding update to Policy
>     and mass updates to fix packages that fail to build as a result.
> 
>  5) Further Discussion

Steve and I discussed this on IRC for a while in advance of his posting
the email here, and I'm supportive of the TC voting on this to help
establish the plan for our next stable release.

FWIW, if voting today I'd vote 12453.

Bdale
[Message part 2 (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 19:21:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Raphael Hertzog <hertzog@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 19:21:03 GMT) Full text and rfc822 format available.

Message #174 received at 604397@bugs.debian.org (full text, mbox):

From: Raphael Hertzog <hertzog@debian.org>
To: Guillem Jover <guillem@debian.org>, 604397@bugs.debian.org
Cc: Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>
Subject: Re: Bug#604397: debian-policy: require build-arch and build-indep targets
Date: Mon, 6 Jun 2011 21:15:41 +0200
Hi,

On Mon, 06 Jun 2011, Guillem Jover wrote:
> And to me that's one of the problems with Build-Options/Features,
> another being the duplicated information. If we consider
> build-arch/build-indep something useful enough to be widely usable
> on all packages producing arch:any + arch:all packages, then making
> this optional is close to keeping status quo, I expect a multi-year
> period to make a dent on packages adding support for this, if at all.

That's true but so is it for any new feature unfortunately. And even with
a flag day, once you have fixed the FTBFS, you're far from having benefits
from that separation. Because most of the packages that do not FTBFS are
still not converted to make usage of it. They would still run the same
build process in both cases.

> If this is considered only useful to a handful of packages, then
> really, what's the point of all the discussions? All that energy
> could have been used in NMUs instead.

How so? If dpkg-buildpackage only calls "build" I don't see what you could
have NMUed.

Except turning build into an empty target and adding
build-arch/build-indep as dependencies of binary-arch/binary-indep
but that would not be nice since it means compilation is run
as root when it's usually not.

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
                      ▶ http://RaphaelHertzog.fr (Français)




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 20:09:06 GMT) Full text and rfc822 format available.

Acknowledgement sent to Russ Allbery <rra@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 20:09:06 GMT) Full text and rfc822 format available.

Message #179 received at 604397@bugs.debian.org (full text, mbox):

From: Russ Allbery <rra@debian.org>
To: debian-ctte@lists.debian.org, 604397@bugs.debian.org
Subject: Re: Request for TC to rule on a course of action for supporting build-arch
Date: Mon, 06 Jun 2011 13:04:59 -0700
Andreas Barth <aba@not.so.argh.org> writes:

> Option 1 also implies forcing debian/rules to be a Makefile, which is
> think is sensible.

Policy already requires this.  The only package in the archive for which
this is not already the case is "leave".

I don't like option #3 because it's something we'll be stuck with forever
and requires packagers update both debian/rules and debian/control to
configure things properly.  One of the reasons why I'm personally fond of
#4 is that it reduces our long-term complexity.  #3 increases our
long-term complexity, I think unnecessarily.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 20:12:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Steve Langasek <vorlon@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 20:12:03 GMT) Full text and rfc822 format available.

Message #184 received at 604397@bugs.debian.org (full text, mbox):

From: Steve Langasek <vorlon@debian.org>
To: Roger Leigh <rleigh@codelibre.net>, 604397@bugs.debian.org
Subject: Re: Bug#604397: debian-policy: require build-arch and build-indep targets
Date: Mon, 6 Jun 2011 13:08:46 -0700
[Message part 1 (text/plain, inline)]
On Mon, Jun 06, 2011 at 02:33:22PM +0100, Roger Leigh wrote:
> While neither of these changes actively "enforces" the addition of
> these targets, neither do any harm, and by actively encouraging
> adoption of the targets it will reduce the total number of NMUs
> required should we go for the approach of a straight change in
> dpkg-buildpackage.

> Neither of these dictate /how/ the transition should proceed, which is
> a separate issue which you brought to the CTTE, but both will be needed
> irrespective of the choice picked by the technical committee.

I don't think that's true at all.  If we go with one of the autodetecting
implementations, there's no need for a policy "should"; we can go straight
to a list of policy "musts" for the required interactions between
Build-Depends and build-arch.  Recommending that maintainers *take
advantage* of the possibility to properly split Build-Depends and
Build-Depends-Indep is not really very interesting to me, once that's done.

> > And if we want to provide a smooth transition instead, using something like
> > Joey's proposed make-first-existing-target interface in bug #598534 (as
> > discussed on debian-devel in
> > http://lists.debian.org/debian-devel/2010/09/msg00704.html), we don't need
> > this to be a policy "should" or "must" at all, because the autobuilders can
> > then DTRT for any package whether or not it implements the build-arch target
> > and the presence of the target merely lets us optimize build times and
> > reduce build dependencies - so it could remain a policy "may" indefinitely
> > (with some tightening of the language about how not having build-arch
> > requires different bits to be in Build-Depends than if you do have it).

> I would see make-first-existing-target as a good method for easing the
> transition, and we could leave it a "may" indefinitely.  But would it
> not be better (long-term) to aim for complete transition to requiring
> build-arch/build-indep for all packages (as for binary-arch/
> binary-indep) and use make-first-existing-target as a useful strategy
> for doing the transition, but not as a long-term feature?  Especially
> since it's a bit of a hack.

Hack or not, once implemented I expect any automatic fallback handling to be
with us for a while.  I think it's realistic to have build-arch supported
for wheezy, but I'm not sure we would want to make it mandatory even in
wheezy+1, because the last packages to implement it are going to be those
that also get no benefit from it.  So this comes down to Policy (and the
buildds) not making a large number of packages instantly buggy without good
reason, and I think it's premature to worry about sunsetting
make-first-existing-target before we've even started to use it.

(Incidentally, if the consensus is that make-first-existing-target is a "bit
of a hack", then I agree with Guillem that it shouldn't be put in the make
package at all, just in dpkg-buildpackage.)

> > Unfortunately I see the same problem with this lintian check as with all the
> > rest - if we can actually check for the existence of the target *reliably*,
> > then we don't need to enforce its presence at all.

> It isn't 100% reliable (it can't be given pattern rules and includes); it
> gives up for dh/cdbs-using packages, but both of these handily already
> support the missing targets, so it's reliable enough to be useful.  If
> the maintainer chose to use odd pattern rules, these could potentially
> cause a false positive.

Have you tried to quantify these false positives?  IME where maintainers
have resisted adopting either dh or cdbs, it's usually *because* they have
their own complex build systems, with pattern rules and includes, that are
not straightforward to convert (if they're willing to at all).

> > > Is there any recent work on the rules checking in make which would allow
> > > dpkg-buildpackage to use the rules if present, but fall back to build
> > > if absent?  This would be the most pragmatic approach, because it will
> > > both provide backwards compatibility with all older source packages, and
> > > use the rules if present in new ones.

> > The patch is outstanding; the make maintainer is TTBOMK currently
> > unavailable for Debian work.  If there's a consensus on
> > debian-policy/devel/ctte that we should go the make-first-existing-target
> > route, I would be more than happy to do an NMU of make to facilitate this.

> As a medium-term solution it would be great if GNU make could itself
> support querying whether or not a makefile supports a given target.
> Since make needs to parse all the includes etc., only make itself can
> do this 100% reliably.  If it's not already done, a feature request
> upstream would be good.

'make -qn' is already an adequate facility for this.  The only disadvantage
to adopting it for build-arch is that not all packages use a
policy-compliant makefile for debian/rules.

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek@ubuntu.com                                     vorlon@debian.org
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 20:21:06 GMT) Full text and rfc822 format available.

Acknowledgement sent to Steve Langasek <vorlon@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 20:21:06 GMT) Full text and rfc822 format available.

Message #189 received at 604397@bugs.debian.org (full text, mbox):

From: Steve Langasek <vorlon@debian.org>
To: Andreas Barth <aba@not.so.argh.org>, debian-ctte@lists.debian.org, 604397@bugs.debian.org
Subject: Re: Request for TC to rule on a course of action for supporting build-arch
Date: Mon, 6 Jun 2011 13:20:00 -0700
[Message part 1 (text/plain, inline)]
On Mon, Jun 06, 2011 at 09:56:22PM +0200, Andreas Barth wrote:
> > >  1) Implement support for calling 'debian/rules build-arch' in place of
> > >     'debian/rules build' by checking for the presence of the target using
> > >     'make -qn'.[1]

> Option 1 also implies forcing debian/rules to be a Makefile, which is
> think is sensible.

Well, it only implies forcing debian/rules to be a makefile *if the package
wants to benefit from build-arch*.

$ make -f config.sub  -qn build-arch; echo $?
config.sub:61: *** missing separator.  Stop.
2
$

That's sufficient to let dpkg-buildpackage conclude "build-arch is not
supported", falling back to debian/rules build, without stirring up the old
arguments about whether we want to keep Policy 4.9.

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek@ubuntu.com                                     vorlon@debian.org
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 20:27:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Andreas Barth <aba@not.so.argh.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 20:27:03 GMT) Full text and rfc822 format available.

Message #194 received at 604397@bugs.debian.org (full text, mbox):

From: Andreas Barth <aba@not.so.argh.org>
To: debian-ctte@lists.debian.org, 604397@bugs.debian.org
Subject: Re: Request for TC to rule on a course of action for supporting build-arch
Date: Mon, 6 Jun 2011 21:56:22 +0200
* Bdale Garbee (bdale@gag.com) [110606 20:59]:
> On Mon, 6 Jun 2011 02:15:37 -0700, Steve Langasek <vorlon@debian.org> wrote:
> > If this were to be put to a vote today, I would propose the following ballot
> > options:
> > 
> >  1) Implement support for calling 'debian/rules build-arch' in place of
> >     'debian/rules build' by checking for the presence of the target using
> >     'make -qn'.[1]
> > 
> >  2) Implement support for calling 'debian/rules build-arch' with a fallback
> >     to 'debian/rules build' by checking whether the output of the build-arch
> >     target matches that of a dummy target.[2]
> > 
> >  3) Implement support for calling 'debian/rules build-arch' in place of
> >     'debian/rules build' if a Build-Options field is set in debian/control
> >     of the source package specifying that this target is supported.[3]
> > 
> >  4) Turn on direct use of 'debian/rules build-arch' on the autobuilders for
> >     all packages in unstable and experimental immediately, with no fallback
> >     if the target does not exist; requires a corresponding update to Policy
> >     and mass updates to fix packages that fail to build as a result.
> > 
> >  5) Further Discussion
> 
> Steve and I discussed this on IRC for a while in advance of his posting
> the email here, and I'm supportive of the TC voting on this to help
> establish the plan for our next stable release.
> 
> FWIW, if voting today I'd vote 12453.

Why 3 below 5?

Option 1 also implies forcing debian/rules to be a Makefile, which is
think is sensible.

My vote as of now would be something along 1254 (and unsure where to
place 3, between 2 and 5, or 5 and 4).


Andi




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 22:06:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Bdale Garbee <bdale@gag.com>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 22:06:03 GMT) Full text and rfc822 format available.

Message #199 received at 604397@bugs.debian.org (full text, mbox):

From: Bdale Garbee <bdale@gag.com>
To: Andreas Barth <aba@not.so.argh.org>, debian-ctte@lists.debian.org, 604397@bugs.debian.org
Subject: Re: Request for TC to rule on a course of action for supporting build-arch
Date: Mon, 06 Jun 2011 16:03:40 -0600
[Message part 1 (text/plain, inline)]
On Mon, 6 Jun 2011 21:56:22 +0200, Andreas Barth <aba@not.so.argh.org> wrote:
> Why 3 below 5?

Introducing a new field that must be filled in and kept (manually) in
sync with information that is already present in the rules file just
doesn't seem like a good solution.

I'm less afraid of 4 than some people would be, particularly with some
time to go before we'd think about freezing our next stable release.
But 1 and 2 both seem plausible to me and would be much less disruptive.

> Option 1 also implies forcing debian/rules to be a Makefile, which is
> think is sensible.

Right.  This has always been part of Debian policy, and I have not been
swayed by those who have tried to argue against it at various times.

Bdale
[Message part 2 (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 22:09:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 22:09:03 GMT) Full text and rfc822 format available.

Message #204 received at 604397@bugs.debian.org (full text, mbox):

From: Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>
To: Bdale Garbee <bdale@gag.com>, 604397@bugs.debian.org
Cc: Steve Langasek <vorlon@debian.org>, debian-ctte@lists.debian.org
Subject: Re: Bug#604397: Request for TC to rule on a course of action for supporting build-arch
Date: Tue, 7 Jun 2011 00:06:18 +0200
[Message part 1 (text/plain, inline)]
On Mon, 6 Jun 2011 02:15:37 -0700, Steve Langasek <vorlon@debian.org> wrote:
> If this were to be put to a vote today, I would propose the following ballot
> options:
> 
>  1) Implement support for calling 'debian/rules build-arch' in place of
>     'debian/rules build' by checking for the presence of the target using
>     'make -qn'.[1]
> 
>  2) Implement support for calling 'debian/rules build-arch' with a fallback
>     to 'debian/rules build' by checking whether the output of the build-arch
>     target matches that of a dummy target.[2]
> 
>  3) Implement support for calling 'debian/rules build-arch' in place of
>     'debian/rules build' if a Build-Options field is set in debian/control
>     of the source package specifying that this target is supported.[3]

The implementation is done now.

>  4) Turn on direct use of 'debian/rules build-arch' on the autobuilders for
>     all packages in unstable and experimental immediately, with no fallback
>     if the target does not exist; requires a corresponding update to Policy
>     and mass updates to fix packages that fail to build as a result.

As one of the few developpers that actually put work toward the issue and not
merely write about it, I like to give some backgrounds

The proposal 3) (which is implemented in dpkg as of today) was devised
following a discussion in Debian policy bug #218893 as a compromise
solution that was agreeable to everyone, then a patch to dpkg was written (bug
#229357). For reasons beyond my control, the patch was actually merged only
today.

During that time, two attempts to use 'make' to guess whether build-arch existed were
made to dpkg and reverted because they did not work in the real world.
See dpkg version 1.10.15.

Proposal 3 is the safest approach, does not require a flag day and preserve backward 
compatibility. And we have it now.

(Please ensure that mails send to bugs assigned to the ctte can reach the ctte list, thanks).

Cheers,
-- 
Bill. <ballombe@debian.org>

Imagine a large red swirl here. 
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 22:15:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Russ Allbery <rra@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 22:15:03 GMT) Full text and rfc822 format available.

Message #209 received at 604397@bugs.debian.org (full text, mbox):

From: Russ Allbery <rra@debian.org>
To: Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>
Cc: 604397@bugs.debian.org, Bdale Garbee <bdale@gag.com>, Steve Langasek <vorlon@debian.org>, debian-ctte@lists.debian.org
Subject: Re: Bug#604397: Request for TC to rule on a course of action for supporting build-arch
Date: Mon, 06 Jun 2011 15:13:52 -0700
Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr> writes:

> The proposal 3) (which is implemented in dpkg as of today) was devised
> following a discussion in Debian policy bug #218893 as a compromise
> solution that was agreeable to everyone, then a patch to dpkg was
> written (bug #229357). For reasons beyond my control, the patch was
> actually merged only today.

Well... acceptable in the sense that it was better than arguing about it
forever, but I've never been particularly happy with it, for the reasons
that Bdale said.  I think we should be able to just do a flag day, but I
could well be too aggressive.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 22:39:08 GMT) Full text and rfc822 format available.

Acknowledgement sent to Roger Leigh <rleigh@codelibre.net>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 22:39:08 GMT) Full text and rfc822 format available.

Message #214 received at 604397@bugs.debian.org (full text, mbox):

From: Roger Leigh <rleigh@codelibre.net>
To: Raphael Hertzog <hertzog@debian.org>, 604397@bugs.debian.org
Cc: Guillem Jover <guillem@debian.org>, Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>
Subject: Re: Bug#604397: debian-policy: require build-arch and build-indep targets
Date: Mon, 6 Jun 2011 23:34:43 +0100
[Message part 1 (text/plain, inline)]
On Mon, Jun 06, 2011 at 09:15:41PM +0200, Raphael Hertzog wrote:
> Hi,
> 
> On Mon, 06 Jun 2011, Guillem Jover wrote:
> > And to me that's one of the problems with Build-Options/Features,
> > another being the duplicated information. If we consider
> > build-arch/build-indep something useful enough to be widely usable
> > on all packages producing arch:any + arch:all packages, then making
> > this optional is close to keeping status quo, I expect a multi-year
> > period to make a dent on packages adding support for this, if at all.
> 
> That's true but so is it for any new feature unfortunately. And even with
> a flag day, once you have fixed the FTBFS, you're far from having benefits
> from that separation. Because most of the packages that do not FTBFS are
> still not converted to make usage of it. They would still run the same
> build process in both cases.

One thought I had today was what will happen with packages using
either cdbs or dh.  Both of these provide build-arch and build-indep
rules, and as a result both can build using those targets today
(though individual packages may of course be broken if they did
things in the wrong rules).  However, each would require updating
individually to actually enable their use.  Autodetection here would
prevent the need for this.

Would it be possible to combine both autodetection /and/ build
features?  That is, enable if in build features or if autodetected.
This would provide convenience for developers if autodetection
works (which will for the vast majority of packages), but would
also permit manual enabling where it does not.  This would allow
the "make -qn" check to supplement Bill's work quite nicely, and
make for much less work for maintainers (less packages needing
NMUing with the build feature added).


Regards,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
   `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 23:04:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Russ Allbery <rra@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 23:04:02 GMT) Full text and rfc822 format available.

Message #219 received at 604397@bugs.debian.org (full text, mbox):

From: Russ Allbery <rra@debian.org>
To: Roger Leigh <rleigh@codelibre.net>
Cc: 604397@bugs.debian.org, Raphael Hertzog <hertzog@debian.org>, Guillem Jover <guillem@debian.org>, Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>
Subject: Re: Bug#604397: debian-policy: require build-arch and build-indep targets
Date: Mon, 06 Jun 2011 15:54:56 -0700
Roger Leigh <rleigh@codelibre.net> writes:

> Would it be possible to combine both autodetection /and/ build
> features?  That is, enable if in build features or if autodetected.
> This would provide convenience for developers if autodetection
> works (which will for the vast majority of packages), but would
> also permit manual enabling where it does not.

When would autodetection fail in that direction?  That's not the worrisome
issue for autodetection; the worrisome issue with autodetection is if it
thinks the rule is available but it actually isn't.

Rather than adding a build feature, I think package maintainers for whom
autodetection fails should just fix the issue that causes it to fail.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Mon, 06 Jun 2011 23:04:15 GMT) Full text and rfc822 format available.

Acknowledgement sent to Steve Langasek <vorlon@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Mon, 06 Jun 2011 23:04:15 GMT) Full text and rfc822 format available.

Message #224 received at 604397@bugs.debian.org (full text, mbox):

From: Steve Langasek <vorlon@debian.org>
To: 629385@bugs.debian.org, 604397@bugs.debian.org
Subject: Re: Bug#604397: Request for TC to rule on a course of action for supporting build-arch
Date: Mon, 6 Jun 2011 16:02:36 -0700
[Message part 1 (text/plain, inline)]
Hi Bill,

(Sending this to 629385@bugs.debian.org, which is the cloned bug actually
assigned to the TC...)

On Tue, Jun 07, 2011 at 12:06:18AM +0200, Bill Allombert wrote:
> The proposal 3) (which is implemented in dpkg as of today) was devised
> following a discussion in Debian policy bug #218893 as a compromise
> solution that was agreeable to everyone, then a patch to dpkg was written (bug
> #229357). For reasons beyond my control, the patch was actually merged only
> today.

I think your read that this was a "compromise solution that was agreeable to
everyone" is incorrect.  Scott James Remnant (a dpkg maintainer at the time)
raised in http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=218893#345 the
same objection about fragility that others have raised since.

I'm actually somewhat surprised that this has been merged at all into dpkg,
given that there clearly isn't agreement among the current dpkg maintainers
that this is the right way forward for build-arch.

> During that time, two attempts to use 'make' to guess whether build-arch
> existed were made to dpkg and reverted because they did not work in the
> real world.  See dpkg version 1.10.15.

Given that there is zero documentation of what went wrong with the previous
implementation, I give zero weight to that statement from the dpkg
maintainer of the time.  Aside from cases where debian/rules isn't actually
a makefile, I am aware of no potential problems with the patch proposed in
http://lists.debian.org/debian-devel/2007/07/msg00048.html (though I would
probably write that somewhat differently today).  If you can point to a
specific problem with that approach, I would certainly take that into
consideration, but unsubstantiated claims that an approach is "impossible"
are equivalent to FUD.

> Proposal 3 is the safest approach,

It is the most error-prone.

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek@ubuntu.com                                     vorlon@debian.org
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Tue, 07 Jun 2011 07:33:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Tue, 07 Jun 2011 07:33:03 GMT) Full text and rfc822 format available.

Message #229 received at 604397@bugs.debian.org (full text, mbox):

From: Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>
To: 629385@bugs.debian.org, 604397@bugs.debian.org
Cc: Steve Langasek <vorlon@debian.org>
Subject: Re: Bug#604397: Request for TC to rule on a course of action for supporting build-arch
Date: Tue, 7 Jun 2011 09:29:12 +0200
On Mon, Jun 06, 2011 at 04:02:36PM -0700, Steve Langasek wrote:
> Hi Bill,
> 
> (Sending this to 629385@bugs.debian.org, which is the cloned bug actually
> assigned to the TC...)
> 
> On Tue, Jun 07, 2011 at 12:06:18AM +0200, Bill Allombert wrote:
> > The proposal 3) (which is implemented in dpkg as of today) was devised
> > following a discussion in Debian policy bug #218893 as a compromise
> > solution that was agreeable to everyone, then a patch to dpkg was written (bug
> > #229357). For reasons beyond my control, the patch was actually merged only
> > today.
> 
> I think your read that this was a "compromise solution that was agreeable to
> everyone" is incorrect.  Scott James Remnant (a dpkg maintainer at the time)
> raised in http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=218893#345 the
> same objection about fragility that others have raised since.

Scott did not take part in the initial discussion, so there was no way his objection
would be taken into account.

Cheers,
-- 
Bill. <ballombe@debian.org>

Imagine a large red swirl here. 




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Tue, 07 Jun 2011 07:45:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Tue, 07 Jun 2011 07:45:04 GMT) Full text and rfc822 format available.

Message #234 received at 604397@bugs.debian.org (full text, mbox):

From: Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>
To: Roger Leigh <rleigh@codelibre.net>
Cc: Raphael Hertzog <hertzog@debian.org>, 604397@bugs.debian.org, Guillem Jover <guillem@debian.org>
Subject: Re: Bug#604397: debian-policy: require build-arch and build-indep targets
Date: Tue, 7 Jun 2011 09:39:59 +0200
On Mon, Jun 06, 2011 at 11:34:43PM +0100, Roger Leigh wrote:
> On Mon, Jun 06, 2011 at 09:15:41PM +0200, Raphael Hertzog wrote:
> > Hi,
> > 
> > That's true but so is it for any new feature unfortunately. And even with
> > a flag day, once you have fixed the FTBFS, you're far from having benefits
> > from that separation. Because most of the packages that do not FTBFS are
> > still not converted to make usage of it. They would still run the same
> > build process in both cases.
> 
> One thought I had today was what will happen with packages using
> either cdbs or dh.  Both of these provide build-arch and build-indep
> rules, and as a result both can build using those targets today
> (though individual packages may of course be broken if they did
> things in the wrong rules).  However, each would require updating
> individually to actually enable their use.  Autodetection here would
> prevent the need for this.

It would not. Currently autobuilder always only install Build-Depends, so
'debian/rules build' has to work with only Build-Depends installed.

This means there has basically two cases for package having build-arch/build-indep:
a) Packages do not have Build-Depends-Indep.
b) Packages have Build-Depends-Indep but the documentation is built in binary-indep 
instead of build-indep.

To get any advantage from this proposal, packages a) need to be changed to have 
a proper Build-Depends-Indep line. Packages b) already provides the split anyway.

Cheers,
-- 
Bill. <ballombe@debian.org>

Imagine a large red swirl here. 




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Tue, 07 Jun 2011 08:06:08 GMT) Full text and rfc822 format available.

Acknowledgement sent to Steve Langasek <vorlon@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Tue, 07 Jun 2011 08:06:11 GMT) Full text and rfc822 format available.

Message #239 received at 604397@bugs.debian.org (full text, mbox):

From: Steve Langasek <vorlon@debian.org>
To: Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>, 604397@bugs.debian.org
Cc: Roger Leigh <rleigh@codelibre.net>, Raphael Hertzog <hertzog@debian.org>, Guillem Jover <guillem@debian.org>
Subject: Re: Bug#604397: debian-policy: require build-arch and build-indep targets
Date: Tue, 7 Jun 2011 01:02:04 -0700
[Message part 1 (text/plain, inline)]
On Tue, Jun 07, 2011 at 09:39:59AM +0200, Bill Allombert wrote:
> > One thought I had today was what will happen with packages using
> > either cdbs or dh.  Both of these provide build-arch and build-indep
> > rules, and as a result both can build using those targets today
> > (though individual packages may of course be broken if they did
> > things in the wrong rules).  However, each would require updating
> > individually to actually enable their use.  Autodetection here would
> > prevent the need for this.

> It would not. Currently autobuilder always only install Build-Depends, so
> 'debian/rules build' has to work with only Build-Depends installed.

> This means there has basically two cases for package having build-arch/build-indep:
> a) Packages do not have Build-Depends-Indep.
> b) Packages have Build-Depends-Indep but the documentation is built in binary-indep 
> instead of build-indep.

> To get any advantage from this proposal, packages a) need to be changed to
> have a proper Build-Depends-Indep line.  Packages b) already provides the
> split anyway.

There are two benefits of this intended change:  avoiding the downloading
and installation of heavy build-dependencies only needed for generation of
architecture-independent packages; and avoiding the build-time generation of
those architecture-independent packages.  Existing packages that have
properly structured their debian/rules targets already for the second case
would get immediate benefit from autodetection, with no further changes.

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek@ubuntu.com                                     vorlon@debian.org
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Tue, 07 Jun 2011 08:57:26 GMT) Full text and rfc822 format available.

Acknowledgement sent to Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Tue, 07 Jun 2011 08:57:33 GMT) Full text and rfc822 format available.

Message #244 received at 604397@bugs.debian.org (full text, mbox):

From: Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>
To: Steve Langasek <vorlon@debian.org>, 604397@bugs.debian.org
Cc: Roger Leigh <rleigh@codelibre.net>, Raphael Hertzog <hertzog@debian.org>, Guillem Jover <guillem@debian.org>
Subject: Re: Bug#604397: debian-policy: require build-arch and build-indep targets
Date: Tue, 7 Jun 2011 10:54:33 +0200
On Tue, Jun 07, 2011 at 01:02:04AM -0700, Steve Langasek wrote:
> On Tue, Jun 07, 2011 at 09:39:59AM +0200, Bill Allombert wrote:
> > > One thought I had today was what will happen with packages using
> > > either cdbs or dh.  Both of these provide build-arch and build-indep
> > > rules, and as a result both can build using those targets today
> > > (though individual packages may of course be broken if they did
> > > things in the wrong rules).  However, each would require updating
> > > individually to actually enable their use.  Autodetection here would
> > > prevent the need for this.
> 
> > It would not. Currently autobuilder always only install Build-Depends, so
> > 'debian/rules build' has to work with only Build-Depends installed.
> 
> > This means there has basically two cases for package having build-arch/build-indep:
> > a) Packages do not have Build-Depends-Indep.
> > b) Packages have Build-Depends-Indep but the documentation is built in binary-indep 
> > instead of build-indep.
> 
> > To get any advantage from this proposal, packages a) need to be changed to
> > have a proper Build-Depends-Indep line.  Packages b) already provides the
> > split anyway.
> 
> There are two benefits of this intended change:  avoiding the downloading
> and installation of heavy build-dependencies only needed for generation of
> architecture-independent packages; and avoiding the build-time generation of
> those architecture-independent packages.  Existing packages that have
> properly structured their debian/rules targets already for the second case
> would get immediate benefit from autodetection, with no further changes.

How ? Can you give an exemple ? autobuilders already do not install Build-Depends-Indep.

Cheers,
-- 
Bill. <ballombe@debian.org>

Imagine a large red swirl here. 




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Tue, 07 Jun 2011 09:15:11 GMT) Full text and rfc822 format available.

Acknowledgement sent to Tollef Fog Heen <tfheen@err.no>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Tue, 07 Jun 2011 09:15:16 GMT) Full text and rfc822 format available.

Message #249 received at 604397@bugs.debian.org (full text, mbox):

From: Tollef Fog Heen <tfheen@err.no>
To: 604397@bugs.debian.org
Cc: debian-ctte@lists.debian.org, rleigh@debian.org
Subject: Re: Request for TC to rule on a course of action for supporting build-arch
Date: Tue, 07 Jun 2011 11:14:14 +0200
]] Steve Langasek 

Hi,

|  4) Turn on direct use of 'debian/rules build-arch' on the autobuilders for
|     all packages in unstable and experimental immediately, with no fallback
|     if the target does not exist; requires a corresponding update to Policy
|     and mass updates to fix packages that fail to build as a result.

I'd be happy to provide hardware to do a full scale rebuild test of the
archive if somebody does the actual work of doing the rebuilds.  rleigh
did it for his sbuild resolver test so I've Cc-ed him to see if he's
interested in doing a test for this too.

Regards,
-- 
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Thu, 09 Jun 2011 22:24:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Thu, 09 Jun 2011 22:24:06 GMT) Full text and rfc822 format available.

Message #254 received at 604397@bugs.debian.org (full text, mbox):

From: Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>
To: Russ Allbery <rra@debian.org>, 604397@bugs.debian.org
Cc: debian-ctte@lists.debian.org, 629385@bugs.debian.org
Subject: Re: Bug#604397: Request for TC to rule on a course of action for supporting build-arch
Date: Fri, 10 Jun 2011 00:21:24 +0200
On Mon, Jun 06, 2011 at 01:04:59PM -0700, Russ Allbery wrote:
> Andreas Barth <aba@not.so.argh.org> writes:
> 
> > Option 1 also implies forcing debian/rules to be a Makefile, which is
> > think is sensible.
> 
> Policy already requires this.  The only package in the archive for which
> this is not already the case is "leave".
> 
> I don't like option #3 because it's something we'll be stuck with forever
> and requires packagers update both debian/rules and debian/control to
> configure things properly.  One of the reasons why I'm personally fond of
> #4 is that it reduces our long-term complexity.  #3 increases our
> long-term complexity, I think unnecessarily.

I have proposed an alternative in the past (which did not get any support, though):
Decide that packages that have a Build-Depends-Indep: field must implement
build-arch/build-indep. This is probably already the case.

This has the same advantage than Build-Options: a program can check if dpkg-buildpackage
will call build-arch just by looking at the control file.

There should be a well-defined interface to know whether dpkg-buildpackage will
call build-arch or build, that does not depend on a specific dpkg-buildpackage 
implementation. I am afraid that any makefile trickery would not be stable enough
for that purpose.

Concerning the long-term complexity: If you look at the bug log, you will see
that developpers objected that the build-arch/build-indep split was unnecessary
complexity for packages that would not get any benefit of it, so Build-Options was
proposed which allowed the split to be optional (so only packages that get some
benefit of it would have to implement it).

(PS: I am unsure how this bug fit the purpose of the TC. There is only one
alternative that have been implemented or even drafted at this stage.)

Cheers,
-- 
Bill. <ballombe@debian.org>

Imagine a large red swirl here.




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Thu, 09 Jun 2011 22:33:06 GMT) Full text and rfc822 format available.

Acknowledgement sent to Russ Allbery <rra@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Thu, 09 Jun 2011 22:33:07 GMT) Full text and rfc822 format available.

Message #259 received at 604397@bugs.debian.org (full text, mbox):

From: Russ Allbery <rra@debian.org>
To: Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>
Cc: 604397@bugs.debian.org, debian-ctte@lists.debian.org, 629385@bugs.debian.org
Subject: Re: Bug#604397: Request for TC to rule on a course of action for supporting build-arch
Date: Thu, 09 Jun 2011 15:28:43 -0700
Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr> writes:

> I have proposed an alternative in the past (which did not get any
> support, though):  Decide that packages that have a Build-Depends-Indep:
> field must implement build-arch/build-indep. This is probably already
> the case.

> This has the same advantage than Build-Options: a program can check if
> dpkg-buildpackage will call build-arch just by looking at the control
> file.

This is a hack, though, that relies on the assumption that any package
with separate build-arch and build-indep targets will have additional
dependencies for build-indep.  This is not necessarily true.  For example,
building architecture-independent bearoff databases for GnuBG doesn't
require any additional packages.

> There should be a well-defined interface to know whether
> dpkg-buildpackage will call build-arch or build, that does not depend on
> a specific dpkg-buildpackage implementation.

I don't agree with this statement in the long run.  I don't see any reason
not to have a goal of having dpkg-buildpackage call build-arch
unconditionally.  The most maintainable, most easily testable, and least
complex way to implement an option in a software system is to not make it
an option.

My goal is to allow everyone developing for Debian three years from now to
not care that this debate ever happened.

> Concerning the long-term complexity: If you look at the bug log, you
> will see that developpers objected that the build-arch/build-indep split
> was unnecessary complexity for packages that would not get any benefit
> of it, so Build-Options was proposed which allowed the split to be
> optional (so only packages that get some benefit of it would have to
> implement it).

I think that the dh program and its widespread adoption has mostly made
that objection obsolete.  For those who are still listing separate targets
in debian/rules, making build-arch and build-indep just depend on build is
not generally difficult.

> (PS: I am unsure how this bug fit the purpose of the TC. There is only
> one alternative that have been implemented or even drafted at this
> stage.)

The TC is currently considering four different options, all of which are
either already implemented or fairly easy to implement in a short time
frame.  I don't think all the options have to be at the point of being
appliable patches before the TC can consider something.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#604397; Package debian-policy. (Thu, 30 Jun 2011 21:39:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Thu, 30 Jun 2011 21:39:03 GMT) Full text and rfc822 format available.

Message #264 received at 604397@bugs.debian.org (full text, mbox):

From: Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>
To: 629385@bugs.debian.org, 604397@bugs.debian.org
Cc: Steve Langasek <vorlon@debian.org>
Subject: Re: Bug#604397: Request for TC to rule on a course of action for supporting build-arch
Date: Thu, 30 Jun 2011 23:34:28 +0200
On Tue, Jun 07, 2011 at 09:29:12AM +0200, Bill Allombert wrote:
> On Mon, Jun 06, 2011 at 04:02:36PM -0700, Steve Langasek wrote:
> > Hi Bill,
> > 
> > (Sending this to 629385@bugs.debian.org, which is the cloned bug actually
> > assigned to the TC...)

Too much magic, but thanks anyway.

> > On Tue, Jun 07, 2011 at 12:06:18AM +0200, Bill Allombert wrote:
> > > The proposal 3) (which is implemented in dpkg as of today) was devised
> > > following a discussion in Debian policy bug #218893 as a compromise
> > > solution that was agreeable to everyone, then a patch to dpkg was written (bug
> > > #229357). For reasons beyond my control, the patch was actually merged only
> > > today.
> > 
> > I think your read that this was a "compromise solution that was agreeable to
> > everyone" is incorrect.  Scott James Remnant (a dpkg maintainer at the time)
> > raised in http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=218893#345 the
> > same objection about fragility that others have raised since.

Scott was not yet the dpkg maintainer when the discussion occurred.

> Scott did not take part in the initial discussion, so there was no way his objection
> would be taken into account.

... but Scott has been the maintainer of dpkg for a sufficiently long time to
implement autodetection if he wanted to, but he did not manage to do it.

And that is the major reason I not impressed by autodetection: it is proposed
every few year as the silver bullet to fix that issue, but it never materialize
while putting the other options to the backburner, and only fix half of the problem
(packages still have to be changed to split Build-Depends-Indep from Build-Depends).

Cheers,
-- 
Bill. <ballombe@debian.org>

Imagine a large red swirl here. 




Added tag(s) pending; removed tag(s) patch. Request was from Russ Allbery <rra@debian.org> to control@bugs.debian.org. (Mon, 27 Aug 2012 15:33:05 GMT) Full text and rfc822 format available.

Message #267 received at 374029-close@bugs.debian.org (full text, mbox):

From: Russ Allbery <rra@debian.org>
To: 374029-close@bugs.debian.org
Subject: Bug#374029: fixed in debian-policy 3.9.4.0
Date: Wed, 19 Sep 2012 06:32:10 +0000
Source: debian-policy
Source-Version: 3.9.4.0

We believe that the bug you reported is fixed in the latest version of
debian-policy, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 374029@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Russ Allbery <rra@debian.org> (supplier of updated debian-policy package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Tue, 18 Sep 2012 21:35:36 -0700
Source: debian-policy
Binary: debian-policy
Architecture: source all
Version: 3.9.4.0
Distribution: unstable
Urgency: low
Maintainer: Debian Policy List <debian-policy@lists.debian.org>
Changed-By: Russ Allbery <rra@debian.org>
Description: 
 debian-policy - Debian Policy Manual and related documents
Closes: 374029 571776 591791 641153 654958 661816 661933 663918 670429 676561
Changes: 
 debian-policy (3.9.4.0) unstable; urgency=low
 .
   * build-arch and build-indep are now mandatory targets in debian/rules,
     implementing the Technical Committee ruling in #629385.  Wording
     review by Jonathan Nieder, Jakub Wilk, and Roger Leigh.
     (Closes: #374029)
   * Resynchronize the archive section list with ftp-master, adding tasks.
     Patch from Charles Plessy.  (Closes: #670429)
   * Policy: Copyright files must be encoded in UTF-8
     Wording: Russ Allbery <rra@debian.org>
     Seconded: Guillem Jover <guillem@debian.org>
     Seconded: Salvatore Bonaccorso <carnil@debian.org>
     Seconded: Simon McVittie <smcv@debian.org>
     Closes: #661933
   * Policy: Prohibit deprecated < and > relations
     Wording: Jakub Wilk <jwilk@debian.org>
     Seconded: Cyril Brulebois <kibi@debian.org>
     Seconded: Russ Allbery <rra@debian.org>
     Closes: #663918
   * Policy: Document the Built-Using field
     Wording: Charles Plessy <plessy@debian.org>
     Seconded: Russ Allbery <rra@debian.org>
     Seconded: Ansgar Burchardt <ansgar@debian.org>
     Closes: #641153
   * Policy: Document the Vcs-* fields
     Wording: Charles Plessy <plessy@debian.org>
     Wording: Jonathan Nieder <jrnieder@gmail.com>
     Seconded: Russ Allbery <rra@debian.org>
     Seconded: Charles Plessy <plessy@debian.org>
     Seconded: Guillem Jover <guillem@debian.org>
     Closes: #654958
   * Policy: Document restrictions on the use of /run for wheezy
     Wording: Roger Leigh <rleigh@debian.org>
     Seconded: Charles Plessy <plessy@debian.org>
     Seconded: Russ Allbery <rra@debian.org>
     Closes: #676561
   * Policy: Rewrite shared library dependency policy to document symbols
     Wording: Russ Allbery <rra@debian.org>
     Wording: Jonathan Nieder <jrnieder@gmail.com>
     Seconded: Raphaël Hertzog <hertzog@debian.org>
     Seconded: Julien Cristau <jcristau@debian.org>
     Closes: #571776
   * Policy: Document generic and upstart-specific init system requirements
     Wording: Steve Langasek <steve.langasek@canonical.com>
     Seconded: Russ Allbery <rra@debian.org>
     Seconded: Adam D. Barratt <adam@adam-barratt.org.uk>
     Closes: #591791
   * Policy: Rely on triggers instead of calling update-mime
     Wording: Charles Plessy <plessy@debian.org>
     Seconded: Russ Allbery <rra@debian.org>
     Seconded: Guillem Jover <guillem@debian.org>
     Closes: #661816
Checksums-Sha1: 
 c728495994bbdabc43055dfbedfd662bba5eb069 1518 debian-policy_3.9.4.0.dsc
 4c6bc2d0eb510313e1b4a0d2a932f4182ffe6f91 704838 debian-policy_3.9.4.0.tar.gz
 ac9ff5a5987343a736fb45af52d3178bae30d37e 2147892 debian-policy_3.9.4.0_all.deb
Checksums-Sha256: 
 c6dbad5976931268283c02903cf0dc3f3bb8dfc86710cab462e0e6c19aab1407 1518 debian-policy_3.9.4.0.dsc
 01ae1a19f7a251dd5c2b078736427f33f04c5f7e38308f874345f1e3e194dca5 704838 debian-policy_3.9.4.0.tar.gz
 c6e22f66e4cd38cbfac944bfebb41fa5608604326c6ecb9dbbd2213f5372ebbd 2147892 debian-policy_3.9.4.0_all.deb
Files: 
 e5683a409d1f740582e960158152b4ba 1518 doc optional debian-policy_3.9.4.0.dsc
 33eafa60a7c79f827adaa1bdb0cdcf83 704838 doc optional debian-policy_3.9.4.0.tar.gz
 2c5c278e5035e26489c3ae76f8c428d2 2147892 doc optional debian-policy_3.9.4.0_all.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iQEcBAEBCAAGBQJQWVEqAAoJEH2AMVxXNt51CfcH/226voYDWjhFjwJJh61d0XBI
3JRRYNqF7rZ59zl4kwEX+QFe/CoKnX1rEceBb9g3cCJ/AO6vU8Z+hhGnpr4eus1v
2BKhO4E8S6vqjtWfiXHIUmkIlGQeuxY3aBMWNZPgQzqEz9Skrc3aDel3zuuiKehE
fTk8Kse0hwTGp5h9nVaXawdZEPKFhcQT2NrhhTE/VmTHuC1EzUTcjOUDeu8tM2xy
r6Zjytz43qqvWinUQNYQXOtjt2zAVV0dw6T9nWcssXOSTD1EZLbfAbaJw9m1VG6G
B9BRhz5xs334/DktrgDw2gKjb4IF2tI3lIPRj12OuGErR+lChgZr4egrA+xyBwM=
=SP2c
-----END PGP SIGNATURE-----




Bug archived. Request was from Debbugs Internal Request <owner@bugs.debian.org> to internal_control@bugs.debian.org. (Mon, 03 Jun 2013 07:49:19 GMT) Full text and rfc822 format available.

Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Sat Apr 19 01:04:08 2014; Machine Name: buxtehude.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.