Debian Bug report logs - #397939
clean rule behavior underspecified and inconsistent with common practice

Package: debian-policy; Maintainer for debian-policy is Debian Policy List <debian-policy@lists.debian.org>; Source for debian-policy is src:debian-policy.

Reported by: Martin Zobel-Helas <zobel@ftbfs.de>

Date: Fri, 10 Nov 2006 15:48:12 UTC

Severity: wishlist

Reply or subscribe to this bug.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, debian-release@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Martin Zobel-Helas <zobel@ftbfs.de>:
New Bug report received and forwarded. Copy sent to debian-release@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: Martin Zobel-Helas <zobel@ftbfs.de>
To: submit@bugs.debian.org
Subject: Proposal: Packages must have a working clean target
Date: Fri, 10 Nov 2006 16:11:22 +0100
Package: debian-policy
Severity: important


Hi,

during the last months i had to review several packages. Quite a number
of packages were not buildable two times (eg. "unrepresentable changes
to source"). Most of these packages used svn-buildpackage or
cvs-buildpackage. This bug is quite annoying as one needs to either
manual interact or run dpkg-source -x again.

I therefore propose a policy change for etch+1, that all packages need
to have a proper working clean target, so a directly rebuild of the
package is possible without manual interaction.

Discussion and rephrasing of that text welcome.

Greetings
Martin
-- 
[root@debian /root]# man real-life
No manual entry for real-life




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Manoj Srivastava <srivasta@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #10 received at 397939@bugs.debian.org (full text, mbox):

From: Manoj Srivastava <srivasta@debian.org>
To: Martin Zobel-Helas <zobel@ftbfs.de>
Cc: 397939@bugs.debian.org
Subject: Re: Bug#397939: Proposal: Packages must have a working clean target
Date: Fri, 10 Nov 2006 10:33:54 -0600
On Fri, 10 Nov 2006 16:11:22 +0100, Martin Zobel-Helas <zobel@ftbfs.de> said: 

> Package: debian-policy Severity: important

> Hi,

> during the last months i had to review several packages. Quite a
> number of packages were not buildable two times
> (eg. "unrepresentable changes to source"). Most of these packages
> used svn-buildpackage or cvs-buildpackage. This bug is quite
> annoying as one needs to either manual interact or run dpkg-source
> -x again.

        Sounds like a bug in the package. Whydo we need to make policy
 that says "Do not create buggy packages"?

> I therefore propose a policy change for etch+1, that all packages
> need to have a proper working clean target, so a directly rebuild of
> the package is possible without manual interaction.

        Policy already states that the clean target must exist, and
 has to undo whatever build did. Violating this precept already seems
 like a bug.

> Discussion and rephrasing of that text welcome.

        I suggest closig this proposal, and filing bugs on pakages
 that do not have a working clean target, in  violation of existing
 policy.

        If people are not following current policy, and we can't
 enforce current policy, making new policy doctums is unlikely to
 help.

        manoj
-- 
Its name is Public Opinion.  It is held in reverence.  It settles
everything. Some think it is the voice of God.  -- Mark Twain
Manoj Srivastava <srivasta@debian.org> <http://www.debian.org/~srivasta/>
1024D/BF24424C print 4966 F272 D093 B493 410B  924B 21BA DABB BF24 424C



Severity set to `wishlist' from `important' Request was from Manoj Srivastava <srivasta@golden-gryphon.com> to control@bugs.debian.org. Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Kurt Roeckx <kurt@roeckx.be>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #17 received at 397939@bugs.debian.org (full text, mbox):

From: Kurt Roeckx <kurt@roeckx.be>
To: Martin Zobel-Helas <zobel@ftbfs.de>, 397939@bugs.debian.org
Subject: Re: Bug#397939: Proposal: Packages must have a working clean target
Date: Fri, 10 Nov 2006 18:23:44 +0100
On Fri, Nov 10, 2006 at 04:11:22PM +0100, Martin Zobel-Helas wrote:
> Package: debian-policy
> Severity: important
> 
> 
> Hi,
> 
> during the last months i had to review several packages. Quite a number
> of packages were not buildable two times (eg. "unrepresentable changes
> to source"). Most of these packages used svn-buildpackage or
> cvs-buildpackage. This bug is quite annoying as one needs to either
> manual interact or run dpkg-source -x again.
> 
> I therefore propose a policy change for etch+1, that all packages need
> to have a proper working clean target, so a directly rebuild of the
> package is possible without manual interaction.

And by "need" you mean MUST?  In the subject you also use the word must.

I would like to see such a change.


Kurt




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Martin Zobel-Helas <zobel@ftbfs.de>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #22 received at 397939@bugs.debian.org (full text, mbox):

From: Martin Zobel-Helas <zobel@ftbfs.de>
To: debian-policy@lists.debian.org
Cc: 397939@bugs.debian.org
Subject: Re: Bug#397939: Proposal: Packages must have a working clean target
Date: Thu, 23 Nov 2006 00:28:14 +0100
Hi,

FYI, Alexander Wirt started rebuilding the archive testing for packages
which have not a proper working clean target.

Expect results in a few days. (Within 90min already 4 packages popped
up).

Greetings
Martin
-- 
[root@debian /root]# man real-life
No manual entry for real-life




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Russ Allbery <rra@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #27 received at 397939@bugs.debian.org (full text, mbox):

From: Russ Allbery <rra@debian.org>
To: 397939@bugs.debian.org
Subject: Re: Bug#397939: Proposal: Packages must have a working clean target
Date: Tue, 19 Dec 2006 16:15:34 -0800
Moving this discussion into the bug, since I'm not going to have a chance
to work on wording any time soon and I want to remember what we talked
about at that point.

Manoj Srivastava <srivasta@debian.org> writes:

>         In all of the following discussion, no one has ever said
>  anything about *WHY*  policy states that clean must undo what build
>  does. Unless we are clear on the rationale for dictum, trying to
>  resolve the issue is like playing blind man's bluff.

>         There are several reasons for wanting a clean target:
>  a) one should be able to iteratively tweak the source code, and so
>     the build/clean cycle should be idempotent.
>  b) The build process must be reproducible -- and this is where I see
>     the aautotools recommendation failing.  If we are supposed to be a
>     part of the free software community, we expect suers to contribute
>     back to us. One of the major areas where such help is needed is
>     debugging -- so someone building in a non-buildd or a non-one-shot
>     build process , should have the same, reproducible results as the
>     developer does.

>     So, if I ship some code, and I am on IRC with a user, they do
>     ./debian/rules clean, install some changes I suggest, we should
>     get the same compiler results -- without such a common baseline,
>     collaboration is made harder.

I agree with those goals.

Part of (a) is basically, "running build, binary, clean, build, and binary
again should result in the same packages the second time as the first,
modulo compiler timestamp differences."  In other words, I think what (a)
captures is that clean should return the package state to a place where it
can be built again with the same results.

The other half of (a) is something like "running build, binary, clean,
then modifying a source file, and then running build and binary again
should result in packages reflecting the change made to the source file."
In other words, clean needs to clean things up sufficiently that
modifications are noticed and acted on in the next build.

I think it's an open question how much we care about what detritus is left
around in the source directory provided that both of the above
requirements are met.  And I think that open question is exactly what this
thread is about.  I would rather require the above two principles, in some
form, than the current Policy language that says clean must restore a
pristine build tree.  The latter is a nice property, but it can be tricky
to provide, many packages don't provide this currently, and it's not clear
to me how really important it is (whether, in other words, not doing so
really introduces bugs).

As for (b), this is a general target that we can't ever meet completely,
since some aspects of the build system are going to change between the
point at which the package enters the archive and the point at which
someone else tries to rebuild it unless it's an arch-independent package.
The compiler will change, binutils will change, there will be a new libc,
etc.  The only difference between autotools and the compiler and that
autotools have historically been less stable for complex packages.  That
being said, for the packages I maintain, I've not had much difficulty with
newer versions of autotools doing something different than older versions.

>         Building packages from source is good -- but in this case, it
>  does introduce another variable as far as auto-tools and helper package
>  versions.  We can say that we find this acceptable -- and that package
>  should build depend on autotools, and _always_ run autotools in the
>  build process.  This would tend to find bugs in those packages quickly,
>  too.

>         Building from sources improves the package quality in the long
>  run, but does impact reproducibility, since now the tool chain
>  components have increased in number.

>         This also stands for lex, yacc, and swig output files.

This is where I'm at.  For those packages where the upstream files are not
sufficient for some reason, I would prefer to always run the autotools so
that I notice when things break.  In practice, things generally don't
break.

>         However, I wonder about the value of artificially reducing the
>  size of the diffs. there are tools for examing the diffs, and for
>  separating out the diffs on autogenerated files from diffs on true
>  source files. 

Yeah, I think diff size is one of the weaker arguments, the more I think
about it.  A more compelling argument from my perspective is that packages
really should be built from source, not from an intermediate form prepared
by the Debian developer.

To some extent, that same argument extends to upstream as well.  I can see
always running autotools period, even if the upstream-provided generated
files are sufficient.  There is no difference in principle, really.  But
as a practical matter, when the upstream files are sufficient, I think it
makes sense to leave well enough alone.  I have no strong principle on
which to base that position; it's just an argument from practicality.

>> I can imagine an argument for removing all of the intermediate files
>> (Makefile.in, configure, etc) in the clean target and rerunning the
>> autotools stuff from the build target.  This would provide a relatively
>> small diff (provided upstream doesn't ship the intermediate files),
>> although I am not sure of the wisdom of this approach.

Upstream basically always ships the intermediate files.  The only
exception in my experience are snapshots.

Doing this doesn't address the issue raised at the beginning of this
thread.  This approach violates the current Policy requirement for the
clean target since, unless one restores all of those generated files from
the original tarball on debian/rules clean, the clean target doesn't
restore the tree to the state it was in before running build.  This is, in
practice, difficult to do unless you maintain a snapshot of the tree
before you run the autotools, since they frequently change *many* files.

>         In my experience, upstream ships ancient versions of
>  intermediate files that often no longer work on hardware that Debian
>  supports.  This has been my experience with fvwm, and before the
>  current release, make.

Yup.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>



Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Russ Allbery <rra@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #32 received at 397939@bugs.debian.org (full text, mbox):

From: Russ Allbery <rra@debian.org>
To: 397939@bugs.debian.org
Subject: Re: Bug#397939: Proposal: Packages must have a working clean target
Date: Tue, 19 Dec 2006 16:20:03 -0800
Bill Allombert <allomber@math.u-bordeaux.fr> writes:
> On Fri, Nov 10, 2006 at 10:48:15AM -0800, Russ Allbery wrote:

>> As previously discussed, it's very difficult to comply with this
>> directive as written if one is following the autotools-dev
>> recommendations for how to regenerate the various autotools files.
>> Before putting too much weight on this directive, I'd really like to
>> find some way of reconciling that, since right now it's a
>> frequently-violated dictate of Policy.

>> Certainly, though, being unable to build a package twice is a bug that
>> should be reported against that package.  (I actually don't know if any
>> of my packages have this problem; some of them have so many build
>> dependencies that I always build them in pbuilder chroot.  Hm.)

> I already proposed we change policy to require that doing
> debian/rules clean
> debian/rules binary
> debian/rules clean
> restore the tree to the state after the first "debian/rules clean"
> providing no packages were installed or upgraded in between.
> This way, it is allowable for debian/rules clean to remove or change
> files shipped in the tarball providing this is idempotent.

I find running the autotools in debian/rules clean extremely
counterintuitive.  It also breaks how I handle my packages within a
revision control system, and given that I use svn-buildpackage in a fairly
standard mode, I expect I am *very* far from the only one.

What this basically requires is putting the generated output of the
autotools files in the diff.  This is only one of the options documented
in autotools-dev, and I think both of the options listed there should be
acceptable.  In practice, neither cause issues.

> I also proposed a different way to deal with config.guess/config.sub
> that comply with the current policy.

Yes, which is a neat approach, but I think it's still an open question
whether such workarounds should be necessary.  While that solution
provides a nice way to satisfy the current policy requirement, I see no
compelling reason *for* the current policy requirement, and therefore
don't see why one shouldn't be permitted to simply replace those files at
build-time with files provided by a build dependency.

If someone wants to use your technique, I certainly wouldn't argue with
them, but from a Policy standpoint I don't see how it causes less bugs or
creates a better distribution or packaging system than doing what many of
us have historically done.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>



Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Bas Wijnen <shevek@fmf.nl>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #37 received at 397939@bugs.debian.org (full text, mbox):

From: Bas Wijnen <shevek@fmf.nl>
To: debian-mentors@lists.debian.org
Cc: 397939@bugs.debian.org
Subject: Re: Lintian: outdated-autotools-helper-file
Date: Mon, 11 Feb 2008 10:53:48 +0100
[Message part 1 (text/plain, inline)]
On Sun, Feb 10, 2008 at 03:48:20PM -0800, Russ Allbery wrote:
> Raphael Geissert <atomo64+debian@gmail.com> writes:
> 
> > Quoting the Debian Policy, section 4.9 Main building script:
> > debian/rules[1]
> >
> >> clean
> >> 
> >> This must undo any effects that the build and binary targets may
> >> have had, except that it should leave alone any output files
> >> created in the parent directory by a run of a binary target.  
> >
> > So according to the policy the clean target must put the original
> > files back on place.
> 
> This Policy dictate as written is pretty widely ignored and (IMO) is
> somewhat hard to defend in several of its implications.  We haven't
> figured out what to say instead, but deleting the files is fairly
> common right now.
> 
> See http://bugs.debian.org/397939 for some additional discussion.

Thank you for this link.  I would like to add a suggestion as a
solution.  IMO the important thing is, as stated by Bill, that "clean"
and "clean; binary; clean" should result in the same tree.  From the log
it seems that people agree that this implies either creating a huge
diff.gz, or running autotools in the clean target.

This is not true if you simply build the whole package from source.
That is, run autotools during build, remove all generated files,
including Makefile.in, configure, etc, in the clean target.

For some reason many people seem to think that the whole package must be
built from source, except for configure and Makefile.in.  I still don't
understand what the idea behind this exception is.  Especially when it
leads to unwanted concequences (unreadable diff.gzs, for example), I
don't think it is a good idea to hold on to the idea that running
autotools is not part of building the package.

Apart from that, this discussion is about users making changes to the
source and compiling a package with those changes in it.  When not
running autotools during build, that doesn't work if the user changes
Makefile.am or configure.ac.  Yet another undesirable effect of this
strange exception to "build everything from source".

I suggest to mandate "remove all generated files in the clean target"
(formulated in a way which includes "generated by upstream", not only
"generated by the build target), which implies "rebuild everything in
the build target".

With the current wording it is allowed to use shipped built files from
the upstream tarball, as long as the source is present.  It is even
allowed to ship the build results (uuencoded, for example) in the
diff.gz and use those.  I suppose we all agree that this is unacceptable
for normal build results.

Now reread the previous paragraph while thinking of Makefile.in instead
of "normal build results".  It is common practice to do exactly that:
ship and use pre-built versions, or even ship them in the diff.gz (which
then gets huge).  Makefile.am is only present as source-file, but it
isn't used at all during the build.  Any changes the user makes will not
be noticed by the build system.

I'd like to hear why this exception is so important for people.  Or
better yet, I'd like to hear that everyone agrees with me, so we can
make this change and finally to the Right Thing. :-)

Thanks,
Bas

-- 
I encourage people to send encrypted e-mail (see http://www.gnupg.org).
If you have problems reading my e-mail, use a better reader.
Please send the central message of e-mails as plain text
   in the message body, not as HTML and definitely not as MS Word.
Please do not use the MS Word format for attachments either.
For more information, see http://pcbcn10.phys.rug.nl/e-mail.html
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Kapil Hari Paranjape <kapil@imsc.res.in>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #42 received at 397939@bugs.debian.org (full text, mbox):

From: Kapil Hari Paranjape <kapil@imsc.res.in>
To: debian-mentors@lists.debian.org, 397939@bugs.debian.org
Subject: Re: Lintian: outdated-autotools-helper-file
Date: Mon, 11 Feb 2008 17:40:58 +0530
[Message part 1 (text/plain, inline)]
On Mon, 11 Feb 2008, Bas Wijnen wrote:
> I suggest to mandate "remove all generated files in the clean target"
> (formulated in a way which includes "generated by upstream", not only
> "generated by the build target), which implies "rebuild everything in
> the build target".
> 
> With the current wording it is allowed to use shipped built files from
> the upstream tarball, as long as the source is present.  It is even
> allowed to ship the build results (uuencoded, for example) in the
> diff.gz and use those.  I suppose we all agree that this is unacceptable
> for normal build results.
> 
> Now reread the previous paragraph while thinking of Makefile.in instead
> of "normal build results".  It is common practice to do exactly that:
> ship and use pre-built versions, or even ship them in the diff.gz (which
> then gets huge).  Makefile.am is only present as source-file, but it
> isn't used at all during the build.  Any changes the user makes will not
> be noticed by the build system.
> 
> I'd like to hear why this exception is so important for people.  Or
> better yet, I'd like to hear that everyone agrees with me, so we can
> make this change and finally to the Right Thing. :-)

It is a good idea if one can use .orig.tar.gz to be the same as the
upstream .tar.gz whenever it is DFSG-free to do so.

One reason is that it becomes easier to verify that the .orig.tar.gz
is the same --- has the same GPG signature, MD5SUM etc.

Another reason (which is the same reason in disguise!) is that it is
easier to see exactly what Debian is changing in the upstream source.

Note that if the upstream's auto-generated files are deleted during
the clean target, then the source *must* be re-packaged to avoid
needless clutter in the .diff.gz which is of a "negative" nature.

So all such source must come with a "get-orig-source" target and a
README.Debian-source which explains the changes made.

Regards,

Kapil.
--
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Russ Allbery <rra@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #47 received at 397939@bugs.debian.org (full text, mbox):

From: Russ Allbery <rra@debian.org>
To: debian-mentors@lists.debian.org
Cc: 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Mon, 11 Feb 2008 09:21:29 -0800
Bas Wijnen <shevek@fmf.nl> writes:

> I suggest to mandate "remove all generated files in the clean target"
> (formulated in a way which includes "generated by upstream", not only
> "generated by the build target), which implies "rebuild everything in
> the build target".

[...]

> I'd like to hear why this exception is so important for people.  Or
> better yet, I'd like to hear that everyone agrees with me, so we can
> make this change and finally to the Right Thing. :-)

It may be less common now, but for many years it was quite common for
upstreams to use very specific versions of autotools, which means that if
Debian had dropped the specific autoconf in question, it wasn't easy to
regenerate the files.  Now, that may be a problem for other reasons since
as you point out it means we don't really have horribly usable source, but
that's the most common practical concern, I think.

Always re-running autoconf and automake would increase the number of
FTBFS's that we'd need to fix.  (Probably for the greater good of free
software, but.)

Also, it's not always easy to figure out which files are generated in
order to remove them, but that's probably programmatically fixable.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Henrique de Moraes Holschuh <hmh@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #52 received at 397939@bugs.debian.org (full text, mbox):

From: Henrique de Moraes Holschuh <hmh@debian.org>
To: debian-mentors@lists.debian.org, 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Mon, 11 Feb 2008 20:22:42 -0200
On Mon, 11 Feb 2008, Kapil Hari Paranjape wrote:
> Note that if the upstream's auto-generated files are deleted during
> the clean target, then the source *must* be re-packaged to avoid
> needless clutter in the .diff.gz which is of a "negative" nature.

Not so.  Deletions are ignored.  Ever tried it?

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Bas Wijnen <wijnen@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #57 received at 397939@bugs.debian.org (full text, mbox):

From: Bas Wijnen <wijnen@debian.org>
To: debian-mentors@lists.debian.org
Cc: 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Tue, 12 Feb 2008 00:16:17 +0100
[Message part 1 (text/plain, inline)]
On Mon, Feb 11, 2008 at 09:21:29AM -0800, Russ Allbery wrote:
> Bas Wijnen <shevek@fmf.nl> writes:
> 
> > I suggest to mandate "remove all generated files in the clean target"
> > (formulated in a way which includes "generated by upstream", not only
> > "generated by the build target), which implies "rebuild everything in
> > the build target".
> 
> [...]
> 
> > I'd like to hear why this exception is so important for people.  Or
> > better yet, I'd like to hear that everyone agrees with me, so we can
> > make this change and finally to the Right Thing. :-)
> 
> It may be less common now, but for many years it was quite common for
> upstreams to use very specific versions of autotools, which means that if
> Debian had dropped the specific autoconf in question, it wasn't easy to
> regenerate the files.  Now, that may be a problem for other reasons since
> as you point out it means we don't really have horribly usable source, but
> that's the most common practical concern, I think.

Oh, I didn't know that.  IMO that would mean the package should really
be in contrib, because we're missing a build dependency in main then
(not used, but AFAIK using pre-built files is not an acceptable way to
avoid getting in contrib in such a situation).

Anyway, I don't think that is still a reason for not running autotools
nowadays, so it's just historical background. :-)

> Always re-running autoconf and automake would increase the number of
> FTBFS's that we'd need to fix.

Not really.  It would lead to many bugs that packages aren't following
policy.  Especially if we start with should and later (possibly) upgrade
it to must, I think it is workable.  In particular because these bugs
don't actually stop a package from being built.  I would be very happy
with consensus that the autotools _should_ be run during build.  The
implementation of actually doing it in all packages may take a while, I
don't have a problem with that.

Unless of course you are talking about cdbs.  When that changes to
running autotools, it likely needs to know if there are extra arguments.
That may indeed result in FTBFSs for many packages which use it (because
they may need, but don't provide, the arguments).  But it's good when
they're fixed as well. :-)

> (Probably for the greater good of free software, but.)

Yes, IMO it's one of those situations where Debian should do what's
Right, not what's Easy (similar to what I wrote about the /bin/sh
bash->dash move on -policy today).

> Also, it's not always easy to figure out which files are generated in
> order to remove them, but that's probably programmatically fixable.

That's in principle a problem for the maintainer to solve, and
secundarily for lintian.  If things can be automated, that may be nice,
but it is not required.  Once policy mandates to remove generated files
in the clean target, it's up to the maintainer to do it.  And if the
maintainer makes a mistake and forgets to remove a file, that's not a
big problem (it would be a bug with this change in policy, but only a
minor one, IMO).

At this moment however, I don't think there is consensus yet that it is
always better to remove all generated files, including
autotools-generated stuff, in the clean target.  After that happens, we
can think about how to implement it in the archive.  Slowly, I would
suggest. :-)  Because it is a good thing if we do this Right, but we
shouldn't break half the archive for it. ;-)

Thanks,
Bas

-- 
I encourage people to send encrypted e-mail (see http://www.gnupg.org).
If you have problems reading my e-mail, use a better reader.
Please send the central message of e-mails as plain text
   in the message body, not as HTML and definitely not as MS Word.
Please do not use the MS Word format for attachments either.
For more information, see http://pcbcn10.phys.rug.nl/e-mail.html
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Russ Allbery <rra@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #62 received at 397939@bugs.debian.org (full text, mbox):

From: Russ Allbery <rra@debian.org>
To: debian-mentors@lists.debian.org
Cc: 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Mon, 11 Feb 2008 15:19:36 -0800
Bas Wijnen <wijnen@debian.org> writes:
> On Mon, Feb 11, 2008 at 09:21:29AM -0800, Russ Allbery wrote:

>> Always re-running autoconf and automake would increase the number of
>> FTBFS's that we'd need to fix.

> Not really.

No, really, I promise it will.  :)  Each time we upgrade autoconf, it will
break a bunch of packages that were doing things that weren't supported.

Now, those really were bugs and should be fixed.  But turning them into
FTBFS bugs does escalate the severity quite a bit.

> It would lead to many bugs that packages aren't following policy.
> Especially if we start with should and later (possibly) upgrade it to
> must, I think it is workable.  In particular because these bugs don't
> actually stop a package from being built.  I would be very happy with
> consensus that the autotools _should_ be run during build.  The
> implementation of actually doing it in all packages may take a while, I
> don't have a problem with that.

Well, I don't do it for mine right now because it takes a long time and
feels kind of pointless, but I'm happy to go along with any consensus.
But that part isn't so much the concern.

> Yes, IMO it's one of those situations where Debian should do what's
> Right, not what's Easy (similar to what I wrote about the /bin/sh
> bash->dash move on -policy today).

I am generally in favor of that, but I also don't have the free time to
volunteer for the release team, who ends up bearing the brunt of us doing
the right thing in this area, so, y'know, easy for me to say.  :)

> That's in principle a problem for the maintainer to solve, and
> secundarily for lintian.

Well, yeah, but it would still be nice to provide some help.

> At this moment however, I don't think there is consensus yet that it is
> always better to remove all generated files, including
> autotools-generated stuff, in the clean target.  After that happens, we
> can think about how to implement it in the archive.  Slowly, I would
> suggest. :-) Because it is a good thing if we do this Right, but we
> shouldn't break half the archive for it. ;-)

Yup.  :)

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Bas Wijnen <wijnen@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #67 received at 397939@bugs.debian.org (full text, mbox):

From: Bas Wijnen <wijnen@debian.org>
To: debian-mentors@lists.debian.org
Cc: 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Tue, 12 Feb 2008 01:08:17 +0100
[Message part 1 (text/plain, inline)]
On Mon, Feb 11, 2008 at 03:19:36PM -0800, Russ Allbery wrote:
> Bas Wijnen <wijnen@debian.org> writes:
> > On Mon, Feb 11, 2008 at 09:21:29AM -0800, Russ Allbery wrote:
> 
> >> Always re-running autoconf and automake would increase the number of
> >> FTBFS's that we'd need to fix.
> 
> > Not really.
> 
> No, really, I promise it will.  :)  Each time we upgrade autoconf, it will
> break a bunch of packages that were doing things that weren't supported.

Ah, that is a point.  But that should not be the case if the packages
build-depend on the right version, or is it still a problem?  I know
automake in particular changes a lot between versions, but that's why I
always build-depend on a version (such as "automake1.10").  That's
recommended as well AFAIK.  So the problem of removing old versions of
automake (such as automake1.8 at the moment) gets bigger, but it
shouldn't make it FTBFS bugs most of the time.

> > Yes, IMO it's one of those situations where Debian should do what's
> > Right, not what's Easy (similar to what I wrote about the /bin/sh
> > bash->dash move on -policy today).
> 
> I am generally in favor of that, but I also don't have the free time to
> volunteer for the release team, who ends up bearing the brunt of us doing
> the right thing in this area, so, y'know, easy for me to say.  :)

I'm happy to report bugs and provide patches for many packages if that
helps doing this properly.

> > That's in principle a problem for the maintainer to solve, and
> > secundarily for lintian.
> 
> Well, yeah, but it would still be nice to provide some help.

Sure. :-)

Thanks,
Bas

-- 
I encourage people to send encrypted e-mail (see http://www.gnupg.org).
If you have problems reading my e-mail, use a better reader.
Please send the central message of e-mails as plain text
   in the message body, not as HTML and definitely not as MS Word.
Please do not use the MS Word format for attachments either.
For more information, see http://pcbcn10.phys.rug.nl/e-mail.html
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Kurt Roeckx <kurt@roeckx.be>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #72 received at 397939@bugs.debian.org (full text, mbox):

From: Kurt Roeckx <kurt@roeckx.be>
To: Russ Allbery <rra@debian.org>, 397939@bugs.debian.org
Cc: debian-mentors@lists.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Tue, 12 Feb 2008 18:13:06 +0100
On Mon, Feb 11, 2008 at 03:19:36PM -0800, Russ Allbery wrote:
> Bas Wijnen <wijnen@debian.org> writes:
> > On Mon, Feb 11, 2008 at 09:21:29AM -0800, Russ Allbery wrote:
> 
> >> Always re-running autoconf and automake would increase the number of
> >> FTBFS's that we'd need to fix.
> 
> > Not really.
> 
> No, really, I promise it will.  :)  Each time we upgrade autoconf, it will
> break a bunch of packages that were doing things that weren't supported.

From a QA point of view it would be nice that we can actually test the
archive with a new version.  We could warn in advance which packages
have a problem.  We can track which still have the problem, and so on.

This would of course assume that it either gets uploaded to
experimental, or that it's not the default version in unstable.


Kurt





Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Bas Wijnen <wijnen@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #77 received at 397939@bugs.debian.org (full text, mbox):

From: Bas Wijnen <wijnen@debian.org>
To: 397939@bugs.debian.org
Cc: autotools-dev@packages.qa.debian.org
Subject: Bug#397939: Consensus about what a proper clean target is?
Date: Wed, 13 Feb 2008 13:17:23 +0100
[Message part 1 (text/plain, inline)]
On Tue, Feb 12, 2008 at 12:16:17AM +0100, Bas Wijnen wrote:
> At this moment however, I don't think there is consensus yet that it is
> always better to remove all generated files, including
> autotools-generated stuff, in the clean target.

Well, I haven't seen many people claiming that it is really bad to do
this so far, so I'll try to make sure that we can record this as
consensus.  Please, if anyone thinks that it should be allowed for the
clean target to keep some generated files, speak up.  And please argue
why you think so, so we can flame^Wtalk about it. :-)

I'll have one argument myself first: If a generated file is needed by
the build, and it cannot be regenerated for some reason, for example as
a workaround for a bug (such as #441126; read only the second e-mail) or
to bootstrap a compiler build, then this is acceptable.  However, this
file may not be overwritten during the build.  If overwriting is
desired, it should be copied, the copy should be overwritten, and the
copy must be removed by the clean target.  In other words, it must be
used as a source file by the build.

This mail will automatically go to -policy.  I've added the
autotools-dev PTS address.  Followups probably won't do that, since they
don't have the X-PTS-Approved header.  So if you're not reading this on
-policy, but have an opinion, please follow the discussion there (or
through the BTS).

Thanks,
Bas

-- 
I encourage people to send encrypted e-mail (see http://www.gnupg.org).
If you have problems reading my e-mail, use a better reader.
Please send the central message of e-mails as plain text
   in the message body, not as HTML and definitely not as MS Word.
Please do not use the MS Word format for attachments either.
For more information, see http://pcbcn10.phys.rug.nl/e-mail.html
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Clint Adams <schizo@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #82 received at 397939@bugs.debian.org (full text, mbox):

From: Clint Adams <schizo@debian.org>
To: debian-policy@lists.debian.org
Cc: debian-mentors@lists.debian.org, 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Thu, 14 Feb 2008 16:43:52 -0500
On Mon, Feb 11, 2008 at 10:53:48AM +0100, Bas Wijnen wrote:
> I suggest to mandate "remove all generated files in the clean target"
> (formulated in a way which includes "generated by upstream", not only
> "generated by the build target), which implies "rebuild everything in
> the build target".

Tell me how I, as an upstream, can use an experimental version of
libtool in that situation.




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Bas Wijnen <wijnen@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #87 received at 397939@bugs.debian.org (full text, mbox):

From: Bas Wijnen <wijnen@debian.org>
To: debian-mentors@lists.debian.org, 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Fri, 15 Feb 2008 00:46:48 +0100
[Message part 1 (text/plain, inline)]
On Thu, Feb 14, 2008 at 04:43:52PM -0500, Clint Adams wrote:
> On Mon, Feb 11, 2008 at 10:53:48AM +0100, Bas Wijnen wrote:
> > I suggest to mandate "remove all generated files in the clean target"
> > (formulated in a way which includes "generated by upstream", not only
> > "generated by the build target), which implies "rebuild everything in
> > the build target".
> 
> Tell me how I, as an upstream, can use an experimental version of
> libtool in that situation.

Upstream can do whatever they want, of course.  Policy only applies to
Debian.  So I'll instead answer the question "How can I, as a packager,
follow this rule when upstream uses an experimental version of libtool?"

If a package requires a compiler which currently is not in Debian
(main), then the package cannot be in main either (even if the compiler
is free).  This situation you describe falls in that category IMO.

A workaround could be to not regenerate the files.  This is how it
is usually done now.  IMO that is incorrect, because the compiler for
every generated file must be in Debian.  The current practise of not
rerunning autotools makes this rule technically unnecessary, but it can
still be violated (and that should still be considered a bug, even if it
doesn't result in a build failure).

Another workaround could be to include the experimental libtool
in the package.  This is not a very good idea, and it would probably be
better to package the new libtool as a separate package (not named
"libtool", to avoid it being used as default) and Build-Depend on that.

Even better would probably be to convince upstream to not use features
of experimental versions.  That may of course not always be possible.
:-)

Does this answer your question?

Thanks,
Bas

-- 
I encourage people to send encrypted e-mail (see http://www.gnupg.org).
If you have problems reading my e-mail, use a better reader.
Please send the central message of e-mails as plain text
   in the message body, not as HTML and definitely not as MS Word.
Please do not use the MS Word format for attachments either.
For more information, see http://pcbcn10.phys.rug.nl/e-mail.html
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Russ Allbery <rra@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #92 received at 397939@bugs.debian.org (full text, mbox):

From: Russ Allbery <rra@debian.org>
To: debian-mentors@lists.debian.org
Cc: 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Thu, 14 Feb 2008 16:02:41 -0800
Bas Wijnen <wijnen@debian.org> writes:

> A workaround could be to not regenerate the files.  This is how it is
> usually done now.  IMO that is incorrect, because the compiler for every
> generated file must be in Debian.  The current practise of not rerunning
> autotools makes this rule technically unnecessary, but it can still be
> violated (and that should still be considered a bug, even if it doesn't
> result in a build failure).

Note that libtool is an unusual case here and isn't the same as Autoconf
or Automake.  The files included in the package (libtool.m4 and ltmain.sh)
are not generated files in the same sense.  Running libtoolize basically
just copies those files from the installed libtool into the package.

I think (but am not at all certain) that ltmain.sh is a generated file in
that I don't think it's maintained in that form by the libtool
maintainers, but if so, whatever generation is done is already done as
part of the installation of libtool.

It's not like Autoconf or Automake where a file in the source is used as
input to a compiler which generates a shell script based on it.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Bas Wijnen <wijnen@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #97 received at 397939@bugs.debian.org (full text, mbox):

From: Bas Wijnen <wijnen@debian.org>
To: debian-mentors@lists.debian.org, 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Fri, 15 Feb 2008 01:37:54 +0100
[Message part 1 (text/plain, inline)]
On Thu, Feb 14, 2008 at 04:02:41PM -0800, Russ Allbery wrote:
> Bas Wijnen <wijnen@debian.org> writes:
> > A workaround could be to not regenerate the files.  This is how it is
> > usually done now.  IMO that is incorrect, because the compiler for every
> > generated file must be in Debian.  The current practise of not rerunning
> > autotools makes this rule technically unnecessary, but it can still be
> > violated (and that should still be considered a bug, even if it doesn't
> > result in a build failure).
> 
> Note that libtool is an unusual case here and isn't the same as Autoconf
> or Automake.  The files included in the package (libtool.m4 and ltmain.sh)
> are not generated files in the same sense.  Running libtoolize basically
> just copies those files from the installed libtool into the package.

Oh, ok.  That changes things a bit.  In that case, not copying them, but
using the included copy instead is similar to using an included version
of a library instead of the package containing it.  This is bad for
security, but not a problem with respect to policy.

> I think (but am not at all certain) that ltmain.sh is a generated file in
> that I don't think it's maintained in that form by the libtool
> maintainers, but if so, whatever generation is done is already done as
> part of the installation of libtool.

In that case, my proposal would require it to be copied from there, and
not used from upstream.  That is wise anyway, but if it would just be a
copy of the source, it would strictly not be required.

Thanks,
Bas

-- 
I encourage people to send encrypted e-mail (see http://www.gnupg.org).
If you have problems reading my e-mail, use a better reader.
Please send the central message of e-mails as plain text
   in the message body, not as HTML and definitely not as MS Word.
Please do not use the MS Word format for attachments either.
For more information, see http://pcbcn10.phys.rug.nl/e-mail.html
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Russ Allbery <rra@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #102 received at 397939@bugs.debian.org (full text, mbox):

From: Russ Allbery <rra@debian.org>
To: debian-mentors@lists.debian.org
Cc: 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Thu, 14 Feb 2008 17:32:47 -0800
Bas Wijnen <wijnen@debian.org> writes:
> On Thu, Feb 14, 2008 at 04:02:41PM -0800, Russ Allbery wrote:

>> Note that libtool is an unusual case here and isn't the same as
>> Autoconf or Automake.  The files included in the package (libtool.m4
>> and ltmain.sh) are not generated files in the same sense.  Running
>> libtoolize basically just copies those files from the installed libtool
>> into the package.

> Oh, ok.  That changes things a bit.  In that case, not copying them, but
> using the included copy instead is similar to using an included version
> of a library instead of the package containing it.  This is bad for
> security, but not a problem with respect to policy.

Given that the files are only used during the build process, it seems
fairly irrelevant to security to me.  (Embedded copies of libltdl are a
different issue, but we already generally disable those.)

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Clint Adams <schizo@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #107 received at 397939@bugs.debian.org (full text, mbox):

From: Clint Adams <schizo@debian.org>
To: 397939@bugs.debian.org
Cc: debian-mentors@lists.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Thu, 14 Feb 2008 21:22:53 -0500
On Thu, Feb 14, 2008 at 04:02:41PM -0800, Russ Allbery wrote:
> Note that libtool is an unusual case here and isn't the same as Autoconf
> or Automake.  The files included in the package (libtool.m4 and ltmain.sh)
> are not generated files in the same sense.  Running libtoolize basically
> just copies those files from the installed libtool into the package.

libtool.m4 is included via aclocal (unless you're not using automake),
and then processed into configure by autoconf (unless you're not using
autoconf).

So if you build-dep on autoconf and automake and try to use sources
written for libtool 2.1, everything will break horribly even if you keep
the static libtool files the same.

We have been stuck with libtool 1.5 for 4 years.




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Russ Allbery <rra@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #112 received at 397939@bugs.debian.org (full text, mbox):

From: Russ Allbery <rra@debian.org>
To: 397939@bugs.debian.org, debian-mentors@lists.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Thu, 14 Feb 2008 18:44:00 -0800
Clint Adams <schizo@debian.org> writes:
> On Thu, Feb 14, 2008 at 04:02:41PM -0800, Russ Allbery wrote:

>> Note that libtool is an unusual case here and isn't the same as
>> Autoconf or Automake.  The files included in the package (libtool.m4
>> and ltmain.sh) are not generated files in the same sense.  Running
>> libtoolize basically just copies those files from the installed libtool
>> into the package.

> libtool.m4 is included via aclocal (unless you're not using automake),
> and then processed into configure by autoconf (unless you're not using
> autoconf).

Oh, yes, good point.  Sorry about that.

> So if you build-dep on autoconf and automake and try to use sources
> written for libtool 2.1, everything will break horribly even if you keep
> the static libtool files the same.
>
> We have been stuck with libtool 1.5 for 4 years.

I was used to upstreams that include libtool.m4 in the package, in which
case I don't believe that running aclocal will overwrite it (but maybe I'm
wrong?).  If they're pulling in libtool macros using aclocal without
keeping a local copy, that indeed will break.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Colin Watson <cjwatson@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #117 received at 397939@bugs.debian.org (full text, mbox):

From: Colin Watson <cjwatson@debian.org>
To: debian-policy@lists.debian.org, debian-mentors@lists.debian.org
Cc: 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Sun, 17 Feb 2008 15:07:59 +0000
On Mon, Feb 11, 2008 at 10:53:48AM +0100, Bas Wijnen wrote:
> This is not true if you simply build the whole package from source.
> That is, run autotools during build, remove all generated files,
> including Makefile.in, configure, etc, in the clean target.
> 
> For some reason many people seem to think that the whole package must be
> built from source, except for configure and Makefile.in.  I still don't
> understand what the idea behind this exception is.  Especially when it
> leads to unwanted concequences (unreadable diff.gzs, for example), I
> don't think it is a good idea to hold on to the idea that running
> autotools is not part of building the package.

It's a well-established principle of the GNU build system that there are
targets that are run by maintainers and there are targets to be run by
people building the package. This division is there to make our lives
easier, and ignoring it, IMHO, is cutting off our nose to spite our
face. (Even the section in the GNU Coding Standards regarding the
maintainer-clean target doesn't go so far as deleting configure.)

The underlying problem here is that users who change configure.ac etc.
need to do some additional manual work to update the build system, and
that this should be done automatically. I agree. However, this is not
the same as regenerating the full build system from scratch on every
build.

You should be aware, if you aren't already, that the process of
generating a package's build system from scratch is often really quite
complicated and may well involve network access, which build daemons do
not have. For example, the usual update process for a package that uses
Gnulib involves rsyncing translations for some messages from
translationproject.org. If you mandate that Debian maintainers must
support full build system regeneration on every build, then you will
force some of them to maintain a separate bootstrapping process which is
guaranteed not to access the network, which will involve duplicating
work already done by upstream and will undoubtedly introduce errors at
some point.

Mere incremental regeneration on demand is much simpler and is often
already supported by upstream build systems without requiring any change
(see below).

> I suggest to mandate "remove all generated files in the clean target"
> (formulated in a way which includes "generated by upstream", not only
> "generated by the build target), which implies "rebuild everything in
> the build target".

I object to this proposal. As an example, I am upstream for man-db and
know that its build system is properly generated. Requiring that I
regenerate it every time I build the package would introduce nothing but
hassle for me, and would not in practice provide significant benefits
for users since the build system already caters for them out of the box
(see below).

> With the current wording it is allowed to use shipped built files from
> the upstream tarball, as long as the source is present.  It is even
> allowed to ship the build results (uuencoded, for example) in the
> diff.gz and use those.  I suppose we all agree that this is unacceptable
> for normal build results.

configure is readable, if you have to. It's not the preferred form for
modification, certainly, but it's not an opaque object file either, and
there's no need to treat it exactly as we would treat opaque object
files.

Bear in mind also that, while upstream maintainers will probably accept
that configure.ac files that don't work with current versions of
Autoconf are bugs, they are also unlikely to have immediate sympathy for
"I rebuilt your package with a version of Autoconf three years newer
than the one you used and it produced a horrible mess; please help me
untangle it". In other words, this proposal will have the effect of
requiring that Debian maintainers become experts in the Autotools before
being able to upload policy-compliant packages again, whereas that
burden could formerly be passed to upstream to be dealt with on their
schedule. I wouldn't expect that figuring out enough about the Autotools
to be able to do this should be beyond most Debian maintainers, but
neither would I expect that they could all deal with some of the weird
stuff I've seen already.

I agree that behavioural changes due to regenerating the build system
are bugs, but what you're suggesting is that all such bugs be elevated
to the status of showstoppers, as it will be impossible to upload
packages without it. I'm sure that that would make the security team's
life a lot harder, for instance - while there are a few vulnerabilities
that require configure.ac changes to fix, in most cases you can get away
without needing to do so.

Furthermore, if as a user you're desperate to make a small tweak to
configure.ac, it *is* often quite possible - if ugly! - to just make the
obvious corresponding change to configure and then get on with your
life. You can't realistically do that with e.g. compiled C object files,
which is one reason we treat them differently.

> Now reread the previous paragraph while thinking of Makefile.in instead
> of "normal build results".  It is common practice to do exactly that:
> ship and use pre-built versions, or even ship them in the diff.gz (which
> then gets huge).  Makefile.am is only present as source-file, but it
> isn't used at all during the build.  Any changes the user makes will not
> be noticed by the build system.

But this simply isn't true across the board! Makefile.am changes are
only ignored if you use AM_MAINTAINER_MODE. One reason people do that is
that it produces more predictable build-dependencies that aren't
affected by timestamp skew. This was more of an issue before bug #105750
was fixed. Nowadays, while I think the jury is still out in some places
regarding AM_MAINTAINER_MODE, the case for it is much less compelling
than it used to be, and the Automake manual explicitly recommends
against it. If you don't use AM_MAINTAINER_MODE, then Makefile.am
changes are automatically noticed and taken into account. A
demonstration:

  $ apt-get source man-db
  [...]
  dpkg-source: extracting man-db in man-db-2.5.1
  dpkg-source: unpacking man-db_2.5.1.orig.tar.gz
  dpkg-source: applying ./man-db_2.5.1-2.diff.gz
  $ cd man-db-2.5.1/
  $ touch configure.ac
  $ debian/rules build
  ./configure --prefix=/usr --libexecdir=\${libdir} \
  [...]
  touch configure-stamp
  dh_testdir
  /usr/bin/make CFLAGS="-O2 -g -Wall" LDFLAGS="-g"
  make[1]: Entering directory `/home/cjwatson/man-db-2.5.1'
  cd . && /bin/bash /home/cjwatson/man-db-2.5.1/tools/missing --run aclocal-1.10 -I m4 -I gnulib/m4
   cd . && /bin/bash /home/cjwatson/man-db-2.5.1/tools/missing --run automake-1.10 --foreign
  cd . && /bin/bash /home/cjwatson/man-db-2.5.1/tools/missing --run autoconf
  /bin/bash ./config.status --recheck
  [...]

(The same goes for changing Makefile.am.)

Rather than incurring the pain of gratuitous full regeneration every
time, we just regenerate it when the user has changed something. Yes,
the user now gets to resolve any problems that might have been
pre-existing, but realistically either the Debian maintainer or the
upstream maintainer is probably going to run into those at some point
anyway.

I think we should recommend (but not require) that AM_MAINTAINER_MODE
not be used, and perhaps work to specify an optional debian/rules target
that regenerates the build system in an appropriate way. That seems to
provide the necessary benefits for users who need to change these files
without imposing an unacceptable burden on developers. I don't think
there's a good cause to go much further than that at this point.

Cheers,

-- 
Colin Watson                                       [cjwatson@debian.org]




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Russ Allbery <rra@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #122 received at 397939@bugs.debian.org (full text, mbox):

From: Russ Allbery <rra@debian.org>
To: debian-mentors@lists.debian.org, 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Sun, 17 Feb 2008 09:41:58 -0800
Colin Watson <cjwatson@debian.org> writes:

> Rather than incurring the pain of gratuitous full regeneration every
> time, we just regenerate it when the user has changed something. Yes,
> the user now gets to resolve any problems that might have been
> pre-existing, but realistically either the Debian maintainer or the
> upstream maintainer is probably going to run into those at some point
> anyway.
>
> I think we should recommend (but not require) that AM_MAINTAINER_MODE
> not be used, and perhaps work to specify an optional debian/rules target
> that regenerates the build system in an appropriate way. That seems to
> provide the necessary benefits for users who need to change these files
> without imposing an unacceptable burden on developers. I don't think
> there's a good cause to go much further than that at this point.

I think this would in some respects be the worst of both worlds.  The
problem with not using AM_MAINTAINER_MODE is that the autotools *may* be
run but generally aren't.  This means that build dependencies only needed
when one modifies those files aren't present or aren't tested.  Modifying
one of those files can suddenly spark the discovery that upstream isn't
compatible with the current autotools, the partial run of Automake can
leave the whole tree in a broken state, and so forth.

But I suppose that's basically the normal argument for AM_MAINTAINER_MODE.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Bas Wijnen <wijnen@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #127 received at 397939@bugs.debian.org (full text, mbox):

From: Bas Wijnen <wijnen@debian.org>
To: debian-mentors@lists.debian.org, 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Sun, 17 Feb 2008 20:08:47 +0100
[Message part 1 (text/plain, inline)]
On Sun, Feb 17, 2008 at 03:07:59PM +0000, Colin Watson wrote:
> On Mon, Feb 11, 2008 at 10:53:48AM +0100, Bas Wijnen wrote:
> > This is not true if you simply build the whole package from source.
> > That is, run autotools during build, remove all generated files,
> > including Makefile.in, configure, etc, in the clean target.
> > 
> > For some reason many people seem to think that the whole package must be
> > built from source, except for configure and Makefile.in.  I still don't
> > understand what the idea behind this exception is.  Especially when it
> > leads to unwanted concequences (unreadable diff.gzs, for example), I
> > don't think it is a good idea to hold on to the idea that running
> > autotools is not part of building the package.
> 
> It's a well-established principle of the GNU build system that there are
> targets that are run by maintainers and there are targets to be run by
> people building the package. This division is there to make our lives
> easier, and ignoring it, IMHO, is cutting off our nose to spite our
> face. (Even the section in the GNU Coding Standards regarding the
> maintainer-clean target doesn't go so far as deleting configure.)

So according to the GCS, even the maintainer shouldn't need to
regenerate configure, or at least not by default.   This doesn't quite
prove your point, that this is a "maintainter-only" thing. ;-)

Anyway, AFAIK the reason that autoconf exists is to allow building
programs reliably, even on exotic architectures.  They may have really
weird shells, or a weird make, or libraries in weird places needing
special flags, etc.

That is a good thing.  I am happy that it is possible to build the
programs on those architectures with relatively little trouble.  One of
the essential things of autoconf there, is that it doesn't require
itself to be installed.  The generated configure script should be very
platform-independent.  It is pre-built exactly for that reason: to make
sure that the build does not require autoconf to be installed.

So far so good.  But Debian isn't an exotic system.  It's GNU with a
Linux kernel.  You can't get more standard than that, from autoconf's
(thus GNU's) point of view.  We don't need this
super-platform-independence on our system.  We have a proper shell, GNU
make, and autoconf and automake can be installed without any trouble.

In other words, there is no reason to use the pre-built files.  Their
reason for existing isn't applicable to Debian.

> The underlying problem here is that users who change configure.ac etc.
> need to do some additional manual work to update the build system, and
> that this should be done automatically. I agree. However, this is not
> the same as regenerating the full build system from scratch on every
> build.

Let's compare it with a Java program.  Would you consider it acceptable
for the packager to build it, uuencode it, put it in the diff.gz and
from debian/rules just uudecode it, instead of regenerating it?  I fail
to see the difference between this and using a pre-built configure and
Makefile.in.  Compiled Java is also platform independent (AFAIK), so
there isn't even a problem with architectures.

> You should be aware, if you aren't already, that the process of
> generating a package's build system from scratch is often really quite
> complicated

Of course it is.  That's exactly the reason that I care so much about
it.  If it would be trivial, everyone could just do it, even without
using debian/rules, or having it as an example.

> and may well involve network access, which build daemons do
> not have.

Building the package should not require network access.  The package is
built using the version which was "the newest" (I hope) when it was
created.  While it could be nice to be able to use newer versions of
things, I agree that we do not want that.  It makes builds less
reproducible, for example.

> For example, the usual update process for a package that uses
> Gnulib involves rsyncing translations for some messages from
> translationproject.org.

And so this step is not needed for the Debian package (but it would be
nice if debian/rules did actually contain a target for it).

> If you mandate that Debian maintainers must support full build system
> regeneration on every build, then you will force some of them to
> maintain a separate bootstrapping process which is guaranteed not to
> access the network, which will involve duplicating work already done
> by upstream and will undoubtedly introduce errors at some point.

While I think Gnulib is not a good example (AFAIK it contains GNU libc
functions for platforms without GNU libc, which are unused if they are
available natively), I see your point.  I think this should be uncommon,
though, and worth the trouble.  Also, if it's really too hard, it can be
decided on a per-package basis to ignore policy and do things in a way
that works.  That means there is a bug in the package, which is hard to
fix, but it doesn't mean the package can't be released.  I'm not
suggesting that breaking this rule should be a release-critical bug.
(At least not yet.  If it turns out that there are no packages (anymore)
which can't follow the rule, then we may change that.)

> Mere incremental regeneration on demand is much simpler and is often
> already supported by upstream build systems without requiring any
> change (see below).

Of course it's simpler to skip a part of the compilation process.  It's
also simpler for Java packages to use pre-built versions instead of
recompiling, but that doesn't make it right to do it.

> > I suggest to mandate "remove all generated files in the clean target"
> > (formulated in a way which includes "generated by upstream", not only
> > "generated by the build target), which implies "rebuild everything in
> > the build target".
> 
> I object to this proposal. As an example, I am upstream for man-db and
> know that its build system is properly generated. Requiring that I
> regenerate it every time I build the package would introduce nothing but
> hassle for me, and would not in practice provide significant benefits
> for users since the build system already caters for them out of the box
> (see below).

The fact that there exist packages which work properly without
recompiling from source doesn't mean it's a good default.  IMO the
default should be to always compile from source.  Yes, that means hassle
for the packager; it's pretty much the whole task of packaging.

The benefits I see are that we know things can be generated, that users
don't have to find out how to regenerate things, and what to install for
it (your package may be fine, but that doesn't mean they all are).  And
also important, that we are sure that the shipped source files are
really the source for what is generated.

> > With the current wording it is allowed to use shipped built files from
> > the upstream tarball, as long as the source is present.  It is even
> > allowed to ship the build results (uuencoded, for example) in the
> > diff.gz and use those.  I suppose we all agree that this is unacceptable
> > for normal build results.
> 
> configure is readable, if you have to. It's not the preferred form for
> modification, certainly, but it's not an opaque object file either, and
> there's no need to treat it exactly as we would treat opaque object
> files.

True, but also not much different.  It is a generated file.

> In other words, this proposal will have the effect of requiring that
> Debian maintainers become experts in the Autotools before being able
> to upload policy-compliant packages again,

Not at all.  Autoconf is pretty stable, so I suppose you're talking
about automake.  Because of its interface instability, you should never
depend on "automake", but always on "automake1.10" (or whatever version
you tested with).  It's also a good idea to force the build process to
use that version.  That requires setting two environment variables
(AUTOMAKE and ACLOCAL).  I suppose you're not suggesting that knowing to
set those makes you an autotools expert (especially when it will be
documented, for example in the developers' reference).

> whereas that burden could formerly be passed to upstream to be dealt
> with on their schedule.

When you're explicitly using a certain version of automake, this is
still the case.  There may be a bit of pushing when the automake
maintainers want to remove the version you're using, but they will wait
until you finished converting to the newer version (or help you with
that).  In fact, this makes things easier for the maintainer: if you
don't have the build-dependency recorded, your automake version may
disappear, leaving you unable to build the new version of the package
(considering worst case, where it doesn't work with the new version and
it's not easy to fix).

> I wouldn't expect that figuring out enough about the Autotools to be
> able to do this should be beyond most Debian maintainers, but neither
> would I expect that they could all deal with some of the weird stuff
> I've seen already.

If really weird stuff is needed to compile the package (from source),
then IMO it's totally reasonable that that weirdness is in debian/rules,
so users can see what's happening, if they want to.

> I agree that behavioural changes due to regenerating the build system
> are bugs, but what you're suggesting is that all such bugs be elevated
> to the status of showstoppers, as it will be impossible to upload
> packages without it.

No no, you misunderstood me.  I'm saying you should get a lintian error
(and a bug) without it, not that you can't upload, and not even that the
bug should be RC.

> I'm sure that that would make the security team's life a lot harder,
> for instance

Huh?  I would think life only gets easier for them...  Packages which do
follow the rule simply don't get uploaded if they do things wrong.  For
packages which ignore the rule, there is no difference.

Perhaps you mean that lots of bugs pop up when "automake" increases its
version?  In that case I apologize for being unclear; I thought it was
commonly known and agreed upon that you should always build-depend on
the versioned package.

> Furthermore, if as a user you're desperate to make a small tweak to
> configure.ac, it *is* often quite possible - if ugly! - to just make the
> obvious corresponding change to configure and then get on with your
> life. You can't realistically do that with e.g. compiled C object files,
> which is one reason we treat them differently.

I agree that this means it is less unacceptable to not regenerate them,
but it doesn't make it right.

Also, it is quite improper to expect this from someone who wants to NMU
the package.

> > Now reread the previous paragraph while thinking of Makefile.in instead
> > of "normal build results".  It is common practice to do exactly that:
> > ship and use pre-built versions, or even ship them in the diff.gz (which
> > then gets huge).  Makefile.am is only present as source-file, but it
> > isn't used at all during the build.  Any changes the user makes will not
> > be noticed by the build system.
> 
> But this simply isn't true across the board! Makefile.am changes are
> only ignored if you use AM_MAINTAINER_MODE.

If you don't use it, you need to do the "touch all targets in the right
order" game if you want to patch anything.  Very ugly as well, if you
ask me.

> If you don't use AM_MAINTAINER_MODE, then Makefile.am changes are
> automatically noticed and taken into account.

I know.  But then the system tries to run executables which aren't in
build-depends.  It works, but it isn't nice.  Also, as I wrote above,
you need to do the touching game if there are any patches.  If you do
this in debian/rules, the user's changes are ignored again.  If you
don't, it's hard to patch these parts in an NMU (since the touching must
be added as well).

Altogether, the behaviour is unpredictable for someone who doesn't know
what's going on.  I think it's reasonable for a user to edit any source
file and use debian/rules to build a new package with it.

> Rather than incurring the pain of gratuitous full regeneration every
> time, we just regenerate it when the user has changed something. Yes,
> the user now gets to resolve any problems that might have been
> pre-existing, but realistically either the Debian maintainer or the
> upstream maintainer is probably going to run into those at some point
> anyway.

I don't think that's very realistic at all for all cases.  For example,
if the Debian maintainer is also upstream, there's only one system.  If
on that system some weird things are installed which make things work
even though they shouldn't, there is no reason to believe he will find
out.  When building the package in pbuilder, though, it will immediately
show.  But only if the files are indeed regenerated during that build.

> I think we should recommend (but not require) that AM_MAINTAINER_MODE
> not be used,

Why?  If you consider it acceptable for a user to edit configure, it
should certainly be no problem to add --enable-maintainer-mode to the
configure invocation.

> and perhaps work to specify an optional debian/rules target that
> regenerates the build system in an appropriate way. That seems to
> provide the necessary benefits for users who need to change these
> files

Not at all.  If it's optional, it's likely that many packages will not
have it.  Also, if the build system doesn't use it by default, it is
likely that many of those targets are never tested and don't actually
work.

> without imposing an unacceptable burden on developers.

I hope you do agree that maintainers should anyway be able to build
their package from source?  The only difference is that it should be
done in debian/rules.  I don't see how this is an "unacceptable burden".

> I don't think there's a good cause to go much further than that at
> this point.

We're currently trying to allow packages to be built twice in a row as a
release goal.  If the clean target would be as I propose, this would
automatically be fulfilled as well.

Also, considering the bug we're writing to, what would you propose that
the clean target should do?  "Remove all generated files, except the
ones that are generated by autotools"?  I can't properly express how
completely wrong I find such an exception when written in Policy...


Thank you for responding.  It would be a pretty big change in Policy, if
adopted, and so it's good when there's some discussion about it, to
detect any problems it might cause.

Thanks,
Bas

-- 
I encourage people to send encrypted e-mail (see http://www.gnupg.org).
If you have problems reading my e-mail, use a better reader.
Please send the central message of e-mails as plain text
   in the message body, not as HTML and definitely not as MS Word.
Please do not use the MS Word format for attachments either.
For more information, see http://pcbcn10.phys.rug.nl/e-mail.html
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Russ Allbery <rra@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #132 received at 397939@bugs.debian.org (full text, mbox):

From: Russ Allbery <rra@debian.org>
To: debian-mentors@lists.debian.org
Cc: 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Sun, 17 Feb 2008 11:15:20 -0800
Bas Wijnen <wijnen@debian.org> writes:

> Autoconf is pretty stable,

This has not been the experience of many of us.  I haven't had a lot of
trouble fixing things for newer releases of Autoconf, but I definitely
have seen issues.  And the Autoconf 2.13 to 2.50 transition and all the
subsequent instability was not that long ago.

> so I suppose you're talking about automake.  Because of its interface
> instability, you should never depend on "automake", but always on
> "automake1.10" (or whatever version you tested with).

Just FYI:

windlord:~> apt-cache policy automake1.10
automake1.10:
  Installed: (none)
  Candidate: (none)
  Version table:
windlord:~> apt-cache policy automake
automake:
  Installed: 1:1.10.1-2
  Candidate: 1:1.10.1-2
  Version table:
     1:1.10.1-3 0
        500 http://exodus.stanford.edu unstable/main Packages
 *** 1:1.10.1-2 0
        990 http://exodus.stanford.edu testing/main Packages
        100 /var/lib/dpkg/status

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Bas Wijnen <wijnen@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #137 received at 397939@bugs.debian.org (full text, mbox):

From: Bas Wijnen <wijnen@debian.org>
To: 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Sun, 17 Feb 2008 20:38:17 +0100
[Message part 1 (text/plain, inline)]
On Sun, Feb 17, 2008 at 11:15:20AM -0800, Russ Allbery wrote:
> Bas Wijnen <wijnen@debian.org> writes:
> 
> > Autoconf is pretty stable,
> 
> This has not been the experience of many of us.  I haven't had a lot of
> trouble fixing things for newer releases of Autoconf, but I definitely
> have seen issues.

So far I haven't, but I readily believe they do exist. :-)

> And the Autoconf 2.13 to 2.50 transition and all the subsequent
> instability was not that long ago.

Ok, but I don't think anything like that should happen again soon.  If
it does become as unstable as automake, we can just use the same
versioning system in our packages.

Also, updating autoconf becomes a bit more tricky, like updating gcc
currently is.  It should all be fine, but when actually doing it, some
packages don't like it anyway.  I think this is acceptable.  There's
always the easy quick-fix of build-depending on the old version (which
should then remain in the archive).

Of course, all this is only needed if it turns out that these problems
aren't just hypothetical.

> > so I suppose you're talking about automake.  Because of its interface
> > instability, you should never depend on "automake", but always on
> > "automake1.10" (or whatever version you tested with).
> 
> Just FYI:
> 
> windlord:~> apt-cache policy automake1.10
> automake1.10:
>   Installed: (none)
>   Candidate: (none)
>   Version table:

It's a virtual package, provided by automake.  When 1.11 is released, I
suppose it becomes a real package.  Perhaps it's nicer to make
"automake" virtual, not "automake1.10", but it doesn't make much
difference IMO.

Thanks,
Bas

-- 
I encourage people to send encrypted e-mail (see http://www.gnupg.org).
If you have problems reading my e-mail, use a better reader.
Please send the central message of e-mails as plain text
   in the message body, not as HTML and definitely not as MS Word.
Please do not use the MS Word format for attachments either.
For more information, see http://pcbcn10.phys.rug.nl/e-mail.html
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Mark Brown <broonie@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #142 received at 397939@bugs.debian.org (full text, mbox):

From: Mark Brown <broonie@debian.org>
To: debian-mentors@lists.debian.org
Cc: 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Sun, 17 Feb 2008 21:29:59 +0000
On Sun, Feb 17, 2008 at 08:08:47PM +0100, Bas Wijnen wrote:

> The fact that there exist packages which work properly without
> recompiling from source doesn't mean it's a good default.  IMO the
> default should be to always compile from source.  Yes, that means hassle
> for the packager; it's pretty much the whole task of packaging.

It's probably worth bearing in mind that there are quite a few people
out there who use autotools for want of a standard alternative and are
either indifferent to it or actively dislike it.  Often people have been
burned by the version incompatbilities in the past and try to avoid
having anything to do with it - I've seen people do things like check
the generated files into their VCS to avoid dealing with the churn.  I'm
not sure how prevalent this is it is an issue.

> > In other words, this proposal will have the effect of requiring that
> > Debian maintainers become experts in the Autotools before being able
> > to upload policy-compliant packages again,

> Not at all.  Autoconf is pretty stable, so I suppose you're talking

Not as stable as, say, C by a long way...

> then IMO it's totally reasonable that that weirdness is in debian/rules,
> so users can see what's happening, if they want to.

OTOH if the standard Debian build process jumps through unusual hoops
like forcing regeneration of all the autotools files that makes it less
useful as a guide for the things you'd actually want to do in the normal
course of affairs.  I know I've found Debian packages very useful for
that when doing things like working on programs for upstream purposes or
for non-Debian things like cross development.

> > I'm sure that that would make the security team's life a lot harder,
> > for instance

> Perhaps you mean that lots of bugs pop up when "automake" increases its
> version?  In that case I apologize for being unclear; I thought it was
> commonly known and agreed upon that you should always build-depend on
> the versioned package.

If you're willing to do things by forcing a particular version in the
general case then this sounds more like something that could be checked
outside the standard package build process.

If you're keen on introducing this I'd suggest doing some work to see
how much effort would be involved in doing so.

> Not at all.  If it's optional, it's likely that many packages will not
> have it.  Also, if the build system doesn't use it by default, it is
> likely that many of those targets are never tested and don't actually
> work.

We already have regular tests for things that aren't caught by the
normal build processes, this could be checked in a similar fashion.

> Also, considering the bug we're writing to, what would you propose that
> the clean target should do?  "Remove all generated files, except the
> ones that are generated by autotools"?  I can't properly express how
> completely wrong I find such an exception when written in Policy...

Personally, I expect the package to restore things to the state they
were in the distribution tarball.

-- 
"You grabbed my hand and we fell into it, like a daydream - or a fever."




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Bas Wijnen <wijnen@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #147 received at 397939@bugs.debian.org (full text, mbox):

From: Bas Wijnen <wijnen@debian.org>
To: debian-mentors@lists.debian.org, 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Sun, 17 Feb 2008 23:55:03 +0100
[Message part 1 (text/plain, inline)]
On Sun, Feb 17, 2008 at 09:29:59PM +0000, Mark Brown wrote:
> On Sun, Feb 17, 2008 at 08:08:47PM +0100, Bas Wijnen wrote:
> 
> > The fact that there exist packages which work properly without
> > recompiling from source doesn't mean it's a good default.  IMO the
> > default should be to always compile from source.  Yes, that means hassle
> > for the packager; it's pretty much the whole task of packaging.
> 
> It's probably worth bearing in mind that there are quite a few people
> out there who use autotools for want of a standard alternative and are
> either indifferent to it or actively dislike it.  Often people have been
> burned by the version incompatbilities in the past and try to avoid
> having anything to do with it - I've seen people do things like check
> the generated files into their VCS to avoid dealing with the churn.  I'm
> not sure how prevalent this is it is an issue.

Yes, that's a good point.  For such cases, I don't mind if the package
would keep the bug for having an improper clean target open and/or
tagged upstream, because it is hard to fix.

> > > In other words, this proposal will have the effect of requiring that
> > > Debian maintainers become experts in the Autotools before being able
> > > to upload policy-compliant packages again,
> 
> > Not at all.  Autoconf is pretty stable, so I suppose you're talking
> 
> Not as stable as, say, C by a long way...

C isn't really that stable.  When we upgrade the default gcc version,
there're always some packages that break.  Of course they break on not
being proper C, but still, the problems exist.  I think there are less
problems with new autoconf versions, but I didn't check that.

> > then IMO it's totally reasonable that that weirdness is in debian/rules,
> > so users can see what's happening, if they want to.
> 
> OTOH if the standard Debian build process jumps through unusual hoops
> like forcing regeneration of all the autotools files that makes it less
> useful as a guide for the things you'd actually want to do in the normal
> course of affairs.

The things are pretty separate IME.  The parts which do the compile
after autotools have been run can easily be found in most cases.  If
not, then the file probably wasn't readable without running autotools
either. ;-)

As an example, I needed 3 extra lines and a Build-Depends for gnujump:

export ACLOCAL=aclocal-1.9 -I /usr/share/autoconf-archive
export AUTOMAKE=automake-1.9
autoreconf -f -i -s

(The last line in the build target.)
When I wrote that, I didn't know automake-1.10 was out (if it was...)
Otherwise I would have used that and tested with it.

> I know I've found Debian packages very useful for that when doing
> things like working on programs for upstream purposes or for
> non-Debian things like cross development.

Yes, I'm regularly using debian/rules as documentation as well.  I don't
usually want to check autotools stuff, but it would sure be nice if I
could.  For example, I think it is very useful in gnujump that
debian/rules (and control) shows that autoconf-archive is needed, and
how it is used, to run aclocal.

> > > I'm sure that that would make the security team's life a lot harder,
> > > for instance
> 
> > Perhaps you mean that lots of bugs pop up when "automake" increases its
> > version?  In that case I apologize for being unclear; I thought it was
> > commonly known and agreed upon that you should always build-depend on
> > the versioned package.
> 
> If you're willing to do things by forcing a particular version in the
> general case then this sounds more like something that could be checked
> outside the standard package build process.

No, I don't want to force a version, I want the package to force it.
That is, if a package requires automake-1.4, it should ask for that.
Build-depending on "automake" is dangerous, because with a new automake
release it's not at all unlikely that you have given yourself an FTBFS
bug.

> If you're keen on introducing this I'd suggest doing some work to see
> how much effort would be involved in doing so.

I'm happy to do so, but don't really see where I could start.  I would
probably want to regenerate the autotools files for packages which
currently depend on autotools-dev.  However, I think there isn't really
a standard way for that.  That's the reason I want it in debian/rules.
:-)

Or do you mean I should manually transition some packages?  I'm happy to
try some which are likely hard.  If anyone has a package which is a good
candidate (and preferably, which they don't mind running autotools at
build time, so my work is actually used), please let me know.  All my
own packages already run autotools (when they use them). :-)

> > Not at all.  If it's optional, it's likely that many packages will not
> > have it.  Also, if the build system doesn't use it by default, it is
> > likely that many of those targets are never tested and don't actually
> > work.
> 
> We already have regular tests for things that aren't caught by the
> normal build processes, this could be checked in a similar fashion.

We could check if an automake upgrade would produce many FTBFSs, if the
packages are already build-depending on automake.  However, most
packages currently don't do that, and it's in the general case not
possible (AFAICS) to run it for them automatically.

> > Also, considering the bug we're writing to, what would you propose that
> > the clean target should do?  "Remove all generated files, except the
> > ones that are generated by autotools"?  I can't properly express how
> > completely wrong I find such an exception when written in Policy...
> 
> Personally, I expect the package to restore things to the state they
> were in the distribution tarball.

That has been suggested, but it would mean backing up generated files
which get overwritten during the build (such as config.guess and
config.sub).  There seems to be agreement that such backing up is not
useful.

Thanks,
Bas

-- 
I encourage people to send encrypted e-mail (see http://www.gnupg.org).
If you have problems reading my e-mail, use a better reader.
Please send the central message of e-mails as plain text
   in the message body, not as HTML and definitely not as MS Word.
Please do not use the MS Word format for attachments either.
For more information, see http://pcbcn10.phys.rug.nl/e-mail.html
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Kurt Roeckx <kurt@roeckx.be>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #152 received at 397939@bugs.debian.org (full text, mbox):

From: Kurt Roeckx <kurt@roeckx.be>
To: debian-mentors@lists.debian.org, 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Mon, 18 Feb 2008 00:41:49 +0100
On Sun, Feb 17, 2008 at 11:55:03PM +0100, Bas Wijnen wrote:
> > > Not at all.  If it's optional, it's likely that many packages will not
> > > have it.  Also, if the build system doesn't use it by default, it is
> > > likely that many of those targets are never tested and don't actually
> > > work.
> > 
> > We already have regular tests for things that aren't caught by the
> > normal build processes, this could be checked in a similar fashion.
> 
> We could check if an automake upgrade would produce many FTBFSs, if the
> packages are already build-depending on automake.  However, most
> packages currently don't do that, and it's in the general case not
> possible (AFAICS) to run it for them automatically.

If there is some standardised rules target that can be used to
regenerate those files, it shouldn't be that hard to make the current
tools run such tests.

I'm wondering how to properly deal with different versions of for
instance automake, and at first guess I would think it should just use
the current default version.  And that would make that target useless
for people who actually build depend on some specific version.

I'm having my doubts that having a new rules target just so that
we can run archive wide tests is a good idea.  Specially if they'd
do the same thing but with different versions in different targets.


Kurt





Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Mark Brown <broonie@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #157 received at 397939@bugs.debian.org (full text, mbox):

From: Mark Brown <broonie@debian.org>
To: debian-mentors@lists.debian.org, 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Mon, 18 Feb 2008 12:47:41 +0000
On Sun, Feb 17, 2008 at 11:55:03PM +0100, Bas Wijnen wrote:
> On Sun, Feb 17, 2008 at 09:29:59PM +0000, Mark Brown wrote:
> > On Sun, Feb 17, 2008 at 08:08:47PM +0100, Bas Wijnen wrote:

> > OTOH if the standard Debian build process jumps through unusual hoops
> > like forcing regeneration of all the autotools files that makes it less
> > useful as a guide for the things you'd actually want to do in the normal
> > course of affairs.

> The things are pretty separate IME.  The parts which do the compile
> after autotools have been run can easily be found in most cases.  If
> not, then the file probably wasn't readable without running autotools
> either. ;-)



> > If you're willing to do things by forcing a particular version in the
> > general case then this sounds more like something that could be checked
> > outside the standard package build process.
> 
> No, I don't want to force a version, I want the package to force it.

By forcing a version I mean doing so in the package.

> > If you're keen on introducing this I'd suggest doing some work to see
> > how much effort would be involved in doing so.

> Or do you mean I should manually transition some packages?  I'm happy to

Yes, that's the sort of thing I meant.

> > We already have regular tests for things that aren't caught by the
> > normal build processes, this could be checked in a similar fashion.

> We could check if an automake upgrade would produce many FTBFSs, if the
> packages are already build-depending on automake.  However, most
> packages currently don't do that, and it's in the general case not
> possible (AFAICS) to run it for them automatically.

What I'm suggesting is that if you add something out of the normal build
process which regenerates autotools stuff (like an extra target in the
rules file, for example) then we could test this without doing it on
every single build.

> > Personally, I expect the package to restore things to the state they
> > were in the distribution tarball.

> That has been suggested, but it would mean backing up generated files
> which get overwritten during the build (such as config.guess and
> config.sub).  There seems to be agreement that such backing up is not
> useful.

My point is that I don't expect the clean target to clean with respect
to anything except the upstream tarball rather than going around making
things clean with respect to upstream revision control or similar.

-- 
"You grabbed my hand and we fell into it, like a daydream - or a fever."




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Bas Wijnen <wijnen@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #162 received at 397939@bugs.debian.org (full text, mbox):

From: Bas Wijnen <wijnen@debian.org>
To: debian-mentors@lists.debian.org, 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Mon, 18 Feb 2008 17:03:24 +0100
[Message part 1 (text/plain, inline)]
On Mon, Feb 18, 2008 at 12:47:41PM +0000, Mark Brown wrote:
> On Sun, Feb 17, 2008 at 11:55:03PM +0100, Bas Wijnen wrote:
> > On Sun, Feb 17, 2008 at 09:29:59PM +0000, Mark Brown wrote:
> > > If you're willing to do things by forcing a particular version in the
> > > general case then this sounds more like something that could be checked
> > > outside the standard package build process.
> > 
> > No, I don't want to force a version, I want the package to force it.
> 
> By forcing a version I mean doing so in the package.

Then I still don't understand your statement above.  What is the thing
that you prefer to check outside the normal build process?

> > > If you're keen on introducing this I'd suggest doing some work to see
> > > how much effort would be involved in doing so.
> 
> > Or do you mean I should manually transition some packages?  I'm happy to
> 
> Yes, that's the sort of thing I meant.

Ok, I'll see what I can do.  I repeat my request for packages which may
be hard to transition.  If I get none of those, I'll just pick some
random packages from the archive which currently build-depend on
autotools-dev.

It would be nice to have some consensus that my transition efforts are
not wasted though.  So I have a question:

Does everyone agree that it would in theory be better to run autotools
during the build process?  In other words, if you don't have to do
anything to your packages for it, would you have a problem with this?

("you" above is addressed to anyone reading this.)

> > > We already have regular tests for things that aren't caught by the
> > > normal build processes, this could be checked in a similar fashion.
> 
> > We could check if an automake upgrade would produce many FTBFSs, if the
> > packages are already build-depending on automake.  However, most
> > packages currently don't do that, and it's in the general case not
> > possible (AFAICS) to run it for them automatically.
> 
> What I'm suggesting is that if you add something out of the normal build
> process which regenerates autotools stuff (like an extra target in the
> rules file, for example) then we could test this without doing it on
> every single build.

I still consider "not building from source" to be a Bad Thing, and
therefore I think this is the wrong approach.

The best argument I've heard so far, is "we care more about other bugs,
and we want to be able to ignore those".  However, I don't really think
this is a big problem.  The program was tested with the same automake
version that it is built with.  Why would it suddenly FTBFS?  I can see
that it might do so while doing the transition to running autotools
during the build.  But that's the maintainer's problem.  They can take
their time (or ask me for help :-) ).  It's not a big problem if they
delay following my proposed rule.  Once they have changed their rules
files, things should just keep working.

And if it doesn't, a dirty workaround of not running autotools can
always be installed, if fixing whatever problem does come up seems to be
too hard.

Build-depending on versioned automake doesn't look really nice, though.
This is how it currently should be done, AFAIK, but it might be better
to recommend against it.  However, in that case great care must be taken
when increasing its version, similar to increasing the default gcc
version.

All this can easily be tested before actually starting the transition,
though, just like with gcc.  Bugs can be filed and fixed to make
packages work with the newer automake before it becomes the default.
Transitions may be some work, but they shouldn't result in too much
breakage.  And if the RMs don't like it, and want us to spend time on
other bugs, they can just say "the newer automake will not be the
default for the next release, therefore bugs with it are not RC".

> > > Personally, I expect the package to restore things to the state they
> > > were in the distribution tarball.
> 
> > That has been suggested, but it would mean backing up generated files
> > which get overwritten during the build (such as config.guess and
> > config.sub).  There seems to be agreement that such backing up is not
> > useful.
> 
> My point is that I don't expect the clean target to clean with respect
> to anything except the upstream tarball rather than going around making
> things clean with respect to upstream revision control or similar.

So you don't want to remove files which are unchanged during the build.
Do you agree on removing all files which were changed though?  I think
even that would require rerunning of some autotools parts in some cases,
but I'm not sure.

And just that statement, "all changed files should be removed by the
clean target", is enough to fix the bug we're talking about.  This rule
automatically makes two builds in a row give identical results (except
for time stamps).

Of course this is a separate point.  IMO clean should remove any file
which was changed during the build.  And secondly, I think build should
regenerate everything it can.  Combined, these can be formulated as
"clean should remove all non-source files", because every shipped
non-source file must be updated (and thus changed) by the build.

Thanks,
Bas

-- 
I encourage people to send encrypted e-mail (see http://www.gnupg.org).
If you have problems reading my e-mail, use a better reader.
Please send the central message of e-mails as plain text
   in the message body, not as HTML and definitely not as MS Word.
Please do not use the MS Word format for attachments either.
For more information, see http://pcbcn10.phys.rug.nl/e-mail.html
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Chris Waters <xtifr@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #167 received at 397939@bugs.debian.org (full text, mbox):

From: Chris Waters <xtifr@debian.org>
To: debian-mentors@lists.debian.org, 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Mon, 18 Feb 2008 12:39:29 -0800
On Sun, Feb 17, 2008 at 08:08:47PM +0100, Bas Wijnen wrote:
> Let's compare it with a Java program.  Would you consider it acceptable
> for the packager to build it, uuencode it, put it in the diff.gz and
> from debian/rules just uudecode it, instead of regenerating it?

Well, I see one big difference.  I often get patches from downstream
to configure.  I, of course, make sure to apply them to the
appropriate .am (or whatever), as well as forwarding them upstream.
But to me, this indicates that downstream often considers the
configure file to be a readable source format.  This cannot be said of
a uuencoded binary.  I think that's an important distinction.

Whether that distinction is sufficient to justify a different set of
rules, I reserve judgement on at this time.

But honestly, I think our job is to deliver full source and binaries.
I _don't_ think we necessarily have to exercise every bit of the
source (e.g. the .am files) on every build.  In fact, my primary
objections to the java example would be a) that it confounds user
expectations, and b) that it would result in huge diffs.  I'm not sure
that either of those objections would apply to the autoconf case.

> The fact that there exist packages which work properly without
> recompiling from source doesn't mean it's a good default.  IMO the
> default should be to always compile from source.  Yes, that means hassle
> for the packager; it's pretty much the whole task of packaging.

I think there's a big difference between recompiling from source as an
end user would do and (re)generating _everything_ as an upstream might
do.  I suspect the ultimate question here is: does Debian serve as a)
a proxy for the user, generating binaries so they don't have to, or b)
as a proxy for upstream?  I tend to lean towards the former position;
it sounds like you lean towards the latter.

Bottom line: it sounds like you think the java example is
fundamentally wrong; I merely see it as flawed, awkward and hard to
maintain: a bad idea in general, but not necessarily wrong.

-- 
Chris Waters           |  Pneumonoultra-        osis is too long
xtifr@debian.org       |  microscopicsilico-    to fit into a single
or xtifr@speakeasy.net |  volcaniconi-          standalone haiku




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Mark Brown <broonie@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #172 received at 397939@bugs.debian.org (full text, mbox):

From: Mark Brown <broonie@debian.org>
To: debian-mentors@lists.debian.org
Cc: 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Mon, 18 Feb 2008 22:14:24 +0000
On Mon, Feb 18, 2008 at 05:03:24PM +0100, Bas Wijnen wrote:
> On Mon, Feb 18, 2008 at 12:47:41PM +0000, Mark Brown wrote:
> > On Sun, Feb 17, 2008 at 11:55:03PM +0100, Bas Wijnen wrote:

> > > No, I don't want to force a version, I want the package to force it.

> > By forcing a version I mean doing so in the package.

> Then I still don't understand your statement above.  What is the thing
> that you prefer to check outside the normal build process?

That we can regenerate the autotools products.

> not wasted though.  So I have a question:

> Does everyone agree that it would in theory be better to run autotools
> during the build process?  In other words, if you don't have to do
> anything to your packages for it, would you have a problem with this?

If I didn't have to do anything - but the maintainer is at least going
to have to upload changes to run autotools.

> Build-depending on versioned automake doesn't look really nice, though.
> This is how it currently should be done, AFAIK, but it might be better
> to recommend against it.  However, in that case great care must be taken
> when increasing its version, similar to increasing the default gcc
> version.

Of course, doing this introduces all the work that was causing people to
raise concerns about this...

> Of course this is a separate point.  IMO clean should remove any file
> which was changed during the build.  And secondly, I think build should
> regenerate everything it can.  Combined, these can be formulated as
> "clean should remove all non-source files", because every shipped
> non-source file must be updated (and thus changed) by the build.

Right, half the thing for me is that I don't see this as being something
that we need to check on each and every single build.

-- 
"You grabbed my hand and we fell into it, like a daydream - or a fever."




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Bas Wijnen <wijnen@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #177 received at 397939@bugs.debian.org (full text, mbox):

From: Bas Wijnen <wijnen@debian.org>
To: debian-mentors@lists.debian.org, 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Tue, 19 Feb 2008 10:50:23 +0100
[Message part 1 (text/plain, inline)]
On Mon, Feb 18, 2008 at 12:39:29PM -0800, Chris Waters wrote:
> But honestly, I think our job is to deliver full source and binaries.
> I _don't_ think we necessarily have to exercise every bit of the
> source (e.g. the .am files) on every build.  In fact, my primary
> objections to the java example would be a) that it confounds user
> expectations, and b) that it would result in huge diffs.  I'm not sure
> that either of those objections would apply to the autoconf case.

At least the huge diff does, that's one of the documented (in
autotools-dev's README.Debian) downsides of not running autotools during
package build.

> > The fact that there exist packages which work properly without
> > recompiling from source doesn't mean it's a good default.  IMO the
> > default should be to always compile from source.  Yes, that means hassle
> > for the packager; it's pretty much the whole task of packaging.
> 
> I think there's a big difference between recompiling from source as an
> end user would do and (re)generating _everything_ as an upstream might
> do.  I suspect the ultimate question here is: does Debian serve as a)
> a proxy for the user, generating binaries so they don't have to, or b)
> as a proxy for upstream?  I tend to lean towards the former position;
> it sounds like you lean towards the latter.

Probably, although I'm not sure who we're proxying from and to then. ;-)

I think providing binary packages which work to users is certainly a
very important part of Debian.  However, that's not everything.  We also
want to keep it free, and one important aspect of that is that we
provide source packages which work as well.

To me, a "working" source package is the complete source plus build
rules (which work ;-) ) for that source.  And that's all the source, not
just a part of it.

Of course, the suggestion to add a debian/rules target which does this
which doesn't have to be run during the normal build process would work
for this as well.  One problem I have with that is that it'll be much
less tested.  This can be fixed by running automated testing, as also
suggested, but I think it will still mean that we will have packages
which can't be compiled from source.  Another problem is that the normal
build rules will still generate a huge and unreadable diff.gz.

I would therefore still prefer to build everything from source.  However
a recommended (or even mandatory) extra target to at least be able to do
it would be acceptable to me.  That doesn't solve the "what should the
clean target do exactly" problem, though.  Is "remove all files which
have been changed during the build" an acceptable answer to that?  If
that means autotools must be rerun during normal build (which I think
will be the case sometimes), is that acceptable to most people?

As a sidestep, I think this target may actually be legally required for
GPL (at least 2 and 3) licenced code.  They say

	For an executable work, complete source code means all the
	source code for all modules it contains, plus any associated
	interface definition files, plus the scripts used to control
	compilation and installation of the executable.
(version 2), and
	The "Corresponding Source" for a work in object code form means
	all the source code needed to generate, install, and (for an
	executable work) run the object code and to modify the work,
	including scripts to control those activities.
(version 3).

In other words, build rules to generate the binary from source must be
present.

If upstream doesn't normally ask from users that they regenerate all
files, that doesn't mean that we should make it as hard for the users as
well.  Because that's the main result of not running autotools IMO: it
makes it harder for the user to make use of the sources.  I think this
also has to do with:

> Well, I see one big difference.  I often get patches from downstream
> to configure.

Perhaps this is because when running debian/rules that's the file that
is used as the source.  If any other file would be changed, they should
include instructions for how that change can be made part of the
package.  (In the simplest case a list of extra build-depends.)

> But to me, this indicates that downstream often considers the
> configure file to be a readable source format.  This cannot be said of
> a uuencoded binary.  I think that's an important distinction.

I do agree with this, though. :-)

> Bottom line: it sounds like you think the java example is
> fundamentally wrong;

Indeed.

> I merely see it as flawed, awkward and hard to maintain: a bad idea in
> general, but not necessarily wrong.

Interesting...  I didn't expect Debian people to disagree with me on
that...  (Just to be clear, that's saying something about my
expectations, not about your opinion being wrong or anything. ;-) )

Thanks,
Bas

-- 
I encourage people to send encrypted e-mail (see http://www.gnupg.org).
If you have problems reading my e-mail, use a better reader.
Please send the central message of e-mails as plain text
   in the message body, not as HTML and definitely not as MS Word.
Please do not use the MS Word format for attachments either.
For more information, see http://pcbcn10.phys.rug.nl/e-mail.html
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Bas Wijnen <wijnen@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #182 received at 397939@bugs.debian.org (full text, mbox):

From: Bas Wijnen <wijnen@debian.org>
To: debian-mentors@lists.debian.org, 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Tue, 19 Feb 2008 10:59:10 +0100
[Message part 1 (text/plain, inline)]
On Mon, Feb 18, 2008 at 10:14:24PM +0000, Mark Brown wrote:
> > Then I still don't understand your statement above.  What is the thing
> > that you prefer to check outside the normal build process?
> 
> That we can regenerate the autotools products.

I answered this in another reply.  Sorry for not merging it with this
one.

> > Does everyone agree that it would in theory be better to run autotools
> > during the build process?  In other words, if you don't have to do
> > anything to your packages for it, would you have a problem with this?
> 
> If I didn't have to do anything - but the maintainer is at least going
> to have to upload changes to run autotools.

What I mean is that I make the required changes to the packages and send
patches to everyone.  All the maintainers need to do then is apply the
patch (and maintain the result, but I'm also happy to help with that).

> > Build-depending on versioned automake doesn't look really nice, though.
> > This is how it currently should be done, AFAIK, but it might be better
> > to recommend against it.  However, in that case great care must be taken
> > when increasing its version, similar to increasing the default gcc
> > version.
> 
> Of course, doing this introduces all the work that was causing people to
> raise concerns about this...

Yes, and I share those concerns, which is why I didn't recommend this.
However, thinking more about it, it really is the Right Thing.  And when
doing it the same way as we handle gcc, I think it shouldn't be causing
too much trouble, even.

> > Of course this is a separate point.  IMO clean should remove any file
> > which was changed during the build.  And secondly, I think build should
> > regenerate everything it can.  Combined, these can be formulated as
> > "clean should remove all non-source files", because every shipped
> > non-source file must be updated (and thus changed) by the build.
> 
> Right, half the thing for me is that I don't see this as being something
> that we need to check on each and every single build.

If we build separate infrastructure to test it, it would likely also try
to do this for every upload.  And preferrably on different (or even all)
architectures we support.  So if we make this whole extra check work
right, it isn't actually costing any less computing time.  Assuming that
is what you have against doing it on every build, that is...

Thanks,
Bas

-- 
I encourage people to send encrypted e-mail (see http://www.gnupg.org).
If you have problems reading my e-mail, use a better reader.
Please send the central message of e-mails as plain text
   in the message body, not as HTML and definitely not as MS Word.
Please do not use the MS Word format for attachments either.
For more information, see http://pcbcn10.phys.rug.nl/e-mail.html
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Kurt Roeckx <kurt@roeckx.be>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #187 received at 397939@bugs.debian.org (full text, mbox):

From: Kurt Roeckx <kurt@roeckx.be>
To: debian-mentors@lists.debian.org, 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Tue, 19 Feb 2008 23:22:04 +0100
On Tue, Feb 19, 2008 at 10:50:23AM +0100, Bas Wijnen wrote:
> As a sidestep, I think this target may actually be legally required for
> GPL (at least 2 and 3) licenced code.  They say
> 
> 	For an executable work, complete source code means all the
> 	source code for all modules it contains, plus any associated
> 	interface definition files, plus the scripts used to control
> 	compilation and installation of the executable.
> (version 2), and
> 	The "Corresponding Source" for a work in object code form means
> 	all the source code needed to generate, install, and (for an
> 	executable work) run the object code and to modify the work,
> 	including scripts to control those activities.
> (version 3).
> 
> In other words, build rules to generate the binary from source must be
> present.

The GPL does not require us to build anything.  Just that the source
are available.

I would also like to point out this exception from the autoconf license:

| As a special exception, the Free Software Foundation gives unlimited
| permission to copy, distribute and modify the configure scripts that
| are the output of Autoconf.  You need not follow the terms of the GNU
| General Public License when using or distributing such scripts, even
| though portions of the text of Autoconf appear in them.  The GNU
| General Public License (GPL) does govern all other use of the material
| that constitutes the Autoconf program.


Kurt





Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Mark Brown <broonie@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #192 received at 397939@bugs.debian.org (full text, mbox):

From: Mark Brown <broonie@debian.org>
To: debian-mentors@lists.debian.org, 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Tue, 19 Feb 2008 23:00:55 +0000
On Tue, Feb 19, 2008 at 10:59:10AM +0100, Bas Wijnen wrote:

> If we build separate infrastructure to test it, it would likely also try
> to do this for every upload.  And preferrably on different (or even all)
> architectures we support.  So if we make this whole extra check work
> right, it isn't actually costing any less computing time.  Assuming that
> is what you have against doing it on every build, that is...

It's not just the computing resources required that concern me, it's
also the effort involved in doing it and the disruption that could be
caused, especially if we were to do things like changing autotools
versions underneath the package.

-- 
"You grabbed my hand and we fell into it, like a daydream - or a fever."




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. Full text and rfc822 format available.

Acknowledgement sent to Bas Wijnen <wijnen@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. Full text and rfc822 format available.

Message #197 received at 397939@bugs.debian.org (full text, mbox):

From: Bas Wijnen <wijnen@debian.org>
To: debian-mentors@lists.debian.org
Cc: 397939@bugs.debian.org
Subject: Re: Bug#397939: Lintian: outdated-autotools-helper-file
Date: Wed, 20 Feb 2008 13:22:14 +0100
[Message part 1 (text/plain, inline)]
On Tue, Feb 19, 2008 at 11:00:55PM +0000, Mark Brown wrote:
> It's not just the computing resources required that concern me, it's
> also the effort involved in doing it and the disruption that could be
> caused, especially if we were to do things like changing autotools
> versions underneath the package.

I think by doing things the gcc-way, there shouldn't be much trouble at
all, and as far as there is, we can delay it until we have time (by
waiting with the upgrade).

But if this is a concern for many people, it is of course possible to do
it in a separate (but not optional, IMO) target.  It should be defined
in a way which allows easy building of the binary package from real
source.  (This should be the normal situation if you call it before
dpkg-buildpackage.  However, it should be mandated that this really
works, and that the normal build doesn't still use some other generated
files.)

Thanks,
Bas

-- 
I encourage people to send encrypted e-mail (see http://www.gnupg.org).
If you have problems reading my e-mail, use a better reader.
Please send the central message of e-mails as plain text
   in the message body, not as HTML and definitely not as MS Word.
Please do not use the MS Word format for attachments either.
For more information, see http://pcbcn10.phys.rug.nl/e-mail.html
[signature.asc (application/pgp-signature, inline)]

Changed Bug title to `clean rule behavior underspecified and inconsistent with common practice' from `Proposal: Packages must have a working clean target'. Request was from Russ Allbery <rra@debian.org> to control@bugs.debian.org. (Mon, 17 Mar 2008 05:24:23 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. (Tue, 11 Sep 2012 03:45:06 GMT) Full text and rfc822 format available.

Acknowledgement sent to Jonathan Nieder <jrnieder@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Tue, 11 Sep 2012 03:45:06 GMT) Full text and rfc822 format available.

Message #204 received at 397939@bugs.debian.org (full text, mbox):

From: Jonathan Nieder <jrnieder@gmail.com>
To: 397939@bugs.debian.org
Cc: Martin Zobel-Helas <zobel@ftbfs.de>
Subject: Re: clean rule behavior underspecified and inconsistent with common practice
Date: Mon, 10 Sep 2012 20:41:36 -0700
Hi,

In 2006, Martin Zobel-Helas wrote:

> during the last months i had to review several packages. Quite a number
> of packages were not buildable two times (eg. "unrepresentable changes
> to source"). Most of these packages used svn-buildpackage or
> cvs-buildpackage. This bug is quite annoying as one needs to either
> manual interact or run dpkg-source -x again.

Of course policy forbids that.  The requirements in policy for
"debian/rules clean" are very stringent --- to avoid the
"unrepresentable changes" it would be enough to _remove_ the modified
(regenerated) files, but policy requires undoing everything the build
target did, or in other words restoring the original files.

In 2008, Henrique de Moraes Holschuh wrote:
> On Mon, 11 Feb 2008, Kapil Hari Paranjape wrote:

>> Note that if the upstream's auto-generated files are deleted during
>> the clean target, then the source *must* be re-packaged to avoid
>> needless clutter in the .diff.gz which is of a "negative" nature.
>
> Not so.  Deletions are ignored.  Ever tried it?

I rely on this when working with packages that use autotools, even
though it is against policy.

Would it make sense to add this exception (removal of files present in
the upstream tarball in the clean target to be ok as long as it
doesn't affect the build) to policy?  Would that be close enough to
the purpose of this bug to discuss here or should it be filed as a
separate proposal?

Thanks,
Jonathan



Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. (Tue, 11 Sep 2012 12:12:09 GMT) Full text and rfc822 format available.

Acknowledgement sent to "Bernhard R. Link" <brlink@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Tue, 11 Sep 2012 12:12:09 GMT) Full text and rfc822 format available.

Message #209 received at 397939@bugs.debian.org (full text, mbox):

From: "Bernhard R. Link" <brlink@debian.org>
To: 397939@bugs.debian.org
Subject: Re: Bug#397939: clean rule behavior underspecified and inconsistent with common practice
Date: Tue, 11 Sep 2012 13:35:45 +0200
* Jonathan Nieder <jrnieder@gmail.com> [120911 05:45]:
> > during the last months i had to review several packages. Quite a number
> > of packages were not buildable two times (eg. "unrepresentable changes
> > to source"). Most of these packages used svn-buildpackage or
> > cvs-buildpackage. This bug is quite annoying as one needs to either
> > manual interact or run dpkg-source -x again.
> 
> Of course policy forbids that.  The requirements in policy for
> "debian/rules clean" are very stringent --- to avoid the
> "unrepresentable changes" it would be enough to _remove_ the modified
> (regenerated) files, but policy requires undoing everything the build
> target did, or in other words restoring the original files.

I disagree.

Policy says:

| This must undo any effects that the build and binary targets may have
| had, except that it should leave alone any output files created in the
| parent directory by a run of a binary target.

It does not do it must undo "everything". Undoing everything would be
impossible (like, how do you revert the timestamps of directories that
got a newer timestamp because there was a file created and then removed
in there?).

Policy only speaks about the "effects" those targets had.

And I think common understanding of this was (at least was in the past)
that removing files not needed for the build is a simple and effective
way to undo those effects, as it results in a working dir aquivalent
for all practical purposes to one where build and binary never ran.

        Bernhard R. Link



Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. (Tue, 11 Sep 2012 13:45:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Jonathan Nieder <jrnieder@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Tue, 11 Sep 2012 13:45:03 GMT) Full text and rfc822 format available.

Message #214 received at 397939@bugs.debian.org (full text, mbox):

From: Jonathan Nieder <jrnieder@gmail.com>
To: "Bernhard R. Link" <brlink@debian.org>
Cc: 397939@bugs.debian.org
Subject: Re: clean rule behavior underspecified and inconsistent with common practice
Date: Tue, 11 Sep 2012 06:44:28 -0700
Bernhard R. Link wrote:
> * Jonathan Nieder <jrnieder@gmail.com> [120911 05:45]:

>>                                 The requirements in policy for
>> "debian/rules clean" are very stringent --- to avoid the
>> "unrepresentable changes" it would be enough to _remove_ the modified
>> (regenerated) files, but policy requires undoing everything the build
>> target did, or in other words restoring the original files.
[...]
> It does not do it must undo "everything". Undoing everything would be
> impossible (like, how do you revert the timestamps of directories that
> got a newer timestamp because there was a file created and then removed
> in there?).
>
> Policy only speaks about the "effects" those targets had.
>
> And I think common understanding of this was (at least was in the past)
> that removing files not needed for the build is a simple and effective
> way to undo those effects, as it results in a working dir aquivalent
> for all practical purposes to one where build and binary never ran.

I'm happy to hear that, though I don't see how the wording in policy
can support it.  Perhaps a simple footnote that mentions that adding
or modifying files is not allowed but removing them is allowed would
take care of that distraction, then?

Thanks,
Jonathan



Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. (Tue, 11 Sep 2012 14:27:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Henrique de Moraes Holschuh <hmh@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Tue, 11 Sep 2012 14:27:03 GMT) Full text and rfc822 format available.

Message #219 received at 397939@bugs.debian.org (full text, mbox):

From: Henrique de Moraes Holschuh <hmh@debian.org>
To: Jonathan Nieder <jrnieder@gmail.com>, 397939@bugs.debian.org
Cc: "Bernhard R. Link" <brlink@debian.org>
Subject: Re: Bug#397939: clean rule behavior underspecified and inconsistent with common practice
Date: Tue, 11 Sep 2012 11:25:06 -0300
On Tue, 11 Sep 2012, Jonathan Nieder wrote:
> Bernhard R. Link wrote:
> > * Jonathan Nieder <jrnieder@gmail.com> [120911 05:45]:
> >>                                 The requirements in policy for
> >> "debian/rules clean" are very stringent --- to avoid the
> >> "unrepresentable changes" it would be enough to _remove_ the modified
> >> (regenerated) files, but policy requires undoing everything the build
> >> target did, or in other words restoring the original files.
> [...]
> > It does not do it must undo "everything". Undoing everything would be
> > impossible (like, how do you revert the timestamps of directories that
> > got a newer timestamp because there was a file created and then removed
> > in there?).
> >
> > Policy only speaks about the "effects" those targets had.
> >
> > And I think common understanding of this was (at least was in the past)
> > that removing files not needed for the build is a simple and effective
> > way to undo those effects, as it results in a working dir aquivalent
> > for all practical purposes to one where build and binary never ran.
> 
> I'm happy to hear that, though I don't see how the wording in policy
> can support it.  Perhaps a simple footnote that mentions that adding
> or modifying files is not allowed but removing them is allowed would
> take care of that distraction, then?

Yes, but as soon as you try that, you will get a bunch of people who
considers that Debian source package trees must at all times work as if it
were an upstream package tree (i.e. not depend on debian/rules targets to
work) to voice in the thread against any non-reversible changes during
build, and the thread will rapidly either die, or degenerate into
dead-horse-beating noise.

Nowadays we could actually do it, in a faint hope that the thread will
actually come to a rough consensus of some sort, and if it doesn't, punt it
to the ctte.  But this is *really* not worth the hassle IMO.

Still, if any of you would like to do it, please wait a bit.  Let's release
Wheezy first.

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh



Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. (Tue, 11 Sep 2012 14:39:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Jonathan Nieder <jrnieder@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Tue, 11 Sep 2012 14:39:03 GMT) Full text and rfc822 format available.

Message #224 received at 397939@bugs.debian.org (full text, mbox):

From: Jonathan Nieder <jrnieder@gmail.com>
To: Henrique de Moraes Holschuh <hmh@debian.org>
Cc: 397939@bugs.debian.org, "Bernhard R. Link" <brlink@debian.org>
Subject: Re: clean rule behavior underspecified and inconsistent with common practice
Date: Tue, 11 Sep 2012 07:37:37 -0700
Henrique de Moraes Holschuh wrote:

> Still, if any of you would like to do it, please wait a bit.  Let's release
> Wheezy first.

I agree that we should not upload any change along these lines to
policy in the near future.  I disagree that we should not find a
consensus about correct behavior and get the wording ready.



Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. (Tue, 11 Sep 2012 15:12:06 GMT) Full text and rfc822 format available.

Acknowledgement sent to Henrique de Moraes Holschuh <hmh@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Tue, 11 Sep 2012 15:12:06 GMT) Full text and rfc822 format available.

Message #229 received at 397939@bugs.debian.org (full text, mbox):

From: Henrique de Moraes Holschuh <hmh@debian.org>
To: Jonathan Nieder <jrnieder@gmail.com>, 397939@bugs.debian.org
Cc: "Bernhard R. Link" <brlink@debian.org>
Subject: Re: Bug#397939: clean rule behavior underspecified and inconsistent with common practice
Date: Tue, 11 Sep 2012 12:09:51 -0300
On Tue, 11 Sep 2012, Jonathan Nieder wrote:
> Henrique de Moraes Holschuh wrote:
> > Still, if any of you would like to do it, please wait a bit.  Let's release
> > Wheezy first.
> 
> I agree that we should not upload any change along these lines to
> policy in the near future.  I disagree that we should not find a
> consensus about correct behavior and get the wording ready.

It will be distracting, and the current status-quo is non-damaging in that
it is not in practice limiting anyone from either using non-reversible or
reversible methods.

IMO we won't benefit from engaging project resources on this matter right
now instead of in four to six months.

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh



Information forwarded to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#397939; Package debian-policy. (Thu, 13 Sep 2012 23:09:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Charles Plessy <plessy@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>. (Thu, 13 Sep 2012 23:09:03 GMT) Full text and rfc822 format available.

Message #234 received at 397939@bugs.debian.org (full text, mbox):

From: Charles Plessy <plessy@debian.org>
To: 397939@bugs.debian.org
Cc: Martin Zobel-Helas <zobel@ftbfs.de>
Subject: Re: Bug#397939: clean rule behavior underspecified and inconsistent with common practice
Date: Fri, 14 Sep 2012 08:04:37 +0900
> In 2006, Martin Zobel-Helas wrote:
> 
> > during the last months i had to review several packages. Quite a number
> > of packages were not buildable two times (eg. "unrepresentable changes
> > to source"). Most of these packages used svn-buildpackage or
> > cvs-buildpackage. This bug is quite annoying as one needs to either
> > manual interact or run dpkg-source -x again.
 
Le Mon, Sep 10, 2012 at 08:41:36PM -0700, Jonathan Nieder a écrit :
> 
> Of course policy forbids that.  The requirements in policy for
> "debian/rules clean" are very stringent --- to avoid the
> "unrepresentable changes" it would be enough to _remove_ the modified
> (regenerated) files, but policy requires undoing everything the build
> target did, or in other words restoring the original files.

Hello everybody,

I also think that the Policy is hard to apply as it is written and does not
correspond to current practice.  First, it is hard for a Debian build system to
distinguish between a change introduced by the developer and a change
introduced by the upstream build system.  Second, the practice of deleting an
autogenerated file to cancel changes is widesperead but not recognised by the
Policy.

If we think about the purpose instead of the wording, things are clearer.  It
has been since Lenny at least that packages to be released in Stable are
expected to be able to be built twice in a row.

How about solving #397939 by writing that the clean target must ensure that
there are no more differences between two builds in a row, compared with two
builds in two fresh source trees ?  Details about reverting build system
effects and deleting files can then go in a footnote.

Have a nice day,

-- 
Charles Plessy
Tsurumi, Kanagawa, Japan



Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Mon Apr 21 02:35:47 2014; Machine Name: beach.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.