Debian Bug report logs - #471263
[checks/patch-systems] suppress patch-system-but-direct-changes-in-diff for generated files

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Daniel Leidert <daniel.leidert@wgdd.de>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: [patch-systems]: please no patch-system-but-direct-changes-in-diff for generated files

Date: Mon, 17 Mar 2008 00:31:36 +0100

Package: lintian
Version: 1.23.46
Severity: wishlist

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Please consider docbook-defguide and docbook-xml/docbook-simple as
examples. In these packages I created a build-system (docbook-defguide)
or catalogs (docbook-xml, docbook-simple). Because I use a patch system
to change upstream sources, lintian complains about the fact, that I do
not create these files by patch.

It simply doesn't make sense to create these files by patch. lintian
should only warn, if changes are really changes to an existing file, but
not if "change" means, that a file has been created.

Would you agree to this? Is it possible to exclude created files from
this warning? I would further vote for excluding
changes.{sub,guess,rpath} and maybe even Makefiles and/or configure
scripts from this test. However, this is hust an optional suggestion.

Regards, Daniel


- -- System Information:
Debian Release: lenny/sid
  APT prefers unstable
  APT policy: (850, 'unstable'), (700, 'testing'), (550, 'stable'), (110, 'experimental')
Architecture: i386 (i686)

Kernel: Linux 2.6.22-3-k7 (SMP w/1 CPU core)
Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash

Versions of packages lintian depends on:
ii  binutils            2.18.1~cvs20080103-1 The GNU assembler, linker and bina
ii  diffstat            1.45-2               produces graph of changes introduc
ii  dpkg-dev            1.14.16.6            package building tools for Debian
ii  file                4.23-2               Determines file type using "magic"
ii  gettext             0.17-2               GNU Internationalization utilities
ii  intltool-debian     0.35.0+20060710.1    Help i18n of RFC822 compliant conf
ii  libparse-debianchan 1.1.1-2              parse Debian changelogs and output
ii  liburi-perl         1.35.dfsg.1-1        Manipulates and accesses URI strin
ii  man-db              2.5.1-3              on-line manual pager
ii  perl [libdigest-md5 5.8.8-12             Larry Wall's Practical Extraction 

lintian recommends no packages.

- -- no debconf information

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFH3a3Ym0bx+wiPa4wRAjTTAKC2LkWSEHDHbecv4Tmuf/VObnKydwCePc3k
oyhr484WQfCGJAF1/KFxe7U=
=hDTQ
-----END PGP SIGNATURE-----

Message #10 received at 471263@bugs.debian.org (full text, mbox, reply):

From: Russ Allbery <rra@debian.org>

To: Daniel Leidert <daniel.leidert@wgdd.de>

Cc: 471263@bugs.debian.org

Subject: Re: Bug#471263: [patch-systems]: please no patch-system-but-direct-changes-in-diff for generated files

Date: Sun, 16 Mar 2008 17:07:21 -0700

Daniel Leidert <daniel.leidert@wgdd.de> writes:

> Package: lintian
> Version: 1.23.46
> Severity: wishlist
>
> Please consider docbook-defguide and docbook-xml/docbook-simple as
> examples. In these packages I created a build-system (docbook-defguide)
> or catalogs (docbook-xml, docbook-simple). Because I use a patch system
> to change upstream sources, lintian complains about the fact, that I do
> not create these files by patch.
>
> It simply doesn't make sense to create these files by patch.

Why not?

> lintian should only warn, if changes are really changes to an existing
> file, but not if "change" means, that a file has been created.

I can see why you did it the way that you did, but I don't see any clear
advantages over doing it that way versus creating the build files in a
dpatch.  It seems like about the same thing to me.  Not warning about
newly created files would miss some of the things that this check is
designed to look for (basically, editing the package without being aware
that there's an existing patch system).  If a patch created a file and
that file was also shipped in the diff, you end up putting the patch
system in an inconsistent state.

Part of the problem is that people use patch systems in a wide variety of
sometimes odd ways.  To some extent, lintian is intentionally being
conservative right now about what uses it's willing to be quiet about,
although I don't know if we'll be able to stick with that in the long run.

> Would you agree to this? Is it possible to exclude created files from
> this warning? I would further vote for excluding
> changes.{sub,guess,rpath} and maybe even Makefiles and/or configure
> scripts from this test. However, this is hust an optional suggestion.

I personally think such changes should be made by running the relevant
Autotools at build time, but I realize this is an ongoing debate and
that's far from the consensus at the moment.  I think we're still trying
to feel out what Lintian's role should be here.  I'd welcome any feedback
from other people reading the mailing list.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>

Message #15 received at 471263@bugs.debian.org (full text, mbox, reply):

From: Daniel Leidert <daniel.leidert@wgdd.de>

To: Russ Allbery <rra@debian.org>

Cc: 471263@bugs.debian.org

Subject: Re: Bug#471263: [patch-systems]: please no patch-system-but-direct-changes-in-diff for generated files

Date: Tue, 25 Mar 2008 21:58:21 +0100

Sorry for the late answer.

Am Sonntag, den 16.03.2008, 17:07 -0700 schrieb Russ Allbery:
> Daniel Leidert <daniel.leidert@wgdd.de> writes:
> 
> > Please consider docbook-defguide and docbook-xml/docbook-simple as
> > examples. In these packages I created a build-system (docbook-defguide)
> > or catalogs (docbook-xml, docbook-simple). Because I use a patch system
> > to change upstream sources, lintian complains about the fact, that I do
> > not create these files by patch.
> >
> > It simply doesn't make sense to create these files by patch.
> 
> Why not?

I could ask the opposite: Why do you think it makes sense to create
files by patch?

> > lintian should only warn, if changes are really changes to an existing
> > file, but not if "change" means, that a file has been created.
> 
> I can see why you did it the way that you did, but I don't see any clear
> advantages over doing it that way versus creating the build files in a
> dpatch.

It makes the life of the package maintainer or any contributor harder.

I also don't create examples in the debian/ directory by patch. I simply
write them and put them under debian/. Why shouldn't I do this for files
outside of debian/ as long as these files are part of the Debian
packaging.

In the above cases we speak about compilation packages, that are not
provided by upstream in this form. In the case of docbook-defguide the
created files are a self-written build-system.

> It seems like about the same thing to me.  Not warning about
> newly created files would miss some of the things that this check is
> designed to look for (basically, editing the package without being aware
> that there's an existing patch system).  If a patch created a file and
> that file was also shipped in the diff, you end up putting the patch
> system in an inconsistent state.

This is true. But it is wrong for changes, that create a file. All you
said is valid for existing files, that are changed. But you cannot put
the patch system in an inconsistent state by creation of files.

I would really suggest to downgrade this warning (W) for existing files
to an information (I). There is no danger and there are clearly
situations, where creation fo these files by patch makes less or at
least only the same sense then creation of these files without a patch
system.

Regards, Daniel

Message #20 received at 471263@bugs.debian.org (full text, mbox, reply):

From: Neil Williams <codehelp@debian.org>

To: 471263@bugs.debian.org

Cc: 471263-submitter@bugs.debian.org

Subject: Please silence lintian for generated changes

Date: Wed, 26 Mar 2008 00:01:49 +0000

[Message part 1 (text/plain, inline)]

My take on this is that lintian is getting the check wrong.

Files that are changed as the result of a patch to a file that is
processed during the build should be ignored - e.g. patching
configure.in|ac should mean that changes in ./configure must be ignored.
Why should a patch system try (and necessarily fail) to
patch ./configure only to (fail to) implement the very change that
patching configure.in|ac and running autoreconf achieves cleanly? (The
patch will fail because ./configure changes frequently due to autotools
updates when configure.in|ac does not.) --maintainer-mode is already
dead, why should I need to return to the days of patching the same
change twice?

In practise, this undermines the entire check because it is not going to
be easy for lintian to identify which files contain generated changes on
a reliable basis.

"this check is
designed to look for (basically, editing the package without being aware
that there's an existing patch system)."

In which case, I fear, the check appears to be over-simplified and I
don't see a reliable method for doing what the check is designed to do
without causing false positives like libgpewidget.

" If a patch created a file and
that file was also shipped in the diff, you end up putting the patch
system in an inconsistent state."

That is a case of trying to patch the same file twice - what we need to
avoid is the check for that problem causing maintainers to patch the
same *change* twice in more than one file when changing only one file
will generate the necessary changes in the second file during an
entirely normal build and without any other intervention.

These files are not being "patched twice" or "patched and changed
again", they are being changed by the effect of a patch on another file
causing a change in output from build tools. To fix that problem, we'd
need a radically different autotools design that strips out large parts
of the _orig.tar.gz and only packages the files that upstream actually
change (Makefile.am and configure.in) not the files that upstream
generate (Makefile.in and ./configure). Even that would be all well and
good for new or developing projects but unachievable on older projects
(like libjpeg6b) that haven't actually updated the build system for some
time. (That model is closer to the RCS model of upstream source -
generate everything everytime.)

If a patch modifies a file that is regenerated during the build using
data contributed from an external tool (like intltool, gettext,
autotools, gtk-doc, docbook, xsltproc, ....) then this patch is actually
far more likely to fail and may even fail simply when trying to rebuild
the package twice in a row. (The patch could not be unapplied cleanly
because the patched file has been changed by the build process.)

Simple example: If a package uses xsltproc to update a manpage from XML
but the package also contains the upstream manpage in order to make
generating the manpage optional for some builds, what is the point of
patching the manpage *and* the XML? Only the XML needs to be patched -
patching the manpage itself simply means that as soon as docbook-xsl is
updated to fix some bug in manpage output, my package becomes RC buggy
because the patch fails to apply - a patch that wasn't actually needed
in the first place. Does the maintainer now need to repackage the
_orig.tar.gz to remove the upstream manpage in order to patch it?

This whole thing was bad enough with --maintainer-mode causing patches
to ./configure and configure.in. (I've still got a few packages with
that problem - hopefully upstream will get it right for the next
release.)

These "patching-a-generated-file" bugs are *more* of a problem because
they cause a FTBFS due to the update of some third party package
(typically when the maintainer is busy with something else or away on
hols). If the generated file is not patched, the patch cannot fail - as
long as the effect of the patch is achieved through a patch to the
"source" file, the package will continue to build when other packages
change.

I found this lintian warning to be a problem with the libgpewidget
package - I cannot patch the changes to the doc/tmpl files because those
changes depend on how gtk-doc processes the doc/tmpl/ files in the first
place. Now it may well be that upstream could do with updating the
doc/tmpl files in the next release, but if there is nothing I can do
about it in the Debian package, why should I need a lintian override?

> > Would you agree to this? Is it possible to exclude created files from
> > this warning? I would further vote for excluding
> > changes.{sub,guess,rpath} and maybe even Makefiles and/or configure
> > scripts from this test. However, this is hust an optional suggestion.
> 
> I personally think such changes should be made by running the relevant
> Autotools at build time, but I realize this is an ongoing debate and
> that's far from the consensus at the moment.  I think we're still trying
> to feel out what Lintian's role should be here.  I'd welcome any feedback
> from other people reading the mailing list.

If there is no consensus on changes to files like ./configure,
Makefile.in (when Makefile.am is patched), why has lintian gone ahead
anyway?

I think the check should only seek to find changes to *files that are
already patched* from also appearing in the .diff.gz - skipping any
files in the .diff.gz that are not listed in the patch set. It is these
"patch-and-change-same-file" bugs that will corrupt the patch process,
not the other changes that affect unrelated (and typically generated)
files.

IMHO it is a bad idea to try to patch a file that is modified during the
build process - wherever possible, patches should be against files that
upstream would normally edit to implement the desired change, not the
files that the build tools generate from those changes. Lintian should
not inadvertently undermine that principle.

-- 


Neil Williams
=============
http://www.data-freedom.org/
http://www.nosoftwarepatents.com/
http://www.linux.codehelp.co.uk/

[signature.asc (application/pgp-signature, inline)]

Message #28 received at 471263@bugs.debian.org (full text, mbox, reply):

From: Danai SAE-HAN (韓達耐) <danai.sae-han@edpnet.be>

To: Debian Bug Tracking System <471263@bugs.debian.org>

Subject: Problem with newly created files via ctangle

Date: Mon, 12 May 2008 14:35:13 +0200

Package: lintian
Version: 1.23.48
Followup-For: Bug #471263

Hi!

I'm the maintainer of "cjk", which produces binary files like latex-cjk-common
and latex-cjk-chinese.

The upstream author uses CWEB, a language that produces C and TeX documentation
files through ctangle and cweave respectively.

I used to leave the .c file, as it would be overwritten anyway when you would
rebuild cjk.  However, when I build it now, lintian produces a warning that
the .c file appears in the diff.gz file, presuming that it is an extra patch
that is not handled through dpatch (which I also use).

To solve this issue, I remove the .c file during the cleaning target of
debian/rules, clean the directory with "debuild clean", recreate the source with
git-buildpackage and then rebuild the whole thing once or twice.

I'm not sure if I've done the Right Thing(tm), but it seems to work regardless.


Best regards



Danai SAE-HAN


-- System Information:
Debian Release: lenny/sid
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.24-hefaistos (PREEMPT)
Locale: LANG=nl_BE.UTF-8, LC_CTYPE=nl_BE.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash

Versions of packages lintian depends on:
ii  binutils         2.18.1~cvs20080103-4+b1 The GNU assembler, linker and bina
ii  diffstat         1.45-2                  produces graph of changes introduc
ii  dpkg-dev         1.14.18                 package building tools for Debian
ii  file             4.24-2                  Determines file type using "magic"
ii  gettext          0.17-2                  GNU Internationalization utilities
ii  intltool-debian  0.35.0+20060710.1       Help i18n of RFC822 compliant conf
ii  libparse-debianc 1.1.1-2                 parse Debian changelogs and output
ii  liburi-perl      1.35.dfsg.1-1           Manipulates and accesses URI strin
ii  man-db           2.5.2-1                 on-line manual pager
ii  perl [libdigest- 5.10.0-10               Larry Wall's Practical Extraction 

lintian recommends no packages.

-- no debconf information

Message #33 received at 471263@bugs.debian.org (full text, mbox, reply):

From: Russ Allbery <rra@debian.org>

To: Danai SAE-HAN (韓達耐) <danai.sae-han@edpnet.be>

Cc: 471263@bugs.debian.org

Subject: Re: Bug#471263: Problem with newly created files via ctangle

Date: Mon, 12 May 2008 10:20:57 -0700

Danai SAE-HAN "(韓達耐)" <danai.sae-han@edpnet.be> writes:

> I used to leave the .c file, as it would be overwritten anyway when you
> would rebuild cjk.  However, when I build it now, lintian produces a
> warning that the .c file appears in the diff.gz file, presuming that it
> is an extra patch that is not handled through dpatch (which I also use).
>
> To solve this issue, I remove the .c file during the cleaning target of
> debian/rules, clean the directory with "debuild clean", recreate the
> source with git-buildpackage and then rebuild the whole thing once or
> twice.
>
> I'm not sure if I've done the Right Thing(tm), but it seems to work
> regardless.

That is indeed exactly the right thing to do.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>

Message #38 received at 471263@bugs.debian.org (full text, mbox, reply):

From: Neil Williams <codehelp@debian.org>

To: lintian-maint@debian.org, 471263@bugs.debian.org

Cc: Debian Devel <debian-devel@lists.debian.org>

Subject: Generated files and patch systems

Date: Sun, 25 May 2008 13:07:56 +0100

[Message part 1 (text/plain, inline)]

I'm getting into a crazy situation with this lintian warning:

patch-system-but-direct-changes-in-diff

I've removed one diversion from libgpewidget but I still need to use one
and this requires a patch to configure.ac using dpatch. This then
regenerates configure and aclocal.m4.

I think patch-system-but-direct-changes-in-diff is just too buggy to be
deployed - that isn't the fault of lintian, I think the "problem" cannot
be detected/determined in a sane manner. Maybe a downgrade to info?

From #471263
> I personally think such changes should be made by running the relevant
> Autotools at build time, but I realize this is an ongoing debate and
> that's far from the consensus at the moment.  I think we're still trying
> to feel out what Lintian's role should be here.  I'd welcome any feedback
> from other people reading the mailing list.
> 
> -- 
> Russ Allbery (rra@debian.org)
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=471263#10

So I am running the relevant autotools at build time but I still get the
warning.

Then a more bizarre problem arrives - the first build I get:
Now running lintian...
W: libgpewidget source: patch-system-but-direct-changes-in-diff aclocal.m4
Finished running lintian.

Any subsequent build produces:
Now running lintian...
W: libgpewidget source: patch-system-but-direct-changes-in-diff aclocal.m4
W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/color-slider.sgml
W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/colordialog.sgml
W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/colorrenderer.sgml
W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/gpeclockface.sgml
W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/gpedialog.sgml
W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/gpehelp.sgml
W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/gtkdatecombo.sgml
W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/gtksimplemenu.sgml
W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/libgpewidget-unused.sgml
W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/link-warning.sgml
W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/pixmaps.sgml
W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/smallbox.sgml
W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/spacing.sgml
Finished running lintian.

I don't see why I should "patch" aclocal.m4 and I consider it a harmful
thing to do seeing as every update of autotools will likely cause the
other changes in aclocal.m4 and the patch will to fail to apply causing
a FTBFS - a much more serious issue.

Having aclocal.m4 in the .diff.gz causes problems with uscan; uupdate
because it does fail to apply from one upstream release to the next but
that is only a problem for me and I can live with it.

What is the solution here?

Do the other dpkg source formats have a solution for generated files?

(PLEASE don't spout on and on about git. I don't use git, I have no
intention of using git - I do not particularly want to use any RCS with
this particular package although I might consider SVN seeing as
svn-buildpackage at least has sane MergeWithUpstream support. The more
noise the git-fanclub make, the less I listen so don't expect a reply
from me if you want to talk about git.)

Do I have to move aclocal.m4 aside in debian/rules and move it back? How
does that help? - it'll still be in the .diff.gz under a different
filename. That doesn't help the build-twice release goal issue. Yes, the
package meets the letter of the release goal by building twice in a row
but not without either spurious lintian overrides or spurious lintian
warnings.

What is the problem that this lintian check is trying to solve, why does
it apply to generated files and is there a sane way to tell the
difference (especially when a "generated file" can still mean a file
present in the upstream source and modified in the .diff.gz because
modifications result from the actions of a third-party build-tool)?

I'm not sure why gtk-doc-tools is making these changes - this is a new
upstream release (although there appear to be differences in the
autotools environment between myself and upstream, hence the aclocal.m4
diff) and the tmp/ files have been updated upstream. Quite why I get no
error with the first build but errors with all subsequent builds I have
no idea. I can reproduce the "silent" build by simply copying the
tmp/*.sgml files back into place from the .orig.tar.gz so those could be
"moved aside" as well but it seems a lot of work if the lintian warning
itself is dubious or cannot be applied in a sane manner. I don't see
that creating patches for these tmpl/ files is correct either - a change
in gtk-doc-tools could easily break many such patches and I do not
consider it "my fault" that those files have been changed, hence no
patches.

This, to me, is where the "all changes are patches" approach falls down.
A patch, IMHO, is a deliberate action - similar to a GnuPG signature on
an email. It should require a conscious act to solve an identifiable
problem - preferably one with a bug report. Changes that result merely
from differences in build environment between upstream and Debian should
not even be in the .diff.gz if at all possible - such changes are, IMHO,
certainly not deliberate actions by the maintainer. Changes that
themselves change independently of the actions of the maintainer cannot
be implemented by the maintainer as patches, imho.

The "silent" build is available from my repo:
dget -x http://www.linux.codehelp.co.uk/packages/pool/main/libg/libgpewidget/libgpewidget_0.116-1.dsc

I won't upload to ftp-master just yet - I do need to do some more tests
to ensure that other GPE packages build and function correctly so that I
don't inadvertently trigger a transition in over 40 packages.
;-)

-- 


Neil Williams
=============
http://www.data-freedom.org/
http://www.nosoftwarepatents.com/
http://www.linux.codehelp.co.uk/

[signature.asc (application/pgp-signature, inline)]

Message #43 received at 471263@bugs.debian.org (full text, mbox, reply):

From: Mark Brown <broonie@debian.org>

To: lintian-maint@debian.org, 471263@bugs.debian.org, Debian Devel <debian-devel@lists.debian.org>

Subject: Re: Generated files and patch systems

Date: Sun, 25 May 2008 13:19:02 +0100

On Sun, May 25, 2008 at 01:07:56PM +0100, Neil Williams wrote:

> So I am running the relevant autotools at build time but I still get the
> warning.

If you run autotools at build time you should also ensure that the
changes which autotools makes are reverted in the clean target.  This
means that your diff doesn't get cluttered with automatically generated
things and ensures that repeated builds of the package produce the same
diff.gz.

This is the same idea as patch systems reverting the changes they make
in the clean target.

> Do I have to move aclocal.m4 aside in debian/rules and move it back? How
> does that help? - it'll still be in the .diff.gz under a different
> filename. That doesn't help the build-twice release goal issue. Yes, the
> package meets the letter of the release goal by building twice in a row
> but not without either spurious lintian overrides or spurious lintian
> warnings.

If you actually move it back in the clean target (rather than just
copying it back) then it won't appear in the diff.gz.

-- 
"You grabbed my hand and we fell into it, like a daydream - or a fever."

Message #48 received at 471263@bugs.debian.org (full text, mbox, reply):

From: Raphael Hertzog <hertzog@debian.org>

To: lintian-maint@debian.org, 471263@bugs.debian.org, Debian Devel <debian-devel@lists.debian.org>

Subject: Re: Generated files and patch systems

Date: Sun, 25 May 2008 20:49:19 +0200

On Sun, 25 May 2008, Mark Brown wrote:
> On Sun, May 25, 2008 at 01:07:56PM +0100, Neil Williams wrote:
> 
> > So I am running the relevant autotools at build time but I still get the
> > warning.
> 
> If you run autotools at build time you should also ensure that the
> changes which autotools makes are reverted in the clean target.  This
> means that your diff doesn't get cluttered with automatically generated
> things and ensures that repeated builds of the package produce the same
> diff.gz.

Exactly, it's the reason why I filed
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=482716 in the first
place.

Removing the files which are regenerated in the configure stage is a
simple way to make sure that that you don't end up with such clutter.

Several of the packages which contain such useless changes in the diff
end up causing problems once you try to convert them to source format 3.0
(quilt) because the changes can't be applied/unapplied at any point of
time if the files are regenerated by some intermediate step.

Some examples:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=482749
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=482733

So I'm in favor of keeping this lintian warning because if the maintainer
makes the effort to keep clean patches, it should also make the effort to
make sure that the package cleans up nicely and doesn't clutter the
.diff.gz.

Cheers,
-- 
Raphaël Hertzog

Le best-seller français mis à jour pour Debian Etch :
http://www.ouaza.com/livre/admin-debian/

Message #53 received at 471263@bugs.debian.org (full text, mbox, reply):

From: Goswin von Brederlow <goswin-v-b@web.de>

To: Neil Williams <codehelp@debian.org>

Cc: lintian-maint@debian.org, 471263@bugs.debian.org, Debian Devel <debian-devel@lists.debian.org>

Subject: Re: Generated files and patch systems

Date: Mon, 26 May 2008 00:28:47 +0200

Neil Williams <codehelp@debian.org> writes:

> I'm getting into a crazy situation with this lintian warning:
>
> patch-system-but-direct-changes-in-diff
>
> I've removed one diversion from libgpewidget but I still need to use one
> and this requires a patch to configure.ac using dpatch. This then
> regenerates configure and aclocal.m4.
>
> I think patch-system-but-direct-changes-in-diff is just too buggy to be
> deployed - that isn't the fault of lintian, I think the "problem" cannot
> be detected/determined in a sane manner. Maybe a downgrade to info?

If you regenerate those files then remove them in clean. If lintian
still complains then that is imho a bug.

If you can't work without those files then the only way out is to save
them and restore them in clean.

MfG
        Goswin

Message #58 received at 471263@bugs.debian.org (full text, mbox, reply):

From: Neil Williams <codehelp@debian.org>

To: 471263@bugs.debian.org

Subject: Re: Generated changes and patch systems

Date: Wed, 28 May 2008 11:56:46 +0100

[Message part 1 (text/plain, inline)]

Oops, sent to the wrong bug report.

On Wed, 2008-05-28 at 11:45 +0100, Neil Williams wrote:
> On Tue, 2008-05-27 at 18:06 -0700, Russ Allbery wrote:
> > Neil Williams <codehelp@debian.org> writes:
> > > On Sun, 2008-05-25 at 08:40 -0700, Russ Allbery wrote:
> > 
> > >> Lots of other packages do this -- one of mine off the top of my head is
> > >> xml-security-c.
> > 
> > > Nope. No mention of aclocal.m4 in debian/rules for that package,
> > > just /usr/share/misc/config.guess and config.sub.
> > 
> > Uh, it's the same problem.  Surely the generalization is obvious?
> 
> It's not quite the same problem because aclocal.m4 is regenerated in the
> clean rule itself because changing it causes ./configure --recheck to
> recreate the (modified) file. So instead of copying it around, it has to
> be deleted - merely because a patch system is in use. Seems crazy to me.
> 
> > > I need to patch configure and configure.ac in this package, that means
> > > that aclocal.m4 is constantly being changed and reverted. It shouldn't
> > > matter - it really should not.
> > 
> > Right, so delete it in the clean rule.
> 
> OK. It's a bit like collateral damage - need a patch to configure,
> patching causes configure --recheck which modifies aclocal.m4 so delete
> aclocal.m4. Hmmm. I don't like it but I'll do it anyway.
> 
> > > I see no purpose in lintian (or dpkg come to that) testing for changes
> > > in aclocal.m4 - IMHO it should simply be exempt from this check.
> > 
> > Absolutely not -- you're introducing unexpected changes to the packaging
> > diff file, 
> 
> Well my point is that the change is entirely expected (because the
> patch modifies a file that is involved in generating the changes) - the
> changes are merely a consequence of the patch. It feels wrong to have to
> add a clean rule for aclocal.m4 as a direct result of patching
> configure.ac when aclocal.m4 was not a problem before the patch. 
> 
> Anyway, the "problem" of the tmpl/* files is far more resistant.
> 
> > >>> I really don't want to do all that for the tmpl/* files as well - I
> > >>> don't see the need, copying dozens of files into foo.safe or
> > >>> foo.upstream and then moving them back?
> > 
> > >> Just delete them then.
> > 
> > > ??? That simply does not work. The problem is that running gtk-doc not
> > > only requires tmpl/*.sgml files to exist but it *then modifies them*!
> > 
> > This is extremely unusual.  Are you *sure* that it does an inplace edit of
> > the files during the build process? 
> 
> $ ls libgpewidget-0.115.orig/doc/tmpl/ -1
> colordialog.sgml
> color-slider.sgml
> dirbrowser.sgml
> errorbox.sgml
> gpeclockface.sgml
> gpehelp.sgml
> gpeiconlistitem.sgml
> gpe-iconlist.sgml
> gpeiconlistview.sgml
> gpetimesel.sgml
> gpewindowlist.sgml
> gtkdatecombo.sgml
> gtksimplemenu.sgml
> infoprint.sgml
> init.sgml
> libgpewidget-unused.sgml
> picturebutton.sgml
> pixmaps.sgml
> popup_menu.sgml
> popup.sgml
> question.sgml
> smallbox.sgml
> spacing.sgml
> stylus.sgml
> translabel.sgml
> tray.sgml
> windows.sgml
> 
> From the build log:
>  gtkdoc-mkdb --module=libgpewidget --source-dir=.. --output-format=xml
> --sgml-mode --output-format=xml --tmpl-dir=tmpl
> 
> Now running lintian...
> W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/color-slider.sgml
> W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/colordialog.sgml
> W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/colorrenderer.sgml
> W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/gpeclockface.sgml
> W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/gpedialog.sgml
> W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/gpehelp.sgml
> W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/gtkdatecombo.sgml
> W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/gtksimplemenu.sgml
> W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/libgpewidget-unused.sgml
> W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/link-warning.sgml
> W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/pixmaps.sgml
> W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/smallbox.sgml
> W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/spacing.sgml
> Finished running lintian.
> 
> 13 out of 27 files modified by gtk-doc.
> 
> Copy those files back into doc/tmpl/ from the upstream source and they
> are still modified at the next build.
> 
> Add a rm -f doc/tmpl/*.sgml to clean:: and the build fails: No rule to
> make doc/tmp/*.sgml
> 
> So, yes, --enable-gtkdoc is modifying files included upstream and which
> are essential to the build process.
> 
> If I drop --enable-gtkdoc I get different contents in the
> libgpewidget-doc package.
> 
> >  This is almost never what build
> > systems do; instead, they generate files from other files using templating
> > systems.  They may ship the results of that operation and then redo it
> > during the build, in which case you need to delete the regenerated files
> > in your clean rule.
> 
> Yes, that is what we might expect from build systems - unfortunately it
> doesn't always work that way. 
> 
> The way gtk-doc works is that it adds a pre-defined gtk-doc.make file
> into the upstream source and it is this file that determines everything
> about how gtk-doc works. If upstream are using a different version, I
> don't see what can be done. --enable-gtkdoc will update the .make file
> according to the installed version on the build system. Disabling gtkdoc
> means packaging outdated documentation.
> 
> It seems to me you're expecting gtk-doc-tools to implement something
> like AM_MAINTAINER_MODE where upstream gtk-doc-tools is allowed to make
> changes to doc/tmpl/*.sgml but Debian gtk-doc-tools is not. (I always
> thought that AM_MAINTAINER_MODE was considered harmful for precisely
> this reason.)
> 
> > If they really expect the files to exist and to edit them in-place during
> > the build, upstream is doing something insane, and it's really something
> > that should be fixed upstream; a lintian warning is drawing your attention
> > to a very broken behavior.
> 
> AFAICT this is not a fault upstream - the tmpl/*.sgml files need to be
> included. I suspect it is merely because upstream don't happen to use
> the same version of gtk-doc-tools. The "they" in the quoted paragraph
> refers to the gtk-doc-tools upstream, not libgpewidget upstream.
> 
> It is not at all insane - every package using gtk-doc-tools has the same
> process. I'm not sure why only libgpewidget is currently showing this
> problem but it is wrong to call it "insane".
> 
> gtk-doc-tools is, in essence, an upstream tool. It is meant to modify
> upstream files during the build, that is the reason it exists. The need
> to run it in the build arises, I'm presuming, from a difference in the
> version of gtk-doc-tools between Debian and the libgpewidget upstream
> team. Debian simply cannot stipulate that a certain version must be used
> upstream. I don't see the bug.
> 
> > Right, lintian is diagnosing build system brokenness.  I'm fairly happy
> > with lintian's behavior here; what it's drawing attention to is exactly
> > the kind of thing that breaks repeated package builds or causes NMUs to
> > introduce spurious package diffs, and is something that should really
> > ideally be fixed.
> 
> Well, I'm not sure that it can be fixed - other than requiring upstream
> to use the same version of gtk-doc-tools as Debian which is frankly,
> absurd.
> 
> > > With these gtk-doc files, it's not so much that the tmpl/*.sgml files
> > > are generated but that a tool essential to the build modifies them in a
> > > way that cannot be patched because the results of the patches are
> > > variable according to that third party tool.
> > 
> > If the tool behaves and only behaves in that way (I haven't checked), that
> > tool is broken and we should fix that tool.
> 
> It probably is behaving that way but there's no accounting for changes
> between versions - or for changes that may yet be made that also tweak
> the output. I simply cannot risk patching these tmpl/*.sgml files and
> having the patches fail at some point in the future, turning a bug in
> lintian into a FTBFS in libgpewidget (which has 40 reverse
> dependencies).
> 
> I still think that lintian is out of line here - this is not a problem
> for the maintainer and it is only the maintainer who needs to worry
> about updating the package from 0.115 to 0.116 and 0.117 etc.
> 
> These files are in the .diff.gz because of the behaviour of a third
> party tool which updates a data source within the upstream source
> according to internal rules for that tool. OK, maybe that tool could be
> improved but that is not necessarily trivial - especially when Debian
> cannot stipulate that any particular upstream team must use a particular
> version or configuration for that tool.
> 
> I haven't been able to identify the supposed bug in gtk-doc-tools and I
> don't see that it is up to me to do that - AFAICT this is something that
> is not a problem in the real world and lintian is imposing unrealistic
> expectations on libgpewidget.
> 
> I'm perfectly happy with the behaviour of gtk-doc-tools in libgpewidget
> (I'd like gtk-doc-tools to be workable in a cross-build environment but
> that's minor because cross-builds rarely care for API documentation
> anyway and it is fairly easy to disable gtk-doc if the docs are not
> wanted.). If others disagree, it is up to them to file a bug in
> gtk-doc-tools and work out just how it should be improved. The mere fact
> of a spurious lintian warning is an annoyance, not a problem but I would
> still like lintian to be more intelligent about these things, accept
> that the world is not 100% perfect and not blame the messenger.
> 
> If this is a bug in libgpewidget that I've missed, please let me know in
> the usual manner, but for now I'm just going to ignore lintian for this
> check.
> 
-- 


Neil Williams
=============
http://www.data-freedom.org/
http://www.nosoftwarepatents.com/
http://www.linux.codehelp.co.uk/

[signature.asc (application/pgp-signature, inline)]

Message #67 received at 471263-done@bugs.debian.org (full text, mbox, reply):

From: Niels Thykier <niels@thykier.net>

To: 471263-done@bugs.debian.org

Subject: Re: [checks/patch-systems] suppress patch-system-but-direct-changes-in-diff for generated files

Date: Sat, 20 Aug 2011 15:08:59 +0200

Hi,

This bug has been marked wontfix for over 3 years and no comments have
appeared since then.  Therefore I am closing it to clean up the bug tracker.

For this particular bug, I believe dpkg-source has better support for
excluding some files from the resulting diff (or the auto-generated
patch), which should cover common cases for this issue.

~Niels

Send a report that this bug log contains spam.