Debian Bug report logs - #592839
dpkg-source option to remove files on unpack: debian/source/remove-files

version graph

Package: dpkg-dev; Maintainer for dpkg-dev is Dpkg Developers <debian-dpkg@lists.debian.org>; Source for dpkg-dev is src:dpkg.

Reported by: Raphael Hertzog <hertzog@debian.org>

Date: Fri, 13 Aug 2010 07:51:02 UTC

Severity: wishlist

Found in version dpkg/1.15.8

Reply or subscribe to this bug.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, ijackson@chiark.greenend.org.uk, debian-devel@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#592839; Package dpkg-dev. (Fri, 13 Aug 2010 07:51:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Raphael Hertzog <hertzog@debian.org>:
New Bug report received and forwarded. Copy sent to ijackson@chiark.greenend.org.uk, debian-devel@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>. (Fri, 13 Aug 2010 07:51:05 GMT) Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: Raphael Hertzog <hertzog@debian.org>
To: submit@bugs.debian.org
Subject: dpkg-source option to remove files on unpack: debian/source/remove-files
Date: Fri, 13 Aug 2010 09:46:46 +0200
[Message part 1 (text/plain, inline)]
Package: dpkg-dev
Version: 1.15.8
Severity: wishlist

As suggested by Ian on -devel (see attachment), it would be nice to have
a way to remove files during unpack of a source package to hide non-free
files from our users without stripping them from the original tarball.

I also prefer this approach over repacking upstream files so let's
implement this feature.

To be consistent with other features like debian/source/include-binaries
I suggest to use a file named debian/source/remove-files (and not a field
in .dsc) but if you have a better name, feel free to suggest it.

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer ◈ [Flattr=20693]

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
                      ▶ http://RaphaelHertzog.fr (Français)
[Message part 2 (message/rfc822, inline)]
From: Ian Jackson <ijackson@chiark.greenend.org.uk>
To: Tanguy Ortolo <tanguy+debian@ortolo.eu>
Cc: debian-devel@lists.debian.org
Subject: Re: Non-recompilable binaries in source and binary packages (Adobe Flash strikes again)
Date: Thu, 12 Aug 2010 19:00:32 +0100
Tanguy Ortolo writes ("Non-recompilable binaries in source and binary packages (Adobe Flash strikes again)"):
> Let us say an upstream tarball contains such a non-recompilable binary
> as a minor component that can be stripped and maybe distributed by other
> means. The debian/rules will not compile it (it cannot), but if it does
> not even include it in the generated binary package, would such a
> package (both source and binary) ?not require a package outside of main
> for compilation or execution?? In other words, to be in main, does the
> maintainer have to strip the problematic file from the binary package,
> or does he have to repack the original tarball stripping it?

The current approach of the project in these cases seems to be that
the right thing to do is to rebuild the source package so that the
non-free pieces are removed.

Personally I think this is a complete waste of everyone's time;
particularly, rebuilding upstream source archives is troublesome and
doesn't really benefit anyone.

I'd much rather you could just write in your .dsc a set of glob
patterns for files to remove, somehow, which dpkg-source would remove
when you unpacked it (unless you told it not to).  That would prevent
the non-free files from being accidentally used during the build
process or shipped (eg when upstream Makefiles change), but would save
a lot of work.  It would also mean that the upstream tarball in our
archive would be the real upstream tarball.  

I can see that there might be the objection that we were then
distrubuting non-free stuff from our main archive section but given
that our archive is almost entirely used by people using our tools,
this is a small price worth paying for not obstructing our work on the
free parts of the package.  Obviously if the non-free parts are
significant in size this doesn't apply.

The benefit would be that we would no longer have to rebuild upstream
source packages where upstream has helpfully included files such as
RFCs (which we usually consider non-free) or other non-free stuff.
Upstreams normally regard it as a convenience for people, to provide
apocrypha etc. in their tarballs, and to be honest they're probably
right.  We are making a rod for our own back by our current practice
of insisting on repackaging their tarballs.

(I'm assuming that the non-recompilable binary is distributable.  If
it isn't then it can't be in our archive at all of course.)

> Same case, but with two binary packages generated, one with the main
> content, the other one with the problematic files appart: can the source
> file be in main? That case does not apply if all the packages associated
> to a single orig tarball must be in the same area.

No.  There is no sensible way to do this.  The problem is inherent:
the binary packages in main have to be rebuildable using the source
package (and supporting binary packages eg compilers) in main.

If you have this situation you have to have two separate source
packages; one in main which builds only the free parts, and one in
non-free which builds only the non-free parts.

Ian.


-- 
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/19556.14016.688915.420234@chiark.greenend.org.uk


Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#592839; Package dpkg-dev. (Fri, 13 Aug 2010 10:30:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Goswin von Brederlow <goswin-v-b@web.de>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>. (Fri, 13 Aug 2010 10:30:03 GMT) Full text and rfc822 format available.

Message #10 received at submit@bugs.debian.org (full text, mbox):

From: Goswin von Brederlow <goswin-v-b@web.de>
To: Raphael Hertzog <hertzog@debian.org>
Cc: 592839@bugs.debian.org, submit@bugs.debian.org
Subject: Re: Bug#592839: dpkg-source option to remove files on unpack: debian/source/remove-files
Date: Fri, 13 Aug 2010 12:27:51 +0200
Raphael Hertzog <hertzog@debian.org> writes:

> Package: dpkg-dev
> Version: 1.15.8
> Severity: wishlist
>
> As suggested by Ian on -devel (see attachment), it would be nice to have
> a way to remove files during unpack of a source package to hide non-free
> files from our users without stripping them from the original tarball.
>
> I also prefer this approach over repacking upstream files so let's
> implement this feature.
>
> To be consistent with other features like debian/source/include-binaries
> I suggest to use a file named debian/source/remove-files (and not a field
> in .dsc) but if you have a better name, feel free to suggest it.
>
> Cheers,

How does that differ (legally, from ftp-masters perspective) from
running the clean target?

From the users side would it really be so bad if those extra files exist
initially after unpacking the source but get removed in debian/rules?

MfG
        Goswin




Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#592839; Package dpkg-dev. (Fri, 13 Aug 2010 10:30:07 GMT) Full text and rfc822 format available.

Acknowledgement sent to Goswin von Brederlow <goswin-v-b@web.de>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>. (Fri, 13 Aug 2010 10:30:07 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#592839; Package dpkg-dev. (Fri, 13 Aug 2010 17:03:08 GMT) Full text and rfc822 format available.

Acknowledgement sent to Russ Allbery <rra@debian.org>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>. (Fri, 13 Aug 2010 17:03:09 GMT) Full text and rfc822 format available.

Message #20 received at 592839@bugs.debian.org (full text, mbox):

From: Russ Allbery <rra@debian.org>
To: 592839@bugs.debian.org, debian-devel@lists.debian.org
Subject: Re: Bug#592839: dpkg-source option to remove files on unpack: debian/source/remove-files
Date: Fri, 13 Aug 2010 09:58:07 -0700
Raphael Hertzog <hertzog@debian.org> writes:

> As suggested by Ian on -devel (see attachment), it would be nice to have
> a way to remove files during unpack of a source package to hide non-free
> files from our users without stripping them from the original tarball.

> I also prefer this approach over repacking upstream files so let's
> implement this feature.

I'm pretty sure ftp-master isn't going to allow source packages with
non-free content in the main archive regardless of whether that content is
hidden on unpack (I certainly wouldn't if I were them), so implementing
this is kind of pointless for Debian.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>




Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#592839; Package dpkg-dev. (Fri, 13 Aug 2010 17:06:06 GMT) Full text and rfc822 format available.

Acknowledgement sent to Michael Gilbert <michael.s.gilbert@gmail.com>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>. (Fri, 13 Aug 2010 17:06:06 GMT) Full text and rfc822 format available.

Message #25 received at 592839@bugs.debian.org (full text, mbox):

From: Michael Gilbert <michael.s.gilbert@gmail.com>
To: 592839@bugs.debian.org, debian-devel@lists.debian.org
Subject: Re: Bug#592839: dpkg-source option to remove files on unpack: debian/source/remove-files
Date: Fri, 13 Aug 2010 13:01:34 -0400
On Fri, 13 Aug 2010 09:58:07 -0700, Russ Allbery wrote:
> Raphael Hertzog <hertzog@debian.org> writes:
> 
> > As suggested by Ian on -devel (see attachment), it would be nice to have
> > a way to remove files during unpack of a source package to hide non-free
> > files from our users without stripping them from the original tarball.
> 
> > I also prefer this approach over repacking upstream files so let's
> > implement this feature.
> 
> I'm pretty sure ftp-master isn't going to allow source packages with
> non-free content in the main archive regardless of whether that content is
> hidden on unpack (I certainly wouldn't if I were them), so implementing
> this is kind of pointless for Debian.

What if the list of binary components was part of the watch file
instead; so that new upstream sources could be automatically stripped of
those non-free bits by uscan right after download.

Best wishes,
Mike




Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#592839; Package dpkg-dev. (Fri, 13 Aug 2010 17:06:08 GMT) Full text and rfc822 format available.

Acknowledgement sent to "Eugene V. Lyubimkin" <jackyf@debian.org>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>. (Fri, 13 Aug 2010 17:06:08 GMT) Full text and rfc822 format available.

Message #30 received at 592839@bugs.debian.org (full text, mbox):

From: "Eugene V. Lyubimkin" <jackyf@debian.org>
To: 592839@bugs.debian.org
Cc: debian-devel@lists.debian.org
Subject: Re: Bug#592839: dpkg-source option to remove files on unpack: debian/source/remove-files
Date: Fri, 13 Aug 2010 20:04:14 +0300
[Message part 1 (text/plain, inline)]
Russ Allbery wrote:
> Raphael Hertzog <hertzog@debian.org> writes:
> 
>> As suggested by Ian on -devel (see attachment), it would be nice to have
>> a way to remove files during unpack of a source package to hide non-free
>> files from our users without stripping them from the original tarball.
> 
>> I also prefer this approach over repacking upstream files so let's
>> implement this feature.
> 
> I'm pretty sure ftp-master isn't going to allow source packages with
> non-free content in the main archive regardless of whether that content is
> hidden on unpack (I certainly wouldn't if I were them), so implementing
> this is kind of pointless for Debian.
> 
Seconded.

-- 
Eugene V. Lyubimkin aka JackYF, JID: jackyf.devel(maildog)gmail.com
C++/Perl developer, Debian Developer

[signature.asc (application/pgp-signature, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#592839; Package dpkg-dev. (Sun, 15 Aug 2010 00:36:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to David Claughton <dave@eclecticdave.com>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>. (Sun, 15 Aug 2010 00:36:03 GMT) Full text and rfc822 format available.

Message #35 received at 592839@bugs.debian.org (full text, mbox):

From: David Claughton <dave@eclecticdave.com>
Cc: 592839@bugs.debian.org
Subject: Re: Bug#592839: dpkg-source option to remove files on unpack: debian/source/remove-files
Date: Sun, 15 Aug 2010 01:33:23 +0100
On 13/08/10 17:58, Russ Allbery wrote:
> Raphael Hertzog <hertzog@debian.org> writes:
> 
>> As suggested by Ian on -devel (see attachment), it would be nice to have
>> a way to remove files during unpack of a source package to hide non-free
>> files from our users without stripping them from the original tarball.
> 
>> I also prefer this approach over repacking upstream files so let's
>> implement this feature.
> 
> I'm pretty sure ftp-master isn't going to allow source packages with
> non-free content in the main archive regardless of whether that content is
> hidden on unpack (I certainly wouldn't if I were them), so implementing
> this is kind of pointless for Debian.
> 

Another use-case might be to remove "convenience copies" of system
libraries.  Might be useful (e.g. for security reasons) to be able to
guarantee that this code isn't being accidentally used by a build (in a
way that can be easily checked by a script).

Cheers,

	David.




Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#592839; Package dpkg-dev. (Mon, 16 Aug 2010 19:03:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Joerg Jaspert <joerg@debian.org>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>. (Mon, 16 Aug 2010 19:03:03 GMT) Full text and rfc822 format available.

Message #40 received at submit@bugs.debian.org (full text, mbox):

From: Joerg Jaspert <joerg@debian.org>
To: Raphael Hertzog <hertzog@debian.org>
Cc: 592839@bugs.debian.org, submit@bugs.debian.org
Subject: Re: Bug#592839: dpkg-source option to remove files on unpack: debian/source/remove-files
Date: Mon, 16 Aug 2010 20:59:16 +0200
> As suggested by Ian on -devel (see attachment), it would be nice to have
> a way to remove files during unpack of a source package to hide non-free
> files from our users without stripping them from the original tarball.

This doesnt sound useful for non-free files, at least not if you want
your package end up in main/contrib, but for the useless source copies
many have, sure it will help there.

-- 
bye, Joerg
<doogie_> linux takes shit and turns it into something useful.
<doogie_> windows takes something useful and turns it into shit




Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#592839; Package dpkg-dev. (Mon, 16 Aug 2010 19:03:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to Joerg Jaspert <joerg@debian.org>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>. (Mon, 16 Aug 2010 19:03:05 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#592839; Package dpkg-dev. (Wed, 18 Aug 2010 08:33:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Goswin von Brederlow <goswin-v-b@web.de>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>. (Wed, 18 Aug 2010 08:33:03 GMT) Full text and rfc822 format available.

Message #50 received at 592839@bugs.debian.org (full text, mbox):

From: Goswin von Brederlow <goswin-v-b@web.de>
To: David Claughton <dave@eclecticdave.com>
Cc: debian-devel@lists.debian.org, 592839@bugs.debian.org
Subject: Re: Bug#592839: dpkg-source option to remove files on unpack: debian/source/remove-files
Date: Wed, 18 Aug 2010 10:29:26 +0200
David Claughton <dave@eclecticdave.com> writes:

> On 13/08/10 17:58, Russ Allbery wrote:
>> Raphael Hertzog <hertzog@debian.org> writes:
>> 
>>> As suggested by Ian on -devel (see attachment), it would be nice to have
>>> a way to remove files during unpack of a source package to hide non-free
>>> files from our users without stripping them from the original tarball.
>> 
>>> I also prefer this approach over repacking upstream files so let's
>>> implement this feature.
>> 
>> I'm pretty sure ftp-master isn't going to allow source packages with
>> non-free content in the main archive regardless of whether that content is
>> hidden on unpack (I certainly wouldn't if I were them), so implementing
>> this is kind of pointless for Debian.
>> 
>
> Another use-case might be to remove "convenience copies" of system
> libraries.  Might be useful (e.g. for security reasons) to be able to
> guarantee that this code isn't being accidentally used by a build (in a
> way that can be easily checked by a script).
>
> Cheers,
>
> 	David.

And I ask again: How does that differ from deleting them in
debian/rules?

Legally that should be the same. And practically you would have the
useless files on the initial source unpack but they would be gone when
debian/rules is invoked the first time. dpkg-source -x could run the
clean target by default to make the files disapear directly.

MfG
        Goswin




Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#592839; Package dpkg-dev. (Tue, 03 Sep 2013 09:51:12 GMT) Full text and rfc822 format available.

Acknowledgement sent to David Bremner <bremner@debian.org>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>. (Tue, 03 Sep 2013 09:51:12 GMT) Full text and rfc822 format available.

Message #55 received at 592839@bugs.debian.org (full text, mbox):

From: David Bremner <bremner@debian.org>
To: 592839@bugs.debian.org
Subject: please consider it anyway, even if useless for non-free files
Date: Tue, 03 Sep 2013 06:47:40 -0300
It is sometimes convenient to keep files deleted in the integration
branch of a version control system even if these files are dfsg free;
most recently I did this to facilitate cherry-picking patches from
upstream of an embedded library (yes, ick, I know).  There is no nice
way of representing this in a quilt series; one easily ends up with
enormous patches.  Currently I do some shenanigans like 

         git diff --diff-filter=D --name-only >> debian/clean

This works, but the dependence on debhelper to unpack is unsatisfactory.

Cheers,

d



Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#592839; Package dpkg-dev. (Tue, 03 Sep 2013 18:48:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to Jonathan Nieder <jrnieder@gmail.com>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>. (Tue, 03 Sep 2013 18:48:05 GMT) Full text and rfc822 format available.

Message #60 received at 592839@bugs.debian.org (full text, mbox):

From: Jonathan Nieder <jrnieder@gmail.com>
To: David Bremner <bremner@debian.org>
Cc: 592839@bugs.debian.org
Subject: Re: dpkg-source option to remove files on unpack: debian/source/remove-files
Date: Tue, 3 Sep 2013 11:45:33 -0700
Hi,

David Bremner wrote:

> [Subject: please consider it anyway, even if useless for non-free files]

Please keep in mind that these emails appear in a crowded inbox, so
the subject line can be a good place to put valuable context.

> It is sometimes convenient to keep files deleted in the integration
> branch of a version control system even if these files are dfsg free;
> most recently I did this to facilitate cherry-picking patches from
> upstream of an embedded library (yes, ick, I know).  There is no nice
> way of representing this in a quilt series; one easily ends up with
> enormous patches.

I don't think I understand this particular use case --- why patch the
embedded library instead of just removing it? --- but David
Claughton's example of being able to simply and reliably remove an
embedded convenience copy of another package in a way that tools like
lintian can notice makes sense to me.

If someone writes a patch, I'll be happy to review it. ;-)

>          git diff --diff-filter=D --name-only >> debian/clean
>
> This works, but the dependence on debhelper to unpack is unsatisfactory.

One can't rely on the clean target having been run before the initial
build anyway, no?

Lazily,
Jonathan



Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#592839; Package dpkg-dev. (Tue, 03 Sep 2013 23:30:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to David Bremner <bremner@debian.org>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>. (Tue, 03 Sep 2013 23:30:04 GMT) Full text and rfc822 format available.

Message #65 received at 592839@bugs.debian.org (full text, mbox):

From: David Bremner <bremner@debian.org>
To: Jonathan Nieder <jrnieder@gmail.com>
Cc: 592839@bugs.debian.org
Subject: Re: dpkg-source option to remove files on unpack: debian/source/remove-files
Date: Tue, 03 Sep 2013 20:27:23 -0300
Jonathan Nieder <jrnieder@gmail.com> writes:

> I don't think I understand this particular use case --- why patch the
> embedded library instead of just removing it? 

The embedded library is actually a fork, with tiny but functionally
significant changes.

d



Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#592839; Package dpkg-dev. (Tue, 03 Sep 2013 23:42:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to Jonathan Nieder <jrnieder@gmail.com>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>. (Tue, 03 Sep 2013 23:42:05 GMT) Full text and rfc822 format available.

Message #70 received at 592839@bugs.debian.org (full text, mbox):

From: Jonathan Nieder <jrnieder@gmail.com>
To: David Bremner <bremner@debian.org>
Cc: 592839@bugs.debian.org
Subject: Re: dpkg-source option to remove files on unpack: debian/source/remove-files
Date: Tue, 3 Sep 2013 16:39:15 -0700
David Bremner wrote:
> Jonathan Nieder <jrnieder@gmail.com> writes:

>> I don't think I understand this particular use case --- why patch the
>> embedded library instead of just removing it? 
>
> The embedded library is actually a fork, with tiny but functionally
> significant changes.

I think we're veering off topic, but if you'd like help getting the
source package that provides the normal library to provide the
modified library as an extra binary package, I'd be happy to work on
that.

Thanks,
Jonathan



Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Sat Apr 19 15:29:24 2014; Machine Name: buxtehude.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.