Debian Bug report logs - #653465
Allow Vcs/Tarball cases where the tarball _and_ the source in Vcs have a (common) $subdir.

version graph

Package: pristine-tar; Maintainer for pristine-tar is Joey Hess <joeyh@debian.org>; Source for pristine-tar is src:pristine-tar.

Reported by: Mike Gabriel <mike.gabriel@das-netzwerkteam.de>

Date: Wed, 28 Dec 2011 16:27:01 UTC

Severity: normal

Tags: patch

Found in version pristine-tar/1.15

Reply or subscribe to this bug.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, Joey Hess <joeyh@debian.org>:
Bug#653465; Package pristine-tar. (Wed, 28 Dec 2011 16:27:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Mike Gabriel <mike.gabriel@das-netzwerkteam.de>:
New Bug report received and forwarded. Copy sent to Joey Hess <joeyh@debian.org>.

Your message specified a Severity: in the pseudo-header, but the severity value whishlist was not recognised. The default severity normal is being used instead. The recognised values are: critical, grave, serious, important, normal, minor, wishlist, fixed.

(Wed, 28 Dec 2011 16:27:05 GMT) Full text and rfc822 format available.


Message #5 received at submit@bugs.debian.org (full text, mbox):

From: Mike Gabriel <mike.gabriel@das-netzwerkteam.de>
To: submit@bugs.debian.org
Subject: Allow Vcs/Tarball cases where the tarball _and_ the source in Vcs have a (common) $subdir.
Date: Wed, 28 Dec 2011 17:22:46 +0100
[Message part 1 (text/plain, inline)]
package: pristine-tar
version: 1.15
severity: whishlist
tags: patch

Hi,

I have stumbled over an upstream use case where I had to bundle  
several upstream projects into one Debian source package.

For this I had to create several upstream branches with branch names  
equal to subproject name. Another requirement was that the base folder  
in each branch only contained _one_ folder with the name of the  
subproject and therein the actual source code of the subproject.

Same with the tarballs from upstream: each tarball  
<subproject>-<ver>.tar.gz contained a single folder <subproject> and  
in this folder one could find the source code of the subproject.

pristine-tar till now does not support such a construct. Attached you  
find a patch that adds functionality for the described use case  
without breaking other use cases (I hope!).

Greets,
Mike

-- 

DAS-NETZWERKTEAM
mike gabriel, dorfstr. 27, 24245 barmissen
fon: +49 (4302) 281418, fax: +49 (4302) 281419

GnuPG Key ID 0xB588399B
mail: mike.gabriel@das-netzwerkteam.de, http://das-netzwerkteam.de

freeBusy:
https://mail.das-netzwerkteam.de/freebusy/m.gabriel%40das-netzwerkteam.de.xfb
[pristine-tar-subdir.patch (text/x-patch, attachment)]
[Message part 3 (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Joey Hess <joeyh@debian.org>:
Bug#653465; Package pristine-tar. (Wed, 28 Dec 2011 16:39:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Mike Gabriel <mike.gabriel@das-netzwerkteam.de>:
Extra info received and forwarded to list. Copy sent to Joey Hess <joeyh@debian.org>. (Wed, 28 Dec 2011 16:39:03 GMT) Full text and rfc822 format available.

Message #10 received at 653465@bugs.debian.org (full text, mbox):

From: Mike Gabriel <mike.gabriel@das-netzwerkteam.de>
To: 653465@bugs.debian.org
Subject: Patch update
Date: Wed, 28 Dec 2011 17:31:16 +0100
[Message part 1 (text/plain, inline)]
Hi,

sorry, the patch I send was an old version, here is the (hopefully)  
correct one...

Mike

-- 

DAS-NETZWERKTEAM
mike gabriel, dorfstr. 27, 24245 barmissen
fon: +49 (4302) 281418, fax: +49 (4302) 281419

GnuPG Key ID 0xB588399B
mail: mike.gabriel@das-netzwerkteam.de, http://das-netzwerkteam.de

freeBusy:
https://mail.das-netzwerkteam.de/freebusy/m.gabriel%40das-netzwerkteam.de.xfb
[pristine-tar-subdir.patch (text/x-patch, attachment)]
[Message part 3 (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org:
Bug#653465; Package pristine-tar. (Wed, 28 Dec 2011 16:57:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Joey Hess <joeyh@debian.org>:
Extra info received and forwarded to list. (Wed, 28 Dec 2011 16:57:03 GMT) Full text and rfc822 format available.

Message #15 received at submit@bugs.debian.org (full text, mbox):

From: Joey Hess <joeyh@debian.org>
To: Mike Gabriel <mike.gabriel@das-netzwerkteam.de>, 653465@bugs.debian.org
Cc: submit@bugs.debian.org
Subject: Re: Bug#653465: Allow Vcs/Tarball cases where the tarball _and_ the source in Vcs have a (common) $subdir.
Date: Wed, 28 Dec 2011 12:54:01 -0400
[Message part 1 (text/plain, inline)]
Mike Gabriel wrote:
> I have stumbled over an upstream use case where I had to bundle
> several upstream projects into one Debian source package.
> 
> For this I had to create several upstream branches with branch names
> equal to subproject name. Another requirement was that the base
> folder in each branch only contained _one_ folder with the name of
> the subproject and therein the actual source code of the subproject.
> 
> Same with the tarballs from upstream: each tarball
> <subproject>-<ver>.tar.gz contained a single folder <subproject> and
> in this folder one could find the source code of the subproject.
> 
> pristine-tar till now does not support such a construct. Attached
> you find a patch that adds functionality for the described use case
> without breaking other use cases (I hope!).

> +	# Test if $subdir is the only folder in $source, if so, also
> +	# use $subdir as subpath for $source...	
> +	opendir(DIR, "$source");
> +	my @FILES = readdir(DIR); 
> +	debug("files: @FILES");
> +
> +	# @FILES eq 3 means: `$subdir .. .'
> +	if (-d "$source$subdir" && length @FILES eq 3) {
> +		$source = "$source$subdir";
> +	}

This only works in the very specific case you mention, where you have a
branch containing a single subdirectory containing the upstream source.

pristine-tar geldelta and gentar can just be run from inside a
subdirectory, and will only use files from that subdirectory. So I think
a better approach would be to make pristine-tar commit and pristine-tar
checkout able to be run from a subdirectory inside the git repository,
and only use files under the current subdirectory. Currently in various
ways I have not investigated in detail, they assume they're run at the
top of the repository.

Once they support running in a subdirectory, you won't even necessarily
need separate branches, and a larger class of use cases will be
supported.

-- 
see shy jo
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org:
Bug#653465; Package pristine-tar. (Wed, 28 Dec 2011 16:57:07 GMT) Full text and rfc822 format available.

Acknowledgement sent to Joey Hess <joeyh@debian.org>:
Extra info received and forwarded to list. (Wed, 28 Dec 2011 16:57:07 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Joey Hess <joeyh@debian.org>:
Bug#653465; Package pristine-tar. (Wed, 28 Dec 2011 18:15:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Mike Gabriel <mike.gabriel@das-netzwerkteam.de>:
Extra info received and forwarded to list. Copy sent to Joey Hess <joeyh@debian.org>. (Wed, 28 Dec 2011 18:15:04 GMT) Full text and rfc822 format available.

Message #25 received at submit@bugs.debian.org (full text, mbox):

From: Mike Gabriel <mike.gabriel@das-netzwerkteam.de>
To: Joey Hess <joeyh@debian.org>
Cc: 653465@bugs.debian.org, submit@bugs.debian.org
Subject: Re: Bug#653465: Allow Vcs/Tarball cases where the tarball _and_ the source in Vcs have a (common) $subdir.
Date: Wed, 28 Dec 2011 19:10:40 +0100
[Message part 1 (text/plain, inline)]
Hi Joey,

On Mi 28 Dez 2011 17:54:01 CET Joey Hess wrote:

> Mike Gabriel wrote:
>> I have stumbled over an upstream use case where I had to bundle
>> several upstream projects into one Debian source package.
>>
>> For this I had to create several upstream branches with branch names
>> equal to subproject name. Another requirement was that the base
>> folder in each branch only contained _one_ folder with the name of
>> the subproject and therein the actual source code of the subproject.
>>
>> Same with the tarballs from upstream: each tarball
>> <subproject>-<ver>.tar.gz contained a single folder <subproject> and
>> in this folder one could find the source code of the subproject.
>>
>> pristine-tar till now does not support such a construct. Attached
>> you find a patch that adds functionality for the described use case
>> without breaking other use cases (I hope!).
>
>> +	# Test if $subdir is the only folder in $source, if so, also
>> +	# use $subdir as subpath for $source...
>> +	opendir(DIR, "$source");
>> +	my @FILES = readdir(DIR);
>> +	debug("files: @FILES");
>> +
>> +	# @FILES eq 3 means: `$subdir .. .'
>> +	if (-d "$source$subdir" && length @FILES eq 3) {
>> +		$source = "$source$subdir";
>> +	}
>
> This only works in the very specific case you mention, where you have a
> branch containing a single subdirectory containing the upstream source.
>
> pristine-tar geldelta and gentar can just be run from inside a
> subdirectory, and will only use files from that subdirectory. So I think
> a better approach would be to make pristine-tar commit and pristine-tar
> checkout able to be run from a subdirectory inside the git repository,
> and only use files under the current subdirectory. Currently in various
> ways I have not investigated in detail, they assume they're run at the
> top of the repository.
>
> Once they support running in a subdirectory, you won't even necessarily
> need separate branches, and a larger class of use cases will be
> supported.
>
> --
> see shy jo
>

I just now encountered exactly such a case you described above... So,  
when do you plan to have the more generic approach (use any subdir in  
source tree for pristine-tar) available? That would be really great to  
have that!!!

Greets,
Mike


-- 

DAS-NETZWERKTEAM
mike gabriel, dorfstr. 27, 24245 barmissen
fon: +49 (4302) 281418, fax: +49 (4302) 281419

GnuPG Key ID 0xB588399B
mail: mike.gabriel@das-netzwerkteam.de, http://das-netzwerkteam.de

freeBusy:
https://mail.das-netzwerkteam.de/freebusy/m.gabriel%40das-netzwerkteam.de.xfb
[Message part 2 (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Joey Hess <joeyh@debian.org>:
Bug#653465; Package pristine-tar. (Wed, 28 Dec 2011 18:15:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to Mike Gabriel <mike.gabriel@das-netzwerkteam.de>:
Extra info received and forwarded to list. Copy sent to Joey Hess <joeyh@debian.org>. (Wed, 28 Dec 2011 18:15:06 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org:
Bug#653465; Package pristine-tar. (Wed, 28 Dec 2011 19:18:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to Joey Hess <joeyh@debian.org>:
Extra info received and forwarded to list. (Wed, 28 Dec 2011 19:18:05 GMT) Full text and rfc822 format available.

Message #35 received at 653465@bugs.debian.org (full text, mbox):

From: Joey Hess <joeyh@debian.org>
To: Mike Gabriel <mike.gabriel@das-netzwerkteam.de>, 653465@bugs.debian.org
Subject: Re: Bug#653465: Allow Vcs/Tarball cases where the tarball _and_ the source in Vcs have a (common) $subdir.
Date: Wed, 28 Dec 2011 15:14:37 -0400
[Message part 1 (text/plain, inline)]
Mike Gabriel wrote:
> I just now encountered exactly such a case you described above...
> So, when do you plan to have the more generic approach (use any
> subdir in source tree for pristine-tar) available? That would be
> really great to have that!!!

I don't know... it's not a priority for me.. maybe you would like to
work on it?

-- 
see shy jo
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Joey Hess <joeyh@debian.org>:
Bug#653465; Package pristine-tar. (Wed, 28 Dec 2011 21:54:08 GMT) Full text and rfc822 format available.

Acknowledgement sent to Mike Gabriel <mike.gabriel@das-netzwerkteam.de>:
Extra info received and forwarded to list. Copy sent to Joey Hess <joeyh@debian.org>. (Wed, 28 Dec 2011 21:54:10 GMT) Full text and rfc822 format available.

Message #40 received at 653465@bugs.debian.org (full text, mbox):

From: Mike Gabriel <mike.gabriel@das-netzwerkteam.de>
To: Joey Hess <joeyh@debian.org>
Cc: 653465@bugs.debian.org
Subject: Re: Bug#653465: Allow Vcs/Tarball cases where the tarball _and_ the source in Vcs have a (common) $subdir.
Date: Wed, 28 Dec 2011 22:50:49 +0100
[Message part 1 (text/plain, inline)]
Hi Joey,

On Mi 28 Dez 2011 20:14:37 CET Joey Hess wrote:

> Mike Gabriel wrote:
>> I just now encountered exactly such a case you described above...
>> So, when do you plan to have the more generic approach (use any
>> subdir in source tree for pristine-tar) available? That would be
>> really great to have that!!!
>
> I don't know... it's not a priority for me.. maybe you would like to
> work on it?

I might have time for that tomorrow... Such a feature is needed for  
one of my projects, so it is a good time to work on that.

I'll send patches to this report...

BTW: where is your development Vcs (pristine-tar upstream)?

Mike


-- 

DAS-NETZWERKTEAM
mike gabriel, dorfstr. 27, 24245 barmissen
fon: +49 (4302) 281418, fax: +49 (4302) 281419

GnuPG Key ID 0xB588399B
mail: mike.gabriel@das-netzwerkteam.de, http://das-netzwerkteam.de

freeBusy:
https://mail.das-netzwerkteam.de/freebusy/m.gabriel%40das-netzwerkteam.de.xfb
[Message part 2 (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org:
Bug#653465; Package pristine-tar. (Thu, 29 Dec 2011 16:09:06 GMT) Full text and rfc822 format available.

Acknowledgement sent to Joey Hess <joeyh@debian.org>:
Extra info received and forwarded to list. (Thu, 29 Dec 2011 16:09:06 GMT) Full text and rfc822 format available.

Message #45 received at 653465@bugs.debian.org (full text, mbox):

From: Joey Hess <joeyh@debian.org>
To: Mike Gabriel <mike.gabriel@das-netzwerkteam.de>
Cc: 653465@bugs.debian.org
Subject: Re: Bug#653465: Allow Vcs/Tarball cases where the tarball _and_ the source in Vcs have a (common) $subdir.
Date: Thu, 29 Dec 2011 12:03:56 -0400
[Message part 1 (text/plain, inline)]
Mike Gabriel wrote:
> I might have time for that tomorrow... Such a feature is needed for
> one of my projects, so it is a good time to work on that.
> 
> I'll send patches to this report...

Thanks.. Don't worry about pretty, a series of hacks that make it work
for you would be very helpful.

> BTW: where is your development Vcs (pristine-tar upstream)?

git://git.kitenet.net/pristine-tar/

-- 
see shy jo
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Joey Hess <joeyh@debian.org>:
Bug#653465; Package pristine-tar. (Tue, 03 Jan 2012 14:18:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Mike Gabriel <mike.gabriel@das-netzwerkteam.de>:
Extra info received and forwarded to list. Copy sent to Joey Hess <joeyh@debian.org>. (Tue, 03 Jan 2012 14:18:05 GMT) Full text and rfc822 format available.

Message #50 received at 653465@bugs.debian.org (full text, mbox):

From: Mike Gabriel <mike.gabriel@das-netzwerkteam.de>
To: Joey Hess <joeyh@debian.org>
Cc: 653465@bugs.debian.org, control@bugs.debian.org
Subject: Re: Bug#653465: Allow Vcs/Tarball cases where the tarball _and_ the source in Vcs have a (common) $subdir.
Date: Tue, 03 Jan 2012 15:15:26 +0100
[Message part 1 (text/plain, inline)]
tags #653465 patch
thanks

Hi Joey,

On Do 29 Dez 2011 17:03:56 CET Joey Hess wrote:

> Mike Gabriel wrote:
>> I might have time for that tomorrow... Such a feature is needed for
>> one of my projects, so it is a good time to work on that.
>>
>> I'll send patches to this report...
>
> Thanks.. Don't worry about pretty, a series of hacks that make it work
> for you would be very helpful.
>
>> BTW: where is your development Vcs (pristine-tar upstream)?
>
> git://git.kitenet.net/pristine-tar/

attached you find an approach that adds a --relsrcbase option to  
pristine-tar which then takes in the relative location of the source  
tree in the Git upstream source refspec.

This --relsrcbase option has to be passed on commit _and_ checkout.  
However, it would be much more genuine if the .delta file would store  
this relsrcbase, so on checkout no extra option would be necessary.

Greets,
Mike




-- 

DAS-NETZWERKTEAM
mike gabriel, dorfstr. 27, 24245 barmissen
fon: +49 (4302) 281418, fax: +49 (4302) 281419

GnuPG Key ID 0xB588399B
mail: mike.gabriel@das-netzwerkteam.de, http://das-netzwerkteam.de

freeBusy:
https://mail.das-netzwerkteam.de/freebusy/m.gabriel%40das-netzwerkteam.de.xfb
[pristine-tar_relsrcbase.patch (text/x-diff, attachment)]
[Message part 3 (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org:
Bug#653465; Package pristine-tar. (Tue, 03 Jan 2012 17:09:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Joey Hess <joeyh@debian.org>:
Extra info received and forwarded to list. (Tue, 03 Jan 2012 17:09:03 GMT) Full text and rfc822 format available.

Message #55 received at 653465@bugs.debian.org (full text, mbox):

From: Joey Hess <joeyh@debian.org>
To: Mike Gabriel <mike.gabriel@das-netzwerkteam.de>
Cc: 653465@bugs.debian.org
Subject: Re: Bug#653465: Allow Vcs/Tarball cases where the tarball _and_ the source in Vcs have a (common) $subdir.
Date: Tue, 3 Jan 2012 13:04:12 -0400
[Message part 1 (text/plain, inline)]
Mike Gabriel wrote:
> attached you find an approach that adds a --relsrcbase option to
> pristine-tar which then takes in the relative location of the source
> tree in the Git upstream source refspec.
> 
> This --relsrcbase option has to be passed on commit _and_ checkout.
> However, it would be much more genuine if the .delta file would
> store this relsrcbase, so on checkout no extra option would be
> necessary.

It seems to me that the next step is to automatically calculate the
$relsrcbase value. Probably that only needs scanning up the parent
directories to find .git . Then either it will need to be stored in the
delta, or the recreation code can simply re-calculate it.

-- 
see shy jo
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Joey Hess <joeyh@debian.org>:
Bug#653465; Package pristine-tar. (Tue, 03 Jan 2012 17:27:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Mike Gabriel <mike.gabriel@das-netzwerkteam.de>:
Extra info received and forwarded to list. Copy sent to Joey Hess <joeyh@debian.org>. (Tue, 03 Jan 2012 17:27:03 GMT) Full text and rfc822 format available.

Message #60 received at 653465@bugs.debian.org (full text, mbox):

From: Mike Gabriel <mike.gabriel@das-netzwerkteam.de>
To: Joey Hess <joeyh@debian.org>
Cc: 653465@bugs.debian.org
Subject: Re: Bug#653465: Allow Vcs/Tarball cases where the tarball _and_ the source in Vcs have a (common) $subdir.
Date: Tue, 03 Jan 2012 18:25:52 +0100
[Message part 1 (text/plain, inline)]
Hi Joey,

On Di 03 Jan 2012 18:04:12 CET Joey Hess wrote:

> Mike Gabriel wrote:
>> attached you find an approach that adds a --relsrcbase option to
>> pristine-tar which then takes in the relative location of the source
>> tree in the Git upstream source refspec.
>>
>> This --relsrcbase option has to be passed on commit _and_ checkout.
>> However, it would be much more genuine if the .delta file would
>> store this relsrcbase, so on checkout no extra option would be
>> necessary.

> It seems to me that the next step is to automatically calculate the
> $relsrcbase value.

Do you think that such a calculation can be reliable with any folder  
structure imaginable? I am not so sure about that.

> Probably that only needs scanning up the parent
> directories to find .git.

What if there are duplicate path names??? This is not very common, I  
know, but we should think of any possible dir tree, even if maybe  
completely crazy...

> Then either it will need to be stored in the
> delta, or the recreation code can simply re-calculate it.

My favorite is a cmdline option (so the user can state the path  
explicitly) and storing this value in the delta file... However, if we  
find a way to detect the relscrbase reliably (by comparing the whole  
manifest vs. the tarball tree?), then I, of course, prefer that, too.

Greets,
Mike


-- 

DAS-NETZWERKTEAM
mike gabriel, dorfstr. 27, 24245 barmissen
fon: +49 (4302) 281418, fax: +49 (4302) 281419

GnuPG Key ID 0xB588399B
mail: mike.gabriel@das-netzwerkteam.de, http://das-netzwerkteam.de

freeBusy:
https://mail.das-netzwerkteam.de/freebusy/m.gabriel%40das-netzwerkteam.de.xfb
[Message part 2 (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org:
Bug#653465; Package pristine-tar. (Tue, 03 Jan 2012 17:48:13 GMT) Full text and rfc822 format available.

Acknowledgement sent to Joey Hess <joeyh@debian.org>:
Extra info received and forwarded to list. (Tue, 03 Jan 2012 17:48:13 GMT) Full text and rfc822 format available.

Message #65 received at 653465@bugs.debian.org (full text, mbox):

From: Joey Hess <joeyh@debian.org>
To: Mike Gabriel <mike.gabriel@das-netzwerkteam.de>
Cc: 653465@bugs.debian.org
Subject: Re: Bug#653465: Allow Vcs/Tarball cases where the tarball _and_ the source in Vcs have a (common) $subdir.
Date: Tue, 3 Jan 2012 13:43:46 -0400
[Message part 1 (text/plain, inline)]
Mike Gabriel wrote:
> What if there are duplicate path names??? This is not very common, I
> know, but we should think of any possible dir tree, even if maybe
> completely crazy...

Git commits to the first .git directory found in either the current
directory or its parent, etc. AFAICS, that's all you need to know; find
the toplevel of the repository, find the current directory, and subtract
the former from the latter.

-- 
see shy jo
[signature.asc (application/pgp-signature, inline)]

Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Thu Apr 17 06:58:56 2014; Machine Name: buxtehude.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.