Debian Bug report logs - #638706
qa.debian.org: Does not like \1 backreferences in watch file (breaks download links)

Package: qa.debian.org; Maintainer for qa.debian.org is debian-qa@lists.debian.org;

Reported by: Matthijs Kooijman <matthijs@stdin.nl>

Date: Sun, 21 Aug 2011 10:06:49 UTC

Severity: normal

Done: Matthijs Kooijman <matthijs@stdin.nl>

Bug is archived. No further changes may be made.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, debian-qa@lists.debian.org:
Bug#638706; Package qa.debian.org. (Sun, 21 Aug 2011 10:06:51 GMT) Full text and rfc822 format available.

Acknowledgement sent to Matthijs Kooijman <matthijs@stdin.nl>:
New Bug report received and forwarded. Copy sent to debian-qa@lists.debian.org. (Sun, 21 Aug 2011 10:06:54 GMT) Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: Matthijs Kooijman <matthijs@stdin.nl>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: qa.debian.org: Does not like \1 backreferences in watch file (breaks download links)
Date: Sun, 21 Aug 2011 11:35:09 +0200
Package: qa.debian.org
Severity: normal

Hi folks,

I noticed that the "download" links on the developer's packages
overview, which allow downloading new upstream version if uscan detects
new versions, are not working right now. At least on this page:

	http://qa.debian.org/developer.php?login=matthijs%40stdin.nl&comaint=yes

The links point to, for example,

	http://qa.debian.org/1/openttd-1-source.tar.gz

which returns an Apache 404 from qa.debian.org.

I'm not sure if the url is wrong, or if this url should redirect but it
doesn't. Not sure where the "1" in the url comes from either, since the
link is supposed to download the source for openttd 1.1.2.

I also noticed that the download links are working for other packages,
so it must be something related to the rewriting happening in my watch
file. For the OpenTTD package mentioned above, the watch file contains:

	options=downloadurlmangle=s/(.*)\/index.html$/\1\/openttd-\1-source.tar.gz/ \
	http://master.binaries.openttd.org/releases/ \
	(\d+(?:\.\d+)*)/index.html

Looking at this file, it seems the "1"'s in the download url come from
the \1 backreferences in the downloadurlmangle option.

Thinking about this a bit further, I suspect that this is caused by
using \1 as a backreference instead of $1 (which is perfectly valid in
Perl and is understood by uscan, but apparently not by the qa.debian.org
scripts).

I'm not sure if this is a bug in my watchfile, or that qa.debian.org
should support this, though. This might be a trival change in the
qa.debian.org scripts, but I'll probably change my backreferences to use
$ instead of \ anyway (seems Perl recommends using $1 instead of \1 as
well).

Also, perhaps my analysis is wrong somewhere, haven't actually confirmed
that the backreferences are the cause (though it seems likely).

Gr.

Matthijs

-- System Information:
Debian Release: wheezy/sid
  APT prefers testing
  APT policy: (990, 'testing'), (500, 'unstable'), (500, 'stable'), (1, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.39-2-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_US.utf8, LC_CTYPE=en_US.utf8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash




Information forwarded to debian-bugs-dist@lists.debian.org, debian-qa@lists.debian.org:
Bug#638706; Package qa.debian.org. (Fri, 26 Aug 2011 21:27:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Giovanni Mascellani <gio@debian.org>:
Extra info received and forwarded to list. Copy sent to debian-qa@lists.debian.org. (Fri, 26 Aug 2011 21:27:04 GMT) Full text and rfc822 format available.

Message #10 received at 638706@bugs.debian.org (full text, mbox):

From: Giovanni Mascellani <gio@debian.org>
To: Matthijs Kooijman <matthijs@stdin.nl>, 638706@bugs.debian.org
Subject: Re: Bug#638706: qa.debian.org: Does not like \1 backreferences in watch file (breaks download links)
Date: Fri, 26 Aug 2011 23:25:49 +0200
[Message part 1 (text/plain, inline)]
Hi Matthijs.

On 21/08/2011 11:35, Matthijs Kooijman wrote:
> I also noticed that the download links are working for other packages,
> so it must be something related to the rewriting happening in my watch
> file. For the OpenTTD package mentioned above, the watch file contains:
> 
> 	options=downloadurlmangle=s/(.*)\/index.html$/\1\/openttd-\1-source.tar.gz/ \
> 	http://master.binaries.openttd.org/releases/ \
> 	(\d+(?:\.\d+)*)/index.html
> 
> Looking at this file, it seems the "1"'s in the download url come from
> the \1 backreferences in the downloadurlmangle option.
> 
> Thinking about this a bit further, I suspect that this is caused by
> using \1 as a backreference instead of $1 (which is perfectly valid in
> Perl and is understood by uscan, but apparently not by the qa.debian.org
> scripts).

Your interpretation appear to be correct. Anyway, I don't think uscan
actually recognizes the \1 notation: if I use the watch file you posted
in the bug report, uscan says:

> $ uscan --report --dehs
> <dehs>
> <package>openttd</package>
> <debian-uversion>1.1.2</debian-uversion>
> <debian-mangled-uversion>1.1.2</debian-mangled-uversion>
> <upstream-version>1.1.2</upstream-version>
> <upstream-url>1/openttd-1-source.tar.gz</upstream-url>
> <status>up to date</status>
> </dehs>

which is exactly your problem. Also, the scripts behind the QA pages
actually call uscan to do the parsing.

BTW, the debian/watch that is in unstable now uses the $1 notation, so
fixes this problem. Anyway, there is another subtle bug:

> $ uscan --report --dehs
> <dehs>
> <package>openttd</package>
> <debian-uversion>1.1.2</debian-uversion>
> <debian-mangled-uversion>1.1.2</debian-mangled-uversion>
> <upstream-version>1.1.2</upstream-version>
> <upstream-url>http://master.binaries.openttd.org/releases/1.1.2/openttd-http://master.binaries.openttd.org/releases/1.1.2-source.tar.gz</upstream-url>
> <status>up to date</status>
> </dehs>

As you may see, the download URL is incorrect, because the mangling
wrongly matches on the whole URL, not only the version number. The
problem can be fixed with somewhat like this (you can use | instead of /
for separating substitution fields, so you have less escaping work to do):

options=downloadurlmangle=s|([^/]*)/index.html$|$1/openttd-$1-source.tar.gz|
\
http://master.binaries.openttd.org/releases/ \
(\d+(?:\.\d+)*)/index.html

Hope this helps.

> I'm not sure if this is a bug in my watchfile, or that qa.debian.org
> should support this, though. This might be a trival change in the
> qa.debian.org scripts, but I'll probably change my backreferences to use
> $ instead of \ anyway (seems Perl recommends using $1 instead of \1 as
> well).

I don't think this is a bug in QA scripts, because it actually depends
on uscan. You should probably file a whishlist bug against uscan asking
for the \1 notation to be supported or, at least, to have some
documentation somewhere about this decision.

Please, let me know your comments if you have any. Otherwise, I'll close
the report.

Giovanni.
-- 
Giovanni Mascellani <mascellani@poisson.phc.unipi.it>
Pisa, Italy

Web: http://poisson.phc.unipi.it/~mascellani
Jabber: g.mascellani@jabber.org / giovanni@elabor.homelinux.org

[signature.asc (application/pgp-signature, attachment)]

Reply sent to Matthijs Kooijman <matthijs@stdin.nl>:
You have taken responsibility. (Tue, 30 Aug 2011 14:39:07 GMT) Full text and rfc822 format available.

Notification sent to Matthijs Kooijman <matthijs@stdin.nl>:
Bug acknowledged by developer. (Tue, 30 Aug 2011 14:39:07 GMT) Full text and rfc822 format available.

Message #15 received at 638706-done@bugs.debian.org (full text, mbox):

From: Matthijs Kooijman <matthijs@stdin.nl>
To: Giovanni Mascellani <gio@debian.org>
Cc: 638706-done@bugs.debian.org
Subject: Re: Bug#638706: qa.debian.org: Does not like \1 backreferences in watch file (breaks download links)
Date: Tue, 30 Aug 2011 15:52:20 +0200
[Message part 1 (text/plain, inline)]
Hi Giovanni,

> Your interpretation appear to be correct. Anyway, I don't think uscan
> actually recognizes the \1 notation: if I use the watch file you posted
> in the bug report, uscan says:
> 
> > $ uscan --report --dehs
> > <dehs>
> > <package>openttd</package>
> > <debian-uversion>1.1.2</debian-uversion>
> > <debian-mangled-uversion>1.1.2</debian-mangled-uversion>
> > <upstream-version>1.1.2</upstream-version>
> > <upstream-url>1/openttd-1-source.tar.gz</upstream-url>
> > <status>up to date</status>
> > </dehs>
> 
> which is exactly your problem. Also, the scripts behind the QA pages
> actually call uscan to do the parsing.
Right, I see what happens there. I had of course tested my watch file,
but just running uscan --report-status reports that the current version
is up to date. However, it does not actually try to download the
(wrongly mangled) url, it only sees if there is an url that could be
succesfully mangled.

So, your right: My watch files are really wrong, I just hadn't noticed
this before.

> BTW, the debian/watch that is in unstable now uses the $1 notation, so
> fixes this problem. Anyway, there is another subtle bug:
> 
> > $ uscan --report --dehs
> > <dehs>
> > <package>openttd</package>
> > <debian-uversion>1.1.2</debian-uversion>
> > <debian-mangled-uversion>1.1.2</debian-mangled-uversion>
> > <upstream-version>1.1.2</upstream-version>
> > <upstream-url>http://master.binaries.openttd.org/releases/1.1.2/openttd-http://master.binaries.openttd.org/releases/1.1.2-source.tar.gz</upstream-url>
> > <status>up to date</status>
> > </dehs>
> 
> As you may see, the download URL is incorrect, because the mangling
> wrongly matches on the whole URL, not only the version number. The
> problem can be fixed with somewhat like this (you can use | instead of /
> for separating substitution fields, so you have less escaping work to do):
> 
> options=downloadurlmangle=s|([^/]*)/index.html$|$1/openttd-$1-source.tar.gz|
> \
> http://master.binaries.openttd.org/releases/ \
> (\d+(?:\.\d+)*)/index.html
Ah, right. Didn't notice that for the same reasons.

> > I'm not sure if this is a bug in my watchfile, or that qa.debian.org
> > should support this, though. This might be a trival change in the
> > qa.debian.org scripts, but I'll probably change my backreferences to use
> > $ instead of \ anyway (seems Perl recommends using $1 instead of \1 as
> > well).
> 
> I don't think this is a bug in QA scripts, because it actually depends
> on uscan. You should probably file a whishlist bug against uscan asking
> for the \1 notation to be supported or, at least, to have some
> documentation somewhere about this decision.
Since perl says \1 is deprecated, I think documenting the lack of
support would be sufficient. I'll open a new bug with a patch in a
minute.

> Please, let me know your comments if you have any. Otherwise, I'll close
> the report.
I've gone ahead and closed it already. Thanks for your thoughts and
explainations.

Gr.

Matthijs
[signature.asc (application/pgp-signature, inline)]

Bug archived. Request was from Debbugs Internal Request <owner@bugs.debian.org> to internal_control@bugs.debian.org. (Wed, 28 Sep 2011 07:32:39 GMT) Full text and rfc822 format available.

Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Sun Apr 20 01:47:00 2014; Machine Name: buxtehude.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.