Debian Bug report logs - #633531
qa.debian.org: popcon tables not updated (at least) since

Package: qa.debian.org; Maintainer for qa.debian.org is debian-qa@lists.debian.org;

Reported by: Tobias Hansen <tobias.han@gmx.de>

Date: Mon, 11 Jul 2011 09:48:02 UTC

Severity: important

Done: Jakub Wilk <jwilk@debian.org>

Bug is archived. No further changes may be made.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, debian-qa@lists.debian.org:
Bug#633531; Package qa.debian.org. (Mon, 11 Jul 2011 09:48:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to Tobias Hansen <tobias.han@gmx.de>:
New Bug report received and forwarded. Copy sent to debian-qa@lists.debian.org. (Mon, 11 Jul 2011 09:48:20 GMT) Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: Tobias Hansen <tobias.han@gmx.de>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: qa.debian.org: popcon graphs not updated since June 25
Date: Mon, 11 Jul 2011 11:27:56 +0200
Package: qa.debian.org
Severity: important


See for example this graph for the time from June 23 until now:

http://qa.debian.org/popcon-graph.php?packages=grub-pc&show_installed=on&show_vote=on&show_old=on&show_recent=on&show_nofiles=on&want_legend=on&want_ticks=on&from_date=2011-06-23&to_date=2011-07-11&hlght_date=&date_fmt=%25Y-%25m&beenhere=1

-- System Information:
Debian Release: wheezy/sid
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.39-2-amd64 (SMP w/2 CPU cores)
Locale: LANG=de_DE.utf8, LC_CTYPE=de_DE.utf8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash




Reply sent to Giovanni Mascellani <gio@debian.org>:
You have taken responsibility. (Wed, 17 Aug 2011 11:48:04 GMT) Full text and rfc822 format available.

Notification sent to Tobias Hansen <tobias.han@gmx.de>:
Bug acknowledged by developer. (Wed, 17 Aug 2011 11:48:08 GMT) Full text and rfc822 format available.

Message #10 received at 633531-done@bugs.debian.org (full text, mbox):

From: Giovanni Mascellani <gio@debian.org>
To: 633531-done@bugs.debian.org
Subject: Closing
Date: Wed, 17 Aug 2011 13:44:41 +0200
[Message part 1 (text/plain, inline)]
This bug appears to be fixed, the graph at the indicated isn't constant
and seems to be up-to-date.

Please, reopen the bug if you still experience the bug.

Giovanni.
-- 
Giovanni Mascellani <mascellani@poisson.phc.unipi.it>
Pisa, Italy

Web: http://poisson.phc.unipi.it/~mascellani
Jabber: g.mascellani@jabber.org / giovanni@elabor.homelinux.org

[signature.asc (application/pgp-signature, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, debian-qa@lists.debian.org:
Bug#633531; Package qa.debian.org. (Wed, 17 Aug 2011 18:27:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Tobias Hansen <tobias.han@gmx.de>:
Extra info received and forwarded to list. Copy sent to debian-qa@lists.debian.org. (Wed, 17 Aug 2011 18:27:03 GMT) Full text and rfc822 format available.

Message #15 received at 633531@bugs.debian.org (full text, mbox):

From: Tobias Hansen <tobias.han@gmx.de>
To: 633531@bugs.debian.org
Cc: Giovanni Mascellani <gio@debian.org>, control@bugs.debian.org
Subject: Re: Bug#633531: marked as done (qa.debian.org: popcon graphs not updated since June 25)
Date: Wed, 17 Aug 2011 20:27:01 +0200
reopen 633531
retitle 633531 qa.debian.org: popcon tables not updated (at least) since 
July
thanks

When I first opened this bug, I memorized some figures from the popcon 
tables and they didn't change ever since:

gedit:           inst 41897
out-of-order:    inst   103
sludge-engine:   inst   114

See for example http://qa.debian.org/popcon.php?package=gedit

So the graphs are updated now, but the tables are not.

Someone also reported this on the mailing list: 
http://lists.debian.org/debian-qa/2011/08/msg00000.html

Best regards,
Tobias




Did not alter fixed versions and reopened. Request was from Debbugs Internal Request <owner@bugs.debian.org> to internal_control@bugs.debian.org. (Wed, 17 Aug 2011 18:27:14 GMT) Full text and rfc822 format available.

Changed Bug title to 'qa.debian.org: popcon tables not updated (at least) since' from 'qa.debian.org: popcon graphs not updated since June 25' Request was from Tobias Hansen <tobias.han@gmx.de> to control@bugs.debian.org. (Wed, 17 Aug 2011 18:27:15 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, debian-qa@lists.debian.org:
Bug#633531; Package qa.debian.org. (Wed, 17 Aug 2011 19:00:08 GMT) Full text and rfc822 format available.

Acknowledgement sent to Giovanni Mascellani <gio@debian.org>:
Extra info received and forwarded to list. Copy sent to debian-qa@lists.debian.org. (Wed, 17 Aug 2011 19:00:08 GMT) Full text and rfc822 format available.

Message #24 received at 633531@bugs.debian.org (full text, mbox):

From: Giovanni Mascellani <gio@debian.org>
To: Tobias Hansen <tobias.han@gmx.de>
Cc: 633531@bugs.debian.org
Subject: Re: Bug#633531: marked as done (qa.debian.org: popcon graphs not updated since June 25)
Date: Wed, 17 Aug 2011 20:58:28 +0200
[Message part 1 (text/plain, inline)]
Hi.

On 17/08/2011 20:27, Tobias Hansen wrote:
> When I first opened this bug, I memorized some figures from the popcon
> tables and they didn't change ever since:
> 
> gedit:           inst 41897
> out-of-order:    inst   103
> sludge-engine:   inst   114
> 
> See for example http://qa.debian.org/popcon.php?package=gedit
> 
> So the graphs are updated now, but the tables are not.

Yes, I see, thanks for the reporting.

Another example is the package mandelbulber, which is quite recent,
shows the graph, but has no numerical data:

http://qa.debian.org/popcon.php?package=mandelbulber

I don't think I'll have time for this in the next days, but let's just
hope for someone to cross this report! :-)

Giovanni.
-- 
Giovanni Mascellani <mascellani@poisson.phc.unipi.it>
Pisa, Italy

Web: http://poisson.phc.unipi.it/~mascellani
Jabber: g.mascellani@jabber.org / giovanni@elabor.homelinux.org

[signature.asc (application/pgp-signature, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, debian-qa@lists.debian.org:
Bug#633531; Package qa.debian.org. (Thu, 18 Aug 2011 22:39:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Yaroslav Halchenko <debian@onerussian.com>:
Extra info received and forwarded to list. Copy sent to debian-qa@lists.debian.org. (Thu, 18 Aug 2011 22:39:03 GMT) Full text and rfc822 format available.

Message #29 received at 633531@bugs.debian.org (full text, mbox):

From: Yaroslav Halchenko <debian@onerussian.com>
To: 633531@bugs.debian.org
Subject: yeap -- tables are out of date
Date: Thu, 18 Aug 2011 18:37:14 -0400
just wanted to confirm that it costed me already quite some time to
figure out that there is indeed a problem ;)

I got confused by difference among numbers presented on the web and downloaded
daily stats file (i.e.  all-popcon-results.gz): e.g.
http://qa.debian.org/popcon.php?package=python-numpy says that python-numpy has
42922 installations BUT

> zgrep 'python-numpy ' all-popcon-results.gz
Package: python-numpy                   29513  8824  6003     4

and the plot on the page is closer to 44000 than 43000. Also the page
signature says "Last generated on Sat Feb 19 13:14:02 2011 UTC" which seems to
be a while.

And why there is such a huge difference between numbers in the
all-popcon-results.gz and what is reported by the webpage...? (probably I do
not know for sure the meaning of the columns ;-) )



-- 
=------------------------------------------------------------------=
Keep in touch                                     www.onerussian.com
Yaroslav Halchenko                 www.ohloh.net/accounts/yarikoptic




Information forwarded to debian-bugs-dist@lists.debian.org, debian-qa@lists.debian.org:
Bug#633531; Package qa.debian.org. (Thu, 01 Sep 2011 13:03:35 GMT) Full text and rfc822 format available.

Acknowledgement sent to Jakub Wilk <jwilk@debian.org>:
Extra info received and forwarded to list. Copy sent to debian-qa@lists.debian.org. (Thu, 01 Sep 2011 13:03:46 GMT) Full text and rfc822 format available.

Message #34 received at 633531@bugs.debian.org (full text, mbox):

From: Jakub Wilk <jwilk@debian.org>
To: 633531@bugs.debian.org
Subject: Bug#633531: qa.debian.org: popcon tables not updated
Date: Thu, 1 Sep 2011 15:00:52 +0200
* Yaroslav Halchenko <debian@onerussian.com>, 2011-08-18, 18:37:
>Also the page signature says "Last generated on Sat Feb 19 13:14:02 
>2011 UTC" which seems to be a while.

This is a red herring - it's a time when the PHP file was generated, not 
when data was generated. However, the data is also out-of-date:
| $ cd /org/qa.debian.org/data/popcon/ && ls -l --time-style=long-iso popcon*
| -rw-rw-r-- 1 qa qa    299549 2011-06-22 15:52 popcon
| -rw-rw-r-- 1 qa qa 134221824 2011-06-22 15:51 popcon.db

/org/qa.debian.org/data/cronjobs/popcon looks like this:
| #!/bin/bash -e
| # CRON=50 15 * * *
| # MAILTO+=geissert@debian.org
|
| umask 002
| cd /org/qa.debian.org/data/popcon
|
| lockfile -r0 .popcon.lock 2> /dev/null || exit 1
|
| ./extract_popcon.py
| ./extract_popcon-tobdb.py
|
| mv -f popcon{-new,}
| mv -f popcon{-new,}.db
|
| rm -f .popcon.lock

The first script is for updating PostgreSQL database (used by DDPO), the 
latter for updating Berkley database (used by everything else). My 
testing shows that extract_popcon-tobdb.py works flawlessly; 
unfortunately I cannot test extract_popcon.py. Given the fact that DDPO 
data also appear to be out of date, I suspect that extract_popcon.py 
fails.

-- 
Jakub Wilk




Reply sent to Jakub Wilk <jwilk@debian.org>:
You have taken responsibility. (Wed, 14 Sep 2011 18:03:09 GMT) Full text and rfc822 format available.

Notification sent to Tobias Hansen <tobias.han@gmx.de>:
Bug acknowledged by developer. (Wed, 14 Sep 2011 18:03:10 GMT) Full text and rfc822 format available.

Message #39 received at 633531-done@bugs.debian.org (full text, mbox):

From: Jakub Wilk <jwilk@debian.org>
To: Yaroslav Halchenko <debian@onerussian.com>, 633531-done@bugs.debian.org
Subject: Re: Bug#633531: yeap -- tables are out of date
Date: Wed, 14 Sep 2011 20:02:01 +0200
* Yaroslav Halchenko <debian@onerussian.com>, 2011-08-18, 18:37:
>just wanted to confirm that it costed me already quite some time to 
>figure out that there is indeed a problem ;)
>
>I got confused by difference among numbers presented on the web and 
>downloaded daily stats file (i.e.  all-popcon-results.gz): e.g.
>http://qa.debian.org/popcon.php?package=python-numpy says that 
>python-numpy has 42922 installations BUT
>
>> zgrep 'python-numpy ' all-popcon-results.gz
>Package: python-numpy                   29513  8824  6003     4

Note that the none of the columns of all-popcon-results.gz is number of 
installations. 29513+8824+6003+4 = 44344, which is not *that* different 
from 42922.

On the qa.d.o side, we had a problem with a stale lockfile, which is 
fixed now.

-- 
Jakub Wilk




Message #40 received at 633531-done@bugs.debian.org (full text, mbox):

From: Yaroslav Halchenko <debian@onerussian.com>
To: 633531-done@bugs.debian.org
Subject: Re: Bug#633531: yeap -- tables are out of date
Date: Wed, 14 Sep 2011 14:04:47 -0400
On Wed, 14 Sep 2011, Jakub Wilk wrote:
> >>zgrep 'python-numpy ' all-popcon-results.gz
> >Package: python-numpy                   29513  8824  6003     4
> Note that the none of the columns of all-popcon-results.gz is number
> of installations. 29513+8824+6003+4 = 44344, which is not *that*
> different from 42922.

yeap -- I have figured it out as well ;)  BTW, this number is around 40%
of all popcon submissions, which I think is quite a ratio.  On Ubuntu's
it is only 20%... more plots in my recent talk:

π’s in Debian or Scientific Debian: NumPy, SciPy and beyond. 
http://neuro.debian.net/_files/Halchenko_EuroScipy11_3_14s_in_Debian.pdf


> On the qa.d.o side, we had a problem with a stale lockfile, which is
> fixed now.
-- 
=------------------------------------------------------------------=
Keep in touch                                     www.onerussian.com
Yaroslav Halchenko                 www.ohloh.net/accounts/yarikoptic




Bug archived. Request was from Debbugs Internal Request <owner@bugs.debian.org> to internal_control@bugs.debian.org. (Thu, 13 Oct 2011 07:35:05 GMT) Full text and rfc822 format available.

Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Fri Apr 18 16:06:48 2014; Machine Name: beach.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.