Debian Bug report logs - #670615
wget: add post-download command hook

version graph

Package: wget; Maintainer for wget is Noël Köthe <noel@debian.org>; Source for wget is src:wget.

Reported by: Witold Baryluk <baryluk@smp.if.uj.edu.pl>

Date: Fri, 27 Apr 2012 11:03:02 UTC

Severity: wishlist

Tags: upstream

Found in version wget/1.13.4-3

Forwarded to https://savannah.gnu.org/bugs/index.php?36797

Reply or subscribe to this bug.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, baryluk@smp.if.uj.edu.pl, Noël Köthe <noel@debian.org>:
Bug#670615; Package wget. (Fri, 27 Apr 2012 11:03:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to Witold Baryluk <baryluk@smp.if.uj.edu.pl>:
New Bug report received and forwarded. Copy sent to baryluk@smp.if.uj.edu.pl, Noël Köthe <noel@debian.org>. (Fri, 27 Apr 2012 11:03:09 GMT) Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: Witold Baryluk <baryluk@smp.if.uj.edu.pl>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: wget: add post-download command hook
Date: Fri, 27 Apr 2012 13:01:03 +0200
Package: wget
Version: 1.13.4-3
Severity: wishlist

Hello,

It would be really great to have a post-download command hook which would be
executed after each file is downloaded successfully by wget.

Currently, when running from scripts it is not easy to do something with
actually downloaded file if it is downloaded using recursion or if file is
generated dynamically, or filename is unknown or redirection took place.

for example
   wget --post-download=md5sum  http://ftp.gnu.org/gnu/bash/bash-4.2.tar.gz

md5sum will be then executed by wget, for example using system() command:
    system("wget bash-4.2.tar.gz")

preferably with some additional environment variables set, like time of
download, wheter it was first download or already downloaded file previously
(but was running with -c flag), what was size, speed, original url, url after
last redirection, wheter redirection took place, what was dns and IP address
used and protocol (http/ftp, ipv4/ipv6, etc), average speed, number of retries,
wheter html links was already changed to local or absolute ones from relative
ones, received cookies/headers, etc.

Actual system command will depend also on -O flag, -c, and whatever we are
using -nc, -nd, -x, nH, --cut-dirs, (and so also -r) and -P.

This command will be very useful in recursive processing as well when using
glob patterns, as well in case of redirects.

Instead of system() command, one can use fork/execve, which will not clobber
environment variables, as well make files with spaces (both downloaded one as
well used script) not a danger.

Thanks,
Witek



-- System Information:
Debian Release: wheezy/sid
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: i386 (i686)

Kernel: Linux 3.2.0-2-686-pae (SMP w/1 CPU core)
Locale: LANG=pl_PL.UTF-8, LC_CTYPE=pl_PL.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages wget depends on:
ii  dpkg           1.16.2
ii  install-info   4.13a.dfsg.1-10
ii  libc6          2.13-30
ii  libgcrypt11    1.5.0-3
ii  libgnutls26    2.12.18-1
ii  libgpg-error0  1.10-3
ii  libidn11       1.24-2
ii  zlib1g         1:1.2.6.dfsg-2

wget recommends no packages.

wget suggests no packages.

-- no debconf information




Information forwarded to debian-bugs-dist@lists.debian.org, Noël Köthe <noel@debian.org>:
Bug#670615; Package wget. (Thu, 05 Jul 2012 22:54:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to NoèlKöthe <INVALID.NOREPLY@gnu.org>:
Extra info received and forwarded to list. Copy sent to Noël Köthe <noel@debian.org>. (Thu, 05 Jul 2012 22:54:03 GMT) Full text and rfc822 format available.

Message #10 received at 670615@bugs.debian.org (full text, mbox):

From: NoèlKöthe <INVALID.NOREPLY@gnu.org>
To: NoèlKöthe <noel@debian.org>, 670615@bugs.debian.org, gscrivano@gnu.org
Subject: [bug #36797] add post-download command hook
Date: Thu, 05 Jul 2012 22:52:22 +0000
URL:
  <http://savannah.gnu.org/bugs/?36797>

                 Summary: add post-download command hook
                 Project: GNU Wget
            Submitted by: nok
            Submitted on: Fr 06 Jul 2012 00:52:21 CEST
                Category: Feature Request
                Severity: 3 - Normal
                Priority: 5 - Normal
                  Status: None
                 Privacy: Public
             Assigned to: None
         Originator Name: 
        Originator Email: 
             Open/Closed: Open
         Discussion Lock: Any
                 Release: None
        Operating System: GNU/Linux
         Reproducibility: None
           Fixed Release: None
         Planned Release: None
              Regression: None
           Work Required: None
          Patch Included: No

    _______________________________________________________

Details:

Hello,

this is a forwarded feature request from http://bugs.debian.org/670615

--8<--
Hello,

It would be really great to have a post-download command hook which would be
executed after each file is downloaded successfully by wget.

Currently, when running from scripts it is not easy to do something with
actually downloaded file if it is downloaded using recursion or if file is
generated dynamically, or filename is unknown or redirection took place.

for example
   wget --post-download=md5sum  http://ftp.gnu.org/gnu/bash/bash-4.2.tar.gz

md5sum will be then executed by wget, for example using system() command:
    system("wget bash-4.2.tar.gz")

preferably with some additional environment variables set, like time of
download, wheter it was first download or already downloaded file previously
(but was running with -c flag), what was size, speed, original url, url after
last redirection, wheter redirection took place, what was dns and IP address
used and protocol (http/ftp, ipv4/ipv6, etc), average speed, number of
retries,
wheter html links was already changed to local or absolute ones from relative
ones, received cookies/headers, etc.

Actual system command will depend also on -O flag, -c, and whatever we are
using -nc, -nd, -x, nH, --cut-dirs, (and so also -r) and -P.

This command will be very useful in recursive processing as well when using
glob patterns, as well in case of redirects.

Instead of system() command, one can use fork/execve, which will not clobber
environment variables, as well make files with spaces (both downloaded one as
well used script) not a danger.

Thanks,
Witek
--8<--

Thank you.




    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?36797>

_______________________________________________
  Nachricht gesendet von/durch Savannah
  http://savannah.gnu.org/





Information forwarded to debian-bugs-dist@lists.debian.org:
Bug#670615; Package wget. (Thu, 05 Jul 2012 22:57:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Noël Köthe <noel@debian.org>:
Extra info received and forwarded to list. (Thu, 05 Jul 2012 22:57:03 GMT) Full text and rfc822 format available.

Message #15 received at 670615@bugs.debian.org (full text, mbox):

From: Noël Köthe <noel@debian.org>
To: Witold Baryluk <baryluk@smp.if.uj.edu.pl>, 670615@bugs.debian.org
Cc: control <control@bugs.debian.org>
Subject: Re: Bug#670615: wget: add post-download command hook
Date: Fri, 06 Jul 2012 00:53:31 +0200
[Message part 1 (text/plain, inline)]
forwarded 670615 https://savannah.gnu.org/bugs/index.php?36797
tags 670615 + upstream
thanks

Hello Witek,

Am Freitag, den 27.04.2012, 13:01 +0200 schrieb Witold Baryluk:

> It would be really great to have a post-download command hook which would be
> executed after each file is downloaded successfully by wget.

I forwarded your request to the upstream authors.

> Currently, when running from scripts it is not easy to do something with
> actually downloaded file if it is downloaded using recursion or if file is
> generated dynamically, or filename is unknown or redirection took place.
> 
> for example
>    wget --post-download=md5sum  http://ftp.gnu.org/gnu/bash/bash-4.2.tar.gz
> 
> md5sum will be then executed by wget, for example using system() command:
>     system("wget bash-4.2.tar.gz")
> 
> preferably with some additional environment variables set, like time of
> download, wheter it was first download or already downloaded file previously
> (but was running with -c flag), what was size, speed, original url, url after
> last redirection, wheter redirection took place, what was dns and IP address
> used and protocol (http/ftp, ipv4/ipv6, etc), average speed, number of retries,
> wheter html links was already changed to local or absolute ones from relative
> ones, received cookies/headers, etc.
> 
> Actual system command will depend also on -O flag, -c, and whatever we are
> using -nc, -nd, -x, nH, --cut-dirs, (and so also -r) and -P.
> 
> This command will be very useful in recursive processing as well when using
> glob patterns, as well in case of redirects.
> 
> Instead of system() command, one can use fork/execve, which will not clobber
> environment variables, as well make files with spaces (both downloaded one as
> well used script) not a danger.


-- 
Noël Köthe <noel debian.org>
Debian GNU/Linux, www.debian.org
[signature.asc (application/pgp-signature, inline)]

Set Bug forwarded-to-address to 'https://savannah.gnu.org/bugs/index.php?36797'. Request was from Noël Köthe <noel@debian.org> to control@bugs.debian.org. (Thu, 05 Jul 2012 22:57:04 GMT) Full text and rfc822 format available.

Added tag(s) upstream. Request was from Noël Köthe <noel@debian.org> to control@bugs.debian.org. (Thu, 05 Jul 2012 22:57:05 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org:
Bug#670615; Package wget. (Tue, 10 Jul 2012 23:30:11 GMT) Full text and rfc822 format available.

Acknowledgement sent to Noël Köthe <noel@debian.org>:
Extra info received and forwarded to list. (Tue, 10 Jul 2012 23:30:11 GMT) Full text and rfc822 format available.

Message #24 received at 670615@bugs.debian.org (full text, mbox):

From: Noël Köthe <noel@debian.org>
To: 670615@bugs.debian.org
Subject: reporter is not reachable anymore
Date: Wed, 11 Jul 2012 01:28:03 +0200
[Message part 1 (text/plain, inline)]
<baryluk@smp.if.uj.edu.pl>: connect to smp.if.uj.edu.pl[149.156.82.206]:25: No
    route to host

-- 
Noël Köthe <noel debian.org>
Debian GNU/Linux, www.debian.org
[signature.asc (application/pgp-signature, inline)]

Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Thu Apr 24 10:44:10 2014; Machine Name: beach.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.