Debian Bug report logs - #199316
apt: bittorrent URI support

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Antony Hilliard <ahilliard@hailmail.net>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: apt: bittorrent URI support

Date: Sun, 29 Jun 2003 18:32:48 -0700

Package: apt
Version: 0.5.5.1
Severity: wishlist


Debian depends a great deal on the generosity of its academic and private partners to provide all the bandwidth for hosting apt's 
extensive software archive.  Ease of installation of this enormous variety of software is one of Debian's biggest strengths and the 
list of packages continues to grow.

As more and more packages grow in size and releases of unstable software see thousands of users upgrading daily, perhaps 
bittorrent could be integrated into apt for certain packages at the servers discretion.  The nano package probably doesn't need it, 
but OpenOffice might.

I don't code so I'm afraid all I can do is ask.  How severe are the technical challenges?


-- System Information:
Debian Release: testing/unstable
Architecture: i386
Kernel: Linux tones 2.4.20 #1 Wed May 28 18:54:01 PDT 2003 i686
Locale: LANG=C, LC_CTYPE=C

Versions of packages apt depends on:
ii  libc6                      2.3.1-17      GNU C Library: Shared libraries an
ii  libgcc1                    1:3.3.1-0pre0 GCC support library
ii  libstdc++5                 1:3.3.1-0pre0 The GNU Standard C++ Library v3

-- no debconf information

Message #10 received at 199316@bugs.debian.org (full text, mbox, reply):

From: Matt Zimmerman <mdz@debian.org>

To: 199316@bugs.debian.org, Antony Hilliard <ahilliard@hailmail.net>

Subject: #199316: apt: bittorrent URI support

Date: Mon, 30 Jun 2003 18:02:05 -0400

See #176733 and elsewhere for information on why this isn't presently
feasible.

-- 
 - mdz

Message #15 received at 199316@bugs.debian.org (full text, mbox, reply):

From: Matt Zimmerman <mdz@debian.org>

To: 199316@bugs.debian.org

Subject: bittorrent notes

Date: Tue, 11 Nov 2003 22:28:26 -0500

It might be possible to counteract the disadvantages of smaller files
somewhat by using the same sort of technique as the bittorrent client, i.e.
continue to serve the file even after it is downloaded.  The time that apt
spends downloading subsequent packages and installing the whole batch could
be used to serve packages from the cache.

I don't think this ties in very well with the current Acquire interface, but
I haven't looked at it with this in mind.  The bittorrents could continue to
run in the background after the download completed, but something would need
to tell them to stop eventually.

The other problem, I believe, is the creation and maintenance of .torrent
files.

-- 
 - mdz

Message #20 received at 199316@bugs.debian.org (full text, mbox, reply):

From: Matt Zimmerman <mdz@debian.org>

To: 199316@bugs.debian.org

Subject: [Arnaud.Kyheng@free.fr: Apt-Torrent project]

Date: Sun, 31 Oct 2004 21:18:23 -0800

Haven't looked at this yet, but it seems relevant...

----- Forwarded message from Arnaud Kyheng <Arnaud.Kyheng@free.fr> -----

Date: Sat, 30 Oct 2004 10:35:53 +0200
From: Arnaud Kyheng <Arnaud.Kyheng@free.fr>
Resent-From: debian-devel@lists.debian.org
To: debian-devel@lists.debian.org
Cc: Arnaud Kyheng <Arnaud.Kyheng@free.fr>
Subject: Apt-Torrent project

Hello,

I love the Debian project, and I have worked on a new development for 
it: Apt-Torrent :)

Apt-Torrent is an apt proxy to the Bittorrent network. For security, the 
package listing, and the .torrent files are downloaded from a regular 
http server, as usual for a package, but then the whole package is 
fetched via bittorrent protocol and forwarded to apt :)

At the moment there is two packages, apt-torrent itself, and 
apt-torrent-server which is an helper package to build a Bittorrent 
repository.

I'm running a very little repository with only official frozen-bubble 
sid's package in order to test apt-torrent :) (might be very slow since 
it's on my own home ADSL connection)

Here is the homepage of the project, where I've written a little 
documentation too:
http://sianka.free.fr

I would like to know what do you think of it :)


Arnaud


-- 
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

----- End forwarded message -----

-- 
 - mdz

Message #25 received at 199316@bugs.debian.org (full text, mbox, reply):

From: Matt Zimmerman <mdz@debian.org>

To: debian-devel@lists.debian.org

Cc: 199316@bugs.debian.org

Subject: Re: Apt-Torrent project

Date: Sun, 31 Oct 2004 21:27:18 -0800

(CCing the BTS, where this feature request is already tracked)

On Sat, Oct 30, 2004 at 10:35:53AM +0200, Arnaud Kyheng wrote:

> I love the Debian project, and I have worked on a new development for 
> it: Apt-Torrent :)
> 
> Apt-Torrent is an apt proxy to the Bittorrent network. For security, the 
> package listing, and the .torrent files are downloaded from a regular 
> http server, as usual for a package, but then the whole package is 
> fetched via bittorrent protocol and forwarded to apt :)

Interesting; I hadn't considered implementing this as a proxy.  It would
probably be better to subclass an existing HTTP server implementation,
rather than to implement a subset of HTTP in your package.

-- 
 - mdz

Message #30 received at 199316@bugs.debian.org (full text, mbox, reply):

From: Arnaud Kyheng <Arnaud.Kyheng@free.fr>

To: Matt Zimmerman <mdz@debian.org>

Cc: debian-devel@lists.debian.org, 199316@bugs.debian.org

Subject: Re: Apt-Torrent project

Date: Mon, 01 Nov 2004 09:55:46 +0100

Matt Zimmerman a écrit :
> (CCing the BTS, where this feature request is already tracked)
> 
> On Sat, Oct 30, 2004 at 10:35:53AM +0200, Arnaud Kyheng wrote:
> 
> 
>>I love the Debian project, and I have worked on a new development for 
>>it: Apt-Torrent :)
>>
>>Apt-Torrent is an apt proxy to the Bittorrent network. For security, the 
>>package listing, and the .torrent files are downloaded from a regular 
>>http server, as usual for a package, but then the whole package is 
>>fetched via bittorrent protocol and forwarded to apt :)
> 
> 
> Interesting; I hadn't considered implementing this as a proxy.  It would
> probably be better to subclass an existing HTTP server implementation,
> rather than to implement a subset of HTTP in your package.

At the moment apt-torrent is implemented like a real http server. This 
line is added in /etc/apt/sources.list by the package:
deb http://127.0.0.1:6968/debian/ unstable main

From the apt point of view, it's a real http package repository, which 
could also be a package source repository (deb-src) as well.


Arnaud

Message #35 received at 199316@bugs.debian.org (full text, mbox, reply):

From: Mike Furr <mfurr@debian.org>

To: 199316@bugs.debian.org

Cc: debian-devel@lists.debian.org, Arnaud Kyheng <Arnaud.Kyheng@free.fr>

Subject: Re: Apt-Torrent project

Date: Mon, 01 Nov 2004 12:07:10 -0500

Arnaud Kyheng wrote:
> I don't agree with the little package problem with Bittorrent. With 
> Bittornado I'm using as a backend, the super-seeder option answer to 
> this problem since if the package is already well available on the 
> network, it'll not answer to the client but let it download from peers.
[...]
> And also, if you think that the tracker overhead not worth a 5k package, 
> you could as well split the downloading system in two. I mean  put only 
>  >= 5k packages files on the apt-torrent server and let the others be 
> fetched directly in http.

This would probably help as long as you didn't abuse super-seeding.  One 
solution may be to only super seed those packages which are smaller than 
some threshold and are also in base or have a priority > standard(or 
something).  Like most things, the distribution of popular packages 
appears to have a zipf distribution(at least, according to popcon), so 
you could also gain efficiency by exploiting this data.

> This can be done easily since apt-torrent is fetching the Packages.gz as 
> usual. I mean I could add a special header in the Packages.gz 
> description to tell the proxy where to download the package direct-http, 
> or apt-torrent-server for example.
Well, I wouldn't edit the Packages.gz file directly since it will no 
longer match the hash in the Release file, I would have this in a 
separate file, if at all.


> My original idea was to save bandwidth of the Debian server, and improve 
> the downloading speed of the packages for users that are even far of a 
> mirror. I found that the Bittorrent was really mature and will fit well. 
>  In the future, I could as well use GNUnet as a backend :)
Although I personally get fantastic download speeds from the push 
primary mirrors, I guess this is not the case everywhere.  I agree that 
moving some load off of the mirror network would be beneficial.

I look forward to trying apt-torrent and hope that it works out well. 
Since it appears that you are not a debian developer, are you looking 
for someone to package/sponsor this?

-Mike

Message #40 received at 199316@bugs.debian.org (full text, mbox, reply):

From: Arnaud Kyheng <Arnaud.Kyheng@free.fr>

To: debian-devel@lists.debian.org

Cc: 199316@bugs.debian.org

Subject: Re: Apt-Torrent project

Date: Tue, 02 Nov 2004 18:19:25 +0100

Mike Furr a écrit :
> This would probably help as long as you didn't abuse super-seeding.  One 
> solution may be to only super seed those packages which are smaller than 
> some threshold and are also in base or have a priority > standard(or 
> something).  Like most things, the distribution of popular packages 
> appears to have a zipf distribution(at least, according to popcon), so 
> you could also gain efficiency by exploiting this data.

That's interesting. At the moment I'm "super-seeding" everything since 
my tracker/seeder is running on my own home ADSL connection which is too 
slow to run a server even for testing purposes.

[...]
> 
> Well, I wouldn't edit the Packages.gz file directly since it will no 
> longer match the hash in the Release file, I would have this in a 
> separate file, if at all.

Mmmm yes, or use two different lines in the sources.list file. One to 
apt-torrent and one regular.

[..]
> I look forward to trying apt-torrent and hope that it works out well. 
> Since it appears that you are not a debian developer, are you looking 
> for someone to package/sponsor this?

That would be very nice.

I've already packaged my program (only for unstable):
http://sianka.free.fr/download.html

Well, I'm a sponsored maintainer (packages gnunet and libextractor). My 
sponsor is Glenn McGrath (bug1@debian.org). I'm also in the applying Q 
to be DD.
I don't know if we can have multiple sponsors :D ?


Arnaud

Message #45 received at 199316@bugs.debian.org (full text, mbox, reply):

From: Steve Cotton <steve0001@s.cotton.clara.co.uk>

To: 199316@bugs.debian.org

Subject: #199316 Can this be closed as done (apt: bittorrent URI support)

Date: Fri, 24 Oct 2008 15:49:58 +0100

This wishlist feature is implemented by packages in Lenny.
Although apt-torrent didn't go in to the official archives, the
debtorrent and apt-transport-debtorrent packages did.

I believe this bug can be closed.
Steve

Message #50 received at 199316-done@bugs.debian.org (full text, mbox, reply):

From: "Eugene V. Lyubimkin" <jackyf.devel@gmail.com>

To: Steve Cotton <steve0001@s.cotton.clara.co.uk>, 199316-done@bugs.debian.org

Subject: Re: Bug#199316: #199316 Can this be closed as done (apt: bittorrent URI support)

Date: Fri, 24 Oct 2008 18:28:24 +0300

[Message part 1 (text/plain, inline)]

Steve Cotton wrote:
> This wishlist feature is implemented by packages in Lenny.
> Although apt-torrent didn't go in to the official archives, the
> debtorrent and apt-transport-debtorrent packages did.
> 
> I believe this bug can be closed.
> Steve
Agreed. Closing.

-- 
Eugene V. Lyubimkin aka JackYF

[signature.asc (application/pgp-signature, attachment)]

Send a report that this bug log contains spam.