Debian Bug report logs - #754242
feature request: support http://asdfasdf.onion without tor://

Reply or subscribe to this bug.

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Hans-Christoph Steiner <hans@eds.org>

To: submit@bugs.debian.org

Subject: feature request: support http://asdfasdf.onion without tor://

Date: Tue, 08 Jul 2014 22:10:25 -0400

[Message part 1 (text/plain, inline)]

Package: apt-transport-tor
Severity: wishlist
Owner: "Hans-Christoph Steiner" <hans@eds.org>

Thanks for writing this!  This may seem like a trivial issue, but I think
it'll help a lot with the usability if users can use "http://" with .onion
addresses.  I have set up a test mirror of debian-security on a Tor Hidden
Service, you are welcome to use it for testing:

http://dju2peblv7upfz3q.onion/debian-security/

I'd write a patch, but my C++ skills are weak and C string handling doesn't
seem appropriate.  C, python, Java for me...

[signature.asc (application/pgp-signature, attachment)]

Message #10 received at 754242@bugs.debian.org (full text, mbox, reply):

From: Tim Retout <diocles@debian.org>

To: Hans-Christoph Steiner <hans@eds.org>

Cc: 754242@bugs.debian.org

Subject: Re: Bug#754242: feature request: support http://asdfasdf.onion without tor://

Date: Thu, 10 Jul 2014 22:50:28 +0100

Hello!

On 9 July 2014 03:10, Hans-Christoph Steiner <hans@eds.org> wrote:
> Thanks for writing this!  This may seem like a trivial issue, but I think
> it'll help a lot with the usability if users can use "http://" with .onion
> addresses.

Because of the design of apt transports, if you put just "http" then
unfortunately the request will get sent to the http transport, not
this one.  And for that to work, you would have to add SOCKS proxy
support to apt itself, which is quite difficult.

One thing I do intend to add is support for URLs like
"tor+http://...", which is quite possible.  Also "tor+https://...".
Do you think this would be any better for usability?

Kind regards,

-- 
Tim Retout <diocles@debian.org>

Message #15 received at 754242@bugs.debian.org (full text, mbox, reply):

From: Hans-Christoph Steiner <hans@eds.org>

To: Tim Retout <diocles@debian.org>

Cc: 754242@bugs.debian.org

Subject: Re: Bug#754242: feature request: support http://asdfasdf.onion without tor://

Date: Tue, 15 Jul 2014 16:29:31 -0400

[Message part 1 (text/plain, inline)]

apt already supports SOCKS proxies, I use one for forcing all my apt traffic
over Tor in /etc/apt/apt.conf:

Acquire::socks::Proxy "socks://127.0.0.1:9050";
Acquire::https::SslForceVersion "TLSv1";
Acquire::http::Pipeline-Depth "20";
Acquire::https::Pipeline-Depth "20";

I like your URL scheme idea.  I think the ideal would be to support it with
all of these URLs:

http://asdfasdfasdfadfadf.onion
https://asdfasdfasdfadfadf.onion
tor+http://mirrors.kernel.org
tor+https://mirrors.kernel.org
tor+ftp://mirrors.kernel.org

etc. Then get rid of tor:// entirely.

.hc

[signature.asc (application/pgp-signature, attachment)]

Message #20 received at 754242@bugs.debian.org (full text, mbox, reply):

From: Tim Retout <diocles@debian.org>

To: Hans-Christoph Steiner <hans@eds.org>

Cc: 754242@bugs.debian.org

Subject: Re: Bug#754242: feature request: support http://asdfasdf.onion without tor://

Date: Fri, 18 Jul 2014 16:46:09 +0100

[Message part 1 (text/plain, inline)]

On Tue, 2014-07-15 at 16:29 -0400, Hans-Christoph Steiner wrote:
> apt already supports SOCKS proxies, I use one for forcing all my apt traffic
> over Tor in /etc/apt/apt.conf:
> 
> Acquire::socks::Proxy "socks://127.0.0.1:9050";

Unfortunately I do not believe this will work - there are various
references to this apt config setting on the internet, but none in the
source code for apt.  Worse still, apt will just silently ignore it and
route your requests over HTTP, ignoring Tor.

Please see: https://bugs.debian.org/744934

Even if SOCKS support were added to apt, you would have to be quite
careful not to leak DNS requests - you need the right sort of SOCKS.

apt-transport-tor tries to make this harder to get wrong, but the
tradeoff is that you need to put "tor" or something similar at the front
of the URLs.

> I like your URL scheme idea.  I think the ideal would be to support it with
> all of these URLs:
> 
> http://asdfasdfasdfadfadf.onion
> https://asdfasdfasdfadfadf.onion

> tor+http://mirrors.kernel.org
> tor+https://mirrors.kernel.org
> tor+ftp://mirrors.kernel.org

I can probably support the last three, but not the first two, under the
current design of apt.

Hope that helps,

-- 
Tim Retout <diocles@debian.org>

[signature.asc (application/pgp-signature, inline)]

Message #25 received at 754242@bugs.debian.org (full text, mbox, reply):

From: Hans-Christoph Steiner <hans@eds.org>

To: Tim Retout <diocles@debian.org>

Cc: 754242@bugs.debian.org

Subject: Re: Bug#754242: feature request: support http://asdfasdf.onion without tor://

Date: Fri, 18 Jul 2014 12:01:39 -0400

[Message part 1 (text/plain, inline)]

On 07/18/2014 11:46 AM, Tim Retout wrote:
> On Tue, 2014-07-15 at 16:29 -0400, Hans-Christoph Steiner wrote:
>> apt already supports SOCKS proxies, I use one for forcing all my apt traffic
>> over Tor in /etc/apt/apt.conf:
>>
>> Acquire::socks::Proxy "socks://127.0.0.1:9050";
> 
> Unfortunately I do not believe this will work - there are various
> references to this apt config setting on the internet, but none in the
> source code for apt.  Worse still, apt will just silently ignore it and
> route your requests over HTTP, ignoring Tor.
> 
> Please see: https://bugs.debian.org/744934
> 
> Even if SOCKS support were added to apt, you would have to be quite
> careful not to leak DNS requests - you need the right sort of SOCKS.
> 
> apt-transport-tor tries to make this harder to get wrong, but the
> tradeoff is that you need to put "tor" or something similar at the front
> of the URLs.

Doh, I get it now, I was misunderstanding the apt.conf syntax.

 Acquire::socks::Proxy "socks://127.0.0.1:9050";

Means proxy all socks:// URLs through socks://127.0.0.1:9050.  So really it
should read:

 Acquire::http::Proxy "socks://127.0.0.1:9050";
 Acquire::https::Proxy "socks://127.0.0.1:9050";
 Acquire::ftp::Proxy "socks://127.0.0.1:9050";



>> I like your URL scheme idea.  I think the ideal would be to support it with
>> all of these URLs:
>>
>> http://asdfasdfasdfadfadf.onion
>> https://asdfasdfasdfadfadf.onion
> 
>> tor+http://mirrors.kernel.org
>> tor+https://mirrors.kernel.org
>> tor+ftp://mirrors.kernel.org
> 
> I can probably support the last three, but not the first two, under the
> current design of apt.
> 
> Hope that helps,

Yes this is good.  Actually the Tor plugin does not need to support the
http:// and https:// onion URLs as long as there is a working SOCKS proxy that
handles the DNS stuff.  Then the user would just need to setup the SOCKS
proxy, and no plugin should be needed to handle these:

http://asdfasdfasdfadfadf.onion
https://asdfasdfasdfadfadf.onion

.hc

[signature.asc (application/pgp-signature, attachment)]

Message #30 received at 754242@bugs.debian.org (full text, mbox, reply):

From: Petter Reinholdtsen <pere@hungry.com>

To: Hans-Christoph Steiner <hans@eds.org>, 754242@bugs.debian.org

Subject: Re: Bug#754242: feature request: support http://asdfasdf.onion without tor://

Date: Fri, 15 Jan 2016 07:46:33 +0100

[Tim Retout]
> Because of the design of apt transports, if you put just "http" then
> unfortunately the request will get sent to the http transport, not
> this one.  And for that to work, you would have to add SOCKS proxy
> support to apt itself, which is quite difficult.

There are two ways to redirect http and https apt requests without rewriting
sources.list.  One is using dpkg-divert to replace /usr/lib/apt/methods/http
with /usr/lib/apt/methods/tor+http, which can be done by local admins
or the apt-transport-tor package without any cooperation with other maintainers,
and the other is to make /usr/lib/apt/methods/http an alternative in the
alternatives system, and pick the default transport implementation
using the alternatives system.

Perhaps something to consider?
-- 
Happy hacking
Petter Reinholdtsen

Message #35 received at 754242@bugs.debian.org (full text, mbox, reply):

From: Hans-Christoph Steiner <hans@eds.org>

To: Petter Reinholdtsen <pere@hungry.com>, 754242@bugs.debian.org

Subject: Re: Bug#754242: feature request: support http://asdfasdf.onion without tor://

Date: Fri, 15 Jan 2016 09:18:48 +0100

I like the combination of those two: make apt-transport-tor set itself
as the http transport by default, using /usr/lib/apt/methods/http in the
alternatives system.

Message #40 received at 754242@bugs.debian.org (full text, mbox, reply):

From: David Kalnischkies <david@kalnischkies.de>

To: 754242@bugs.debian.org

Cc: Hans-Christoph Steiner <hans@eds.org>

Subject: Re: feature request: support http://asdfasdf.onion without tor://

Date: Mon, 3 Oct 2016 17:59:48 +0200

[Message part 1 (text/plain, inline)]

Control: noowner -1
Control: tags -1 + wontfix

On Tue, Jul 08, 2014 at 10:10:25PM -0400, Hans-Christoph Steiner wrote:
> Thanks for writing this!  This may seem like a trivial issue, but I think
> it'll help a lot with the usability if users can use "http://" with .onion
> addresses.  I have set up a test mirror of debian-security on a Tor Hidden

I had played with implementing it in apt, but ultimatively decided that
this is too much magic and potentially even dangerous to enable by
default, which is why I am tagging this wontfix.


The good thing about requiring 'tor+http' is that it is clear that this
will either be sent via Tor or not at all. If just 'http' would be
allowed this isn't clear anymore: It becomes system state dependent with
silent "failure" modes. It is a frequent problem in user support
channels that a program has "mysteriously" disappeared from the system
– usually it turns out that the user missed the remark that apt is
removing the package containing that program.

Now, with a normal program the user will get an error while trying to
start it.  If the user had removed apt-transport-tor through everything
"keeps working"… That is not so much a problem with .onion addresses,
those would "just" end up as unresolveable domains – at least if you
aren't talking to a rogue DNS server (and apt will actually not ask DNS
servers about .onions by default because of that). It contains other
anomalies through: What will happen if a "normal" http request redirects
you to an onion address? (Remember, a MITM could be editing the http
request). Following the request means that we potentially contact a MITM
provided onion service which has a good chance of knowing your Tor
identify now (the default tries to avoid that by host-specific
circuits), but at least knows that you have a Tor setup.  Same problem
if you do the reverse as a rogue onion service who wants to figure out
your non-Tor identity.  That pre-depends on the user not having non-Tor
traffic entirely forbidden of course, but not everyone has all the time
on every system and as it isn't a trivial setup yet mistakes happen.

As such, I prefer a one-time safe error if you happen to have forgotten
to use tor for an onion address, rather than having it magically work,
but hiding hard errors in the long run. It is also good for consitency
as for the non-onion repositories you will need to use tor+http for
exactly the reason I mentioned first anyhow.

That said, if you really really really must, there are multiple options
to have the "normal" http behave like tor+http, the simplest one might
be to just set Acquire::http::Proxy. The way described in the README to
disable http can actually be used to "redirect" it to another method,
too, if you don't use the special value 'false' but e.g. 'tor+http'. But
as said, I don't think this is a good idea by default – or at all.

btw: If you happen to use an .onion address with http only:

Err:1 http://vwakviie2ienjx6t.onion/
  Direct connection to .onion domains is blocked by default. If you meant to use Tor remember to use tor+http instead of http.

Disclaimer: This response pre-depends on the recently uploaded 0.3
version in combination with apt 1.3.


Best regards

David Kalnischkies

P.S.: I clear the "Owner" field as evidently nobody was working on implementing
this – which is what claiming ownership means: "I will implement that" to avoid
duplicated work by multiple people working on the same bug at the same time.

[signature.asc (application/pgp-signature, inline)]

Message #49 received at 754242@bugs.debian.org (full text, mbox, reply):

From: Patrick Schleizer <adrelanos@riseup.net>

To: 754242@bugs.debian.org

Date: Sat, 14 Jan 2017 22:33:00 +0000

Is this actually implemented despite saying wontfix?

Acquire::BlockDotOnion "false";

allows connecting to http://asdfasdf.onion without tor://

Thank you for implementing 'Acquire::BlockDotOnion "false";' - very
useful for Whonix!

Best regards,
Patrick

Message #54 received at 754242@bugs.debian.org (full text, mbox, reply):

From: David Kalnischkies <david@kalnischkies.de>

To: Patrick Schleizer <adrelanos@riseup.net>, 754242@bugs.debian.org

Subject: Re: Bug#754242: support http://asdfasdf.onion without tor://

Date: Wed, 18 Jan 2017 13:51:48 +0100

[Message part 1 (text/plain, inline)]

On Sat, Jan 14, 2017 at 10:33:00PM +0000, Patrick Schleizer wrote:
> Is this actually implemented despite saying wontfix?

Well, it's implemented as described in my previous mail in this buglog:
The setting: Dir::Bin::Methods::http "http+tor"; will give you what this
bugreport requests – well, not really, it will route all traffic through
tor, not just these http-lines which have a .onion address as described
in the request. I really don't like such magic and think it would be
dangerous which is why its tagged wontfix, but more details in last
mail.

The in this bugreport requested feature is distinct (in some ways even
the opposite) of what you refer to next through:

> Thank you for implementing 'Acquire::BlockDotOnion "false";' - very
> useful for Whonix!

The option is fashioned after a similar option available in firefox: By
default if we end up trying to perform DNS queries such a try will be
canceled and fails without contacting a DNS server – that is so that by
default a misconfiguration will not cause you to leak your
onion-browsing attempt to a (potentially very) remote DNS server or even
be suspect to an evil DNS server resolving the address to some fake…

I guess Whonix is setup in a way that there is no communication to
the outside world without Tor so there is no danger of a mis-
configuration exposing you in some way. In such a world you don't need
a-t-tor at all. You will need that config knob through as you want apt
to contact a DNS server for all its needs – which your network-stack
will deal with (as in rewriting it to be routed over tor and such).

That said, it might make sense to use a-t-tor anyhow even if not
strictly needed as it will deal better with certain tor anomalies given
that it knows tor is involved reporting better errors (like telling you
that the .onion address you typo'ed is too long/short; saying
"unreachable host" if a service is… well, not reachable, instead of
saying "TTL expired" which is reported by Tor and technically more
correct but unhelpful), will use different circuits for different
sources and stuff.


Best regards

David Kalnischkies

[signature.asc (application/pgp-signature, inline)]

Send a report that this bug log contains spam.