Debian Bug report logs - #836266
parcimonie should not modify the user's dirmngr.conf

Reply or subscribe to this bug.

Display info messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Kyuma Ohta <whatisthis.sowhat@gmail.com>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: dirmngr: Please disable "use-tor" by default.

Date: Thu, 01 Sep 2016 17:00:19 +0900

Package: dirmngr
Version: 2.1.15-2
Severity: important

Dear Maintainer,
Running gnupg2 with dirmngr, sometimes failed to connect to keyserver.
I checked configuration files, automatically add "use-tor" to
~/.gnupg/dirmngr.conf .
So, I stopped dirmngr via gpgconf, and delete this line of dirmngr.conf,
and running gnupg2 to access to keyserver, this succeeded.

But, after about 15min, re-run gnupg2 to access to keyserver, faied.
I checked dirmngr.conf again, added "use-tor" again (；´Д｀)

At some ISPs, tor connection has rejected by default, maybe gnupg2
fails to connect to keyserver by default.

Best regards.
Ohta.

-- System Information:
Debian Release: stretch/sid
  APT prefers unstable-debug
  APT policy: (500, 'unstable-debug'), (500, 'unstable'), (500, 'stable'), (1, 'experimental-debug'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.7.0-1-amd64 (SMP w/8 CPU cores)
Locale: LANG=ja_JP.UTF-8, LC_CTYPE=ja_JP.UTF-8 (charmap=UTF-8) (ignored: LC_ALL set to ja_JP.UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages dirmngr depends on:
ii  adduser        3.115
ii  libassuan0     2.4.3-1
ii  libc6          2.24-1
ii  libgcrypt20    1.7.3-1
ii  libgnutls30    3.5.3-3
ii  libgpg-error0  1.24-1
ii  libksba8       1.3.4-4
ii  libldap-2.4-2  2.4.42+dfsg-2+b2
ii  libnpth0       1.2-3
ii  lsb-base       9.20160629

Versions of packages dirmngr recommends:
ii  gnupg  2.1.15-2

Versions of packages dirmngr suggests:
ii  tor  0.2.8.7-1

-- no debconf information

Message #10 received at 836266@bugs.debian.org (full text, mbox, reply):

From: Felipe Sateler <fsateler@debian.org>

To: 836266@bugs.debian.org

Subject: Re: dirmngr: Please disable "use-tor" by default.

Date: Wed, 7 Sep 2016 12:31:17 -0300

On Thu, 01 Sep 2016 17:00:19 +0900 Kyuma Ohta
<whatisthis.sowhat@gmail.com> wrote:
> Package: dirmngr
> Version: 2.1.15-2
> Severity: important
>
> Dear Maintainer,
> Running gnupg2 with dirmngr, sometimes failed to connect to keyserver.
> I checked configuration files, automatically add "use-tor" to
> ~/.gnupg/dirmngr.conf .
> So, I stopped dirmngr via gpgconf, and delete this line of dirmngr.conf,
> and running gnupg2 to access to keyserver, this succeeded.
>
> But, after about 15min, re-run gnupg2 to access to keyserver, faied.
> I checked dirmngr.conf again, added "use-tor" again (；´Д｀)
>
> At some ISPs, tor connection has rejected by default, maybe gnupg2
> fails to connect to keyserver by default.
>

Not only this, but dirmngr does not detect the availability of tor,
and thus fails to work if tor is not installed (or uninstalled after
the first use).


Saludos

Message #15 received at 836266@bugs.debian.org (full text, mbox, reply):

From: Daniel Kahn Gillmor <dkg@fifthhorseman.net>

To: Felipe Sateler <fsateler@debian.org>, 836266@bugs.debian.org, 836266-submitter@bugs.debian.org

Subject: Re: [pkg-gnupg-maint] Bug#836266: dirmngr: Please disable "use-tor" by default.

Date: Thu, 08 Sep 2016 01:31:44 +0200

[Message part 1 (text/plain, inline)]

Control: reassign 836266 parcimonie
Control: affects 836266 + dirmngr

On Wed 2016-09-07 17:31:17 +0200, Felipe Sateler wrote:
> On Thu, 01 Sep 2016 17:00:19 +0900 Kyuma Ohta <whatisthis.sowhat@gmail.com> wrote:
>> Package: dirmngr
>> Version: 2.1.15-2
>> Severity: important
>>
>> Dear Maintainer,
>> Running gnupg2 with dirmngr, sometimes failed to connect to keyserver.
>> I checked configuration files, automatically add "use-tor" to
>> ~/.gnupg/dirmngr.conf .

dirmngr defaults to having use-tor set to false, and does not
automatically add use-tor to ~/.gnupg/dirmngr.conf

>> So, I stopped dirmngr via gpgconf, and delete this line of dirmngr.conf,
>> and running gnupg2 to access to keyserver, this succeeded.
>>
>> But, after about 15min, re-run gnupg2 to access to keyserver, faied.
>> I checked dirmngr.conf again, added "use-tor" again

I suspect that you have parcimonie installed, since it appears to
automatically ask dirmngr to use tor:

  https://codesearch.debian.net/search?q=use-tor&perpkg=1

Do you have it installed?  Is it configured and running for you?

>> At some ISPs, tor connection has rejected by default, maybe gnupg2
>> fails to connect to keyserver by default.
>
> Not only this, but dirmngr does not detect the availability of tor,
> and thus fails to work if tor is not installed (or uninstalled after
> the first use).

If dirmngr is configured to use tor, and tor is off or broken or
uninstalled, it should indeed fail to work (rather than contacting the
keyservers without using tor).  that's proper behavior, i think.
(though i agree that better error messages or warnings would make the
failure easier to diagnose and understand, and would be a positive
improvement)

I'm reassigning this bug to parcimonie, because that's where use-tor is
being set automatically.

           --dkg

[signature.asc (application/pgp-signature, inline)]

Message #29 received at 836266@bugs.debian.org (full text, mbox, reply):

From: Gabriel Filion <gabster@lelutin.ca>

To: 836266@bugs.debian.org

Subject: Re: dirmngr: Please disable "use-tor" by default.

Date: Tue, 22 Nov 2016 06:40:00 +0000

[Message part 1 (text/plain, inline)]

Hi there,

For what it's worth I've been experiencing the same issue up to now:
impossible to search or get keys with "use-tor" enabled, which was
forced into the config file seemingly by parcimonie.

However it's just been resolved by an upgrade of gnupg/dirmngr to the
latest version in sid, 2.1.16-2.

I believe the fix to use-tor issues was added in 2.1.15-9 by using adns
for better DNS resolution.

[signature.asc (application/pgp-signature, attachment)]

Message #34 received at 836266@bugs.debian.org (full text, mbox, reply):

From: intrigeri <intrigeri@debian.org>

To: 836266@bugs.debian.org

Subject: Re: [Pkg-privacy-maintainers] Bug#836266: dirmngr: Please disable "use-tor" by default.

Date: Tue, 06 Dec 2016 16:34:26 +0100

Dear users and fellow maintainers,

I am very much in doubt about what to do with this bug report.

Ideally, I think that parcimonie would start its own dirmngr instance,
fork the configuration of the default instance, and enable use-tor in
its own copy. I guess that's doable, but I am not convinced it's worth
the effort.

Let me make my case in favour of not changing anything. Given the
purpose of parcimonie, in the use cases I can think of, it makes lots
of sense that key fetches done outside of parcimonie go through Tor as
well. Moreover, one may see the current behaviour (forcing use-tor
into dirmngr.conf) as a fail-safe mechanism that makes running e.g.
"gpg --refresh-keys" slightly less harmful (in the threat model
parcimonie is concerned about) than if it were done without using Tor.
IIRC some OpenPGP GUIs (Seahorse, Enigmail) make it really easy to
trigger such an operation, possibly without realizing the
consequences. So it may be argued that the current behaviour is the
best parcimonie can provide to those who feel the need to install it:
one may argue that it does the right thing, even if that does not
match the most advanced users' expectations.

Now, I have to admit that the currently resulting UX is sometimes
painful. In the past few months I often had to ask dirmngr to forget
that it thought my configured keyserver was down, otherwise it would
simply not even try, although my network connectivity + Tor client
were up again. But I did not notice any such problem recently, which
makes my life easier again. I suspect that dirmngr was recently
improved in this respect.

Thoughts, opinions, user experience feedback and patches are
welcome :)

> However it's just been resolved by an upgrade of gnupg/dirmngr to the
> latest version in sid, 2.1.16-2.

I concur: current dirmngr works just fine (again) for me.

> I believe the fix to use-tor issues was added in 2.1.15-9 by using adns
> for better DNS resolution.

Note that the adns support was then reverted in 2.1.16-2, "due to lack
of security support (Closes: #845078)".

Cheers,
-- 
intrigeri

Message #39 received at 836266@bugs.debian.org (full text, mbox, reply):

From: Daniel Kahn Gillmor <dkg@fifthhorseman.net>

To: intrigeri <intrigeri@debian.org>, 836266@bugs.debian.org

Subject: Re: [Pkg-privacy-maintainers] Bug#836266: Bug#836266: dirmngr: Please disable "use-tor" by default.

Date: Tue, 06 Dec 2016 12:55:15 -0500

On Tue 2016-12-06 10:34:26 -0500, intrigeri wrote:
> Now, I have to admit that the currently resulting UX is sometimes
> painful. In the past few months I often had to ask dirmngr to forget
> that it thought my configured keyserver was down, otherwise it would
> simply not even try, although my network connectivity + Tor client
> were up again. But I did not notice any such problem recently, which
> makes my life easier again. I suspect that dirmngr was recently
> improved in this respect.

I concur with intrigeri that this is not a problem that should be fixed
in parcimonie.

dirmngr has a range of problems right now with its understanding of
which hosts are alive and dead.  I've been reporting them upstream, some
of them have been fixed, and some haven't yet.

For example:

   https://bugs.gnupg.org/gnupg/issue2438

some of them interact with tor, some interact with hkps, and others are
simply problems with lower-level DNS apis, etc.

avoiding the use of tor for parcimonie seems like it would (a) not solve
the underlying problems, and (b) be more likely to expose users to
network surveillance.

       --dkg

Message #44 received at 836266@bugs.debian.org (full text, mbox, reply):

From: intrigeri <intrigeri@debian.org>

To: Daniel Kahn Gillmor <dkg@fifthhorseman.net>

Cc: 836266@bugs.debian.org

Subject: Re: [Pkg-privacy-maintainers] Bug#836266: Bug#836266: dirmngr: Please disable "use-tor" by default.

Date: Wed, 07 Dec 2016 15:09:33 +0100

Hi,

Daniel Kahn Gillmor:
> dirmngr has a range of problems right now with its understanding of
> which hosts are alive and dead.  I've been reporting them upstream, some
> of them have been fixed, and some haven't yet.

Cool, glad there's WIP on this front, and thanks for your work :)

> avoiding the use of tor for parcimonie seems like it would (a) not solve
> the underlying problems, and (b) be more likely to expose users to
> network surveillance.

Sure. To be fair though, I don't think anyone suggested that
parcimonie stops using Tor. I think that what's at stake is whether it
is acceptable that parcimonie configures dirmngr to use Tor, in a way
that applies to *all* other uses of dirmngr.

Cheers,
-- 
intrigeri

Message #49 received at 836266@bugs.debian.org (full text, mbox, reply):

From: intrigeri <intrigeri@debian.org>

To: 836266@bugs.debian.org

Subject: Re: Bug#836266: [Pkg-privacy-maintainers] Bug#836266: dirmngr: Please disable "use-tor" by default.

Date: Sat, 24 Dec 2016 09:08:56 +0100

intrigeri:
>> I believe the fix to use-tor issues was added in 2.1.15-9 by using adns
>> for better DNS resolution.

> Note that the adns support was then reverted in 2.1.16-2, "due to lack
> of security support (Closes: #845078)".

… and 2.1.17 upstream includes a change that might improve things in
this area:

 * dirmngr: Support for the ADNS library has been removed.  Instead
   William Ahern's Libdns is now source included and used on all
   platforms.  This enables Tor support on all platforms.

Cheers,
-- 
intrigeri

Message #54 received at 836266@bugs.debian.org (full text, mbox, reply):

From: Antoine Beaupre <anarcat@debian.org>

To: intrigeri <intrigeri@debian.org>, 836266@bugs.debian.org

Cc: Daniel Kahn Gillmor <dkg@fifthhorseman.net>

Subject: Re: Bug#836266: [Pkg-privacy-maintainers] Bug#836266: Bug#836266: dirmngr: Please disable "use-tor" by default.

Date: Tue, 10 Jan 2017 14:15:43 -0500

[Message part 1 (text/plain, inline)]

Package: parcimonie
Version: 0.10.2-4
Followup-For: Bug #836266

I suffered from this same problem here on a fresh stretch install:
parcimonie rewrote my dirmngr.conf to add use-tor and broke all
network operations *outside* of parcimonie.

This is a change of behavior and a significant regression from jessie,
where parcimonie would do its thing on the side and would leave the
default gpg configuration alone for the most part.

While I am sensitive to the argument that *all* gpg network operations
go through tor, I do not think this policy should be *enforced* by
parcimonie, let alone silently and after a non-user-visible
timeout. Having something rewrite a configuration file behind your
back is one of the most troubling and confusing user experiences we
can provide to our users and I believe we should avoid doing that as
much as possible.

"use-tor" should be a *runtime* configuration that parcimonie uses. I do
not care much how this is implemented: it can be a separate dirmngr, or
something that trickles down from the gpg commandline to dirmngr and
reconfigures it on the fly.

I *want* to fetch certain keys *directly*, *not* going through a tor
exit node. There are simple and valid reasons to still be able to do that
while having parcimonie to run in the background, if only to verify
certain key material outside of the tor network.

The fact that dirmngr and gpg have horrible failure modes and error
messages certainly doesn't help here, but shouldn't keep parcimonie from
doing the right thing and not destroy the policies i set in my
configuration.

As things stand now, I see no choice but to stop using parcimonie, which
means:

 1. i will not update my keyring in a timely manner anymore, or;
 2. i will reveal my keyring social graph to the keyserver and
    possible attackers

That seems like the opposite of what parcimonie is trying to
accomplish.

A.

[signature.asc (application/pgp-signature, inline)]

Message #59 received at 836266@bugs.debian.org (full text, mbox, reply):

From: Daniel Kahn Gillmor <dkg@fifthhorseman.net>

To: Antoine Beaupre <anarcat@debian.org>, 836266@bugs.debian.org

Subject: Re: [Pkg-privacy-maintainers] Bug#836266: Bug#836266: Bug#836266: dirmngr: Please disable "use-tor" by default.

Date: Tue, 10 Jan 2017 19:28:59 -0500

[Message part 1 (text/plain, inline)]

On Tue 2017-01-10 14:15:43 -0500, Antoine Beaupre wrote:
> As things stand now, I see no choice but to stop using parcimonie, which
> means:
>
>  1. i will not update my keyring in a timely manner anymore, or;
>  2. i will reveal my keyring social graph to the keyserver and
>     possible attackers
>
> That seems like the opposite of what parcimonie is trying to
> accomplish.

fwiw, the ideal long-term fix here is that the logic of parcimonie gets
folded into dirmngr itself, and just works automatically if tor is
available.

i've got sketches for how this would work if anyone has the time to work
on it.

Please see https://bugs.gnupg.org/gnupg/issue1827 for more details.

   --dkg

[signature.asc (application/pgp-signature, inline)]

Message #64 received at 836266@bugs.debian.org (full text, mbox, reply):

From: Phil Morrell <debian@emorrp1.name>

To: Debian Bug Tracking System <836266@bugs.debian.org>

Subject: Re: dirmngr: Please disable "use-tor" by default.

Date: Thu, 12 Apr 2018 20:51:07 +0100

Package: parcimonie
Version: 0.10.2-4
Followup-For: Bug #836266

words



-- System Information:
Debian Release: 9.4
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.9.0-6-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8), LANGUAGE=en_GB:en (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages parcimonie depends on:
ii  dirmngr                      2.1.18-8~deb9u1
ii  gnupg                        2.1.18-8~deb9u1
ii  gnupg2                       2.1.18-8~deb9u1
ii  libclone-perl                0.38-2+b1
ii  libconfig-general-perl       2.63-1
ii  libfile-homedir-perl         1.00-1
ii  libfile-which-perl           1.21-1
ii  libgnupg-interface-perl      0.52-9
ii  libipc-system-simple-perl    1.25-3
ii  liblist-moreutils-perl       0.416-1+b1
ii  libmoo-perl                  2.002005-1
ii  libmoox-late-perl            0.015-2
ii  libmoox-options-perl         4.023-1
ii  libnamespace-clean-perl      0.27-1
ii  libpath-tiny-perl            0.100-1
ii  libtime-duration-parse-perl  0.13-1
ii  libtry-tiny-perl             0.28-1
ii  libtype-tiny-perl            1.000005-1
ii  libtypes-path-tiny-perl      0.005-1
ii  perl                         5.24.1-3+deb9u2
ii  torsocks                     2.2.0-1+deb9u1

Versions of packages parcimonie recommends:
pn  gnupg-curl              <none>
ii  libglib-perl            3:1.324-1
ii  libgtk3-perl            0.030-1
ii  liblocale-gettext-perl  1.07-3+b1
ii  libnet-dbus-glib-perl   0.33.0-2+b1
ii  libnet-dbus-perl        1.1.0-4+b1
ii  libpango-perl           1.227-1+b1
ii  libtime-duration-perl   1.20-1
ii  tor                     0.2.9.14-1

parcimonie suggests no packages.

-- no debconf information

Message #69 received at 836266@bugs.debian.org (full text, mbox, reply):

From: Phil Morrell <debian@emorrp1.name>

To: Debian Bug Tracking System <836266@bugs.debian.org>

Subject: Re: dirmngr: Please disable "use-tor" by default.

Date: Thu, 12 Apr 2018 21:22:45 +0100

[Message part 1 (text/plain, inline)]

Package: parcimonie
Version: 0.10.2-4
Followup-For: Bug #836266

I want to add my 2c to this bug report, sharing the same user
frustrations as anarcat above. I don't know if any more recent tooling
versions (be that parcimonie, dirmngr, gnupg, torsocks) have improved
the situation, as it's not in stretch-backports.

In the absence of a longer term solution, parcimonie should respect user
edits to dirmngr.conf i.e. I don't have a massive objection to it adding
use-tor initially, but if I've removed it (perhaps temporarily to
receive a single key without tor errors), then don't get into an editing
war with me. I'm even happy if this disables parcimonie until I put it
back (with a log window message).

When I see the parcimonie log error:

	Failed to fetch key 6ACBAD6A729326258CF725C6E7519C8D747F00DC: gpg: keyserver receive failed: No data
	 at /usr/share/perl5/App/Parcimonie/Daemon.pm line 350.

I now run this to fix the tor connections:

	systemctl --user restart dirmngr.socket

I realise this is a dirmngr issue, but it's also a parcimonie issue as a
"privacy-friendly helper to refresh a GnuPG keyring" which is likely to
be run by people like me trying to get into best practices. You said
above you're unsure "what to do with this bug report", at the very least
I'd like it documented in the man-page (if my workaround above is
correct in the general case). Ideally in the short to medium term,
parcimonie could detect a series of sequential (likely) tor-related
errors and explicitly write this in the logs, perhaps with the socket
restart recommendation, perhaps lengthening the sleep to e.g. 2hrs so it
can be fixed in user scale time.

-- System Information:
Debian Release: 9.4
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.9.0-6-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8), LANGUAGE=en_GB:en (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages parcimonie depends on:
ii  dirmngr                      2.1.18-8~deb9u1
ii  gnupg                        2.1.18-8~deb9u1
ii  gnupg2                       2.1.18-8~deb9u1
ii  libclone-perl                0.38-2+b1
ii  libconfig-general-perl       2.63-1
ii  libfile-homedir-perl         1.00-1
ii  libfile-which-perl           1.21-1
ii  libgnupg-interface-perl      0.52-9
ii  libipc-system-simple-perl    1.25-3
ii  liblist-moreutils-perl       0.416-1+b1
ii  libmoo-perl                  2.002005-1
ii  libmoox-late-perl            0.015-2
ii  libmoox-options-perl         4.023-1
ii  libnamespace-clean-perl      0.27-1
ii  libpath-tiny-perl            0.100-1
ii  libtime-duration-parse-perl  0.13-1
ii  libtry-tiny-perl             0.28-1
ii  libtype-tiny-perl            1.000005-1
ii  libtypes-path-tiny-perl      0.005-1
ii  perl                         5.24.1-3+deb9u2
ii  torsocks                     2.2.0-1+deb9u1

Versions of packages parcimonie recommends:
pn  gnupg-curl              <none>
ii  libglib-perl            3:1.324-1
ii  libgtk3-perl            0.030-1
ii  liblocale-gettext-perl  1.07-3+b1
ii  libnet-dbus-glib-perl   0.33.0-2+b1
ii  libnet-dbus-perl        1.1.0-4+b1
ii  libpango-perl           1.227-1+b1
ii  libtime-duration-perl   1.20-1
ii  tor                     0.2.9.14-1

parcimonie suggests no packages.

-- no debconf information

[signature.asc (application/pgp-signature, inline)]

Message #76 received at 836266@bugs.debian.org (full text, mbox, reply):

From: Antoine Beaupré <anarcat@debian.org>

To: 836266@bugs.debian.org

Cc: Daniel Kahn Gillmor <dkg@fifthhorseman.net>, intrigeri <intrigeri@debian.org>

Subject: Re: [Pkg-privacy-maintainers] Bug#836266: Bug#836266: Bug#836266: Bug#836266: dirmngr: Please disable "use-tor" by default.

Date: Mon, 27 Aug 2018 22:06:06 -0400

On 2017-01-10 18:28:59, Daniel Kahn Gillmor wrote:
> On Tue 2017-01-10 14:15:43 -0500, Antoine Beaupre wrote:
>> As things stand now, I see no choice but to stop using parcimonie, which
>> means:
>>
>>  1. i will not update my keyring in a timely manner anymore, or;
>>  2. i will reveal my keyring social graph to the keyserver and
>>     possible attackers
>>
>> That seems like the opposite of what parcimonie is trying to
>> accomplish.
>
> fwiw, the ideal long-term fix here is that the logic of parcimonie gets
> folded into dirmngr itself, and just works automatically if tor is
> available.
>
> i've got sketches for how this would work if anyone has the time to work
> on it.
>
> Please see https://bugs.gnupg.org/gnupg/issue1827 for more details.

We're now a year and a half later and I've upgraded my box to Debian
Buster. Remembering this issue (and that I had no mechanism for key
refresh still!) I figured I would give it a shot again. So I installed
parcimonie and let it run for a while. It seems to do its thing but it's
hard to tell as I'm not sure if there are logs anywhere.

That said, `use-tor` is still as problematic as it was back in
stretch. As a random example, I tried to search for a key on a
keyserver, and gpg just failed to get anything. The diagnostics are, as
usual, fairly limited, but the error Phil Morrell got is pretty typical:

       Failed to fetch key 6ACBAD6A729326258CF725C6E7519C8D747F00DC: gpg: keyserver receive failed: No data

Since it's not the first time I have had this problem, I have full debug
enabled in dirmngr (hello metadata leakage!). This translate into this
error:

2018-08-27 21:20:17 dirmngr[9652.6] erreur d'accès à « https://193.164.133.100:443/pks/lookup?op=index&options=mr&search=milan%40debian%2Eorg » : état HTTP 502
2018-08-27 21:20:17 dirmngr[9652.6] command 'KS_SEARCH' failed: Pas de données
2018-08-27 21:20:17 dirmngr[9652.6] DBG: chan_6 -> ERR 167772218 Pas de données <Dirmngr>

What's dumb here is that dirmngr doesn't fallback to other
servers. Repeated attempts to search keys will fail with the same
incomprehensible and useless error message. ("HTTP status 502" would
actually be more useful for debugging, for example, along with which
host is responsible - but GnuPG only blesses us only with "no data".) So
the bug in dirmngr here at least is that it doesn't rotate to the next
available server in the pool when failing with tor.

The workaround, as Morrell explained, is to restart dirmngr which
magically picks another host and moves on.

I know this is not Parcimonie's fault. It's gnupg's fault or, more
precisely, dirmngr's, but it seems difficult to change things over
there: this would require rewriting dirmngr's network routines or
reimplementing parcimonie within dirmngr itself.

After spending years fighting with GnuPG at various levels, neither
looks very attractive or accessible to me as a developer right now.

Instead, I've started thinking about what a parcimonie rewrite would
look like, one that would *not* depend on dirmngr (or, in fact, any
specific OpenPGP implementation). If you permit, I would like to use
this space to brainstorm such a design, which can be broken up into the
following step:

 1. build a random list of keys to fetch, idempotently
 2. talk with Tor
 3. fetch keys from a keyserver
 4. validate the keys (!)
 5. reinject in the data store

In details, it would look something like this:

 1. The first step is to enumerate keys. This requires talking to the
    keystore: it can be done with gpg --export but that means a lot of
    data. Parsing the `--list-keys --with-colons` output might be
    mandated here, as much as it hurts me to even think about this. This
    would load a list of fingerprints to refresh. This list should be
    sorted internally, and then copied and shuffled so we have a list of
    keys to iterate over reliably. This process can be repeated after a
    timeout: new keys would be added, sorted, to the sorted list and
    then added in a random location in the shuffled list.

 2. Fetching Tor is not that complicated, and is the cornerstone of this
    program. Talking to it is simply like talking with a SOCKS5 proxy,
    something which Python requests supports since 2.10, if my memory
    serves me right (jessie-backports and above).

 3. Then parcimonie also needs to talk to keyservers: right now it lets
    gnupg do the talking, but this is actually fairly easy as well, as
    HKP was implemented on top of a few HTTP verbs. I have implemented
    such a client in the PGPy library in a few lines of code:

    https://github.com/SecurityInnovation/PGPy/blob/d5e46733df34f14f83bda5ed2bc0bcc13bd971e3/pgpy/types.py#L267

 4. This actually parses the packet as well and this is where things get
    a little more complicated: what's an acceptable response from a
    keyserver?  This is another thing that's delegated to GnuPG right
    now, but it would be interesting to formalize this and (self-?)
    authenticate the key material. Or can we delegate *just* that bit to
    GnuPG?

 5. Then the last step is simply to feed the resulting key material back
    into the keystore, either gpg --import or whatever backend we want
    supported.

All this doesn't seem that complicated to me. The tricky bit is the gate
to keep garbage and hostile keys from going into the keyring. It would
probably be where the rubber meets the road, as there's an incredible
variety of stuff on keyservers. Not that many programs or libraries can
parse this reliably or at all, GnuPG included. PGPy, in particular, is
not very well tooled for this just yet and cannot correctly parse ECC
keys, for example.

I would welcome feedback on how this could be done, or if it's just an
incredibly stupid idea.

Thanks, and sorry for hijacking this thread with such wild ideas. :)

A.

-- 
Rock journalism is people who can't write, interviewing people who can't
talk, in order to provide articles for people who can't read.
                        - Frank Zappa

Message #81 received at 836266@bugs.debian.org (full text, mbox, reply):

From: Antoine Beaupré <anarcat@debian.org>

To: 836266@bugs.debian.org

Cc: Daniel Kahn Gillmor <dkg@fifthhorseman.net>, intrigeri <intrigeri@debian.org>

Subject: Re: [Pkg-privacy-maintainers] Bug#836266: Bug#836266: Bug#836266: Bug#836266: dirmngr: Please disable "use-tor" by default.

Date: Mon, 27 Aug 2018 22:50:36 -0400

>  4. This actually parses the packet as well and this is where things get
>     a little more complicated: what's an acceptable response from a
>     keyserver?  This is another thing that's delegated to GnuPG right
>     now, but it would be interesting to formalize this and (self-?)
>     authenticate the key material. Or can we delegate *just* that bit to
>     GnuPG?

I guess this whole re-implementation feasibility question can be
summarized as such:

    Is `gpg --import` safe to run against untrusted data? If not, how
    does it differ from `gpg --recv-keys`?

A.

-- 
They say that time changes things, but you actually have to change
them yourself.           - Andy Warhol

Message #86 received at 836266@bugs.debian.org (full text, mbox, reply):

From: intrigeri <intrigeri@debian.org>

To: Antoine Beaupré <anarcat@debian.org>, 836266@bugs.debian.org

Cc: Daniel Kahn Gillmor <dkg@fifthhorseman.net>

Subject: Re: [Pkg-privacy-maintainers] Bug#836266: Bug#836266: Bug#836266: Bug#836266: dirmngr: Please disable "use-tor" by default.

Date: Tue, 30 Oct 2018 19:08:32 +0100

Hi!

Antoine Beaupré:
> I know this is not Parcimonie's fault. It's gnupg's fault or, more
> precisely, dirmngr's, but it seems difficult to change things over
> there: this would require rewriting dirmngr's network routines

… at least so they're network-status aware and don't treat "my system
is offline" as "the keyserver is down".

> or reimplementing parcimonie within dirmngr itself.

Sure, that would be ideal. Note that it *also* requires fixing dirmngr.

> Instead, I've started thinking about what a parcimonie rewrite would
> look like, one that would *not* depend on dirmngr (or, in fact, any
> specific OpenPGP implementation). If you permit, I would like to use
> this space to brainstorm such a design […]

I'm glad you're bringing such out-of-the-box thinking in this space!
It's refreshing. I did not put much thought into it yet but at first
glance, your design makes sense to me.

> All this doesn't seem that complicated to me. The tricky bit is the gate
> to keep garbage and hostile keys from going into the keyring.

Agreed, that was my concern as well.

> I would welcome feedback on how this could be done, or if it's just an
> incredibly stupid idea.

I'll happily let you reuse the parcimonie name once you have it
working with good enough™ backwards compatibility with the
current interfaces.

> Thanks, and sorry for hijacking this thread with such wild ideas. :)

By all means, please keep going wild :)

Cheers,
-- 
intrigeri

Message #91 received at 836266@bugs.debian.org (full text, mbox, reply):

From: Antoine Beaupré <anarcat@debian.org>

To: intrigeri <intrigeri@debian.org>, 836266@bugs.debian.org

Cc: Daniel Kahn Gillmor <dkg@fifthhorseman.net>

Subject: Re: [Pkg-privacy-maintainers] Bug#836266: Bug#836266: Bug#836266: Bug#836266: dirmngr: Please disable "use-tor" by default.

Date: Tue, 30 Oct 2018 14:28:50 -0400

On 2018-10-30 19:08:32, intrigeri wrote:

[...]

>> Instead, I've started thinking about what a parcimonie rewrite would
>> look like, one that would *not* depend on dirmngr (or, in fact, any
>> specific OpenPGP implementation). If you permit, I would like to use
>> this space to brainstorm such a design […]
>
> I'm glad you're bringing such out-of-the-box thinking in this space!
> It's refreshing. I did not put much thought into it yet but at first
> glance, your design makes sense to me.

Thanks! :)

>> All this doesn't seem that complicated to me. The tricky bit is the gate
>> to keep garbage and hostile keys from going into the keyring.
>
> Agreed, that was my concern as well.

As it turns out, while doing a review of the upcoming gpg2 update for
stretch, I found out that GnuPG supports "filters" in gpg --import,
through the `--import-filter` commandline option. Some key improvements
of that are being backported to stretch as well. This is basically what
we'd be looking at to implement this crucial component:

https://manpages.debian.org/unstable/gpg/gpg.1.en.html#FILTER_EXPRESSIONS

Through those, there should be a way to avoid importing secret keys and
make sure the imported key material matches (one of?) the UID we are
looking at. It's too bad we can't restrict on a fingerprint there...

Something to think about, anyways.

>> I would welcome feedback on how this could be done, or if it's just an
>> incredibly stupid idea.
>
> I'll happily let you reuse the parcimonie name once you have it
> working with good enough™ backwards compatibility with the
> current interfaces.

Glad to hear that.

A.
-- 
Sous un gouvernement qui emprisonne injustement, la place de l’homme
juste est aussi en prison.
                        - La désobéissance civile, Henry David Thoreau

Message #98 received at 836266@bugs.debian.org (full text, mbox, reply):

From: intrigeri <intrigeri@debian.org>

To: 836266@bugs.debian.org

Subject: Re: Bug#836266: parcimonie should not modify the user's dirmngr.conf

Date: Tue, 21 Apr 2020 13:52:43 +0200

Hi,

intrigeri (2016-12-06):
> Dear users and fellow maintainers,
>
> I am very much in doubt about what to do with this bug report.
>
> Ideally, I think that parcimonie would start its own dirmngr instance,
> fork the configuration of the default instance, and enable use-tor in
> its own copy. I guess that's doable, but I am not convinced it's worth
> the effort.

I've researched this avenue a bit today and the best design I could
think of is this:

 - Introduce dirmngr-with-tor.{socket,service} user units,
   that only differ from the existing dirmngr.{socket,service} units
   like this:

    - ListenStream=%t/gnupg/S.dirmngr-with-tor
    - ExecStart=/usr/bin/dirmngr --supervised --use-tor

   Additionally, one would need to figure out how what to do with
   "ExecReload=/usr/bin/gpgconf --reload dirmngr".

 - Give gpg(1) a command-line argument to specify which dirmngr socket
   shall be used.

   If there's already a way to do this, that does not require patching
   GnuPG, I'm all ears.

Then, parcimonie-like programs could use that new option to ask gpg(1)
to use the extra, torified dirmngr instance. And parcimonie could stop
modifying the user's dirmngr.conf.

FWIW, while looking for a hopefully simpler fix, I've also thought
about:

 - Use "gpg --dirmngr-program"

   As long as there's already an instance listening on S.dirmngr,
   gpg seems to ignore this. I guess that makes sense.

 - Run "gpg --recv-keys" in an environment where S.dirmngr
   has been replaced by a torified dirmngr's socket.

   I understand that creating such an environment requires one of:

    - a setuid-root helper, such as bwrap

    - root privileges (e.g. to bind-mount stuff) → probably not
      OK in this context

    - user namespaces enabled, which I doubt we can assume in this
      context

    - symlink'ing dance + custom $GPGHOME → sounds fragile and racy

    - kill dirmngr and run our own temporarily → sounds fragile and
      racy

And of course, one should not forget about:

 - anarcat's idea of replacing the "download + validate/filter +
   import" part of gpg --recv-keys's functionality with something
   that does not use dirmngr.

 - Some day, someone will teach dirmngr to do parcimonie's job :)

Send a report that this bug log contains spam.