Debian Bug report logs - #865495
tor >= 0.2.7.4-rc-1 renders CAP_NET_BIND_SERVICE on server transport plugins ineffective

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: David Fifield <david@bamsoftware.com>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: tor >= 0.2.7.4-rc-1 renders CAP_NET_BIND_SERVICE on server transport plugins ineffective

Date: Wed, 21 Jun 2017 19:56:40 -0700

Package: tor
Version: 0.2.9.10-1
Severity: normal

I'm trying to have a server transport plugin listen on a port <1024. In
the past, it has worked to set CAP_NET_BIND_SERVICE on the pluggable
transport binary (e.g. /usr/bin/obfs4proxy). But that no longer works
with the Debian tor packages; the transport binary gets a "permission
denied" error trying to bind to the port. Bisection suggests that it
stopped working in tor_0.2.7.4-rc-1.

Here are instructions to reproduce. Building a fresh VM isn't necessary;
I only did that to show it doesn't depend on weird configurations in
other packages.

Build a base VM and boot it. When I did this, it installed tor_0.2.9.10-1.
	$ sudo /usr/sbin/vmdebootstrap --verbose --image=test.img --size=1G --distribution=stretch --grub --package=tor --package=obfs4proxy --package=libcap2-bin
	$ qemu-img convert -f raw -O qcow2 test.img test.qcow2
	$ kvm -cpu host -hda test.qcow2 -m 2G

Set the capability on /usr/bin/obfs4proxy:
	# setcap 'cap_net_bind_service=+eip' /usr/bin/obfs4proxy

Add these lines to /etc/tor/torrc:
	ORPort 9001
	BridgeRelay 1
	PublishServerDescriptor 0
	AssumeReachable
	ExtORPort auto
	ServerTransportPlugin obfs4 exec /usr/bin/obfs4proxy
	ServerTransportListenAddr obfs4 0.0.0.0:443

Restart tor with `service tor restart` and check /var/log/tor/log. You
will see this warning:
	[warn] Server managed proxy encountered a method error. (obfs4 listen tcp 0.0.0.0:443: bind: permission denied)
The message that would appear instead if the setcap were working is:
	[notice] Registered server transport 'obfs4' at '[::]:443'

I tried bisecting with manually installed packages from
snapshot.debian.org. First I needed to install an older libssl1.0.0 to
support the older versions of tor:
	# wget http://snapshot.debian.org/archive/debian-security/20170127T195950Z/pool/updates/main/o/openssl/libssl1.0.0_1.0.1t-1%2Bdeb8u6_amd64.deb
	# dpkg -i libssl1.0.0_1.0.1t-1+deb8u6_amd64.deb
Then, for each tor deb, I did:
	# dpkg -i tor_${VER}_amd64.deb  # say 'N' if it prompts to overwrite torrc
	# tail -n 50 /var/log/tor/log   # check whether obfs4 worked

These are the versions I tried:
http://snapshot.debian.org/archive/debian-security/20170610T045408Z/pool/updates/main/t/tor/tor_0.2.5.14-1_amd64.deb
http://snapshot.debian.org/archive/debian/20150713T215209Z/pool/main/t/tor/tor_0.2.6.10-1_amd64.deb
http://snapshot.debian.org/archive/debian/20150731T034629Z/pool/main/t/tor/tor_0.2.7.2-alpha-1_amd64.deb
http://snapshot.debian.org/archive/debian/20150925T213031Z/pool/main/t/tor/tor_0.2.7.3-rc-1_amd64.deb
http://snapshot.debian.org/archive/debian/20151023T214618Z/pool/main/t/tor/tor_0.2.7.4-rc-1_amd64.deb
http://snapshot.debian.org/archive/debian/20151121T033923Z/pool/main/t/tor/tor_0.2.7.5-1_amd64.deb
http://snapshot.debian.org/archive/debian/20151211T214445Z/pool/main/t/tor/tor_0.2.7.6-1_amd64.deb
http://snapshot.debian.org/archive/debian/20170305T153308Z/pool/main/t/tor/tor_0.2.9.10-1_amd64.deb

Here is a summary of the bisection results:
0.2.5.14-1	Registered server transport 'obfs4' at '[::]:443'
0.2.6.10-1	Registered server transport 'obfs4' at '[::]:443'
0.2.7.2-alpha-1	Registered server transport 'obfs4' at '[::]:443'
0.2.7.3-rc-1	Registered server transport 'obfs4' at '[::]:443'
0.2.7.4-rc-1	obfs4 listen tcp 0.0.0.0:443: bind: permission denied
0.2.7.5-1	obfs4 listen tcp 0.0.0.0:443: bind: permission denied
0.2.7.6-1	obfs4 listen tcp 0.0.0.0:443: bind: permission denied
0.2.9.10-1	obfs4 listen tcp 0.0.0.0:443: bind: permission denied

The changelog for 0.2.7.4-rc-1 mentions apparmor, so I would guess that
is the source of the problem:
	tor (0.2.7.4-rc-1) experimental; urgency=medium

	  * New upstream version.
	  * Include identity tag for syslog logging feature from master.  This is
	    bug#17194 upstream.
	  * Support multiple instances (closes: #791393).
	  * Split systemd service timeout into start and stop timeout, and raise
	    them to 120 and 60 seconds from 45 (closes: tor#16398).
	  * Enable apparmor support for the default tor service (re: #761404).
	    Apparmor is not yet being enabled for any other tor instance.

A fix I tried that _doesn't_ work is to set `KeepBindCapabilities 1`
in torrc. That only causes this message to appear in the log:
	[warn] KeepBindCapabilities set, but no capability support on this system.

I'm not asking for a change in the Debian packaging, necessarily. I'd be
happy with an apparmor/systemd setting or an alternate way of running
transport plugins on low ports. I used obfs4proxy as an example above,
but the reason I need port 443 specifically is for Let's Encrypt
integration in snowflake-server, which is an HTTPS server.


-- System Information:
Debian Release: 9.0
  APT prefers stable
  APT policy: (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 4.9.0-3-amd64 (SMP w/1 CPU core)
Locale: LANG=, LC_CTYPE= (charmap=ANSI_X3.4-1968), LANGUAGE= (charmap=ANSI_X3.4-1968)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages tor depends on:
ii  adduser              3.115
ii  init-system-helpers  1.48
ii  libc6                2.24-11
ii  libevent-2.0-5       2.0.21-stable-3
ii  libseccomp2          2.3.1-2.1
ii  libssl1.1            1.1.0f-3
ii  libsystemd0          232-25
ii  lsb-base             9.20161125
ii  zlib1g               1:1.2.8.dfsg-5

Versions of packages tor recommends:
ii  logrotate    3.11.0-0.1
pn  tor-geoipdb  <none>
pn  torsocks     <none>

Versions of packages tor suggests:
pn  apparmor-utils       <none>
pn  mixmaster            <none>
ii  obfs4proxy           0.0.7-1+b2
pn  obfsproxy            <none>
pn  socat                <none>
pn  tor-arm              <none>
pn  torbrowser-launcher  <none>

-- Configuration Files:
/etc/tor/torrc changed [not included]

-- no debconf information

Message #10 received at 865495@bugs.debian.org (full text, mbox, reply):

From: Peter Palfrader <weasel@debian.org>

To: David Fifield <david@bamsoftware.com>, 865495@bugs.debian.org

Subject: Re: Bug#865495: tor >= 0.2.7.4-rc-1 renders CAP_NET_BIND_SERVICE on server transport plugins ineffective

Date: Thu, 22 Jun 2017 07:30:11 +0000

On Wed, 21 Jun 2017, David Fifield wrote:

> Package: tor
> Version: 0.2.9.10-1
> Severity: normal
> 
> I'm trying to have a server transport plugin listen on a port <1024. In
> the past, it has worked to set CAP_NET_BIND_SERVICE on the pluggable
> transport binary (e.g. /usr/bin/obfs4proxy). But that no longer works
> with the Debian tor packages; the transport binary gets a "permission
> denied" error trying to bind to the port. Bisection suggests that it
> stopped working in tor_0.2.7.4-rc-1.

Does it start working again if you add CAP_NET_BIND_SERVICE to the
CapabilityBoundingSet in the tor@default.service (or tor@.service)
systemd unit file?

-- 
                            |  .''`.       ** Debian **
      Peter Palfrader       | : :' :      The  universal
 https://www.palfrader.org/ | `. `'      Operating System
                            |   `-    https://www.debian.org/

Message #15 received at 865495@bugs.debian.org (full text, mbox, reply):

From: David Fifield <david@bamsoftware.com>

To: Peter Palfrader <weasel@debian.org>

Cc: 865495@bugs.debian.org

Subject: Re: Bug#865495: tor >= 0.2.7.4-rc-1 renders CAP_NET_BIND_SERVICE on server transport plugins ineffective

Date: Thu, 22 Jun 2017 07:18:08 -0700

On Thu, Jun 22, 2017 at 07:30:11AM +0000, Peter Palfrader wrote:
> On Wed, 21 Jun 2017, David Fifield wrote:
> 
> > Package: tor
> > Version: 0.2.9.10-1
> > Severity: normal
> > 
> > I'm trying to have a server transport plugin listen on a port <1024. In
> > the past, it has worked to set CAP_NET_BIND_SERVICE on the pluggable
> > transport binary (e.g. /usr/bin/obfs4proxy). But that no longer works
> > with the Debian tor packages; the transport binary gets a "permission
> > denied" error trying to bind to the port. Bisection suggests that it
> > stopped working in tor_0.2.7.4-rc-1.
> 
> Does it start working again if you add CAP_NET_BIND_SERVICE to the
> CapabilityBoundingSet in the tor@default.service (or tor@.service)
> systemd unit file?

CAP_NET_BIND_SERVICE is already present in the CapabilityBoundingSet of
both tor@default.service and tor@.service.

I tried adding it to the [Service] section of tor.service, and that
didn't work. I'm using the default tor instance (i.e. not one created
with tor-instance-create). I'm using the main debian.org tor package
(which don't have tor-instance-create), not the deb.torproject.org one.


==> /lib/systemd/system/tor.service <==
# This service is actually a systemd target,
# but we are using a service since targets cannot be reloaded.

[Unit]
Description=Anonymizing overlay network for TCP (multi-instance-master)

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/bin/true
ExecReload=/bin/true

[Install]
WantedBy=multi-user.target


==> /lib/systemd/system/tor@.service <==
[Unit]
Description=Anonymizing overlay network for TCP (instance %i)
After=network.target nss-lookup.target
PartOf=tor.service
ReloadPropagatedFrom=tor.service

[Service]
Type=notify
NotifyAccess=all
PIDFile=/var/run/tor-instances/%i/tor.pid
PermissionsStartOnly=yes
ExecStartPre=/usr/bin/install -Z -m 02755 -o _tor-%i -g _tor-%i -d /var/run/tor-instances/%i
ExecStartPre=/bin/sed -e 's/@@NAME@@/%i/g; w /var/run/tor-instances/%i.defaults' /usr/share/tor/tor-service-defaults-torrc-instances
ExecStartPre=/usr/bin/tor --defaults-torrc /var/run/tor-instances/%i.defaults -f /etc/tor/instances/%i/torrc --verify-config
ExecStart=/usr/bin/tor --defaults-torrc /var/run/tor-instances/%i.defaults -f /etc/tor/instances/%i/torrc
ExecReload=/bin/kill -HUP ${MAINPID}
KillSignal=SIGINT
TimeoutStartSec=300
TimeoutStopSec=60
Restart=on-failure
LimitNOFILE=65536

# Hardening
NoNewPrivileges=yes
PrivateTmp=yes
PrivateDevices=yes
ProtectHome=yes
ProtectSystem=full
ReadOnlyDirectories=/
# We would really like to restrict the next item to [..]/%i but we can't,
# as systemd does not support that yet.  See also #781730.
ReadWriteDirectories=-/var/lib/tor-instances
ReadWriteDirectories=-/var/run
CapabilityBoundingSet=CAP_SETUID CAP_SETGID CAP_NET_BIND_SERVICE CAP_DAC_OVERRIDE

[Install]
WantedBy=multi-user.target


==> /lib/systemd/system/tor@default.service <==
[Unit]
Description=Anonymizing overlay network for TCP
After=network.target nss-lookup.target
PartOf=tor.service
ReloadPropagatedFrom=tor.service

[Service]
Type=notify
NotifyAccess=all
PIDFile=/var/run/tor/tor.pid
PermissionsStartOnly=yes
ExecStartPre=/usr/bin/install -Z -m 02755 -o debian-tor -g debian-tor -d /var/run/tor
ExecStartPre=/usr/bin/tor --defaults-torrc /usr/share/tor/tor-service-defaults-torrc -f /etc/tor/torrc --RunAsDaemon 0 --verify-config
ExecStart=/usr/bin/tor --defaults-torrc /usr/share/tor/tor-service-defaults-torrc -f /etc/tor/torrc --RunAsDaemon 0
ExecReload=/bin/kill -HUP ${MAINPID}
KillSignal=SIGINT
TimeoutStartSec=300
TimeoutStopSec=60
Restart=on-failure
LimitNOFILE=65536

# Hardening
AppArmorProfile=system_tor
NoNewPrivileges=yes
PrivateTmp=yes
PrivateDevices=yes
ProtectHome=yes
ProtectSystem=full
ReadOnlyDirectories=/
ReadWriteDirectories=-/proc
ReadWriteDirectories=-/var/lib/tor
ReadWriteDirectories=-/var/log/tor
ReadWriteDirectories=-/var/run
CapabilityBoundingSet=CAP_SETUID CAP_SETGID CAP_NET_BIND_SERVICE CAP_DAC_OVERRIDE

Message #20 received at 865495@bugs.debian.org (full text, mbox, reply):

From: David Fifield <david@bamsoftware.com>

To: 865495@bugs.debian.org

Subject: Re: Bug#865495: tor >= 0.2.7.4-rc-1 renders CAP_NET_BIND_SERVICE on server transport plugins ineffective

Date: Fri, 23 Jun 2017 09:57:31 -0700

I've just discovered a ticket in the Tor bug tracker about the same
issue. The user there had upgraded to 0.2.7.6-1~d80.jessie+1 when the
problem started occurring.

"obfs4proxy cannot bind to <1024 port with systemd hardened service unit"
https://bugs.torproject.org/18356

However, the workaround suggested there (setting NoNewPrivileges=no) did
not work for me. I set NoNewPrivileges=no in both
/lib/systemd/system/tor/tor@default.service and
/lib/systemd/system/tor/tor@.service.

Message #25 received at 865495@bugs.debian.org (full text, mbox, reply):

From: David Fifield <david@bamsoftware.com>

To: 865495@bugs.debian.org

Subject: Re: Bug#865495: tor >= 0.2.7.4-rc-1 renders CAP_NET_BIND_SERVICE on server transport plugins ineffective

Date: Fri, 23 Jun 2017 10:07:58 -0700

On Fri, Jun 23, 2017 at 09:57:31AM -0700, David Fifield wrote:
> I've just discovered a ticket in the Tor bug tracker about the same
> issue. The user there had upgraded to 0.2.7.6-1~d80.jessie+1 when the
> problem started occurring.
> 
> "obfs4proxy cannot bind to <1024 port with systemd hardened service unit"
> https://bugs.torproject.org/18356
> 
> However, the workaround suggested there (setting NoNewPrivileges=no) did
> not work for me. I set NoNewPrivileges=no in both
> /lib/systemd/system/tor/tor@default.service and
> /lib/systemd/system/tor/tor@.service.

Never mind that last point. Setting NoNewPrivileges=no *does* work. I
didn't know that I had to run `systemctl daemon-reload` after modifying
the service files and before restarting tor. With the reload, it works:
	[notice] Registered server transport 'snowflake' at '[::]:443'

Message #32 received at 865495-done@bugs.debian.org (full text, mbox, reply):

From: Peter Palfrader <weasel@debian.org>

To: David Fifield <david@bamsoftware.com>, 865495-done@bugs.debian.org

Subject: Re: Bug#865495: tor >= 0.2.7.4-rc-1 renders CAP_NET_BIND_SERVICE on server transport plugins ineffective

Date: Tue, 14 Jan 2020 08:56:50 +0000

On Fri, 23 Jun 2017, David Fifield wrote:

> On Fri, Jun 23, 2017 at 09:57:31AM -0700, David Fifield wrote:
> > I've just discovered a ticket in the Tor bug tracker about the same
> > issue. The user there had upgraded to 0.2.7.6-1~d80.jessie+1 when the
> > problem started occurring.
> > 
> > "obfs4proxy cannot bind to <1024 port with systemd hardened service unit"
> > https://bugs.torproject.org/18356
> > 
> > However, the workaround suggested there (setting NoNewPrivileges=no) did
> > not work for me. I set NoNewPrivileges=no in both
> > /lib/systemd/system/tor/tor@default.service and
> > /lib/systemd/system/tor/tor@.service.
> 
> Never mind that last point. Setting NoNewPrivileges=no *does* work. I
> didn't know that I had to run `systemctl daemon-reload` after modifying
> the service files and before restarting tor. With the reload, it works:
> 	[notice] Registered server transport 'snowflake' at '[::]:443'

Great.  So if you want your service to be able to gain extra privileges,
you set NoNewPrivileges to false in your local override file.

That's not something I consider reasonable for the general tor package.

Cheers,
-- 
                            |  .''`.       ** Debian **
      Peter Palfrader       | : :' :      The  universal
 https://www.palfrader.org/ | `. `'      Operating System
                            |   `-    https://www.debian.org/

Message #37 received at 865495@bugs.debian.org (full text, mbox, reply):

From: David Fifield <david@bamsoftware.com>

To: 865495@bugs.debian.org

Subject: Re: Bug#865495: tor >= 0.2.7.4-rc-1 renders CAP_NET_BIND_SERVICE on server transport plugins ineffective

Date: Mon, 10 Feb 2020 11:33:40 -0700

On Tue, Jan 14, 2020 at 08:56:50AM +0000, Peter Palfrader wrote:
> Great.  So if you want your service to be able to gain extra privileges,
> you set NoNewPrivileges to false in your local override file.

Thanks for the hint about an override file. That's better than editing
/lib/systemd/system/tor@* because it will survive upgrades of the tor
package.

For the benefit of anyone who finds this bug report, here is what I did
to use an override or "drop-in" file.
https://bugs.torproject.org/18356#comment:10

	$ systemctl edit tor@.service tor@default.service
In the first editor that appears, enter the following text, then save
and quit:
	[Service]
	NoNewPrivileges=no
A second editor will appear. Enter the same text, then save and quit.
	[Service]
	NoNewPrivileges=no
If all goes well, you will have two new files under /etc:
	/etc/systemd/system/tor@.service.d/override.conf
	/etc/systemd/system/tor@default.service.d/override.conf
Restart tor. There is no need to run "systemctl daemon-reload".
	$ service tor restart

Send a report that this bug log contains spam.