Debian Bug report logs - #849417
nagios-nrpe-server: segfault during SSL negotiation with older NRPE 2.15 plugin

version graph

Package: nagios-nrpe-server; Maintainer for nagios-nrpe-server is Debian Nagios Maintainer Group <pkg-nagios-devel@lists.alioth.debian.org>; Source for nagios-nrpe-server is src:nagios-nrpe (PTS, buildd, popcon).

Reported by: adam@webcheckout.net

Date: Mon, 26 Dec 2016 20:27:02 UTC

Severity: normal

Tags: fixed-upstream, upstream

Found in version nagios-nrpe/3.0.1-3

Fixed in version nagios-nrpe/3.1.0-1~exp1

Done: Bas Couwenberg <sebastic@debian.org>

Bug is archived. No further changes may be made.

Forwarded to https://github.com/NagiosEnterprises/nrpe/issues/91

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, Debian Nagios Maintainer Group <pkg-nagios-devel@lists.alioth.debian.org>:
Bug#849417; Package nagios-nrpe-server. (Mon, 26 Dec 2016 20:27:04 GMT) (full text, mbox, link).


Acknowledgement sent to adam@webcheckout.net:
New Bug report received and forwarded. Copy sent to Debian Nagios Maintainer Group <pkg-nagios-devel@lists.alioth.debian.org>. (Mon, 26 Dec 2016 20:27:04 GMT) (full text, mbox, link).


Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Adam Di Carlo <adam@onshored.com>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: nagios-nrpe-server: segfault during SSL negotiation with older NRPE 2.15 plugin
Date: Mon, 26 Dec 2016 15:06:34 -0500
Package: nagios-nrpe-server
Version: 3.0.1-3
Severity: normal

Given a situation where a debian/stable (Jessie) server is polling an
NRPE node running the latest unstable NRPE server, with all debugging
enabled (ssl_logging=-1), I am getting the following segfault, as reported in
/var/log/syslog:

Dec 26 14:49:38 salsa nrpe[14736]: Connection from 192.168.1.5 port 59564
Dec 26 14:49:38 salsa nrpe[14736]: Host address is in allowed_hosts
Dec 26 14:49:38 salsa kernel: [176235.037105] nrpe[14736]: segfault at 50000335 ip 00007fd44f408496 sp 00007ffd5abfb418 error 4 in libc-2.24.so[7fd44f388000+195000]


However, if I rachet down the SSL debugging, e.g., ssl_logging=0x03,
the segfault disappears. 


-- System Information:
Debian Release: stretch/sid
  APT prefers testing
  APT policy: (1001, 'testing'), (300, 'unstable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.8.0-2-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages nagios-nrpe-server depends on:
ii  adduser              3.115
ii  init-system-helpers  1.46
ii  libc6                2.24-8
ii  libssl1.0.2          1.0.2j-4
ii  libwrap0             7.6.q-25
ii  lsb-base             9.20161125

Versions of packages nagios-nrpe-server recommends:
ii  monitoring-plugins        2.2-2
ii  monitoring-plugins-basic  2.2-2

Versions of packages nagios-nrpe-server suggests:
pn  xinetd | inetd  <none>

-- Configuration Files:
/etc/default/nagios-nrpe-server changed:
USE_SSL=1

/etc/nagios/nrpe.cfg changed:
log_facility=daemon
debug=1
pid_file=/var/run/nagios/nrpe.pid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127.0.0.1,192.168.1.5
dont_blame_nrpe=1
allow_bash_command_substitution=0
command_timeout=60
connection_timeout=300
ssl_version=SSLv2+
ssl_logging=-1
command[check_users]=/usr/lib/nagios/plugins/check_users -w 5 -c 10
command[check_load]=/usr/lib/nagios/plugins/check_load -w 15,10,5 -c 30,25,20
command[check_hda1]=/usr/lib/nagios/plugins/check_disk -w 20% -c 10% -p /dev/hda1
command[check_zombie_procs]=/usr/lib/nagios/plugins/check_procs -w 5 -c 10 -s Z
command[check_total_procs]=/usr/lib/nagios/plugins/check_procs -w 150 -c 200
include=/etc/nagios/nrpe_local.cfg
include_dir=/etc/nagios/nrpe.d/

/etc/nagios/nrpe_local.cfg changed:
include=/etc/shared/nagios/nrpe.cfg
command[check_swap]=/usr/lib/nagios/plugins/check_dummy 0
command[check_total_procs]=/usr/lib/nagios/plugins/check_procs -w 800 -c 1200
command[check_users]=/usr/lib/nagios/plugins/check_users -w 60 -c 100


-- no debconf information



Information forwarded to debian-bugs-dist@lists.debian.org, Debian Nagios Maintainer Group <pkg-nagios-devel@lists.alioth.debian.org>:
Bug#849417; Package nagios-nrpe-server. (Mon, 26 Dec 2016 21:33:04 GMT) (full text, mbox, link).


Acknowledgement sent to Sebastiaan Couwenberg <sebastic@xs4all.nl>:
Extra info received and forwarded to list. Copy sent to Debian Nagios Maintainer Group <pkg-nagios-devel@lists.alioth.debian.org>. (Mon, 26 Dec 2016 21:33:04 GMT) (full text, mbox, link).


Message #10 received at 849417@bugs.debian.org (full text, mbox, reply):

From: Sebastiaan Couwenberg <sebastic@xs4all.nl>
To: adam@webcheckout.net, 849417@bugs.debian.org
Subject: Re: [Pkg-nagios-devel] Bug#849417: nagios-nrpe-server: segfault during SSL negotiation with older NRPE 2.15 plugin
Date: Mon, 26 Dec 2016 22:16:32 +0100
[Message part 1 (text/plain, inline)]
Control: tags -1 unreproducible moreinfo

Hi Adam,

Thanks for reporting this issue. Unfortunately I cannot reproduce it.

On 12/26/2016 09:06 PM, Adam Di Carlo wrote:
> Given a situation where a debian/stable (Jessie) server is polling an
> NRPE node running the latest unstable NRPE server, with all debugging
> enabled (ssl_logging=-1), I am getting the following segfault, as reported in
> /var/log/syslog:
> 
> Dec 26 14:49:38 salsa nrpe[14736]: Connection from 192.168.1.5 port 59564
> Dec 26 14:49:38 salsa nrpe[14736]: Host address is in allowed_hosts
> Dec 26 14:49:38 salsa kernel: [176235.037105] nrpe[14736]: segfault at 50000335 ip 00007fd44f408496 sp 00007ffd5abfb418 error 4 in libc-2.24.so[7fd44f388000+195000]
> 
> However, if I rachet down the SSL debugging, e.g., ssl_logging=0x03,
> the segfault disappears. 

To help reproduce this issue, can you clarify how nagios-nrpe-server is
configured. I assume that you configured SSL before removing the -n
option of the nrpe daemon? Do you use a CA certificate, or self-signed?

-- System Information:
> -- Configuration Files:
> /etc/default/nagios-nrpe-server changed:
> USE_SSL=1

Please note that the /etc/default/nagios-nrpe-server changed in
nagios-nrpe (3.0.1-3) because of the systemd service file.

The USE_SSL option is no longer used, instead the NRPE_OPTS variable is
used to disable SSL in both the init script and systemd service file.
The default content is now as attached.

> /etc/nagios/nrpe.cfg changed:
> log_facility=daemon
> debug=1
> pid_file=/var/run/nagios/nrpe.pid
> server_port=5666
> nrpe_user=nagios
> nrpe_group=nagios
> allowed_hosts=127.0.0.1,192.168.1.5
> dont_blame_nrpe=1
> allow_bash_command_substitution=0
> command_timeout=60
> connection_timeout=300
> ssl_version=SSLv2+
> ssl_logging=-1

It doesn't look like you configured SSL, but you did enable the feature.

To use SSL in NRPE 3.x you'll need to configure at least a certificate
file (ssl_cert_file) and its key (ssl_privatekey_file), e.g. for the
snakeoil certificate generated by the ssl-cert package:

 ssl_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
 ssl_privatekey_file=/etc/ssl/private/ssl-cert-snakeoil.key

For proper SSL certificates you also need to configure the path to the
CA certificate (including intermediate certificates) in ssl_cacert_file.

Also note that setting dont_blame_nrpe=1 has no effect, the package is
not configured with --enable-command-args.

Kind Regards,

Bas

-- 
 GPG Key ID: 4096R/6750F10AE88D4AF1
Fingerprint: 8182 DE41 7056 408D 6146  50D1 6750 F10A E88D 4AF1
[nagios-nrpe-server (text/plain, attachment)]

Added tag(s) moreinfo and unreproducible. Request was from Sebastiaan Couwenberg <sebastic@xs4all.nl> to 849417-submit@bugs.debian.org. (Mon, 26 Dec 2016 21:33:04 GMT) (full text, mbox, link).


Information forwarded to debian-bugs-dist@lists.debian.org, Debian Nagios Maintainer Group <pkg-nagios-devel@lists.alioth.debian.org>:
Bug#849417; Package nagios-nrpe-server. (Tue, 27 Dec 2016 23:57:03 GMT) (full text, mbox, link).


Acknowledgement sent to Adam Di Carlo <adam@onshored.com>:
Extra info received and forwarded to list. Copy sent to Debian Nagios Maintainer Group <pkg-nagios-devel@lists.alioth.debian.org>. (Tue, 27 Dec 2016 23:57:03 GMT) (full text, mbox, link).


Message #17 received at 849417@bugs.debian.org (full text, mbox, reply):

From: Adam Di Carlo <adam@onshored.com>
To: Sebastiaan Couwenberg <sebastic@xs4all.nl>
Cc: 849417@bugs.debian.org
Subject: Re: [Pkg-nagios-devel] Bug#849417: nagios-nrpe-server: segfault during SSL negotiation with older NRPE 2.15 plugin
Date: Tue, 27 Dec 2016 18:43:59 -0500
Sebastiaan Couwenberg <sebastic@xs4all.nl> writes:

> Thanks for reporting this issue. Unfortunately I cannot reproduce it.

Oh dear.

> To help reproduce this issue, can you clarify how nagios-nrpe-server is
> configured. I assume that you configured SSL before removing the -n
> option of the nrpe daemon? Do you use a CA certificate, or
> self-signed?

Hmm, actually I left all those settings (ssl_cacert_file, ssl_cert_file,
ssl_privatekey_file) commented out.

FYI, I'm trying to interoperate with nagios-nrpe-plugin from jessie
(version 2.15-1), which doesn't seem to have any way to configure a CA
or client cert.  Any advice is welcome.

-- 
.....Adam Di Carlo....adam@debian.org.....<URL:http://www.debian.org/>



Information forwarded to debian-bugs-dist@lists.debian.org, Debian Nagios Maintainer Group <pkg-nagios-devel@lists.alioth.debian.org>:
Bug#849417; Package nagios-nrpe-server. (Wed, 28 Dec 2016 04:45:03 GMT) (full text, mbox, link).


Acknowledgement sent to Adam Di Carlo <adam@onshored.com>:
Extra info received and forwarded to list. Copy sent to Debian Nagios Maintainer Group <pkg-nagios-devel@lists.alioth.debian.org>. (Wed, 28 Dec 2016 04:45:03 GMT) (full text, mbox, link).


Message #22 received at 849417@bugs.debian.org (full text, mbox, reply):

From: Adam Di Carlo <adam@onshored.com>
To: Sebastiaan Couwenberg <sebastic@xs4all.nl>
Cc: 849417@bugs.debian.org
Subject: Re: [Pkg-nagios-devel] Bug#849417: nagios-nrpe-server: segfault during SSL negotiation with older NRPE 2.15 plugin
Date: Tue, 27 Dec 2016 23:41:57 -0500
Sebastiaan Couwenberg <sebastic@xs4all.nl> writes:

>> -- Configuration Files:
>> /etc/default/nagios-nrpe-server changed:
>> USE_SSL=1
>
> Please note that the /etc/default/nagios-nrpe-server changed in
> nagios-nrpe (3.0.1-3) because of the systemd service file.
>
> The USE_SSL option is no longer used, instead the NRPE_OPTS variable is
> used to disable SSL in both the init script and systemd service file.
> The default content is now as attached.

Gotit.

I'll work my way through your instructions, attempt to fix my interop
issue.  Its always *overconfiguration* that gets me.

Thank you for taking the time to help!


However, no matter my legacy misconfig, isn't it still problematic to
segfault like this?  Let me know if a backtrace would help.

-- 
...Adam Di Carlo...<adam@onshored.com>.......<URL:http://www.onshored.com/>



Information forwarded to debian-bugs-dist@lists.debian.org, Debian Nagios Maintainer Group <pkg-nagios-devel@lists.alioth.debian.org>:
Bug#849417; Package nagios-nrpe-server. (Wed, 28 Dec 2016 06:33:04 GMT) (full text, mbox, link).


Acknowledgement sent to Sebastiaan Couwenberg <sebastic@xs4all.nl>:
Extra info received and forwarded to list. Copy sent to Debian Nagios Maintainer Group <pkg-nagios-devel@lists.alioth.debian.org>. (Wed, 28 Dec 2016 06:33:04 GMT) (full text, mbox, link).


Message #27 received at 849417@bugs.debian.org (full text, mbox, reply):

From: Sebastiaan Couwenberg <sebastic@xs4all.nl>
To: Adam Di Carlo <adam@onshored.com>
Cc: 849417@bugs.debian.org
Subject: Re: [Pkg-nagios-devel] Bug#849417: nagios-nrpe-server: segfault during SSL negotiation with older NRPE 2.15 plugin
Date: Wed, 28 Dec 2016 07:29:45 +0100
On 12/28/2016 05:41 AM, Adam Di Carlo wrote:
> Sebastiaan Couwenberg <sebastic@xs4all.nl> writes:
> 
>>> -- Configuration Files:
>>> /etc/default/nagios-nrpe-server changed:
>>> USE_SSL=1
>>
>> Please note that the /etc/default/nagios-nrpe-server changed in
>> nagios-nrpe (3.0.1-3) because of the systemd service file.
>>
>> The USE_SSL option is no longer used, instead the NRPE_OPTS variable is
>> used to disable SSL in both the init script and systemd service file.
>> The default content is now as attached.
> 
> Gotit.
> 
> I'll work my way through your instructions, attempt to fix my interop
> issue.  Its always *overconfiguration* that gets me.

As documented in /usr/share/doc/nagios-nrpe-server/NEWS.Debian.gz which
is shown to you on upgrade when you have apt-listchanges installed:

"
  SSL support is disabled by default, the reworked SSL/TLS support in
  NRPE requires configuration before it can be used. Read the
  instructions in /usr/share/doc/nagios-nrpe-server/README.SSL.md.gz
  before enabling SSL support in /etc/default/nagios-nrpe-server.

  The default check_nrpe command in check_nrpe.cfg has been updated
  to disable SSL by default too. The check_nrpe_ssl command has been
  added to connect to the NRPE daemon over SSL.

  Beware that the new NRPE daemon only works with old check_nrpe
  plugins when SSL support is disabled on both sides, likewise the
  new check_nrpe plugin only works with the old NRPE daemon when SSL
  support is disabled.

  To use SSL between the NRPE client and server, configuring Stunnel
  is recommended.
"

Once all systems have upgraded to NRPE 3.x using its SSL support is an
option, but that will take some time (no other distributions have
upgraded to 3.x yet).

> Thank you for taking the time to help!
> 
> 
> However, no matter my legacy misconfig, isn't it still problematic to
> segfault like this?  Let me know if a backtrace would help.

Due to the signal handler in NRPE you won't easily get a backtrace since
SIGSEGV is caught too and NRPE just continues instead of terminating. If
you can get a backtrace (with debug symbols installed) that would be
helpful.

Kind Regards,

Bas

-- 
 GPG Key ID: 4096R/6750F10AE88D4AF1
Fingerprint: 8182 DE41 7056 408D 6146  50D1 6750 F10A E88D 4AF1



Information forwarded to debian-bugs-dist@lists.debian.org, Debian Nagios Maintainer Group <pkg-nagios-devel@lists.alioth.debian.org>:
Bug#849417; Package nagios-nrpe-server. (Wed, 28 Dec 2016 18:09:03 GMT) (full text, mbox, link).


Acknowledgement sent to Adam Di Carlo <adam@onshored.com>:
Extra info received and forwarded to list. Copy sent to Debian Nagios Maintainer Group <pkg-nagios-devel@lists.alioth.debian.org>. (Wed, 28 Dec 2016 18:09:03 GMT) (full text, mbox, link).


Message #32 received at 849417@bugs.debian.org (full text, mbox, reply):

From: Adam Di Carlo <adam@onshored.com>
To: Sebastiaan Couwenberg <sebastic@xs4all.nl>
Cc: 849417@bugs.debian.org
Subject: Re: [Pkg-nagios-devel] Bug#849417: nagios-nrpe-server: segfault during SSL negotiation with older NRPE 2.15 plugin
Date: Wed, 28 Dec 2016 13:07:16 -0500
Sebastiaan Couwenberg <sebastic@xs4all.nl> writes:

> As documented in /usr/share/doc/nagios-nrpe-server/NEWS.Debian.gz which
> is shown to you on upgrade when you have apt-listchanges installed:
[...]
>   Beware that the new NRPE daemon only works with old check_nrpe
>   plugins when SSL support is disabled on both sides, likewise the
>   new check_nrpe plugin only works with the old NRPE daemon when SSL
>   support is disabled.

Oh!  I totally didn't see that.  Ok.  So what I'm trying to do will
never work and I need to disable SSL for all NRPE servers as well as on
my (Jessie) nagios server.

>   To use SSL between the NRPE client and server, configuring Stunnel
>   is recommended.

I suppose that disabling SSL, so long as I also disable the NRPE
argument processing on the older NRPEs which allow it, won't create too
many security issues on an internal network.  The most an attacker could
do, assuming they could spoof my the one allowed IP that commands can
come from, is run the checks configured on the NRPE server.  So, there
is a denial-of-service risk here but not much more than that....

Pardon me for failing to RTM here.

> Due to the signal handler in NRPE you won't easily get a backtrace since
> SIGSEGV is caught too and NRPE just continues instead of terminating. If
> you can get a backtrace (with debug symbols installed) that would be
> helpful.

Ok, I'll give it a whack.  Lets leave the bug in "moreinfo" until I get
that.  I do believe I need to rebuild the package with '-g' to get
symbols out, which I've done.  Off to work for now but I'll give this
another attempt, should have result by no later than end of day tomorrow.

-- 
...Adam Di Carlo...<adam@onshored.com>.......<URL:http://www.onshored.com/>



Information forwarded to debian-bugs-dist@lists.debian.org, Debian Nagios Maintainer Group <pkg-nagios-devel@lists.alioth.debian.org>:
Bug#849417; Package nagios-nrpe-server. (Wed, 28 Dec 2016 18:42:02 GMT) (full text, mbox, link).


Acknowledgement sent to Sebastiaan Couwenberg <sebastic@xs4all.nl>:
Extra info received and forwarded to list. Copy sent to Debian Nagios Maintainer Group <pkg-nagios-devel@lists.alioth.debian.org>. (Wed, 28 Dec 2016 18:42:02 GMT) (full text, mbox, link).


Message #37 received at 849417@bugs.debian.org (full text, mbox, reply):

From: Sebastiaan Couwenberg <sebastic@xs4all.nl>
To: Adam Di Carlo <adam@onshored.com>
Cc: 849417@bugs.debian.org
Subject: Re: [Pkg-nagios-devel] Bug#849417: nagios-nrpe-server: segfault during SSL negotiation with older NRPE 2.15 plugin
Date: Wed, 28 Dec 2016 19:38:44 +0100
[Message part 1 (text/plain, inline)]
On 12/28/2016 07:07 PM, Adam Di Carlo wrote:
> Sebastiaan Couwenberg <sebastic@xs4all.nl> writes:
> 
>> As documented in /usr/share/doc/nagios-nrpe-server/NEWS.Debian.gz which
>> is shown to you on upgrade when you have apt-listchanges installed:
> [...]
>>   Beware that the new NRPE daemon only works with old check_nrpe
>>   plugins when SSL support is disabled on both sides, likewise the
>>   new check_nrpe plugin only works with the old NRPE daemon when SSL
>>   support is disabled.
> 
> Oh!  I totally didn't see that.  Ok.  So what I'm trying to do will
> never work and I need to disable SSL for all NRPE servers as well as on
> my (Jessie) nagios server.

You only need to disable SSL for NRPE >= 3.0. The SSL support for NRPE
2.x still works.

For example, on my jessie server I changed the check_nrpe commands to
match the configuration in NRPE 3.x (see attached check_nrpe.cfg) by
modifying /etc/nagios-plugins/config/check_nrpe.cfg.

In the service configuration I changed all check_nrpe_1arg commands to
check_nrpe_ssl, and for the hosts running testing/unstable I changed it
to check_nrpe. Once the jessie systems get upgraded to stretch their
service configuration will be changed to use check_nrpe instead of
check_nrpe_ssl too.

>>   To use SSL between the NRPE client and server, configuring Stunnel
>>   is recommended.
> 
> I suppose that disabling SSL, so long as I also disable the NRPE
> argument processing on the older NRPEs which allow it, won't create too
> many security issues on an internal network.  The most an attacker could
> do, assuming they could spoof my the one allowed IP that commands can
> come from, is run the checks configured on the NRPE server.  So, there
> is a denial-of-service risk here but not much more than that....

The SSL support in NRPE 2.x never got you much security on your internal
network (it did not verify the hostname for example), it mostly
obfuscated tcpdumps. Disabling the NRPE arguments brings you much more
security that the (broken) SSL support in NRPE 2.x.

> Pardon me for failing to RTM here.
> 
>> Due to the signal handler in NRPE you won't easily get a backtrace since
>> SIGSEGV is caught too and NRPE just continues instead of terminating. If
>> you can get a backtrace (with debug symbols installed) that would be
>> helpful.
> 
> Ok, I'll give it a whack.  Lets leave the bug in "moreinfo" until I get
> that.  I do believe I need to rebuild the package with '-g' to get
> symbols out, which I've done.  Off to work for now but I'll give this
> another attempt, should have result by no later than end of day tomorrow.

The debug symbols are already available, no need to a rebuild. Just
install the nagios-nrpe-server-dbgsym package. You may need to configure
the sources for that first, e.g. for unstable:

# Debug packages
deb http://debug.mirrors.debian.org/debian-debug/ unstable-debug main
contrib non-free

Kind Regards,

Bas

-- 
 GPG Key ID: 4096R/6750F10AE88D4AF1
Fingerprint: 8182 DE41 7056 408D 6146  50D1 6750 F10A E88D 4AF1
[check_nrpe.cfg (text/plain, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Nagios Maintainer Group <pkg-nagios-devel@lists.alioth.debian.org>:
Bug#849417; Package nagios-nrpe-server. (Sun, 01 Jan 2017 03:09:04 GMT) (full text, mbox, link).


Acknowledgement sent to Adam Di Carlo <adam@onshored.com>:
Extra info received and forwarded to list. Copy sent to Debian Nagios Maintainer Group <pkg-nagios-devel@lists.alioth.debian.org>. (Sun, 01 Jan 2017 03:09:04 GMT) (full text, mbox, link).


Message #42 received at 849417@bugs.debian.org (full text, mbox, reply):

From: Adam Di Carlo <adam@onshored.com>
To: Sebastiaan Couwenberg <sebastic@xs4all.nl>
Cc: 849417@bugs.debian.org
Subject: Re: [Pkg-nagios-devel] Bug#849417: nagios-nrpe-server: segfault during SSL negotiation with older NRPE 2.15 plugin
Date: Sat, 31 Dec 2016 22:07:24 -0500
Sebastiaan Couwenberg <sebastic@xs4all.nl> writes:

> The debug symbols are already available, no need to a rebuild. Just
> install the nagios-nrpe-server-dbgsym package.
[...]

Thanks, that system is new to me!

>>> Due to the signal handler in NRPE you won't easily get a backtrace since
>>> SIGSEGV is caught too and NRPE just continues instead of terminating. If
>>> you can get a backtrace (with debug symbols installed) that would be
>>> helpful.

It didn't really give me too much trouble.  I think gdb replaces all the
signal handlers anyhow.

To recap my current behavior, in case things maybe changed subtly here,
here's the logging I get in daemon.log with ssl_debug set to 0x0f:

Dec 31 21:37:22 salsa nrpe[24931]: Allowing connections from: 127.0.0.1,192.168.1.5
Dec 31 21:37:27 salsa nrpe[24935]: Connection from 192.168.1.5 port 42463
Dec 31 21:37:27 salsa nrpe[24935]: Host address is in allowed_hosts
Dec 31 21:37:27 salsa nrpe[24935]: Error: Could not complete SSL handshake with 192.168.1.5: 1
Dec 31 21:37:27 salsa nrpe[24935]: Connection from 192.168.1.5 closed.


Whereas if I set it to 0xff:
Dec 31 21:36:23 salsa nrpe[24897]: Allowing connections from: 127.0.0.1,192.168.1.5
Dec 31 21:36:30 salsa nrpe[24899]: Connection from 192.168.1.5 port 41951
Dec 31 21:36:30 salsa nrpe[24899]: Host address is in allowed_hosts

and then in kernl.log:
Dec 31 21:36:30 salsa kernel: [632644.965865] nrpe[24899]: segfault at
b0935335 ip 00007f3fafd3d496 sp 00007ffee43c9dc8 error 4 in
libc-2.24.so[7f3fafcbd000+195000]


Here's my gdb session and the best backtrace I was able to get out:

# gdb /usr/sbin/nrpe 24967
(gdb) set follow-fork-mode child
(gdb) c
Continuing.
[New process 25047]
[New process 25048]

Thread 3.1 "nrpe" received signal SIGSEGV, Segmentation fault.
[Switching to process 25048]
strlen () at ../sysdeps/x86_64/strlen.S:106
106	../sysdeps/x86_64/strlen.S: No such file or directory.
#0  strlen () at ../sysdeps/x86_64/strlen.S:106
#1  0x00007fc8e3c34da3 in _IO_vfprintf_internal (s=s@entry=0x561cf790d280, format=<optimized out>, format@entry=0x561cf6be9eb8 "Error: Could not complete SSL handshake with %s: %s", 
    ap=0x7fff6996e188) at vfprintf.c:1637
#2  0x00007fc8e3ce2f66 in ___vfprintf_chk (fp=fp@entry=0x561cf790d280, flag=flag@entry=1, format=format@entry=0x561cf6be9eb8 "Error: Could not complete SSL handshake with %s: %s", 
    ap=ap@entry=0x7fff6996e188) at vfprintf_chk.c:33
#3  0x00007fc8e3ccfad8 in __GI___vsyslog_chk (pri=<optimized out>, flag=1, fmt=0x561cf6be9eb8 "Error: Could not complete SSL handshake with %s: %s", ap=ap@entry=0x7fff6996e188)
    at ../misc/syslog.c:222
#4  0x00007fc8e3ccffd2 in __syslog_chk (pri=<optimized out>, flag=<optimized out>, fmt=<optimized out>) at ../misc/syslog.c:129
#5  0x0000561cf6be51ba in syslog (__fmt=0x561cf6be9eb8 "Error: Could not complete SSL handshake with %s: %s", __pri=3) at /usr/include/x86_64-linux-gnu/bits/syslog.h:31
#6  handle_conn_ssl (sock=<optimized out>, ssl_ptr=0x561cf78f7b70) at ./nrpe.c:1753
#7  0x0000561cf6be6a53 in handle_connection (sock=6) at ./nrpe.c:1491
#8  0x0000561cf6be7085 in wait_for_connections () at ./nrpe.c:1198
#9  0x0000561cf6be71c3 in run_src () at ./nrpe.c:506
#10 0x0000561cf6be288c in main (argc=<optimized out>, argv=<optimized out>) at ./nrpe.c:198

(gdb) frame 6
#6  handle_conn_ssl (sock=<optimized out>, ssl_ptr=0x561cf78f7b70) at ./nrpe.c:1753
1753	./nrpe.c: No such file or directory.
nerrs = 0
c = <optimized out>
buffer = "\000\000\000\000\000\000\000\000\324\006\000\000\000\000\000\000\250\310\311\344\310\177\000\000\220\375\276\343\310\177\000\000\070п\343\310\177\000\000SI\250\344\310\177\000\000\324\006\000\000\000\000\000\000\070п\343\310\177\000\000\250\310\311\344\310\177\000\000\070\343\226i\377\177\000\000\064\343\226i\377\177\000\000\313B\250\344\310\177\000\000\020\265\370\343\310\177\000\000(\252\370\343\310\177\000\000\070\343\226i\377\177\000\000\066\025\025e\000\000\000\000TT\224\001\000\000\000\000\070п\343\310\177\000\000\020\344\226i\377\177\000\000\220\375\276\343\310\177\000\000\064\343\226i\377\177\000\000\000\344\226i\377\177\000\000PF\306\344\310\177\000\000\b", '\000' <repeats 47 times>...
ssl = 0x561cf78f7b70
peer = <optimized out>
rc = <optimized out>
x = <optimized out>


Let me know if you're still stumped.   I think my next step would be to
have to try to hack sources and come up with a diff which fixes matters.

Also, I'm clearly missing some debug symbols, covering
.../sysdeps/x86_64/strlen.S, but not sure what package I need to install
to cover that.

-- 
...Adam Di Carlo...<adam@onshored.com>.......<URL:http://www.onshored.com/>



Information forwarded to debian-bugs-dist@lists.debian.org, Debian Nagios Maintainer Group <pkg-nagios-devel@lists.alioth.debian.org>:
Bug#849417; Package nagios-nrpe-server. (Sun, 01 Jan 2017 11:18:03 GMT) (full text, mbox, link).


Acknowledgement sent to Sebastiaan Couwenberg <sebastic@xs4all.nl>:
Extra info received and forwarded to list. Copy sent to Debian Nagios Maintainer Group <pkg-nagios-devel@lists.alioth.debian.org>. (Sun, 01 Jan 2017 11:18:03 GMT) (full text, mbox, link).


Message #47 received at 849417@bugs.debian.org (full text, mbox, reply):

From: Sebastiaan Couwenberg <sebastic@xs4all.nl>
To: Adam Di Carlo <adam@onshored.com>
Cc: 849417@bugs.debian.org
Subject: Re: [Pkg-nagios-devel] Bug#849417: nagios-nrpe-server: segfault during SSL negotiation with older NRPE 2.15 plugin
Date: Sun, 1 Jan 2017 12:16:15 +0100
tags 849417 - unreproducible moreinfo
tags 849417 + upstream
forwarded 849417 https://github.com/NagiosEnterprises/nrpe/issues/91
thanks

Hi Adam,

Thanks for the additional debugging. I've now been able to reproduce the
issue on a Debian unstable VM, and have forwarded the issue upstream.

On 01/01/2017 04:07 AM, Adam Di Carlo wrote:
> Sebastiaan Couwenberg <sebastic@xs4all.nl> writes:
> 
>> The debug symbols are already available, no need to a rebuild. Just
>> install the nagios-nrpe-server-dbgsym package.
> [...]
> 
> Thanks, that system is new to me!

Debug packages have existed for quite some time, the automatic dbgsym
packages are new in stretch, see: https://wiki.debian.org/DebugPackage

> Let me know if you're still stumped.   I think my next step would be to
> have to try to hack sources and come up with a diff which fixes matters.

That would be excellent, please forward your proposed fix upstream.

> Also, I'm clearly missing some debug symbols, covering
> .../sysdeps/x86_64/strlen.S, but not sure what package I need to install
> to cover that.

You need the libc source for that.

Kind Regards,

Bas

-- 
 GPG Key ID: 4096R/6750F10AE88D4AF1
Fingerprint: 8182 DE41 7056 408D 6146  50D1 6750 F10A E88D 4AF1



Removed tag(s) unreproducible and moreinfo. Request was from Sebastiaan Couwenberg <sebastic@xs4all.nl> to control@bugs.debian.org. (Sun, 01 Jan 2017 11:18:04 GMT) (full text, mbox, link).


Added tag(s) upstream. Request was from Sebastiaan Couwenberg <sebastic@xs4all.nl> to control@bugs.debian.org. (Sun, 01 Jan 2017 11:18:05 GMT) (full text, mbox, link).


Set Bug forwarded-to-address to 'https://github.com/NagiosEnterprises/nrpe/issues/91'. Request was from Sebastiaan Couwenberg <sebastic@xs4all.nl> to control@bugs.debian.org. (Sun, 01 Jan 2017 11:18:06 GMT) (full text, mbox, link).


Added tag(s) fixed-upstream. Request was from Bas Couwenberg <sebastic@debian.org> to control@bugs.debian.org. (Mon, 03 Apr 2017 17:03:06 GMT) (full text, mbox, link).


Reply sent to Bas Couwenberg <sebastic@debian.org>:
You have taken responsibility. (Wed, 19 Apr 2017 18:09:07 GMT) (full text, mbox, link).


Notification sent to adam@webcheckout.net:
Bug acknowledged by developer. (Wed, 19 Apr 2017 18:09:07 GMT) (full text, mbox, link).


Message #60 received at 849417-close@bugs.debian.org (full text, mbox, reply):

From: Bas Couwenberg <sebastic@debian.org>
To: 849417-close@bugs.debian.org
Subject: Bug#849417: fixed in nagios-nrpe 3.1.0-1~exp1
Date: Wed, 19 Apr 2017 18:06:00 +0000
Source: nagios-nrpe
Source-Version: 3.1.0-1~exp1

We believe that the bug you reported is fixed in the latest version of
nagios-nrpe, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 849417@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Bas Couwenberg <sebastic@debian.org> (supplier of updated nagios-nrpe package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.8
Date: Wed, 19 Apr 2017 19:28:05 +0200
Source: nagios-nrpe
Binary: nagios-nrpe-server nagios-nrpe-plugin
Architecture: source amd64
Version: 3.1.0-1~exp1
Distribution: experimental
Urgency: medium
Maintainer: Debian Nagios Maintainer Group <pkg-nagios-devel@lists.alioth.debian.org>
Changed-By: Bas Couwenberg <sebastic@debian.org>
Description:
 nagios-nrpe-plugin - Nagios Remote Plugin Executor Plugin
 nagios-nrpe-server - Nagios Remote Plugin Executor Server
Closes: 445976 691328 849417 856658 859223
Changes:
 nagios-nrpe (3.1.0-1~exp1) experimental; urgency=medium
 .
   * New upstream release.
     (closes: #849417, #445976, #691328)
   * Fix typo in manpage.
     (closes: #856658)
   * Drop 10_reproducible_build.patch, applied upstream.
     Refresh remaining patches.
   * Update build dependency for OpenSSL 1.1.0.
     (closes: #859223)
   * Add patch to fix FTBFS with -Werror=format-security.
Checksums-Sha1:
 ed7d88e82dcdfedc3df17f5771d9dcc8871d94c3 2127 nagios-nrpe_3.1.0-1~exp1.dsc
 4ddbdbb50e4a5dfd6efe89515a24b76dba4b98ae 501028 nagios-nrpe_3.1.0.orig.tar.gz
 225d9eed9a0881ba7c4d3f235d00a07c8f65482a 13736 nagios-nrpe_3.1.0-1~exp1.debian.tar.xz
 3806f969b05a8f18962285b970dfbee223c96911 38426 nagios-nrpe-plugin-dbgsym_3.1.0-1~exp1_amd64.deb
 204ef8ca21b80ac524cdac2800df95f127424ec5 31352 nagios-nrpe-plugin_3.1.0-1~exp1_amd64.deb
 8435f0c6dc2e7d9c9016b9869b87bb3f2daa2637 57448 nagios-nrpe-server-dbgsym_3.1.0-1~exp1_amd64.deb
 4886c0e82fdbd06a1182f1f89d4abf5cbf9bf286 349812 nagios-nrpe-server_3.1.0-1~exp1_amd64.deb
 86e059855a5814dab27a58712befde83ebbd88e3 6131 nagios-nrpe_3.1.0-1~exp1_amd64.buildinfo
Checksums-Sha256:
 591c2c4180d83c80f8861c5e2a9bcb49c3333ee03f7d899ff40e8cddb666d32d 2127 nagios-nrpe_3.1.0-1~exp1.dsc
 f37598beade42a33a1869f99ddf9930b3c7bce38ab407d671d4469ef7233b507 501028 nagios-nrpe_3.1.0.orig.tar.gz
 4ed54f3a4e20246dad1ee90feca842d467ba4892680d15321117ea89eede31be 13736 nagios-nrpe_3.1.0-1~exp1.debian.tar.xz
 93eeb909f826554d7f5c1f6c337af789f6ea1beb848602da5dc53a4a0efc3f8f 38426 nagios-nrpe-plugin-dbgsym_3.1.0-1~exp1_amd64.deb
 0925a59316fa3646d5abbbf205722ab5f2f989f64a686ab8c3d37e9ff192aeee 31352 nagios-nrpe-plugin_3.1.0-1~exp1_amd64.deb
 7bcdd61f6bf6f1e44522fdd252cfb3b1f9cff70b4608f2e2eedd36b28dfb0cc0 57448 nagios-nrpe-server-dbgsym_3.1.0-1~exp1_amd64.deb
 dc00dbb24bf3ea79356f738d022ef9ca6f1a695efa082fbf9174e8a1459d1411 349812 nagios-nrpe-server_3.1.0-1~exp1_amd64.deb
 9ed14e777849972ad6a06e46bb5656ae069fb81f904f4a7025a00fc925b1b7b8 6131 nagios-nrpe_3.1.0-1~exp1_amd64.buildinfo
Files:
 6a2237e7402a92e4d0ca67418cd1afa3 2127 net optional nagios-nrpe_3.1.0-1~exp1.dsc
 95112ca774de7f8e856c5c4b42c1fb48 501028 net optional nagios-nrpe_3.1.0.orig.tar.gz
 cb79a0339885ef8fdcff5303edf006eb 13736 net optional nagios-nrpe_3.1.0-1~exp1.debian.tar.xz
 ae90c242029adfc52f9826585f660f8d 38426 debug extra nagios-nrpe-plugin-dbgsym_3.1.0-1~exp1_amd64.deb
 d003f3b986b609858f78e608686039ed 31352 net optional nagios-nrpe-plugin_3.1.0-1~exp1_amd64.deb
 7a7036dbc8d948cf0dbc9ccf556927a0 57448 debug extra nagios-nrpe-server-dbgsym_3.1.0-1~exp1_amd64.deb
 cf645f53a0719739a38e4f460be275ee 349812 net optional nagios-nrpe-server_3.1.0-1~exp1_amd64.deb
 031c97b725a955f2fbebd277ab250217 6131 net optional nagios-nrpe_3.1.0-1~exp1_amd64.buildinfo

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCgAGBQJY96NVAAoJEGdQ8QrojUrxzaoP/0CPjAaTih32H1uf/2E9AxJr
Plw9Lbvjxc3Uhs2w25beROVq1NdhZ+m1OChRA9/g8Ijy38a1RWT9UBTm/agcvDwh
8T4LXyJv2DraTp9cLr3FddBb3wZ4t7GNFaoWHr2mWfeZMoiT7Qh6eJSHTZmpltoh
MWi/OfO5v0j/9ocsihBIQUWWDFu4tXpycEsn0Npx7zymFDWp6M+XmgH/rTKbMJK3
ahd3Sz4R4z/crKhZrwOLSBePlTFmFIrwFVIL3JykVctFN4RI+RvmAjSk8zk/MoCT
TEqkSjC7nCT9uvODZHCSZcIuZYTfA7O6jlG4VIjoPfkHxrHKZBYeB0eEbeuWumOb
Q2nmsXAvl1sTp86+xKhb/TZSFQ2IbFYJridLuv72bkOJTxfzH8Y9pUj/b1c4pLTR
Iv0mooXELQd2zHH0yKZmAfehpHucyeufeHYlh4hBBcj82ORroSVMtN+85CpyZ2O9
4UDub//kFes3IEKVuMRoMssi1/LIYC6fWsHLyoacqqWLDlXKDLZPCPfd7dHggwbH
XNokev3ZBNT9+bE4CCURDvlES945dA2eehGDFv0Ys1yXAKZcSLRoxkC+4r8/+6d9
jhkd5Y9io5PHgJbVlW1dfxf+may6Qku3z+WQdlqbyWTW7UaFpxkgEI0fBoH3YGXt
RdoSTXnETHVQm/Z2Yt/e
=j7Ii
-----END PGP SIGNATURE-----




Bug archived. Request was from Debbugs Internal Request <owner@bugs.debian.org> to internal_control@bugs.debian.org. (Fri, 23 Jun 2017 07:25:33 GMT) (full text, mbox, link).


Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Wed Jan 10 08:16:42 2018; Machine Name: beach

Debian Bug tracking system

Debbugs is free software and licensed under the terms of the GNU Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.

Copyright © 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson, 2005-2017 Don Armstrong, and many other contributors.