Debian Bug report logs - #340601
ldapsearch hangs when using ldap for /etc/hosts

version graph

Package: ldap-utils; Maintainer for ldap-utils is Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>; Source for ldap-utils is src:openldap.

Reported by: Arthur de Jong <arthur@west.nl>

Date: Thu, 24 Nov 2005 14:03:01 UTC

Severity: important

Tags: confirmed, patch, upstream

Found in version ldap-utils/2.2.23-8

Fixed in version openldap2.3/2.4.7-6

Done: Steve Langasek <vorlon@debian.org>

Bug is archived. No further changes may be made.

Forwarded to http://www.openldap.org/its/index.cgi/?findid=5363

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, Torsten Landschoff <torsten@debian.org>:
Bug#340601; Package ldap-utils. Full text and rfc822 format available.

Acknowledgement sent to Arthur de Jong <arthur@west.nl>:
New Bug report received and forwarded. Copy sent to Torsten Landschoff <torsten@debian.org>. Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: Arthur de Jong <arthur@west.nl>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: ldapsearch hangs when using ldap for /etc/hosts
Date: Thu, 24 Nov 2005 14:53:37 +0100
Subject: ldapsearch hangs when using ldap for /etc/hosts
Package: ldap-utils
Version: 2.2.23-8
Severity: important
File: /usr/bin/ldapsearch

ldapsearch hangs when host nss information is retrieved from an LDAP
server and a user uses ldapsearch to search a different LDAP server.

The relevant parts of the relevant configfiles:
  /etc/nsswitch.conf:
    hosts:          files ldap dns
  /etc/libnss-ldap.conf:
    host 10.10.10.10
    base dc=domain,dc=tld
    ldap_version 3
  /etc/hosts:
    10.10.10.10   ldapserver1

When I run ldapsearch with the name of an LDAP server it hangs (searches
on ldapserver1 work as expected):

% ldapsearch -v -h ldapserver2 -D '....' -x -W 'uid=*'
ldap_initialize( ldap://ldapserver2 )
Enter LDAP Password:
*ldapsearch hangs*

(ldapserver2 does not have the address 10.10.10.10 and is not listed
in /etc/hosts)

% strace ldapsearch -v -h ldapserver2 -D '....' -x -W 'uid=*'
[...] 
rt_sigaction(SIGPIPE, {SIG_IGN}, {SIG_IGN}, 8) = 0
geteuid32()                             = 181
futex(0xb7c7aafc, FUTEX_WAKE, 2147483647) = 0
open("/etc/libnss-ldap.conf", O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=9108, ...}) = 0
mmap2(NULL, 131072, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7a76000
read(3, "###DEBCONF###\n# the configuratio"..., 131072) = 9108
read(3, "", 131072)                     = 0
close(3)                                = 0
munmap(0xb7a76000, 131072)              = 0
geteuid32()                             = 181
futex(0xb7fd1598, FUTEX_WAIT, 2, NULL

If I substitute the ip address of ldapserver2 ldapsearch produces the
expected results.

It appears that the LDAP library uses some locking mechanism to ensure
that it does not do two requests at the same time. Without looking at
the code, it appears that this lock is aquired before the LDAP server
name lookup is done which results in a deadlock.

This problem also occurs for any nss lookup if the ip address of
ldapserver1 is not listed in /etc/hosts (even if the ip address is
referenced in libnss-ldap.conf).

-- System Information:
Debian Release: 3.1
  APT prefers testing
  APT policy: (51, 'testing'), (50, 'unstable')
Architecture: i386 (i686)
Kernel: Linux 2.6.11.12
Locale: LANG=en_US, LC_CTYPE=en_GB (charmap=ISO-8859-1)

Versions of packages ldap-utils depends on: 
ii  libc6                       2.3.2.ds1-22 GNU C Library: Shared libraries an
ii  libldap-2.2-7               2.2.23-8     OpenLDAP libraries
ii  libsasl2                    2.1.19-1.5   Authentication abstraction library
ii  libssl0.9.7                 0.9.7e-3     SSL shared libraries

-- 
-- arthur de jong - arthur@west.nl - west consulting b.v. --



Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#340601; Package ldap-utils. Full text and rfc822 format available.

Acknowledgement sent to James Andrewartha <trs80@ucc.gu.uwa.edu.au>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #10 received at 340601@bugs.debian.org (full text, mbox):

From: James Andrewartha <trs80@ucc.gu.uwa.edu.au>
To: 340601@bugs.debian.org
Subject: nss-ldap and hosts resolution workaround
Date: Mon, 05 Nov 2007 19:58:44 +0900
Have you tried replacing libnss-ldap with libnss-ldapd (only available
in testing/unstable) ?

James Andrewartha




Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#340601; Package ldap-utils. Full text and rfc822 format available.

Acknowledgement sent to Steve Langasek <vorlon@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #15 received at 340601@bugs.debian.org (full text, mbox):

From: Steve Langasek <vorlon@debian.org>
To: 340601@bugs.debian.org, 340601-submitter@bugs.debian.org, James Andrewartha <trs80@ucc.gu.uwa.edu.au>
Subject: Re: nss-ldap and hosts resolution workaround
Date: Mon, 5 Nov 2007 12:01:58 -0500
> Have you tried replacing libnss-ldap with libnss-ldapd (only available
> in testing/unstable) ?

The bug submitter is the maintainer and author of nss-ldapd, so I suspect he
may have done so ;)

But you didn't send your message to the bug submitter.  Forwarding now.

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
vorlon@debian.org                                   http://www.debian.org/




Message sent on to Arthur de Jong <arthur@west.nl>:
Bug#340601. Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#340601; Package ldap-utils. Full text and rfc822 format available.

Acknowledgement sent to Arthur de Jong <adejong@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #23 received at 340601@bugs.debian.org (full text, mbox):

From: Arthur de Jong <adejong@debian.org>
To: Steve Langasek <vorlon@debian.org>, 340601@bugs.debian.org
Cc: James Andrewartha <trs80@ucc.gu.uwa.edu.au>
Subject: Re: Bug#340601: nss-ldap and hosts resolution workaround
Date: Tue, 06 Nov 2007 22:23:11 +0100
[Message part 1 (text/plain, inline)]
On Mon, 2007-11-05 at 12:01 -0500, Steve Langasek wrote:
> > Have you tried replacing libnss-ldap with libnss-ldapd (only
> > available in testing/unstable) ?
>
> The bug submitter is the maintainer and author of nss-ldapd, so I
> suspect he may have done so ;)
>
> But you didn't send your message to the bug submitter.  Forwarding
> now.

Thanks. I will plug nss-ldapd a little more now ;-)

Yes I'm using nss-ldapd now and it solves this particular problem pretty
well. It also solves some other problems because of a much simpler
architecture.

The ldapsearch command would now do something like:

  ldapsearch
  |- NSS host lookup for LDAP server
  |  \- send request to nslcd ->  nslcd
  |                               \- does LDAP lookup for hostname
  \- does LDAP search  

So only one instance of OpenLDAP is active in each application which
simplifies things greatly. Due to the architecture change and some
refactoring I was also able to reduce the amount of code by 50%.

The downside is that nss-ldapd is not yet as stable as nss_ldap. A
memory leak has been reported (#447997) that seems to not have been
fully dealt with at this time and nss-ldapd has obviously not had as
much in-the-field testing as nss_ldap.

Back to the bugreport. I'm not really sure if bug #340601 is really a
bug in OpenLDAP. I think there is some locking done in OpenLDAP that is
not strictly necessary on glibc but this is based on an examination of
the source code I did a year ago so take it with a grain of salt.

-- 
-- arthur - adejong@debian.org - http://people.debian.org/~adejong --
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#340601; Package ldap-utils. Full text and rfc822 format available.

Acknowledgement sent to Steve Langasek <vorlon@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #28 received at 340601@bugs.debian.org (full text, mbox):

From: Steve Langasek <vorlon@debian.org>
To: Arthur de Jong <adejong@debian.org>, 340601@bugs.debian.org
Cc: James Andrewartha <trs80@ucc.gu.uwa.edu.au>
Subject: Re: [Pkg-openldap-devel] Bug#340601: nss-ldap and hosts resolution workaround
Date: Tue, 6 Nov 2007 23:35:11 -0500
On Tue, Nov 06, 2007 at 10:23:11PM +0100, Arthur de Jong wrote:
> Back to the bugreport. I'm not really sure if bug #340601 is really a
> bug in OpenLDAP. I think there is some locking done in OpenLDAP that is
> not strictly necessary on glibc but this is based on an examination of
> the source code I did a year ago so take it with a grain of salt.

It's plausible to me that it is an OpenLDAP bug, it might be caused by the
fact that we have two separate ldap libraries linked in in the case of
ldapsearch+nss_ldap and this might be causing symbol confusion.  If so, this
bug should shake out once openldap 2.4 is packaged.

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
vorlon@debian.org                                   http://www.debian.org/




Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#340601; Package ldap-utils. Full text and rfc822 format available.

Acknowledgement sent to Steve Langasek <vorlon@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #33 received at 340601@bugs.debian.org (full text, mbox):

From: Steve Langasek <vorlon@debian.org>
To: 340601@bugs.debian.org
Subject: Re: ldapsearch hangs when using ldap for /etc/hosts
Date: Sat, 9 Feb 2008 19:38:24 -0800
tags 340601 confirmed
thanks

I can confirm that this is still an issue even after transitioning to
libnss-ldap built against libldap-2.4-2.  This is easily reproducible by
configuring libnss-ldap to point at ldap://127.0.0.1 (with or without slapd
installed), setting 'hosts: files ldap dns' (or variant) in
/etc/nsswitch.conf, and running 'ldapsearch -H ldap://unknown-host/'.

Here is a gdb backtrace of a hung ldapsearch:

(gdb) bt
#0  0x00002b4065305174 in __lll_lock_wait () from /lib/libpthread.so.0
#1  0x00002b4065300b08 in _L_lock_104 () from /lib/libpthread.so.0
#2  0x00002b4065300470 in pthread_mutex_lock () from /lib/libpthread.so.0
#3  0x00002b40638e3ca7 in ldap_connect_to_host ()
   from /usr/lib/libldap-2.4.so.2
#4  0x00002b40638cf46a in ldap_int_open_connection ()
   from /usr/lib/libldap-2.4.so.2
#5  0x00002b40638e16d0 in ldap_new_connection () from /usr/lib/libldap-2.4.so.2
#6  0x00002b40638cf38d in ldap_open_defconn () from /usr/lib/libldap-2.4.so.2
#7  0x00002b40638e219f in ldap_send_initial_request ()
   from /usr/lib/libldap-2.4.so.2
#8  0x00002b40638d83e7 in ldap_sasl_bind () from /usr/lib/libldap-2.4.so.2
#9  0x00002b40638d88e9 in ldap_simple_bind () from /usr/lib/libldap-2.4.so.2
#10 0x00002b406572561b in ?? () from /lib/libnss_ldap.so.2
#11 0x00002b40657272d0 in ?? () from /lib/libnss_ldap.so.2
#12 0x00002b406572796e in ?? () from /lib/libnss_ldap.so.2
#13 0x00002b406572841b in ?? () from /lib/libnss_ldap.so.2
#14 0x00002b406572ac53 in _nss_ldap_gethostbyname2_r ()
   from /lib/libnss_ldap.so.2
#15 0x00002b4065059c3e in ?? () from /lib/libc.so.6
#16 0x00002b406505af19 in getaddrinfo () from /lib/libc.so.6
#17 0x00002b40638e3cbf in ldap_connect_to_host ()
   from /usr/lib/libldap-2.4.so.2
#18 0x00002b40638cf46a in ldap_int_open_connection ()
   from /usr/lib/libldap-2.4.so.2
#19 0x00002b40638e16d0 in ldap_new_connection () from /usr/lib/libldap-2.4.so.2
#20 0x00002b40638cf38d in ldap_open_defconn () from /usr/lib/libldap-2.4.so.2
#21 0x00002b40638e219f in ldap_send_initial_request ()
   from /usr/lib/libldap-2.4.so.2
#22 0x00002b40638d49cf in ldap_extended_operation ()
   from /usr/lib/libldap-2.4.so.2
#23 0x00002b40638d4b69 in ldap_extended_operation_s ()
   from /usr/lib/libldap-2.4.so.2
#24 0x00002b40638f1565 in ldap_start_tls_s () from /usr/lib/libldap-2.4.so.2
#25 0x0000000000407ddd in ?? ()
#26 0x000000000040559a in ?? ()
#27 0x00002b4064fb81c4 in __libc_start_main () from /lib/libc.so.6
#28 0x00000000004030f9 in ?? ()
#29 0x00007fff47408b18 in ?? ()
#30 0x0000000000000000 in ?? ()

So the issue is that ldap_start_tls_s() calls getaddrinfo(), which of course
recurses into libldap as needed and tries to start a connection, and boom.

From libraries/libldap/os-ip.c:ldap_connect_to_host():

#ifdef LDAP_R_COMPILE
	/* most getaddrinfo(3) use non-threadsafe resolver libraries */
	ldap_pvt_thread_mutex_lock(&ldap_int_resolv_mutex);
#endif

That looks to me like a good reason for a glibc-specific patch to the
code... :)

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek@ubuntu.com                                     vorlon@debian.org




Tags added: confirmed Request was from Steve Langasek <vorlon@debian.org> to control@bugs.debian.org. (Sun, 10 Feb 2008 03:39:02 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#340601; Package ldap-utils. Full text and rfc822 format available.

Acknowledgement sent to Steve Langasek <vorlon@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #40 received at 340601@bugs.debian.org (full text, mbox):

From: Steve Langasek <vorlon@debian.org>
To: 340601@bugs.debian.org
Subject: Re: ldapsearch hangs when using ldap for /etc/hosts
Date: Sat, 9 Feb 2008 20:59:46 -0800
[Message part 1 (text/plain, inline)]
tags 340601 patch
thanks

Here is a patch that fixes the deadlock for me.  Fellow maintainers, is this
ok to commit to the package, or would you prefer I clean it up first so that
it's suitable for upstream submission?

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek@ubuntu.com                                     vorlon@debian.org
[openldap2.3-340601.diff (text/x-diff, attachment)]

Tags added: patch Request was from Steve Langasek <vorlon@debian.org> to control@bugs.debian.org. (Sun, 10 Feb 2008 05:00:03 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#340601; Package ldap-utils. Full text and rfc822 format available.

Acknowledgement sent to Russ Allbery <rra@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #47 received at 340601@bugs.debian.org (full text, mbox):

From: Russ Allbery <rra@debian.org>
To: Steve Langasek <vorlon@debian.org>
Cc: 340601@bugs.debian.org
Subject: Re: [Pkg-openldap-devel] Bug#340601: ldapsearch hangs when using ldap for /etc/hosts
Date: Sat, 09 Feb 2008 21:12:35 -0800
Steve Langasek <vorlon@debian.org> writes:

> tags 340601 patch
> thanks
>
> Here is a patch that fixes the deadlock for me.  Fellow maintainers, is
> this ok to commit to the package, or would you prefer I clean it up
> first so that it's suitable for upstream submission?

I think it's fine to commit to the package for right now.  I'll go file an
upstream bug about the issue so that they're aware of it; my guess is that
they may have other ideas for how to fix it.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>




Noted your statement that Bug has been forwarded to http://www.openldap.org/its/index.cgi/?findid=5363. Request was from Russ Allbery <rra@debian.org> to control@bugs.debian.org. (Sun, 10 Feb 2008 05:21:01 GMT) Full text and rfc822 format available.

Tags added: upstream Request was from Russ Allbery <rra@debian.org> to control@bugs.debian.org. (Sun, 10 Feb 2008 05:21:02 GMT) Full text and rfc822 format available.

Tags added: pending Request was from vorlon@alioth.debian.org to control@bugs.debian.org. (Sun, 10 Feb 2008 06:09:04 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#340601; Package ldap-utils. Full text and rfc822 format available.

Acknowledgement sent to Steve Langasek <vorlon@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #58 received at 340601@bugs.debian.org (full text, mbox):

From: Steve Langasek <vorlon@debian.org>
To: Russ Allbery <rra@debian.org>
Cc: 340601@bugs.debian.org
Subject: Re: [Pkg-openldap-devel] Bug#340601: ldapsearch hangs when using ldap for /etc/hosts
Date: Sat, 9 Feb 2008 23:20:49 -0800
On Sat, Feb 09, 2008 at 09:12:35PM -0800, Russ Allbery wrote:

> > Here is a patch that fixes the deadlock for me.  Fellow maintainers, is
> > this ok to commit to the package, or would you prefer I clean it up
> > first so that it's suitable for upstream submission?

> I think it's fine to commit to the package for right now.  I'll go file an
> upstream bug about the issue so that they're aware of it; my guess is that
> they may have other ideas for how to fix it.

Ok, I've scaled back the patch a bit before committing it because a deeper
search leaves me uncertain that res_query and dn_expand are thread-safe even
in current versions of glibc.  Dropping the mutex for getaddrinfo() and
getnameinfo() is sufficient to fix this bug, in any case.

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek@ubuntu.com                                     vorlon@debian.org




Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#340601; Package ldap-utils. Full text and rfc822 format available.

Acknowledgement sent to Russ Allbery <rra@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #63 received at 340601@bugs.debian.org (full text, mbox):

From: Russ Allbery <rra@debian.org>
To: Steve Langasek <vorlon@debian.org>
Cc: 340601@bugs.debian.org
Subject: Re: [Pkg-openldap-devel] Bug#340601: ldapsearch hangs when using ldap for /etc/hosts
Date: Sun, 10 Feb 2008 10:06:14 -0800
Steve Langasek <vorlon@debian.org> writes:

> Ok, I've scaled back the patch a bit before committing it because a
> deeper search leaves me uncertain that res_query and dn_expand are
> thread-safe even in current versions of glibc.  Dropping the mutex for
> getaddrinfo() and getnameinfo() is sufficient to fix this bug, in any
> case.

I have now discussed this and the related fact that we're using libldap_r
for ldapsearch (which from upstream's perspective is the actual problem)
with upstream.  Upstream's stance on this is:

 * Using libldap_r for anything other than slapd is flatly unsupported and
   considered a bug.  We should not be doing that.  We should be treating
   libldap_r as a private library only for slapd.

 * libldap has no supported thread-safe API.  Threaded programs that link
   against libldap are required to handle locking themselves.

 * The root underlying problem would then be trying to use libnss-ldap and
   slapd together on the same system at the same time, because libnss-ldap
   pulls libldap into slapd's namespace.  Upstream's opinion is that
   libnss-ldap is broken and this regard and libnss-ldapd may be better.

 * People really shouldn't put hosts into LDAP; LDAP is a heavy-weight
   protocol that is not suited for use as a DNS resolver.

The last we can communicate back to the user and perhaps even put into the
documentation for libnss-ldap and libnss-ldapd.  For the rest, here is the
outline of an upstream-acceptable solution, which I'd love to be able to
get at.

 * Revert the change to link everything against libldap_r and ship only
   libldap in the libldap package (which will require nasty transition
   stuff, but putting that side for right now).  Adjust the shlibs in the
   package accordingly, of course.

 * Ship libldap_r in the slapd package or in a separate package referenced
   by slapd and clearly document in the README.Debian for that package
   that those libraries are intended for use only with slapd and any other
   use is not supported.

 * Make slapd conflict with libnss-ldap so that people can't run both of
   them on the same system.  Unfortunately, since libnss-ldapd provides
   libnss-ldap, this is trickier than I'd like it to be.

I'm guessing that this is going to break other things, but I don't know
what.  Comments?

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>




Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#340601; Package ldap-utils. Full text and rfc822 format available.

Acknowledgement sent to Russ Allbery <rra@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #68 received at 340601@bugs.debian.org (full text, mbox):

From: Russ Allbery <rra@debian.org>
To: 340601@bugs.debian.org
Subject: Fwd: (ITS#5363) ldapsearch hangs when using ldap for host in nsswitch
Date: Sun, 10 Feb 2008 10:53:23 -0800
[Message part 1 (text/plain, inline)]
Additional discussion with upstream.  I did a quick Google search and
couldn't uncover the specific details that Howard refers to here.  Given
that their packages are a commercial product of Symas, it feels rude to
ask him specific details.

The obvious interpretation would be to statically link libnss-ldap against
the appropriate LDAP libraries, which of course could be done and would
avoid a lot of the problem, but has other implications.  The fancier
possibility of using a separate daemon and IPC is basically what
libnss-ldapd does, as I understand it.

I'd really like to get back to a place where we don't have these sorts of
differences from upstream, since they cause a lot of problems when trying
to diagnose bugs and incorporate upstream fixes.

[Message part 2 (message/rfc822, inline)]
From: Howard Chu <hyc@symas.com>
To: rra@stanford.edu
Cc: openldap-its@openldap.org
Subject: Re: (ITS#5363) ldapsearch hangs when using ldap for host in nsswitch
Date: Sun, 10 Feb 2008 10:22:32 -0800
rra@stanford.edu wrote:
> We'll talk this over and see if there's a way that we can either move from
> nss-ldap or try to get it out of the places where it causes problems,
> although that's going to be really rough going as you can probably
> imagine.  libnss-ldap has over 2,800 installations that report popcon
> statistics, which means a *lot* of people are using it.
>
> Thinking out-loud here a bit (because I really want to solve this problem
> in the long run in a way that you can support), libnss-ldap and slapd on
> the same system are instant death unless everything uses libldap_r.  If we
> have slapd conflict with libnss-ldap and then have only slapd use
> libldap_r (and make people use libnss-ldapd on systems that are also
> running slapd), we may solve a fair chunk of the problem.  The other place
> where libldap is pulled in is via PAM, but slapd itself should never need
> to call into PAM (even indirectly).

You might consider suggesting to PADL that they contribute pam/nss_ldap to the 
OpenLDAP Project, if you want us to take an active role in solving this. Until 
then, the discussion just doesn't belong here on the OpenLDAP ITS. I'll note 
that I've described the solution Symas uses in our commercial distribution of 
pam/nss-ldap on the pam/nss-ldap mailing lists in the past. Our package has 
none of these issues.

Hint: since dynamic linker dependencies are causing the conflict, the solution 
is not to rely on the dynamic linker. In this particular case, using the 
dynamic linker introduces a security vulnerability anyway.

> If any slapd backends load dynamic libraries that in turn load libldap,
> the world may also explode, but this should only affect back-perl, which
> isn't working right now anyway due to the libtool RTLD_GLOBAL issues.

>> The other obvious step in this situation is never to use LDAP for
>> hostname resolution. ("Doctor, it hurts when I do this." - "Don't do
>> that.") LDAP is a much heavier weight protocol than DNS; as such it's
>> simply the wrong tool for the job. (Note - I like using DNS servers
>> backed by LDAP, but that's a completely different situation.)
>
> Okay.  I can at least pass that along to the user.

Or patch nss_ldap to simply return a failure whenever it's invoked for 
hostname resolution.
-- 
   -- Howard Chu
   Chief Architect, Symas Corp.  http://www.symas.com
   Director, Highland Sun        http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP     http://www.openldap.org/project/

Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#340601; Package ldap-utils. Full text and rfc822 format available.

Acknowledgement sent to Steve Langasek <vorlon@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #73 received at 340601@bugs.debian.org (full text, mbox):

From: Steve Langasek <vorlon@debian.org>
To: Russ Allbery <rra@debian.org>, 340601@bugs.debian.org
Subject: Re: [Pkg-openldap-devel] Bug#340601: Bug#340601: ldapsearch hangs when using ldap for /etc/hosts
Date: Sun, 10 Feb 2008 20:16:50 -0800
On Sun, Feb 10, 2008 at 10:06:14AM -0800, Russ Allbery wrote:
> > Ok, I've scaled back the patch a bit before committing it because a
> > deeper search leaves me uncertain that res_query and dn_expand are
> > thread-safe even in current versions of glibc.  Dropping the mutex for
> > getaddrinfo() and getnameinfo() is sufficient to fix this bug, in any
> > case.

> I have now discussed this and the related fact that we're using libldap_r
> for ldapsearch (which from upstream's perspective is the actual problem)
> with upstream.  Upstream's stance on this is:

>  * Using libldap_r for anything other than slapd is flatly unsupported and
>    considered a bug.  We should not be doing that.  We should be treating
>    libldap_r as a private library only for slapd.

>  * libldap has no supported thread-safe API.  Threaded programs that link
>    against libldap are required to handle locking themselves.

Wow, ok, I'm very not happy with this answer.  I think that *all* libraries
should have thread-safe APIs; and aside from this minor blemish, libldap_r
appears to already be a perfectly good thread-safe, reentrant library,
supported by upstream or not.

We have the following callers using libldap in Debian which also have need
of thread-safety and currently take no additional steps to ensure libldap is
thread-safe:

- libpq5
- libnss-ldapd
- evolution-data-server
- libexchange-storage1.2-3
- libcurl3
- kdepimlibs5
- claws-mail (for bonus points, this app launches each LDAP query in a
  separate thread... :)
- strongswan
- libnss-ldap
- libpam-ldap
- python-ldap
- openoffice.org-core
- libsasl2-modules-ldap
- libpam-smbpass
- libsmbclient
- freeradius-ldap (uses a mutex to ensure any given LDAP connection is only
  used by one thread at a time, but otherwise assumes that multiple LDAP
  connections can be going at once)

... and probably others besides, the list of reverse-depends is long and I
haven't looked at them all.

Moving the locking up the stack when we already have a perfectly thread-safe
libldap available would be a significant regression.

>  * The root underlying problem would then be trying to use libnss-ldap and
>    slapd together on the same system at the same time, because libnss-ldap
>    pulls libldap into slapd's namespace.  Upstream's opinion is that
>    libnss-ldap is broken and this regard and libnss-ldapd may be better.

This particular bug is not related to using libnss-ldap and slapd together
on the same system; it only requires one to try using an LDAP *client* on a
system that uses nss_ldap for hosts resolution.

The concerns about symbol collisions between libldap and libldap_r are
specific to slapd only if we concede that libldap_r should be treated as a
private library not intended for public consumption, which I am not because
the non-threaded libldap is not an adequate replacement for a number of
applications without duplicating a lot of work that's already been done
centrally.

>  * People really shouldn't put hosts into LDAP; LDAP is a heavy-weight
>    protocol that is not suited for use as a DNS resolver.

I happen to agree with that, but nss_ldap nevertheless implements all of the
NSS services, and supporting the hosts implementation for glibc-based
systems is a trivial change.  Eliminating a lock that we *know* we don't
need is hardly a bad thing, anyway.

> The last we can communicate back to the user and perhaps even put into the
> documentation for libnss-ldap and libnss-ldapd.  For the rest, here is the
> outline of an upstream-acceptable solution, which I'd love to be able to
> get at.

>  * Revert the change to link everything against libldap_r and ship only
>    libldap in the libldap package (which will require nasty transition
>    stuff, but putting that side for right now).  Adjust the shlibs in the
>    package accordingly, of course.

Frankly, I would rather continue to ship libldap_r publically, even without
upstream's blessing for this configuration, than inflict threading problems
on the reverse-dependencies.  We *know* that libldap_r is thread-safe
because slapd itself exercises it in the "supported" configuration, and we
already have to deal with frequently-changing libldap sonames primarily
because of the ABI changes that are specific to the internal ldap_pvt_*
functions - we ought to get some benefit for the migration troubles that's
going to cause.

>  * Ship libldap_r in the slapd package or in a separate package referenced
>    by slapd and clearly document in the README.Debian for that package
>    that those libraries are intended for use only with slapd and any other
>    use is not supported.

If that's really upstream's position, then they've been doing this wrong
from the start.  If libldap_r isn't supposed to be public, then they
should have been building it as a convenience library and linking it
statically into slapd; they should have kept all of the ldap_pvt_*/ldap_int_*
functions *private*, instead of exporting them in libldap; and they should
be maintaining a stable ABI for what is a pretty major system library
instead of forcing rebuilds for every slapd internal ABI change.

I'm dismayed.  All of the problems with using libldap_r as the system
libldap would be fixable over time, but it becomes considerably less
feasible without upstream's support of the effort.

Was your discussion with upstream somewhere public?

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek@ubuntu.com                                     vorlon@debian.org




Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#340601; Package ldap-utils. Full text and rfc822 format available.

Acknowledgement sent to Quanah Gibson-Mount <quanah@zimbra.com>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #78 received at 340601@bugs.debian.org (full text, mbox):

From: Quanah Gibson-Mount <quanah@zimbra.com>
To: Steve Langasek <vorlon@debian.org>, 340601@bugs.debian.org, Russ Allbery <rra@debian.org>
Subject: Re: [Pkg-openldap-devel] Bug#340601: Bug#340601: Bug#340601: ldapsearch hangs when using ldap for /etc/hosts
Date: Sun, 10 Feb 2008 20:44:13 -0800
--On Sunday, February 10, 2008 8:16 PM -0800 Steve Langasek 
<vorlon@debian.org> wrote:

> Was your discussion with upstream somewhere public?

ITS#5363

--Quanah

--

Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration




Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#340601; Package ldap-utils. Full text and rfc822 format available.

Acknowledgement sent to Russ Allbery <rra@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #83 received at 340601@bugs.debian.org (full text, mbox):

From: Russ Allbery <rra@debian.org>
To: 340601@bugs.debian.org
Subject: Re: Bug#340601: ldapsearch hangs when using ldap for /etc/hosts
Date: Mon, 11 Feb 2008 09:05:41 -0800
Steve Langasek <vorlon@debian.org> writes:

> Wow, ok, I'm very not happy with this answer.  I think that *all*
> libraries should have thread-safe APIs; and aside from this minor
> blemish, libldap_r appears to already be a perfectly good thread-safe,
> reentrant library, supported by upstream or not.
>
> We have the following callers using libldap in Debian which also have
> need of thread-safety and currently take no additional steps to ensure
> libldap is thread-safe:

Quick question here: Do you (or anyone else on the list) know what Red Hat
does with all of these software packages, many of which they also ship?
Are they also ignoring upstream and linking to libldap_r, or are they
doing something else?

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>




Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#340601; Package ldap-utils. Full text and rfc822 format available.

Acknowledgement sent to Howard Chu <hyc@highlandsun.com>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #88 received at 340601@bugs.debian.org (full text, mbox):

From: Howard Chu <hyc@highlandsun.com>
To: 340601@bugs.debian.org
Subject: libldap issues
Date: Mon, 11 Feb 2008 23:53:17 -0800
>>  * libldap has no supported thread-safe API.  Threaded programs that link
>>    against libldap are required to handle locking themselves.
>
> Wow, ok, I'm very not happy with this answer.

Russ left out a pertinent comment. I said:
>>> Until somebody decides to invest some energy into writing a new C API spec,
>>> (simply dusting off the expired C API draft won't cut it) there is not and
>>> will never be any official support for threading/re-entrancy in the LDAP API.

> I think that *all* libraries should have thread-safe APIs;

We all agree on that. Thus this page
http://scratchpad.wikia.com/wiki/LDAP_C_API

but it's not going to happen until more people get involved.

> and aside from this minor blemish, libldap_r
> appears to already be a perfectly good thread-safe, reentrant library,
> supported by upstream or not.

Ideally libldap would be re-entrant and free-threaded. But there are so many 
other problems with the API already, we're not interested in putting any more 
bandaids on it. We need a ground-up effort at defining a new API. We may not 
have to throw out 100% of what we have today, but there's no point in 
continuing to patch it. It's junk, and needs to be rewritten.

> We have the following callers using libldap in Debian which also have need
> of thread-safety and currently take no additional steps to ensure libldap is
> thread-safe:

> - libnss-ldap

Not true, nss_ldap already locks itself to prevent re-entrance. As such, it 
*must* only be linked with libldap, not libldap_r. (Remember, nss_ldap was 
written to work with other vendors' libraries too, and none of them are 
re-entrant, so it has to do the locking itself.)
-- 
  -- Howard Chu
  Chief Architect, Symas Corp.  http://www.symas.com
  Director, Highland Sun        http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP     http://www.openldap.org/project/




Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#340601; Package ldap-utils. Full text and rfc822 format available.

Acknowledgement sent to Russ Allbery <rra@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #93 received at 340601@bugs.debian.org (full text, mbox):

From: Russ Allbery <rra@debian.org>
To: 340601@bugs.debian.org
Subject: Re: Bug#340601: ldapsearch hangs when using ldap for /etc/hosts
Date: Tue, 12 Feb 2008 00:03:42 -0800
Russ Allbery <rra@debian.org> writes:

> Quick question here: Do you (or anyone else on the list) know what Red
> Hat does with all of these software packages, many of which they also
> ship?  Are they also ignoring upstream and linking to libldap_r, or are
> they doing something else?

Red Hat statically links nss-ldap.  That explains a lot.  I bet they're
just living with all the other (less common) library conflicts.  I wonder
if they statically link pam-ldap as well, since that's the other major
source.

(Thanks to Quanah for the investigation.)

So, the other thought that occurred to me: what if libldap and libldap_r
both had symbol versioning with different versions?  In that case, they
should be able to share process namespace with each other, which would
avoid the problem of libldap getting sucked into slapd's namespace via,
say, back-perl and would also let the problems of any package that chooses
to link directly to libldap_r be only the problems of that package and not
more general library conflict problems.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>




Reply sent to Steve Langasek <vorlon@debian.org>:
You have taken responsibility. Full text and rfc822 format available.

Notification sent to Arthur de Jong <arthur@west.nl>:
Bug acknowledged by developer. Full text and rfc822 format available.

Message #98 received at 340601-close@bugs.debian.org (full text, mbox):

From: Steve Langasek <vorlon@debian.org>
To: 340601-close@bugs.debian.org
Subject: Bug#340601: fixed in openldap2.3 2.4.7-6
Date: Fri, 29 Feb 2008 07:02:04 +0000
Source: openldap2.3
Source-Version: 2.4.7-6

We believe that the bug you reported is fixed in the latest version of
openldap2.3, which is due to be installed in the Debian FTP archive:

ldap-utils_2.4.7-6_amd64.deb
  to pool/main/o/openldap2.3/ldap-utils_2.4.7-6_amd64.deb
libldap-2.4-2-dbg_2.4.7-6_amd64.deb
  to pool/main/o/openldap2.3/libldap-2.4-2-dbg_2.4.7-6_amd64.deb
libldap-2.4-2_2.4.7-6_amd64.deb
  to pool/main/o/openldap2.3/libldap-2.4-2_2.4.7-6_amd64.deb
libldap2-dev_2.4.7-6_amd64.deb
  to pool/main/o/openldap2.3/libldap2-dev_2.4.7-6_amd64.deb
openldap2.3_2.4.7-6.diff.gz
  to pool/main/o/openldap2.3/openldap2.3_2.4.7-6.diff.gz
openldap2.3_2.4.7-6.dsc
  to pool/main/o/openldap2.3/openldap2.3_2.4.7-6.dsc
slapd-dbg_2.4.7-6_amd64.deb
  to pool/main/o/openldap2.3/slapd-dbg_2.4.7-6_amd64.deb
slapd_2.4.7-6_amd64.deb
  to pool/main/o/openldap2.3/slapd_2.4.7-6_amd64.deb



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 340601@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Steve Langasek <vorlon@debian.org> (supplier of updated openldap2.3 package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.7
Date: Thu, 28 Feb 2008 22:15:17 -0800
Source: openldap2.3
Binary: slapd ldap-utils libldap-2.4-2 libldap-2.4-2-dbg libldap2-dev slapd-dbg
Architecture: source amd64
Version: 2.4.7-6
Distribution: unstable
Urgency: low
Maintainer: Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>
Changed-By: Steve Langasek <vorlon@debian.org>
Description: 
 ldap-utils - OpenLDAP utilities
 libldap-2.4-2 - OpenLDAP libraries
 libldap-2.4-2-dbg - Debugging information for OpenLDAP libraries
 libldap2-dev - OpenLDAP development libraries
 slapd      - OpenLDAP server (slapd)
 slapd-dbg  - Debugging information for the OpenLDAP server (slapd)
Closes: 320073 340601 452438 452950 463460 465588 465784
Changes: 
 openldap2.3 (2.4.7-6) unstable; urgency=low
 .
   [ Updated debconf translations ]
   * Dutch, thanks to Bart Cornelis <cobaco@skolelinux.no>.  Closes: #452950.
   * Brazilian Portuguese, thanks to Eder L. Marques <frolic@debian-ce.org>.
     Closes: #463460.
   * German, thanks to Helge Kreutzmann <debian@helgefjell.de>.
     Closes: #465784.
 .
   [ Steve Langasek ]
   * Relax build-dependency on libsasl2-dev now that the versioned dependency
     is satisfied by all extant versions (including in oldstable), fixing a
     lintian warning about versioned build-deps on Debian revisions.
   * Avoid using a mutex around getaddrinfo() and getnameinfo() calls, which
     are guaranteed by glibc to be threadsafe; this fixes a deadlock when
     using nss_ldap for host lookups.  Closes: #340601.
   * debian/libldap2-dev.manpages: install all of man3/* instead of
     enumerating specific manpages to install.  Closes: #320073.
   * Add new patch, sasl-cleartext-strncasecmp, to correct a regression that
     prevented the use of the {CLEARTEXT} password scheme with SASL.
     Closes LP: #191563.
   * drop LGPL from debian/copyright; there is no longer any code under this
     license in the package.
   * Drop patch gnutls-altname-nulterminated; it's been determined that the
     "length" discrepancy was a bug in gnutls, and fixed in that package.
   * debian/configure.options: explicitly pass --with-odbc=unixodbc, so
     that we depend on the right ODBC implementation when both happen to
     be installed at build time.
 .
   [ Russ Allbery ]
   * Add a stamp file for the configure rule to avoid rerunning configure
     needlessly.  Closes: #465588.
   * Don't create the openldap user if slapd has been configured to run as
     a different user.  If slapd has been configured to run as openldap, do
     create the user on reconfigure.  Closes: #452438.
   * Reformat, reorganize, and update slapd's README.Debian.
     - Include SASL configuration information.
     - Remove LDBM information, since upstream no longer even ships LDBM
       and the debconf prompting and maintainer scripts already take care
       of any lingering databases.
     - Document the differences between the Debian OpenLDAP packages and
       upstream.
Files: 
 3fd5c06009c26c6fd096119067af1bb5 1397 net optional openldap2.3_2.4.7-6.dsc
 c90efb0e3e9d3ea9527f9e5a779bca7d 141637 net optional openldap2.3_2.4.7-6.diff.gz
 1e948da4592a7fdcda1b9295633f674d 1423100 net optional slapd_2.4.7-6_amd64.deb
 96a86c8570752c87fc7d91618e7a7c1a 260926 net optional ldap-utils_2.4.7-6_amd64.deb
 eac1ca03d4a22f51aa4096ea3c7b9699 199320 libs optional libldap-2.4-2_2.4.7-6_amd64.deb
 8b96e6f11ff478e4b020f9dfe3e6ddcf 289354 libdevel extra libldap-2.4-2-dbg_2.4.7-6_amd64.deb
 035136210cc17a73c423c0b9aff0c593 863266 libdevel extra libldap2-dev_2.4.7-6_amd64.deb
 a19fa3f23c2555014b464c4019a0b543 3531468 net extra slapd-dbg_2.4.7-6_amd64.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFHx6b+KN6ufymYLloRArSrAJ9seWbbp4qs8uw03rATuX2VTvJ41wCfTMQB
EagHzMDIXwQYwK2pCMsHBJc=
=drA5
-----END PGP SIGNATURE-----





Bug archived. Request was from Debbugs Internal Request <owner@bugs.debian.org> to internal_control@bugs.debian.org. (Sat, 29 Mar 2008 07:26:09 GMT) Full text and rfc822 format available.

Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Thu Apr 17 03:59:15 2014; Machine Name: buxtehude.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.