Debian Bug report logs - #421946
openldap2.3: build against newer Berkeley DB

version graph

Package: slapd; Maintainer for slapd is Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>; Source for slapd is src:openldap.

Reported by: Clint Adams <schizo@debian.org>

Date: Wed, 2 May 2007 16:00:02 UTC

Severity: serious

Tags: sid, squeeze

Found in version openldap/2.4.11-1

Fixed in version openldap/2.4.15-1

Done: Steve Langasek <vorlon@debian.org>

Bug is archived. No further changes may be made.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#421946; Package openldap2.3. Full text and rfc822 format available.

Acknowledgement sent to Clint Adams <schizo@debian.org>:
New Bug report received and forwarded. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: Clint Adams <schizo@debian.org>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: openldap2.3: build against newer Berkeley DB
Date: Wed, 2 May 2007 11:53:15 -0400
Package: openldap2.3
Version: 2.3.30-5
User: pkg-db-devel@lists.alioth.debian.org
Usertags: db4.2 oldbdb

Please update openldap2.3 to build against db4.5 so that we can get rid of
db4.2.



Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#421946; Package openldap2.3. Full text and rfc822 format available.

Acknowledgement sent to Russ Allbery <rra@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #10 received at 421946@bugs.debian.org (full text, mbox):

From: Russ Allbery <rra@debian.org>
To: Clint Adams <schizo@debian.org>
Cc: 421946@bugs.debian.org
Subject: Re: openldap2.3: build against newer Berkeley DB
Date: Wed, 30 May 2007 20:29:21 -0700
openldap2.3 is still building against db4.2 at upstream's specific
recommendation.  According to upstream, db4.5 (and db4.4) have regressions
from db4.2 in supporting OpenLDAP.

They have hopes for db4.6, once it's finally released.

This doesn't necessarily mean that we shouldn't switch to db4.5 anyway,
depending on the severity of the regressions, but I wanted to note in the
bug log that there was a reason why this hadn't been done other than
simple lack of time.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>



Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#421946; Package openldap2.3. Full text and rfc822 format available.

Acknowledgement sent to Roberto C. Sánchez <roberto@connexer.com>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #15 received at 421946@bugs.debian.org (full text, mbox):

From: Roberto C. Sánchez <roberto@connexer.com>
To: pkg-openldap-devel@lists.alioth.debian.org, 421946@bugs.debian.org
Subject: Re: [Pkg-openldap-devel] Bug#421946: openldap2.3: build against newer Berkeley DB
Date: Thu, 31 May 2007 10:12:48 -0400
[Message part 1 (text/plain, inline)]
On Wed, May 30, 2007 at 08:29:21PM -0700, Russ Allbery wrote:
> openldap2.3 is still building against db4.2 at upstream's specific
> recommendation.  According to upstream, db4.5 (and db4.4) have regressions
> from db4.2 in supporting OpenLDAP.
> 
> They have hopes for db4.6, once it's finally released.
> 
> This doesn't necessarily mean that we shouldn't switch to db4.5 anyway,
> depending on the severity of the regressions, but I wanted to note in the
> bug log that there was a reason why this hadn't been done other than
> simple lack of time.
> 
This is something that we on the cyrus-sasl2 team are also dealing with.
I think that what has held us back so far from going out on a limb is
the integral nature of cyrus-sasl2 for most systems.  I would argue that
openldap is in a similar situation, since it is often used as the source
of authentication information.  Any breakage there has the potential to
be catastrophic.  Of course, I am doubtful that db4.2 will be supported
for an another release cycle.  It is rather old at this point, after
all.

Regards,

-Roberto

-- 
Roberto C. Sánchez
http://people.connexer.com/~roberto
http://www.connexer.com
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#421946; Package openldap2.3. Full text and rfc822 format available.

Acknowledgement sent to Steve Langasek <vorlon@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #20 received at 421946@bugs.debian.org (full text, mbox):

From: Steve Langasek <vorlon@debian.org>
To: pkg-openldap-devel@lists.alioth.debian.org
Cc: 421946@bugs.debian.org
Subject: Re: [Pkg-openldap-devel] Bug#421946: openldap2.3: build against newer Berkeley DB
Date: Thu, 31 May 2007 17:08:01 -0700
On Thu, May 31, 2007 at 10:12:48AM -0400, Roberto C. Sánchez wrote:
> On Wed, May 30, 2007 at 08:29:21PM -0700, Russ Allbery wrote:
> > openldap2.3 is still building against db4.2 at upstream's specific
> > recommendation.  According to upstream, db4.5 (and db4.4) have regressions
> > from db4.2 in supporting OpenLDAP.

> > They have hopes for db4.6, once it's finally released.

> > This doesn't necessarily mean that we shouldn't switch to db4.5 anyway,
> > depending on the severity of the regressions, but I wanted to note in the
> > bug log that there was a reason why this hadn't been done other than
> > simple lack of time.

> This is something that we on the cyrus-sasl2 team are also dealing with.
> I think that what has held us back so far from going out on a limb is
> the integral nature of cyrus-sasl2 for most systems.

FWIW, while I would say cyrus-sasl2 is fairly integral, the bdb support
/within/ cyrus-sasl2 is not as integral.  I don't use any of the sasl
methods on my systems that look up to a local db.

Anyway, if there are problems, better that they be detected at the beginning
of the release cycle when it's easier to get any upstream bdb problems fixed
in time for lenny.

Cheers,
-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
vorlon@debian.org                                   http://www.debian.org/



Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#421946; Package openldap2.3. Full text and rfc822 format available.

Acknowledgement sent to Quanah Gibson-Mount <quanah@zimbra.com>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #25 received at 421946@bugs.debian.org (full text, mbox):

From: Quanah Gibson-Mount <quanah@zimbra.com>
To: pkg-openldap-devel@lists.alioth.debian.org
Cc: 421946@bugs.debian.org
Subject: Re: [Pkg-openldap-devel] Bug#421946: openldap2.3: build against newer Berkeley DB
Date: Thu, 31 May 2007 17:14:10 -0700
--On Thursday, May 31, 2007 5:08 PM -0700 Steve Langasek 
<vorlon@debian.org> wrote:

> On Thu, May 31, 2007 at 10:12:48AM -0400, Roberto C. Sánchez wrote:
>> On Wed, May 30, 2007 at 08:29:21PM -0700, Russ Allbery wrote:
>> > openldap2.3 is still building against db4.2 at upstream's specific
>> > recommendation.  According to upstream, db4.5 (and db4.4) have
>> > regressions from db4.2 in supporting OpenLDAP.
>
>> > They have hopes for db4.6, once it's finally released.
>
>> > This doesn't necessarily mean that we shouldn't switch to db4.5 anyway,
>> > depending on the severity of the regressions, but I wanted to note in
>> > the bug log that there was a reason why this hadn't been done other
>> > than simple lack of time.
>
>> This is something that we on the cyrus-sasl2 team are also dealing with.
>> I think that what has held us back so far from going out on a limb is
>> the integral nature of cyrus-sasl2 for most systems.
>
> FWIW, while I would say cyrus-sasl2 is fairly integral, the bdb support
> /within/ cyrus-sasl2 is not as integral.  I don't use any of the sasl
> methods on my systems that look up to a local db.
>
> Anyway, if there are problems, better that they be detected at the
> beginning of the release cycle when it's easier to get any upstream bdb
> problems fixed in time for lenny.

My hope is 4.6 will be out long before lenny.  It is the first release 
since BDB 4.2 that I've seen actually outperform 4.2 across the board.  4.4 
& 4.5 were both noticeably slower in read & write tests.

--Quanah

--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration



Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#421946; Package openldap2.3. Full text and rfc822 format available.

Acknowledgement sent to Steve Langasek <vorlon@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #30 received at 421946@bugs.debian.org (full text, mbox):

From: Steve Langasek <vorlon@debian.org>
To: Quanah Gibson-Mount <quanah@zimbra.com>, 421946@bugs.debian.org
Cc: pkg-openldap-devel@lists.alioth.debian.org
Subject: Re: [Pkg-openldap-devel] Bug#421946: Bug#421946: openldap2.3: build against newer Berkeley DB
Date: Thu, 31 May 2007 17:22:30 -0700
On Thu, May 31, 2007 at 05:14:10PM -0700, Quanah Gibson-Mount wrote:
> --On Thursday, May 31, 2007 5:08 PM -0700 Steve Langasek 
> <vorlon@debian.org> wrote:

> > Anyway, if there are problems, better that they be detected at the
> > beginning of the release cycle when it's easier to get any upstream bdb
> > problems fixed in time for lenny.

> My hope is 4.6 will be out long before lenny.  It is the first release 
> since BDB 4.2 that I've seen actually outperform 4.2 across the board.  4.4 
> & 4.5 were both noticeably slower in read & write tests.

Is performance the only problem with 4.4 and 4.5 for OpenLDAP?  If so, I
think the BDB maintainers would be well within their rights to ask for
removal of 4.2 from unstable anyway, even if 4.6 weren't on the horizon.

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
vorlon@debian.org                                   http://www.debian.org/



Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#421946; Package openldap2.3. Full text and rfc822 format available.

Acknowledgement sent to Quanah Gibson-Mount <quanah@zimbra.com>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #35 received at 421946@bugs.debian.org (full text, mbox):

From: Quanah Gibson-Mount <quanah@zimbra.com>
To: 421946@bugs.debian.org
Cc: pkg-openldap-devel@lists.alioth.debian.org
Subject: Re: [Pkg-openldap-devel] Bug#421946: Bug#421946: openldap2.3: build against newer Berkeley DB
Date: Thu, 31 May 2007 17:26:52 -0700
--On Thursday, May 31, 2007 5:22 PM -0700 Steve Langasek 
<vorlon@debian.org> wrote:

> On Thu, May 31, 2007 at 05:14:10PM -0700, Quanah Gibson-Mount wrote:
>> --On Thursday, May 31, 2007 5:08 PM -0700 Steve Langasek
>> <vorlon@debian.org> wrote:
>
>> > Anyway, if there are problems, better that they be detected at the
>> > beginning of the release cycle when it's easier to get any upstream bdb
>> > problems fixed in time for lenny.
>
>> My hope is 4.6 will be out long before lenny.  It is the first release
>> since BDB 4.2 that I've seen actually outperform 4.2 across the board.
>> 4.4  & 4.5 were both noticeably slower in read & write tests.
>
> Is performance the only problem with 4.4 and 4.5 for OpenLDAP?  If so, I
> think the BDB maintainers would be well within their rights to ask for
> removal of 4.2 from unstable anyway, even if 4.6 weren't on the horizon.

Well, if high performance is a critical factor to an LDAP service, this is 
a severe issue.  Yes, that is the only problem I've encountered with 4.4 
and 4.5.  Is it enough of an issue I would avoid it? Yes, it is.  What 
Debian and its BDB maintainers choose to do about it depends, I guess, on 
whether or not they understand why that would be an issue for users or if 
it matters so much to them.

--Quanah

--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration



Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#421946; Package openldap2.3. Full text and rfc822 format available.

Acknowledgement sent to Russ Allbery <rra@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #40 received at 421946@bugs.debian.org (full text, mbox):

From: Russ Allbery <rra@debian.org>
To: 421946@bugs.debian.org
Subject: Re: [Pkg-openldap-devel] Bug#421946: openldap2.3: build against newer Berkeley DB
Date: Thu, 31 May 2007 17:41:46 -0700
Steve Langasek <vorlon@debian.org> writes:

> Is performance the only problem with 4.4 and 4.5 for OpenLDAP?  If so, I
> think the BDB maintainers would be well within their rights to ask for
> removal of 4.2 from unstable anyway, even if 4.6 weren't on the horizon.

Note that going with a slower BDB probably isn't an acceptable solution
for Stanford, which means that if Debian goes this direction, I'll have to
keep maintaining a local fork of the packages, which means that I won't be
testing the Debian packages in a production environment.

Currently I have to do this anyway for other reasons, but I'm trying to
reduce the divergence as much as I can, and currently once 2.4 is released
there's no reason why I can't run the Debian packages directly.  But going
to a slower BDB is probably a non-option for us.

This is *not* necessarily a good reason for Debian to make decisions; it's
more feedback from someone who wants to use the Debian packages that this
could be a deal-breaker in our willingness to use the packages in Debian
verbatim.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>



Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#421946; Package openldap2.3. Full text and rfc822 format available.

Acknowledgement sent to Steve Langasek <vorlon@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #45 received at 421946@bugs.debian.org (full text, mbox):

From: Steve Langasek <vorlon@debian.org>
To: Quanah Gibson-Mount <quanah@zimbra.com>
Cc: 421946@bugs.debian.org, pkg-openldap-devel@lists.alioth.debian.org
Subject: Re: [Pkg-openldap-devel] Bug#421946: Bug#421946: openldap2.3: build against newer Berkeley DB
Date: Thu, 31 May 2007 18:41:19 -0700
On Thu, May 31, 2007 at 05:26:52PM -0700, Quanah Gibson-Mount wrote:
> > Is performance the only problem with 4.4 and 4.5 for OpenLDAP?  If so, I
> > think the BDB maintainers would be well within their rights to ask for
> > removal of 4.2 from unstable anyway, even if 4.6 weren't on the horizon.

> Well, if high performance is a critical factor to an LDAP service, this is 
> a severe issue.  Yes, that is the only problem I've encountered with 4.4 
> and 4.5.  Is it enough of an issue I would avoid it? Yes, it is.  What 
> Debian and its BDB maintainers choose to do about it depends, I guess, on 
> whether or not they understand why that would be an issue for users or if 
> it matters so much to them.

I don't doubt that not getting stuck maintaining 5 versions of libdb4 in
lenny (db4.2, db4.3, db4.4, db4.5, and the not-yet-released db4.6) is more
important to them than an application-specific performance issue with db4.5.

I'm also confident that they would entertain other reasonable solutions for
the performance issue that don't involve shipping a separate copy of libdb
in lenny for the benefit of a single reverse-dependency.

Anyway, even though we don't want to ship it in that state for lenny (and
db4.6 seems sure to solve that), is there any reason not to build openldap
against db4.5 right now for unstable, to get feedback in parallel about any
other issues there might be?

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
vorlon@debian.org                                   http://www.debian.org/



Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#421946; Package openldap2.3. Full text and rfc822 format available.

Acknowledgement sent to Russ Allbery <rra@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #50 received at 421946@bugs.debian.org (full text, mbox):

From: Russ Allbery <rra@debian.org>
To: 421946@bugs.debian.org
Subject: Re: [Pkg-openldap-devel] Bug#421946: Bug#421946: openldap2.3: build against newer Berkeley DB
Date: Thu, 31 May 2007 18:45:26 -0700
Steve Langasek <vorlon@debian.org> writes:

> I don't doubt that not getting stuck maintaining 5 versions of libdb4 in
> lenny (db4.2, db4.3, db4.4, db4.5, and the not-yet-released db4.6) is
> more important to them than an application-specific performance issue
> with db4.5.

It's not that bad.  db4.3 is badly broken and nothing should use it, and I
don't know of any advantages of db4.4 over db4.5.  It's really only two
versions.

I certainly do understand why they don't want to maintain even two
versions, though.

> Anyway, even though we don't want to ship it in that state for lenny
> (and db4.6 seems sure to solve that), is there any reason not to build
> openldap against db4.5 right now for unstable, to get feedback in
> parallel about any other issues there might be?

Well, I think it's the wrong technical choice for the openldap packages,
and it will reduce my ability to test and increase the amount of
divergence that I have to deal with, but none of those are particularly
strong objections.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>



Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#421946; Package openldap2.3. Full text and rfc822 format available.

Acknowledgement sent to Albert Chin <debian-openldap-devel@mlists.thewrittenword.com>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #55 received at 421946@bugs.debian.org (full text, mbox):

From: Albert Chin <debian-openldap-devel@mlists.thewrittenword.com>
To: pkg-openldap-devel@lists.alioth.debian.org
Cc: 421946@bugs.debian.org
Subject: Re: [Pkg-openldap-devel] Bug#421946: openldap2.3: build against newer Berkeley DB
Date: Tue, 24 Jul 2007 16:21:33 -0500
On Thu, May 31, 2007 at 05:14:10PM -0700, Quanah Gibson-Mount wrote:
> --On Thursday, May 31, 2007 5:08 PM -0700 Steve Langasek 
> <vorlon@debian.org> wrote:
> 
> > On Thu, May 31, 2007 at 10:12:48AM -0400, Roberto C. S??nchez wrote:
> >> On Wed, May 30, 2007 at 08:29:21PM -0700, Russ Allbery wrote:
> >> > openldap2.3 is still building against db4.2 at upstream's specific
> >> > recommendation.  According to upstream, db4.5 (and db4.4) have
> >> > regressions from db4.2 in supporting OpenLDAP.
> >
> >> > They have hopes for db4.6, once it's finally released.
> >
> >> > This doesn't necessarily mean that we shouldn't switch to db4.5 anyway,
> >> > depending on the severity of the regressions, but I wanted to note in
> >> > the bug log that there was a reason why this hadn't been done other
> >> > than simple lack of time.
> >
> >> This is something that we on the cyrus-sasl2 team are also dealing with.
> >> I think that what has held us back so far from going out on a limb is
> >> the integral nature of cyrus-sasl2 for most systems.
> >
> > FWIW, while I would say cyrus-sasl2 is fairly integral, the bdb support
> > /within/ cyrus-sasl2 is not as integral.  I don't use any of the sasl
> > methods on my systems that look up to a local db.
> >
> > Anyway, if there are problems, better that they be detected at the
> > beginning of the release cycle when it's easier to get any upstream bdb
> > problems fixed in time for lenny.
> 
> My hope is 4.6 will be out long before lenny.  It is the first release 
> since BDB 4.2 that I've seen actually outperform 4.2 across the board.  4.4 
> & 4.5 were both noticeably slower in read & write tests.

4.6.18 is now available. Would be nice to know if it seems like a
worthy replacement for 4.2.

-- 
albert chin (china@thewrittenword.com)



Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#421946; Package openldap2.3. Full text and rfc822 format available.

Acknowledgement sent to Quanah Gibson-Mount <quanah@zimbra.com>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #60 received at 421946@bugs.debian.org (full text, mbox):

From: Quanah Gibson-Mount <quanah@zimbra.com>
To: Albert Chin <debian-openldap-devel@mlists.thewrittenword.com>, pkg-openldap-devel@lists.alioth.debian.org
Cc: 421946@bugs.debian.org
Subject: Re: [Pkg-openldap-devel] Bug#421946: openldap2.3: build against newer Berkeley DB
Date: Tue, 24 Jul 2007 14:26:01 -0700

--On July 24, 2007 4:21:33 PM -0500 Albert Chin 
<debian-openldap-devel@mlists.thewrittenword.com> wrote:

> 4.6.18 is now available. Would be nice to know if it seems like a
> worthy replacement for 4.2.

Other than the API changing significantly from the BDB 4.6 pre-releases, 
there's at least one serious problem with them.  The openldap developers 
are corresponding with sleepycat to see what can be done about it.

--Quanah

--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration



Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#421946; Package openldap2.3. Full text and rfc822 format available.

Acknowledgement sent to Quanah Gibson-Mount <quanah@zimbra.com>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #65 received at 421946@bugs.debian.org (full text, mbox):

From: Quanah Gibson-Mount <quanah@zimbra.com>
To: Albert Chin <debian-openldap-devel@mlists.thewrittenword.com>, pkg-openldap-devel@lists.alioth.debian.org
Cc: 421946@bugs.debian.org
Subject: Re: [Pkg-openldap-devel] Bug#421946: openldap2.3: build against newer Berkeley DB
Date: Fri, 17 Aug 2007 07:40:51 -0700
--On Tuesday, July 24, 2007 2:26 PM -0700 Quanah Gibson-Mount 
<quanah@zimbra.com> wrote:

>
>
> --On July 24, 2007 4:21:33 PM -0500 Albert Chin
> <debian-openldap-devel@mlists.thewrittenword.com> wrote:
>
>> 4.6.18 is now available. Would be nice to know if it seems like a
>> worthy replacement for 4.2.
>
> Other than the API changing significantly from the BDB 4.6 pre-releases,
> there's at least one serious problem with them.  The openldap developers
> are corresponding with sleepycat to see what can be done about it.


On the downside testing of BDB 4.6.19 finds it to be 2-3 times *slower* 
than BDB 4.2 or BDB 4.5.  While BDB 4.6.3 remains 25% faster than BDB 4.2. 
Correspondance with sleepycat to figure out what happened continues...

--Quanah

--

Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration



Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#421946; Package openldap2.3. Full text and rfc822 format available.

Acknowledgement sent to Quanah Gibson-Mount <quanah@zimbra.com>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #70 received at 421946@bugs.debian.org (full text, mbox):

From: Quanah Gibson-Mount <quanah@zimbra.com>
To: 421946@bugs.debian.org
Cc: pkg-openldap-devel@lists.alioth.debian.org
Subject: Re: OpenLDAP 2.3.38 and Berkeley DB 4.6.19 (fwd)
Date: Tue, 28 Aug 2007 14:03:57 -0700
Here's the relevant information.

Looks like x86_64 requires a specific option.  Maybe it'd be best to 
compile it that way for all platforms.

--Quanah

------------ Forwarded Message ------------
Date: Tuesday, August 28, 2007 1:47 PM -0700
From: Howard Ch
To: Quanah Gibson-Mount
Cc: openldap-software@openldap.org
Subject: Re: OpenLDAP 2.3.38 and Berkeley DB 4.6.19

Quanah Gibson-Mount wrote:
> --On Tuesday, August 28, 2007 2:44 PM -0400 matthew sporleder
> <msporleder@gmail.com> wrote:
>
>> That's an unfortunate development from oracle considering the
>> presentation that was released showing bdb4.6 to be very fast.
>
> Yup.  I believe Howard is working with Sleepycat to find out what was
> done  to 4.6 to make it suddenly be 3-4 times slower instead of
> significantly  faster as it was in the early releases.

Yes, it's been a long conversation so far. While their new memory manager
(inspired by Jong-Hyuk Choi's research on BerkeleyDB performance in
OpenLDAP) is a great improvement, their new lock manager is not.

This may not be an issue on all platforms. On x86 though, they have a
hybrid mutex mechanism which is enabled by default. It uses both assembly
language spinlocks and pthread mutexes, first spinning in the assembly
language lock some number of times before falling back onto the mutex. They
claimed that this improved performance in their tests, because pthread
mutexes can be very expensive on some platforms. In my tests on x86_64
Linux however, it just forced CPU usage to 100% (200% actually, dualcore)
and slowed down overall throughput.

You can give an explicit "--with-mutex=POSIX/pthread" argument when
configuring BerkeleyDB to avoid the hybrid mutex scheme, in which case
performance of BDB 4.6.19 seems to match what we obtained with BDB 4.6.3.

If you were using BerkeleyDB 4.5.20 before, the early BDB 4.6 releases will
work well for you. The only difference between 4.5.20 and 4.6.1 was in the
memory manager. 4.6.2 just tweaked some portability issues to support BREW
(Qualcomm's cellphone programming environment). 4.6.3 added the ability to
specify different cache priorities per database operation. (I didn't track
what changed between 4.6.3 and 4.6.18.)

And for anyone curious - you can read ITS#3851
http://www.openldap.org/its/index.cgi/Development?id=3851 for the
background on the problems in BerkeleyDB's memory allocator. While there
was a fair amount of debate as to whether Jong's proposed solution was of
any benefit, it was pretty clear that the existing code in BerkeleyDB could
be improved. But the changes in the memory allocator may not visibly affect
you if you're running a small-to-medium sized database. It's only apparent
when the total volume of data is much larger than the BerkeleyDB cache,
because that's the condition that leads to memory fragmentation, and it's
the fragmentation that causes the allocator's performance to degrade.

The same kind of problem affects the slapd entry cache, when the number of
active entries is much larger than the entry cache, and you're using the
glibc malloc. (At least, for glibc 2.3 and older. I haven't retested with
glibc 2.5 yet.)

To sum up - if you're already using BDB 4.5.20 with OpenLDAP 2.3 and the
performance is acceptable, I wouldn't be in any hurry to upgrade to BDB
4.6. Releases 4.6.18 and 4.6.19 are incompatible with OpenLDAP 2.3. BDB
4.6.1 is a drop-in replacement for 4.5.20 though.
-- 
  -- Howard Chu
  Chief Architect, Symas Corp.  http://www.symas.com
  Director, Highland Sun        http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP     http://www.openldap.org/project/

---------- End Forwarded Message ----------



--

Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration



Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#421946; Package openldap2.3. Full text and rfc822 format available.

Acknowledgement sent to Clint Adams <schizo@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #75 received at 421946@bugs.debian.org (full text, mbox):

From: Clint Adams <schizo@debian.org>
To: Quanah Gibson-Mount <quanah@zimbra.com>, 421946@bugs.debian.org
Cc: pkg-openldap-devel@lists.alioth.debian.org, pkg-db-devel@lists.alioth.debian.org
Subject: Re: Bug#421946: OpenLDAP 2.3.38 and Berkeley DB 4.6.19 (fwd)
Date: Wed, 29 Aug 2007 00:02:30 -0400
On Tue, Aug 28, 2007 at 02:03:57PM -0700, Quanah Gibson-Mount wrote:
> Here's the relevant information.
>
> Looks like x86_64 requires a specific option.  Maybe it'd be best to 
> compile it that way for all platforms.

Are there reproducible tests/benchmarks that we can use to figure out
whether or not this actually makes sense on any given architecture?

> ------------ Forwarded Message ------------
> Date: Tuesday, August 28, 2007 1:47 PM -0700
> From: Howard Ch
> To: Quanah Gibson-Mount
> Cc: openldap-software@openldap.org
> Subject: Re: OpenLDAP 2.3.38 and Berkeley DB 4.6.19
>
> Quanah Gibson-Mount wrote:
>> --On Tuesday, August 28, 2007 2:44 PM -0400 matthew sporleder
>> <msporleder@gmail.com> wrote:
>>
>>> That's an unfortunate development from oracle considering the
>>> presentation that was released showing bdb4.6 to be very fast.
>>
>> Yup.  I believe Howard is working with Sleepycat to find out what was
>> done  to 4.6 to make it suddenly be 3-4 times slower instead of
>> significantly  faster as it was in the early releases.
>
> Yes, it's been a long conversation so far. While their new memory manager
> (inspired by Jong-Hyuk Choi's research on BerkeleyDB performance in
> OpenLDAP) is a great improvement, their new lock manager is not.
>
> This may not be an issue on all platforms. On x86 though, they have a
> hybrid mutex mechanism which is enabled by default. It uses both assembly
> language spinlocks and pthread mutexes, first spinning in the assembly
> language lock some number of times before falling back onto the mutex. They
> claimed that this improved performance in their tests, because pthread
> mutexes can be very expensive on some platforms. In my tests on x86_64
> Linux however, it just forced CPU usage to 100% (200% actually, dualcore)
> and slowed down overall throughput.
>
> You can give an explicit "--with-mutex=POSIX/pthread" argument when
> configuring BerkeleyDB to avoid the hybrid mutex scheme, in which case
> performance of BDB 4.6.19 seems to match what we obtained with BDB 4.6.3.
>
> If you were using BerkeleyDB 4.5.20 before, the early BDB 4.6 releases will
> work well for you. The only difference between 4.5.20 and 4.6.1 was in the
> memory manager. 4.6.2 just tweaked some portability issues to support BREW
> (Qualcomm's cellphone programming environment). 4.6.3 added the ability to
> specify different cache priorities per database operation. (I didn't track
> what changed between 4.6.3 and 4.6.18.)
>
> And for anyone curious - you can read ITS#3851
> http://www.openldap.org/its/index.cgi/Development?id=3851 for the
> background on the problems in BerkeleyDB's memory allocator. While there
> was a fair amount of debate as to whether Jong's proposed solution was of
> any benefit, it was pretty clear that the existing code in BerkeleyDB could
> be improved. But the changes in the memory allocator may not visibly affect
> you if you're running a small-to-medium sized database. It's only apparent
> when the total volume of data is much larger than the BerkeleyDB cache,
> because that's the condition that leads to memory fragmentation, and it's
> the fragmentation that causes the allocator's performance to degrade.
>
> The same kind of problem affects the slapd entry cache, when the number of
> active entries is much larger than the entry cache, and you're using the
> glibc malloc. (At least, for glibc 2.3 and older. I haven't retested with
> glibc 2.5 yet.)
>
> To sum up - if you're already using BDB 4.5.20 with OpenLDAP 2.3 and the
> performance is acceptable, I wouldn't be in any hurry to upgrade to BDB
> 4.6. Releases 4.6.18 and 4.6.19 are incompatible with OpenLDAP 2.3. BDB
> 4.6.1 is a drop-in replacement for 4.5.20 though.
> -- 
>   -- Howard Chu
>   Chief Architect, Symas Corp.  http://www.symas.com
>   Director, Highland Sun        http://highlandsun.com/hyc/
>   Chief Architect, OpenLDAP     http://www.openldap.org/project/
>
> ---------- End Forwarded Message ----------



Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#421946; Package openldap2.3. Full text and rfc822 format available.

Acknowledgement sent to Quanah Gibson-Mount <quanah@zimbra.com>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #80 received at 421946@bugs.debian.org (full text, mbox):

From: Quanah Gibson-Mount <quanah@zimbra.com>
To: Clint Adams <schizo@debian.org>, 421946@bugs.debian.org
Cc: pkg-openldap-devel@lists.alioth.debian.org, pkg-db-devel@lists.alioth.debian.org
Subject: Re: Bug#421946: OpenLDAP 2.3.38 and Berkeley DB 4.6.19 (fwd)
Date: Wed, 29 Aug 2007 11:15:58 -0700

--On August 29, 2007 12:02:30 AM -0400 Clint Adams <schizo@debian.org> 
wrote:

> On Tue, Aug 28, 2007 at 02:03:57PM -0700, Quanah Gibson-Mount wrote:
>> Here's the relevant information.
>>
>> Looks like x86_64 requires a specific option.  Maybe it'd be best to
>> compile it that way for all platforms.
>
> Are there reproducible tests/benchmarks that we can use to figure out
> whether or not this actually makes sense on any given architecture?

I've used slamd for this in the past.  It'd require being able to run 
against the various architectures.

--Quanah

--

Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration



Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#421946; Package openldap2.3. Full text and rfc822 format available.

Acknowledgement sent to Clint Adams <schizo@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #85 received at 421946@bugs.debian.org (full text, mbox):

From: Clint Adams <schizo@debian.org>
To: Quanah Gibson-Mount <quanah@zimbra.com>
Cc: 421946@bugs.debian.org, pkg-db-devel@lists.alioth.debian.org, pkg-openldap-devel@lists.alioth.debian.org
Subject: Re: [Pkg-db-devel] Bug#421946: OpenLDAP 2.3.38 and Berkeley DB 4.6.19 (fwd)
Date: Wed, 29 Aug 2007 20:41:50 -0400
On Wed, Aug 29, 2007 at 11:15:58AM -0700, Quanah Gibson-Mount wrote:
> I've used slamd for this in the past.  It'd require being able to run 
> against the various architectures.

I was hoping for something simpler that we could theoretically stuff
at the end of the BDB test suite.



Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#421946; Package openldap2.3. Full text and rfc822 format available.

Acknowledgement sent to Quanah Gibson-Mount <quanah@zimbra.com>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #90 received at 421946@bugs.debian.org (full text, mbox):

From: Quanah Gibson-Mount <quanah@zimbra.com>
To: Clint Adams <schizo@debian.org>
Cc: 421946@bugs.debian.org, pkg-db-devel@lists.alioth.debian.org, pkg-openldap-devel@lists.alioth.debian.org
Subject: Re: [Pkg-db-devel] Bug#421946: OpenLDAP 2.3.38 and Berkeley DB 4.6.19 (fwd)
Date: Wed, 29 Aug 2007 20:01:27 -0700
--On Wednesday, August 29, 2007 8:41 PM -0400 Clint Adams 
<schizo@debian.org> wrote:

> On Wed, Aug 29, 2007 at 11:15:58AM -0700, Quanah Gibson-Mount wrote:
>> I've used slamd for this in the past.  It'd require being able to run
>> against the various architectures.
>
> I was hoping for something simpler that we could theoretically stuff
> at the end of the BDB test suite.

Well, I basically test BDB via OpenLDAP, so that's my familiarity. ;)  I'm 
not sure exactly what the test case to develop purely using BDB.  Someone 
else may have some ideas.

--Quanah

--

Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration



Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#421946; Package openldap2.3. Full text and rfc822 format available.

Acknowledgement sent to Howard Chu <hyc@highlandsun.com>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #95 received at 421946@bugs.debian.org (full text, mbox):

From: Howard Chu <hyc@highlandsun.com>
To: 421946@bugs.debian.org
Subject: openldap2.3: build against newer Berkeley DB
Date: Mon, 03 Sep 2007 23:23:22 -0700
>> Looks like x86_64 requires a specific option.  Maybe it'd be best to 
>> compile it that way for all platforms.
> 
> Are there reproducible tests/benchmarks that we can use to figure out
> whether or not this actually makes sense on any given architecture?

The test I used most recently was to run test008 in the OpenLDAP test suite and 
average the runtimes over 5-10 runs for each BDB version. In my tests I ran 
with SLAPD_DEBUG=0 so the only I/O traffic is from BDB and not from debug 
logging. I also ran with an extremely small cachesize setting (5, the default 
is 1000) to further aggravate the locking contention in the underlying DB. 
Under normal conditions (where the cache is not so ridiculously undersized for 
the workload) the differences are not as apparent.

Also note that the runtimes for this test are non-deterministic since they are 
affected by deadlock retries, and their pattern is unpredictable on most systems.
-- 
  -- Howard Chu
  Chief Architect, Symas Corp.  http://www.symas.com
  Director, Highland Sun        http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP     http://www.openldap.org/project/



Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#421946; Package openldap2.3. Full text and rfc822 format available.

Acknowledgement sent to Clint Adams <schizo@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #100 received at 421946@bugs.debian.org (full text, mbox):

From: Clint Adams <schizo@debian.org>
To: Howard Chu <hyc@highlandsun.com>
Cc: 421946@bugs.debian.org
Subject: Re: openldap2.3: build against newer Berkeley DB
Date: Mon, 8 Oct 2007 23:04:17 -0400
On Mon, Sep 03, 2007 at 11:23:22PM -0700, Howard Chu wrote:
> The test I used most recently was to run test008 in the OpenLDAP test suite 
> and average the runtimes over 5-10 runs for each BDB version. In my tests I 
> ran with SLAPD_DEBUG=0 so the only I/O traffic is from BDB and not from 
> debug logging. I also ran with an extremely small cachesize setting (5, the 
> default is 1000) to further aggravate the locking contention in the 
> underlying DB. Under normal conditions (where the cache is not so 
> ridiculously undersized for the workload) the differences are not as 
> apparent.
>
> Also note that the runtimes for this test are non-deterministic since they 
> are affected by deadlock retries, and their pattern is unpredictable on 
> most systems.

Does the hybrid mutex performance fix in db 4.6.21 solve this problem?




Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#421946; Package openldap2.3. Full text and rfc822 format available.

Acknowledgement sent to Howard Chu <hyc@highlandsun.com>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #105 received at 421946@bugs.debian.org (full text, mbox):

From: Howard Chu <hyc@highlandsun.com>
To: Clint Adams <schizo@debian.org>
Cc: 421946@bugs.debian.org
Subject: Re: openldap2.3: build against newer Berkeley DB
Date: Mon, 08 Oct 2007 20:42:00 -0700
Clint Adams wrote:
> On Mon, Sep 03, 2007 at 11:23:22PM -0700, Howard Chu wrote:
>> The test I used most recently was to run test008 in the OpenLDAP test suite 
>> and average the runtimes over 5-10 runs for each BDB version. In my tests I 
>> ran with SLAPD_DEBUG=0 so the only I/O traffic is from BDB and not from 
>> debug logging. I also ran with an extremely small cachesize setting (5, the 
>> default is 1000) to further aggravate the locking contention in the 
>> underlying DB. Under normal conditions (where the cache is not so 
>> ridiculously undersized for the workload) the differences are not as 
>> apparent.
>>
>> Also note that the runtimes for this test are non-deterministic since they 
>> are affected by deadlock retries, and their pattern is unpredictable on 
>> most systems.
> 
> Does the hybrid mutex performance fix in db 4.6.21 solve this problem?
> 
I haven't tested 4.6.21 yet. Looking at the diff from 4.6.19 I see they simply 
inserted a yield() into the spinlock loop. I expect that this will introduce 
other problems on NPTL systems, since yield()'s behavior is so unfriendly in NPTL.

-- 
  -- Howard Chu
  Chief Architect, Symas Corp.  http://www.symas.com
  Director, Highland Sun        http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP     http://www.openldap.org/project/




Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#421946; Package openldap2.3. Full text and rfc822 format available.

Acknowledgement sent to Howard Chu <hyc@highlandsun.com>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #110 received at 421946@bugs.debian.org (full text, mbox):

From: Howard Chu <hyc@highlandsun.com>
To: Clint Adams <schizo@debian.org>
Cc: 421946@bugs.debian.org
Subject: Re: openldap2.3: build against newer Berkeley DB
Date: Mon, 08 Oct 2007 21:00:17 -0700
Clint Adams wrote:
> On Mon, Sep 03, 2007 at 11:23:22PM -0700, Howard Chu wrote:
>> The test I used most recently was to run test008 in the OpenLDAP test suite 
>> and average the runtimes over 5-10 runs for each BDB version. In my tests I 
>> ran with SLAPD_DEBUG=0 so the only I/O traffic is from BDB and not from 
>> debug logging. I also ran with an extremely small cachesize setting (5, the 
>> default is 1000) to further aggravate the locking contention in the 
>> underlying DB. Under normal conditions (where the cache is not so 
>> ridiculously undersized for the workload) the differences are not as 
>> apparent.
>>
>> Also note that the runtimes for this test are non-deterministic since they 
>> are affected by deadlock retries, and their pattern is unpredictable on 
>> most systems.
> 
> Does the hybrid mutex performance fix in db 4.6.21 solve this problem?

I've just rerun my tests using both BDB 4.2.52 and 4.6.21 and the behavior is 
as expected. On an otherwise idle machine, both perform well, completing our 
concurrency test in about 31 seconds.

With a CPU-hog running in the background, the test with BDB 4.2.52 takes only 
37 seconds, while BDB 4.6.21 takes 1:42. Watching with top you can see that 
BDB 4.6.21 gets a lot less CPU than BDB 4.2.52. This is the problem with using 
yield() on an NPTL system - whereas on most POSIX systems yield() only yields 
control to some other thread in the current process, on NPTL yield() gives up 
the CPU for the entire process.
-- 
  -- Howard Chu
  Chief Architect, Symas Corp.  http://www.symas.com
  Director, Highland Sun        http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP     http://www.openldap.org/project/




Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#421946; Package openldap2.3. Full text and rfc822 format available.

Acknowledgement sent to Clint Adams <schizo@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. Full text and rfc822 format available.

Message #115 received at 421946@bugs.debian.org (full text, mbox):

From: Clint Adams <schizo@debian.org>
To: Howard Chu <hyc@highlandsun.com>
Cc: 421946@bugs.debian.org
Subject: Re: openldap2.3: build against newer Berkeley DB
Date: Fri, 7 Mar 2008 09:43:27 -0500
On Mon, Oct 08, 2007 at 09:00:17PM -0700, Howard Chu wrote:
> I've just rerun my tests using both BDB 4.2.52 and 4.6.21 and the 
> behavior is as expected. On an otherwise idle machine, both perform well, 
> completing our concurrency test in about 31 seconds.
>
> With a CPU-hog running in the background, the test with BDB 4.2.52 takes 
> only 37 seconds, while BDB 4.6.21 takes 1:42. Watching with top you can 
> see that BDB 4.6.21 gets a lot less CPU than BDB 4.2.52. This is the 
> problem with using yield() on an NPTL system - whereas on most POSIX 
> systems yield() only yields control to some other thread in the current 
> process, on NPTL yield() gives up the CPU for the entire process.

Were these run with or without a ShmKey / DB_SYSTEM_MEM ?




Bug reassigned from package `openldap2.3' to `openldap2.3,openldap'. Request was from Steve Langasek <vorlon@debian.org> to control@bugs.debian.org. (Mon, 07 Jul 2008 18:45:08 GMT) Full text and rfc822 format available.

Bug marked as found in version openldap2.3/2.3.30-5. Request was from Steve Langasek <vorlon@debian.org> to control@bugs.debian.org. (Mon, 07 Jul 2008 18:45:08 GMT) Full text and rfc822 format available.

Bug marked as found in version openldap/2.4.10-2. Request was from Steve Langasek <vorlon@debian.org> to control@bugs.debian.org. (Mon, 07 Jul 2008 18:45:09 GMT) Full text and rfc822 format available.

Bug reassigned from package `openldap2.3,openldap' to `openldap'. Request was from Clint Adams <schizo@debian.org> to control@bugs.debian.org. (Sat, 20 Sep 2008 15:00:14 GMT) Full text and rfc822 format available.

Bug marked as found in version 2.4.10-2. Request was from Clint Adams <schizo@debian.org> to control@bugs.debian.org. (Sat, 20 Sep 2008 15:00:14 GMT) Full text and rfc822 format available.

Bug marked as found in version 2.4.10-3. Request was from Clint Adams <schizo@debian.org> to control@bugs.debian.org. (Sat, 20 Sep 2008 15:00:15 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>:
Bug#421946; Package openldap. (Thu, 09 Oct 2008 22:24:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Howard Chu <hyc@highlandsun.com>:
Extra info received and forwarded to list. Copy sent to Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>. (Thu, 09 Oct 2008 22:24:02 GMT) Full text and rfc822 format available.

Message #132 received at 421946@bugs.debian.org (full text, mbox):

From: Howard Chu <hyc@highlandsun.com>
To: Clint Adams <schizo@debian.org>
Cc: 421946@bugs.debian.org
Subject: Re: openldap2.3: build against newer Berkeley DB
Date: Thu, 09 Oct 2008 15:12:56 -0700
Clint Adams wrote:
> On Mon, Oct 08, 2007 at 09:00:17PM -0700, Howard Chu wrote:
>> I've just rerun my tests using both BDB 4.2.52 and 4.6.21 and the
>> behavior is as expected. On an otherwise idle machine, both perform well,
>> completing our concurrency test in about 31 seconds.
>>
>> With a CPU-hog running in the background, the test with BDB 4.2.52 takes
>> only 37 seconds, while BDB 4.6.21 takes 1:42. Watching with top you can
>> see that BDB 4.6.21 gets a lot less CPU than BDB 4.2.52. This is the
>> problem with using yield() on an NPTL system - whereas on most POSIX
>> systems yield() only yields control to some other thread in the current
>> process, on NPTL yield() gives up the CPU for the entire process.
>
> Were these run with or without a ShmKey / DB_SYSTEM_MEM ?
>
Both with and without DB_SYSTEM_MEM; that's really irrelevant to the poor 
locking behavior.

-- 
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/




Tags added: pending Request was from vorlon@alioth.debian.org to control@bugs.debian.org. (Tue, 17 Feb 2009 18:15:02 GMT) Full text and rfc822 format available.

Bug reassigned from package `openldap' to `slapd'. Request was from Clint Adams <schizo@debian.org> to control@bugs.debian.org. (Wed, 18 Feb 2009 01:54:05 GMT) Full text and rfc822 format available.

Bug marked as found in version 2.4.11-1. Request was from Clint Adams <schizo@debian.org> to control@bugs.debian.org. (Wed, 18 Feb 2009 01:54:05 GMT) Full text and rfc822 format available.

Severity set to `serious' from `normal' Request was from Clint Adams <schizo@debian.org> to control@bugs.debian.org. (Wed, 18 Feb 2009 01:54:06 GMT) Full text and rfc822 format available.

Tags added: squeeze, sid Request was from Steve Langasek <vorlon@debian.org> to control@bugs.debian.org. (Wed, 18 Feb 2009 02:15:03 GMT) Full text and rfc822 format available.

Reply sent to Steve Langasek <vorlon@debian.org>:
You have taken responsibility. (Wed, 25 Feb 2009 01:15:02 GMT) Full text and rfc822 format available.

Notification sent to Clint Adams <schizo@debian.org>:
Bug acknowledged by developer. (Wed, 25 Feb 2009 01:15:02 GMT) Full text and rfc822 format available.

Message #147 received at 421946-close@bugs.debian.org (full text, mbox):

From: Steve Langasek <vorlon@debian.org>
To: 421946-close@bugs.debian.org
Subject: Bug#421946: fixed in openldap 2.4.15-1
Date: Wed, 25 Feb 2009 00:47:08 +0000
Source: openldap
Source-Version: 2.4.15-1

We believe that the bug you reported is fixed in the latest version of
openldap, which is due to be installed in the Debian FTP archive:

ldap-utils_2.4.15-1_amd64.deb
  to pool/main/o/openldap/ldap-utils_2.4.15-1_amd64.deb
libldap-2.4-2-dbg_2.4.15-1_amd64.deb
  to pool/main/o/openldap/libldap-2.4-2-dbg_2.4.15-1_amd64.deb
libldap-2.4-2_2.4.15-1_amd64.deb
  to pool/main/o/openldap/libldap-2.4-2_2.4.15-1_amd64.deb
libldap2-dev_2.4.15-1_amd64.deb
  to pool/main/o/openldap/libldap2-dev_2.4.15-1_amd64.deb
openldap_2.4.15-1.diff.gz
  to pool/main/o/openldap/openldap_2.4.15-1.diff.gz
openldap_2.4.15-1.dsc
  to pool/main/o/openldap/openldap_2.4.15-1.dsc
openldap_2.4.15.orig.tar.gz
  to pool/main/o/openldap/openldap_2.4.15.orig.tar.gz
slapd-dbg_2.4.15-1_amd64.deb
  to pool/main/o/openldap/slapd-dbg_2.4.15-1_amd64.deb
slapd_2.4.15-1_amd64.deb
  to pool/main/o/openldap/slapd_2.4.15-1_amd64.deb



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 421946@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Steve Langasek <vorlon@debian.org> (supplier of updated openldap package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.8
Date: Tue, 24 Feb 2009 14:27:35 -0800
Source: openldap
Binary: slapd ldap-utils libldap-2.4-2 libldap-2.4-2-dbg libldap2-dev slapd-dbg
Architecture: source amd64
Version: 2.4.15-1
Distribution: unstable
Urgency: low
Maintainer: Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>
Changed-By: Steve Langasek <vorlon@debian.org>
Description: 
 ldap-utils - OpenLDAP utilities
 libldap-2.4-2 - OpenLDAP libraries
 libldap-2.4-2-dbg - Debugging information for OpenLDAP libraries
 libldap2-dev - OpenLDAP development libraries
 slapd      - OpenLDAP server (slapd)
 slapd-dbg  - Debugging information for the OpenLDAP server (slapd)
Closes: 421946 497697
Changes: 
 openldap (2.4.15-1) unstable; urgency=low
 .
   * New upstream version
     - Fixes a bug with the pcache overlay not returning cached entries
       (closes: #497697)
     - Update evolution-ntlm patch to apply to current Makefiles.
     - (tentatively) drop gnutls-ciphers, since this bug was reported to be
       fixed upstream in 2.4.8.  The fix applied in 2.4.8 didn't match the
       patch from the bug report, so this should be watched for regressions.
   * Build against db4.7 instead of db4.2 at last!  Closes: #421946.
   * Build with --disable-ndb, to avoid a misbuild when libmysqlclient is
     installed in the build environment.
   * Add -D_GNU_SOURCE to CFLAGS, apparently required for building with
     current headers in unstable
Checksums-Sha1: 
 168b5f88fd375d8b305a8d4da88c4a5dd552a287 1794 openldap_2.4.15-1.dsc
 bee25478ea4400d7363a3e1b127c69fe397e201b 4339060 openldap_2.4.15.orig.tar.gz
 ff3763f263c8b4abf799fe20aa8497f051d36d13 145857 openldap_2.4.15-1.diff.gz
 0b4808ce64c7491b52416720d215a6469a392a39 1550066 slapd_2.4.15-1_amd64.deb
 5d0b72021b2d44c0a3e12098475300bf3696f2f2 275064 ldap-utils_2.4.15-1_amd64.deb
 9c9a276ed65a4fda15a650a27db6d3204399a389 211178 libldap-2.4-2_2.4.15-1_amd64.deb
 6716c5dc8567097c299851ace644a19151f9b3ba 320940 libldap-2.4-2-dbg_2.4.15-1_amd64.deb
 6eae9177b1f299201ca9d9cc6c50784dae07eb97 935548 libldap2-dev_2.4.15-1_amd64.deb
 c1460e59184b192a5c61acb626a5b67d9a6d7de0 3888836 slapd-dbg_2.4.15-1_amd64.deb
Checksums-Sha256: 
 a0ed4dcfe948a80a1ee05b5b43385b3d39d7f9859d1e799cce533290e0b9f0ad 1794 openldap_2.4.15-1.dsc
 3c45f16930212a74e0c6bd5e61bbf8152f238ffdb44bed430ae61d87d1ff5177 4339060 openldap_2.4.15.orig.tar.gz
 6ecc9e5339a1f4a13cddcd07dfcec3acc9770ba7181a55fc7ac666c9e4761388 145857 openldap_2.4.15-1.diff.gz
 eb82f1272773f286560a38150ff75e1ad0b48c4f103454202585ffb0ea9cfb35 1550066 slapd_2.4.15-1_amd64.deb
 a15b679a1acca2f2b3055fb15be854495c3f78fb23bfd979d36cf88becd80929 275064 ldap-utils_2.4.15-1_amd64.deb
 2d0b193513bfefb20f679b55087de0fad24dcee088132f280859110a1df03c1c 211178 libldap-2.4-2_2.4.15-1_amd64.deb
 044375606db5b2002557a43163f324fdab7b4590ff82b4e19df1106753cd811b 320940 libldap-2.4-2-dbg_2.4.15-1_amd64.deb
 251d317c493963bf07f760e5a6b30b0328f2e4957a00333c5238b4da4757653c 935548 libldap2-dev_2.4.15-1_amd64.deb
 d8364ef73d6c3c4716827d4c0b823b71cec825e788b10b86b7a73074b372c7e1 3888836 slapd-dbg_2.4.15-1_amd64.deb
Files: 
 79302feb3697357229a9d1aa51fe84fe 1794 net optional openldap_2.4.15-1.dsc
 7b3ce3de2f252a1e88cd55c68551b73d 4339060 net optional openldap_2.4.15.orig.tar.gz
 4ce43a9e6f5b9d4fab5e4ade7a6c463e 145857 net optional openldap_2.4.15-1.diff.gz
 37eecf5158017e3cbfd5df8dcc70d504 1550066 net optional slapd_2.4.15-1_amd64.deb
 c41cc4b08968de3254235cf20ab7b502 275064 net optional ldap-utils_2.4.15-1_amd64.deb
 909dcc294871200df99be49eefccc7a8 211178 libs optional libldap-2.4-2_2.4.15-1_amd64.deb
 e20adcec6c279e8197914e7de5797916 320940 libdevel extra libldap-2.4-2-dbg_2.4.15-1_amd64.deb
 e875cb8035bdb88a4db95d6d06041545 935548 libdevel extra libldap2-dev_2.4.15-1_amd64.deb
 f8fd36c3615949c1d0d3c8b6c7a284b9 3888836 net extra slapd-dbg_2.4.15-1_amd64.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iD8DBQFJpJHsKN6ufymYLloRAjdBAKDRCztGrNUMbGmGBDnA5PL5DJaozQCfQBta
0hVW1HpRzgS4ua7DqZnUycQ=
=j/f4
-----END PGP SIGNATURE-----





Bug archived. Request was from Debbugs Internal Request <owner@bugs.debian.org> to internal_control@bugs.debian.org. (Mon, 07 Mar 2011 08:59:41 GMT) Full text and rfc822 format available.

Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Fri Apr 18 21:03:57 2014; Machine Name: beach.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.