[glue-wg] On BDII performance was Re: When is data stale?

[Paul Millar]

Hi Florido,

Sorry for the delay in replying.

On 22/04/15 11:45, Florido Paganelli wrote:
[..]
> As you can see above, you just bechmarked the top of the iceberg.

Agreed; although it's nice to see that many site-level BDIIs are very 
fast, while a very small fraction are (perhaps disproportionally) slow. 
  This could be influencing the latency, as the top-level BDII waits for 
all site-level BDIIs to complete sending before generating the LDIF and 
updating the slapd.

[...]
> Over the years Laurence managed to shorten down this update time with
> several smart ideas, that include also enterprise-level techniques like
> replication, and probably partial LDIF documents where applicable. I
> think in this way he avoided having two LDAP servers.

That is my understanding too: Laurence has improved the situation 
greatly over the years.


> You have to understand that the LDAP technology is intended for data
> that changes rarely, and we're using it for an almost real-time system.
> One more hint of the fact that is a bad monitoring tool...
>
> Trust me, 30 mins is a great achievement for a technology that never was
> meant to do what we use it for. I might have several arguments against
> BDII code but not about its performance.

I think we have to agree to differ slightly on this point.

I certainly agree with you that OpenLDAP software is designed for read 
performance rather than update (or write) performance, with the 
consequence that it doesn't work so well for all the use-cases we might 
want.

I haven't benchmarked the complete BDII update life-cycle, especially 
not for lcg-bdii.cern.ch: the top-level BDII that I often use as a 
reference point.  I don't know how much time is spent preparing the 
LDIF, compared to the time OpenLDAP takes to process the changes.  If 
generating the LDIF takes the majority of the time then perhaps this 
could be improved: currently this is done with perl and python, perhaps 
alternative technologies would be faster?

However, I still feel that there's nothing intrinsic in what was want to 
achieve that prevents us using LDAP the network protocol (i.e., not 
necessarily OpenLDAP) to achieving much reduced latencies.


> The problems we're facing with ARC while trying to move to other
> technologies is that query times for LDAP are faster than other
> investigated technologies (e.g. REST web services)
> Update times are horrible, but for people is more important to have the
> queries fast than the information fresh it seems...
> And I can say this because in ARC we put also jobs in the LDAP database,
> which is EXTREME for today's numbers (i.e. O(10000) jobs).
> It's nice(?) that these numbers match those that Stephen mentioned.

Yes, I guess so.  It's living on the "bleeding edge" that triggers 
innovation, right?

Cheers,

Paul.