[Duckwater-discuss] [sparks-discuss] Code-review request for Duckwater Phase 0

[Nicolas Williams]

On Thu, May 29, 2008 at 05:55:40PM -0500, Nicolas Williams wrote:
>    Also, if I understand correctly we could have practically every
>    process blocked waiting for ldap_cachemgr to do something, is that
>    correct?  If so, then I think this is a bit sub-optimal.  I'd much,
>    much rather we find a way to avoid such waits.  I don't mind lots of
>    processes blocking on nscd, for example, if they are calling getXbyYs
>    and the name service backends are blocking.

After talking to Michen and looking at the code I have these additional
comments:

 - 2^16 threads created with default stack size (1MB for 32-bit
   programs) is just not possible in a 32-bit program (since that would
   require a virtual memory address space of 2^36 bytes just for the
   stacks.

   So at a bare minimum MAX_CHG_THREADS is set to a number so large that
   were it tested then ldap_cachemgr would run out of resources.


 - To date all the other door calls served ldap_cachemgr are
   non-blocking.

   Now we're adding a door call that blocks on a condition variable.

   This means that if the status change door calls consume all available
   door server threads then the other door calls will block too.

   I recommend creating a separate door for these blocking calls.  Bind
   a small number of threads to it, or maybe even just one -- no need to
   have thousands of threads blocking on a condition variable in
   ldap_cachemgr when they can be blocking on one in the door's knob in
   the kernel.  This way you can have far, far fewer threads in
   ldap_cachemgr, which a much, much smaller memory footprint.


 - Please file a CR for determining what stack size we need in
   ldap_cachemgr and setting its threads' stack sizes to that.


Nico
--