[ug-bosug] Fwd: [cslug] malloc problem on Solaris

[Binu Jose Philip]

On 9/14/07, S h i v <shivakumar.gn at gmail.com> wrote:
> > ---------- Forwarded message ----------
> > From:  vijay das <lovingvijay2002 at yahoo.com>
> > To: cslug at yahoogroups.com
> >
> >  The default Sun Solaris malloc library is generally
> >  considered inefficient.
> >  For high scalability and high performance, it is best
> >  to use 3rd party memory manager library.  For example,
> >  I have simple/pure C/C++ program
> >
> >  1.  Allocates, say 512 MB memory.
> >  2.  Free the memory allocated in step 1.
> >  3.  Allocate, memory in step1 + 10MB.
> >  4.  Free the memory in step 3.
> >
>
> I havent run any benchmarks or tests to compare the efficiency of the
> memory management, so cant comment on the *considered inefficient*
> part.
>
> But the example above does not give an indication of proper/improper
> memory management due to reasons mentioned below.

Right. Use prstat and look at the 'rss' column, this is the resident set size.
That tells you how much physical memory is allocated for the process.

If you want a more modern memory allocator use libumem, it is much
better than third party allocators. The old allocator for most Unices the
buddy allocator, does not scale well and the performance is average.
libumem is a slab allocator - the same as what solaris VM and ZFS use.

http://blogs.sun.com/ahl/entry/solaris_10_top_11_20
http://blogs.sun.com/bonwick/category/Slab+Allocator
http://citeseer.ist.psu.edu/bonwick94slab.html

cheers
Binu

> >  If I use 'pmap' to watch the meomory *after* step 2
> >  and step 4, it still says that much memory is still in
> >  use by the process.
>
> As far as my understanding goes, with default memory management in
> Solaris, the heap mapping is virtual memory, so requesting memory with
> malloc does not allocate physical memory. It allocates the virtual
> address space. Physical address gets allocated on first reference
> (triggered by a page fault). On freeing the memory, the memory is not
> reclaimed immediately by the system but remain in the heap mapping as
> unused pages. Subsequent allocations will reuse these pages and not
> get it allocated from the system thus yielding better efficiency (do a
> memory allocation again after free and the memory as shown by
> prstat/pmap doesnt grow).
>
> There is a system page scanner that runs in the background and
> reclaims the unused pages when system moves towards mem crunch
> situation.
> The "Solaris Internals" book covers this in great detail in sections
> "9 Virtual Memory" & "10. Physical Memory".
>
> Others who have additional info may add to this.
>
> -Shiv
> _______________________________________________
> ug-bosug mailing list
> List-Unsubscribe: mailto:ug-bosug-unsubscribe at opensolaris.org
> List-Owner: mailto:ug-bosug-owner at opensolaris.org
> List-Archives: http://www.opensolaris.org/jive/forum.jspa?forumID=54
>