Very Computer

I have heard that Solaris will allocate pages of memory to a process and
does not release them  - ever by default.  And I've heard that there is a
tuning "thing" for the OS
that allows it to free up pages of memory that have not been accessed for X
period of time and is configurable.

Can anyone verify and tell me what the name of the "thing" is?

On Wed, 27 Nov 2002 09:01:29 -0500, "Jesse Waters"

Then you heard wrong.

Quote:>And I've heard that there is a tuning "thing" for the OS that allows it to
>free up pages of memory that have not been accessed for X period of
>time and is configurable.

There are tuneable parameters for reclaiming memory. However they are
best left alone unless you really know what you're doing. And I would
hazard a guess that since you're asking you probably don't meet that
criteria :-)

But, if you still wish to proceed the following tuneables are
available:

- fastscan
- slowscan
- physmem
- lotsfree
- desfree
- minfree
- throttlefree
- handspreadpages
- max_page_get
- maxpgio
- tune_t_fsflushr
- autoup

If you're interested in learning more about these parameters refer to
the book: "Sun Performance and Tuning - Second Edition" by "Adrian
*croft and Richard Pettit" [ ISBN 0-13-095249-4 ]. It is an
excellent book that discusses memory tuning and much more.

Quote:>Can anyone verify and tell me what the name of the "thing" is?

Is this the result of seeing a small amount of free memory in the
vmstat output? Prior to Solaris 8 the vmstat output usually showed a
few megabytes of free memory. This was a result of Solaris caching
filesystem data. By default Solaris will cache filesystem data thus
giving the appearance that the system is low on memory. However if the
system needs memory for an application the cached filesystem data will
be released and allocated to application. In Solaris 8 I believe that
the way memory is reported was changed to reflect only the memory
allocated to processes. While the filesystem data is still cached its
usage is no longer is reported in the vmstat output.

Josh

Thankx for the tip, and I do not know enough to go at it on my own but I can
ask, can't I! ;-)
I have a process that will consume RAM, swap pages out to swap space and
never reclaim it.  I wondered if there is a tool that could reclaim that
because the RSS and SZ continue to grow.  The obvious fix is to stop the
memory leak but I do not have that option.  Also I'm not knowledgeable
enough to correct the problem myself but there is a good Sun Admin here that
could if there is, in fact, a way to do it.
-jcw

Unless by chance the original poster was thinking about
the default malloc() arena...freed memory is not returned
to the system, just the arena within the process address space.

-Jimmo

ANother way would be to re-start the broken process.

Quote:> enough to correct the problem myself but there is a good Sun Admin here that
> could if there is, in fact, a way to do it.

Heh - if he's that good, he wouldn't need you to tell him
what's wrong, or how to fix it...  ;-)

--
Rich Teer

President,
Rite Online Inc.

Voice: +1 (250) 979-1638
URL: http://www.rite-online.net

Yeah, we do that already!

Quote:> > enough to correct the problem myself but there is a good Sun Admin here
that
> > could if there is, in fact, a way to do it.

> Heh - if he's that good, he wouldn't need you to tell him
> what's wrong, or how to fix it...  ;-)

No argument here, but I have to say that years ago I did some unix admin on
osf/1 and now that I work with Sun I want to learn all I can.  This one is
not the type to teach me anything.  If I want something I have to tell him
exactly what I want and he'll do it.  Other than that there is no use in
asking him questions like these.

Quote:> --
> Rich Teer

> President,
> Rite Online Inc.

> Voice: +1 (250) 979-1638
> URL: http://www.rite-online.net

ah, so there really could be something to this nebulus "thing"!

How could these pages be reclaimed?  The kernel cannot know that the
process never needs the memory again.

Looks like the process has a memory leak and it should be fixed.

Casper
--
Expressed in this posting are my opinions.  They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

Good idea.

Quote:> not the type to teach me anything.  If I want something I have to tell him
> exactly what I want and he'll do it.  Other than that there is no use in
> asking him questions like these.

Ah.  One of those "job security" idiots.  He won't teach you
anything, because he isn't confident enough in his own abilities.

--
Rich Teer

President,
Rite Online Inc.

Voice: +1 (250) 979-1638
URL: http://www.rite-online.net

Very few Unices or freenixes offer such a feature.  AIX is the only commercial
Unix that allows free()-ed memory to be subsequently given back to the OS by
way of a library call.  It has come to my attention that Linux is still con-
sidering such a capability via an enhanced malloc() (in conjunction with AIX's
SIGDANGER signal).  

Other than AIX through its supplied dislaim() call that does this, all Unices
are able to return memory back to the OS via low-level sbrk() calls; but that
becomes sticky and problematic (which is why its better if the OS would just
supply a wrapper function to take all the guesswork out of it).

[El Toro]:

Quote:

>   Very few Unices or freenixes offer such a feature.  AIX is the
>   only commercial Unix that allows free()-ed memory to be
>   subsequently given back to the OS by way of a library call.  It
>   has come to my attention that Linux is still con- sidering such a
>   capability via an enhanced malloc() (in conjunction with AIX's
>   SIGDANGER signal).

this is a standard feature in the malloc found in GNU libc 2.x, but by
default it is only used on large objects.  you can adjust the
behaviour using mallopt(3c).  the overhead from setting up and tearing
down the memory mapping of /dev/zero is quite great, so you really
don't want to use an mmap() for every allocated object.

Kjetil T.

Quote:> [El Toro]:

> >   Very few Unices or freenixes offer such a feature.  AIX is the
> >   only commercial Unix that allows free()-ed memory to be
> >   subsequently given back to the OS by way of a library call.  It
> >   has come to my attention that Linux is still con- sidering such a
> >   capability via an enhanced malloc() (in conjunction with AIX's
> >   SIGDANGER signal).

> this is a standard feature in the malloc found in GNU libc 2.x, but by
> default it is only used on large objects. ...

Right.  Solaris of mmap based alternatives of malloc too.

-Jimmo

Quote:> [El Toro]:

> >   Very few Unices or freenixes offer such a feature.  AIX is the
> >   only commercial Unix that allows free()-ed memory to be
> >   subsequently given back to the OS by way of a library call.  It
> >   has come to my attention that Linux is still con- sidering such a
> >   capability via an enhanced malloc() (in conjunction with AIX's
> >   SIGDANGER signal).

> this is a standard feature in the malloc found in GNU libc 2.x, but by
> default it is only used on large objects.  you can adjust the
> behaviour using mallopt(3c).  the overhead from setting up and tearing
> down the memory mapping of /dev/zero is quite great, so you really
> don't want to use an mmap() for every allocated object.

Ah, thanks.  So they've implemented the first half already.  Do you know
what progress they've made on the SIGDANGER implementation, so that the
OOM killer can be more intelligent?  Hmm, I guess that's a bit off-topic,
though.  Please respond in private if you know the answer.

What versions of Solaris mmap() provides the ability to release memory
back to the OS?  I don't see one mentioned in the Sol8 manpages, nor a
mention of such a feature from mallopt().  Is this a Solaris 9 thing?

Thanks in advance.

That should have read "Solaris has mmap based alternatives of
malloc too".  Perils of writing with one hand while nursing
an unwell baby with the other :)

Quote:> What versions of Solaris mmap() provides the ability to release memory
> back to the OS?  I don't see one mentioned in the Sol8 manpages, nor a
> mention of such a feature from mallopt().  Is this a Solaris 9 thing?

I was referring to things like libmapmalloc which pull
in a version of malloc() that uses mmap("/dev/zero") as
the source of memory.

free() then leads to munmap() once a whole page has been
freed and this, of course, returns the page to the
system and doesn't keep it allocated to the process
address space.

Cheers
Jimmo