Solaris tuning parameter

Solaris tuning parameter

Post by Jesse Water » Thu, 28 Nov 2002 23:01:29



I have heard that Solaris will allocate pages of memory to a process and
does not release them  - ever by default.  And I've heard that there is a
tuning "thing" for the OS
that allows it to free up pages of memory that have not been accessed for X
period of time and is configurable.

Can anyone verify and tell me what the name of the "thing" is?

 
 
 

Solaris tuning parameter

Post by Josh Mcke » Fri, 29 Nov 2002 01:02:32


On Wed, 27 Nov 2002 09:01:29 -0500, "Jesse Waters"


>I have heard that Solaris will allocate pages of memory to a process and
>does not release them  - ever by default.

Then you heard wrong.

Quote:>And I've heard that there is a tuning "thing" for the OS that allows it to
>free up pages of memory that have not been accessed for X period of
>time and is configurable.

There are tuneable parameters for reclaiming memory. However they are
best left alone unless you really know what you're doing. And I would
hazard a guess that since you're asking you probably don't meet that
criteria :-)

But, if you still wish to proceed the following tuneables are
available:

- fastscan
- slowscan
- physmem
- lotsfree
- desfree
- minfree
- throttlefree
- handspreadpages
- max_page_get
- maxpgio
- tune_t_fsflushr
- autoup

If you're interested in learning more about these parameters refer to
the book: "Sun Performance and Tuning - Second Edition" by "Adrian
*croft and Richard Pettit" [ ISBN 0-13-095249-4 ]. It is an
excellent book that discusses memory tuning and much more.

Quote:>Can anyone verify and tell me what the name of the "thing" is?

Is this the result of seeing a small amount of free memory in the
vmstat output? Prior to Solaris 8 the vmstat output usually showed a
few megabytes of free memory. This was a result of Solaris caching
filesystem data. By default Solaris will cache filesystem data thus
giving the appearance that the system is low on memory. However if the
system needs memory for an application the cached filesystem data will
be released and allocated to application. In Solaris 8 I believe that
the way memory is reported was changed to reflect only the memory
allocated to processes. While the filesystem data is still cached its
usage is no longer is reported in the vmstat output.

Josh

 
 
 

Solaris tuning parameter

Post by Jesse Water » Fri, 29 Nov 2002 03:15:54



> On Wed, 27 Nov 2002 09:01:29 -0500, "Jesse Waters"

> >I have heard that Solaris will allocate pages of memory to a process and
> >does not release them  - ever by default.

> Then you heard wrong.

> >And I've heard that there is a tuning "thing" for the OS that allows it
to
> >free up pages of memory that have not been accessed for X period of
> >time and is configurable.

> There are tuneable parameters for reclaiming memory. However they are
> best left alone unless you really know what you're doing. And I would
> hazard a guess that since you're asking you probably don't meet that
> criteria :-)

> But, if you still wish to proceed the following tuneables are
> available:

> - fastscan
> - slowscan
> - physmem
> - lotsfree
> - desfree
> - minfree
> - throttlefree
> - handspreadpages
> - max_page_get
> - maxpgio
> - tune_t_fsflushr
> - autoup

> If you're interested in learning more about these parameters refer to
> the book: "Sun Performance and Tuning - Second Edition" by "Adrian
>*croft and Richard Pettit" [ ISBN 0-13-095249-4 ]. It is an
> excellent book that discusses memory tuning and much more.

> >Can anyone verify and tell me what the name of the "thing" is?

> Is this the result of seeing a small amount of free memory in the
> vmstat output? Prior to Solaris 8 the vmstat output usually showed a
> few megabytes of free memory. This was a result of Solaris caching
> filesystem data. By default Solaris will cache filesystem data thus
> giving the appearance that the system is low on memory. However if the
> system needs memory for an application the cached filesystem data will
> be released and allocated to application. In Solaris 8 I believe that
> the way memory is reported was changed to reflect only the memory
> allocated to processes. While the filesystem data is still cached its
> usage is no longer is reported in the vmstat output.

> Josh

Thankx for the tip, and I do not know enough to go at it on my own but I can
ask, can't I! ;-)
I have a process that will consume RAM, swap pages out to swap space and
never reclaim it.  I wondered if there is a tool that could reclaim that
because the RSS and SZ continue to grow.  The obvious fix is to stop the
memory leak but I do not have that option.  Also I'm not knowledgeable
enough to correct the problem myself but there is a good Sun Admin here that
could if there is, in fact, a way to do it.
-jcw
 
 
 

Solaris tuning parameter

Post by Jimm » Fri, 29 Nov 2002 03:41:43



> On Wed, 27 Nov 2002 09:01:29 -0500, "Jesse Waters"

> >I have heard that Solaris will allocate pages of memory to a process and
> >does not release them  - ever by default.

> Then you heard wrong.

Unless by chance the original poster was thinking about
the default malloc() arena...freed memory is not returned
to the system, just the arena within the process address space.

-Jimmo

 
 
 

Solaris tuning parameter

Post by Rich Tee » Fri, 29 Nov 2002 05:35:04



> I have a process that will consume RAM, swap pages out to swap space and
> never reclaim it.  I wondered if there is a tool that could reclaim that
> because the RSS and SZ continue to grow.  The obvious fix is to stop the
> memory leak but I do not have that option.  Also I'm not knowledgeable

ANother way would be to re-start the broken process.

Quote:> enough to correct the problem myself but there is a good Sun Admin here that
> could if there is, in fact, a way to do it.

Heh - if he's that good, he wouldn't need you to tell him
what's wrong, or how to fix it...  ;-)

--
Rich Teer

President,
Rite Online Inc.

Voice: +1 (250) 979-1638
URL: http://www.rite-online.net

 
 
 

Solaris tuning parameter

Post by Jesse Water » Sat, 30 Nov 2002 21:44:13




> > I have a process that will consume RAM, swap pages out to swap space and
> > never reclaim it.  I wondered if there is a tool that could reclaim that
> > because the RSS and SZ continue to grow.  The obvious fix is to stop the
> > memory leak but I do not have that option.  Also I'm not knowledgeable

> ANother way would be to re-start the broken process.

Yeah, we do that already!

Quote:> > enough to correct the problem myself but there is a good Sun Admin here
that
> > could if there is, in fact, a way to do it.

> Heh - if he's that good, he wouldn't need you to tell him
> what's wrong, or how to fix it...  ;-)

No argument here, but I have to say that years ago I did some unix admin on
osf/1 and now that I work with Sun I want to learn all I can.  This one is
not the type to teach me anything.  If I want something I have to tell him
exactly what I want and he'll do it.  Other than that there is no use in
asking him questions like these.
Quote:> --
> Rich Teer

> President,
> Rite Online Inc.

> Voice: +1 (250) 979-1638
> URL: http://www.rite-online.net

 
 
 

Solaris tuning parameter

Post by Jesse Water » Sat, 30 Nov 2002 21:45:18




> > On Wed, 27 Nov 2002 09:01:29 -0500, "Jesse Waters"

> > >I have heard that Solaris will allocate pages of memory to a process
and
> > >does not release them  - ever by default.

> > Then you heard wrong.

> Unless by chance the original poster was thinking about
> the default malloc() arena...freed memory is not returned
> to the system, just the arena within the process address space.

> -Jimmo

ah, so there really could be something to this nebulus "thing"!
 
 
 

Solaris tuning parameter

Post by Casper H.S. Di » Sat, 30 Nov 2002 22:06:33



>I have a process that will consume RAM, swap pages out to swap space and
>never reclaim it.  I wondered if there is a tool that could reclaim that
>because the RSS and SZ continue to grow.  The obvious fix is to stop the
>memory leak but I do not have that option.  Also I'm not knowledgeable
>enough to correct the problem myself but there is a good Sun Admin here that
>could if there is, in fact, a way to do it.

How could these pages be reclaimed?  The kernel cannot know that the
process never needs the memory again.

Looks like the process has a memory leak and it should be fixed.

Casper
--
Expressed in this posting are my opinions.  They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

 
 
 

Solaris tuning parameter

Post by Rich Tee » Sun, 01 Dec 2002 09:17:01



> No argument here, but I have to say that years ago I did some unix admin on
> osf/1 and now that I work with Sun I want to learn all I can.  This one is

Good idea.

Quote:> not the type to teach me anything.  If I want something I have to tell him
> exactly what I want and he'll do it.  Other than that there is no use in
> asking him questions like these.

Ah.  One of those "job security" idiots.  He won't teach you
anything, because he isn't confident enough in his own abilities.

--
Rich Teer

President,
Rite Online Inc.

Voice: +1 (250) 979-1638
URL: http://www.rite-online.net

 
 
 

Solaris tuning parameter

Post by El To » Tue, 03 Dec 2002 07:21:37




> > On Wed, 27 Nov 2002 09:01:29 -0500, "Jesse Waters"

> > >I have heard that Solaris will allocate pages of memory to a process and
> > >does not release them  - ever by default.

> > Then you heard wrong.

> Unless by chance the original poster was thinking about
> the default malloc() arena...freed memory is not returned
> to the system, just the arena within the process address space.

> -Jimmo

Very few Unices or freenixes offer such a feature.  AIX is the only commercial
Unix that allows free()-ed memory to be subsequently given back to the OS by
way of a library call.  It has come to my attention that Linux is still con-
sidering such a capability via an enhanced malloc() (in conjunction with AIX's
SIGDANGER signal).  

Other than AIX through its supplied dislaim() call that does this, all Unices
are able to return memory back to the OS via low-level sbrk() calls; but that
becomes sticky and problematic (which is why its better if the OS would just
supply a wrapper function to take all the guesswork out of it).

 
 
 

Solaris tuning parameter

Post by Kjetil Torgrim Homm » Tue, 03 Dec 2002 08:50:51


[El Toro]:

Quote:

>   Very few Unices or freenixes offer such a feature.  AIX is the
>   only commercial Unix that allows free()-ed memory to be
>   subsequently given back to the OS by way of a library call.  It
>   has come to my attention that Linux is still con- sidering such a
>   capability via an enhanced malloc() (in conjunction with AIX's
>   SIGDANGER signal).

this is a standard feature in the malloc found in GNU libc 2.x, but by
default it is only used on large objects.  you can adjust the
behaviour using mallopt(3c).  the overhead from setting up and tearing
down the memory mapping of /dev/zero is quite great, so you really
don't want to use an mmap() for every allocated object.

Kjetil T.

 
 
 

Solaris tuning parameter

Post by Jimm » Tue, 03 Dec 2002 16:51:26



Quote:> [El Toro]:

> >   Very few Unices or freenixes offer such a feature.  AIX is the
> >   only commercial Unix that allows free()-ed memory to be
> >   subsequently given back to the OS by way of a library call.  It
> >   has come to my attention that Linux is still con- sidering such a
> >   capability via an enhanced malloc() (in conjunction with AIX's
> >   SIGDANGER signal).

> this is a standard feature in the malloc found in GNU libc 2.x, but by
> default it is only used on large objects. ...

Right.  Solaris of mmap based alternatives of malloc too.

-Jimmo

 
 
 

Solaris tuning parameter

Post by El To » Wed, 04 Dec 2002 03:16:51



Quote:> [El Toro]:

> >   Very few Unices or freenixes offer such a feature.  AIX is the
> >   only commercial Unix that allows free()-ed memory to be
> >   subsequently given back to the OS by way of a library call.  It
> >   has come to my attention that Linux is still con- sidering such a
> >   capability via an enhanced malloc() (in conjunction with AIX's
> >   SIGDANGER signal).

> this is a standard feature in the malloc found in GNU libc 2.x, but by
> default it is only used on large objects.  you can adjust the
> behaviour using mallopt(3c).  the overhead from setting up and tearing
> down the memory mapping of /dev/zero is quite great, so you really
> don't want to use an mmap() for every allocated object.

Ah, thanks.  So they've implemented the first half already.  Do you know
what progress they've made on the SIGDANGER implementation, so that the
OOM killer can be more intelligent?  Hmm, I guess that's a bit off-topic,
though.  Please respond in private if you know the answer.
 
 
 

Solaris tuning parameter

Post by El To » Wed, 04 Dec 2002 03:18:40




> > [El Toro]:

> > >   Very few Unices or freenixes offer such a feature.  AIX is the
> > >   only commercial Unix that allows free()-ed memory to be
> > >   subsequently given back to the OS by way of a library call.  It
> > >   has come to my attention that Linux is still con- sidering such a
> > >   capability via an enhanced malloc() (in conjunction with AIX's
> > >   SIGDANGER signal).

> > this is a standard feature in the malloc found in GNU libc 2.x, but by
> > default it is only used on large objects. ...

> Right.  Solaris of mmap based alternatives of malloc too.

> -Jimmo

What versions of Solaris mmap() provides the ability to release memory
back to the OS?  I don't see one mentioned in the Sol8 manpages, nor a
mention of such a feature from mallopt().  Is this a Solaris 9 thing?

Thanks in advance.

 
 
 

Solaris tuning parameter

Post by Jimm » Wed, 04 Dec 2002 03:30:27




> > ...

> > Right.  Solaris of mmap based alternatives of malloc too.

That should have read "Solaris has mmap based alternatives of
malloc too".  Perils of writing with one hand while nursing
an unwell baby with the other :)

Quote:> What versions of Solaris mmap() provides the ability to release memory
> back to the OS?  I don't see one mentioned in the Sol8 manpages, nor a
> mention of such a feature from mallopt().  Is this a Solaris 9 thing?

I was referring to things like libmapmalloc which pull
in a version of malloc() that uses mmap("/dev/zero") as
the source of memory.

free() then leads to munmap() once a whole page has been
freed and this, of course, returns the page to the
system and doesn't keep it allocated to the process
address space.

Cheers
Jimmo

 
 
 

1. Tuning solaris 5.8 system parameters

I'd like to measure cache miss rate and (I'm not sure it's duable on
Solaris 5.8) tlbmiss. Also I appreciate to know the primary/secondary
cache size and cpu type to the report it will be.

How could we customize the computer configuration parameters  such as
cache hit ratio, buffer size,
and so on. Have you a documentation whcih explains how to adjust the
system parameters concerning the memory, hard cache, I/O ......

I appreciate your help. Thanks.

Fran?ois REYGAGNE. Software Architect.
opt[e]way S.A., 2881 route des Crtes, BP308
06906 Sophia Antipolis Cedex, FRANCE
tl: +33 (0)4 92 95 27 01    http://www.opteway.com

2. .forward...., .signature

3. Network tuning parameters in Solaris 2.5.1

4. Help - Slackware - CDROM - TVGA

5. Setting 2.5.1 kernel tuning parameters

6. cirrus5430 bitblt time-out

7. BSD memory tuning parameters on Linux

8. FS: SUN X7004 256MB Memory for U/E 1, 2, 30, 60,

9. tuning IPC parameters under linux

10. help ... tuning kernal parameters

11. TCP congestion stats + performance tuning parameters

12. Oracle tuning parameters on AIX

13. streams buffer fails need tune parameter