Very Computer

I have a memory-intensive real-time algorithm that runs continously.
The memory it uses is malloced and freed (actually 'new'ed and
'delete'd but you get the idea).

Now, after maybe several hours, I find that the "SIZE" of the program
as given by the UNIX utility "top" has increased. The memory being
actually used by the system ("RES" in top) remains within the normal
bounds.  SIZE continues to increase, with no set pattern, while "RES"
remains within range.

I have run the program under purify but even when the SIZE has increased,
there are absolutely no leaks. Not even 4 bytes!

Questions:

1) Why could this (SIZE increases, RES okay) happen? How can we fix it?

2) Hypothesis:  maybe the memory is getting fragmented and so, the program
keeps continously adding to its SIZE in order to get the required memory.
How do I _really return_ allocated memory to the system instead of free-ing
but tagging it for future use by the program?

thanks for any suggestions!

I don't know which flavor of UNIX you are using, but on some, malloc()
does not do garbage collection.  In Solaris 2.x there is the choice
of using libmalloc which is announced as "space efficient, but slower".

Quote:>I have a memory-intensive real-time algorithm that runs continously.
>The memory it uses is malloced and freed (actually 'new'ed and
>'delete'd but you get the idea).

>Now, after maybe several hours, I find that the "SIZE" of the program
>as given by the UNIX utility "top" has increased. The memory being
>actually used by the system ("RES" in top) remains within the normal
>bounds.  SIZE continues to increase, with no set pattern, while "RES"
>remains within range.

>I have run the program under purify but even when the SIZE has increased,
>there are absolutely no leaks. Not even 4 bytes!

>Questions:

>1) Why could this (SIZE increases, RES okay) happen? How can we fix it?

>2) Hypothesis:  maybe the memory is getting fragmented and so, the program
>keeps continously adding to its SIZE in order to get the required memory.
>How do I _really return_ allocated memory to the system instead of free-ing
>but tagging it for future use by the program?

>thanks for any suggestions!

writes:

 Valliappa> 1) Why could this (SIZE increases, RES okay) happen? How
 Valliappa> can we fix it?

That may be an artifact of the malloc() implementation; some systems
seem to be biased towards grabbing more memory from the O/S rather
than reusing space already freed.

 Valliappa> 2) Hypothesis: maybe the memory is getting fragmented and
 Valliappa> so, the program keeps continously adding to its SIZE in
 Valliappa> order to get the required memory.  How do I _really
 Valliappa> return_ allocated memory to the system instead of free-ing
 Valliappa> but tagging it for future use by the program?

Fragmentation can cause the memory used to be much larger than expected,
yes.

As for actually returning free memory to the system, one way to do
that is to provide your own specialised allocator that uses mmap() to
attach blocks of memory to the process (either by specifying MAP_ANON
or by opening /dev/zero and mapping that). These allocations can be
returned to the system simply by unmapping the blocks. You need to map
fairly large chunks and then carve them up for your own use; use
sysconf(_SC_PAGE_SIZE) to get the system page granularity.

--
Andrew.

comp.unix.programmer FAQ: see <URL: http://www.erlenstar.demon.co.uk/unix/>

Of course in the process of writing your own memory allocator you may end up
solving the problem of getting more and more memory from the system making it
unnecessary for you to return memory to the system.

    Bill Lanam

[>...what are you talking about....what's this got to do with C ???????

------- Just because YOU don't know, doesn't mean it does not make
sense. In fact, it makes perfect sense to me, and it has to do with
C (it's a C program that has a problem with it).
--
 Mariusz Zydyk                           http://www.ucalgary.ca/~mszydyk/

You do this by using the global new to allocate a chunk of chars and
then
you can manage this memory on your own.
implement a myClass::new and a myclass::delete to return a free adress
when called (and flag id busy) and to flag it free in case of delete.
You don't need to worry about constructors and destructors.
When someone says "myClass *X(new myClass(1,2,3)", myClass's own new
will only be called to provide the memory.

--
-----------------------------------------


-----------------------------------------

Hi Valliappa,

I worked hard looking for bad use of *alloc() and free() in the code,
then found the problem: the program was a server handling one request at
a time (no fork()'s), something like:

for(;;) {
 newsock = accept(...);
 infile = fdopen(newsock,...);
 outfile = fdopen(newsock,...);
 --- process request ---
 close(newsock);

Quote:}

Cheers, julio