is mmap memory available after closing process?

is mmap memory available after closing process?

Post by Stefan Lisch » Fri, 09 Nov 2001 23:28:17



Hi,

I read hundreds of postings to the subject "mmap or shm*" and now i
only have last things, that are not really clear to me.

1. is it right: when i mmap a file in a process with MAP_SHARED and
another process is started slightly after the first and mmap's the
same file, the second process gets the data "already read" bye the
first process from memory.

2. is it right: They both use the same physical memory.

3. is it right: that when process 2 wants some data from that mmap'ed
file which hasn't been used by process 1, this data is read from disk
and placed into mmap'ed memory, so process 1 can also use it.

4. if 3 is right, mmap doesn't read the whole file on start, at which
size does mmap read data into memory (like 4k blocks)

5. when the two processes are closed and a 3rd process starts half a
second after 1,2 closed, #3 has to read the file again, cause "when
the last process which mmap's the same file is closing, the shared
memory is removed"

6. The better solution for scenario #5 is a forking process who get a
shared memory by shm_get, read the whole file into it and fork()'s on
demand. disconnecting shared memory is made by the parent process by
his own "caching" algorithms (in case #5 i hold the shared memory till
10 seconds after last process is using it, so process 3 could benefit
from "my caching")

7. shared memory with shm_get isn't slower than mmap

i hope i'm at least right in the half of these thesis

thanx Stefan Lischke

 
 
 

is mmap memory available after closing process?

Post by Andrew Giert » Sat, 10 Nov 2001 00:27:49


 Stefan> Hi,
 Stefan> I read hundreds of postings to the subject "mmap or shm*" and
 Stefan> now i only have last things, that are not really clear to me.

 Stefan> 1. is it right: when i mmap a file in a process with
 Stefan> MAP_SHARED and another process is started slightly after the
 Stefan> first and mmap's the same file, the second process gets the
 Stefan> data "already read" bye the first process from memory.

yes, on any decent implementation

 Stefan> 2. is it right: They both use the same physical memory.

yes, on any decent implementation

 Stefan> 3. is it right: that when process 2 wants some data from that
 Stefan> mmap'ed file which hasn't been used by process 1, this data
 Stefan> is read from disk and placed into mmap'ed memory, so process
 Stefan> 1 can also use it.

yes, on any decent implementation

 Stefan> 4. if 3 is right, mmap doesn't read the whole file on start,
 Stefan> at which size does mmap read data into memory (like 4k
 Stefan> blocks)

The way the file is actually read will depend on the implementation,
and also perhaps on any advice given via madvise(). For example, the
implementation may detect sequential reads and start reading in pages
ahead of the ones actually necessary; or the filesystem block size may
be larger than the page size.

 Stefan> 5. when the two processes are closed and a 3rd process starts
 Stefan> half a second after 1,2 closed, #3 has to read the file
 Stefan> again, cause "when the last process which mmap's the same
 Stefan> file is closing, the shared memory is removed"

This is hopefully _not_ true on a decent implementation. In
particular, systems with unified VM/buffer-cache will generally _not_
discard the file data from RAM after all the mappings are removed, and
a subsequent process that maps the same file will not need to go back
to disk to re-read.

 Stefan> 6. The better solution for scenario #5 is a forking process
 Stefan> who get a shared memory by shm_get, read the whole file into
 Stefan> it and fork()'s on demand. disconnecting shared memory is
 Stefan> made by the parent process by his own "caching" algorithms
 Stefan> (in case #5 i hold the shared memory till 10 seconds after
 Stefan> last process is using it, so process 3 could benefit from "my
 Stefan> caching")

this is likely to be unnecessary overhead

 Stefan> 7. shared memory with shm_get isn't slower than mmap

it will normally be about the same (though I have run into architectures
where it is faster, if the same parts of the mapping are being actively
used by several processes, but I would regard that as fairly unusual)

--
Andrew.

comp.unix.programmer FAQ: see <URL: http://www.erlenstar.demon.co.uk/unix/>
                           or <URL: http://www.whitefang.com/unix/>

 
 
 

is mmap memory available after closing process?

Post by Casper H.S. Dik - Network Security Engine » Fri, 09 Nov 2001 23:43:13


[[ PLEASE DON'T SEND ME EMAIL COPIES OF POSTINGS ]]


>1. is it right: when i mmap a file in a process with MAP_SHARED and
>another process is started slightly after the first and mmap's the
>same file, the second process gets the data "already read" bye the
>first process from memory.

If the first process looked at it; not if it was just mmaped but
not accessed.

Quote:>2. is it right: They both use the same physical memory.

Correct.

Quote:>3. is it right: that when process 2 wants some data from that mmap'ed
>file which hasn't been used by process 1, this data is read from disk
>and placed into mmap'ed memory, so process 1 can also use it.

Correct.

Quote:>4. if 3 is right, mmap doesn't read the whole file on start, at which
>size does mmap read data into memory (like 4k blocks)

At the system pagesize or disk system cluster size (multiple pages)
"implementation dependent".

The minimum granularity is system pagesize.

Quote:>5. when the two processes are closed and a 3rd process starts half a
>second after 1,2 closed, #3 has to read the file again, cause "when
>the last process which mmap's the same file is closing, the shared
>memory is removed"

No, because the memory may longer on in the cache after last close;
pages may be put on the free memory list but can often be recovered.

Quote:>6. The better solution for scenario #5 is a forking process who get a
>shared memory by shm_get, read the whole file into it and fork()'s on
>demand. disconnecting shared memory is made by the parent process by
>his own "caching" algorithms (in case #5 i hold the shared memory till
>10 seconds after last process is using it, so process 3 could benefit
>from "my caching")

It's a more elborate solution and it's doubtful whether it's better.

Quote:>7. shared memory with shm_get isn't slower than mmap

Depending on the implementation it can be faster, share more resources.

Can be slower too.

Casper
--
Expressed in this posting are my opinions.  They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

 
 
 

1. walk the process memory (mmap)

Hi

I'd like to implement a memory mapper that does the following:

1. use mmap() to map a file into memory (private copy)
2. read (most) and write (few) to the memory
3. on something like a "commit" operation: scan all pages for modification

My first idea was using a SIGSEGV handler on memory that initially is
read-only and only after the first write access gets opened for write
through mprotect() in the signal handler (plus some bookkeeping to know
which pages where modified afterwards).

However my "research" on the web seems to indicate that there are some
portability and reliability issues with the siginfo fault address in the
SIGSEGV handler.

I would also like to avoid the overhead of SIGSEGV handling during the
transaction and do the modification handling in one sweep afterwards (and
only in case of commit).

On Win32 I can use the VirtualQueryEx API to learn about my virtual memory.
Is there no such thing on U*X?

Thanx for all help,
Mark

2. TEAC QUAD speed CDROM

3. question about process memory usage and process memory limits

4. Would like some comments :)

5. Total available free memory for new processes ?

6. Another satisfied customer

7. Setting Max Memory available for a process in Solaris 2.6

8. problem of traceroute to virtual IP on Solaris2.4

9. mmap, virtual memory, physical memory

10. available process memory

11. Limit memory available for user processes

12. Available process memory on OSF 3.2

13. mmap() of file, then close and unlink