mmap: using it to mmap Queue Data structure

mmap: using it to mmap Queue Data structure

Post by sat.. » Tue, 13 Jul 1999 04:00:00



Hi all,
  I am trying to implement a queue which contains a structure as its member
node. Also to make the queue persistent we are using the mmap unix system
call to map the contenets of memory to a file. so that if the process shuts
down abruptly then we can retriev the contents of the queue by loading the
memory mapped file. In this manner we are trying to build persistence of
objects without using flat files or a database, as this should result in
better performance than normal file I/O. Does anybody see any problems in
the above approach.

In that there are a few issues:-

1. The structure which is to mmapped contains fields which are pointers as
its data members. So the size of the struct is not known before hand. Is
there a way around to allocate exact size of memory depending on the size of
structure instance.

for example the fileds of struct are:
struct QueueNode {
                char * event_name;
                char * description;

Quote:};

then if we use mmap() to allocate memory for five such nodes using
mmap(addr, 5*sizeof(struct QueueNode), PROT_READ|PROT_WRITE, MAP_SHARED,
mtfd, 0);

then the mmap function just allocates memory for the pointers i.e. 4 bytes
for each char*.

When such instances of structs are populated in the mmapped files, this
creates problems. As the actual contents of event_name and description are
not written to the mmapped file. The work around is to make the fileds as
arrays instead of pointers.

Is there a better solution so that we can allocate memory using mmap while
still keeping the fields of the struct as pointers.

2) Second query is related to the deletion of nodes from such mmapped queue.

  For deletion of nodes do we need to use munmap() or free system call is
sufficient.

3) Thirdly do we need to explicitly close the file once the insertion of
nodes is over. In case of abrupt shutdown of the process is there any
guarnatee that the mmapped file would not contain any garbage.

Any clues to the above queries??

Thanks
Satender

 
 
 

mmap: using it to mmap Queue Data structure

Post by John Bens » Thu, 15 Jul 1999 04:00:00


     1.  Using pointers in an mmaped file presupposes that
         you can always map it to the same virtual addresses,
         or that the system will accidently do that for you.
         Not a terribly good idea.  You'd be better to use
         relative pointers, to which the anchor address of the
         map would be added to make memory pointers.

     2.  Any sort of complex data structure will be trash if
         the system stops catistrophically.  Use some sort of
         flat journal from which you can re-create your data
         lump if it gets corrupted.  Frequent use of close()
         can keep the journal disk copy fairly up-to-date.



 
 
 

mmap: using it to mmap Queue Data structure

Post by Philip Bro » Thu, 15 Jul 1999 04:00:00



Quote:>...
>     2.  Any sort of complex data structure will be trash if
>         the system stops catistrophically.  Use some sort of
>         flat journal from which you can re-create your data
>         lump if it gets corrupted.  Frequent use of close()
>         can keep the journal disk copy fairly up-to-date.

there's a particular ioctl that will flush out a mmaped file to disk.
sorry, don't remmeber what it is though.

--
[Trim the no-bots from my address to reply to me by email!]
[ Do NOT email-CC me on posts. Pick one or the other.]
 --------------------------------------------------
The word of the day is mispergitude

 
 
 

mmap: using it to mmap Queue Data structure

Post by Richard L. Hamilt » Fri, 16 Jul 1999 04:00:00





>>...
>>     2.  Any sort of complex data structure will be trash if
>>         the system stops catistrophically.  Use some sort of
>>         flat journal from which you can re-create your data
>>         lump if it gets corrupted.  Frequent use of close()
>>         can keep the journal disk copy fairly up-to-date.

> there's a particular ioctl that will flush out a mmaped file to disk.
> sorry, don't remmeber what it is though.

I don't know about an ioctl().  Are you thinking of msync(3c), or
perhaps the underlying memcntl(2)?

--
ftp> get |fortune
377 I/O error: smart remark generator failed

Bogonics: the primary language inside the Beltway


 
 
 

mmap: using it to mmap Queue Data structure

Post by Philip Bro » Fri, 16 Jul 1999 04:00:00



Quote:>I don't know about an ioctl().  Are you thinking of msync(3c), or
>perhaps the underlying memcntl(2)?

probably msync()

--
[Trim the no-bots from my address to reply to me by email!]
[ Do NOT email-CC me on posts. Pick one or the other.]
 --------------------------------------------------
The word of the day is mispergitude

 
 
 

1. mmap diferrent data structures

  Hi guys,

      I would like to save a hash and a trie on a memory mapped
file and save it on disk. But I don't like the idea that
I can access a memory mapped file as if it were an array.

  The hash uses buckets to solve collisons.

Please Help
Thank You greatly
Hany

2. Netscape mouse locks up under Solaris 2.5

3. FIFO queue using MMAP? Help Please?

4. DR3 on IMac?

5. ALSA + mmap or OSS emulation + mmap producing stutering sound

6. winmodems

7. MMAP cache problem in 2.4.x, Re: [parisc-linux] unreliable mmap (fwd)

8. Solaris 2.4 I/O and interactive desktop performance

9. Q: mmap() files in linux vs.data in memory

10. mmap a data file on x86 and sparc

11. Code/data in mmap'ped address space..?

12. how to share data with mmap()?