File I/O improvements using file locking and/or mem-mapping???

File I/O improvements using file locking and/or mem-mapping???

Post by Dann » Wed, 29 Jan 2003 02:34:39



My program reads (open, read, close) the contents of a file from a set
of about 25 files continuously Each time it's a different file. Is
there a way to improve the performance of the file reads, from
"outside" the program, by some form of file locking in memory
before/when the program starts?
 
 
 

File I/O improvements using file locking and/or mem-mapping???

Post by Andrew Giert » Thu, 30 Jan 2003 08:48:15


 Danny> My program reads (open, read, close) the contents of a file
 Danny> from a set of about 25 files continuously Each time it's a
 Danny> different file. Is there a way to improve the performance of
 Danny> the file reads, from "outside" the program, by some form of
 Danny> file locking in memory before/when the program starts?

you don't specify what system you're using.

On some systems, you can mmap() and then mlock() the files from a
privileged process (depending on the system, you may also have to read
from each page of the file to force them to be faulted in the first
time). This is only a win if the files are small enough to be locked
in memory, _and_ the other memory usage (and the system's VM tuning)
is such that the file pages would otherwise have been freed for other
uses in between reads from your program.

--
Andrew.

comp.unix.programmer FAQ: see <URL: http://www.erlenstar.demon.co.uk/unix/>

 
 
 

File I/O improvements using file locking and/or mem-mapping???

Post by ahowar » Thu, 30 Jan 2003 15:26:52



Quote:> My program reads (open, read, close) the contents of a file from a set
> of about 25 files continuously Each time it's a different file. Is
> there a way to improve the performance of the file reads, from
> "outside" the program, by some form of file locking in memory
> before/when the program starts?

place them in a shared memory segment and use semaphores to coordinate
reads/write.  i'm assuming the files only change sometimes and so only need
updating in memory every so often.  you can map the files in with a perl
script - but read them from any language with SysVIPC bindings.

-a

--

 ====================================
 | Ara Howard
 | NOAA Forecast Systems Laboratory
 | Information and Technology Services
 | Data Systems Group
 | R/FST 325 Broadway
 | Boulder, CO 80305-3328

 | Phone:  303-497-7238
 | Fax:    303-497-7259
 ====================================

 
 
 

1. File locking text file in linux using C

Hi,

I'm relatively new to programming, I'm learning C.
I'm working with text files for CGI currently, and
I heard about file locking, it sounds like something
that I could use.  Does anyone have a snippet
code of this function? I saw an example for it
in a dos based program, but the library in unix
does not exist (or is different, I don't know what),
so I got a an error.  I have no idea how to write it
from scractch so any help will be greatly
appreciated, thank you!

-neuron

2. compaq proliant 1600

3. Any way to flush to disk mem-mapped file writes?

4. Linux on apple G4

5. Q. wipe behaves different (mem-mapped file)

6. Problem with D-Link DE-528CT w/Charter@home

7. mmap/file locking vs. shared mem/semaphores

8. CMedia CMI8738 soundcard

9. File corruption accessing files on a large-file-enabled fs using RM-Cobol

10. writing to a mem-mapped device using modules

11. Need System Call to return %MEM (kernel mem) used by process

12. vi question - mapping a function key in .exrc file that uses a yanked word

13. How to generate the map file when I am using CC