Waking up After a File Changes.

Waking up After a File Changes.

Post by Srir » Tue, 01 Jul 2003 15:51:06



Sir,
  I have 10 files which are accessed by a process. The process Changes
the contents of these files randomly. I want to have one more process
that should print a message whenever a file has been changed. The Two
processes are totally different and have no relation whatsoever.

How do i implement it very efficiently.

Thanks,
Sriram

 
 
 

Waking up After a File Changes.

Post by Michael Kerris » Tue, 01 Jul 2003 17:22:32



Quote:>Sir,
>  I have 10 files which are accessed by a process. The process Changes
>the contents of these files randomly. I want to have one more process
>that should print a message whenever a file has been changed. The Two
>processes are totally different and have no relation whatsoever.

>How do i implement it very efficiently.

There is no standard way of doing this.  A simple, inefficient,
portable way of doing it w=ould be to have the monitoring process poll
the files and/or directories with stat(2).

However, various implementatations have non-standard tools / APIs to
do this (e.g., Linux has dnotify and a FAM implementation is
available), so it might help to say which platform(s) you are aiming
at.

Cheers

Michael

 
 
 

Waking up After a File Changes.

Post by Lew Pitch » Tue, 01 Jul 2003 21:36:43





>>Sir,
>>  I have 10 files which are accessed by a process. The process Changes
>>the contents of these files randomly. I want to have one more process
>>that should print a message whenever a file has been changed. The Two
>>processes are totally different and have no relation whatsoever.

>>How do i implement it very efficiently.

>There is no standard way of doing this.  A simple, inefficient,
>portable way of doing it w=ould be to have the monitoring process poll
>the files and/or directories with stat(2).

The OP /could/ use named pipes, Unix sockets, TCP/IP sockets, shared memory, or
other Unix IPC tools to signal between processes. The general technique would
have the source process write the file-name to the IPC tool (i.e. write to a
named pipe or Unix socket), and the target process perform a blocking read
against the IPC tool. The blocking read would unblock when the source process
does it's write, thus providing the target process with it's data in a timely,
unpolled, manner. The target process would block on the next unsatisfied read,
awaiting the source process to tell it another filename.

Shared memory and semiphores would work in a similar manner; the two processes
would co-operate in a producer/consumer fashion. The semiphore could be a mutex
(with the shared memory carrying both a 'filename-is-valid' flag, and the
filename itself), or as an event counter (with the shared memory carrying a
filename queue). The procedure would be the same as the blocking read, with the
consumer process awaiting on the semiphore, and reading the filename from shared
memory, while the producer populates the shared memory with the filename, and
sets the semiphore accordingly.

--
Lew Pitcher
IT Consultant, Enterprise Technology Solutions
Toronto Dominion Bank Financial Group

(Opinions expressed are my own, not my employers')

 
 
 

Waking up After a File Changes.

Post by Chuck Dillo » Tue, 01 Jul 2003 22:53:14



> Sir,
>   I have 10 files which are accessed by a process. The process Changes
> the contents of these files randomly. I want to have one more process
> that should print a message whenever a file has been changed. The Two
> processes are totally different and have no relation whatsoever.

> How do i implement it very efficiently.

> Thanks,
> Sriram

In addition to what's already been suggested the monitoring process
could monitor the system calls of the process that changes the files IF
it has permission to do so that is.  IOW, the monitor could use popen()
to fire off a truss or strace process watching I/O calls of the writer
program.  The monitor would scan the output of truss/strace and make
decisions based on what's seen.  For example, if the writing process
opens, writes and closes you might be able to simply monitor open
calls.  If the writing process keeps open handles to the files and
writes data to them randomly you would have to monitor all opens,
closes and writes to identify when something is written to a file of
interest.

The problems with this include:
        a) Non-portable.
        b) Heuristics based.  You have to figure out how to identify the
conditions you are looking for from the truss/strace output.
        c) Fragile... If complex heuristics are involved, when the writer is
updated/changed your heuristics will likely have to change.
        d) If you have to monitor all writes and the writer does a lot of
extraneous writing you're better off polling with stat/fstat.

-- ced

--
Chuck Dillon
Senior Software Engineer
NimbleGen Systems Inc.

 
 
 

Waking up After a File Changes.

Post by William Aher » Thu, 03 Jul 2003 02:41:22




>>Sir,
>>  I have 10 files which are accessed by a process. The process Changes
>>the contents of these files randomly. I want to have one more process
>>that should print a message whenever a file has been changed. The Two
>>processes are totally different and have no relation whatsoever.

>>How do i implement it very efficiently.

> There is no standard way of doing this.  A simple, inefficient,
> portable way of doing it w=ould be to have the monitoring process poll
> the files and/or directories with stat(2).

> However, various implementatations have non-standard tools / APIs to
> do this (e.g., Linux has dnotify and a FAM implementation is
> available), so it might help to say which platform(s) you are aiming
> at.

Linux has dnotify, however you can only register a listener for events
through a directory file descriptor. So, you can know when _some_file has
been modified in a directory, but you have to manually test all of the files
in that directory to deduce which one(s) changed. I'd stay away from FAM,
because i don't see it widely available (unless you are admining the machine
yourself, can patch the kernel and also run the daemon). dnotify should
suffice and is automatically available on all recent Linux kernels.

Free/Open/NetBSD has kqueue, which is much nicer (for most cases at least).
With kqueue you can wait for events on any file descriptor (w/ the caveat
that file modifications do not themeslves trigger an event for any
parent directory).

There are two shell utilities you can use. For dnotify:

        [http://www.student.lu.se/~nbi98oli/dnotify.html]

For kqueue:

        [http://www.25thandclement.com/~william/projects/watch.html]

I'm working on a portable kqueue implementation which would try to mimic the
BSD kqueue semantics. Initially it is meant for Linux, and then possibly on
Solaris, et al.

        [http://www.25thandclement.com/~william/projects/libkqueue.html]

Watch is useable (w/ a quirk regarding recycled inodes, which shouldn't
worry you if you're only interested in modifications, not file creation and
deletion). libkqueue is still being mocked up.

- Bill

 
 
 

Waking up After a File Changes.

Post by Michael Bruschkewit » Wed, 09 Jul 2003 00:03:25




Quote:> Sir,

I would really like to know what is the meaning of this greeting in
newsgroups...

Quote:>   I have 10 files which are accessed by a process. The process Changes
> the contents of these files randomly. I want to have one more process
> that should print a message whenever a file has been changed. The Two
> processes are totally different and have no relation whatsoever.

At least proc 2 should know the name or pid of proc 1.
Do you know the file names?

Quote:

> How do i implement it very efficiently.

What means: efficiently? Less work? Less latency? Less ressources?
From your requirement: print a mess - I guess: work less :)
 check files every 5 sec should enough.

Try loop (sleep, stat)  on these files. See struct stat member st_mtime.
Just an idea.

see also /proc/<pid>/fd (man -s4 proc, section fd)

Possible idea: mmap()

-mb