Event service for linux (monitoring a file for being changed)

Event service for linux (monitoring a file for being changed)

Post by seliou » Mon, 24 May 1999 04:00:00



Hi,

does Linux offer a way to write a event driven app, that get's notified
that a certain file changed ??

I mean, not by polling time-stamps and stuff. It should not cause
overhead..

The best should be to register a call-back function to a file-name, and
when the file is changed the call-back function is called !!

Who knows ??

 
 
 

Event service for linux (monitoring a file for being changed)

Post by Benedikt Hube » Tue, 25 May 1999 04:00:00


Hi

I don't know how the "less" utility accomplishes this behaviour
but you may look at its sources

Beni


> Hi,

> does Linux offer a way to write a event driven app, that get's notified
> that a certain file changed ??

> I mean, not by polling time-stamps and stuff. It should not cause
> overhead..

> The best should be to register a call-back function to a file-name, and
> when the file is changed the call-back function is called !!

> Who knows ??


 
 
 

Event service for linux (monitoring a file for being changed)

Post by Kevin Burto » Tue, 25 May 1999 04:00:00


Let us know how you do with this.  I have been wondering how to do this
myself.

Under Win32 it is easy.  Call the API which watches files and it returns
a pointer to an object wich you call WaitForSingleObject() on.  I would
love to find out this for Linux.


> Hi

> I don't know how the "less" utility accomplishes this behaviour
> but you may look at its sources

> Beni


> > Hi,

> > does Linux offer a way to write a event driven app, that get's notified
> > that a certain file changed ??

> > I mean, not by polling time-stamps and stuff. It should not cause
> > overhead..

> > The best should be to register a call-back function to a file-name, and
> > when the file is changed the call-back function is called !!

> > Who knows ??

--
Kevin A. Burton
Internet Guy
 
 
 

Event service for linux (monitoring a file for being changed)

Post by luis malheir » Tue, 25 May 1999 04:00:00


Isn't the "select" system call what you are looking for?

Try a "man select".

Cheers,
Luis

--
    Luis Malheiro,
    ALCATEL TELECOM

 
 
 

Event service for linux (monitoring a file for being changed)

Post by luis malheir » Tue, 25 May 1999 04:00:00


Ooops! Sorry, you want a event driven thing...
Can you use pthreads + select?

Cheers,
Luis


> Isn't the "select" system call what you are looking for?

> Try a "man select".

 
 
 

Event service for linux (monitoring a file for being changed)

Post by seliou » Tue, 25 May 1999 04:00:00


Quote:> Under Win32 it is easy.  Call the API which watches files and it returns
> a pointer to an object wich you call WaitForSingleObject() on.  I would
> love to find out this for Linux.

 I think applications like miniterm (in the LPG samples) have such a method to
watch serial data...

But I'm afraid it might require a addition to the VFS layer !!

Still, it's mandatory for linux to have both config files for each app, and a
general tool to configure them...

 
 
 

Event service for linux (monitoring a file for being changed)

Post by G. Sumner Haye » Tue, 25 May 1999 04:00:00



> Hi,

> does Linux offer a way to write a event driven app, that get's notified
> that a certain file changed ??

You should probably read the comp.unix.programmer FAQ.  Question 2.6:
: 2.6 How do I find out if a file has been updated by another
: process?

: This is close to being a Frequently Unanswered Question, because
people
: asking it are often looking for some notification from the system when
a
: file or directory is changed, and there is no portable way of getting
: this. (IRIX has a non-standard facility for monitoring file accesses,
: but I've never heard of it being available in any other flavour.)
:
: In general, the best you can do is to use fstat() on the file. (Note:
: the overhead on fstat() is quite low, usually much lower than the
: overhead of stat().) By watching the mtime and ctime of the file, you
: can detect when it is modified, or deleted/linked/renamed. This is a
bit
: kludgy, so you might want to rethink why you want to do it.

Quote:> I mean, not by polling time-stamps and stuff. It should not cause
> overhead..

Can't happen AFAIK.  I'd love to see some sort of Linux-specific
interface to do this, but I don't believe there is one.  Let me know
if I'm wrong.

If you're only worried about certain kinds of fds, then select(),
poll(),
and I/O completion notification via realtime signals (2.2 kernels w/
glibc
2.1 only) might be of some interest.  They won't help you find out when
other processes have written to a regular file, though.  You're stuck
with a thread or process that polls with fstat() periodically and
notifies
the main thread over a socket or some such when there's an interesting
change.  You might be able to reduce overhead if you have a ton of
programs interested in this by writing a modification daemon that only
has to be run once on the system (and communicates over Unix-domain
sockets or sends queued signals with some sort of identifier).

If you tell us what you're trying to accomplish there might be an
alternative way of doing this.

Let me know if you come up with a better solution.

--Sumner

 
 
 

Event service for linux (monitoring a file for being changed)

Post by Kaz Kylhe » Wed, 26 May 1999 04:00:00




Quote:>Let us know how you do with this.  I have been wondering how to do this
>myself.

>Under Win32 it is easy.  Call the API which watches files and it returns
>a pointer to an object wich you call WaitForSingleObject() on.  I would
>love to find out this for Linux.

Tell that to the guys who wrote Windows NT explorer and other shit Windoze
programs.  You are always having to hit F5 to refresh their file lists!
 
 
 

Event service for linux (monitoring a file for being changed)

Post by Kaz Kylhe » Wed, 26 May 1999 04:00:00



>Ooops! Sorry, you want a event driven thing...
>Can you use pthreads + select?

Select always returns positive on a regular file. It cannot be used to detect
modifications. Polling timestamps periodically is the only way.
 
 
 

Event service for linux (monitoring a file for being changed)

Post by Gues » Fri, 28 May 1999 04:00:00



> >Let us know how you do with this.  I have been wondering how to do this
> >myself.

> >Under Win32 it is easy.  Call the API which watches files and it returns
> >a pointer to an object wich you call WaitForSingleObject() on.  I would
> >love to find out this for Linux.

> Tell that to the guys who wrote Windows NT explorer and other shit Windoze
> programs.  You are always having to hit F5 to refresh their file lists!

Because the NT implementation is buggy and very unfriendly, i.e. if you want to
know whenever any file within some directory tree is modified, you get an
event that some file within that tree has been modified, but no indication of
which file was modified.  Also, the NT notification is known to 'drop' some
changes every now and then, especially when made through network connections.

Now, all the above aside: having such a service in Linux (one that actually
would work) would be a nice thing.

 
 
 

Event service for linux (monitoring a file for being changed)

Post by Seliou » Fri, 28 May 1999 04:00:00


I have build a prototype of a administration program. It is a simple
process, that allows clients (local or remote) to read out configuration
data (like /etc/hosts, /etc/smb.conf, etc.)

Anyway, because most linux/unix people don't like the idea of a central
registry store, my application has to interface between configuration files
(ascii) and comprihensible data (structures).

This works GREAT, unless someone or something (application) would change the
configuration directly, in the file.

Ofcourse this can be monitored by timestamps, but I want my service to
process changes realtime !! It is very important to make this program work,
because for instance:

Let's say you select usergroup 600 and it's users to be synchronised between
several linux systems. When you're 40 year old admin (who still uses
adduser, or maybe even his own script) adds a user, this user must also be
exported to other systems..

My application allows this, but I DON"T accept it that the application would
poll this.

Only when:

- a client request reaches my LMA CPU may be used
- someone changes a configuration file directly CPU may be used

Furthermore, my application may not be spending CPU power, because it would
simply add up to piles of programs that will make linux as slow as other
OS's.

 
 
 

Event service for linux (monitoring a file for being changed)

Post by Emile van berge » Fri, 28 May 1999 04:00:00



>I have build a prototype of a administration program. It is a simple
>process, that allows clients (local or remote) to read out configuration
>data (like /etc/hosts, /etc/smb.conf, etc.)

>Anyway, because most linux/unix people don't like the idea of a central
>registry store, my application has to interface between configuration files
>(ascii) and comprihensible data (structures).

Ah. So I gather you have built your own parser to every configuration
file supported. Well, let's hope the Samba folks don't change smb.conf's
syntax then ;-)

BTW, the more unixy way of doing such a thing is NIS/NIS+. You won't
need to sync /etc/passwd realtime across machines, as a central database
(containing plain ascii btw.) is queried. You could extend it to support
other files as well.

Quote:>This works GREAT, unless someone or something (application) would change the
>configuration directly, in the file.

So you permit every luser on your system to change system configuration
data. Hmmm. 'nuf said.

Quote:>Ofcourse this can be monitored by timestamps, but I want my service to
>process changes realtime !! It is very important to make this program work,
>because for instance:

>Let's say you select usergroup 600 and it's users to be synchronised between
>several linux systems. When you're 40 year old admin (who still uses
>adduser, or maybe even his own script) adds a user, this user must also be
>exported to other systems..

Use NIS. Problem solved. ;-)

Quote:>My application allows this, but I DON"T accept it that the application would
>poll this.

Bah, if you _don't accept_ it, stop using Linux (or any Unix, for that
matter!). Alternatively, the source is there, do with it as you please.

Quote:>Only when:

>- a client request reaches my LMA CPU may be used
>- someone changes a configuration file directly CPU may be used

>Furthermore, my application may not be spending CPU power, because it would
>simply add up to piles of programs that will make linux as slow as other
>OS's.

Well, if you're that persisting, I guess you should learn Linux's VFS
system and enhance select(). If you don't feel like doing that, I'd
suggest lowering the yelling a bit if you want to get something done by
the kernel guys and gals...

Just my 2c.

--

M.vr.gr. / Best regards,


This e-mail message is 100% electronically degradeable and produced
on a GNU/Linux system.

 
 
 

Event service for linux (monitoring a file for being changed)

Post by Bj?rn Rees » Fri, 28 May 1999 04:00:00



> My application allows this, but I DON"T accept it that the application would
> poll this.

At the moment you don't really have an option.

While such an event service certainly is a nice idea, I am not sure
that it is a good idea. Consider that this event service has to
examine which applications wants a notification every time a file
is modified. This can amount to quite some overhead when you modify
many files (for example during compilation of a program.)

Why don't you simply implement periodic polling as see if it poses
a problem in practise? (which I doubt -- both concerning CPU usage
and real-time responsitivity.)

 
 
 

Event service for linux (monitoring a file for being changed)

Post by Jeffrey L Straszhei » Fri, 28 May 1999 04:00:00


<snip>

Quote:> > Tell that to the guys who wrote Windows NT explorer and other shit Windoze
> > programs.  You are always having to hit F5 to refresh their file lists!

<snip>

Quote:> Now, all the above aside: having such a service in Linux (one that actually
> would work) would be a nice thing.

Very true. I get the feeling by reading these groups that some folks
around here haven't bothered reading what the Win32 kernel specs are.
This is not a compliment of how Win32 is implemented: NT is crap, Win95/8
is worse, and putting the GUI in the kernel was the worse decision
ever, but all that being said, the way Win32 treats kernel ojects is
far better than how unix treats its. Currently, I'm doing a project to
provide an application level sockets library for the Eiffel language.
The control structure basically wraps the select() command to dispatch
I/O handlers. Pretty basic stuff. However, say that in addition to
responding to pending I/0, the developer would like her application
to respond to a pending semephore. Too bad in unix. SVR5 IPC objects don't
have file descriptors. I suppose there might be some solution for this
(and if you know one feel free to post), but in Win32 it's easy. The
common interface for kernel objects in Win32 was a great idea and we
should recognize that. So what am I saying? Simply that it is a good
idea to stop and recognize those parts of something else that are well
done even when, as a whole, you should reject it.

{Please note that when I describe some aspect of Win32 as being "well
done", my compliment is limited to what I've found in books, not to
what I've found in actually shipping code from Microsoft.}

-- Jeffrey Straszheim          
-- Systems Engineer, Programmer
-- http://www.shadow.net/~stimuli
-- stimuli AT shadow DOT net

 
 
 

Event service for linux (monitoring a file for being changed)

Post by G. Sumner Haye » Sat, 29 May 1999 04:00:00



> the way Win32 treats kernel ojects is
> far better than how unix treats its. Currently, I'm doing a project to
> provide an application level sockets library for the Eiffel language.
> The control structure basically wraps the select() command to dispatch
> I/O handlers. Pretty basic stuff. However, say that in addition to
> responding to pending I/0, the developer would like her application
> to respond to a pending semephore. Too bad in unix. SVR5 IPC objects
> don't have file descriptors.

SysV IPC doesn't fit the Unix programming model.  That's well known.
The Unix way would be to use sockets for synchronization and mmap for
shm.  POSIX shm is much nicer than SysV.

On the subject, how about select() on a directory returning when the
directory changes?  I know that idea's been thrown around before, but
I don't recall the conclusion of the conversation.  It's only a
partial solution.  I don't think the select() semantics on files are
really useful atm, but I'm not sure how to fix them.  Maybe select
on an fd in append mode would return only if you're not at EOF; that
way tail -f would be able to select on file appends.  I have no idea
what that would break conceptually.  I'm sure that it'd break a bunch
of stuff that relies on current semantics.

Quote:> The common interface for kernel objects in Win32 was a great idea
> and we should recognize that. So what am I saying? Simply that it is
> a good idea to stop and recognize those parts of something else that
> are well done even when, as a whole, you should reject it.

Ugh.  Don't even get me started on WaitForMultipleObjectsEx.  We did
take TransmitFile if I'm not mistaken (possibly via HP/UX).  But
Linux does it right and doesn't muck up the API to work around systemic
problems (unlike HP/UX and NT).

--Sumner

 
 
 

1. Event monitoring service for file changes

Hi

I need to know when a file has changed, under a given directory root,
but I don't want to go down the polling road as there could
potentially be millions of files and this would suck resource.  I have
seen many questions such as this, but I've never seen it answered.  Is
it possible for this sort of event notification for files under Linux?

Any ideas would be greatly appreciated.

Thanks in advance

Matt

2. Help! A Ethernet Card Question.

3. Looking for a freely available Event Service for UNIX

4. Newbie help please

5. Free Classified Ads: Job listings , Real Estates, Events, Services, Personal and more!

6. Linux and Sun A1000 Storage Array

7. Kernel Event Notification Service

8. DHCP client (dhclient) question

9. pcmcia: don't inform "driver services" of cardbus-related events

10. How to monitor changes made on a file/dir?

11. File change monitor for Unix?

12. Change CRT monitor to LCD monitor

13. Efficiently Monitor for changes in the file system?