Serializing data & device access across file ops and interrupt handler

Serializing data & device access across file ops and interrupt handler

Post by Floy » Tue, 30 Apr 2002 07:12:12



I am writing a device driver that must serialize access to driver data and
registers on the device across file operations (open, close, ioctl, etc) and
interrupt handling operations.

I use a sempahore [ down_interruptable() and up() ] when enterring and leaving
the file operations but, as I have discovered, these cannot be used in the
interrupt handler.

For the time being I have added the use of spin locks. I must still use the
sempahore for file operations because spin_lock_irqsave  doesn't seem to do
anything else but disable interrupts one a single processor system.

Some of the file operations are rather long and I would prefer not to have the
CPU's interrupts diabled all the time.  Also some of the interrupt handler's
tasks are involved and I would like to delegate them to a tasklet or BH
mechanism.  However, from what I understand, those cannot use sempahores
either.  Is this true?  Or can a BH use semaphores?

Here is what I would like to do but I don't know if Linux supports it:

I would like to use something like a kernel thread - a task I can start up
when my module loads and kill when it unloads.  The task would perform all of
the interrupt handler functions but not at the interrupt level so it could use
the sempahore to sleep and protect the data it needs to access.  The task
would start off sleeping in wait_event().  The actual interrupt level handler
function would then wake it up.  Upon waking the task would obtain the
semaphore, perform the actions to service the interrupt, release the semaphone
then go back to sleep with wait_event() unless the interrupt level handler has
incremented an interrupt count signifying there may be more work to do.
The driver's unload routine would also wake up the interrupt service thread
after setting a flag that says it should exit.

I thought of using a user-mode daemon that would use a special ioctl command
that does the interrupt service loop.  But I would prefer to keep everything
within the driver code and managed by the driver.  I don't like the prospect
of having to maintain the separate daemon - checking to see if it dies for
some reason or even having to provide a mechanism to stop and restart it if
the driver module is unloaded and reloaded.

Can a tasklet be used in such a way?  Is it possible to start a tasklet in the
module initialization function and can that tasklet use semaphores?
Is there a kernel thread mechanism in Linux?

 
 
 

Serializing data & device access across file ops and interrupt handler

Post by Kasper Dupon » Tue, 30 Apr 2002 16:31:57



> I am writing a device driver that must serialize access to driver data and
> registers on the device across file operations (open, close, ioctl, etc) and
> interrupt handling operations.

> I use a sempahore [ down_interruptable() and up() ] when enterring and leaving
> the file operations but, as I have discovered, these cannot be used in the
> interrupt handler.

You can use down_trylock() and up() from an interrupt handler.
But down() and down_interruptible() is not going to work. (If
the semaphore can be aquired immediately all three down variants
will work from an interrupt handler. If not down_trylock() will
just tell you that it wasn't possible while down_interruptible()
and down() will crash your system. Retrying down_trylock() will
also crash your system.)

Quote:

> For the time being I have added the use of spin locks. I must still use the
> sempahore for file operations because spin_lock_irqsave  doesn't seem to do
> anything else but disable interrupts one a single processor system.

It is true that on current Linux versions spinlocks are no-ops on
single processor systems. That will change in future versions.
But don't think too much about different spinlock implementations,
the important fact is that you may not sleep while holding a
spinlock. While holding a spinlock calling any function that could
sleep is going to introduce races on single processor systems and
deadlocks on multiprocessor systems.

Quote:

> Some of the file operations are rather long and I would prefer not to have the
> CPU's interrupts diabled all the time.

Of course, don't keep interrupts disabled longer time than
absolutely necesarry. But it is even worse than that. You may not
call any function that could sleep. So you really need a semaphore.

Quote:> Also some of the interrupt handler's
> tasks are involved and I would like to delegate them to a tasklet or BH
> mechanism.  However, from what I understand, those cannot use sempahores
> either.  Is this true?  Or can a BH use semaphores?

AFAIR you may not sleep in a BH. Both down_interruptible() and
down() will sleep.

Quote:

> Here is what I would like to do but I don't know if Linux supports it:

> I would like to use something like a kernel thread - a task I can start up
> when my module loads and kill when it unloads.  The task would perform all of
> the interrupt handler functions but not at the interrupt level so it could use
> the sempahore to sleep and protect the data it needs to access.  The task
> would start off sleeping in wait_event().  The actual interrupt level handler
> function would then wake it up.  Upon waking the task would obtain the
> semaphore, perform the actions to service the interrupt, release the semaphone
> then go back to sleep with wait_event() unless the interrupt level handler has
> incremented an interrupt count signifying there may be more work to do.
> The driver's unload routine would also wake up the interrupt service thread
> after setting a flag that says it should exit.

That would be possible. Though stopping a kernel thread when the
module is being unloaded is very hard to do without race
conditions. (But it is possible.)

Quote:

> I thought of using a user-mode daemon that would use a special ioctl command
> that does the interrupt service loop.  But I would prefer to keep everything
> within the driver code and managed by the driver.  I don't like the prospect
> of having to maintain the separate daemon - checking to see if it dies for
> some reason or even having to provide a mechanism to stop and restart it if
> the driver module is unloaded and reloaded.

I don't see any point in doing this in user mode. So just keep it
in kernel mode.

Quote:

> Can a tasklet be used in such a way?  Is it possible to start a tasklet in the
> module initialization function and can that tasklet use semaphores?

I'm not sure about those tasklets.

Quote:> Is there a kernel thread mechanism in Linux?

Yes, check question number five on this page:
http://www.daimi.au.dk/~kasperd/comp.os.linux.development.faq.html

--
Kasper Dupont -- der bruger for meget tid p? usenet.


 
 
 

Serializing data & device access across file ops and interrupt handler

Post by Floy » Wed, 01 May 2002 09:30:43


Thanks for the invaluable tips and the pointer on kernel threads.  The
O'Reilly book doesn't mention kernel threads.  Nor does it mention
the daemonize function.  Your FAQ is a great resource.  Once I knew what to
look for, I was able to find good examples in the Kernel source tree and come
up with an implementation that will work - once I iron out some details
dealing with the device's interrupt mask and status registers.

I still need to find some more infor on the meaning of the flags passed to
kernel_threads but it seems like a a value of zero will suffice for my
purposes.


> Floyd asked:
>> Is there a kernel thread mechanism in Linux?

>Yes, check question number five on this page:
>http://www.daimi.au.dk/~kasperd/comp.os.linux.development.faq.html

>--
>Kasper Dupont -- der bruger for meget tid p? usenet.


 
 
 

Serializing data & device access across file ops and interrupt handler

Post by Kasper Dupon » Wed, 01 May 2002 14:52:16



> I still need to find some more infor on the meaning of the flags passed to
> kernel_threads but it seems like a a value of zero will suffice for my
> purposes.

The man page about clone would be one place to look. The fork() system
call will put SIGCHLD in this field, and khubd is started with
CLONE_FS | CLONE_FILES | CLONE_SIGHAND.

Since daemonize will give up userspace resources anyway it might be
slightly more efficient to clone them to avoid the default copying.
Otherwise I don't see any purpose of using the CLONE_* flags.

--
Kasper Dupont -- der bruger for meget tid p? usenet.

 
 
 

1. Is it possible to write an interrupt handler w/o a device driver

Hi!

I'm writing some routines to play sound under Solaris 2.1 (x86).  They
work great, but I would like to be able to handle interrupts sent
from the soundcard.  Can this be done without writing a device driver ?
(I don't have any docs on device drivers or the SunPro compiler, I'm
using gcc.)

Thanks for any help.

__________________________________________________________________________
   _
  | |       |                         Anders Skar
  |_|  _   _|  _   _  _      Trondheim College of Engineering

  | | | | |_| |_  |   _|
__________________________________________________________________________
   Why run Windows when Dosshell is faster and offers taskswitching too?

2. 3 COM: NEED HELP ON LAPTOP !!!

3. Interrupt handler without a device driver?

4. NVidia 0.93 glx driver strange thing...

5. setsockopt( fh, IPPROTO_TCP, TCP_NODELAY, &op, sizeof( op ) )

6. Netgear Trouble 3 (Pinging)

7. AHA2940 & Interrupt handler

8. Trying to get to grips with Linux

9. Help!, retrieving data from files spread across directories.

10. Handlers, Handlers, Handlers

11. Gator eWallet Insecure User Data files Encryption and Gator BackUp / Banner Server Access/File retrieving

12. Networked two linux computers, can't access files across?

13. Windows network file access to server across the Internet ?