Async I/O implementation Details

Async I/O implementation Details

Post by MrSigno » Tue, 12 Sep 2000 02:55:46



Hi,

    I originally posted a very similar question to that posted below in
    comp.programming.threads, but received no replies

    So here's hoping

    In respect of HP-UX, Solaris, TRU64, AIX and IRIX and various Linux's,
    how have the different OS's implemented Async I/O ... ie POSIX.4

    1) Modify the OS so that the OS supports the Async I/O specification

    2) Cheat, and support it via threads, thus 100 async i/o requests
         have 100 threads doing the I/O for you

    3) ??

    Thanks inadvance

-Fred

 
 
 

Async I/O implementation Details

Post by John Ingra » Wed, 13 Sep 2000 12:32:01


You are going to have problems implementing the 2nd  option on some unixes.
Some Unixes do not have threads.  Sco Openserver comes to mind.  They have a
some thing like threads but its not build into the os.


Quote:> Hi,

>     I originally posted a very similar question to that posted below in
>     comp.programming.threads, but received no replies

>     So here's hoping

>     In respect of HP-UX, Solaris, TRU64, AIX and IRIX and various Linux's,
>     how have the different OS's implemented Async I/O ... ie POSIX.4

>     1) Modify the OS so that the OS supports the Async I/O specification

>     2) Cheat, and support it via threads, thus 100 async i/o requests
>          have 100 threads doing the I/O for you

>     3) ??

>     Thanks inadvance

> -Fred


 
 
 

Async I/O implementation Details

Post by Bob Stumpf » Thu, 14 Sep 2000 04:51:14



> You are going to have problems implementing the 2nd  option on some unixes.
> Some Unixes do not have threads.  Sco Openserver comes to mind.  They have a
> some thing like threads but its not build into the os.



> > Hi,

> >     I originally posted a very similar question to that posted below in
> >     comp.programming.threads, but received no replies

> >     So here's hoping

> >     In respect of HP-UX, Solaris, TRU64, AIX and IRIX and various Linux's,
> >     how have the different OS's implemented Async I/O ... ie POSIX.4

> >     1) Modify the OS so that the OS supports the Async I/O specification

> >     2) Cheat, and support it via threads, thus 100 async i/o requests
> >          have 100 threads doing the I/O for you

> >     3) ??

> >     Thanks inadvance

> > -Fred

HPUX and Solaris now support async I/O primarily thru method [2] above.  The
last version of Irix (4.2?) which I saw did have specific aio_xxx functions
(which actually resulted, the first time they were called in the creation of
multiple 'sprocs' which did the actual I/O for you).  I'm not sure why you refer
to multi-threading as "cheating" though, since the aioxxx/aio_xxx functions of
other systems actually result (typically) in the kernels of those OS's spawning
the threads in kernel mode to do the I/O operations.  The use of multi-threading
to do async I/O is actually an optimization of the earlier methods which results
in a "cleaner" form of coding...you (the programmer) have to pay attention to
the synchronization of threads and application shared data, which is often
referenced when doing I/O operations...this is a relatively small price to pay
for programs which are simpler to understand, and easier to maintain.

Bob.
--

========================================
R.J.Stumpf
Software Engineering,
Open Image Core Product Group


========================================

  bob_stumpf.vcf
< 1K Download
 
 
 

Async I/O implementation Details

Post by MrSigno » Fri, 15 Sep 2000 11:34:34


Regardless of whether a thread issues a synchronous call, or
an asynchronous call, both are obviously actually handled by
the OS itself. When the I/O completes it will let us know.

Quote:>>>> Why cheating ?

What i dislike about the Threads implementation is if you
have a busy server, say several thousand connections you
also have several thousand threads. This is a LOT of
contention / context switching. After all, if you only
have two cpu's, you can NOT have more two threads active
at any time.

Thus for those OS's that support async I/O directly within the
Kernel (OpenVMS, TRU64, WNT), you can queue multiple async
requests, and keep just a couple of threads really busy, and
thus avoid context swicthing.

Also, everything I have read, suggest that the Async I/O model,
though more complicated, is usually superior in respect of
throughput
==================================================


> > You are going to have problems implementing the 2nd  option on some
unixes.
> > Some Unixes do not have threads.  Sco Openserver comes to mind.  They
have a
> > some thing like threads but its not build into the os.



> > > Hi,

> > >     I originally posted a very similar question to that posted below
in
> > >     comp.programming.threads, but received no replies

> > >     So here's hoping

> > >     In respect of HP-UX, Solaris, TRU64, AIX and IRIX and various
Linux's,
> > >     how have the different OS's implemented Async I/O ... ie POSIX.4

> > >     1) Modify the OS so that the OS supports the Async I/O
specification

> > >     2) Cheat, and support it via threads, thus 100 async i/o requests
> > >          have 100 threads doing the I/O for you

> > >     3) ??

> > >     Thanks inadvance

> > > -Fred

> HPUX and Solaris now support async I/O primarily thru method [2] above.
The
> last version of Irix (4.2?) which I saw did have specific aio_xxx
functions
> (which actually resulted, the first time they were called in the creation
of
> multiple 'sprocs' which did the actual I/O for you).  I'm not sure why you
refer
> to multi-threading as "cheating" though, since the aioxxx/aio_xxx
functions of
> other systems actually result (typically) in the kernels of those OS's
spawning
> the threads in kernel mode to do the I/O operations.  The use of
multi-threading
> to do async I/O is actually an optimization of the earlier methods which
results
> in a "cleaner" form of coding...you (the programmer) have to pay attention
to
> the synchronization of threads and application shared data, which is often
> referenced when doing I/O operations...this is a relatively small price to
pay
> for programs which are simpler to understand, and easier to maintain.

> Bob.
> --

> ========================================
> R.J.Stumpf
> Software Engineering,
> Open Image Core Product Group


> ========================================

 
 
 

Async I/O implementation Details

Post by John S. Dys » Sun, 17 Sep 2000 04:00:00




Quote:> Regardless of whether a thread issues a synchronous call, or
> an asynchronous call, both are obviously actually handled by
> the OS itself. When the I/O completes it will let us know.

>>>>> Why cheating ?

> What i dislike about the Threads implementation is if you
> have a busy server, say several thousand connections you
> also have several thousand threads. This is a LOT of
> contention / context switching. After all, if you only
> have two cpu's, you can NOT have more two threads active
> at any time.

> Thus for those OS's that support async I/O directly within the
> Kernel (OpenVMS, TRU64, WNT), you can queue multiple async
> requests, and keep just a couple of threads really busy, and
> thus avoid context swicthing.

> Also, everything I have read, suggest that the Async I/O model,
> though more complicated, is usually superior in respect of
> throughput

On FreeBSD (due to time constraints), I implemented Async I/O
as a hybrid.  In the case of subsystems not designed for it,
the FreeBSD scheme would create threads to provide context
for the I/O.  On subsystems that could easily support Async
I/O, I implemented the 'right' way.  The additional overhead
to support both was small, yet could provide a small amount
of improvement.  As contributors decided that better Async I/O
support was needed in any given subsystem, it could be provided
without major changes at the higher levels of the code.

It isn't an ideal solution, but is a workaround that could be
morphed into something close to ideal.  The implementation
supported the standard aio semantics at the system call interface.

John

 
 
 

1. Async IO calls - Linux implementations?

I am attempting to port an application that uses asynchronous disk I/O
calls. It is currently implemented on Solaris and other platforms where
aioread/aiowrite/aiocancel/aiowait (solaris) and lio_listio() functions
are implmented.

Are there any implementations of async I/O for Linux 2.x?

Thanks!

Robert

2. Modem set to answer call in?

3. Implementation detail of fsck.

4. Announcing Gcal-2.40

5. SNORT Implementation detail

6. Where are the scripts that load the modules located?

7. Q: Implementation details putmsg/getmsg

8. Epson STYLUS Color with Linux?

9. implementation details of getnetbyname()

10. SMP clusters implementation details

11. NFS implementation details

12. Problem adding tty port using 128 Port Async Adapter with 16 port Async Node

13. To async or not not async