Making libraries thread-safe

Making libraries thread-safe

Post by Arthur Gol » Fri, 10 Dec 1999 04:00:00



Hey all:

Does anyone have any information or pointers on what kind of
semaphores/mutexes should be used to make a library thread-safe (as well
as signal safe) in a situation where:

1) There's no way to know if an app is threaded at all--and if it _is_
what kind of threading it's using (Netscape seems to be an example of
this, at least the version linked to the glibc-2.0 series).

2) The Right way to do it (i.e. pthreads semaphores--sem_init(),
sem_wait(), sem_post() ) isn't really available as an option (there are
deep reasons for this that have to do with the nature of the library; I
can elaborate on request).

One situation that may _not_ have a solution would be running on top of
a non-pre-emptive thread system (a thread spinning on a lock in a
handler would just keep spinning); fortunately `gotcha' cases--if
they're well-understood--are OK.

Many thanks.
--ag

--
Artie Gold, Austin, TX  (finger the cs.utexas.edu account for more info)

--
A: Look for a lawyer who speaks Aramaic...about trademark infringement.

 
 
 

Making libraries thread-safe

Post by Kaz Kylhe » Fri, 10 Dec 1999 04:00:00



>Hey all:

>Does anyone have any information or pointers on what kind of
>semaphores/mutexes should be used to make a library thread-safe (as well
>as signal safe) in a situation where:

You should probably use the POSIX interface provided by the system library.

Quote:>1) There's no way to know if an app is threaded at all--and if it _is_
>what kind of threading it's using (Netscape seems to be an example of
>this, at least the version linked to the glibc-2.0 series).

If you don't know what kind of threading the application is using, then you
can't easily make your library thread-safe, other than through the use of
hardware primitives. (E.g. using atomic compare-exchange to implement wait-free
algorithms).

One way to make a library thread-safe is to avoid using static data, and write
the functions in a reentrant manner. A reasonble compromise may be that
operations on distinct objects are safe.  For example, if you wrote a library
that mainpulates binary trees, say, it might be reasonable to make it safe in
the face of access to distinct trees. This is easily accomplished without any
interlocks. The threaded application has to use its own locking if it wants to
access the same binary tree using multiple threads.

Quote:>2) The Right way to do it (i.e. pthreads semaphores--sem_init(),
>sem_wait(), sem_post() ) isn't really available as an option (there are

Semaphores are rarely, if ever The Right Way to do it. That distinction belongs
to condition variables.

For merely making code thread safe, you may want a locking mechanism, not a
synchronzation mechanism: hence mutexes. pthread_mutex_lock(),
pthread_mutex_unlock().

Quote:>One situation that may _not_ have a solution would be running on top of
>a non-pre-emptive thread system (a thread spinning on a lock in a
>handler would just keep spinning); fortunately `gotcha' cases--if
>they're well-understood--are OK.

You might have to make your library open-ended. If it was a C++ library, you
could use a template mechanism described in Andrew Koenig's _Ruminations on
C++_. His example uses I/O, but it applies to threadig primitives as well.

The idea is to leave your library open-ended as to which interface it will use
for I/O (or in this case for sychronization) by templating it.

The template parameters are abstract things that provide the synchronization
operations.

The user of your library simply has to implement suitable classes which he can
supply as template parameters.

E.g.

    template <class Mutex, class Condition> class Queue {
    private:
        Mutex m_mutex;
        Condition m_cond;
    public:
        void Enqueue(Queuenode *qn)
        {
            mutex.Lock();
            // do the operation
            mutex.Unlock();
        }

        Queuenode *BlockingDequeue()
        {
            mutex.Lock();

            while (IsEmpty())
                m_cond.Wait(&mutex);

            Queuenode *out = DequeueUnsafe();

            mutex.Unlock();

            return out;
        }
    };

The idea here is that the user must implement a mutex and condition class which
have all the right oeprations, like Lock(), Unlock() and Wait().  The user
instantiates your template over his or her synchronization classes.

Of course, you don't have a fixed binary interface anymore. There are drawbacks
to temlates and C++ in general.

You could do a similar thing in C. Whatever it takes to leave the
synchronization interface open-ended.

Here is one simple way: declare a hard rule that that the user must provide
management for mutex objects, which must fit into 32 bytes.  Then in all your
objects that need protecting, leave a properly aligned 32 byte buffer.  Also,
require that the user implement lock and unlock functions having specific
names. Leave these names as unresolved references in your library:

    extern void user_mutex_init(void *buffer32bytes);
    extern void user_mutex_lock(void *buffer32bytes);
    extern void user_mutex_ulock(void *buffer32bytes);

The user must implement these functions otherwise your library won't link.  The
functions simply treat the given buffer as a mutex object.  Your library can
then declare mutex objects wherever it needs to simply by reserving that much
aligned space. It calls back into the app to have these objects initialized and
to lock and unlock them.  Problem solved: you can hack this into working over
nearly any API.  (Though I can see hassles with Win32 critical sections).

You have to make a list of the likely target API's and decide on the precise
interface.

 
 
 

Making libraries thread-safe

Post by Arthur Gol » Tue, 14 Dec 1999 04:00:00


Kaz:

Thanks for your response.
Unfortunately, the options you have suggested won't work in my case;
I'll (hopefully) clarify what my needs are with comments below.



> >Hey all:

> >Does anyone have any information or pointers on what kind of
> >semaphores/mutexes should be used to make a library thread-safe (as well
> >as signal safe) in a situation where:

> You should probably use the POSIX interface provided by the system library.

> >1) There's no way to know if an app is threaded at all--and if it _is_
> >what kind of threading it's using (Netscape seems to be an example of
> >this, at least the version linked to the glibc-2.0 series).

> If you don't know what kind of threading the application is using, then you
> can't easily make your library thread-safe, other than through the use of
> hardware primitives. (E.g. using atomic compare-exchange to implement wait-free
> algorithms).

Which might be the way I have to go. I ws hoping to avoid having to
"roll my own" mutex, but that's looking unavoidable.
Quote:

> One way to make a library thread-safe is to avoid using static data, and write
> the functions in a reentrant manner. A reasonble compromise may be that
> operations on distinct objects are safe.  For example, if you wrote a library
> that mainpulates binary trees, say, it might be reasonable to make it safe in
> the face of access to distinct trees. This is easily accomplished without any
> interlocks. The threaded application has to use its own locking if it wants to
> access the same binary tree using multiple threads.

> >2) The Right way to do it (i.e. pthreads semaphores--sem_init(),
> >sem_wait(), sem_post() ) isn't really available as an option (there are

> Semaphores are rarely, if ever The Right Way to do it. That distinction belongs
> to condition variables.

Au contraire (and it's not your fault, since I didn't provide enough
information).
The reason I indicated that semaphores were the "Right Way" was the fact
that the library in question only does its real work in a signal
handler, specifically a handler for SIGSEGV.
In the most-recent (AFAIK) libc info pages, it states the following:
"It is not safe to call mutex functions from a signal handler.  In
particular, calling `pthread_mutex_lock' or `pthread_mutex_unlock' from
a signal handler may deadlock the calling thread."
Further:
"It is not safe to call the condition variable functions from a signal
handler. In particular, calling `pthread_cond_signal' or
`pthread_cond_broadcast' from a signal handler may deadlock the calling
thread."

Quote:

> For merely making code thread safe, you may want a locking mechanism, not a
> synchronzation mechanism: hence mutexes. pthread_mutex_lock(),
> pthread_mutex_unlock().

> >One situation that may _not_ have a solution would be running on top of
> >a non-pre-emptive thread system (a thread spinning on a lock in a
> >handler would just keep spinning); fortunately `gotcha' cases--if
> >they're well-understood--are OK.

> You might have to make your library open-ended. If it was a C++ library, you
> could use a template mechanism described in Andrew Koenig's _Ruminations on
> C++_. His example uses I/O, but it applies to threadig primitives as well.

Nope. C.
[snipped]
Again, this isn't possible.
The point of the library is to virtualize signal handling and virtual
memory (we'll use it as a preloaded library). We cannot expect to have
any access to the application itself, except as a binary. We also cannot
expect any _cooperation_ from the app.

It continues to sound like creating our own mutex will be the only way
to go; even in cases where we can determine what kind of threading is
going on (for example by doing a dlsym(RTLD_NEXT,"__pthread_mutex_init")
) we would have the additional problem of sharing library code, also a
no-no (the first purpose of the library is as a general virtual memory
tracer; the fundamental notion is that we use mprotect to forbid access
to the address space and respond to the SIGSEGVs by making note of what
page the app attempted to touch and unprotecting it.)

As regards reentrancy, the only real problem we have in regard to the
fact that we need a virtualized page table--which is global to the
process by its very nature.

Again, many thanks. I would more than welcome any future comments you
might have.

--ag
--
Artie Gold, Austin, TX  (finger the cs.utexas.edu account for more info)

--
A: Look for a lawyer who speaks Aramaic...about trademark infringement.

 
 
 

1. Making a library thread-safe

Hello,

I have been assigned the task of making a library thread-safe.  I'm
seeking advice on the following issues:

1) Sun's "Multithreaded Programming Guide" states that modules of a
multi-threaded application should be compiled with either -D_REENTRANT
or -D_POSIX_C_SOURCE=199506L (or both).  To get a single-threaded
application, none of these flags should be defined.  Page 167 of the
May 2002 version of this guide says: "This applies to every module of
an application."

Does this imply that it is impossible to create a single thread-safe
version of the library that can be linked into both single-threaded
and multi-threaded applications?

[I would prefer to not have to create two or perhaps four versions of
the library for each possible combination of the above mentioned
flags.]

2) The library has an error code global variable, much like `errno' in
the Standard C library.  This error code variable must somehow be
converted into thread-specific data.

The nicest solution would be to use thread-local storage (TLS), with
support from the linker, thread-library, compiler, and assembler.  The
Solaris 9 12/02 "Linker and Libraries Guide" suggests that the linker
supports TLS, but I cannot find anything about how to use this from
within C source code.

Perhaps this feature is not quite ready for use yet?

Otherwise, I suppose I must use the pthread_key_create,
pthread_setspecific, and pthread_getspecific functions?  And the key
must be created using a pthread_once_t control unless the user is
required to create the key explicitly?

Thanks in advance for any advice on these questions.

---

Hugin Expert A/S
Niels Jernes Vej 10
9220 Aalborg East
DENMARK

2. Problem Compiling Kernel

3. Using thread-safe C library functions.

4. Need help in implementing asynchronous communication.

5. using thread-safe library code without -lpthreads

6. Driver for IRWIN 40MB, labled by HP

7. Thread-safe C libraries in Solaris?

8. Mouse Problem in X

9. Solaris 8 threads: If a routine is Async-Signal-Safe is it also thread Safe?

10. mmap, thread-safe, and signal-safe

11. Thread-safe and Signal-safe

12. Making gcc thread safe.

13. How to make a thread safe library