Very Computer

by **p..** » Fri, 21 Apr 2000 04:00:00

Hi, unix gurus:

Here's my situation, we have a server which, when runs, will consume a
large amount of memory(about 400M) by caching a lot of tables. We can
run multiple instances of the server on one machine, but each of them
will have its own copy of cached tables, which is of course a waste. To
get rid of this limitation, some folks here are talking about
using "shared memory" to handle the table caching, but I really doubt
about this:

1. I believe shared memory is just a mapping mechanism, so if one
process has ,say, 100M shared memory, other process will still need
100M physical memory to map it, am I right ?
2. shared memory itself has some limitation, I'm not sure whether it
can handle as much as 400-500M.(we're running HP-UX 10.20 now).

So my question is if shared memory is not an option, what is the
typical way to handle a situation like this? I'm thinking about to have
a separate data server which handles all database transaction/cacheing
and it talks with other processors through some kind of IPC or RPCs,
and multi-threading might be an option too. Are there any commercial
products which can help here?

BTW, we're using hp-ux 10.20 with sybase 11.xx.

Any opinion/suggestion is very much appreciated.

-Peter

Sent via Deja.com http://www.deja.com/
Before you buy.

by **Paul D. Smi** » Fri, 21 Apr 2000 04:00:00

p> 1. I believe shared memory is just a mapping mechanism, so if one
p> process has ,say, 100M shared memory, other process will still need
p> 100M physical memory to map it, am I right ?

No.

You're thinking of mmap(), for doing memory-mapped files. That's quite
different than shared memory.

Shared memory does indeed allow multiple processes on a system to access
the exact same memory, and not require their own copies of it.

--
-------------------------------------------------------------------------------

"Please remain calm...I may be mad, but I am a professional." --Mad Scientist
-------------------------------------------------------------------------------
These are my opinions---Nortel Networks takes no responsibility for them.

by **Andrew Gabri** » Fri, 21 Apr 2000 04:00:00

Quote:>Hi, unix gurus:

>Here's my situation, we have a server which, when runs, will consume a
>large amount of memory(about 400M) by caching a lot of tables. We can
>run multiple instances of the server on one machine, but each of them
>will have its own copy of cached tables, which is of course a waste. To
>get rid of this limitation, some folks here are talking about
>using "shared memory" to handle the table caching, but I really doubt
>about this:

>1. I believe shared memory is just a mapping mechanism, so if one
>process has ,say, 100M shared memory, other process will still need
>100M physical memory to map it, am I right ?

No. Both processes will be referencing the same physical memory.

Quote:>2. shared memory itself has some limitation, I'm not sure whether it
>can handle as much as 400-500M.(we're running HP-UX 10.20 now).

I don't know what HP-UX's limits are on shared memory segment size.
There are different types of shared memory, and which you use depends
on factors such as how your multiple processes are started up. I do
recall reading that HP-UX has a rather small limit on the number of
shared segments it can handle efficiently without having to start
swapping some hardware mapping tables, but you should only need one
so that's unlikely to be a problem unless you have other applications
which also use shared memory.

Quote:>So my question is if shared memory is not an option, what is the
>typical way to handle a situation like this? I'm thinking about to have
>a separate data server which handles all database transaction/cacheing
>and it talks with other processors through some kind of IPC or RPCs,
>and multi-threading might be an option too. Are there any commercial
>products which can help here?

Shared memory almost certainly is an option. You will require some
sort of synchronisation objects such as mutexs to protect areas of
shared memory from simultaneous reading and updating and resyncing
of processor caches. Shared memory in the form of multi-threading
might be a good way to go - it depends how much work is involved to
make the non-shared memory parts of your application thread-safe.
A single non-threaded data server process could become a bottle-neck
which doesn't easily scale.

Quote:>BTW, we're using hp-ux 10.20 with sybase 11.xx.

HP-UX was rather a late-commer to kernel-aware threading; you'd better
check if HP-UX 10.20 has it - you might need to go to HP-UX 11 in order
to persue the threads route. (My vague recollection is that is appeared
in 10.30 which was a developer-only release, but I might be wrong.)

--
Andrew Gabriel
Consultant Software Engineer

Very Computer

Is "shared memory" a way to reduce physical memory usage?

Is "shared memory" a way to reduce physical memory usage?

Is "shared memory" a way to reduce physical memory usage?

Is "shared memory" a way to reduce physical memory usage?