Solaris 10: Increasing the process data space; shared memory segments & intimate shared memory problems

Solaris 10: Increasing the process data space; shared memory segments & intimate shared memory problems

Post by Roger P. Johnso » Sat, 06 May 2006 09:09:46



I have a Progress 10 (64 bit) Open Edge DB server running on Solaris 10u1 which runs on an E450 w/4GB of RAM.
I have a project and assigned the project as the default for the user to run the db server in.
In that project I have the resource project.max-shm-memory = 3GB.

There is a startup parameter to the database server (-pinshm) that will lock the shared memory segments into RAM so they are not be paged out. This is the intimate shared memory option in the system call to shmat().

This is the problem I am having:
When I start the db server specifying the -pinshm option to lock the segments, if the buffer pool size is approximately 1843666944 bytes (or 225057 db blocks; where 1 db block is 8192 bytes) or less, this works and the segments are locked according to ipcs -A; but when I specify a buffer pool greater than 1843666944 bytes (such as one block more 225058) and the -pinshm option, the db server konks out as in (shown is a 2GB buffer pool in the -B arg):

$ proserve apprise -B 262144 -bibufs 30 -L 100000 -H localhost -S appriselive -Ma 10 -Mi 3 -Mn 10 -n 100 -pinshm
OpenEdge Release 10.0B as of Thu Aug  5 21:55:46 EDT 2004
18:35:03 BROKER   : The data space of the process is not enough for the shm segment (1176) <---- THIS HERE ERROR
18:35:05 BROKER   : Removed shared memory with segment_id: 1
18:35:05 BROKER   : Removed shared memory with segment_id: 0
18:35:06 BROKER   : ** This process terminated with exit code 2. (8619)
$

Without using the -pinshm option to lock the segments in RAM, I can successfully start the db server using a buffer pool upto 3GB sucessfully as in:
$proserve apprise -B 262144 -bibufs 30 -L 100000 -H localhost -S appriselive -Ma 10 -Mi 3 -Mn 10 -n 100
OpenEdge Release 10.0B as of Thu Aug  5 21:55:46 EDT 2004
18:41:25 BROKER  0: Multi-user session begin. (333)
18:41:25 BROKER  0: Begin Physical Redo Phase at 0 . (5326)
18:41:25 BROKER  0: Physical Redo Phase Completed at blk 0 off 532 upd 0. (7161)
18:41:25 BROKER  0: Started for appriselive using TCP, pid 611. (5644)
$

QUESTION: when the shared memory segments are locked into ram, does this increase the process space in some manner where as not locking the shared memoary segments doesn't and is there a way to increase the users process size? I don't "see" it in the resource_controls(5).

As an experiment, I modified an example from W. Richard Stevens book "Advanced Programming in the UNIX Environment" for creating, attaching, and deleting shared memory segments and I changed it to use intimate shared memory -- to lock in the segments to see if it can be done on this system.

Here I create 1x3GB shared memory segment that is locked in with his modifed program:
$ ./tshm3 -n 1 -s 3221225472
This Stevens modifed program that will create 1 segment 3GB in size (I can post the source if needed to see how I do it).

From another terminal window I can see under the ISMATTCH column that the one process is attached to the locked in segment:
$ ipcs -A
IPC status from <running system> as of Thu May  4 18:47:53 CDT 2006
Message Queues:
[message queue snip]
T         ID      KEY        MODE        OWNER    GROUP  CREATOR   CGROUP NATTCH      SEGSZ  CPID  LPID   ATIME    DTIME    CTIME  ISMATTCH         PROJECT
Shared Memory:
m          5   0x8        --rw------- apprisei apprisei apprisei apprisei      1 3221225472   634   634 18:47:19 no-entry 18:47:07        1 user.appriseinc
[semaphore snip]
$

So I don't know what to think. I am not sure that I can claim an apples to apples comparision as I am assuming that the Progress implementation has to do the same thing.

I've searched Progress "Knowledge Base" and there is no definite answer from them so I thought I'd try here.
Here are some of their answers:
i. "Exact cause is unknown at the time of this writing (8/30/2004). Increase SHMMAX,SHMSEG and MAXUMEM".
ii."Either incress the process data space or reduce the size of the used shared memory by decreasin -B, -c, and/or -L".
iii."Insufficient continguous shared memory exists to launch the database with the parameters configured"
iv. "System was out of free memory"

case i: I don't see a resource control to change the SHMSEG segment size like you could in Solaris 9; Can't find anything about MAXUMEM for Solars.
case ii: Decreasing -B works as mentioned, I have no change in outcome decreaseing the -L parameter (just number of locks)
case iii: Even afer a fresh reboot I can only lock in approximately 225057 blocks in the buffer pool
case iv: I can see swap being allocated as the segments are created but it doesn't run out of free memory nor swap; plus the C program tshm3 works.

As a side note, couple years back I tried all this on a V1280 w/16GB ram with Solaris 9. I could only get about 7GB of shared memory as locked in. I had to fiddle with the shared memory segment size and the number of shared segments a process could attach over its /etc/system file in order to get it to work.

I thought I'ld try here first, as the Progress Support doesn't sound to promising based on their "Knowledge Base". How can they not know? I understand it to be that they develop their database server product on Solaris. Does anyone have any ideas of what to change or do?

TIA
-Roger

Here is the output of the process resource settings used:
pro10.0B-run>prctl $$
process: 588: /bin/sh
NAME    PRIVILEGE       VALUE    FLAG   ACTION                       RECIPIENT
process.max-port-events
        privileged      65.5K       -   deny                                 -
        system          2.15G     max   deny                                 -
process.max-msg-messages
        privileged      8.19K       -   deny                                 -
        system          4.29G     max   deny                                 -
process.max-msg-qbytes
        privileged      64.0KB      -   deny                                 -
        system          16.0EB    max   deny                                 -
process.max-sem-ops
        privileged        512       -   deny                                 -
        system          2.15G     max   deny                                 -
process.max-sem-nsems
        privileged        512       -   deny                                 -
        system          32.8K     max   deny                                 -
process.max-address-space
        privileged      16.0EB    max   deny                                 -
        system          16.0EB    max   deny                                 -
process.max-file-descriptor
        basic             256       -   deny                               588
        privileged      65.5K       -   deny                                 -
        system          2.15G     max   deny                                 -
process.max-core-size
        privileged      8.00EB    max   deny                                 -
        system          8.00EB    max   deny                                 -
process.max-stack-size
        basic           8.00MB      -   deny                               588
        privileged      8.00EB      -   deny                                 -
        system          8.00EB    max   deny                                 -
process.max-data-size
        privileged      16.0EB    max   deny                                 -
        system          16.0EB    max   deny                                 -
process.max-file-size
        privileged      8.00EB    max   deny,signal=XFSZ                     -
        system          8.00EB    max   deny                                 -
process.max-cpu-time
        privileged      18.4Es    inf   signal=XCPU                          -
        system          18.4Es    inf   none                                 -
task.max-cpu-time
        system          18.4Es    inf   none                                 -
task.max-lwps
        system          2.15G     max   deny                                 -
project.max-contracts
        privileged      10.0K       -   deny                                 -
        system          2.15G     max   deny                                 -
project.max-device-locked-memory
        privileged       250MB      -   deny                                 -
        system          16.0EB    max   deny                                 -
project.max-port-ids
        privileged      8.19K       -   deny                                 -
        system          65.5K     max   deny                                 -
project.max-shm-memory
        privileged      3.00GB      -   deny                                 -
        system          16.0EB    max   deny                                 -
project.max-shm-ids
        privileged        128       -   deny                                 -
        system          16.8M     max   deny                                 -
project.max-msg-ids
        privileged        128       -   deny                                 -
        system          16.8M     max   deny                                 -
project.max-sem-ids
        privileged        128       -   deny                                 -
        system          16.8M     max   deny                                 -
project.max-crypto-memory
        privileged       999MB      -   deny                                 -
        system          16.0EB    max   deny                                 -
project.max-tasks
        system          2.15G     max   deny                                 -
project.max-lwps
        system          2.15G     max   deny                                 -
project.cpu-shares
        privileged          1       -   none                                 -
        system          65.5K     max   none                                 -
zone.max-lwps
        system          2.15G     max   deny                                 -
zone.cpu-shares
        privileged          1       -   none                                 -
        system          65.5K     max   none                                 -
pro10.0B-run>

 
 
 

1. increasing # of shared memory segments on Solaris

I'm running Solaris 2.3 on a Sparc 10 multiprocessor machine.  I need
to be able to increase the number of shared memory segments in order
to run a multithreaded application (the system default is 6).  Does
anyone know how to do this?  I understand I have to edit the /etc/system
file, but I don't know the command, and the man pages aren't very
helpful.  Pointers to useful books covering this and similar subjects
would also be appreciated.

Mike

--
------------------------------------------------------------

Department of Computation and Neural Systems, Caltech 216-76
------------------------------------------------------------

2. ISDN TA internal cards and drivers support

3. 10 Shared memory segments in AIX

4. error msg...

5. Increase default number of shared memory segments?

6. identd

7. Increase Shared Memory Segment on SPARC

8. S3 Trio 765 Video Drivers Needed

9. Increasing Maximum SHARED MEMORY segment.

10. How to increase Shared memory segments

11. intimate shared memory in Solaris 2.6

12. AIX Shared Memory Segment (Not enough memory)

13. mapping SysV shared memory to data segment