Very Computer

Hi,
we have two rs6000 (a 320H and 340) being used by different kinds of users.
Some of them do large molecular calculations which create HUGE (600-700 MB) tem
porary files for the molecular integrals. When their program begins writing the
se huge files and then reading them as needed, our machines almost stop and oth
er users are almost unable to use the system. The slowness means 30-60 sec to l
ogon, 1 minute for X windows to create a new window, a long time for even an ls
, etc. Most other users do a lot of editing so they actually need better respon
se time from the machine.
Is there anyway to correct this situation (besides killing those huge calculati
ons, of course)?
Thanks for your answers,
                      Joao Damas

|> Hi,
|> we have two rs6000 (a 320H and 340) being used by different kinds of users.
|> Some of them do large molecular calculations which create HUGE (600-700 MB) tem
|> porary files for the molecular integrals. When their program begins writing the
|> se huge files and then reading them as needed, our machines almost stop and oth
|> er users are almost unable to use the system. The slowness means 30-60 sec to l
|> ogon, 1 minute for X windows to create a new window, a long time for even an ls
|> , etc. Most other users do a lot of editing so they actually need better respon
|> se time from the machine.
|> Is there anyway to correct this situation (besides killing those huge calculati
|> ons, of course)?
|> Thanks for your answers,
|>                       Joao Damas

Run those processes at a fixed priority of 126 or slightly smaller;
with dynamic priorities and your situtation, the niceness scale is
inadequate.
--
Konrad Haedener                             Phone: +41 31 65 42 25
Institute for Physical Chemistry            FAX:   +41 31 65 39 94
University of Berne

Quote:>Hi,
>we have two rs6000 (a 320H and 340) being used by different kinds of users.
>Some of them do large molecular calculations which create HUGE (600-700 MB) tem
>porary files for the molecular integrals. When their program begins writing the
>se huge files and then reading them as needed, our machines almost stop and oth
>er users are almost unable to use the system. The slowness means 30-60 sec to l
>ogon, 1 minute for X windows to create a new window, a long time for even an ls
>, etc. Most other users do a lot of editing so they actually need better respon
>se time from the machine.
>Is there anyway to correct this situation (besides killing those huge calculati
>ons, of course)?

Lower the priority of the cruncher programs with nice or renice.

Look into using the I/O write queue length limitation feature. Ther is
a panel in SMIT to do this. However, describing how to do it here is quite
difficult.  Id suggest playing with some nonzero values and see if that helps.
Cant give you more info since Im not in front of my 6000 right now.

Add more memory.

Add faster disks or more disk arms and rearrange the files on those drives.

|> Hi,
|> we have two rs6000 (a 320H and 340) being used by different kinds of users.
|> Some of them do large molecular calculations which create HUGE (600-700 MB) tem
|> porary files for the molecular integrals. When their program begins writing the
|> se huge files and then reading them as needed, our machines almost stop and oth
|> er users are almost unable to use the system. The slowness means 30-60 sec to l
|> ogon, 1 minute for X windows to create a new window, a long time for even an ls
|> , etc. Most other users do a lot of editing so they actually need better respon
|> se time from the machine.
|> Is there anyway to correct this situation (besides killing those huge calculati
|> ons, of course)?
|> Thanks for your answers,
|>                       Joao Damas

1) Consider installation of NQS with fixed priority patch
   (obtainable e.g. from this site)
2) If these large, molecular calculations are Gaussian 9x Hartree-Fock
   ab-initio calculations, use the direct SCF method. This reduces
    disk-access considerably and is faster on the rs6k.

Hope that helps...
--
Martin Schuetz                              Phone: +41 31 65 42 40
Institute for Physical Chemistry            FAX:   +41 31 65 44 99
University of Berne

Quote:>   Lower the priority of the cruncher programs with nice or renice.

Unlikely to help very much. IO won't be effected and the crunchers will
just slurp up any spare cpu time. You're not competing for cpu but IO.

Quote:>   Look into using the I/O write queue length limitation feature. Ther is
>   a panel in SMIT to do this. However, describing how to do it here is quite
>   difficult.  Id suggest playing with some nonzero values and see if that
>   helps.

This sounds very interesting. Roughly how do I get to this panel?
I can't see it on my 3.1.5 system and I certainly would like to!
Alternatively, what's the command name?

John Rowe
Dept Physics
Exeter University
UK

Quote:>Hi,
>we have two rs6000 (a 320H and 340) being used by different kinds of users.
>Some of them do large molecular calculations which create HUGE (600700 MB) tem
>porary files for the molecular integrals. When their program begins writing the
>se huge files and then reading them as needed, our machines almost stop and oth
>er users are almost unable to use the system. The slowness means 3060 sec to l
>ogon, 1 minute for X windows to create a new window, a long time for even an ls
>, etc. Most other users do a lot of editing so they actually need better respon
>se time from the machine.
>Is there anyway to correct this situation (besides killing those huge calculati
>ons, of course)?

Try turning on I/O Pacing, which is accessed via "smit system"
fastpath, choose the "characterstics of the operating system" option.
If you ask for help in the fields, it will give some information. We
use 17 for the highwater mark and 5 for the lowwater mark (IBM
suggested "startingpoint" values). Values for the highwater mark
should be (according to IBM) along the formula '4*(num pages)+1'. The
"AIX Tuning Gude" (IBM Publication SC23236501) contains further
information about I/O Pacing, which is basically used to prevent large
I/O operations from monopolizing the CPU. Overall system performance
goes down when using this feature; it is used (in general) where you
want to favor interactive users over long I/O bound jobs.

                           L. Scott Emmons
                      CableData  Research Center
                     csusac.csus.edu!cdsac!scotte
                                KC6NFP

I was under the impression that the kernel *raises* the priority of
I/O pending processes because they are waiting for hardware interupts,
therefore the "nice" value has little or no effect on I/O bound
processes.

Our remedy for heavy I/O bound quantum chemical jobs is to have dedicated
workstations (320s) that run just that process (with no logins or competing
processes). We greatly enhance the throughput of work done as opposed to
running processes simultaneously. All interactive work is done on the
faster machines that also run "lightweight" simulations etc.

--
-----------------------------------------------------------------------------

University of Sydney NSW 2006 Australia        061-2-692 4137