Help! Can you guess the problem with this server?

Help! Can you guess the problem with this server?

Post by Phillip Fiedl » Sun, 19 May 2002 11:01:35



Samples from vmstat, iostat, and prstat

#vmstat
 procs     memory            page            disk          faults      cpu
 r b w   swap  free  re  mf pi po fr de sr s6 s1 s1 s1   in   sy   cs us sy id
 7 0 0 2057744 563288 25 84  0  0  0  0  0  0  2  1  2 2434 38005 32344 89 11 0
 6 0 0 2059440 564904 30  7  3  0  0  0  0  0  1  1  1 2417 29911 26551 93 7 0
 6 0 0 2059440 564904 18 15 342 0  0  0  0  0  1  1  1 2101 30208 27394 91 9 0
 7 0 0 2059400 564872 29 90  1  0  0  0  0  0  1  1  1 2431 22430 14489 82 16 1

#iostat
   tty        sd6          sd136         sd137         sd138           cpu
 tin tout kps tps serv  kps tps serv  kps tps serv  kps tps serv   us sy wt id
   0    0   0   0    0    6   1    1   18   2    3   11   1    3   90  8  0  1
   0    0   0   0    0    6   1    2    8   1    2   13   2    5   93  6  0  0
   0    0   0   0    0    6   1   11    2   0    4    6   1    4   91  7  0  2
   0    0   0   0    0   14   2    2   27   2    4   16   1    2   96  3  0  0

#prstat
   PID USERNAME  SIZE   RSS STATE  PRI NICE      TIME  CPU PROCESS/NLWP      
 26784 db2      1692M 1657M run     10    0   5:04.57  19% db2sysc/4
 26930 db2      1692M 1657M run     22    0   4:40.13  19% db2sysc/4
 26929 db2      1692M 1657M run     22    0   4:40.42  18% db2sysc/4
 26785 db2      1692M 1657M cpu1    21    0   5:05.26  18% db2sysc/4
 26364 mqm       135M   96M sleep    0    0   0:00.00 3.8% java/9
 26367 mqm       105M   69M sleep    0    0   0:00.00 2.9% java/9
 26362 mqm       106M   62M sleep   40    0   0:00.00 2.4% java/9

 
 
 

Help! Can you guess the problem with this server?

Post by Hemant Shar » Sun, 19 May 2002 14:33:39


Around 80% of cpu is occupied by db2sync process not much of swapping
or i/o .
System must be very slow mainly due to db2sync .I don't know what
db2sync does and there might be a patch or some kernel parameter which
needs tuning.

Hemant Sharma
http://www.adminschoice.com
----------------------------
"Whenever you need a helping hand --there is one at the end of your
arm !!"


> Samples from vmstat, iostat, and prstat

> #vmstat
>  procs     memory            page            disk          faults      cpu
>  r b w   swap  free  re  mf pi po fr de sr s6 s1 s1 s1   in   sy   cs us sy id
>  7 0 0 2057744 563288 25 84  0  0  0  0  0  0  2  1  2 2434 38005 32344 89 11 0
>  6 0 0 2059440 564904 30  7  3  0  0  0  0  0  1  1  1 2417 29911 26551 93 7 0
>  6 0 0 2059440 564904 18 15 342 0  0  0  0  0  1  1  1 2101 30208 27394 91 9 0
>  7 0 0 2059400 564872 29 90  1  0  0  0  0  0  1  1  1 2431 22430 14489 82 16 1

> #iostat
>    tty        sd6          sd136         sd137         sd138           cpu
>  tin tout kps tps serv  kps tps serv  kps tps serv  kps tps serv   us sy wt id
>    0    0   0   0    0    6   1    1   18   2    3   11   1    3   90  8  0  1
>    0    0   0   0    0    6   1    2    8   1    2   13   2    5   93  6  0  0
>    0    0   0   0    0    6   1   11    2   0    4    6   1    4   91  7  0  2
>    0    0   0   0    0   14   2    2   27   2    4   16   1    2   96  3  0  0

> #prstat
>    PID USERNAME  SIZE   RSS STATE  PRI NICE      TIME  CPU PROCESS/NLWP      
>  26784 db2      1692M 1657M run     10    0   5:04.57  19% db2sysc/4
>  26930 db2      1692M 1657M run     22    0   4:40.13  19% db2sysc/4
>  26929 db2      1692M 1657M run     22    0   4:40.42  18% db2sysc/4
>  26785 db2      1692M 1657M cpu1    21    0   5:05.26  18% db2sysc/4
>  26364 mqm       135M   96M sleep    0    0   0:00.00 3.8% java/9
>  26367 mqm       105M   69M sleep    0    0   0:00.00 2.9% java/9
>  26362 mqm       106M   62M sleep   40    0   0:00.00 2.4% java/9


 
 
 

Help! Can you guess the problem with this server?

Post by Brian Pa » Sun, 19 May 2002 15:05:45



> Samples from vmstat, iostat, and prstat

> #vmstat
>  procs     memory            page            disk          faults      cpu
>  r b w   swap  free  re  mf pi po fr de sr s6 s1 s1 s1   in   sy   cs us sy id
>  7 0 0 2057744 563288 25 84  0  0  0  0  0  0  2  1  2 2434 38005 32344 89 11 0
>  6 0 0 2059440 564904 30  7  3  0  0  0  0  0  1  1  1 2417 29911 26551 93 7 0
>  6 0 0 2059440 564904 18 15 342 0  0  0  0  0  1  1  1 2101 30208 27394 91 9 0
>  7 0 0 2059400 564872 29 90  1  0  0  0  0  0  1  1  1 2431 22430 14489 82 16 1

> #iostat
>    tty        sd6          sd136         sd137         sd138           cpu
>  tin tout kps tps serv  kps tps serv  kps tps serv  kps tps serv   us sy wt id
>    0    0   0   0    0    6   1    1   18   2    3   11   1    3   90  8  0  1
>    0    0   0   0    0    6   1    2    8   1    2   13   2    5   93  6  0  0
>    0    0   0   0    0    6   1   11    2   0    4    6   1    4   91  7  0  2
>    0    0   0   0    0   14   2    2   27   2    4   16   1    2   96  3  0  0

> #prstat
>    PID USERNAME  SIZE   RSS STATE  PRI NICE      TIME  CPU PROCESS/NLWP      
>  26784 db2      1692M 1657M run     10    0   5:04.57  19% db2sysc/4
>  26930 db2      1692M 1657M run     22    0   4:40.13  19% db2sysc/4
>  26929 db2      1692M 1657M run     22    0   4:40.42  18% db2sysc/4
>  26785 db2      1692M 1657M cpu1    21    0   5:05.26  18% db2sysc/4
>  26364 mqm       135M   96M sleep    0    0   0:00.00 3.8% java/9
>  26367 mqm       105M   69M sleep    0    0   0:00.00 2.9% java/9
>  26362 mqm       106M   62M sleep   40    0   0:00.00 2.4% java/9

My hypothesis is that you're seeing the results of expensive
join operations within whatever queries those db2sysc processes
are running.  There's no visible I/O bottleneck, and you're
not paging significantly.  And with the CPU usage at about
90% user and 10% system, most of the time is being spent in
application code, not in system calls.  My suggested next step
would be take a look at the queries being executed by that
database engine to see which ones are taking a long time to
run.

--Brian

 
 
 

Help! Can you guess the problem with this server?

Post by Dennis Clark » Tue, 21 May 2002 05:16:27



> My hypothesis is that you're seeing the results of expensive
> join operations within whatever queries those db2sysc processes
> are running.  There's no visible I/O bottleneck, and you're
> not paging significantly.  And with the CPU usage at about
> 90% user and 10% system, most of the time is being spent in
> application code, not in system calls.  My suggested next step
> would be take a look at the queries being executed by that
> database engine to see which ones are taking a long time to
> run.

> --Brian

If he has no authority to modify the application SQL queries then I'd suggest
throwing hardware at it.  A V880 is a good start.  Sometimes the only way to
increase performance is to either change the application or change the
hardware.  Usually one or the other given a well tuned system.  We don't know
what this system is doing from a tuning perspective so he may as well start
thinking of faster CPU's unless five day result turn around time is okay with
him.  :)

Dennis

 
 
 

1. config.guess can't guess host type under glibc 2.1.2?

Several software packages use a script called config.guess.
I've traced it through and this script generates a test C
program called dummy.c.

I've got a glibc 2.1.2 system, C compiler:

171-> gcc -v
Reading specs from /usr/lib/gcc-lib/i586-linux-libc6/2.8.1/specs
gcc version 2.8.1

When I try to compile the C program which is:

172-> cat ~/dummy.c
#include <features.h>
#if defined(__GLIBC__)
extern char __libc_version[];
extern char __libc_release[];
#endif
main(argc, argv)
     int argc;
     char *argv[];
{
#if defined(__GLIBC__)
  printf("%s %s\n", __libc_version, __libc_release);
#else
  printf("unkown\n");
#endif
  return 0;

I get the following error:

170-> gcc -o dummy ~/dummy.c
/tmp/cca056781.o: In function `main':
/tmp/cca056781.o(.text+0x4): undefined reference to `__libc_release'
/tmp/cca056781.o(.text+0x9): undefined reference to `__libc_version'

Any ideas why this fails? In features.h, __GLIBC__ is defined. Maybe I messed
something up when I upgraded my C libs from 2.0.7pre6 to 2.1.2?

My system:

174-> uname -a
Linux boomer 2.2.13 #1 Sun Dec 26 07:35:29 EST 1999 i686 unknown

2. VME bus driver - which?

3. PPP problem to Windows server ( I guess! )

4. insmod NVdriver no workee!

5. guess i have to use Microsoft ISA server

6. Q: FreeBSD 2.2.2 cannot read multisession cdroms?

7. It's not bad canned meat...

8. From single-user-mode, using floppy, volcheck: possible?

9. Canned Proxy URLs to Filter

10. CGI: apache canned response

11. For those of you tired of canned distributions...

12. Canned "ping"