Very Computer

by **Gopal** » Tue, 08 Jan 2002 04:22:20

We have an E10k domain ( mission critical ) that had lots of stability and
performance problems. Sun has suggested to apply 106541-18 kernel patch (
stating that they have observed too many XCALs happening on our 10 cpu
domain ). But after applying this patch, the opensource LSOF got
significantly slowed down. Prior to applying this patch lsof used to run for
5-10secs and give 38000+ open file report ( when run simply as, lsof -n ).
Now it is taking more than 8minutes even when domain has no load. Under load
conditions, I could not risk running lsof.
Procfs utilities:
pfiles: when run against an oracle process, it randomly hangs. The only way
to get out of this situation is kill the pfiles command. This leaves the
target Process in a "STOP" state, and there is no means to put the stopped
process back to run state.
truss: truss frequently is* the process against which it is run ( the
process is originally run by oracle ) dumping a "Psetrun: Device busy" lines
at a rapid rate.

Overall we have observed that after this patch, system tools "find, ps etc.,
" are running longer and are taking more CPU cycles.

Did anyone encounter similar experience.

tia
Gopi.

by **Vic Abe** » Tue, 08 Jan 2002 20:59:01

>We have an E10k domain ( mission critical ) that had lots of stability and
>performance problems. Sun has suggested to apply 106541-18 kernel patch (
>stating that they have observed too many XCALs happening on our 10 cpu
>domain ). But after applying this patch, the opensource LSOF got
>significantly slowed down. Prior to applying this patch lsof used to run for
>5-10secs and give 38000+ open file report ( when run simply as, lsof -n ).
>Now it is taking more than 8minutes even when domain has no load. Under load
>conditions, I could not risk running lsof.
>Procfs utilities:
>pfiles: when run against an oracle process, it randomly hangs. The only way
>to get out of this situation is kill the pfiles command. This leaves the
>target Process in a "STOP" state, and there is no means to put the stopped
>process back to run state.
>truss: truss frequently is* the process against which it is run ( the
>process is originally run by oracle ) dumping a "Psetrun: Device busy" lines
>at a rapid rate.
>Overall we have observed that after this patch, system tools "find, ps etc.,
>" are running longer and are taking more CPU cycles.
>Did anyone encounter similar experience.

While I can't offer any insight into why the patch might be slowing
your system, I can suggest that you read section 3.2 of the lsof
FAQ. It suggests some options you can use to speed lsof response.
Using one of them might give you a clue to what is slowing your
system.

That lsof FAQ section lists the option of disabling lsof's reading
of the kernel name cache with -C as one possible way to make lsof
run faster. That may have particular relevance to your situation.

Changes in the kernel name cache made available after Solaris 8
FCS slowed lsof's processing of the kernel name cache until an lsof
modification was made available at lsof revision 4.50. It could
be that the patch you applied contained in its bundle the kernel
name cache change that slowed lsof.

I'd also suggest if you have an lsof revision earlier than 4.50,
you update to the latest lsof revision, 4.60.

Vic Abell, lsof author

Very Computer

Procfs tools and Truss utliity are broken, after Kernel patch 106541-18

Procfs tools and Truss utliity are broken, after Kernel patch 106541-18

Procfs tools and Truss utliity are broken, after Kernel patch 106541-18