Solaris 2.2 tcp/ip performance

Solaris 2.2 tcp/ip performance

Post by Casper H.S. D » Sun, 20 Jun 1993 06:57:20



When we first had Solaris 2.1, I did a few TCP/IP performance
experiments with ttcp (SunOS 4 binary) and concluded that it wasn't
bad but that it wasn't good.

Today I repeated the experiments. The results really surprise me,
in a pleasant way.

The following numbers were obtained with ttcp, 16MB data
16k buffer size in SunOS 4, no bffersize specified in Solaris 2.2.
The machines are all identical ELCs on connected on a lightly loaded
ethernet. One of the reasons I chose ELCs is because they are relatively
slow and the SunOS 4.1.1 TCP/IP cannot fill the ethernet with an ELC.
All machines used exactly the same (SunOS 4.1.x) ttcp binary.

Results:
                        KB/s            system time     elapsed time
                                        send/recv

SunOS 4.1.1

ELC/loopback            2030            2.5/5.3         8.

ELC/ELC                 732             5.2/6.1         22.

SunOS 5.2

ELC/loopback            3396            2.0/2.1         4.8

ELC/ELC                 1005            2.9/3.8         16.3

Bonus:

SunOS 5.2 SS10/41 lo    9930            0.7/0.6         1.66

Can other people reproduce these results?

Quote:>classic quote : Solardog 2.2 is almost up to the speed of 4.1.x

Casper
 
 
 

Solaris 2.2 tcp/ip performance

Post by Steinar Ha » Sun, 20 Jun 1993 23:57:12


Quote:> ethernet. One of the reasons I chose ELCs is because they are relatively
> slow and the SunOS 4.1.1 TCP/IP cannot fill the ethernet with an ELC.
> All machines used exactly the same (SunOS 4.1.x) ttcp binary.

That seems strange. I have been able to saturate an unloaded Ethernet
quite well with both IPCs and ELCs, getting more than 1020 KByte/s with
two IPCs, somewhat better with two ELCs.

Steinar Haug, system/networks administrator
SINTEF RUNIT, University of Trondheim, NORWAY


 
 
 

Solaris 2.2 tcp/ip performance

Post by Casper H.S. D » Mon, 21 Jun 1993 00:02:54



>> ethernet. One of the reasons I chose ELCs is because they are relatively
>> slow and the SunOS 4.1.1 TCP/IP cannot fill the ethernet with an ELC.
>> All machines used exactly the same (SunOS 4.1.x) ttcp binary.
>That seems strange. I have been able to saturate an unloaded Ethernet
>quite well with both IPCs and ELCs, getting more than 1020 KByte/s with
>two IPCs, somewhat better with two ELCs.

You should not only note that the tcp/ip numbers I got for the ELCs
were higher for Solaris 2.2, but also that the Solaris 2.2 machines
did it in half the CPU time.

Someone advised me to use netperf. With the ELCs running Solaris 2.2
I consistently get:

% ./netperf -s 16384 -S 16384 -H <host> -fK
 TCP STREAM TEST
 Recv   Send    Send                          
 Socket Socket  Message  Elapsed              
 Size   Size    Size     Time     Throughput  
 bytes  bytes   bytes    secs.    KBytes/sec  

 16384  16384   16384    10.00    1058.07  

 (consistent: between 1050 and 1060 KB/s)

This is on the net that should be busier.

The SunOS 4.1.1 ELCs give me numbers that aren't as consistent and range
from 700 to 950.

My original remark stands: I believe that Solaris 2.2 TCP/IP performance
is better (by a margin) than SunOS 4.1.1. (Especially because the amount
of CPU time consumed is 50% less in Solaris 2.2. It is more efficient)
It is easier for a Solaris 2.2 ELC to achieve maximum TCP/IP bandwith
than it is for a SunOS 4.1.1 ELC. I believe that it is the CPU usage
that matters most: almost all workstations can saturate an unloaded
ethernet. What counts is the CPU cycles they have left after that.

It may well be that something is amiss with some of the hardware used in the
SunOS 4.1.1 test. (It is possible that the Solaris 2.2 ELCs live on the
same concentrator board, whereas the SunOS 4.1.1 ELCs may live on seperate
boards and that that matters in the achieved thruput)
None of this matters for the loopback TCP/IP test, where Solaris 2.2
is much faster.

SunOS 4.1.1 is more efficient when it comes to sending smaller tcp/ip
packets. The cross-over point is at about 100 bytes/packet.

If someone can give me any pointers as to how I can better compare the TCP/IP
implementations, I'm open for suggestions.

Casper

 
 
 

Solaris 2.2 tcp/ip performance

Post by Jens-Uwe Mag » Mon, 21 Jun 1993 15:32:47


I have done some datagram tests and it appears to chew more system time
than before. But this might be due to my use of scatter/gather I/O, as
I do not always have everything handy in one buffer. I am really
missing something like getmsgv/putmsgv.

-----------------------------------------------------


Friedrich-Ebert-Str. 38         Tel.: +49 511 413073

 
 
 

Solaris 2.2 tcp/ip performance

Post by Mark Boolooti » Fri, 25 Jun 1993 04:14:29



Quote:>My original remark stands: I believe that Solaris 2.2 TCP/IP performance
>is better (by a margin) than SunOS 4.1.1. (Especially because the amount
>of CPU time consumed is 50% less in Solaris 2.2. It is more efficient)
>It is easier for a Solaris 2.2 ELC to achieve maximum TCP/IP bandwith
>than it is for a SunOS 4.1.1 ELC. I believe that it is the CPU usage
>that matters most: almost all workstations can saturate an unloaded
>ethernet. What counts is the CPU cycles they have left after that.

In speaking with an individual in the performance group at Sun, I was told
that TCP/IP performance of SunOS 5.1 (and SunOS 5.2) was anywhere from equal
that of SunOS 4.x (presumably 4.1.3) to 80% of SunOS 4.x.  This was due
primarily to implementing TCP and IP as STREAMS modules, which resulted in
a net loss of performance.  I believe this information was collected for the
Sun4m architecture.

I haven't spent any time looking at the performance of these systems, but
I would recommend comparing between Solaris 2.2 and SunOS 4.1.3 (weren't there
some tweaks in the 4.1.3 network code which gave improved performance over
4.1.1?).

It is hard for me to imagine that the SunOS5.x STREAMS implementation of
TCP and IP could beat the SunOS 4.x network code.  Maybe Sun got around
to fixing up the network code.  

At any rate, I am somewhat skeptical of Sun's move to implement TCP/IP as
STREAMS modules.  Other vendors, such as SGI and Cray, although SYSV-based,
have avoided STREAMS when it came to the network code.  This was done, by
and large, for performance reasons.  

I'll be interested to see how future Sun network performance stacks up with
vendors such as SGI and DEC (anybody have FDDI numbers for SunOS 5.x?).

mb
--

Disclaimer:  booloo speaks for booloo and no other.

 
 
 

Solaris 2.2 tcp/ip performance

Post by Casper H.S. D » Fri, 25 Jun 1993 06:46:39



>In speaking with an individual in the performance group at Sun, I was told
>that TCP/IP performance of SunOS 5.1 (and SunOS 5.2) was anywhere from equal
>that of SunOS 4.x (presumably 4.1.3) to 80% of SunOS 4.x.  This was due
>primarily to implementing TCP and IP as STREAMS modules, which resulted in
>a net loss of performance.  I believe this information was collected for the
>Sun4m architecture.

There was a measurable difference between 5.1 and 5.2, so I don't
think that you can say lump 5.1 and 5.2 performance together.
5.2 was released just last month, and from what I heard their
was quite a difference between the last 5.2 betas and 5.2 itself.

Quote:>I haven't spent any time looking at the performance of these systems, but
>I would recommend comparing between Solaris 2.2 and SunOS 4.1.3 (weren't there
>some tweaks in the 4.1.3 network code which gave improved performance over
>4.1.1?).

Actuall, I did look into this. We have only a few machines running 4.1.3,
but the performance test (50-70% more thruput on the loopback interface)
hold for 4.1.3 equally.

The original tests I've run are broken, though. The concentrators we
use start flashing red light when confronted with large mounts
of traffic. Most SPARC based machien can saturate ethernet.

Quote:>It is hard for me to imagine that the SunOS5.x STREAMS implementation of
>TCP and IP could beat the SunOS 4.x network code.  Maybe Sun got around
>to fixing up the network code.  

Hard to imagine? Please, do explain. The 4.1.x networking code
isn't really worlds best. Suns performed really poor in FDDI.

Also, I do not see why STREAMS based code should be slower.
There are no extra copies of the data involved. (Data passed
down through streams modules isn't copied)

Quote:>At any rate, I am somewhat skeptical of Sun's move to implement TCP/IP as
>STREAMS modules.  Other vendors, such as SGI and Cray, although SYSV-based,
>have avoided STREAMS when it came to the network code.  This was done, by
>and large, for performance reasons.  

Presumably because the people at USL were writing a reference implementation.
Their productivity was measured in added features, not removed bugs
or improved performance. It shows. Things that are intersting from
a research point of few, like squeezing the last byte out of the bandwidth
or minimizing protocol overhead, showed up in ready to use socket
code. It was much easier for CRAY and SGI to get the socket code than
it was to reimplement those ideas in STREAMS.

Quote:>I'll be interested to see how future Sun network performance stacks up with
>vendors such as SGI and DEC (anybody have FDDI numbers for SunOS 5.x?).

The FDDI numbers will be interesting. It's easy to outperform SunOS 4.x
in that respect. You can do that with a flashlight and a piece
of fiber.

It is possible that STREAMS has a higher per packet overhead,
each packet must be handled explicitly at each layer,
but for the movement of bytes to the network layer, I've
yet to find someone to explain to me why `~STREAMS are inherently
slower for TCP/IP''.

It may be that TCP/IP is here to stay. In that case STREAMS will
by you nothing. (Or almost nothing). But if we get OSI on a large
scale, STREAMS code doesn't have to be rewritten. Socket code
will have to be changed.

Casper

 
 
 

Solaris 2.2 tcp/ip performance

Post by Rick Jon » Sat, 26 Jun 1993 04:49:57


There are all kinds of factors involved in TCP/IP performance. While a
BSD-based transport might have better CPU utilization, the
STREAMS-based transport might have bigger default windows. This is why
it is important to state as many parms as you can. It is also good to
do more than one type of test - say latency (request/response) and
throughput. Explore the entire space.

rick jones
I am in a maze of twisty little opinions, all my own...

 
 
 

1. tcp/ip hangs on Solaris 2.2

Hi netlanders,

I have several problems running Solaris 2.2 (patches installed) on a Sun 10/40.
A process listens on a socket, starts child-processes and continues listening.
All further communication on the established connection is done by the
child, until the child dies. The parent starts listening after the fork()
call and will create new children on connection requests.

On Solaris2.2, the process hangs after a few hours and does not accept
connections anymore, but I don't know why. On other OS, I never had problems
like this. And Solaris itself seems to crash later: First, no tcp/ip is
possible anymore, on the second stage I get a silent death of the kernel.

One of the interesting things is, that closed sockets sometimes remain (as
netstat displays) as long as the listening process exists, no matter if the
child is dead or not.

Has anybody ever had similar problems with Solaris? Any help would be
greatly appreciated.

Thanks in advance
Joachim
--
 ----------------------------------------------------------------------------

 BeBit Infotechnik GmbH, Mannheim.   Voice: +49 621-459-2652 Fax: 621-459-2501
 > type c) to continue, d) to dump core or b) to reboot:

2. Help C program in linux help help

3. TCP/IP address change under Solaris 2.2

4. Atomic Shared Memory Writes

5. Solaris 2.5.1 TCP/IP faces Solaris2.6 TCP/IP

6. Differences between binary & ascii modes in ftp

7. High Performance TCP/IP & Solaris 2.3

8. LINUX and Multispin 6x CD-ROM driver from NEC

9. Bad TCP/IP performance with Solaris 2.5

10. Tuning Linux 2.2 IP for coast-to-coast performance

11. TCP/IP stack for Wabi 2.2

12. what got broken in 2.2 kernel's TCP/IP?