Some NAPI results

Some NAPI results

Post by jama » Fri, 05 Oct 2001 10:20:08



Since we are discussing this, heres more data ;->

---------- Forwarded message ----------
Date: Wed, 3 Oct 2001 20:42:43 +0200

Hello!

Anyway. Uppsala University (35.000 students, 4000 emploees)
has now SMP/NAPI in production.
But reniceed ksoftirqd to -10. Kernel 2.4.10poll SMP.

USER       PID %CPU %MEM  SIZE   RSS TTY STAT START   TIME COMMAND
root         1  0.0  0.0  1224   512  ?  S    18:08   0:03 init [5]
root         2  0.0  0.0     0     0  ?  SW   18:08   0:00 (keventd)
root         3 74.2  0.0     0     0  ?  RW<  18:08 100:02 (ksoftirqd_CPU0)
root         4 73.9  0.0     0     0  ?  SW<  18:08  99:39 (ksoftirqd_CPU1)
root         5  0.0  0.0     0     0  ?  SW   18:08   0:00 (kswapd)
root         6  0.0  0.0     0     0  ?  SW   18:08   0:00 (bdflush)
root         7  0.0  0.0     0     0  ?  SW   18:08   0:00 (kupdated)
root        52  0.0  0.0   796   192  ?  S    18:08   0:00 voff
root       570  0.3  7.4 38696 38184  ?  S    18:08   0:28 /sbin/gated
.

With real data packet load seems to be well distributed.
Plus some CPU TX collisions. :-)

L-uu1:/# cat /proc/net/softnet_stat
076b152d 00002382 076893e2 00000000 00000000 00000000 00000000 00000000 00060511
07678c10 0000247c 076a7e5c 00000000 00000000 00000000 00000000 00000000 00061b88

L-uu1:/# ipchains -L -n | wc -l
    465

L-uu1:/# cat /proc/net/drivers/tulip_eth0
eth0: UP Locked MII Full DuplexLink UP
Admin up    2 hour(s) 17 min 6 sec
Last input  NOW
Last output NOW
5min RX bit/s   94.5 M
5min TX bit/s   69.2 M
5min RX pkts/s  14981
5min TX pkts/s  13971
5min TX errors  0
5min RX errors  0
5min RX dropped 0
5min TX dropped 0
5min collisions 0
5min RX missed errors   0
NormalIntr       24221027
AbnormalIntr     1
RxIntr   20983232
RxNoBuf  280
RxDied   0
RxJabber         0
TxIntr   15696981
TxDied   0
TxNoBuf  8512291
TimerIntr        0
TotalIntr        23459016     packets-per-int 5
TX-frees in IRQ/xmit 56077416/66612498 TX-queue stop  3731551
Fb: no 0 low 0 mod 0 high 0 drp 0
Poll start start/pkts    1962176135/128376006
Poll exit done/not_done/oom      701671/1961474461/0
Memory allocation failure (small skb)  0

L-uu1:/# cat /proc/interrupts
           CPU0       CPU1
  0:     412299     413160    IO-APIC-edge  timer
  1:         80         96    IO-APIC-edge  keyboard
  2:          0          0          XT-PIC  cascade
  8:          0          0    IO-APIC-edge  rtc
 14:      10791      10371    IO-APIC-edge  ide0
 15:         18        106    IO-APIC-edge  ide1
 16:    8568624    8569699   IO-APIC-level  eth1
 17:   11830639   11830125   IO-APIC-level  eth0, eth2

Cheers.

                                                --ro

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

1. Info: NAPI performance at "low" loads

NAPI network drivers mask the rx interrupts in their interrupt handler,
and reenable them in dev->poll(). In the worst case, that happens for
every packet. I've tried to measure the overhead of that operation.

The cpu time needed to recieve 50k packets/sec:

without NAPI:   53.7 %
with NAPI:      59.9 %

50k packets/sec is the limit for NAPI, at higher packet rates the forced
mitigation kicks in and every interrupt recieves more than one packet.

The cpu time was measured by busy-looping in user space, the numbers
should be accurate to less than 1 %.
Summary: with my setup, the overhead is around 11 %.

Could someone try to reproduce my results?

Sender:
  # sendpkt <target ip> 1 <10..50, go get a good packet rate>

Receiver:
  $ loadtest

Please disable any interrupt  mitigation features of your nic, otherwise
the mitigation will dramatically change the needed cpu time.
The sender sends ICMP echo reply packets, evenly spaced by
"memset(,,n*512)" between the syscalls.
The cpu load was measured with a user space app that calls
"memset(,,16384)" in a tight loop, and reports the number of loops per
second.

I've used a patched tulip driver, the current NAPI driver contains a
loop that severely slows down the nic under such loads.

The patch and my test apps are at

http://www.q-ag.de/~manfred/loadtest

hardware setup:
        Duron 700, VIA KT 133
                no IO APIC, i.e. slow 8259 XT PIC.
        Accton tulip clone, ADMtek comet.
        crossover cable
        Sender: Celeron 1.13 GHz, rtl8139

--
        Manfred

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

2. server-status and load (SS) (I think?)

3. NAPI support for multiple RX rings

4. Stupid shell I/O redirection question...

5. NAPI patch against 2.4.18

6. apache 1.2.6 and jserv 1.0b3

7. NAPI eepro100 bug fixed

8. Cannot logout from script.

9. napi backport to 2.4?

10. e1000 fixes (NAPI)

11. NAPI for eepro100

12. problem with eepro100 NAPI driver

13. Broadcom/Tigon and NAPI