fasterdata.es.netfasterdata.es.netESnet Network Performance Knowledge Base
Performance Testing

nuttcp

Recent versions of nuttcp (versions 7.1.1+) have some new features that can be quite useful. Here is a summary of some of these.

TCP retransmit info (Linux only)

    Starting in version 6.2.7, nuttcp includes retransmit info for Linux. This is much more convenient that using tcpdump to determine if the path has lots of congestion.

server: nuttcp -S
client: nuttcp -T20 -i1 server_hostname
   0.8125 MB /   1.00 sec =    6.8150 Mbps     0 retrans
   0.7500 MB /   1.00 sec =    6.2907 Mbps     0 retrans
   0.7500 MB /   1.00 sec =    6.2907 Mbps     0 retrans
   0.8125 MB /   1.00 sec =    6.8150 Mbps     0 retrans
   0.8125 MB /   1.00 sec =    6.8150 Mbps     0 retrans
   0.7500 MB /   1.00 sec =    6.2907 Mbps    24 retrans
   0.6875 MB /   1.00 sec =    5.7665 Mbps    17 retrans
   0.8125 MB /   1.00 sec =    6.8150 Mbps    11 retrans
   1.0625 MB /   1.00 sec =    8.9119 Mbps     3 retrans
   1.1250 MB /   1.00 sec =    9.4361 Mbps     4 retrans
   1.3125 MB /   1.00 sec =   11.0088 Mbps     5 retrans
   1.3125 MB /   1.00 sec =   11.0088 Mbps     5 retrans
   1.4375 MB /   1.00 sec =   12.0573 Mbps     5 retrans
   1.3125 MB /   1.00 sec =   11.0088 Mbps     5 retrans
UDP Burst mode

    Starting with version 6.2.8, nuttcp includes a 'burst mode' for UDP, which is useful to find paths that are constrained by network devices with not enough buffering.

    e.g.: Send 300 Mbps of UDP in bursts of 20 packets for 5 seconds

 ./nuttcp -u -Ri300m/20 -i 1 -T5 nettest.lbl.gov
   35.7607 MB /   1.00 sec =  299.9241 Mbps     0 / 36619 ~drop/pkt  0.00 ~%loss
   35.7617 MB /   1.00 sec =  300.0018 Mbps     0 / 36620 ~drop/pkt  0.00 ~%loss
   35.5645 MB /   1.00 sec =  298.3730 Mbps     2 / 36420 ~drop/pkt 0.00549 ~%loss
   33.3262 MB /   1.00 sec =  279.5649 Mbps  2253 / 36379 ~drop/pkt  6.19 ~%loss
   34.6924 MB /   1.00 sec =  291.0208 Mbps  1095 / 36620 ~drop/pkt  2.99 ~%loss

  175.1836 MB /   5.00 sec =  293.7936 Mbps 99 %TX 11 %RX 3350 / 182738 drop/pkt 1.83 %loss

This amount of loss is tolerable. Now send a 300 Mbps in bursts of 50 packets:

./nuttcp -u -Ri300m/50 -i 1 -T5 nettest.lbl.gov
   23.8232 MB /   1.00 sec =  199.8394 Mbps 12238 / 36633 ~drop/pkt 33.41 ~%loss
   25.2227 MB /   1.00 sec =  211.5836 Mbps 10783 / 36611 ~drop/pkt 29.45 ~%loss
   25.1846 MB /   1.00 sec =  211.2405 Mbps 10816 / 36605 ~drop/pkt 29.55 ~%loss
   24.2969 MB /   1.00 sec =  203.8392 Mbps 11754 / 36634 ~drop/pkt 32.08 ~%loss
   25.1484 MB /   1.00 sec =  210.8927 Mbps 10864 / 36616 ~drop/pkt 29.67 ~%loss

  123.7090 MB /   5.00 sec =  207.5136 Mbps 99 %TX 12 %RX 56470 / 183148 drop/pkt 30.83 %loss

And the level of loss become excessive, showing that some device in the path needs bigger buffers. Note that bursts of > 50 packets is common with TCP over paths with a RTT > 20 ms

nuttcp also supports 3rd party mode, similar to bwctl:

 nuttcp -P 5099/5099 -T30 -i2 sendhost receivehost

10G UDP testing

nuttcp is definitely the best tool for high-speed UDP testing. To get a full 10Gbps using UDP requires the right MTU size (9K), the right packet size (8972), and an larger buffer size. For example:

nuttcp -l8972 -T30 -u -w4m -R9.9G -i1 192.168.2.11
Other documentation: