Packet Loss
Even a very small amount have packet loss can have a huge impact on performance.
Take this recent example of a bad router line card from ESnet. Using packet filter counters, we verified that we were losing on average of 1 packets out of 22000 packets, or 0.0046% of the packets in one direction.
All hosts are connected by a 10G NIC, and no paths were congested. The MTU was set to 9000bytes. Note the with default MTUs of 1500, the impact of packet loss is even greater.
We compare traditional TCP Reno, Hamilton TCP (htcp), and throughput predicted by the Mathis equation. The Mathis equation states the for TCP Reno, the maximum throughput is bound by this formula:
This figure show the performance impact of this loss with increasing latency, annotated with common ranges:
(view unannotated PDF, view larger unannotated PNG)
The figure below is a zoomed in version of the the bottom section of the plot, showing more clearly that htcp is about twice as fast as Reno (results with cubic are very similar).
As you can see the performance impact of .0046% loss is huge for high latency paths. On the 90ms path, we achieved 490Mbps vs. 8.2Gbps in the direction with no packet loss, or almost 17 times slower.
Also see: Switch's TCP Throughput Calculator
