40G/100G Tuning

For hosts with 40G/100G Ethernet NICs, there are some additional things you'll want to tune to maximize throughput.

The most important things to configure are:

• CPU governor to 'performance'
• TCP buffer size set to the maximum (2GB)
• make sure you are using the correct cores for both IRQ and user processes
• enable on 'fair queuing' (FQ) with a good pacing rate for your environment.

No other tuning should be needed for modern Linux OS's (RHEL/CentOS 7.2 and higher, or systems with a 4.x kernel).

Note: some folks have reported that FQ may decrease single stream 100G throughput. We have not yet confirmed this, but believe FQ will still help for multi-stream performance, and therefore is recommended for most use cases.

For more details on 100G tuning, see this presentation from September 2016.

CPU clock rate still matters a lot for 40G/100G flows  If you care about the throughput of single flows, a higher CPU clock rate is  important.  In general, you need a CPU clock rate of at least 3GHz to achieve 40Gbps per flow.

To find other publicly accessible 40/100G hosts to run tests with, see:

• 40G hosts
• 100G hosts