Menu
Home » Host Tuning » Linux » 100G+ Network Tuning » Other Tuning
Host Tuning

Other Tuning

November 5, 2024

This page contains a collection of other 100G tuning options. In our testing on ESnet hardware, these settings did not always help, especially on a LAN. But we include them here in case you wish to experiment with them.  Some of these settings might be useful with older OSes. Please let us know your results if they work for you.

We recommend leaving these at the default settings, and none of these seem to impact performance on newer versions of Linux:

  • Interrupt Coalescence (ethtool -c)
    • Should be on by default, no need to change timing values rx-usecs or tx-usecs
  • LRO and GRO (ethtool -k)
    • defaults (LRO = off, GRO = on) work best
  • increase txqueuelen: (/sbin/ip link set dev $ETH txqueuelen 10000)
    • default of 1000 works fine on a LAN, but 10000 usually helps on a WAN
  • Increasing net.core.netdev settings (in /etc/sysctl.conf) also sometimes help on a WAN. But we've also seen cases where changing these value decreases throughput considerably, so use with caution!

    • Default 'netdev' values are the following:

       net.core.netdev_budget=300 
 net.core.netdev_budget_usecs=2000 
 net.core.netdev_max_backlog=1000

      We've seen throughput improvements on some hosts with the following settings:

      net.core.netdev_budget=600 
      net.core.netdev_budget_usecs=4000 
      net.core.netdev_max_backlog=250000

        Or with higher latency paths, these settings may improve throughput even more:

      net.core.netdev_budget=8000 
      net.core.netdev_budget_usecs=100000 
      net.core.netdev_max_backlog=1000000

We recommend playing with these values in your environment and see what works best for you.

Other Tuning