ESnet 40G perfSONAR Host

ESnet has constructed a perfSONAR host with a 40Gbps NIC.  The specs for this host are:

  • Super X10SL7-F Motherboard
  • 1 Intel Xeon E3-1275 v3 3.5GHz (Turbo 3.9GHz) Quad Cores
  • Mellanox ConnectX-3 40G ethernet NIC
  • 32GB DDRIII 1600 ECC Only Certified Memory
  • Seagate 1TB SAS 6GB/s, 7200RPM ST1000NM002

The tuning settings for this host are:

Add to /etc/sysctl/conf
# increase TCP max buffer size setable using setsockopt() net.core.rmem_max = 536870912 net.core.wmem_max = 536870912 # increase Linux autotuning TCP buffer limit net.ipv4.tcp_rmem = 4096 87380 268435456 net.ipv4.tcp_wmem = 4096 65536 268435456 # don't cache ssthresh from previous connection net.ipv4.tcp_no_metrics_save = 1   Add to /etc/rc.d/rc.local:
/bin/cpupower frequency-set -g performance
#not needed in CentOS7
#/sbin/ifconfig eth3 txqueuelen 10000

Note that this host only has a single CPU socket, so IRQ and core selection tuning is not required.