ESnet 40G perfSONAR Host
ESnet has deployed a perfSONAR host in Boston, MA with a 40Gbps NIC.
NOTE WELL - this host is experiencing hardware problems, and is functioning poorly and/or intermittently.
The specs for this host are:
- Super X10SL7-F Motherboard
- 1 Intel Xeon E3-1275 v3 3.5GHz (Turbo 3.9GHz) Quad Cores
- Mellanox ConnectX-3 40G ethernet NIC
- 32GB DDRIII 1600 ECC Only Certified Memory
- Seagate 1TB SAS 6GB/s, 7200RPM ST1000NM002
The tuning settings for this host are:
Add to /etc/sysctl/conf
# increase TCP max buffer size setable using setsockopt() net.core.rmem_max = 536870912 net.core.wmem_max = 536870912 # increase Linux autotuning TCP buffer limit net.ipv4.tcp_rmem = 4096 87380 268435456 net.ipv4.tcp_wmem = 4096 65536 268435456 # don't cache ssthresh from previous connection net.ipv4.tcp_no_metrics_save = 1 Add to /etc/rc.d/rc.local:
/bin/cpupower frequency-set -g performance
#not needed in CentOS7
#/sbin/ifconfig eth3 txqueuelen 10000
Note that this host only has a single CPU socket, so IRQ and core selection tuning is not required.
To run tests to this host, install the perfSONAR tool suite:
rpm -hUv http://software.internet2.edu/rpms/el7/x86_64/main/RPMS/perfSONAR-repo-0.8-1.noarch.rpm
yum -y install perfsonar-tools
bwctl -c bost-pt2.es.net -T iperf3 -w 256M -t30