ESnet perfSONAR services
ESnet maintains throughput and latency test hosts at ESnet points of presence (PoPs) as well as near the site network boundary of many Department of Energy facilities. The primary perfSONAR services ESnet provides are throughput testing (via bwctl) and delay/loss testing (via OWAMP). In addition, ESnet provides test DTNs with test data sets available for download via Globus, and by command-line GridFTP.
Note that while ESnet manages an extensive perfSONAR infrastructure, many other networks and institutions run their own perfSONAR infrastructures as well. This is the power of perfSONAR - network problems can be quickly diagnosed by using the resources the community collectively deploys.
Machine Connectivity and Configuration
The ESnet test servers are connected to the ESnet network via a racklan switch (1Gbps copper ports), and the throughput testers are additionally connected via 10Gbps fiber connections to the hub/site router at each PoP. The following image illustrates how this connectivity looks logically:
ESnet perfSONAR Hosts
In 2015-16 ESnet upgraded all it's perfSONAR hosts to the following configuration:
- Super X10SL7-F Motherboard
- 1 Intel Xeon E3-1275 v3 3.5GHz (Turbo 3.9GHz) (8 cores)
- 32GB DDRIII 1600 ECC Only Certified Memory
- Intel X520-SR2 10GB Dual SFP+ with Dual Optics
- Seagate 1TB SAS 6GB/s, 7200RPM ST1000NM002
These new "combined" hosts run loss, latency, and traceroute tests on the 1G NIC, and throughput tests on the 10G (or possibly 40G NIC). With a 40G NIC, these hosts can drive TCP at 39Gbps.
ESnet perfSONAR hosts
ESnet perfSONAR limits file
ESnet permits tests to ESnet throughput testers from any ESnet site, and from any scientific or research institution that is connected to the global research and education network infrastructure. This includes US laboratories and universities, as well as research laboratories and universities in Africa, Asia, Australia, Europe, and Latin America. This is accomplished by including the global R&E routing table (the set of IP prefixes accessible via peerings with R&E networks) in the /etc/pscheduler/limits.conf file (or /etc/bwctl-server.limits for perfSONAR v3.x) used to control test access to the ESnet throughput test hosts. Other logical groups of addresses are added to the file from time to time as the needs of the scientific community evolve. The perfSONAR limits page has more details. Note that UDP testing to ESnet hosts is limited to 100Mbps.
Managing local packet filters
If your site maintains ACLs, follow the list of subnets that ESnet uses.
perfSONAR measurement archives
There is a list of ESnet measurement archives at the following URL: http://stats.es.net/ServicesDirectory/