DTN Tuning

Tuning your DTN host is extremely important. We have seen overall IO throughput of a DTN more than double with proper tuning.

Tuning can be as much art as a science. Due to differences in hardware, it's hard to give concrete running advice. In general you should attempt to tune one thing at a time, and runs some benchmarks to see if it made a difference. Some sample benchmark commands are shown here.

Here are some tuning settings that we have found do make a difference. Note that you should always use the most recent version of the OS, as performance optimizations for new hardware are added to every release.

Additional information on tuning for 40G/100G/200G hosts can be found here.

Network

Network tuning is the most important thing to pay attention to. Be sure to following the the advice in our Linux Tuning Guide.

Packet Pacing

DTN testing by ESnet and several others has confirmed that you will reduce packet loss and maximize DTN throughput by using packet pacing to throttle network traffic to 80-90% of the NIC speed. For example for a 10G NIC running GridFTP doing 4 parallel streams, we strongly recommend using 'tc' to set the per-stream maximum to 2Gbps. e.g.:

/sbin/tc qdisc add dev ethN root fq maxrate 2gbit

BIOS

For older PCI gen3 and gen4-based hosts, you should enable “turbo boost”, and disable SMT/hyperthreading and node interleaving. More information on BIOS tunings are described in these documents: AMD and Intel (2.1 BIOS Recommendations). For 100G DTNs, be sure to look at our 100G Host Tuning information.

File System

We recommend using the xfs file system in Linux for DTN nodes. mkfs defaults are recommended. However, there are guidelines for matching RAID stripe unit and stripe width to optimize for performance.

RAID Controller

Different RAID controllers provide different tuning controls. Check the documentation for your controller and use the settings recommended to optimize for large file reading. You will usually want to disable any “smart” controller built-in options, as they are typically designed for different workflows.