Network Requirements and Expectations

Time Needed to Transfer Data

As a first step in improving your network performance, it is critical to have a baseline understanding of what speed you should expect from your network connection under ideal conditions.

The following shows how long it takes to transfer 1 Terabyte of data across various speed networks:

Data Transfer Rate Table

A full set of sample network requirements is available in this table: (download pdf): 

Expectations

Note that on a high-speed network it takes less time to transfer a terabyte of data than one might expect. For example, this table tells us that in order to transfer 10 terabyte of data in 24 hours only requires a network of 1 Gbps.

In general it is not a good idea to try to completely saturate the network, as you will likely end up causing problems for both yourself and others trying to use the same link. A good rule of thumb is that for periodic transfers it should be possible to get throughput equivalent to 1/4 to 1/3 of a shared path that has nominal background load.

For example, if you know your receiving host is connected to 1 Gbps Ethernet, then a target of speed of 250-300 Mbps is reasonable. You can adjust the number of parallel streams (as described on the data transfer tools page) that you are using to achieve this. However most DOE labs and large universities are connected at speeds of at least 10 Gbps, and most LANs are at least 1 Gbps, so if you don't get at least 200 Mbps, there may be a problem that needs to be addressed.

Also note that with network speeds of 1Gbps or higher, the disk subsystem will likely be a bottleneck.