Data Transfer Tools
|
The goal of this section is to help you maximize your wide-area network bulk data transfer performance by using file transfer tools that are designed to maximize network throughput. A number of tools are described below. |
Recommended ToolsFor most use cases we recommend using either Globus Online or GridFTP for reasons given in the background section. |
Sample Results: disk-to-disk testing from Berkeley, CA to Argonne, IL (near Chicago). RTT = 53 ms, network capacity = 10Gbps, RAID = 4 disks, RAID Level-0. Note that to get more than 1 Gbps (125 MB/s) disk to disk requires RAID.
| Tool | Throughput |
| scp | 140 Mbps (17.5 MB/s) |
| HPN patched scp, 1 disk | 760 Mbps (95 MB/s) |
| HPN patched scp, RAID disk |
1.2 Gbps (150 MB/s) |
| GridFTP, 1 stream, 1 disk |
760 Mbps (95 MB/s) |
| GridFTP, 1 stream, RAID disk |
1.4 Gbps (175 MB/s) |
| GridFTP, 4 streams, RAID disk |
5.4 Gbps (675 MB/s) |
| GridFTP, 8 streams, RAID disk |
6.6 Gbps (825 MB/s) |
Background
Using the right tools, you should be able to move 1 TB between sites in less than 8 hours. Read More »
