Some DTN designees are meant to be 'enclosed', and have a large amount of local storage to facilitate sending and receiving files:
Other designs are possible, namely by adding in additional connections that lead to other forms of storage. In the examples below we can utilize storage provided by a dedicated Storage Area Network (SAN), or a parallel file system:
Adding this functionality to a DTN increases the utility. In an environment where a filesystem is shared between a computing resource and data transfer nodes, the act of transferring files places the data in a location that does not require additional copy steps:
Security can be applied to each component independently. In our example above, the DTNs may be exposed to the world via a Science DMZ architecture and could be restricted in terms of the ports that are opened, and the subnets used for communication. The filesystem could be behind an institutional firewall, and have settings applied to allow for direct communication with login nodes, processing nodes, and the DTNs:
Data flow between DTNs (via Globus) is described below. These interactions are described in several well known port ranges and communication between other DTNs and a control channel to the Amazon cloud.
Filesystem Tuning Guides
Filesystems require extensive tuning to keep up with the speed of the DTN. If this tuning is not performed, it can become a bottleneck for the data transfer activity. The following guides discuss ways to tune filesystems:
Contact [email protected] if you have updates or corrections for information on fasterdata.