Guide to Bulk Data Transfer over a WAN

Search
Quick Start Guide for ssh-based GridFTP

The ability to use sshd to provide secure authentication for GridFTP starting with in the Globus 4.2 release. You do not need to install all of Globus to use GridFTP.

Here is a 'quick start guide' to install GridFTP with ssh support only (i.e.: no X509 support). Do these steps on both the client and server hosts.

wget http://www-unix.globus.org/ftppub/gt4/4.2.1/installers/src/gt4.2.1-all-source-installer.tar.gz
tar xvfz gt4.2.1-all-source-installer.tar.gz
cd gt4.2.1-all-source-installer
./configure  --prefix /path/to/install   (ignore any java/ant warnings)
make gridftp  install  (and wait a LONG time. Very slow to build)

A faster way to install GridFTP is via the Virtural Data Toolkit (VDT). This version of GridFTP is part of the VDT Lotman package. VDT supports several flavors of Linux and Mac OSX. A list of currently supported VDT platforms is here.

To configure GridFTP for ssh authentication do the following (both client and server hosts):

setenv GLOBUS_LOCATION /path/to/install 
source $GLOBUS_LOCATION/etc/globus-user-env.csh 

On client hosts:

$GLOBUS_LOCATION/setup/globus/setup-globus-gridftp-sshftp 

On server hosts (run as root):

$GLOBUS_LOCATION/setup/globus/setup-globus-gridftp-sshftp -server 

or (non-root option)

$GLOBUS_LOCATION/setup/globus/setup-globus-gridftp-sshftp -server -nonroot 

The GridFTP server is now automatically launched via sshd. Here are some sample commands:

    # directory listing
    globus-url-copy -list sshftp://gridhost.foo.gov/tmp/
    # copy file /etc/group
    globus-url-copy sshftp://gridhost.foo.gov/etc/group file:/tmp/group
    # parallel transfer of file /tmp/mybigdatafile
    globus-url-copy -p 4 sshftp://gridhost.foo.gov/tmp/mybigdatafile file:/tmp/myfile
    # test network throughput
    globus-url-copy -vb -p 4 -tcp-bs 16M sshftp://gridhost.lbl.gov/dev/zero sshftp://myhost.bar.gov/dev/null

More information on configuring and running GridFTP.

Explaination of globus-url-copy command line options.

Starting with Globus version 4.2 , GridFTP also includes the ability to analyze disk vs. network bottlenecks using the NetLogger toolkit. For more information see: See http://www.cedps.net/index.php/Gridftp-netlogger. This version also supports a UDP-based transport option using the "-udt" flag, which might be worth trying on congested paths.


© 2008-2009, ESnet

Privacy and Security Notice