Guide to Bulk Data Transfer over a WAN

Workshops
Search
Host Tuning

To obtain the maximum possible TCP throughput, it is usually necessary to tune the TCP settings. In order to tune the host, one must understand a bit about TCP buffers. Please read our short overview of TCP buffers. A complete explanation is available in the PSC TCP Tuning Guide.

Below is a brief summary of how to check and set the maximum TCP buffer size. These examples set the maximum buffer size to 4 MB, which is a reasonable value for most of today's high-speed networks.

For Linux/FreeBSD/MacOSX/Solaris, you can run this script to check your TCP buffer size.

More details on how to determine the best buffer size and other host tuning options are described in the TCP Tuning Guide.

Linux:

    To check the maximum buffer size, type the following, and make sure the values are at least 4 MB:

    sysctl -a | grep "net.core.*max" 
    sysctl -a | grep "net.ipv4.*mem" 
            

    To set the maximum buffer size, add these lines to the file /etc/sysctl.conf, and then run "sysctl -p".

    # increase TCP max buffer size
    net.core.rmem_max = 4194304
    net.core.wmem_max = 4194304
    # increase Linux autotuning TCP buffer limits
    net.ipv4.tcp_rmem = 4096 87380 4194304 
    net.ipv4.tcp_wmem = 4096 65536 4194304
        

Windows XP:

    The easiest way to tune TCP under Windows XP is to get DrTCP from "DSL Reports". Set the "Tcp Receive Window" to your computed BDP (e.g. 4000000), turn on "Window Scaling" "Selective Acks", and "Time Stamping".

    A more powerful (and more complicated) tool for tuning Windows XP TCP is the SG TCP Optimizer.

Windows Vista:

    Default maximum TCP buffer in Vista is 16 MB, so no tuning is needed.

Mac OSX and FreeBSD:

    To check the maximum buffer size, type the following, and make sure the values is at least 4 MB:

    sysctl kern.ipc.maxsockbuf 
            

    To set the maximum buffer size, add these lines to the file /etc/sysctl.conf:

    kern.ipc.maxsockbuf=4194304
        
    and then run:
    sysctl -w kern.ipc.maxsockbuf=4194304
        

Solaris:

    To check the maximum buffer size, type the following, and make sure the values are at least 4 MB:

    ndd /dev/tcp tcp_max_buf 
    ndd /dev/tcp tcp_cwnd_max 
        

    To set the maximum buffer size, create and execute a boot script with the following lines:

    ndd -set /dev/tcp tcp_max_buf 4194304
    ndd -set /dev/tcp tcp_cwnd_max 4194304
        

More detailed information on host tuning is available at http://dsd.lbl.gov/TCP-tuning/ and http://www.psc.edu/networking/projects/tcptune/.


Warning: TCP Buffers that are too large can cause some applications to behave poorly (typically causing sluggish interactive response) and risk running the system out of memory. Therefore one should be careful using large default socket buffers on OSes without TCP autotuning.

Also, Setting the TCP buffers to the wrong value (0: for example) can effectively disable some systems. ESnet takes no responsibility for any problems you encounter due to changing your system settings.


© 2007, ESnet