Step by Step Guide to deploying a bwctl server

The most basic perfSONAR installation (and most useful for troubleshooting) is to install a bwctl server and to register it into the perfSONAR "Global Lookup Service" so that others can find it and run tests to it. These instructions are for a Redhat/Centos Linux i386 system. Note that Centos 5.x is the only OS officially supported by perfSONAR-PS, but it should work with any Redhat-based system.

First, make sure ntp is running on your host and the clock is synced. bwtcl will fail if the client and server hosts are not in sync. You can use this command to verify that ntp is running correctly:

 /usr/sbin/ntpq -p

Next install required perfSONAR software as follows (as root):

 wget http://software.internet2.edu/Internet2-repo-0.1-2.noarch.rpm
 rpm --import http://software.internet2.edu/rpms/RPM-GPG-KEY-Internet2
 rpm -ivh Internet2-repo-0.1-2.noarch.rpm
 # add RPMforge repo for additional perl modules if necessary, for example:
 # rpm -Uhv http://apt.sw.be/redhat/el5/en/i386/rpmforge/RPMS/rpmforge-release-0.3.6-1.el5.rf.i386.rpm
 # or
 # rpm -Uhv http://apt.sw.be/redhat/el5/en/x86_64/rpmforge/RPMS//rpmforge-release-0.3.6-1.el5.rf.x86_64.rpm
 yum install perl-perfSONAR_PS-LSRegistrationDaemon
 yum install bwctl-client
 yum install bwctl-server
 yum install iperf

Note that yum will install boot scripts, creates pseudo users, etc.

The bwctld.limits file that is part of the distribution allows testing from anywhere. Some users find this default is too open for their site, and would prefer to restrict access to a smaller community. You should read the bwctld.limits file, and its associated man page to understand how to configure your system.

ESnet uses a bwctld.limits file that some sites may find useful. This file is based on the routing table and is updated regularly. It implements the following general policies.

• Allow unrestricted UDP tests from ESnet test system prefixes.
• Allow up to 200Mbps UDP tests from ESnet sites.
• Deny UDP tests from any other locations.
• Allow TCP tests from IPV4 addresses in the global Research and Education community routing table.
• Deny TCP tests from everywhere else.

To use the ESnet bwctld.limits file, get this file from ESnet as follows:

 cd /etc/bwctld
 mv bwctld.limits bwctld.limits.dist
 wget http://stats1.es.net/sample_configs/bwctld.limits

Next start bwctld:

 /etc/init.d/bwctld start

To test that it is running correctly, log into another host, install the bwctl client and iperf, then run:

 bwctl -c yourhost -f m -x

Next install a cron job to update limits file once per day from the master version at ESnet. This will insure you only accept bwtcl connections from other Research and Education networks. Note that the file is updated on the ESnet server between 20:00 and 21:00 Pacific Time, so it is best to schedule your cron job outside that time window:

 cd /etc/bwctld
 wget http://stats1.es.net/sample_configs/update_limits.sh
 chmod +x update_limits.sh
 crontab -e
  add this line:
   1 1 * * * (/etc/bwctld/update_limits.sh) > /var/log/update_limits.log  2>&1

The next step is to configure and start the ls_registration_daemon. This will register your server in the Global Lookup Service:

 cd /opt/perfsonar_ps/ls_registration_daemon/etc
 mv ls_registration_daemon.conf ls_registration_daemon.conf.dist
 wget http://stats1.es.net/sample_configs/ls_registration_daemon.conf

Edit ls_registration_daemon.conf to include your host info, then start the daemon:

 /etc/init.d/ls_registration_daemon start

Look at /var/log/perfsonar/ls_registration_daemon.log for any error messages.

To verify that registration worked, look for your host here a couple hours after starting ls_registration_daemon: http://stats1.es.net/cgi-bin/directory.cgi.

It is important to make sure that your host is properly tuned for maximum TCP performance on the WAN. You should verify that cubic, not reno, is the default TCP congestion control algorithm, and that the maximum TCP buffers are big enough for your paths of interest. For more information see: http://fasterdata.es.net/TCP-tuning/linux.html.

Note: Sites with firewalls will also need to open the ports defined by iperf_port and peer_port in /etc/bwctld/bwctld.conf (5001-5050 and 6001-6050), and also port 4823 for bwctl and port 8095 for the Lookup Service. See firewall rules for more information.

For more information see: psps.perfsonar.net and www.perfsonar.net.