Step by Step Guide to deploying a bwctl server
The most basic perfSONAR installation (and most useful for troubleshooting) is to install
a bwctl server and to register it into the perfSONAR "Global Lookup Service"
so that others can find it and run tests to it. These instructions are for a Redhat/Centos
Linux i386 system. Note that Centos 5.x is the only OS officially supported by perfSONAR-PS,
but it should work with any Redhat-based system.
First, make sure ntp is running on your host and the clock is synced. bwtcl
will fail if the client and server hosts are not in sync. You can use this command
to verify that ntp is running correctly:
/usr/sbin/ntpq -p
Next install required perfSONAR software as follows (as root):
wget http://software.internet2.edu/Internet2-repo-0.1-2.noarch.rpm
rpm --import http://software.internet2.edu/rpms/RPM-GPG-KEY-Internet2
rpm -ivh Internet2-repo-0.1-2.noarch.rpm
yum install perl-perfSONAR_PS-LSRegistrationDaemon
yum install bwctl-client
yum install bwctl-server
yum install iperf
Note that yum will install boot scripts, creates pseudo users, etc.
The bwctld.limits file that is part of the distribution allows testing from anywhere.
Some users find this default is too open for their site, and would prefer to restrict access to
a smaller community. You should read the bwctld.limits file, and its associated man
page to understand how to configure your system.
ESnet uses a bwctld.limits file that some sites may find useful. This file is
based on the routing table and is updated regularly. It implements the following
general policies.
-
Allow unrestricted UDP tests from ESnet test system prefixes.
-
Allow up to 200Mbps UDP tests from ESnet sites.
-
Deny UDP tests from any other locations.
-
Allow TCP tests from IPV4 addresses in the global Research and Education community routing table.
-
Deny TCP tests from everywhere else.
To use the ESnet bwctld.limits file, get this file from ESnet as follows:
cd /etc/bwctld
mv bwctld.limits bwctld.limits.dist
wget http://stats1.es.net/sample_configs/bwctld.limits
Next start bwctld:
/etc/init.d/bwctld start
To test that it is running correctly, log into another host, install the bwctl client and iperf, then run:
bwctl -c yourhost -f m -x
Next install a cron job to update limits file once per day from the master version at ESnet. This
will insure you only accept bwtcl connections from other Research and Education networks. Note
that the file is updated on the ESnet server between 20:00 and 21:00 Pacific Time, so it is
best to schedule your cron job outside that time window:
cd /etc/bwctld
wget http://stats1.es.net/sample_configs/update_limits.sh
chmod +x update_limits.sh
crontab -e
add this line:
1 1 * * * (/etc/bwctld/update_limits.sh) > /var/log/update_limits.log 2>&1
The next step is to configure and start the ls_registration_daemon. This
will register your server in the Global Lookup Service:
cd /opt/perfsonar_ps/ls_registration_daemon/etc
mv ls_registration_daemon.conf ls_registration_daemon.conf.dist
wget http://stats1.es.net/sample_configs/ls_registration_daemon.conf
Edit ls_registration_daemon.conf to include your host info,
then start the daemon:
/etc/init.d/ls_registration_daemon start
Look at /var/log/perfsonar/ls_registration_daemon.log for any error messages.
To verify that registration worked, look for your host here a couple hours after starting
ls_registration_daemon:
http://stats1.es.net/cgi-bin/directory.cgi.
It is important to make sure that your host is properly tuned for maximum TCP performance on
the WAN. You should verify that cubic, not reno, is the default TCP congestion control algorithm,
and that the maximum TCP buffers are big enough for your paths of interest. For more
information see:
http://fasterdata.es.net/TCP-tuning/linux.html.
Note: Sites with firewalls will also need to open the ports defined by iperf_port and peer_port in /etc/bwctld/bwctld.conf
(5001-5050 and 6001-6050), and also port 4823 for bwctl and port 8095 for the Lookup Service.
See firewall rules for more information.
For more information see:
psps.perfsonar.net and
www.perfsonar.net.
