Deploy SPLAY on your PlanetLab slice

PlanetLab is a world-wide testbed used by researchers to test and evaluate distributed systems with realistic network and failures conditions. In the following tutorial we will guide you through the deployment of SPLAY daemons on your PL slice. We assume that:

  1. you're familiar with PL terminology;
  2. you have full access to your PL slice;
  3. you have uploaded your public key through the PL web form (available upon login in the PL's homepage); we assume this key is in ~/.ssh/planetlab-key
    1. be aware that the the diffusion of your key on all the machines in your slight can take up to one hour

A. Install the SPLAY Controller

Install the SPLAY controller to be used by the splayds in your slice. We suggest to install it on a rather stable machine with enough memory. This machine must have a public IP address, to be reachable by the SPLAY daemons once deployed at remote nodes on your slice. In the remainder of this tutorial, we assume the public address of the controller is splay.yourdomain.com. Start the controller using the following command, provided in the SPLAY distribution:
./controller_fork.rb 

B. PL Slice configuration

We now add a the currently alive nodes to your PL slice. You can use this simple python script to do so:

#!/usr/bin/python
import xmlrpclib
import sys
import urllib
if len(sys.argv) != 4:
	print "Usage: ./add_nodes_to_slice.py pl_username pl_pwd pl_slice_name"
	sys.exit(2)
alive_nodes_query = " http://comon.cs.princeton.edu/status/tabulator.cgi?table=table_nodeviewshort&format=nameonly&persite=1&select='resptime>0"
alive_no_problems="http://comon.cs.princeton.edu/status/tabulator.cgi?format=nameonly&table=table_nodeviewshort&select='resptime%20%3E%200%20&&%20((drift%20%3E%201m%20||%20(dns1udp%20%3E%2080%20&&%20dns2udp%20%3E%2080)%20||%20gbfree%20%3C%205%20||%20sshstatus%20%3E%202h)%20==%200)'"
query=alive_no_problems
urllib.urlretrieve(query,"nodes.txt" )
api_server = xmlrpclib.ServerProxy('https://www.planet-lab.org/PLCAPI/')
auth = {}
auth['Username'] = sys.argv[1] 
auth['AuthString'] = sys.argv[2] 
auth['AuthMethod'] = "password"
node_list = [line.strip() for line in open("nodes.txt")]
api_server.AddSliceToNodes(auth, sys.argv[3], node_list)

You must provide your PL's username,password (the same ones you use to log into PlanetLab website) and your slice's name as input values. Assuming your slice name is pl_slice_name, you can save the script above into a file add_nodes_to_my_slice.py and execute it from no particular location:

chmod u+x add_nodes_to_my_slice.py
./add_nodes_to_my_slice.py my_pl_name my_pl_pwd  pl_slice_name

Once this script finishes, the file nodes.txt will provide you the list of nodes currently assigned in your slice. To further test your SSH configuration, you can execute the following command, that will try to SSH to the first element in the list:

ssh -o "StrictHostKeyChecking no" -i ~/.ssh/planetlab-key pl_slice_name@`head -n 1 nodes.txt`
[pl_slice_name@planet-lab2 ~]$ 

C. Deployment

There are two files to be made available via a public address, to be download by all the machines in your PL slice. The first is available here. This archive provides all the required libraries and executables to be used on PL. Please note that the default limits for the daemons are very restrictive: if you wish to modify them, unzip the archive, edit the file splayd_planetlab/splayd/settings.lua accordingly.

You must must now upload the file splayd_planetlab.tar.gz to a public web server: this file will be downloaded by all the nodes in your PL slice. In the remainder of this tutorial, we assume the archive has been uploaded and made available through the address http://splay.yourdomain.com/splayd_planetlab.tar.gz.

The second required file is the following bash script. Remember to edit the values of controller_address and pl_slice_name (lines 2 and 3):

#!/bin/bash
controller_address="splay.yourdomain.com"
pl_slice_name="my_slice"
 
killall master
killall splayd
killall jobd
sleep 1
sudo killall -9 master
sudo killall -9 splayd
sudo killall -9 jobd
 
echo "SSLFIX"
if [ ! -e /lib/libssl.so.5 ] ; then
  if [ -e /lib/libssl.so.6 ] ; then
    sudo ln -s /lib/libssl.so.6 /lib/libssl.so.5
  fi
fi
 
if [ ! -e /lib/libcrypto.so.5 ] ; then
  if [ -e /lib/libcrypto.so.6 ] ; then
    sudo ln -s /lib/libcrypto.so.6 /lib/libcrypto.so.5
  fi
fi
 
echo "CLEAN"
 
rm install.sh
sudo rm -fr splayd_planetlab
rm  splayd_planetlab.tar.*
 
rd=$RANDOM
wget "http://$controller_address/splayd_planetlab.tar.gz?${rd}"
tar xpvf splayd_planetlab.tar.gz
if [[ $? != 0 ]]; then
  sleep 5
  rm splayd_planetlab.tar.gz
  wget "http://$controller_address/splayd_planetlab.tar.gz?${rd}"
  tar xpvf splayd_planetlab.tar.gz
fi
 
hostname=`hostname`
l=${#hostname}
settings=/home/${pl_slice_name}/splayd_planetlab/splayd/settings.lua
sed -i -e "s/KEY/$hostname/" -e "s/NAME/$hostname/" $settings
let "port = 11000 + $RANDOM % 10"
sed -i -e "s/PORT/$port/" -e "s/CONTROLLER/$controller_address/" $settings
echo "BOOT (NG)"
#if ! grep "run.sh" /etc/rc.local > /dev/null ; then
echo "echo '#!/bin/sh' > /etc/rc.local" > local
echo "echo 'touch /var/lock/subsys/local' >> /etc/rc.local" >> local
echo "echo 'su -l $pl_slice_name -c /home/$pl_slice_name/splayd_planetlab/run.sh'  >> /etc/rc.local" >> local
sudo bash local
rm local
#fi
echo "RUN"
cd /home/${pl_slice_name}/splayd_planetlab
./run.sh
sleep 1
a=`ps aux`
if echo $a | grep "splayd" > /dev/null ; then
  echo "RUNNING"
else
  echo "PROBLEM"
fi

Save the content of this script in a file called local_install.sh. You must upload it to your public web server. In the remainder of this tutorial, we assume the file has been uploaded and made publicly available at the address http://splay.yourdomain.com/local_install.sh.

Then, you need to save the content of the following bash script into a file called host_install.sh:

#!/bin/bash
url="http://splay.yourdomain.com/"
script="local_install.sh"
if [[ $1 == "" ]]; then
	echo "${0} <hostname> [<install type>]"
	exit
fi
h=$1
if [[ $2 == "" ]]; then
	type="standard"
else
	type=$2
fi
if [[ $type != "standard" && $type != "dev" ]]; then
	echo "install type not recognized"
	exit
fi
# $RANDOM to protect against proxy/cache
if [[ $type == "standard" ]]; then
	script_t="local_install.sh?$RANDOM"
	slice="my_slice"
fi
echo "Install type: ${type} on ${1} (${slice})"
ssh -o StrictHostKeyChecking=no -i ~/.ssh/planetlab-key ${slice}@$h "rm -f ${script}; wget ${url}${script_t}; mv ${script_t} ${script}; chmod 755 ${script};./${script}"

Finally the last script, to be saved in to a file called pl_install.sh:

#!/bin/bash
if [[ $1 == "" ]]; then
	echo "${0} <node file> [<install type>]"
	exit
fi
if [[ $2 == "" ]]; then
	type="standard"
else
	type=$2
fi
if [[ $type != "standard" && $type != "dev" ]]; then
	echo "install type not recognized"
	exit
fi
echo "Install type: ${type}"
rm -fr logs
mkdir logs
rm -f my_known_hosts
touch my_known_hosts
for h in `cat $1`; do
	./host_install.sh $h $type > logs/$h.log 2>&1 &
	sleep 1
done
echo FINISHED
If you've followed all the steps correctly so fare, you should have in the same directory three files:
local_install.sh
nodes.txt
pl_install.sh
If this the case, you can finally go ahead to deploy SPLAY on all the nodes on your PL slice:
chmod u+x pl_install.sh
./pl_install.sh nodes.txt
VoilĂ  !

Comments, questions and suggestions are welcome: info@splay-project.org