Installation

 

Installation is a three stage process.  You will need to install the MyRT run-time overlay, the MyCluster scripts, and the Condor/SGE binaries.

 

1. Install MyRT.  Uncompress/untar the latest MyRT source package, and change to the source code directory.  Run configure, make, and make install.  Prerequisite: make, gcc

 

ewalker:~$ zcat myrt-0.5.X.tar.gz | tar xvf – ewalker:~$ cd myrt-0.5.X/ ewalker:~$ ./configure ewalker:~$ make ewalker:~$ make install

 

The installation will install all binaries in $HOME/myrt by default.  It will then create a symbolic link from $HOME/.myrt_root to $HOME/myrt/bin.  You can change the location of the installation by using the –prefix flag when you invoke configure.  However, the installation will still create a symbolic link from $HOME/.myrt_root to this new location.

 

2. Install MyCluster scripts.  Uncompress/untar the latest MyCluster source package, and change to the source directory.  Run the siteinstaller.csh script.  Prerequisite: csh, bash, sed, awk, tar, gzip, perl, screen.

 

ewalker:~$ zcat mycluster-v2-X.tar.gz | tar xvf – ewalker:~$ cd mycluster-v2-X ewalker:~$ ./siteinstaller.csh

 

The installer will configure the appropriate proxy agents for your system.  These proxy agents are responsible for submitting/maintaining job proxies in your cluster.  The job proxies are used to provision CPUs into your personal clusters when you create your virtual clusters.  Therefore, if you are installing on a LSF managed cluster, you will need to select the LSF option in the agent selection menu shown below.  If you are installing on a personal workstation, or on an Amazon.com EC2 node, select the “Desktop” option in the menu.

 

During the installation, you will need to provide the installer with a directory where all the binaries/scripts will be installed.  This directory will then be the value for the mandatory $MYCLUSTER_LOCATION environment variable.

 

ewalker:~$ ./siteinstaller.csh Version 2.0.5 Created on Mon Jun 15 13:27:05 CDT 2009   Please select the agent you would like installed for provisioning CPUs:       1.  Load Sharing Facility (LSF)     2.  Portable Batch System (PBS)     3.  IBM LoadLeveler (LL)     4.  Sun Grid Engine (SGE)     5.  Desktop(EC2/SSH)       6.  Quit   (Choice)

 

3. Install Condor/SGE binaries.  The installer, siteinstaller.csh, will automatically ask if you would like to install Condor/SGE binaries.  This is for the personal Condor and SGE clusters that MyCluster creates for you.  The installation process installs a private version of Condor/SGE because it needs to create script wrappers for some of the Condor/SGE commands/daemons.  The MyCluster installation does not interfere with any previously installed versions of Condor/SGE in your environment.

 

A snapshot of the Condor/SGE installation menu is shown below.   The menu choices allow installing Condor, SGE, or both system binaries.  During the Condor and SGE installations, you will need to provide the installer with the directories where all the binaries/scripts will be installed. The supplied installation directories will then be the values for the mandatory $CONDOR_WORK_DIR and $SGE_WORK_DIR environment variables for the Condor and SGE installations respectively.

 

Please select the personal job management interface you would like installed:       1.  Condor     2.  Sun Grid Engine (SGE)     3.  Condor and SGE       4.  Quit   (Choice)

 

A screenshot of an example MyCluster installation session can be found here.

Configuration

After completing the above steps, look for $MYCLUSTER_LOCATION/Warning.log to check for any specific warnings related to your installation.

To use your installation, add $MYCLUSTER_LOCATION/mycluster.sh[.csh] to your $HOME/.bashrc[.cshrc], source your new environment, and begin using the vo-login command line tool.   You can check the manual page for the vo-login command line tool with “man vo-login”.

Using the pre-built EC2 AMI

We also provide a pre-built EC2 AMI where MyCluster V2 is already installed and configured (ami-81b252e8).  You can use this as a client node from which to create virtual clusters using nodes provisioned from the Amazon.com EC2 service, or the NSF TeraGrid.

 

In order to access the Amazon.com EC2 service, you will need to follow the installation instructions in the Amazon.com EC2 Getting Started Guide.

 

In particular, you will need to create an account with Amazon.com AWS, and install/setup the EC2 command line tools, i.e. ec2run, ec2kill, ec2din, etc.  Once this is done, you can run an instance of our pre-built AMI, and SSH into the mycluster account (password: mycluster) in the running instance.

 

ewalker:~$ ec2run ami-81b252e8 RESERVATION     r-d7ddaabe      603116669825    default INSTANCE        i-1b93ca72      ami-81b252e8    pending   …[sic]   ewalker:~$ ec2din RESERVATION     r-d7ddaabe      603116669825    default INSTANCE        i-1b93ca72      ami-81b252e8    ec2-67-202-0-81.compute-1.amazonaws.com …[sic]   ewalker:~$ ssh mycluster@ec2-67-202-0-81.compute-1.amazonaws.com mycluster@ec2-67-202-0-81.compute-1.amazonaws.com's password:

 

After you have successfully logged into the mycluster account in the running AMI instance, copy your private key and X509 certificate into $HOME/.ec2.  Then set the environment variables EC2_CERT and EC2_PRIVATE_KEY appropriately in $HOME/.bashrc file, remembering to source the new environment after doing this.  You are now able to use the ec2_pool or vo-login command as documented in the user-guide.

 

This material is based in whole or in part on work supported by the National Science Foundation (NSF) under the NSF Middleware Initiative — Grant No. OCI-0721931.  Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF.