RADICAL-Pilot requires the following packages:
- Python >= 2.7 (including development headers)
- virtualenv >= 1.11
- pip == 1.4.1
If you plan to use RADICAL-Pilot on remote machines, you would also require to setup a password-less ssh login to the particular machine. (help)
All dependencies are installed automatically by the installer. Besides that, RADICAL-Pilot needs access to a MongoDB database that is reachable from the internet. User groups within the same institution or project usually share a single MongoDB instance. MongoDB is standard software and available in most Linux distributions. At the end of this section, we provide brief instructions how to set up a MongoDB server and discuss some advanced topics, like SSL support and authentication to increased the security of RADICAL-Pilot.
To install RADICAL-Pilot in a virtual environment, open a terminal and run:
If your shell is BASH,
virtualenv --system-site-packages $HOME/ve source $HOME/ve/bin/activate pip install radical.pilot
If your shell is CSH,
virtualenv --system-site-packages $HOME/ve source $HOME/ve/bin/activate.csh pip install radical.pilot rehash
For a quick sanity check, to make sure that the the packages have been installed properly, run:
$ radicalpilot-version 0.37
The exact output will obviously depend on the exact version of RP which got installed.
** Installation is complete !**
3.3. Preparing the Environment¶
3.3.1. MongoDB Service¶
RP requires access to a MongoDB server. The MongoDB server is used to store and retrieve operational data during the execution of an application using RADICAL-Pilot. The MongoDB server must be reachable from both, the host that runs the RP application and the target resource which runs the pilots.
If you want to run your application on your laptop or private workstation, but run your MD tasks on a remote HPC cluster, installing MongoDB on your laptop or workstation won’t work. Your laptop or workstations usually does not have a public IP address and is hidden behind a masked and firewalled home or office network. This means that the components running on the HPC cluster will not be able to access the MongoDB server.
Any MongoDB installation should work out, as long as RP is allowed to create databases and collections (which is the default user setting in MongoDB).
The MongoDB location is communicated to RP via the environment variable
RADICAL_PILOT_DBURL. The value will have the form
export RADICAL_PILOT_DBURL="mongodb://user:pass@host:port/dbname" export RADICAL_PILOT_DBURL="mongodb://host:port/dbname"
Many MongoDB instances are by default unsecured, and thus do not require the
user:pass@ part of the URL. For production runs, and for runs on large
secured resources, we strongly recommend the usage of a secured MongoDB
dbname component in the database url can be any valid MongoDB database
name (ie. it musy not contain dots). RP will create that DB on the fly.
A MongoDB server can support more than one user. In an environment where
multiple users use RP applications, a single MongoDB server for all users
/ hosts is usually sufficient. We recommend the use of separate databases per
user though, so please set the
dbname to something like
Install your own MongoDB
Once you have identified a host that can serve as the new home for MongoDB, installation is straight forward. You can either install the MongoDB server package that is provided by most Linux distributions, or follow the installation instructions on the MongoDB website:
There are multiple commercial providers of hosted MongoDB services, some of them offering free usage tiers. We have had some good experience with the following:
3.3.2. Setup SSH Access to Target Resources¶
An easy way to setup SSH Access to multiple remote machines is to create a file
~/.ssh/config. Suppose the url used to access a specific machine is
email@example.com. You can create an entry in this config file as
# contents of $HOME/.ssh/config Host mach1 HostName machine.example.com User foo
Now you can login to the machine by using
ssh mach1. Please make also sure
that your ssh keys are registered on the target resources – while RP can in
principle handle password based login, the repeated prompts for passwords makes
RP applications very difficult to use.
This should return the version of the RADICAL-Pilot installation, e.g.,
If virtualenv is not installed on your system, you can try the following.
wget --no-check-certificate https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.9.tar.gz tar xzf virtualenv-1.9.tar.gz python virtualenv-1.9/virtualenv.py $HOME/ve source $HOME/ve/bin/activate
Many installation problems boil down to one of two causes: an Anaconda based Python distribution, or an incompatible version of pip/setuptools.
Many recent systems, specifically in the academic community, install Python in its incarnation as Anaconda Distribution. RP is not yet able to function in that environment. While support of Anaconda is planned in the near future, you will have to revert to a ‘normal’ Python distribution to use RP.
Python supports a large variety of module deployment paths:
pip being the most prominent ones for non-compilable
modules. RP only supports
pip, and even for pip we do not attempt to keep
up with its vivid evolution. If you encounter pip errors, please downgrade pip
$ pip install --upgrade pip==1.4.1
If you continue to encounter problems, please also fix the version of setuptools
$ pip install --upgrade setuptools==0.6c11
If you encounter any errors, please do not hesitate to contact us via the mailing list:
We also appreciate issues and bug reports via our public github tracker: