Installation

These installation instructions are meant for users to get up and running with LLAMA with minimal effort. See the developer instructions for information on developer dependencies and tools as well as further background documentation.

System Requirements

Make sure you have at least 4GB of memory (physical or virtual; :ref:`swap space`_ is fine) and 15GB of free space on your file system.

Installing Conda

LLAMA depends on LIGO tools that are only distributed via conda, so you’ll need to install an Anaconda python distribution to get up and running (developer notes on Conda <migrating-to-conda>): Conda installs are done on a per-user basis, so you won’t need to use sudo for any of the below. Start by installing the latest version of Conda:

curl -O https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh

Log out and log back in again, then activate conda-forge and install LIGO tools:

conda activate
conda config --add channels conda-forge
# old method: use LIGO's environment
# wget -q https://git.ligo.org/lscsoft/conda/raw/master/environment-py36.yml
# conda env create -f environment-py36.yml
curl -O https://raw.githubusercontent.com/stefco/llama-env/master/llama-py36.yml
conda env create -f llama-py36.yml

Activate the LIGO virtualenv (NOTE: You will need to do this every time you want to use this python setup! Consider putting this command in your .bashrc file.):

conda activate llama-py36

Clone the LLAMA repository into ~/dev/multimessenger-pipeline:

mkdir -p ~/dev
cd ~/dev
git clone git@bitbucket.org:stefancountryman/multimessenger-pipeline.git
cd multimessenger-pipeline

Fetch all data files (make sure you have git-lfs installed)

git lfs fetch
git lfs checkout

Install dependencies:

curl -O https://raw.githubusercontent.com/stefco/llama-env/master/requirements.txt
pip install -r requirements.txt

Install the pipeline in developer mode:

python setup.py develop

Confirm installation succeeded by seeing if you can print the help command from the command-line interface (CLI):

llama --help

Optionally, run LLAMA’s test suite to make sure things are working okay (though note that many tests will fail if you haven’t entered your authentication credentials for external services):

make test

That’s it! All important llama tools can be accessed at the command line as subcommands of the llama command; run llama --help to see what’s available. The llama CLI follows the same structure as the llama python modules, which you can import into your python scripts.

Setting Up a Production Server

You can set up a production server environment on a Debian server as follows. You can skip the installation steps for Docker and Docker Compose if you already have them installed.

Install Docker

The LLAMA production environment runs in a Docker container to massively simplify deployment and reproducibility. On Debian:

curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add -
apt-key fingerprint 0EBFCD88
|
add-apt-repository \
    "deb [arch=amd64] https://download.docker.com/linux/debian \
    $(lsb_release -cs) \
    stable"
apt-get update
apt-get install -y docker-ce docker-ce-cli containerd.io
docker run hello-world

Install Docker Compose

We use Docker Compose to turn on all LLAMA components at once and to keep them running. Install Docker Compose with:

curl -L \
    "https://github.com/docker/compose/releases/download/1.24.1/docker-compose-$(uname -s)-$(uname -m)" \
    -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
docker-compose --version

Log in to Docker Cloud

The LLAMA production docker images are in a private repository on Docker Cloud. You will need to be added to the stefco/llama Docker Cloud repository (contact Stefan Countryman for details) and log in on the production server using docker login:

docker login

You’ll be promted for your Docker Cloud login credentials; enter them to log in.

Get docker-compose.yml

docker-compose.yml is a file that describes how to combine Docker containers together; since LLAMA has a few moving parts that need to work together, it saves us the trouble of having to remember what steps are necessary to turn the pipeline on and keep it running. You can think of it as a shortcut for calling a ton of docker commands every time we want to start up our app.

You can always pull the latest version of docker-compose.yml from the LLAMA repository with this command:

pushd ~ && git archive \
        --remote=git@bitbucket.org:stefancountryman/multimessenger-pipeline.git \
        HEAD docker-compose.yml \
    | tar -x && popd

Starting LLAMA Production App

Fortunately, all the hard work is done for you in docker-compose.yml; you just need to start it with docker-compose up:

docker-compose up