LAPP/Galaxy Docker Appliance

WIP (Work In Progress)

Contents

1. Introduction
2. Pre-requisites
3. Usage
4. Creating Appliances
5. Service Containers
6. Galaxy Configuration
7. Docker Compose

Introduction

A LAPPS/Galaxy Docker Appliance, or simply an appliance, is a network of Docker containers running a Galaxy instance configured with a selected set of LAPPS Grid web services. Where the web services are also running in one or more other Docker containers. Typically services are organized into Docker containers by family, that is one container for StanfordNLP service, another container for Lingpipe, one for a MASC Datasource and Solr instance etc.

Pre-requisites

The make-appliance script expects the Tool Config Editor is installed and the tce script used to launch the editor is available on the $PATH. A pre-built binary distribution of the Tool Conf Editory is available here.

Groovy (2.4.x or later) is required to run the YamlBuilder.groovy script that is used to generate the docker-compose.yml file.

Usage

$> ./make-appliance identifier module [module ...]

where the identifier is just a prefix for names of Docker containers, and a module is the name of the Docker containers to include in the appliance.

EXAMPLE

$> ./make-appliance gate masc oaqa
$> docker-compose up

Creating Appliances

Assembling an appliance requires three things.

1. The Docker images. The specifications for several services are included here for reference and more will be added. However, images can be pulled from any repository, they do not need to be defined here.
2. The Galaxy customizations. Galaxy's tool_conf.xml file will need to be updated and the tool XML wrappers and scripts need to be copied to Galaxy's tool directory. Note: tools are not installed from Galaxy's Tool Shed since the tools used in an appliance do not make sense outside the context of the appliance. That is, other users would not be able to install the tools and use them so it does not make sense to deploy them to a tool shed.
3. The docker-compose.yml file needed to wire everything together.

Some scripts and tools are provided to automate as much of the task as possible, but it is not too difficult to do everything by hand.

Service Containers

The web services running in the Docker containers should be SOAP services that support the LAPP Grid API. While not a hard requirement, it does make it easier to compose workflows in Galaxy if all service speak the same APIs and data formats. Implementing a LAPPS Grid service is beyond the scope of this document, but see this tutorial for information on writing a LAPPS Grid web service. The remainder of this document assumes the Service.war file containing a service named MyCustomService has already be created.

There are two ways to run a war file in a Docker container:

1. Run the lappsgrid/tomcat7 image directly and mount the directory containing the Service.war file as /var/lib/tomcat7/webapps, or
2. Create a new Docker image based on lappsgrid/tomcat7, or any other Tomcat7 based container.

1 Running The Tomcat Container

Create a new directory and copy the Service.war file. For this example it is assumed the Service.war file has been copied to /home/user/webapp, but any local directory will do. The use the -v option to mount this as a volume in the Docker image

$> docker run -d -p 8080:8080 -v /home/user/webapp:/var/lib/tomcat7/webapps

After Docker has started the container the service can be accessed at http://localhost:8080/Service/services/MyCustomService.

Tomcat will expand and run all .war file it finds in /var/lib/tomcat7/webapps so if /home/user/webapp contains more than one .war file Tomcat will run them all.

2 Building a Docker Image

Create a Dockerfile based on the lappsgrid/tomcat7 image and copy the war file to /var/lib/tomcat7/webapps.

FROM lappsgrid/tomcat7
COPY ./Service.war /var/lib/tomcat7/webapps

See here for more information on creating Docker images.

Galaxy Configuration

TODO

1. Explain tool wrappers. See https://docs.galaxyproject.org/en/latest/dev/schema.html
2. Script used to run the tool
3. Updating the tool_conf.xml file.

Docker Compose

TODO

1. YamlBuilder.groovy