This page gives an overview of the structure of the code in this repository.
More information about the contents of each directory can be found in README.md files in each directory.
Scripts which are used to build the Python virtual environment needed by EAS Control, and which also build all the Docker containers which comprise the pipeline prototype.
Each Docker container is in a separate directory, with its own Dockerfile detailing the steps which need to be followed to build the container.
The container eas_base contains the core pipeline code, and also implements a few core pipeline tasks.
The other containers are all derived from eas_base, and implement tasks which need additional software to be installed. These provide a model for how to implement new pipeline tasks in their own Docker containers.
The directory contains the scripts used to deploy the prototype into a Kubernetes cluster, and to scale the deployment according to demand. It also contains code to initialise the database schema, the job queues, and for submitting jobs to the pipeline.
A brief description of all the scripts comprising EAS Control can be found here.
Storage for input files used by the prototype, for intermediate file products, and for output results.
The directory datadir_output is used to store all the file products generated by the pipeline.
The directory datadir_local is used to store local configuration settings, and the Python virtual environment used by the pipeline.
The directory datadir_input is nominally used to store input files for the pipeline to work on, though it practice it's not used by many pipeline tasks currently, since most tasks work on files generated by preceeding tasks. Currently the only pipeline task which reads data from this directory is ingest_external_lcs, which ingests the LCs generated by the lightcurve stitching working group and makes them accessible to other tasks.
This contains JSON files defining example task chains which can be run through the pipeline. Some of these task chains take a few minutes to run; others may take a couple of weeks.
Documentation lives here.
This code is developed and maintained by Dominic Ford, at the Institute of Astronomy, Cambridge.