Concepts

Pipelines

A pipeline represents some form of automation. Within Candor, the following assumptions are made about pipelines:

• A pipeline may have multiple stages, which will be (linearly) executed one after another.
• Pipeline stages all perform work in a single working directory, which gets passed from one stage to the next.
• If a stage fails, all subsequent stages are skipped and the pipeline fails.
• If the all stages pass, select files from the working directory may be permanently archived, or other actions such as updating stages can be taken.

Ecosystem

Candor consists of a single dashboard and an arbitrary amount of runners, both of which can be dockerized. Moreover, the concept of archiving artifacts is achieved by connecting both the dashboard and runners to S3 storage.

When a pipeline runs, the runner will create a container for each stage of the pipeline, and run this stage within a container. Once complete, the working directory of one stage will be pipelined to the next.

The dashboard keeps an inventory of pipelines, their builds and when to trigger them. Similarly, the dashboard also oversees the management of users and runners.

Terminology

• The dashboard: The website and underlying controller that keeps inventory of pipelines, users, runners, files, etc.
• A runner: A service that executes a pipeline.
• A pipeline: An automation that consists of linear steps, each of which execute within a container, that can end up uploading files.
• A pipeline plan: A description of the pipeline that the runner takes. It consists of steps and a description of what to archive.
• A pipeline config: A high-level overview of the pipeline that is used by the dashboard. It consists of everything present in a plan, but also additional functionality such as constraints, parameters and plugins.
• A run: An execution of a pipeline.
• An archive: A set of files that were retained after a pipeline run.
• A log: The text output (standard output, error) of a pipeline run.