Home
Prerequisite: You have an understanding of Clear Linux and using the mixing process to create your own Clear Linux based distribution (distro) that integrates custom software and content. More information can be found on How to Clear.
A Downstream Clear is a Clear Linux based distribution, in other words: an independent OS, that follows the Clear Linux architecture and can optionally reuse Clear Linux content as part of its releases.
Sustaining your custom Clear Linux based distro after it has been built and acting as an Operating System Vendor (OSV) can be a significant commitment. There are pieces of infrastructure that can be deployed as part of a system to help automate the life-cycle of your Clear Linux based distro.
The clr-distro-factory is a set of tools intended to automate part of this life-cycle. To better understand what those tools are and how to use it, let's first understand what it takes to run a minimal 'DevOps' for a Clear Linux-based distro.
Developing and Maintaining a Clear Linux based OS
The complete process required to sustain a Clear Linux-based distro can be divided in two major workflows, each one with its own challenges and life-cycle: Content Workflow and (Distro) Release Workflow.
There are multiple ways to execute upon the goals of each workflow. Teams maintaining Clear Linux based distros can pick whichever tools they see fit to achieve those goals. The amount of content being managed and intended usage of the final distribution are the main drivers on deciding which tools to pick for each job. So let's first understand what those workflows aim to achieve and then discuss on how to implement one possible solution.
The Content workflow is composed of all the processes and methodologies that are used to handle the content of the distribution, which goes from detecting a new release from a software project all the way to have it built in the form of an RPM package file. Clear Linux uses RPM files as an intermediary artifact to track software dependencies, therefore, those artifacts will serve as one of the inputs for the next step of the process.
The Distro Release Workflow, Release Workflow for short, is the set of processes and methodologies used to gather a given set of content in the form of RPMs, transform it into a form that is suitable for 'swupd' to consume once hosted on a web server, allowing that deployed instances of your distro (clients) connect to your web server and update their OS. The Release Workflow is also responsible for the generation of metadata about each release, allowing clients to know what is inside a given release among other uses. Still in its scope, is the creation of images based on your latest release to allow for the installation and usage of your distro.
One possible solution that fulfill The Big Picture requirements is the following:
The clr-distro-factory project provides the Release Workflow implementation for the solution proposed above. It is intended to be driven by Jenkins but that is a soft requirement, any CI/CD solution should do since reliance on Jenkins is kept at minimum (one can even run it manually from a bash shell).
The code is organized as the following:
- Root folder contains code that is used across pipelines
- Each folder represents one pipeline
- Each script within the folder is either a stage or step of that pipeline
- Each folder contains a pipeline file that describes the sequence on which the stages and steps needs to be executed
The code style can be found on Shell-Code-Style-and-Considerations and code quality is also enforced by shellcheck (via make check).
One of the goals of clr-distro-factory is to provide a uniform way to execute the Release Workflow that can be shared among multiple instances, therefore there is the need to split process ('How') from the data ('What'). So the first act of each pipeline on clr-distro-factory (the 'How') is to fetch what it is called a clr-distro-factory-config repository (the 'What'). The clr-distro-factory-config is a git repository that contains all the data needed by the workflow to run and is unique to each downstream instance. It contains information like the downstream name, infrastructure host names, where to find bundles definitions etc. The clr-distro-factory-config needs to be based on this template. Every time clr-distro-factory is released, a matching release is also made at clr-distro-factory-config, so users know what data is needed in order to run a given release of clr-distro-factory.
The canonical way to use clr-distro-factory is to create a Jenkins Pipeline job for each Pipeline listed below. For the pipeline configuration, set Pipeline script from SCM -> Git, point to clr-distro-factory repository and pick one release as 'Branch to build' (i.e. refs/tags/clr-distro-factory-<version>). For the Script path, set the path to the appropriate pipeline file.
For specific settings of each Pipeline, refer to their specific documentation.
When it comes to 'DevOps' there is no "one size fits all." QA is usually the main example of this. Teams usually implement QA in different ways with different tools (that is why there is no QA stage enforced on this repository). That was one of the main drivers to keep each stage the most isolated as possible from others, allowing for the most reuse of the common parts of this project on deployments that deals with different requirements.
It should be easy to keep track of improvements on clr-distro-factory while adding custom steps and stages to its pipelines. To do so, the only thing that is required is for one to use a fork clr-distro-factory instead of the main repository, adding changes on top of a given release and rebasing it after each release.