Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-actions: Refactor execution container generation GH workflow; Add workflow for Ubuntu Jammy. #321

Merged
merged 22 commits into from
Mar 11, 2024
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
f7ae499
Refactor Docker image workflows and add `ubuntu-jammy` Docker image b…
junhaoliao Mar 9, 2024
eef4cbf
Reformat Docker image build workflow YAML files.
junhaoliao Mar 9, 2024
83a29d7
Move YAML files out from `.github/workflow` subdirectories to the dir…
junhaoliao Mar 9, 2024
ba3fefd
Rename YAML files extension to `.yml` for consistency.
junhaoliao Mar 9, 2024
ce1a4f5
Rename GitHub workflow YAML files extension to `.yaml` for consistency.
junhaoliao Mar 9, 2024
27a3c71
Refactor reusable CLP image build workflow to a reusable GitHub Action.
junhaoliao Mar 9, 2024
13dd153
Revert accidental changes in `clp-core-build.yaml`.
junhaoliao Mar 9, 2024
b1109b2
Move env var CONTAINER_IMAGE_REGISTRY to an optional input since GitH…
junhaoliao Mar 9, 2024
10fea7c
Update GitHub actions paths.
junhaoliao Mar 9, 2024
8560e37
Refine GitHub workflow clp-core-build.yaml trigger paths.
junhaoliao Mar 9, 2024
4fc87f0
Refactor GitHub actions for image build process.
junhaoliao Mar 9, 2024
44b2d95
Update github workflow file paths.
junhaoliao Mar 9, 2024
0eef743
Add configurable container registry credentials.
junhaoliao Mar 9, 2024
a61d5e9
Update README.md to add the new Ubuntu Jammy execution environment.
junhaoliao Mar 10, 2024
cd431d0
Refactor execution image workflow and action inputs naming.
junhaoliao Mar 11, 2024
16b8e0b
Refine paths in GitHub workflow filters.
junhaoliao Mar 11, 2024
1a6910e
Separated ubuntu_focal and ubuntu_jammy image build in GitHub workflo…
junhaoliao Mar 11, 2024
1199200
Revert Yoda conditions.
junhaoliao Mar 11, 2024
1108033
Ensure execution image builds need filter-relevant-changes.
junhaoliao Mar 11, 2024
80cf0de
Add step to "Set up Docker Buildx".
junhaoliao Mar 11, 2024
31b86d9
Update blob pattern in clp-execution-image-build workflow file.
junhaoliao Mar 11, 2024
a1aa679
Update blob pattern in `dorny/paths-filter` in clp-execution-image-bu…
junhaoliao Mar 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 57 additions & 0 deletions .github/actions/clp-execution-image-build/action.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
name: "clp-execution-image-build"
description: "Builds a container image that contains the dependencies necessary
to run the CLP package."

inputs:
containerImageRegistry:
kirkrodrigues marked this conversation as resolved.
Show resolved Hide resolved
default: "ghcr.io"
description: "Container Image Registry"
kirkrodrigues marked this conversation as resolved.
Show resolved Hide resolved
required: false
containerImageRegistryUsername:
default: "${{github.actor}}"
description: "Container Image Registry Username"
required: false
containerImageRegistryPassword:
default: ""
description: "Container Image Registry Password"
required: false
platformId:
description: "Platform ID of the container. e.g., ubuntu."
kirkrodrigues marked this conversation as resolved.
Show resolved Hide resolved
required: true
platformVersionId:
description: "Platform VERSION_ID / VERSION_CODENAME of the container.
e.g., jammy, focal."
kirkrodrigues marked this conversation as resolved.
Show resolved Hide resolved
required: true

runs:
using: "composite"
steps:
- uses: "docker/login-action@v3"
kirkrodrigues marked this conversation as resolved.
Show resolved Hide resolved
with:
registry: "${{inputs.containerImageRegistry}}"
username: "${{inputs.containerImageRegistryUsername}}"
password: "${{inputs.containerImageRegistryPassword}}"

- id: "sanitization"
shell: "bash"
run: |
# Docker doesn't support repository names with uppercase characters, so we convert the
# name to lowercase here.
echo "REPOSITORY=$(echo '${{github.repository}}' | tr '[:upper:]' '[:lower:]')" \
>> "$GITHUB_OUTPUT"

- id: "meta"
uses: "docker/metadata-action@v5"
with:
images: "${{inputs.containerImageRegistry}}/${{steps.sanitization.outputs.REPOSITORY}}\
/clp-execution-x86-${{inputs.platformId}}-${{inputs.platformVersionId}}"

- if: "github.event_name != 'pull_request' && github.ref == 'refs/heads/main'"
uses: "docker/build-push-action@v5"
with:
context: "./"
file: "./tools/docker-images/\
clp-execution-base-${{inputs.platformId}}-${{inputs.platformVersionId}}/Dockerfile"
push: true
tags: "${{steps.meta.outputs.tags}}"
labels: "${{steps.meta.outputs.labels}}"
6 changes: 4 additions & 2 deletions .github/workflows/clp-core-build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,16 @@ name: "clp-core-build"
on:
pull_request:
paths:
- ".github/actions/**"
- ".github/actions/clp-core-build/action.yaml"
- ".github/actions/clp-core-build-containers/action.yaml"
- ".github/workflows/clp-core-build.yaml"
- ".gitmodules"
- "components/core/**"
- "!components/core/tools/scripts/lib_install/macos-12/**"
push:
paths:
- ".github/actions/**"
- ".github/actions/clp-core-build/action.yaml"
- ".github/actions/clp-core-build-containers/action.yaml"
- ".github/workflows/clp-core-build.yaml"
- ".gitmodules"
- "components/core/**"
Expand Down
57 changes: 0 additions & 57 deletions .github/workflows/clp-execution-image-build.yaml

This file was deleted.

41 changes: 41 additions & 0 deletions .github/workflows/clp-execution-image-ubuntu-focal.yaml
kirkrodrigues marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: "clp-execution-image-ubuntu-focal"

on:
pull_request:
paths:
- ".github/actions/clp-execution-image-build/action.yaml"
- ".github/workflows/clp-execution-image-ubuntu-focal.yaml"
- "components/core/tools/scripts/lib_install/*"
- "tools/docker-images/clp-execution-base-ubuntu-focal/**"
push:
paths:
- ".github/actions/clp-execution-image-build/action.yaml"
- ".github/workflows/clp-execution-image-ubuntu-focal.yaml"
- "components/core/tools/scripts/lib_install/*"
- "tools/docker-images/clp-execution-base-ubuntu-focal/**"
workflow_dispatch:

concurrency:
group: "${{github.workflow}}-${{github.ref}}"
# Cancel in-progress jobs for efficiency
cancel-in-progress: true

jobs:
build:
runs-on: "ubuntu-latest"
steps:
- uses: "actions/checkout@v3"
with:
submodules: "recursive"

- name: "Workaround actions/runner-images/issues/6775"
shell: "bash"
run: "chown $(id -u):$(id -g) -R ."

- uses: "./.github/actions/clp-execution-image-build"
with:
containerImageRegistry: "ghcr.io"
containerImageRegistryUsername: "${{github.actor}}"
containerImageRegistryPassword: "${{secrets.GITHUB_TOKEN}}"
platformId: "ubuntu"
platformVersionId: "focal"
41 changes: 41 additions & 0 deletions .github/workflows/clp-execution-image-ubuntu-jammy.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: "clp-execution-image-ubuntu-jammy"

on:
pull_request:
paths:
- ".github/actions/clp-execution-image-build/action.yaml"
- ".github/workflows/clp-execution-image-ubuntu-jammy.yaml"
- "components/core/tools/scripts/lib_install/*"
- "tools/docker-images/clp-execution-base-ubuntu-jammy/**"
push:
paths:
- ".github/actions/clp-execution-image-build/action.yaml"
- ".github/workflows/clp-execution-image-ubuntu-jammy.yaml"
- "components/core/tools/scripts/lib_install/*"
- "tools/docker-images/clp-execution-base-ubuntu-jammy/**"
workflow_dispatch:

concurrency:
group: "${{github.workflow}}-${{github.ref}}"
# Cancel in-progress jobs for efficiency
cancel-in-progress: true

jobs:
build:
runs-on: "ubuntu-latest"
steps:
- uses: "actions/checkout@v3"
with:
submodules: "recursive"

- name: "Workaround actions/runner-images/issues/6775"
shell: "bash"
run: "chown $(id -u):$(id -g) -R ."

- uses: "./.github/actions/clp-execution-image-build"
with:
containerImageRegistry: "ghcr.io"
containerImageRegistryUsername: "${{github.actor}}"
containerImageRegistryPassword: "${{secrets.GITHUB_TOKEN}}"
platformId: "ubuntu"
platformVersionId: "focal"
15 changes: 12 additions & 3 deletions docs/gh-workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,10 +65,19 @@ need it for one OS since users can use the container on any OS.

This workflow builds CLP-core on macOS and runs its unit tests.

## [clp-execution-image-build](../.github/workflows/clp-execution-image-build.yaml)
## [clp-execution-image-ubuntu-jammy](../.github/workflows/clp-excution-image-ubuntu-jammy.yaml)

This workflow builds a container image that contains the dependencies necessary to run the CLP
package.
This workflow calls
[clp-execution-image-build](../.github/workflows/actions/clp-execution-image-build/action.yaml)
to build a container image that contains the dependencies necessary to run the CLP package on Ubuntu
22.04 (Jammy).

## [clp-execution-image-ubuntu-focal](../.github/workflows/clp-excution-image-ubuntu-focal.yaml)

Similar to
[clp-execution-image-ubuntu-jammy](../.github/workflows/clp-excution-image-ubuntu-jammy.yaml), this
workflow builds a container image that contains the dependencies necessary to run the CLP package on Ubuntu
20.04 (Focal).

## [clp-lint](../.github/workflows/clp-lint.yaml)

Expand Down
16 changes: 0 additions & 16 deletions tools/docker-images/clp-execution-base-focal/Dockerfile

This file was deleted.

16 changes: 0 additions & 16 deletions tools/docker-images/clp-execution-base-jammy/Dockerfile

This file was deleted.

16 changes: 16 additions & 0 deletions tools/docker-images/clp-execution-base-ubuntu-focal/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
FROM ubuntu:focal AS BASE

WORKDIR /root

RUN mkdir -p ./tools/docker-images/clp-execution-base-ubuntu-focal
ADD ./tools/docker-images/clp-execution-base-ubuntu-focal/setup-scripts ./tools/docker-images/clp-execution-base-ubuntu-focal/setup-scripts

RUN ./tools/docker-images/clp-execution-base-ubuntu-focal/setup-scripts/install-prebuilt-packages.sh

# Remove cached files
RUN apt-get clean \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

# Flatten the image
FROM scratch
COPY --from=BASE / /
16 changes: 16 additions & 0 deletions tools/docker-images/clp-execution-base-ubuntu-jammy/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
FROM ubuntu:jammy AS BASE

WORKDIR /root

RUN mkdir -p ./tools/docker-images/clp-execution-base-ubuntu-jammy
ADD ./tools/docker-images/clp-execution-base-ubuntu-jammy/setup-scripts ./tools/docker-images/clp-execution-base-ubuntu-jammy/setup-scripts

RUN ./tools/docker-images/clp-execution-base-ubuntu-jammy/setup-scripts/install-prebuilt-packages.sh

# Remove cached files
RUN apt-get clean \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

# Flatten the image
FROM scratch
COPY --from=BASE / /
Loading