Skip to content

Commit

Permalink
Create GitHub action to automate CodeFlare operator release
Browse files Browse the repository at this point in the history
  • Loading branch information
sutaakar committed Jun 23, 2023
1 parent 9a7746b commit 0464938
Show file tree
Hide file tree
Showing 4 changed files with 139 additions and 54 deletions.
130 changes: 108 additions & 22 deletions .github/workflows/tag-and-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,35 +7,75 @@ on:
version:
description: 'Tag to be used for operator image'
required: true
default: '0.0.0-dev'
default: 'v0.0.0-dev'
replaces:
description: 'The previous semantic version that this tag replaces.'
required: true
default: '0.0.0-dev'
default: 'v0.0.0-dev'
mcad-version:
description: 'Published version of multi-cluster-app-dispatcher'
required: true
default: 'v0.0.0-dev'
codeflare-sdk-version:
description: 'Published version of CodeFlare-SDK'
required: true
default: 'v0.0.0-dev'
instascale-version:
description: 'Published version of InstaScale'
required: true
default: 'v0.0.0-dev'
is-latest:
description: 'Select if the built image should be tagged as latest'
required: true
type: boolean
branch:
description: 'Branch to be used for tag and release'
required: true
default: 'main'
quay-organization:
description: 'Quay organization used to push the built images to'
required: true
default: 'project-codeflare'
community-operators-prod-fork-organization:
description: 'Owner of forked community-operators-prod repository used to push bundle files to'
required: true
default: 'project-codeflare'
community-operators-prod-organization:
description: 'Owner of target community-operators-prod repository used to open a PR against'
required: true
default: 'redhat-openshift-ecosystem'

jobs:
push:
runs-on: ubuntu-latest

# Permission required to create a release
permissions:
contents: write

steps:
- uses: actions/checkout@v3
with:
ref: "${{ github.event.inputs.branch }}"

- name: Verify that release doesn't exist yet
shell: bash {0}
run: |
gh release view ${{ github.event.inputs.version }}
status=$?
if [[ $status -eq 0 ]]; then
echo "Release ${{ github.event.inputs.version }} already exists."
exit 1
fi
env:
GITHUB_TOKEN: ${{ github.TOKEN }}

- name: Activate cache
uses: actions/cache@v3
with:
path: /cache
key: ${{ runner.os }}-cache-${{ hashFiles('**/go.sum', '.pre-commit-config.yaml') }}

- name: Create tag
uses: actions/github-script@v6
with:
script: |
github.rest.git.createRef({
owner: context.repo.owner,
repo: context.repo.repo,
ref: 'refs/tags/${{ github.event.inputs.version }}',
sha: context.sha
})
- name: Install operator-sdk
run: make install-operator-sdk

Expand All @@ -46,18 +86,64 @@ jobs:
password: ${{ secrets.QUAY_TOKEN }}
registry: quay.io

- name: Image Build
- name: Image Build and Push
run: |
make build
make bundle
make image-build -e IMG=quay.io/project-codeflare/codeflare-operator:${SOURCE_TAG}
podman tag quay.io/project-codeflare/codeflare-operator:${SOURCE_TAG} quay.io/project-codeflare/codeflare-operator:latest
make image-build -e IMG=quay.io/${{ github.event.inputs.quay-organization }}/codeflare-operator:${{ github.event.inputs.version }}
make image-push -e IMG=quay.io/${{ github.event.inputs.quay-organization }}/codeflare-operator:${{ github.event.inputs.version }}
- name: Image Push as latest tag
if: ${{ inputs.is-latest }}
run: |
podman tag quay.io/${{ github.event.inputs.quay-organization }}/codeflare-operator:${{ github.event.inputs.version }} quay.io/${{ github.event.inputs.quay-organization }}/codeflare-operator:latest
make image-push -e IMG=quay.io/${{ github.event.inputs.quay-organization }}/codeflare-operator:latest
- name: Build bundle and create PR in OpenShift community operators repository
run: |
git config --global user.email "[email protected]"
git config --global user.name "CodeFlare CI"
make openshift-community-operator-release
env:
SOURCE_TAG: ${{ github.event.inputs.version }}
VERSION: ${{ github.event.inputs.version }}
PREVIOUS_VERSION: ${{ github.event.inputs.replaces }}
INSTASCALE_VERSION: ${{ github.event.inputs.instascale-version }}
MCAD_VERSION: ${{ github.event.inputs.mcad-version }}
GH_TOKEN: ${{ secrets.GH_PAT }}
IMAGE_ORG_BASE: quay.io/${{ github.event.inputs.quay-organization }}
OPERATORS_REPO_FORK_ORG: ${{ github.event.inputs.community-operators-prod-fork-organization }}
OPERATORS_REPO_ORG: ${{ github.event.inputs.community-operators-prod-organization }}

- name: Adjust Compatibility Matrix in readme
run: |
sed -i -E "s/(.*CodeFlare Operator.*)v[0-9]+\.[0-9]+\.[0-9]+(.*)/\1${{ github.event.inputs.version }}\2/" README.md
sed -i -E "s/(.*Multi-Cluster App Dispatcher.*)v[0-9]+\.[0-9]+\.[0-9]+(.*)/\1${{ github.event.inputs.mcad-version }}\2/" README.md
sed -i -E "s/(.*CodeFlare-SDK.*)v[0-9]+\.[0-9]+\.[0-9]+(.*)/\1${{ github.event.inputs.codeflare-sdk-version }}\2/" README.md
sed -i -E "s/(.*InstaScale.*)v[0-9]+\.[0-9]+\.[0-9]+(.*)/\1${{ github.event.inputs.instascale-version }}\2/" README.md
- name: Adjust MCAD and InstaScale dependencies in the code
run: |
sed -i -E "s/(.*MCAD_VERSION \?= )v[0-9]+\.[0-9]+\.[0-9]+(.*)/\1${{ github.event.inputs.mcad-version }}\2/" Makefile
sed -i -E "s/(.*INSTASCALE_VERSION \?= )v[0-9]+\.[0-9]+\.[0-9]+(.*)/\1${{ github.event.inputs.instascale-version }}\2/" Makefile
sed -i -E "s|(.*quay.io/project-codeflare/mcad-controller:release-)v[0-9]+\.[0-9]+\.[0-9]+(.*)|\1${{ github.event.inputs.mcad-version }}\2|" config/internal/mcad/deployment.yaml.tmpl
sed -i -E "s/(.*instascale-controller:)v[0-9]+\.[0-9]+\.[0-9]+(.*)/\1${{ github.event.inputs.instascale-version }}\2/" controllers/testdata/instascale_test_results/case_1/deployment.yaml
sed -i -E "s/(.*instascale-controller:)v[0-9]+\.[0-9]+\.[0-9]+(.*)/\1${{ github.event.inputs.instascale-version }}\2/" controllers/testdata/instascale_test_results/case_2/deployment.yaml
- name: Commit readme changes back to repository
uses: stefanzweifel/git-auto-commit-action@v4
with:
commit_message: Update dependency versions for release ${{ github.event.inputs.version }}
file_pattern: 'README.md controllers/defaults.go *.yaml *.tmpl Makefile'
branch: "${{ github.event.inputs.branch }}"

- name: Image Push
- name: Creates a release in GitHub
run: |
make image-push -e IMG=quay.io/project-codeflare/codeflare-operator:${SOURCE_TAG}
make image-push -e IMG=quay.io/project-codeflare/codeflare-operator:latest
gh release create ${{ github.event.inputs.version }} --target ${{ github.event.inputs.branch }} --generate-notes
# Edit notes to add there compatibility matrix
sed --null-data -E "s/(.*<\!-- Compatibility Matrix start -->)(.*)(<\!-- Compatibility Matrix end -->.*)/\2/" README.md > release-notes.md
echo "" >> release-notes.md
echo "$(gh release view --json body --jq .body)" >> release-notes.md
gh release edit ${{ github.event.inputs.version }} --notes-file release-notes.md
rm release-notes.md
env:
SOURCE_TAG: ${{ github.event.inputs.version }}
GITHUB_TOKEN: ${{ github.TOKEN }}
shell: bash
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ For building and pushing a new version of the bundled operator image:
- `make bundle-push -e IMAGE_TAG_BASE=<image-repo/image-name> VERSION=<new semver> PREVIOUS_VERSION=<semver to replace>`

To create a new openshift-community-operator-release:
- `make openshift-community-operator-release -e IMAGE_TAG_BASE=<image-repo/image-name> VERSION=<new semver> PREVIOUS_VERSION=<semver to replace>`
- `make openshift-community-operator-release -e IMAGE_TAG_BASE=<image-repo/image-name> VERSION=<new semver> PREVIOUS_VERSION=<semver to replace> GH_TOKEN=<GitHub token for pushing bundle content to forked repository>`

## Testing
The CodeFlare Operator currently has unit tests and pre-commit checks
Expand Down
37 changes: 22 additions & 15 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,13 +1,15 @@
# VERSION defines the project version for the bundle.
# Update this value when you upgrade the version of your project.
# To re-generate a bundle for another specific version without changing the standard setup, you can:
# - use the VERSION as arg of the bundle target (e.g make bundle VERSION=0.0.2)
# - use environment variables to overwrite this value (e.g export VERSION=0.0.2)
# - use the VERSION as arg of the bundle target (e.g make bundle VERSION=v0.0.2)
# - use environment variables to overwrite this value (e.g export VERSION=v0.0.2)
# best if we could detect this. If we cannot, we need to document it somewhere.
# then we can add a patch in the `PHONY: bundle`
# BUNDLE_VERSION is declared as bundle versioning doesn't use semver

PREVIOUS_VERSION ?= 0.0.0-dev
VERSION ?= 0.0.0-dev
PREVIOUS_VERSION ?= v0.0.0-dev
VERSION ?= v0.0.0-dev
BUNDLE_VERSION ?= $(VERSION:v%=%)

# INSTASCALE_VERSION defines the default version of the InstaScale controller
INSTASCALE_VERSION ?= v0.0.4
Expand All @@ -19,6 +21,11 @@ MCAD_REF ?= release-${MCAD_VERSION}
MCAD_REPO ?= github.com/project-codeflare/multi-cluster-app-dispatcher
MCAD_CRD ?= ${MCAD_REPO}/config/crd?ref=${MCAD_REF}

# OPERATORS_REPO_ORG points to GitHub repository organization where bundle PR is opened against
# OPERATORS_REPO_FORK_ORG points to GitHub repository fork organization where bundle build is pushed to
OPERATORS_REPO_ORG ?= redhat-openshift-ecosystem
OPERATORS_REPO_FORK_ORG ?= project-codeflare

# CHANNELS define the bundle channels used in the bundle.
# Add a new line here if you would like to change its default config. (E.g CHANNELS = "candidate,fast,stable")
# To re-generate a bundle for other specific channels without changing the standard setup, you can:
Expand Down Expand Up @@ -53,10 +60,10 @@ INSTASCALE_IMAGE ?= $(IMAGE_ORG_BASE)/instascale-controller:$(INSTASCALE_VERSION

# BUNDLE_IMG defines the image:tag used for the bundle.
# You can use it as an arg. (E.g make bundle-build BUNDLE_IMG=<some-registry>/<project-name-bundle>:<tag>)
BUNDLE_IMG ?= $(IMAGE_TAG_BASE)-bundle:v$(VERSION)
BUNDLE_IMG ?= $(IMAGE_TAG_BASE)-bundle:$(VERSION)

# BUNDLE_GEN_FLAGS are the flags passed to the operator-sdk generate bundle command
BUNDLE_GEN_FLAGS ?= -q --overwrite --version $(VERSION) $(BUNDLE_METADATA_OPTS)
BUNDLE_GEN_FLAGS ?= -q --overwrite --version $(BUNDLE_VERSION) $(BUNDLE_METADATA_OPTS)

# USE_IMAGE_DIGESTS defines if images are resolved via tags or digests
# You can enable this value if you would like to use SHA Based Digests
Expand All @@ -67,7 +74,7 @@ ifeq ($(USE_IMAGE_DIGESTS), true)
endif

# Image URL to use all building/pushing image targets
IMG ?= ${IMAGE_TAG_BASE}:v${VERSION}
IMG ?= ${IMAGE_TAG_BASE}:${VERSION}
# ENVTEST_K8S_VERSION refers to the version of kubebuilder assets to be downloaded by envtest binary.
ENVTEST_K8S_VERSION = 1.24.2

Expand Down Expand Up @@ -308,7 +315,7 @@ bundle: defaults manifests kustomize install-operator-sdk ## Generate bundle man
$(KUSTOMIZE) fn run config/crd/mcad --image gcr.io/kpt-fn/apply-setters:v0.2.0 -- MCAD_CRD=$(MCAD_CRD)
cd config/manager && $(KUSTOMIZE) edit set image controller=$(IMG)
cd config/manifests && $(KUSTOMIZE) edit add patch --patch '[{"op":"add", "path":"/metadata/annotations/containerImage", "value": "$(IMG)" }]' --kind ClusterServiceVersion
cd config/manifests && $(KUSTOMIZE) edit add patch --patch '[{"op":"add", "path":"/spec/replaces", "value": "codeflare-operator.v$(PREVIOUS_VERSION)" }]' --kind ClusterServiceVersion
cd config/manifests && $(KUSTOMIZE) edit add patch --patch '[{"op":"add", "path":"/spec/replaces", "value": "codeflare-operator.$(PREVIOUS_VERSION)" }]' --kind ClusterServiceVersion
$(KUSTOMIZE) build config/manifests | $(OPERATOR_SDK) generate bundle $(BUNDLE_GEN_FLAGS)
$(MAKE) validate-bundle
git restore config/*
Expand All @@ -321,13 +328,13 @@ bundle-build: bundle ## Build the bundle image.
bundle-push: ## Push the bundle image.
$(MAKE) image-push IMG=$(BUNDLE_IMG)

.PHONY: openshift-community-operator-releases
.PHONY: openshift-community-operator-release
openshift-community-operator-release: install-gh-cli bundle ## build bundle and create PR in OpenShift community operators repository
gh repo clone git@github.com:project-codeflare/community-operators-prod.git
cd community-operators-prod && git pull upstream main && git push origin main
cp -r bundle community-operators-prod/operators/codeflare-operator/$(VERSION)
cd community-operators-prod && git checkout -b codeflare-release-$(VERSION) && git add operators/codeflare-operator/$(VERSION)/* && git commit -s -m "add bundle manifests codeflare version $(VERSION)" && git push origin codeflare-release-$(VERSION)
gh pr create --repo redhat-openshift-ecosystem/community-operators-prod --title "CodeFlare $(VERSION)" --body "New release of codeflare operator" --head project-codeflare:codeflare-release-$(VERSION) --base main
git clone https://$(GH_TOKEN)@github.com/$(OPERATORS_REPO_FORK_ORG)/community-operators-prod.git
cd community-operators-prod && git remote add upstream https://github.com/$(OPERATORS_REPO_ORG)/community-operators-prod.git && git pull upstream main && git push origin main
cp -r bundle community-operators-prod/operators/codeflare-operator/$(BUNDLE_VERSION)
cd community-operators-prod && git checkout -b codeflare-release-$(BUNDLE_VERSION) && git add operators/codeflare-operator/$(BUNDLE_VERSION)/* && git commit -m "add bundle manifests codeflare version $(BUNDLE_VERSION)" && git push origin codeflare-release-$(BUNDLE_VERSION)
gh pr create --repo $(OPERATORS_REPO_FORK_ORG)/community-operators-prod --title "CodeFlare $(BUNDLE_VERSION)" --body "New release of codeflare operator" --head $(OPERATORS_REPO_ORG):codeflare-release-$(BUNDLE_VERSION) --base main
rm -rf community-operators-prod

.PHONY: opm
Expand All @@ -352,7 +359,7 @@ endif
BUNDLE_IMGS ?= $(BUNDLE_IMG)

# The image tag given to the resulting catalog image (e.g. make catalog-build CATALOG_IMG=example.com/operator-catalog:v0.2.0).
CATALOG_IMG ?= $(IMAGE_TAG_BASE)-catalog:v$(VERSION)
CATALOG_IMG ?= $(IMAGE_TAG_BASE)-catalog:$(VERSION)

# Set CATALOG_BASE_IMG to an existing catalog image tag to add $BUNDLE_IMGS to that image.
ifneq ($(origin CATALOG_BASE_IMG), undefined)
Expand Down
24 changes: 8 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# codeflare-operator
Operator for installation and lifecycle management of CodeFlare distributed workload stack, starting with MCAD and InstaScale

<!-- Don't delete these comments, they are used to generate Compatibility Matrix table -->
<!-- Compatibility Matrix start -->
CodeFlare Stack Compatibility Matrix

| Component | Version |
Expand All @@ -10,6 +12,7 @@ CodeFlare Stack Compatibility Matrix
| CodeFlare-SDK | v0.4.4 |
| InstaScale | v0.0.4 |
| KubeRay | v0.5.0 |
<!-- Compatibility Matrix end -->

## Release process

Expand All @@ -23,23 +26,12 @@ Release steps:

2. Check result of [tag-and-build.yml](https://github.com/project-codeflare/codeflare-operator/actions/workflows/tag-and-build.yml) GitHub action, it should pass.

3. Update CodeFlare Stack Compatibility Matrix in operator README.
3. Verify that compatibility matrix in [README](https://github.com/project-codeflare/codeflare-operator/blob/main/README.md) was properly updated.

4. Update InstaScale and MCAD versions:
- in [Makefile](https://github.com/project-codeflare/codeflare-operator/blob/02e14b535b4f7172b0b809bcae4025008a1a968b/Makefile#L12-L16).
- in [mcad/deployment.yaml.tmpl](https://github.com/project-codeflare/codeflare-operator/blob/main/config/internal/mcad/deployment.yaml.tmpl#L28)
- in [controllers/defaults.go](https://github.com/project-codeflare/codeflare-operator/blob/main/controllers/defaults.go) by running `make defaults`
- in [controllers/testdata/instascale_test_results/case_1/deployment.yaml](https://github.com/project-codeflare/codeflare-operator/blob/main/controllers/testdata/instascale_test_results/case_1/deployment.yaml) and [controllers/testdata/instascale_test_results/case_2/deployment.yaml](https://github.com/project-codeflare/codeflare-operator/blob/main/controllers/testdata/instascale_test_results/case_2/deployment.yaml)
4. Verify that opened pull request to [OpenShift community operators repository](https://github.com/redhat-openshift-ecosystem/community-operators-prod) has proper content.

5. Create a release in CodeFlare operator repository, release notes should include new support matrix.
5. Once PR is merged, update component stable tags to point at the latest image release.

6. Open a pull request to OpenShift community operators repository with latest bundle using make command, check that the created PR has proper content.
```
make openshift-community-operator-release
```
6. Announce the new release in slack and mail lists, if any.

7. Once merged, update component stable tags to point at the latest image release.

8. Announce the new release in slack and mail lists, if any.

9. Update the Distributed Workloads component in ODH (also copy/update the compatibility matrix).
7. Update the Distributed Workloads component in ODH (also copy/update the compatibility matrix).

0 comments on commit 0464938

Please sign in to comment.