From ac212b4ff22e05a1ed1dfc103e45a8602620ea2e Mon Sep 17 00:00:00 2001 From: Adam Kaplan Date: Thu, 18 Jul 2024 23:16:01 -0400 Subject: [PATCH] Copy missing root files and docs Copy over docs and other missing root-level files --- DEVELOPMENT.md | 59 +++++++++++++++ OWNERS | 22 ++++++ docs/development/local-development.md | 64 ++++++++++++++++ docs/development/olm-development.md | 105 ++++++++++++++++++++++++++ docs/development/releasing.md | 80 ++++++++++++++++++++ docs/shipwrightbuild.md | 30 ++++++++ 6 files changed, 360 insertions(+) create mode 100644 DEVELOPMENT.md create mode 100644 OWNERS create mode 100644 docs/development/local-development.md create mode 100644 docs/development/olm-development.md create mode 100644 docs/development/releasing.md create mode 100644 docs/shipwrightbuild.md diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md new file mode 100644 index 00000000..28f9472b --- /dev/null +++ b/DEVELOPMENT.md @@ -0,0 +1,59 @@ +# Developer Guide + +Want to add a new feature (or fix bugs) in the operator? This document will help you understand how +to build, test, and deploy the operator. + +## Prerequisites + +Before you begin, ensure you have have a development environment well suited for Go and Kubernetes +development: + +- Install [Go](https://go.dev/doc/install) version 1.21 or higher. We recommend using an + interactive development environment (IDE) that supports go, such as + [GoLand](https://www.jetbrains.com/go/) or [VSCode](https://code.visualstudio.com/). +- Install a container engine that can build and run container images. We recommend + [docker](https://docs.docker.com/get-docker/) or [podman](https://podman.io/). +- Obtain access to a Kubernetes cluster that you have administrative permissions on. This can be a + "micro" distribution such as [kind](https://kind.sigs.k8s.io/), or a fully managed cluster like + [GKE](https://cloud.google.com/kubernetes-engine). We recommend `kind` for local development. + + +## Operator Foundations + +This project builds a Kubernetes [operator](https://operatorframework.io/what/) for Shipwright +projects, such as [build](https://github.com/shipwright-io/build). It is scaffolded with +[Operator SDK](https://sdk.operatorframework.io/) and its related Kubernetes development framework, +[kubebuilder](https://book.kubebuilder.io/). Those who are new to Kubernetes concepts like +controllers and custom resources are encouraged to try the +[Operator SDK go tutorial](https://sdk.operatorframework.io/docs/building-operators/golang/tutorial/) +first. + +Operator SDK and kubebuilder are strongly opinionated when it comes to project structure. +Contributors should use `operator-sdk` [commands](https://sdk.operatorframework.io/docs/cli/) or +`kubebuilder` [commands](https://github.com/kubernetes-sigs/kubebuilder) when adding new custom +resources or controllers. Code within the `pkg/` directory is generally safe from these +generators and should be reserved for core reconciliation logic ("business logic"). + +This project also generates an +[operator bundle](https://sdk.operatorframework.io/docs/olm-integration/tutorial-bundle/), which is +used to deploy the operator on a cluster with +[Operator Lifecycle Manager](https://olm.operatorframework.io/) (OLM). Refer to the +[OLM core tasks](https://olm.operatorframework.io/docs/tasks/) for the steps needed to deploy an +operator with OLM. + + +## Development Flows + +Most features and bug fixes only require the operator and its related custom resource definitions +to be deployed on Kubernetes. The [local development guide](/docs/development/local-development.md) +describes how to build, deploy, and test your changes against a Kubernetes cluster. + +Changes that modify the `bundle` directory, however, may require the operator to be packaged and +deployed with Operator Lifecycle Manager. Refer to the +[OLM development guide](/docs/development/olm-development.md) for instructions on how to build, +deploy, and test the operator with OLM. + +## Releasing the Operator + +For maintainers who are asked to issue releases, please refer to the +[release process](./docs/development/releasing.md) document. diff --git a/OWNERS b/OWNERS new file mode 100644 index 00000000..1728c424 --- /dev/null +++ b/OWNERS @@ -0,0 +1,22 @@ +# See the OWNERS docs: https://git.k8s.io/community/contributors/guide/owners.md + +approvers: +- coreydaley +- otaviof +- qu1queee +- adambkaplan +- SaschaSchwarze0 +- HeavyWombat +- jkhelil + +reviewers: +- coreydaley +- otaviof +- qu1queee +- adambkaplan +- SaschaSchwarze0 +- HeavyWombat +- jkhelil + +emeritus_approvers: +- sbose78 #2022-09-12 diff --git a/docs/development/local-development.md b/docs/development/local-development.md new file mode 100644 index 00000000..1a6961d1 --- /dev/null +++ b/docs/development/local-development.md @@ -0,0 +1,64 @@ +# Local Operator Development + +## Prerequisites + +* Ensure you have access to a Kubernetes cluster via `kubectl` with cluster admin permissions. +* Install [Go](https://go.dev/doc/install) version 1.21 or higher. + +## Building locally + +Run `make build` to compile the operator binary. +The resulting binary will be saved to `bin/operator`. + +To build the container image for local use, run `make container-push IMAGE_REPO=ko.local`. +This will do the following: + +1. Compile the application. +2. Create a container image. +3. Push the container image to your local Docker daemon, with the ref `ko.local/operator:`. + +The following make options can be set: + +* `IMAGE_REPO` - defaults to `ghcr.io/shipwright-io/operator`. + The following special repositories can be used for testing: + + * `ko.local` - this pushes the image to the local Docker daemon. + * `kind.local` - pushes to a local KinD cluster. + +* `VERSION` - defaults to the current version of Shipwright. + This must be a valid [semantic version](https://semver.org/), and will be used as the tag for the resulting image. + +Refer to the [ko documentation](https://ko.build/) for more information. + +## Deploy to Kubernetes + +To test the operator on a Kubernetes cluster, you first must have the following: + +* Access to a Kubernetes cluster v1.20 or higher, with cluster admin permissions. +* Install [Tekton operator](https://github.com/tektoncd/operator) v0.50 or higher on the cluster. + +```bash +$ export KUBECONFIG=/path/to/kubeconfig +$ kubectl apply -f https://github.com/tektoncd/pipeline/releases/download/v0.21.0/release.notags.yaml +``` + +If pushing to an external image registry, you may need to provide credentials to ko: + +```bash +$ make ko +$ ko login -u -p +``` + +Next, use the `make deploy` command with appropriate `IMAGE_REPO` and `VERSION` arguments to deploy the operator to the cluster. + +```bash +$ make deploy IMAGE_REPO="/" VERSION="" +``` + +_Note:_ + +Scripts in `hack` folder may require `sed` (GNU), therefore in platforms other than Linux you may have it with a different name. For instance, on macOS it's usually named `gsed`, in this case provide the `SED_BIN` make variable with the alternative name. + +```bash +$ make build SED_BIN=gsed ... +``` diff --git a/docs/development/olm-development.md b/docs/development/olm-development.md new file mode 100644 index 00000000..8b5478ef --- /dev/null +++ b/docs/development/olm-development.md @@ -0,0 +1,105 @@ +# OLM Development + +The Shipwright operator is meant to be deployed on a cluster using the +[Operator Lifecycle Manager](https://olm.operatorframework.io/) (OLM). +OLM provides mechanisms to support over the air upgrades and automatically deploy related operators +that are packaged in operator [catalogs](https://olm.operatorframework.io/). +Additional steps need to be taken to ensure the operator can be deployed with OLM. + +## Prerequisites + +* Ensure you have access to a Kubernetes cluster via `kubectl` with cluster admin permissions. +* Install [Go](https://go.dev/doc/install) version 1.21 or higher. +* Install OLM on your cluster. This can be done using the `make install-olm` command. +* Ability to push to a container registry that is accessible inside your Kubernetes cluster. + +## Step 1: Build and push the operator and bundle + +Run `make bundle-push IMAGE_REPO=`, pushing to a container registry that is +accessible inside your Kubernetes cluster. +This make command will push the operator and [operator bundle](https://olm.operatorframework.io/docs/tasks/creating-operator-bundle/) +to the container registry. +An operator bundle is an OCI artifact that tells OLM how to deploy your operator. + +Using `ko.local` or `kind.local` for `IMAGE_REPO` is not recommended, as this will not push the +resulting images to an OCI-compliant container registry. +If you are using [KinD](https://kind.sigs.k8s.io/), follow the instructions on how to configure a +[local registry](https://kind.sigs.k8s.io/docs/user/local-registry/). +This will let you use `localhost:` as your container registry. + +## Step 2: Build and push an operator catalog + +Next, run `make catalog-push IMAGE_REPO=`. +This will build and push an [operator catalog](https://olm.operatorframework.io/docs/tasks/creating-a-catalog/), +which packages your test operator bundle with the other operators available on [operatorhub.io](https://operatorhub.io). +The built catalog uses the new [file-based catalog](https://olm.operatorframework.io/docs/reference/file-based-catalogs/) +architecture, and packages the upstream Tekton operator along with the current released Shipwright +operator. +The catalog is rendered from JSON manifests in the `test/catalog` directory, and full contents of +the built catalog can be inspected in the `_output/catalog` directory. + +As in step 1, be sure to use the same container registry for the `IMAGE_REPO` argument. + +## Step 3: Deploy the operator using the catalog image + +Finally, deploy the operator using `make catalog-run IMAGE_REPO=`, using the same +value for `IMAGE_REPO` as in the previous steps. +This will run a script that does the following: + +1. Creates a custom [CatalogSource](https://olm.operatorframework.io/docs/tasks/make-catalog-available-on-cluster/) + and [OperatorGroup](https://olm.operatorframework.io/docs/advanced-tasks/operator-scoping-with-operatorgroups/), + which allows the operators in step 3's catalog to be installed anywhere on the cluster. +2. Creates a [Subscription](https://olm.operatorframework.io/docs/tasks/install-operator-with-olm/), + which instructs OLM to install the operator and any dependent operators. +3. Checks that the operator has successfully been installed and rolled out. + +Once the script completes, the Shipwright and Tekton operators will be installed on the cluster. + +## Testing a Release + +To test a release that has not been published to the Kubernetes Operators +[OperatorHub](https://operatorhub.io/), do the following: + +### Step 1: Build and Push an Operator Catalog + +Like Step 2 above, you will need to publish a catalog containing the candidate operator release. +Run the following command to set the correct `BUNDLE_IMG` for the catalog: + +```sh +$ version= # example: 0.13.0-rc0 +$ make catalog-push IMAGE_REPO= VERSION="$version" BUNDLE_IMG="ghcr.io/shipwright-io/operator/operator-bundle:v$version" +``` + +### Step 2: Deploy the operator + +Similar to Step 3 above, deploy the operator with the catalog image: + +```sh +$ version= # example: 0.13.0-rc0 +$ make catalog-run IMAGE_REPO= VERSION="$version" +``` + +## Troubleshooting + +### `sed` Command Not Found + +Scripts in `hack` folder may require `sed` (GNU) and assume they are running on Linux. +On platforms other than Linux, use the `SED_BIN` make variable to use a different command for `sed`. +For instance, on MacOS sed functions are provided by `gsed`: + +```bash +$ make catalog-run SED_BIN=gsed ... +``` + +### Catalog Source Fails - Cannot Access Registry Over grpc + +OLM uses `grpc` by default to pull catalog sources from OCI artifacts. +This protocol requires HTTP/2, which is not supported in some circumstances (example: hosting +the catalog and bundle on a registry deployed on KinD). +To fall back to HTTP-based pull, set the `USE-HTTP` make variable to `true` when building/pushing +the test catalog: + +```bash +$ make catalog-push USE-HTTP="true" ... +$ make catalog-run ... +``` diff --git a/docs/development/releasing.md b/docs/development/releasing.md new file mode 100644 index 00000000..184390fc --- /dev/null +++ b/docs/development/releasing.md @@ -0,0 +1,80 @@ +# Release Process + +This document outlines the steps required to release the operator. This document assumes that you +have achieved the "Approver"/"Maintainer" status, and have permission to manually trigger GitHub +Actions on this repo. + +## Release Candidates (`X.Y.0-rcN`) + +### Step 0: Set Up the Release Branch + +Run the "Create Release Branch" GitHub Action, providing a valid Semantic Version with a major and +minor revision number. Example: `v0.13`. + +### Step 1: Create a Release Candidate + +This is ideal for `.0` releases, where there is most risk of "blocking" bugs. Release candidates +can be skipped for z-stream releases. + +Run the "Release" GitHub action, with the following parameters: + +- New tag: provide a semantic version _without the leading `v`_. Use a trailing `-` to add patch + suffix. Ex: 0.13.0-rc0. +- Previous tag: provide a semantic version _that matches a tag on the GitHub repo_. Ex: `v0.12.0`. +- Ref: use the branch for the release in question. Ex: `release-v0.13`. + +This will draft a release for GitHub - it will not publish a tag or release note. + +### Step 2: Publish Draft Release + +Find the draft release. Edit the release as follows: + +- Add a leading `v` suffix to the generated tag and release name. Ex: `0.13.0-rc0` becomes `v0.13.0-rc0`. +- Change the "Draft release notes" title to "What's Changed". + +Once you're happy, publish the draft release note. If an item is missing from the release note, +review the pull requests that should be included. Each desired PR should have: + +1. A release note. +2. A `kind/*` label, such as `kind/feature` or `kind/bug`. + +### Step 3: Verify the Bundle + +Once the release candidate is published, broadcast the candidate build to the community in Slack +and the shipwright-dev mailing list. Refer to the "Testing a Release" section of the +[OLM Development](./olm-development.md#testing-a-release) guide for instructions on how to test the +release candidate. + +### Step 4: Triage/Fix bugs + +Once the release candidate is published, the community should file any issues as bugs in GitHub. It +is up to maintainers to determine which bugs are "release blockers" and which ones can be addressed +in a future release. + +### Step 5: Repeat Release Candidates + +Repeat steps 1-4 as needed if a "release blocker" issue is identified. + +## Official Releases + +Before proceeding with an official release, ensure that the +[release branch](#step-0-set-up-the-release-branch) has been set up. + +### Step 1: Bump versions for release + +Once bugs have been triaged and the community feels ready, submit pull requests to bump the +`VERSION` make variable. For a new release, there should be two pull requests: + +1. One for the `main` branch to update the minor semantic version. Ex: Update `0.12.0-rc0` to + `0.13.0-rc0` +2. One for the `release-v*` branch, dropping any release candidate patch suffixes and/or updating + the patch semantic version itself. Ex: `0.13.0` to `0.13.1`. + +In both cases, run `make bundle` and commit any generated changes as part of the pull request. +Work with the community to get these pull requests merged. + +### Step 2: Publish the Release + +Repeat the process in [Step 1](#step-1-create-a-release-candidate) and +[Step 2](#step-2-publish-draft-release) above to create the release. For an official release, the +version should adhere to the `X.Y.Z` format (not extra dashes). diff --git a/docs/shipwrightbuild.md b/docs/shipwrightbuild.md new file mode 100644 index 00000000..877fc815 --- /dev/null +++ b/docs/shipwrightbuild.md @@ -0,0 +1,30 @@ +# ShipwrightBuild Custom Resource + +When the Shipwright Operator is installed with the Operator Lifecycle Manager, the +`ShipwrightBuild` [custom resource definition](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) is added to your cluster. +This custom resource is used to install and configure Shipwright Builds on your cluster. +The current operator will install version `0.12.0` of Builds. + +When the `ShipwrightBuild` instance is created, the following components are installed: + +- The custom resources to run Shipwright builds (`ClusterBuildStrategy`, `BuildStrategy`, `Build`, + `BuildRun`). +- Shipwright Build's controller, conversion webhook, and associated CA certificates. +- The following example `ClusterBuildStrategies`: + - `buildah-shipwright-managed-push` + - `buildah-strategy-managed-push` + - `buildkit` + - `buildpacks-v3` + - `buildpacks-v3-heroku` + - `kaniko` + - `kaniko-trivy` + - `ko` + - `source-to-image` + + +## ShipwrightBuild Reference + +| Field | Description | +| ----- | ----------- | +| spec.targetNamespace | The target namespace where Shipwright Build will be deployed. If omitted, this will default to `shipwright-build` | +| status.conditions | Conditions which report the status of Shipwright Build. Current reported conditions:

- `Ready` |