Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move admission logic to scaffolded webhook #195

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from
Draft

Move admission logic to scaffolded webhook #195

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
8efd553
Move admission logic to scaffolded webhook
LCaparelli Dec 16, 2020
2884258
Break FakeClient into FakeClient and FakeDisc
LCaparelli Dec 17, 2020
952b87b
Fix tests
LCaparelli Dec 22, 2020
63e2970
Enable kustomization for webhooks and cert-manager
LCaparelli Dec 22, 2020
9a86fc7
Fix webhook group
LCaparelli Feb 1, 2021
accc48c
Inform reader no changes or validation takes place in e2e tests
LCaparelli Feb 1, 2021
28bd7f3
Create auto-generation for webhookless installer manifest
LCaparelli Feb 2, 2021
cb1732a
Fallback to in-reconcile admission if webhooks are not in use
LCaparelli Feb 2, 2021
428388b
Make webhookless installer generator work on CI
LCaparelli Feb 4, 2021
acd762c
Add and document dependency on cert-manager
LCaparelli Feb 4, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .github/workflows/nexus-operator-integration-checks.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,13 @@ jobs:
with:
go-version: ${{ env.GO_VERSION }}
id: go
- name: Setup Python
uses: actions/setup-python@v1
with:
python-version: 3.7
- name: Install yq
run: |
pip install yq
- name: Check Vet
run: |
make generate
Expand Down Expand Up @@ -66,6 +73,14 @@ jobs:
restore-keys: |
${{ runner.os }}-go-cache-

- name: Setup Python
uses: actions/setup-python@v1
with:
python-version: 3.7
- name: Install yq
run: |
pip install yq

- name: Cache Operator SDK
uses: actions/cache@v2
with:
Expand Down Expand Up @@ -131,6 +146,13 @@ jobs:
key: ${{ runner.os }}-go-${{ env.GO_VERSION }}
restore-keys: |
${{ runner.os }}-go-${{ env.GO_VERSION }}
- name: Setup Python
uses: actions/setup-python@v1
with:
python-version: 3.7
- name: Install yq
run: |
pip install yq
- name: Install Operator SDK
run: |
./hack/ci/install-operator-sdk.sh
Expand Down
177 changes: 32 additions & 145 deletions CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,24 +2,37 @@

## Have found a bug or have a feature request?

Please, open an issue for us. That will really help us improving the Operator and it would benefit other users such as yourself. There are templates ready for bug reporting and feature request to make things easier for you.
Please, open an issue for us. That will really help us improving the Operator and it would benefit other users such as
yourself. There are templates ready for bug reporting and feature request to make things easier for you.

## Have any questions?

We're happy to answer! Either [open an issue](https://github.com/m88i/nexus-operator/issues) or send an email to our mailing list: [[email protected]](mailto:[email protected]).
We're happy to answer! Either [open an issue](https://github.com/m88i/nexus-operator/issues) or send an email to our
mailing list: [[email protected]](mailto:[email protected]).

## Are you willing to send a PR?

Before sending a PR, consider opening an issue first. This way we can discuss your approach and the motivations behind it while also taking into account other development efforts.
Before sending a PR, consider opening an issue first. This way we can discuss your approach and the motivations behind
it while also taking into account other development efforts.

Regarding your local development environment:

1. We use [golint-ci](https://golangci-lint.run/) to check the code. Consider [integrating it in your favorite IDE](https://golangci-lint.run/usage/integrations/) to avoid failing in the CI
2. **Always** run `make test` before sending a PR to make sure the license headers and the manifests are updated (and of course the unit tests are passing)
3. Consider adding a new [end-to-end](https://sdk.operatorframework.io/docs/golang/e2e-tests/) test case covering your scenario and make sure to run `make test-e2e` before sending the PR
4. Make sure to always keep your version of `Go` and the `operator-sdk` on par with the project. The current version information can be found at [the go.mod file](go.mod)
1. We use [golint-ci](https://golangci-lint.run/) to check the code.
Consider [integrating it in your favorite IDE](https://golangci-lint.run/usage/integrations/) to avoid failing in the
CI
2. **Always** run `make test` before sending a PR to make sure the license headers and the manifests are updated (and of
course the unit tests are passing)
3. Consider adding a new [end-to-end](https://sdk.operatorframework.io/docs/building-operators/golang/testing/) test
case covering your scenario in `controllers/nexus_controller_test.go` and make sure to run `make test` before sending
the PR
4. Make sure to always keep your version of `go` and the `operator-sdk` on par with the project.

To run all unit tests, build the image, run the E2E tests with that image and push, all in one go, you may run `make pr-prep`. If any tests fail or if the build fails, the process will be terminated so that you make the necessary adjustments. If they are all successful, you'll be prompted to push your commited changes.
- go 1.15
- operator-sdk v1.2.0

To run all tests and push, all in one go, you may run `make pr-prep`. If any tests fail or if the build fails, the
process will be terminated so that you make the necessary adjustments. If they are all successful, you'll be prompted to
push your committed changes.

```shell
$ make pr-prep
Expand All @@ -32,9 +45,11 @@ Pushing to origin/pr-prep
# (output omitted)
```

If you don't inform remote name and branch, it will use "origin" as the remote and your current branch (the defaults, which appear between "[]"). Double check if the information is correct.
If you don't inform remote name and branch, it will use "origin" as the remote and your current branch (the defaults,
which appear between "[]"). Double check if the information is correct.

If you don't want to go over the interactive prompt every time, you can push with the defaults using the `PUSH_WITH_DEFAULTS` environment variable:
If you don't want to go over the interactive prompt every time, you can push with the defaults using the
`PUSH_WITH_DEFAULTS` environment variable:

```shell
$ PUSH_WITH_DEFAULTS=TRUE make pr-prep
Expand All @@ -46,140 +61,12 @@ Pushing to origin/pr-prep

## E2E Testing

If you added a new functionality and are willing to add some end-to-end (E2E) testing of your own, please add a test case to `test/e2e/nexus_test.go`.

The test case structure allows you to name your test appropriately (try naming it in a way it's clear what it's testing), provide a Nexus CR that the Operator will use to generate the other resources, provide additional checks your feature may require and provide a custom cleanup function if necessary.

Then each test case is submitted to a series of checks which should make sure everything on the cluster is as it should, based on the Nexus CR that has been defined.

Let's take our smoke test as an example to go over the test cases structure:

```go
testCases := []struct {
name string (1)
input *v1alpha1.Nexus (2)
cleanup func() error (3)
additionalChecks []func(nexus *v1alpha1.Nexus) error (4)
}{
{
name: "Smoke test: no persistence, nodeport exposure", (1)
input: &v1alpha1.Nexus{ (2)
ObjectMeta: metav1.ObjectMeta{
Name: nexusName,
Namespace: namespace,
},
Spec: defaultNexusSpec, (5)
},
cleanup: tester.defaultCleanup, (3)
additionalChecks: nil, (4)
},
```

> (1): the test case's name. In this scenario we're testing a deployment with all default values, no persistence and exposed via Node Port.<br>
> (2): the Nexus CR which the Operator will use to orchestrate and maintain your Nexus3 deployment<br>
> (3): a cleanup function which should be ran after the test has been completed<br>
> (4): additional checks your test case may need<br>
> (5): the base, default Nexus CR specification which should be used for testing. Modify this to test your own features<br>

**Important**: although the operator will set the defaults on the Nexus CR you provide it with, the tests will use your original CR for comparison, so be sure to make a completely valid Nexus CR for your test case as it *will not* be modified to insert default values.

### Custom Nexus CR

If your test requires modifications to the default Nexus CR, you can do so directly and concisely when defining the test case by making use of anonymous functions.

For example:

```go
{
name: "Networking: ingress with no TLS",
input: &v1alpha1.Nexus{
ObjectMeta: metav1.ObjectMeta{
Name: nexusName,
Namespace: namespace,
},
Spec: func() v1alpha1.NexusSpec {
spec := *defaultNexusSpec.DeepCopy()
spec.Networking = v1alpha1.NexusNetworking{Expose: true, ExposeAs: v1alpha1.IngressExposeType, Host: "test-example.com"}
return spec
}(),
},
cleanup: tester.defaultCleanup,
additionalChecks: nil,
},
```
If you added a new functionality and are willing to add some end-to-end (E2E) testing of your own, please add a test
case to `controllers/nexus_controller_test.go`.

When defining the Nexus's specification in this case we're actually calling an anonymous function that acquires the default spec, modifies the required fields and then returns that spec, thus making the necessary changes for the test.

### Custom Cleanup functions

Our test cases make use of [functions first-class citizenship in Go](https://golang.org/doc/codewalk/functions/) by declaring the cleanup function as a field from the test case. This way it's possible to specify our own custom cleanup function for a test.

In previous examples, `tester.defaultCleanup` was used, which simply deletes all Nexus CRs in the namespace, but you may want to do some additional computation when cleaning up, such as counting to 5 (intentionally useless to promote simplicity in this example):

```go
{
name: "Test Example: this counts to 5 during cleanup and uses the default cleanup once done",
input: &v1alpha1.Nexus{
ObjectMeta: metav1.ObjectMeta{
Name: nexusName,
Namespace: namespace,
},
Spec: defaultNexusSpec,
},
cleanup: func() error {
for i := 0; i < 5; i++ {
tester.t.Logf("Count: %d", i)
}
return tester.defaultCleanup()
},
additionalChecks: nil,
},
```

It's possible, of course, to not use the default cleanup function at all, but be sure to actually delete the resources you created if they conflict with other test cases (the framework itself will delete the whole namespace once the tests are done):

```go
{
name: "Test Example: this only counts to 5 during cleanup and does not delete anything",
input: &v1alpha1.Nexus{
ObjectMeta: metav1.ObjectMeta{
Name: nexusName,
Namespace: namespace,
},
Spec: defaultNexusSpec,
},
cleanup: func() error {
for i := 0; i < 5; i++ {
tester.t.Logf("Count: %d", i)
}
return nil
},
additionalChecks: nil,
},
```
All mutating (including default values) and validating logic lives in our admission webhooks, which are not in place
when the e2e suite runs. Because of that, no changes or checks are performed against Nexus CRs being reconciled. Be sure
to always write tests which use valid Nexus resources.

### Running additional checks

If your testing needs to check something that isn't already checked by default you may add functions to perform these checks as the function that is responsible for running the default checks will receive them as [variadic arguments](https://gobyexample.com/variadic-functions).

In another useless yet simple example, let's also make sure that 5 is greater than 4 when performing our checks:

```go
{
name: "Test Example: this will also check if 5 > 4",
input: &v1alpha1.Nexus{
ObjectMeta: metav1.ObjectMeta{
Name: nexusName,
Namespace: namespace,
},
Spec: defaultNexusSpec,
},
cleanup: tester.defaultCleanup,
additionalChecks: []func(nexus *v1alpha1.Nexus)error{
func(nexus *v1alpha1.Nexus) error {
assert.Greater(tester.t, 5, 4)
return nil
},
},
},
```
In the future, after we migrate to operator-sdk v1.3.0 and kubebuilder plugin v3, scaffolding will create a separate
suite for testing the webhooks.
14 changes: 11 additions & 3 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ BUNDLE_METADATA_OPTS ?= $(BUNDLE_CHANNELS) $(BUNDLE_DEFAULT_CHANNEL)
# Image URL to use all building/pushing image targets
IMG ?= controller:latest
# Produce CRDs that work back to Kubernetes 1.11 (no version conversion)
CRD_OPTIONS ?= "crd:trivialVersions=true"
CRD_OPTIONS ?= "crd:trivialVersions=true,preserveUnknownFields=false"

# Image URL for the operator final image
OPERATOR_IMG ?= quay.io/m88i/nexus-operator:$(VERSION)
Expand All @@ -36,14 +36,18 @@ all: manager
ENVTEST_ASSETS_DIR=$(shell pwd)/testbin
# Needed to support k8s.io/api/networking/v1 Ingress
K8S_VERSION=1.19.0
test: generate-installer fmt vet bundle
test: generate-installer fmt vet bundle test-only

# just test without generating anything, use wisely
test-only:
mkdir -p ${ENVTEST_ASSETS_DIR}
test -f ${ENVTEST_ASSETS_DIR}/setup-envtest.sh || curl -sSLo ${ENVTEST_ASSETS_DIR}/setup-envtest.sh https://raw.githubusercontent.com/kubernetes-sigs/controller-runtime/master/hack/setup-envtest.sh
sed -i "s,#\!.*,#\!\/bin\/bash,g" ${ENVTEST_ASSETS_DIR}/setup-envtest.sh
sed -i "/pipefail/d" ${ENVTEST_ASSETS_DIR}/setup-envtest.sh
source ${ENVTEST_ASSETS_DIR}/setup-envtest.sh; ENVTEST_K8S_VERSION=$(K8S_VERSION) fetch_envtest_tools $(ENVTEST_ASSETS_DIR); setup_envtest_env $(ENVTEST_ASSETS_DIR); go test ./... -coverprofile cover.out

generate-installer: generate manifests kustomize

generate-installer: generate manifests kustomize generate-webhookless-installer
cd config/manager && $(KUSTOMIZE) edit set image controller=$(OPERATOR_IMG)
$(KUSTOMIZE) build config/default > nexus-operator.yaml

Expand Down Expand Up @@ -157,3 +161,7 @@ create_namespace=true
run_with_image=true
pr-prep:
CREATE_NAMESPACE=$(create_namespace) RUN_WITH_IMAGE=$(run_with_image) ./hack/pr-prep.sh

# Generate the installer without webhook configs, secrets and what not
generate-webhookless-installer:
./hack/generate-webhookless-installer.sh
45 changes: 40 additions & 5 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ Table of Contents
* [Table of Contents](#table-of-contents)
* [Nexus Operator](#nexus-operator)
* [Pre Requisites](#pre-requisites)
* [Kubernetes API dependencies](#kubernetes-api-dependencies)
* [Quick Install](#quick-install)
* [Openshift](#openshift)
* [Clean up](#clean-up)
Expand All @@ -20,8 +21,8 @@ Table of Contents
* [Networking](#networking)
* [Use NodePort](#use-nodeport)
* [Network on OpenShift](#network-on-openshift)
* [Network on Kubernetes 1.14 ](#network-on-kubernetes-114)
* [NGINX Ingress troubleshooting](#nginx-ingress-troubleshooting)
* [Network on Kubernetes 1.14+](#network-on-kubernetes-114)
* [NGINX Ingress troubleshooting](#nginx-ingress-troubleshooting)
* [TLS/SSL](#tlsssl)
* [Persistence](#persistence)
* [Minikube](#minikube)
Expand All @@ -41,19 +42,53 @@ Table of Contents

A Nexus OSS Kubernetes Operator based on the [Operator SDK](https://github.com/operator-framework/operator-sdk).

You can find us at [OperatorHub](https://operatorhub.io/operator/nexus-operator-m88i) or at the ["Operators" tab in your OpenShift 4.x web console](https://docs.openshift.com/container-platform/4.4/operators/olm-adding-operators-to-cluster.html), just search for "Nexus". If you don't have access to [OLM](https://github.com/operator-framework/operator-lifecycle-manager), try installing it manually [following our quick installation guide](#quick-install).
You can find us at [OperatorHub](https://operatorhub.io/operator/nexus-operator-m88i) or at
the ["Operators" tab in your OpenShift 4.x web console](https://docs.openshift.com/container-platform/4.4/operators/olm-adding-operators-to-cluster.html)
, just search for "Nexus". If you don't have access
to [OLM](https://github.com/operator-framework/operator-lifecycle-manager), try installing it
manually [following our quick installation guide](#quick-install).

If you have any questions please either [open an issue](https://github.com/m88i/nexus-operator/issues) or send an email to the mailing list: [[email protected]](mailto:[email protected]).
If you have any questions please either [open an issue](https://github.com/m88i/nexus-operator/issues) or send an email
to the mailing list: [[email protected]](mailto:[email protected]).

## Pre Requisites

- [`kubectl` installed](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
- Kubernetes or OpenShift cluster available (minishift, minikube or crc also supported)
- Cluster admin credentials to install the Operator

### Kubernetes API dependencies

Starting on v0.6.0, the Operator relies on [cert-manager](https://cert-manager.io/) for generating and injecting the TLS
certificate for
the [admission webhooks](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#admission-webhooks)
we use. When installing the Operator via OLM (e.g., following instruction
on [Operator Hub](https://operatorhub.io/operator/nexus-operator-m88i)), cert-manager will be automatically installed in
your cluster. When installing using other methods, it's necessary
to [manually install it](https://cert-manager.io/docs/installation/).

However, if you simply want to try out the operator without webhooks, you may install it with:

```bash
VERSION=<version from GitHub releases page>

kubectl apply -f https://github.com/m88i/nexus-operator/releases/download/${VERSION}/webhookless-nexus-operator.yaml
```

We strongly advise using webhooks as they significantly improve user experience by reporting validation errors
instantly (which you would need to look for in the logs otherwise). They also persist any changes made to the Nexus
resource (such as default values), which would otherwise only happen during runtime (opaque to users).

Additionally, there is also a weak dependency on
the [Prometheus Operator's](https://github.com/prometheus-operator/prometheus-operator) `ServiceMonitor` CRD. The
Operator will run fine without it, but if this API is available it will be used to improve monitoring. For more
information check their [CRDs doc](https://github.com/prometheus-operator/prometheus-operator#customresourcedefinitions)
and [quickstart guide](https://github.com/prometheus-operator/prometheus-operator#quickstart).

## Quick Install

The installation procedure will create a Namespace named `nexus-operator-system` and will install every resources needed for the operator to run:
The installation procedure will create a Namespace named `nexus-operator-system` and will install every resources needed
for the operator to run:

```bash
# requires python and kubectl
Expand Down
7 changes: 3 additions & 4 deletions RELEASE_NOTES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
## Version 0.5.0
## Version 0.6.0

### Enhancements
- #198 - Upgrade to Go v1.15.6
- #199 - Upgrade to operator-sdk v1.2.0

- # 174 - Use Admission Webhooks to set default values and validate to Nexus CR

### Bug Fixes
- #191 - Pod fails to start after modifying the Nexus resource
Loading