Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: testing with cel policies #519

Merged
merged 27 commits into from
May 22, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
c1b9241
testing with cel policies
JaydipGabani May 2, 2024
aa12045
--set enableK8sNativeValidation=false for rego engine
JaydipGabani May 2, 2024
67b246f
fixing uploading artifacts in ci
JaydipGabani May 2, 2024
1a4e6d1
removing blank line
JaydipGabani May 2, 2024
120407b
adding engine rego
JaydipGabani May 3, 2024
be18548
adding engine rego
JaydipGabani May 3, 2024
bf316d9
correcting required label cel policy
JaydipGabani May 6, 2024
d560171
Merge branch 'master' into CEL-source
JaydipGabani May 6, 2024
45e9b6a
removing CEL from policies
JaydipGabani May 8, 2024
f7b47b8
Using sed to modify gk.yml
JaydipGabani May 8, 2024
18cd62a
adding required label policy and fixing deployment for gk
JaydipGabani May 9, 2024
a965f5a
fixing ci
JaydipGabani May 9, 2024
fbd06cc
adding not allowed label value example for required label policy
JaydipGabani May 9, 2024
960614d
testing cel with 3.16+
JaydipGabani May 10, 2024
ad3ab1d
changing required label cel
JaydipGabani May 10, 2024
77fc249
only testing rego for 3.15
JaydipGabani May 10, 2024
58a9b57
fixing ci
JaydipGabani May 10, 2024
466d8eb
fixing ci
JaydipGabani May 10, 2024
b9ab792
fixing examples
JaydipGabani May 10, 2024
f8a4a7e
fixing CEL code
JaydipGabani May 11, 2024
f07d100
fixing ci
JaydipGabani May 11, 2024
823c3bb
fixing ci
JaydipGabani May 11, 2024
6d917f7
Merge branch 'master' into CEL-source
JaydipGabani May 13, 2024
a4dce92
merging with master
JaydipGabani May 14, 2024
89a2df0
merging in master and testing with gk 3.16.1
JaydipGabani May 22, 2024
6fe0ef2
testing with gk 3.16.1
JaydipGabani May 22, 2024
bfb2e89
testing with gk 3.16.2
JaydipGabani May 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 9 additions & 5 deletions .github/workflows/workflow.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,8 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
gatekeeper: [ "release-3.13", "release-3.14" ]
gatekeeper: [ "3.14.2", "3.15.1" ]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we prefer to use specific patch versions instead of the latest?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK, we have always tested with past two release versions. I am fine with testing it with latest, unless others have any objections.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using the branch instead of the tag ensures we are always testing against the latest patch version without having to push a PR to update it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK helm can only install specifi version, I am not sure if it can take a branch and install the latest patch on that branch. If we do not use helm, then we would need to use sed or something similar to modify the file and set the flag manually.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 to @JaydipGabani i think this is a better approach now since we previously installed from yaml directly

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah I see. I'm ok with merging this as is. but it would be better to not having to bump the tag all the time. @JaydipGabani you can create an issue to do it as a follow up.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure!

Copy link
Member

@sozercan sozercan May 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to bump the patch version. We'll need to bump the minor version, which is same as today. I am not sure if there's a way we can automate that, but this can be a part of post GK release checklist

engine: [ "cel", "rego" ]
JaydipGabani marked this conversation as resolved.
Show resolved Hide resolved
name: "Integration test on Gatekeeper ${{ matrix.gatekeeper }}"
steps:
- name: Harden Runner
Expand All @@ -81,7 +82,7 @@ jobs:
mkdir -p $GITHUB_WORKSPACE/bin
echo "$GITHUB_WORKSPACE/bin" >> $GITHUB_PATH
make integration-bootstrap
make deploy GATEKEEPER_VERSION=${{ matrix.gatekeeper }}
make deploy GATEKEEPER_VERSION=${{ matrix.gatekeeper }} POLICY_ENGINE=${{ matrix.engine }}

- name: Run integration test
run: |
Expand All @@ -96,7 +97,7 @@ jobs:
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
if: ${{ always() }}
with:
name: logs-int-test-${{ matrix.gatekeeper }}
name: logs-int-test-${{ matrix.gatekeeper }}-${{ matrix.engine }}
path: |
logs-*.json
require_suites:
Expand Down Expand Up @@ -127,7 +128,10 @@ jobs:
make require-sync
gator-verify:
runs-on: ubuntu-latest
name: "Verify assertions in suite.yaml files"
strategy:
matrix:
engine: [ "cel", "rego" ]
name: "Verify assertions in suite.yaml files for ${{ matrix.engine }} policies"
steps:
- name: Harden Runner
uses: step-security/harden-runner@63c24ba6bd7ba022e95695ff85de572c04a18142 # v2.7.0
Expand All @@ -136,4 +140,4 @@ jobs:

- uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4
- run: |
make verify-gator-dockerized
make verify-gator-dockerized POLICY_ENGINE=${{ matrix.engine }}
26 changes: 20 additions & 6 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,11 @@ KIND_VERSION ?= 0.17.0
# note: k8s version pinned since KIND image availability lags k8s releases
KUBERNETES_VERSION ?= 1.26.0
KUSTOMIZE_VERSION ?= 4.5.5
GATEKEEPER_VERSION ?= release-3.11
GATEKEEPER_VERSION ?= 3.15.1
BATS_VERSION ?= 1.8.2
GATOR_VERSION ?= 3.11.0
GATOR_VERSION ?= 3.15.1
GOMPLATE_VERSION ?= 3.11.6
POLICY_ENGINE ?= rego

REPO_ROOT := $(shell git rev-parse --show-toplevel)
WEBSITE_SCRIPT_DIR := $(REPO_ROOT)/scripts/website
Expand All @@ -31,21 +32,34 @@ integration-bootstrap:
TERM=dumb ${GITHUB_WORKSPACE}/bin/kind create cluster --image kindest/node:v${KUBERNETES_VERSION} --wait 5m --config=test/kind_config.yaml

deploy:
kubectl apply -f https://raw.githubusercontent.com/open-policy-agent/gatekeeper/${GATEKEEPER_VERSION}/deploy/gatekeeper.yaml
helm repo add gatekeeper https://open-policy-agent.github.io/gatekeeper/charts
ifeq ($(POLICY_ENGINE), rego)
helm install gatekeeper/gatekeeper --name-template=gatekeeper --namespace gatekeeper-system --create-namespace --version=${GATEKEEPER_VERSION} --set enableK8sNativeValidation=false
JaydipGabani marked this conversation as resolved.
Show resolved Hide resolved
else ifeq ($(POLICY_ENGINE), cel)
helm install gatekeeper/gatekeeper --name-template=gatekeeper --namespace gatekeeper-system --create-namespace --version=${GATEKEEPER_VERSION} --set enableK8sNativeValidation=true
endif

uninstall:
kubectl delete -f https://raw.githubusercontent.com/open-policy-agent/gatekeeper/${GATEKEEPER_VERSION}/deploy/gatekeeper.yaml
helm delete gatekeeper --namespace gatekeeper-system

test-integration:
bats -t test/bats/test.bats

.PHONY: verify-gator
verify-gator:
gator verify ./...
ifeq ($(POLICY_ENGINE), rego)
gator verify ./... --experimental-enable-k8s-native-validation=false
else ifeq ($(POLICY_ENGINE), cel)
gator verify ./... --experimental-enable-k8s-native-validation=true
endif

.PHONY: verify-gator-dockerized
verify-gator-dockerized: __build-gator
$(docker) run -i -v $(shell pwd):/gatekeeper-library gator-container verify ./...
ifeq ($(POLICY_ENGINE), rego)
JaydipGabani marked this conversation as resolved.
Show resolved Hide resolved
$(docker) run -i -v $(shell pwd):/gatekeeper-library gator-container verify ./... --experimental-enable-k8s-native-validation=false
else ifeq ($(POLICY_ENGINE), cel)
$(docker) run -i -v $(shell pwd):/gatekeeper-library gator-container verify ./... --experimental-enable-k8s-native-validation=true
endif

.PHONY: build-gator
__build-gator:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
version: 1.0.2
name: k8srequiredlabels
displayName: Required Labels
createdAt: "2024-05-02T19:01:02Z"
description: Requires resources to contain specified labels, with values matching provided regular expressions.
digest: e41978589a21f07237a098331543f99ede07098acac8d80a4d44cc050a1ad618
license: Apache-2.0
homeURL: https://open-policy-agent.github.io/gatekeeper-library/website/requiredlabels
keywords:
- gatekeeper
- open-policy-agent
- policies
readme: |-
# Required Labels
Requires resources to contain specified labels, with values matching provided regular expressions.
install: |-
### Usage
```shell
kubectl apply -f https://raw.githubusercontent.com/open-policy-agent/gatekeeper-library/master/artifacthub/library/general/requiredlabels/1.0.2/template.yaml
```
provider:
name: Gatekeeper Library
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
resources:
- template.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabels
metadata:
name: all-must-have-owner
spec:
match:
kinds:
- apiGroups: [""]
kinds: ["Namespace"]
parameters:
message: "All namespaces must have an `owner` label that points to your company username"
labels:
- key: owner
allowedRegex: "^[a-zA-Z]+.agilebank.demo$"
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
apiVersion: v1
kind: Namespace
metadata:
name: allowed-namespace
labels:
owner: user.agilebank.demo
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
apiVersion: v1
kind: Namespace
metadata:
name: disallowed-namespace
17 changes: 17 additions & 0 deletions artifacthub/library/general/requiredlabels/1.0.2/suite.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
kind: Suite
apiVersion: test.gatekeeper.sh/v1alpha1
metadata:
name: requiredlabels
tests:
- name: must-have-owner
template: template.yaml
constraint: samples/all-must-have-owner/constraint.yaml
cases:
- name: example-allowed
object: samples/all-must-have-owner/example_allowed.yaml
assertions:
- violations: no
- name: example-disallowed
object: samples/all-must-have-owner/example_disallowed.yaml
assertions:
- violations: yes
76 changes: 76 additions & 0 deletions artifacthub/library/general/requiredlabels/1.0.2/template.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
name: k8srequiredlabels
annotations:
metadata.gatekeeper.sh/title: "Required Labels"
metadata.gatekeeper.sh/version: 1.0.2
description: >-
Requires resources to contain specified labels, with values matching
provided regular expressions.
spec:
crd:
spec:
names:
kind: K8sRequiredLabels
validation:
openAPIV3Schema:
type: object
properties:
message:
type: string
labels:
type: array
description: >-
A list of labels and values the object must specify.
items:
type: object
properties:
key:
type: string
description: >-
The required label.
allowedRegex:
type: string
description: >-
If specified, a regular expression the annotation's value
must match. The value must contain at least one match for
the regular expression.
targets:
- target: admission.k8s.gatekeeper.sh
code:
- engine: K8sNativeValidation
source:
validations:
- expression: '[object, oldObject].exists(obj, obj != null && has(obj.metadata) && variables.params.labels.all(entry, has(obj.metadata.labels) && entry.key in obj.metadata.labels))'
messageExpression: '"missing required label, requires all of: " + variables.params.labels.map(entry, entry.key).join(", ")'
- expression: '[object, oldObject].exists(obj, obj != null && !variables.params.labels.exists(entry, has(obj.metadata.labels) && entry.key in obj.metadata.labels && !string(obj.metadata.labels[entry.key]).matches(string(entry.allowedRegex))))'
message: "regex mismatch"
rego: |
JaydipGabani marked this conversation as resolved.
Show resolved Hide resolved
JaydipGabani marked this conversation as resolved.
Show resolved Hide resolved
package k8srequiredlabels

get_message(parameters, _default) := _default {
not parameters.message
}

get_message(parameters, _) := parameters.message

violation[{"msg": msg, "details": {"missing_labels": missing}}] {
provided := {label | input.review.object.metadata.labels[label]}
required := {label | label := input.parameters.labels[_].key}
missing := required - provided
count(missing) > 0
def_msg := sprintf("you must provide labels: %v", [missing])
msg := get_message(input.parameters, def_msg)
}

violation[{"msg": msg}] {
value := input.review.object.metadata.labels[key]
expected := input.parameters.labels[_]
expected.key == key
# do not match if allowedRegex is not defined, or is an empty string
expected.allowedRegex != ""
not regex.match(expected.allowedRegex, value)
def_msg := sprintf("Label <%v: %v> does not satisfy allowed regex: %v", [key, value, expected.allowedRegex])
msg := get_message(input.parameters, def_msg)
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
version: 1.0.2
name: k8spspallowprivilegeescalationcontainer
displayName: Allow Privilege Escalation in Container
createdAt: "2024-05-02T19:01:02Z"
description: Controls restricting escalation to root privileges. Corresponds to the `allowPrivilegeEscalation` field in a PodSecurityPolicy. For more information, see https://kubernetes.io/docs/concepts/policy/pod-security-policy/#privilege-escalation
digest: 64107d0d609554fcc6740d2a2c45bd5d221c1eedaaaee3a409aec827aa196a64
license: Apache-2.0
homeURL: https://open-policy-agent.github.io/gatekeeper-library/website/allow-privilege-escalation
keywords:
- gatekeeper
- open-policy-agent
- policies
readme: |-
# Allow Privilege Escalation in Container
Controls restricting escalation to root privileges. Corresponds to the `allowPrivilegeEscalation` field in a PodSecurityPolicy. For more information, see https://kubernetes.io/docs/concepts/policy/pod-security-policy/#privilege-escalation
install: |-
### Usage
```shell
kubectl apply -f https://raw.githubusercontent.com/open-policy-agent/gatekeeper-library/master/artifacthub/library/pod-security-policy/allow-privilege-escalation/1.0.2/template.yaml
```
provider:
name: Gatekeeper Library
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
resources:
- template.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sPSPAllowPrivilegeEscalationContainer
metadata:
name: psp-allow-privilege-escalation-container
spec:
match:
kinds:
- apiGroups: [""]
kinds: ["Pod"]
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
apiVersion: v1
kind: Pod
metadata:
name: nginx-privilege-escalation-disallowed
labels:
app: nginx-privilege-escalation
spec:
ephemeralContainers:
- name: nginx
image: nginx
securityContext:
allowPrivilegeEscalation: true
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
apiVersion: v1
kind: Pod
metadata:
name: nginx-privilege-escalation-allowed
labels:
app: nginx-privilege-escalation
spec:
containers:
- name: nginx
image: nginx
securityContext:
allowPrivilegeEscalation: false
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
apiVersion: v1
kind: Pod
metadata:
name: nginx-privilege-escalation-disallowed
labels:
app: nginx-privilege-escalation
spec:
containers:
- name: nginx
image: nginx
securityContext:
allowPrivilegeEscalation: true
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
kind: AdmissionReview
apiVersion: admission.k8s.io/v1beta1
request:
operation: "UPDATE"
object:
apiVersion: v1
kind: Pod
metadata:
name: nginx-privilege-escalation-disallowed
labels:
app: nginx-privilege-escalation
spec:
containers:
- name: nginx
image: nginx
securityContext:
allowPrivilegeEscalation: true
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
kind: Suite
apiVersion: test.gatekeeper.sh/v1alpha1
metadata:
name: allow-privilege-escalation
tests:
- name: allow-privilege-escalation
template: template.yaml
constraint: samples/psp-allow-privilege-escalation-container/constraint.yaml
cases:
- name: example-allowed
object: samples/psp-allow-privilege-escalation-container/example_allowed.yaml
assertions:
- violations: no
- name: example-disallowed
object: samples/psp-allow-privilege-escalation-container/example_disallowed.yaml
assertions:
- violations: yes
- name: disallowed-ephemeral
object: samples/psp-allow-privilege-escalation-container/disallowed_ephemeral.yaml
assertions:
- violations: yes
- name: update
object: samples/psp-allow-privilege-escalation-container/update.yaml
assertions:
- violations: no
Loading