Apiserversource webhook test #5327

aliok · 2021-04-29T10:00:46Z

Part of #4921

Proposed Changes

🧹 Add a test that makes sure webhook kicks in to validate ApiServerSource updates

Pre-review Checklist

At least 80% unit test coverage
E2E tests for any new behavior
Docs PR for any user-facing impact
Spec PR for any new API feature
Conformance test for any change to the spec

Release Note

Docs

knative-prow-robot · 2021-04-29T10:00:55Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please ask for approval from aliok after the PR has been reviewed.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

test/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

codecov · 2021-04-29T10:07:52Z

Codecov Report

Merging #5327 (adb332e) into main (c2d6649) will decrease coverage by 0.59%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #5327      +/-   ##
==========================================
- Coverage   82.95%   82.36%   -0.60%     
==========================================
  Files         211      189      -22     
  Lines        6291     5902     -389     
==========================================
- Hits         5219     4861     -358     
+ Misses        742      720      -22     
+ Partials      330      321       -9

Impacted Files	Coverage Δ
pkg/apis/sources/v1beta1/apiserver_conversion.go
pkg/apis/sources/v1beta1/sinkbinding_lifecycle.go
pkg/apis/sources/v1beta1/apiserver_types.go
pkg/apis/sources/v1beta1/container_lifecycle.go
pkg/apis/sources/v1beta1/ping_lifecycle.go
pkg/apis/sources/v1beta1/ping_validation.go
pkg/apis/sources/v1beta1/sinkbinding_defaults.go
pkg/apis/sources/v1beta1/apiserver_lifecycle.go
pkg/apis/sources/v1beta1/apiserver_validation.go
pkg/apis/sources/v1beta1/container_types.go
... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c2d6649...adb332e. Read the comment docs.

aliok · 2021-04-29T11:55:52Z

@vaikas Following up on #5304

I've added a test for ApiServerSource to make sure webhook is triggered and it does the validation correctly.
It doesn't do the entire validation checks for ApiServerSource. There's only 1 case that checks webhook rejects invalid spec. I guess that serves our purpose with the original problem: making sure webhook validation is configured correctly.

I didn't want to go beyond this point w/o checking with you first.

I wrote the test as if it is a conformance test but there is no spec for ApiServerSource in https://github.com/knative/specs/tree/main/specs/eventing. I see there's a task for that here: #4933.
I would rather do this same PR for other resources instead of the context switch to writing a spec. So, let's solve all the problems in this PR and I can write tests that check webhook validation configuration for other resources.

Question: The tests in apiserversource_validation_test.go checks all the validations. Should we retest these rules in the conformance tests?

…d to refetch the resource. ...and, we do that already!

test/rekt/features/apiserversource/readyness.go

vaikas

Thanks! Couple of tiny things since still in wip mode :)

test/rekt/apiserversource_test.go

test/rekt/resources/apiserversource/apiserversource.go

vaikas · 2021-04-29T15:06:07Z

test/rekt/resources/apiserversource/apiserversource_test.go

+
+	sinkRef := &duckv1.KReference{
+		Kind:       "sinkkind",
+		Namespace:  "sinknamespace",


I wonder if above this we might want to make the comment about namespace being not used? Or above where you skipped it? Cause here when you create it with this namespace, it might not be clear to the user why L136 is not sinknamespace?

Actually, the namespace shouldn't be skipped here as ApiServerSource allows cross namespace sink. I was think it doesn't. So, I will cancel the skipping and also fix the test code.

I pushed a commit for not skipping the namespace.

aliok · 2021-04-29T19:47:46Z

test/rekt/resources/account_role/role.go

+				// TODO: skip for now
+				//"resourceNames":   resource.ResourceNames,
+				//"nonResourceURLs": resource.NonResourceURLs,


I didn't want to add these for now in order not to waste time and focus on getting feedback on the general idea of the PR.

Would we ever need these? Happy to add code that handles these too, if necessary.

aliok · 2021-04-29T19:50:33Z

test/rekt/resources/account_role/role.yaml

+{{ end }}
+
+{{ if .rules }}
+rules:
+  {{ range $_, $rule := .rules }}


I wanted to extend and reuse the existing account_role here.
In the future if we need to add more functionality to this template, I would create a separate template; but it is manageable for now.

n3wscott · 2021-04-29T20:24:11Z

test/rekt/apiserversource_test.go

+	sourceName := "myapiserversource"
+
+	// Install and wait for a Ready ApiServerSource.
+	env.Prerequisite(ctx, t, apiserversource.GoesReady(sourceName))


I would expect the source to have an install step or have the name of the resource passed in but I am not seeing either.

n3wscott · 2021-04-29T20:26:36Z

test/rekt/features/apiserversource/control_plane.go

+			// wait until the ApiServerSource is ready or fail.
+			// otherwise, some race happens and Kubernetes tells us to refetch the object as it was updated, although
+			// we just refetched the object a couple of lines above.
+			apiserversource.IsReady(apiServerSource.Name)(ctx, t)


this will race with the update, likely you will test IsReady before the reconciler acts on the above Update and never see the resource go unready and then ready again.

n3wscott · 2021-04-29T20:28:06Z

test/rekt/features/apiserversource/control_plane.go

+
+	f.Stable("EventMode").
+		Must("ApiServerSource MUST allow retrieving the event payloads of ObjectReference or ResourceEvent",
+			eventMode()).


this test does not seem to test what the spec is saying. The test confirms we can set the value, but the conformance line is talking about the shape of the event payload I believe, so your test will need to get more involved and see the event on the data plane side.

n3wscott · 2021-04-29T20:30:58Z

test/rekt/features/apiserversource/readiness.go

+
+	sacmName := feature.MakeRandomK8sName("apiserversource")
+	f.Setup("Create Service Account for ApiServerSource",
+		account_role.Install(sacmName,


If you look at the broker/channel changes we found that this pattern of combining Install with GoesReady was not idea for test composition (my bad). I would recommend this being split into two methods: GoesReady which tests that the thing you pass is does go ready (and we have some helpers for that. and Install which installs the ApiServerSource resource.

n3wscott · 2021-04-29T20:32:52Z

test/rekt/resources/account_role/role_test.go

@@ -196,3 +197,76 @@ func Example_addressableResolver() {
 	//   name: addressable-resolver-collector-foo
 	//   apiGroup: rbac.authorization.k8s.io
 }
+
+func Example_withRoleAndRules() {


👍 I like this testing pattern

aliok · 2021-04-29T22:04:25Z

/close

I will close this one. I will write a separate test that checks the webhook configuration.

knative-prow-robot · 2021-04-29T22:04:30Z

@aliok: Closed this PR.

In response to this:

/close

I will close this one. I will write a separate test that checks the webhook configuration.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

aliok added 4 commits April 29, 2021 12:18

Add apiserversource rekt resources

b1a3967

Add apiserversource rekt resources

2a88ece

Adapt account_role rekt resource to accept rules

0cf73ee

ApiServerSource eventMode conformance tests

f667c5b

knative-prow-robot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Apr 29, 2021

knative-prow-robot added the area/test-and-release Test infrastructure, tests or release label Apr 29, 2021

google-cla bot added the cla: yes Indicates the PR's author has signed the CLA. label Apr 29, 2021

aliok changed the title ~~Apiserversource webhook test~~ [WIP] Apiserversource webhook test Apr 29, 2021

knative-prow-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 29, 2021

Print the unexpected error

e676726

Wait until ApiServerSource is ready, otherwise Kubernetes says we nee…

3b92783

…d to refetch the resource. ...and, we do that already!

lionelvillard reviewed Apr 29, 2021

View reviewed changes

test/rekt/features/apiserversource/readyness.go Outdated Show resolved Hide resolved

vaikas reviewed Apr 29, 2021

View reviewed changes

aliok added 4 commits April 29, 2021 22:16

Rename readyness.go to readiness.go

c7002bd

Fix some copy paste mistakes. shhh!

a2533bb

Allow cross namespace sink

d3f63b6

Add comments explaining the wait for ready

adb332e

aliok commented Apr 29, 2021

View reviewed changes

aliok changed the title ~~[WIP] Apiserversource webhook test~~ Apiserversource webhook test Apr 29, 2021

knative-prow-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 29, 2021

n3wscott reviewed Apr 29, 2021

View reviewed changes

knative-prow-robot closed this Apr 29, 2021

aliok mentioned this pull request Apr 30, 2021

[v1] Test to make sure webhook validates ApiServerSource #5329

Merged

5 tasks

aliok deleted the apiserversource-webhook-test branch February 16, 2022 11:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Apiserversource webhook test #5327

Apiserversource webhook test #5327

aliok commented Apr 29, 2021

knative-prow-robot commented Apr 29, 2021

codecov bot commented Apr 29, 2021 •

edited

Loading

aliok commented Apr 29, 2021

vaikas left a comment

vaikas Apr 29, 2021

aliok Apr 29, 2021

aliok Apr 29, 2021

aliok Apr 29, 2021

aliok Apr 29, 2021 •

edited

Loading

n3wscott Apr 29, 2021

n3wscott Apr 29, 2021

n3wscott Apr 29, 2021

n3wscott Apr 29, 2021

n3wscott Apr 29, 2021

aliok commented Apr 29, 2021

knative-prow-robot commented Apr 29, 2021

Apiserversource webhook test #5327

Apiserversource webhook test #5327

Conversation

aliok commented Apr 29, 2021

Proposed Changes

Pre-review Checklist

knative-prow-robot commented Apr 29, 2021

codecov bot commented Apr 29, 2021 • edited Loading

Codecov Report

aliok commented Apr 29, 2021

vaikas left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aliok Apr 29, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aliok commented Apr 29, 2021

knative-prow-robot commented Apr 29, 2021

codecov bot commented Apr 29, 2021 •

edited

Loading

aliok Apr 29, 2021 •

edited

Loading