Sync upstream main #133

Signed-off-by: Nic Cope <[email protected]>

Crossplane uses a controller engine to dynamically start claim and XR controllers when a new XRD is installed. Before this commit, each controller gets at least one cache. This is because when I built this functionality, you couldn't stop a single informer within a cache (a cache is basically a map of informers by GVK). When realtime composition is enabled, there are even more caches. One per composed resource GVK. A GVK routed cache routes cache lookups to these various delegate caches. Meanwhile, controller-runtime recently made it possible to stop an informer within a cache. It's also been possible to remove an event handler from an informer for some time (since Kubernetes 1.26). kubernetes-sigs/controller-runtime#2285 kubernetes-sigs/controller-runtime#2046 This commit uses a single client, backed by a single cache, across all dynamic controllers (specifically the definition, offered, claim, and XR controllers). Compared to the current implementation, this commit: * Takes fewer global locks when realtime compositions are enabled. Locking is now mostly at the controller scope. * Works with the breaking changes to source.Source introduced in controller-runtime v0.18. :) I think this makes the realtime composition code a little easier to follow by consolodating it into the ControllerEngine, but that's pretty subjective. Signed-off-by: Nic Cope <[email protected]>

Signed-off-by: Neeraj Nagure <[email protected]>

Signed-off-by: Chuan-Yen Chiang <[email protected]> Signed-off-by: Chuan-Yen Chiang <[email protected]>

…ore. Signed-off-by: Chuan-Yen Chiang <[email protected]>

Signed-off-by: Chuan-Yen Chiang <[email protected]>

Use a single cache for all dynamic controllers (i.e. XRs and claims)

Signed-off-by: Chuan-Yen Chiang <[email protected]>

…kout-digest chore(deps): update actions/checkout digest to a5ac7e5 (master)

Signed-off-by: Neeraj Nagure <[email protected]>

Unset some UP env vars to avoid unexpect test behaviour.

Enhanced error logging

…l-action-digest chore(deps): update github/codeql-action digest to 9fdb3e4 (master)

…to v0.8.2

…ocker-docker-credential-helpers-0.x fix(deps): update module github.com/docker/docker-credential-helpers to v0.8.2 (master)

Signed-off-by: Philippe Scorsolini <[email protected]>

…angci-lint-1.x chore(deps): update dependency golangci/golangci-lint to v1.58.2 (master)

Add CloudScript adopter. Signed-off-by: Helias Rodrigues <[email protected]>

Update ADOPTERS.md

Signed-off-by: Jared Watts <[email protected]>

adopters: add Zuru Tech Italy to ADOPTERS.md

…cov-action-digest chore(deps): update codecov/codecov-action digest to 125fc84 (master)

Signed-off-by: Philippe Scorsolini <[email protected]>

chore: ignore checklist for renovate prs

…-trivy-action-0.x

Signed-off-by: Philippe Scorsolini <[email protected]>

…olang-jwt-jwt-v5-5.x fix(deps): update module github.com/golang-jwt/jwt/v5 to v5.2.1 (master)

Signed-off-by: Jared Watts <[email protected]>

build: allow CRD patch dir to already exist

….19.1 Signed-off-by: Jared Watts <[email protected]>

Signed-off-by: Nic Cope <[email protected]>

So that it stops warning about it. Signed-off-by: Nic Cope <[email protected]>

The current format uses }, { style, which I find makes it much more laborious to edit, especially when I want to quickly copy and modify a block. Signed-off-by: Nic Cope <[email protected]>

Signed-off-by: Nic Cope <[email protected]>

We could reimplement this with Earthly but I suspect it's not used. Signed-off-by: Nic Cope <[email protected]>

I'm not currently reimplementing the crds.clean target that trims the leading --- from the CRDs because it's unclear whether it's still needed. Signed-off-by: Nic Cope <[email protected]>

This is hacky and I hate it, but I can't think of a better approach. Signed-off-by: Nic Cope <[email protected]>

Its version isn't specified there anymore. Signed-off-by: Nic Cope <[email protected]>

Signed-off-by: Nic Cope <[email protected]>

TARGETPLATFORM is incorrect on MacOS, where TARGETOS will be Linux. There we want USERPLATFORM, i.e. Darwin. Signed-off-by: Nic Cope <[email protected]>

Proposal: Try using Earthly instead of Make for one release

Signed-off-by: Nic Cope <[email protected]>

I thought the experimental --aws flag did this, but it seems like it might not. There's not much documentation on it, so for now just pass AWS credentials explicitly. Signed-off-by: Nic Cope <[email protected]>

Signed-off-by: Nic Cope <[email protected]>

We'll have to see whether this helps at all. I expect it to slow down master builds, since we'll spend time pushing everything to cache. For PR builds, I hope it'll improve some by being able to avoid compiling / linting / testing when nothing changed. Signed-off-by: Nic Cope <[email protected]>

Enable Earthly remote caching, pass AWS credentials explicitly

…kport-action-2.x chore(deps): update zeebe-io/backport-action action to v2.5.0 (master)

Signed-off-by: Nic Cope <[email protected]>

For both required environment variables to be set we need >>, or the second will just overwrite the first. Signed-off-by: Nic Cope <[email protected]>

I dropped this thinking a little more detail wouldn't hurt. Turned out it was there for a reason - there's so much output the CI job logs get truncated. Signed-off-by: Nic Cope <[email protected]>

fix(deps): update module github.com/google/go-containerregistry to v0.19.1

Fix Earthly remote cache, and typo in Renovate configuration

This was lost in translation from Make -> Earthly. Signed-off-by: Nic Cope <[email protected]>

…cies chore(deps): pin dependencies (master)

…-action-digest chore(deps): update docker/login-action digest to 0d4c9c5 (master)

Set --cache-control flag when copying Helm index

Signed-off-by: Nic Cope <[email protected]>

Enforce immutable fields using CEL rules Signed-off-by: Neeraj Nagure <[email protected]>

Signed-off-by: Nic Cope <[email protected]>

Right now the order is important. See earthly/earthly#4177 Signed-off-by: Nic Cope <[email protected]>

The repo has two forms of tag, and this is the one we're interested in. Signed-off-by: Nic Cope <[email protected]>

chore(deps): update dependency helm/helm to v3.15.1 (master)

Signed-off-by: Nic Cope <[email protected]>

Bump golangci-lint to v1.59.0

Fix earthly E2E test results, CodeQL Renovate version

Signed-off-by: Nic Cope <[email protected]>

Fix EARTHLY_VERSION matching in GitHub Workflows

…m-docs-1.x chore(deps): update dependency norwoodj/helm-docs to v1.13.1 (master)

…hly-0.x chore(deps): update dependency earthly/earthly to v0.8.13 (master)

…lf-gotestsum-1.x chore(deps): update dependency gotestyourself/gotestsum to v1.12.0 (master)

It needs to be in the Docker container, not the runner. Signed-off-by: Nic Cope <[email protected]>

Right now master has an Earthfile and all release branches have a Makefile. Eventually they'll all arrive at one implementation or the other depending on whether we decide to adopt Earthly. Signed-off-by: Nic Cope <[email protected]>

We download release artifacts, so releases are a better signal. With tags, it's possible a new tag is created without a corresponding release. This would be a false positive, since we wouldn't actually be able to download a release artifact. Signed-off-by: Nic Cope <[email protected]>

The URL should include the channel, e.g. see: https://github.com/crossplane/build/blob/b0dfb8fbe8c626e3e0f0b6efdfac926d750adf48/makelib/helm.mk#L154 Signed-off-by: Nic Cope <[email protected]>

Fix Renovate running `earthly`

${GITHUB_REF##*/} somewhat confusingly resolves to 'merge' for PRs and (e.g.) 'master' for merges to a branch. I'm not sure 'merge' is what we _really_ want (as opposed to the PR number) but it's what the build submodule does, so keep compatibility with that. EARTHLY_GIT_BRANCH was working for master, but was resolving to HEAD for PR builds. Signed-off-by: Nic Cope <[email protected]>

Signed-off-by: Nic Cope <[email protected]>

…lecthomas-kong-0.x fix(deps): update module github.com/alecthomas/kong to v0.9.0 (master)

Match build submodule artifact pushing behavior

…ws-smithy-go-1.x fix(deps): update module github.com/aws/smithy-go to v1.20.2 (master)

…o-git-go-git-v5-5.x fix(deps): update module github.com/go-git/go-git/v5 to v5.12.0 (master)

…li-2.x chore(deps): update amazon/aws-cli docker tag to v2.15.61 (master)

crossplane/crossplane#5765 Follow up to the above PR, which broke it. It should be refs/heads, not refs/head. Signed-off-by: Nic Cope <[email protected]>

Fix check for master branch to enable pushing

…igs-kind-0.x chore(deps): update dependency kubernetes-sigs/kind to v0.23.0 (master)

Signed-off-by: Neeraj Nagure <[email protected]>

…rometheus-client_golang-1.x fix(deps): update module github.com/prometheus/client_golang to v1.19.1 (master)

Signed-off-by: Neeraj Nagure <[email protected]>

…l-action-digest chore(deps): update github/codeql-action digest to f079b84 (master)

The `+multiplatform-build` step includes `+go-multiplatform-build`, so similarly the `build` step should include `+go-build`. When calling the `+go-build` target, the OS/arch that is used is the native, but in some cases, eg on Docker Desktop on Mac OS X, emulation is used, thus the $NATIVEPLATFORM and $TARGETPLATFORM variables incorrectly report linux/amd64. The USERPLATFORM var though always returns the correct OS/arch of the host that is triggering the build. When calling the `+build` target locally, the expectation is to produce a binary for our own system, so that we can use/test it afterwards. Thus, this commit changes the `+build` target to call `+go-build` but for platform=$USERPLATFORM. Co-authored-by: Nic Cope <[email protected]> Signed-off-by: Theo Chatzimichos <[email protected]>

…ld_go fix(Earthfile): Build also the go binaries when running +build

This is a regression compared to the Makefile era, see for example https://releases.crossplane.io/stable/v1.16.0/bin/windows_amd64/ and also what install.sh expects. Signed-off-by: Theo Chatzimichos <[email protected]>

fix(Earthfile): Add .exe extension in the windows binaries

Signed-off-by: Blake R <[email protected]>

Signed-off-by: johnathan-sq <[email protected]>

fix(runtime): remove container after run

fixes #5707 Add a loader with configurable concurrency to load resources in concurrent manner. The xrm client delegates to the loader for resource load and supports a functional option to set the concurrency. Add a `--concurrency` flag for the `crank beta trace` command and configure the xrm client appropriately. Signed-off-by: gotwarlost <[email protected]>

* sorts children after concurrent load using the same algorithm that crossplane uses to sort resource refs * sets default QPS and burst when not set to the same values as a recent PR * updates test structure to match existing tests Signed-off-by: gotwarlost <[email protected]>

Signed-off-by: gotwarlost <[email protected]>

* remove context from the loader interface when not needed * change member name from l to rl * simplify tests to lose regexes * add concurrency docs * remove module global for channel capacity, pass in as parameter * use range over ints instead of for loops since linter complains otherwise * do not run goroutines when there are no child refs to add * add more docs for sorting logic and why it is needed Signed-off-by: gotwarlost <[email protected]>

Signed-off-by: Blake R <[email protected]>

…constraints Add Topology Spread Constraints to Helm Chart

Signed-off-by: dalton hill <[email protected]>

Composition Function Events and Status Conditions

Signed-off-by: Mehmet Enes <[email protected]>

Add unknown field check to `beta validate`

Signed-off-by: Jared Watts <[email protected]>

README: add SIG-docs to the list of SIGs

Signed-off-by: Mehmet Enes <[email protected]>

Co-authored-by: Ezgi Demirel <[email protected]> Signed-off-by: Mehmet Enes <[email protected]>

Signed-off-by: Mehmet Enes <[email protected]>

improve trace client performance using concurrent resource load

Signed-off-by: stellrust <[email protected]>

Signed-off-by: Cem Mergenci <[email protected]>

Signed-off-by: Jiri Tyr <[email protected]>

Fix broken links

Signed-off-by: Mehmet Enes <[email protected]>

Signed-off-by: Lawrence Aiello <[email protected]> Signed-off-by: aiell0 <[email protected]>

Signed-off-by: aiell0 <[email protected]>

docs: add Rogo to ADOPTERS.md

Add configuration.meta/crossplane.yaml support for dependencies to beta validate

Signed-off-by: Mehmet Enes <[email protected]>

Co-authored-by: Jared Watts <[email protected]> Signed-off-by: Mehmet Enes <[email protected]>

Support dependecies with version constraints in crank validate

Use more global cache directory at validate

Signed-off-by: Mehmet Enes <[email protected]>

Co-authored-by: Jared Watts <[email protected]> Signed-off-by: Mehmet Enes <[email protected]>

Signed-off-by: Mehmet Enes <[email protected]>

download/cache full graph of dependencies at `beta validate`

…ible [security]

chore: fix comment for struct field

Signed-off-by: Alper Rifat Ulucinar <[email protected]>

This happened when I ran `earthly +reviewable`. Not sure how this got into the master branch? Signed-off-by: Nic Cope <[email protected]>

Previously if a composed resource appeared in an XR's spec.resourceRefs but didn't have a controller reference the XR would refuse to garbage collect it. The XR would then remove the composed resource from its resource refs, effectively orphaning it. Now if the composed resource has _no_ controller, the XR will delete it. Most likely it was owned by the XR, then had its controller ref stripped (e.g. due to being backed up and restored using a tool like Velero). If the composed resource is controlled by another resource, we'll now return an error rather than silently orphaning it. Signed-off-by: Nic Cope <[email protected]>

…m-docker-docker-vulnerability fix(deps): update module github.com/docker/docker to v25.0.6+incompatible [security] (master)

Signed-off-by: Philippe Scorsolini <[email protected]>

Delete resources that don't have a controller but appear in resourceRefs

Signed-off-by: Philippe Scorsolini <[email protected]>

This breaks extra resource handling out of the main function composer loop (and crossplane beta render loop). It's now implemented as a FunctionRunner that wraps the existing FunctionRunner implementation, fetching extra resources as needed. Signed-off-by: Nic Cope <[email protected]>

Signed-off-by: Lucas Käldström <[email protected]>

Implement extra resource handling as a `FunctionRunner`

Signed-off-by: Nic Cope <[email protected]>

We want to promote the v1beta1 function proto to v1. This isn't a breaking change. There's actually no changes - we just want to signal that it's reached v1. Crossplane (the client) must continue to support v1beta1 functions (i.e. servers) in order to remain compatible with existing functions before they update to SDKs that use the v1 protobuf. To do this, Crossplane will first attempt to send a v1 RPC. If the server responds that v1 is unimplemented, it'll fall back to v1beta1. The v1beta1 protos are now automatically replicated from v1. So all changes made to v1 will be propagated to v1beta1. This means Crossplane can work entirely with v1 types internally. When it needs to fall back to send a v1beta1 RunFunctionRequest, it converts a v1 request by serializing it to bytes, then deserializing it into a v1beta1 request. As far as I can tell this is the best way to convert a Go protobuf message to an identical message of a different type. Signed-off-by: Nic Cope <[email protected]>

Signed-off-by: Nic Cope <[email protected]>

I missed this when moving it out from crossplane beta xpkg. Signed-off-by: Nic Cope <[email protected]>

This method includes the version (v1 or v1beta1). Including it in the metrics will help folks identify whether their control planes are running any functions that are still speaking only v1beta1. Signed-off-by: Nic Cope <[email protected]>

It's still generally relevant, but it was at the functions test which isn't beta anymore. Signed-off-by: Nic Cope <[email protected]>

Promote composition functions to v1, and deprecate patch and transform

Signed-off-by: Jared Watts <[email protected]>

Documentation housekeeping for graduation application

Using Update instead of CreateOrUpdate avoids a race by failing with a conflict if the underlying Secret changed between the call to Get and the call to Update. Signed-off-by: Jonathan Oddy <[email protected]>

Fix race condition creating certs in init.

Signed-off-by: Philippe Scorsolini <[email protected]>

chore(deps): bump crossplane-runtime to v1.18.0-rc.0

Signed-off-by: Philippe Scorsolini <[email protected]>

chore: add 1.17 release branch to renovate

Signed-off-by: Philippe Scorsolini <[email protected]>

Add secrets.crossplane.io grant rules to RBAC manager roles, even though the feature is alpha.

ci: --push on promote

Apparently you get a magic URL in your Docker config file when you omit the registry. It seems to be needed to successfully push. Signed-off-by: Nic Cope <[email protected]>

Omit docker login registry arg when promoting tag in Docker Hub

…o crd spec Signed-off-by: Christian Franco <[email protected]>

fix: ensure x-kubernetes-preserve-unknown-fields is copied from xrd to crd spec

Signed-off-by: Caio Almeida <[email protected]>

docs: changing gympass adopters to wellhub company

Signed-off-by: Philippe Scorsolini <[email protected]>

Co-authored-by: Jared Watts <[email protected]> Signed-off-by: Philippe Scorsolini <[email protected]>

chore: drop 1.14 and update 1.17 date from release table

… master to main. Signed-off-by: Jean du Plessis <[email protected]>

Signed-off-by: Jean du Plessis <[email protected]>

Code updates after renaming `master` → `main`

Signed-off-by: Jean du Plessis <[email protected]>

Update e2e tests by adding a description for `features.New` calls

They were made immutable by mistake. Signed-off-by: Nic Cope <[email protected]>

…ut-digest chore(deps): update actions/checkout digest to 692973e (main)

Make composition revision numbers mutable

Delete packages in lock having same name and distinct identifier. Signed-off-by: Jose Francisco Dillet Alfonso <[email protected]>

Signed-off-by: Jose Francisco Dillet Alfonso <[email protected]>

Signed-off-by: Christopher Haar <[email protected]>

…move-beta fix(ga): remove beta for CompositionModePipeline

Fix "Missing node in tree error" after updating a package source

Retry on conflict during CRD dry-run updates in XRD validation webhook

Signed-off-by: Hasan Turken <[email protected]>

Signed-off-by: Eric Deitrick <[email protected]>

Update ADOPTERS.md

…am-main

Sync upstream main #133

Sync upstream main #133

Commits on May 20, 2024

Commits on May 21, 2024

Commits on May 22, 2024

Commits on May 23, 2024

Commits on May 24, 2024

Commits on May 25, 2024

Commits on May 26, 2024

Commits on May 28, 2024

Commits on May 29, 2024

Commits on May 30, 2024

Commits on May 31, 2024

Commits on Jun 1, 2024

Commits on Jun 4, 2024

Commits on Jun 9, 2024

Commits on Jun 10, 2024

Commits on Jun 12, 2024

Commits on Jun 20, 2024

Commits on Jun 21, 2024

Commits on Jun 22, 2024

Commits on Jun 24, 2024

Commits on Jun 26, 2024

Commits on Jun 28, 2024

Commits on Jun 29, 2024

Commits on Jun 30, 2024

Commits on Jul 3, 2024

Commits on Jul 4, 2024

Commits on Jul 11, 2024

Commits on Jul 12, 2024

Commits on Jul 13, 2024

Commits on Jul 15, 2024

Commits on Jul 17, 2024

Commits on Jul 18, 2024

Commits on Jul 19, 2024

Commits on Jul 22, 2024

Commits on Jul 23, 2024

Commits on Jul 25, 2024

Commits on Jul 30, 2024

Commits on Jul 31, 2024

Commits on Aug 2, 2024

Commits on Aug 6, 2024

Commits on Aug 10, 2024

Commits on Aug 12, 2024

Commits on Aug 13, 2024

Commits on Aug 14, 2024

Commits on Aug 15, 2024

Commits on Aug 16, 2024

Commits on Aug 17, 2024

Commits on Aug 18, 2024

Commits on Aug 19, 2024

Commits on Aug 20, 2024

Commits on Aug 21, 2024

Commits on Aug 22, 2024

Commits on Aug 28, 2024

Commits on Aug 29, 2024

Commits on Aug 30, 2024

Commits on Aug 31, 2024

Commits on Sep 3, 2024

Commits on Sep 4, 2024

Commits on Sep 5, 2024

Commits on Sep 6, 2024

Commits on Sep 10, 2024

Commits on Sep 11, 2024

Commits on Sep 12, 2024

Commits on Sep 13, 2024