-
Notifications
You must be signed in to change notification settings - Fork 22
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update Kubernetes Cluster controller and add support for meta.externalName #43
base: main
Are you sure you want to change the base?
Conversation
Signed-off-by: Vlad Fratila <[email protected]>
4300017
to
19d3784
Compare
Signed-off-by: Vlad Fratila <[email protected]>
Signed-off-by: Vlad Fratila <[email protected]>
Signed-off-by: Vlad Fratila <[email protected]>
Signed-off-by: Vlad Fratila <[email protected]>
Signed-off-by: Vlad Fratila <[email protected]>
Signed-off-by: Vlad Fratila <[email protected]>
Applications are being installed on Create, but not added on Update. App name is
|
Signed-off-by: Vlad Fratila <[email protected]>
I'd like to leave a note here. The Previously the code would call Update and trigger resource updates. This made it so that the reconciler never issued proper log messages. With this change, you can clearly see when the reconciler requests a Create or Update; or whenever it sees an error. |
@@ -43,11 +43,16 @@ type CivoKubernetesConnectionDetails struct { | |||
type CivoKubernetesSpec struct { | |||
xpv1.ResourceSpec `json:",inline"` | |||
Name string `json:"name"` | |||
Region string `json:"region,omitempty"` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The destination region is set in the ProviderConfig, so we're not setting in the spec of the resource, so I would keep as it is
// +required | ||
// +immutable | ||
// NOTE: This can only be set at creation time. Changing this value after creation will not move the cluster into another network.. | ||
NetworkID *string `json:"networkId"` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
interesting addition, I would set is as optional to be backward compatible and I would add the optional firewallID
and the optional firewallRule
because civo API use these. If they're absent (as is situation), we still use the default ones.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see, the network is actually required in the API, but the Terraform module has this field as optional. This is because when the network is missing, it queries the Civo API to find out the default network ID by doing client.GetDefaultNetwork()
.
I'll add the same thing here, and it should work to get the network back to optional, and make it consistent with Terraform.
pools: | ||
- id: "8382e422-dcdd-461f-afb4-2ab67f171c3e" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it's very uneasy to generate UUID and pass to the manifest, but it doesn't mean we can sent friendly names like these, because they must be unique in our database. I would left like it is. In the API we can auto generate those, but then it means we cannot update through crossplane if we don't provide a reference at all.
civoClient, err := civocli.NewCivoClient(string(s.Data["credentials"]), providerConfig.Spec.Region) | ||
civoClient, err := civogo.NewClient(string(s.Data["credentials"]), providerConfig.Spec.Region) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why removing the civocli
wrapper object?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
civocli
looks to be an old package committed to this repo. In the meantime, I see that the new API is implemented in the civogo
client. I think this is the right way to go, perhaps the API team can confirm?
return managed.ExternalObservation{ResourceExists: false}, err | ||
return managed.ExternalObservation{}, nil |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if err != nil
why removing the error return and return nil
error?
civoCluster, err := e.civoClient.GetK3sCluster(cr.Spec.Name) | ||
|
||
// at the first call, this id will be the cluster name; civo should return 404 | ||
existingClusterID := meta.GetExternalName(cr.GetObjectMeta()) | ||
civoCluster, err := e.civoClient.GetKubernetesCluster(existingClusterID) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same as above: not sure what meta.GetExternalName would return for a crossplane resource: it should fetch the annotation crossplane.io/external-name but this doesn't correspond to the clusterID as it's requested by the GetKubernetesCluster method (that now, as I see, it's never used).
|
||
clusterRegion := cr.Spec.Region | ||
if clusterRegion == "" { | ||
clusterRegion = e.civoClient.Region | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we don't use region in the spec, it's implicit in the provider configs
err = e.civoClient.CreateNewK3sCluster(cr.Spec.Name, cr.Spec.Pools, cr.Spec.Applications, cr.Spec.CNIPlugin, cr.Spec.Version) | ||
kc := &civogo.KubernetesClusterConfig{ | ||
Name: cr.Spec.Name, | ||
Region: clusterRegion, | ||
NetworkID: *cr.Spec.NetworkID, | ||
Pools: cr.Spec.Pools, | ||
Applications: strings.Join(cr.Spec.Applications, ","), | ||
Tags: strings.Join(cr.Spec.Tags, " "), | ||
} | ||
|
||
if cr.Spec.CNIPlugin != nil { | ||
kc.CNIPlugin = *cr.Spec.CNIPlugin | ||
} | ||
if cr.Spec.Version != nil { | ||
kc.KubernetesVersion = *cr.Spec.Version | ||
} | ||
|
||
if cr.Spec.FirewallID != nil { | ||
kc.InstanceFirewall = *cr.Spec.FirewallID | ||
} | ||
|
||
newCluster, err := e.civoClient.NewKubernetesClusters(kc) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would readd the wrapper function and move the creation of the createCluster request in the originary class. Besides that, if you added optional networkID
, firewallID
, firewallRule
I would pass those to the request OR, in their absence, I would pass the default network ID as it is now, without removing that logic.
meta.SetExternalName(cr, newCluster.ID) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is interesting, effectively then you can use the function you proposed above to directly fetch the cluster by ID :claps
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, external-name
is Crossplane magic to allow mapping Cloud IDs to Kubernetes IDs. During Create, a controller is supposed to fetch it and write it here.
Civo API uses cloud-generated IDs for resource identification, so this maps to using external-name
in Crossplane. It looks like the Civo API always generates IDs, so this should be the main way of doing things. Because you can reference this ID directly to the API in most cases, and you would get the resource back. This is exactly how Upjet providers implement this functionality, so the main AWS/GCP/Azure Crossplane providers all store the Cloud IDs via external-name
.
Your comment above about pools
IDs is interesting, I need to look more into that. Right now, I don't use the pools
api to directly update pools, since there is no pools
CRD. So this works for pool updates, just because it uses a cluster-scoped Civo API.
if err := e.civoClient.UpdateK3sCluster(desiredCivoCluster, remoteCivoCluster, providerConfig); err != nil { | ||
desiredClusterConfig := &civogo.KubernetesClusterConfig{ | ||
Pools: desiredCivoCluster.Spec.Pools, | ||
Region: desiredCivoCluster.Spec.Region, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@RealHarshThakur thoughts if we should introduce Region
in the crossplane resource's spec? Now we don't have those.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Region is required by the API, so this is why it needs to be passed here. I think the CRD should have it too, it would be good to follow the API as close as possible. And right now, it still picks up the default Region from the ProviderConfig.
Another argument in favour of the region is that if you want to deploy in multiple regions, you can use the same ProviderConfig.
I ran into this problem that cluster updates are never performed. While trying to debug I actually made good headway.
The problem is that the Kubernetes Civo API changed quite a bit, and now only works via Cluster ID (which makes a ton of sense). The current controller for clusters has two problems:
Changes:
Fixes #41
I have:
make reviewable test
to ensure this PR is ready for review.Tests
Open issues
NOTE: I've bumped
/build
to latest, since I couldn't runmake generate
anymore.