Skip to content

Commit

Permalink
refactor: move connectivity under global key (#294) (#310)
Browse files Browse the repository at this point in the history
* move connectivity under global key

* move baseDomain under global connectivity

* update CI values

* move connectivity requirement to correct location

* correct missed variable location

* update changelog

* update docs
  • Loading branch information
glitchcrab authored Jun 7, 2024
1 parent d6ddb5d commit e28a497
Show file tree
Hide file tree
Showing 19 changed files with 443 additions and 418 deletions.
23 changes: 23 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,29 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

- Remove the interface to set `etcd` and `coredns` images to let kubeadm take care of it.

### **Breaking change**.

<details>
<summary>How to migrate values</summary>

Using `yq`, migrate to the new values layout with the following command:

```bash
#!/bin/bash
yq eval --inplace 'with(select(.connectivity != null); .global.connectivity = .connectivity) |
with(select(.baseDomain != null); .global.connectivity.baseDomain = .baseDomain) |
del(.connectivity) |
del(.baseDomain)' values.yaml
```

</details>

### Changed

- Move Helm values property `.Values.connectivity` to `.Values.global.connectivity`.
- Move Helm values property `.Values.baseDomain` to `.Values.global.connectivity.baseDomain`.

## [0.52.1] - 2024-05-16

### Fixed
Expand Down
96 changes: 48 additions & 48 deletions helm/cluster-cloud-director/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,57 +35,58 @@ Properties within the `.internal` top-level object
| `internal.useAsManagementCluster` | **Display as management cluster**|**Type:** `boolean`<br/>**Default:** `false`|

### Connectivity
Properties within the `.connectivity` top-level object
Properties within the `.global.connectivity` object
Configurations related to cluster connectivity such as container registries.

| **Property** | **Description** | **More Details** |
| :----------- | :-------------- | :--------------- |
| `connectivity.containerRegistries` | **Container registries** - Endpoints and credentials configuration for container registries.|**Type:** `object`<br/>**Default:** `{}`|
| `connectivity.containerRegistries.*` |**None**|**Type:** `array`<br/>|
| `connectivity.containerRegistries.*[*]` |**None**|**Type:** `object`<br/>|
| `connectivity.containerRegistries.*[*].credentials` | **Credentials** - Credentials for the endpoint.|**Type:** `object`<br/>|
| `connectivity.containerRegistries.*[*].credentials.auth` | **Auth** - Base64-encoded string from the concatenation of the username, a colon, and the password.|**Type:** `string`<br/>|
| `connectivity.containerRegistries.*[*].credentials.identitytoken` | **Identity token** - Used to authenticate the user and obtain an access token for the registry.|**Type:** `string`<br/>|
| `connectivity.containerRegistries.*[*].credentials.password` | **Password** - Used to authenticate for the registry with username/password.|**Type:** `string`<br/>|
| `connectivity.containerRegistries.*[*].credentials.username` | **Username** - Used to authenticate for the registry with username/password.|**Type:** `string`<br/>|
| `connectivity.containerRegistries.*[*].endpoint` | **Endpoint** - Endpoint for the container registry.|**Type:** `string`<br/>|
| `connectivity.network` | **Network**|**Type:** `object`<br/>|
| `connectivity.network.controlPlaneEndpoint` | **Control plane endpoint** - Kubernetes API endpoint.|**Type:** `object`<br/>|
| `connectivity.network.controlPlaneEndpoint.host` | **Host**|**Type:** `string`<br/>|
| `connectivity.network.controlPlaneEndpoint.port` | **Port number**|**Type:** `integer`<br/>**Default:** `6443`|
| `connectivity.network.extraOvdcNetworks` | **Extra OVDC networks** - OVDC networks to attach VMs to, additionally.|**Type:** `array`<br/>|
| `connectivity.network.extraOvdcNetworks[*]` |**None**|**Type:** `string`<br/>|
| `connectivity.network.hostEntries` | **Host entries**|**Type:** `array`<br/>|
| `connectivity.network.hostEntries[*]` |**None**|**Type:** `object`<br/>|
| `connectivity.network.hostEntries[*].fqdn` | **FQDN**|**Type:** `string`<br/>|
| `connectivity.network.hostEntries[*].ip` | **IP address**|**Type:** `string`<br/>|
| `connectivity.network.loadBalancers` | **Load Balancers**|**Type:** `object`<br/>|
| `connectivity.network.loadBalancers.vipSubnet` | **Virtual IP subnet** - Virtual IP CIDR for the external network.|**Type:** `string`<br/>|
| `connectivity.network.pods` | **Pods**|**Type:** `object`<br/>|
| `connectivity.network.pods.cidrBlocks` |**None**|**Type:** `array`<br/>|
| `connectivity.network.pods.cidrBlocks[*]` |IPv4 address range, in CIDR notation.|**Type:** `string`<br/>**Example:** `"10.244.0.0/16"`<br/>|
| `connectivity.network.services` | **Services**|**Type:** `object`<br/>|
| `connectivity.network.services.cidrBlocks` |**None**|**Type:** `array`<br/>|
| `connectivity.network.services.cidrBlocks[*]` |IPv4 address range, in CIDR notation.|**Type:** `string`<br/>**Example:** `"10.244.0.0/16"`<br/>|
| `connectivity.network.staticRoutes` | **Static routes**|**Type:** `array`<br/>|
| `connectivity.network.staticRoutes[*]` |**None**|**Type:** `object`<br/>|
| `connectivity.network.staticRoutes[*].destination` | **Destination** - IPv4 address range in CIDR notation.|**Type:** `string`<br/>**Example:** `"10.128.0.0/16"`<br/>|
| `connectivity.network.staticRoutes[*].via` | **Via**|**Type:** `string`<br/>|
| `connectivity.ntp` | **Time synchronization (NTP)** - Servers/pools to synchronize this cluster's clocks with.|**Type:** `object`<br/>|
| `connectivity.ntp.pools` | **Pools**|**Type:** `array`<br/>|
| `connectivity.ntp.pools[*]` | **Pool**|**Type:** `string`<br/>**Example:** `"ntp.ubuntu.com"`<br/>|
| `connectivity.ntp.servers` | **Servers**|**Type:** `array`<br/>|
| `connectivity.ntp.servers[*]` | **Server**|**Type:** `string`<br/>|
| `connectivity.proxy` | **Proxy** - Whether/how outgoing traffic is routed through proxy servers.|**Type:** `object`<br/>|
| `connectivity.proxy.enabled` | **Enable**|**Type:** `boolean`<br/>|
| `connectivity.proxy.secretName` | **Secret name** - Name of a secret resource used by containerd to obtain the HTTP_PROXY, HTTPS_PROXY, and NO_PROXY environment variables. If empty the value will be defaulted to <clusterName>-cluster-values.|**Type:** `string`<br/>**Value pattern:** `^[a-z0-9-]{0,63}$`<br/>|
| `connectivity.shell` | **Shell access**|**Type:** `object`<br/>|
| `connectivity.shell.osUsers` | **OS Users** - Configuration for OS users in cluster nodes.|**Type:** `array`<br/>**Default:** `[{"name":"giantswarm","sudo":"ALL=(ALL) NOPASSWD:ALL"}]`|
| `connectivity.shell.osUsers[*]` | **User**|**Type:** `object`<br/>|
| `connectivity.shell.osUsers[*].name` | **Name** - Username of the user.|**Type:** `string`<br/>**Value pattern:** `^[a-z][-a-z0-9]+$`<br/>|
| `connectivity.shell.osUsers[*].sudo` | **Sudoers configuration** - Permissions string to add to /etc/sudoers for this user.|**Type:** `string`<br/>|
| `connectivity.shell.sshTrustedUserCAKeys` | **Trusted SSH cert issuers** - CA certificates of issuers that are trusted to sign SSH user certificates.|**Type:** `array`<br/>**Default:** `["ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIM4cvZ01fLmO9cJbWUj7sfF+NhECgy+Cl0bazSrZX7sU [email protected]"]`|
| `connectivity.shell.sshTrustedUserCAKeys[*]` |**None**|**Type:** `string`<br/>|
| `global.connectivity.baseDomain` | **Base DNS domain**|**Type:** `string`<br/>|
| `global.connectivity.containerRegistries` | **Container registries** - Endpoints and credentials configuration for container registries.|**Type:** `object`<br/>**Default:** `{}`|
| `global.connectivity.containerRegistries.*` |**None**|**Type:** `array`<br/>|
| `global.connectivity.containerRegistries.*[*]` |**None**|**Type:** `object`<br/>|
| `global.connectivity.containerRegistries.*[*].credentials` | **Credentials** - Credentials for the endpoint.|**Type:** `object`<br/>|
| `global.connectivity.containerRegistries.*[*].credentials.auth` | **Auth** - Base64-encoded string from the concatenation of the username, a colon, and the password.|**Type:** `string`<br/>|
| `global.connectivity.containerRegistries.*[*].credentials.identitytoken` | **Identity token** - Used to authenticate the user and obtain an access token for the registry.|**Type:** `string`<br/>|
| `global.connectivity.containerRegistries.*[*].credentials.password` | **Password** - Used to authenticate for the registry with username/password.|**Type:** `string`<br/>|
| `global.connectivity.containerRegistries.*[*].credentials.username` | **Username** - Used to authenticate for the registry with username/password.|**Type:** `string`<br/>|
| `global.connectivity.containerRegistries.*[*].endpoint` | **Endpoint** - Endpoint for the container registry.|**Type:** `string`<br/>|
| `global.connectivity.network` | **Network**|**Type:** `object`<br/>|
| `global.connectivity.network.controlPlaneEndpoint` | **Control plane endpoint** - Kubernetes API endpoint.|**Type:** `object`<br/>|
| `global.connectivity.network.controlPlaneEndpoint.host` | **Host**|**Type:** `string`<br/>|
| `global.connectivity.network.controlPlaneEndpoint.port` | **Port number**|**Type:** `integer`<br/>**Default:** `6443`|
| `global.connectivity.network.extraOvdcNetworks` | **Extra OVDC networks** - OVDC networks to attach VMs to, additionally.|**Type:** `array`<br/>|
| `global.connectivity.network.extraOvdcNetworks[*]` |**None**|**Type:** `string`<br/>|
| `global.connectivity.network.hostEntries` | **Host entries**|**Type:** `array`<br/>|
| `global.connectivity.network.hostEntries[*]` |**None**|**Type:** `object`<br/>|
| `global.connectivity.network.hostEntries[*].fqdn` | **FQDN**|**Type:** `string`<br/>|
| `global.connectivity.network.hostEntries[*].ip` | **IP address**|**Type:** `string`<br/>|
| `global.connectivity.network.loadBalancers` | **Load Balancers**|**Type:** `object`<br/>|
| `global.connectivity.network.loadBalancers.vipSubnet` | **Virtual IP subnet** - Virtual IP CIDR for the external network.|**Type:** `string`<br/>|
| `global.connectivity.network.pods` | **Pods**|**Type:** `object`<br/>|
| `global.connectivity.network.pods.cidrBlocks` |**None**|**Type:** `array`<br/>|
| `global.connectivity.network.pods.cidrBlocks[*]` |IPv4 address range, in CIDR notation.|**Type:** `string`<br/>**Example:** `"10.244.0.0/16"`<br/>|
| `global.connectivity.network.services` | **Services**|**Type:** `object`<br/>|
| `global.connectivity.network.services.cidrBlocks` |**None**|**Type:** `array`<br/>|
| `global.connectivity.network.services.cidrBlocks[*]` |IPv4 address range, in CIDR notation.|**Type:** `string`<br/>**Example:** `"10.244.0.0/16"`<br/>|
| `global.connectivity.network.staticRoutes` | **Static routes**|**Type:** `array`<br/>|
| `global.connectivity.network.staticRoutes[*]` |**None**|**Type:** `object`<br/>|
| `global.connectivity.network.staticRoutes[*].destination` | **Destination** - IPv4 address range in CIDR notation.|**Type:** `string`<br/>**Example:** `"10.128.0.0/16"`<br/>|
| `global.connectivity.network.staticRoutes[*].via` | **Via**|**Type:** `string`<br/>|
| `global.connectivity.ntp` | **Time synchronization (NTP)** - Servers/pools to synchronize this cluster's clocks with.|**Type:** `object`<br/>|
| `global.connectivity.ntp.pools` | **Pools**|**Type:** `array`<br/>|
| `global.connectivity.ntp.pools[*]` | **Pool**|**Type:** `string`<br/>**Example:** `"ntp.ubuntu.com"`<br/>|
| `global.connectivity.ntp.servers` | **Servers**|**Type:** `array`<br/>|
| `global.connectivity.ntp.servers[*]` | **Server**|**Type:** `string`<br/>|
| `global.connectivity.proxy` | **Proxy** - Whether/how outgoing traffic is routed through proxy servers.|**Type:** `object`<br/>|
| `global.connectivity.proxy.enabled` | **Enable**|**Type:** `boolean`<br/>|
| `global.connectivity.proxy.secretName` | **Secret name** - Name of a secret resource used by containerd to obtain the HTTP_PROXY, HTTPS_PROXY, and NO_PROXY environment variables. If empty the value will be defaulted to <clusterName>-cluster-values.|**Type:** `string`<br/>**Value pattern:** `^[a-z0-9-]{0,63}$`<br/>|
| `global.connectivity.shell` | **Shell access**|**Type:** `object`<br/>|
| `global.connectivity.shell.osUsers` | **OS Users** - Configuration for OS users in cluster nodes.|**Type:** `array`<br/>**Default:** `[{"name":"giantswarm","sudo":"ALL=(ALL) NOPASSWD:ALL"}]`|
| `global.connectivity.shell.osUsers[*]` | **User**|**Type:** `object`<br/>|
| `global.connectivity.shell.osUsers[*].name` | **Name** - Username of the user.|**Type:** `string`<br/>**Value pattern:** `^[a-z][-a-z0-9]+$`<br/>|
| `global.connectivity.shell.osUsers[*].sudo` | **Sudoers configuration** - Permissions string to add to /etc/sudoers for this user.|**Type:** `string`<br/>|
| `global.connectivity.shell.sshTrustedUserCAKeys` | **Trusted SSH cert issuers** - CA certificates of issuers that are trusted to sign SSH user certificates.|**Type:** `array`<br/>**Default:** `["ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIM4cvZ01fLmO9cJbWUj7sfF+NhECgy+Cl0bazSrZX7sU [email protected]"]`|
| `global.connectivity.shell.sshTrustedUserCAKeys[*]` |**None**|**Type:** `string`<br/>|

### Control plane
Properties within the `.controlPlane` top-level object
Expand Down Expand Up @@ -218,7 +219,6 @@ Properties within the `.providerSpecific` top-level object

| **Property** | **Description** | **More Details** |
| :----------- | :-------------- | :--------------- |
| `baseDomain` | **Base DNS domain**|**Type:** `string`<br/>**Default:** `"k8s.test"`|
| `cluster-shared` | **Library chart**|**Type:** `object`<br/>|
| `managementCluster` | **Management cluster name** - The Cluster API management cluster that manages this cluster.|**Type:** `string`<br/>|
| `provider` | **Cluster API provider name**|**Type:** `string`<br/>|
Expand Down
11 changes: 6 additions & 5 deletions helm/cluster-cloud-director/ci/ci-values.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
baseDomain: "test.gigantic.io"
controlPlane:
catalog: giantswarm
replicas: 1
Expand All @@ -11,10 +10,6 @@ controlPlane:
groupsClaim: "groups"
usernamePrefix: ""
groupsPrefix: ""
connectivity:
network:
loadBalancers:
vipSubnet: 10.205.9.254/24
nodePools:
worker:
class: default
Expand All @@ -35,3 +30,9 @@ internal:
kubernetesVersion: v1.25.13+vmware.1
teleport:
enabled: true
global:
connectivity:
network:
loadBalancers:
vipSubnet: 10.205.9.254/24
baseDomain: "test.gigantic.io"
11 changes: 6 additions & 5 deletions helm/cluster-cloud-director/ci/test-wc-values.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
baseDomain: "test.gigantic.io"
controlPlane:
catalog: giantswarm
replicas: 1
Expand All @@ -13,10 +12,6 @@ controlPlane:
groupsClaim: "groups"
usernamePrefix: ""
groupsPrefix: ""
connectivity:
network:
loadBalancers:
vipSubnet: 10.205.9.254/24
nodePools:
worker:
class: default
Expand All @@ -41,3 +36,9 @@ metadata:
servicePriority: highest
internal:
kubernetesVersion: v1.25.13+vmware.1
global:
connectivity:
network:
loadBalancers:
vipSubnet: 10.205.9.254/24
baseDomain: "test.gigantic.io"
4 changes: 2 additions & 2 deletions helm/cluster-cloud-director/files/etc/containerd/config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ sandbox_image = "{{ .Values.internal.sandboxContainerImage.registry }}/{{ .Value

[plugins."io.containerd.grpc.v1.cri".registry]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
{{- range $host, $config := .Values.connectivity.containerRegistries }}
{{- range $host, $config := .Values.global.connectivity.containerRegistries }}
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."{{$host}}"]
endpoint = [
{{- range $value := $config -}}
Expand All @@ -34,7 +34,7 @@ sandbox_image = "{{ .Values.internal.sandboxContainerImage.registry }}/{{ .Value
]
{{- end }}
[plugins."io.containerd.grpc.v1.cri".registry.configs]
{{ range $host, $config := .Values.connectivity.containerRegistries -}}
{{ range $host, $config := .Values.global.connectivity.containerRegistries -}}
{{ range $value := $config -}}
{{ with $value.credentials -}}
[plugins."io.containerd.grpc.v1.cri".registry.configs."{{$value.endpoint}}".auth]
Expand Down
2 changes: 1 addition & 1 deletion helm/cluster-cloud-director/files/etc/teleport.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ ssh_service:
ins: {{ .Values.managementCluster }}
mc: {{ .Values.managementCluster }}
cluster: {{ include "resource.default.name" $ }}
baseDomain: {{ .Values.baseDomain }}
baseDomain: {{ .Values.global.connectivity.baseDomain }}
proxy_service:
enabled: "no"
https_keypairs: []
Expand Down
4 changes: 2 additions & 2 deletions helm/cluster-cloud-director/templates/_cluster_dns.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,11 @@
replaced with .10.
*/}}
{{- define "clusterDNS" -}}
{{- $serviceCidrBlock := index .Values.connectivity.network.services.cidrBlocks 0 -}}
{{- $serviceCidrBlock := index .Values.global.connectivity.network.services.cidrBlocks 0 -}}
{{- $mask := int (mustRegexReplaceAll `^.*/(\d+)$` $serviceCidrBlock "${1}") -}}
{{- if gt $mask 24 -}}
{{- fail (printf ".Values.connectivity.network.services.cidrBlocks[0]=%q mask must be <= 24" $serviceCidrBlock) -}}
{{- fail (printf ".Values.global.connectivity.network.services.cidrBlocks[0]=%q mask must be <= 24" $serviceCidrBlock) -}}
{{- end -}}
{{- mustRegexReplaceAll `^(\d+\.\d+\.\d+).*$` $serviceCidrBlock "${1}.10" -}}
Expand Down
18 changes: 9 additions & 9 deletions helm/cluster-cloud-director/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ use the cluster-apps-operator created secret <clusterName>-cluster-values as def
*/}}
{{- define "containerdProxySecret" -}}
{{- $defaultContainerdProxySecret := printf "%s-systemd-proxy" (include "resource.default.name" . ) -}}
{{ .Values.connectivity.proxy.secretName | default $defaultContainerdProxySecret }}
{{ .Values.global.connectivity.proxy.secretName | default $defaultContainerdProxySecret }}
{{- end -}}

{{- define "containerdProxyConfig" -}}
Expand Down Expand Up @@ -110,7 +110,7 @@ use the cluster-apps-operator created secret <clusterName>-cluster-values as def
[Service]
Type=oneshot
RemainAfterExit=yes
{{- range $.Values.connectivity.network.staticRoutes}}
{{- range $.Values.global.connectivity.network.staticRoutes}}
ExecStart=/bin/bash -c "ip route add {{ .destination }} via {{ .via }}"
{{- end -}}
{{- end }}
Expand Down Expand Up @@ -174,29 +174,29 @@ files:
{{- end }}
{{- include "sshFiles" . | nindent 2}}
{{- include "containerdConfig" . | nindent 2 }}
{{- if $.Values.connectivity.proxy.enabled }}
{{- if $.Values.global.connectivity.proxy.enabled }}
{{- include "containerdProxyConfig" . | nindent 2}}
{{- end }}
{{- if and $.Values.internal.teleport.enabled $.Values.connectivity.proxy.enabled }}
{{- if and $.Values.internal.teleport.enabled $.Values.global.connectivity.proxy.enabled }}
{{- include "teleportProxyConfig" . | nindent 2}}
{{- end }}
{{- if $.Values.internal.teleport.enabled }}
{{- include "teleportFiles" . | nindent 2}}
{{- end }}
{{- if $.Values.connectivity.network.staticRoutes }}
{{- if $.Values.global.connectivity.network.staticRoutes }}
{{- if eq $.Values.providerSpecific.vmBootstrapFormat "cloud-config" }}
{{- include "staticRoutes" . | nindent 2}}
{{- end }}
{{- end }}

preKubeadmCommands:
- /bin/test ! -d /var/lib/kubelet && (/bin/mkdir -p /var/lib/kubelet && /bin/chmod 0750 /var/lib/kubelet)
{{- if $.Values.connectivity.proxy.enabled }}
{{- if $.Values.global.connectivity.proxy.enabled }}
- systemctl daemon-reload
- systemctl restart containerd
{{- end }}
{{- include "hostEntries" .}}
{{- if $.Values.connectivity.network.staticRoutes }}
{{- if $.Values.global.connectivity.network.staticRoutes }}
{{- if eq $.Values.providerSpecific.vmBootstrapFormat "cloud-config" }}
- systemctl daemon-reload
- systemctl enable --now static-routes.service
Expand Down Expand Up @@ -236,9 +236,9 @@ placementPolicy: {{ .currentClass.placementPolicy }}
storageProfile: {{ .currentClass.storageProfile }}
diskSize: {{ mul .currentClass.diskSizeGB 1024 1024 1024 }}
vmNamingTemplate: {{ $.providerSpecific.vmNamingTemplate }}
{{- if $.connectivity.network.extraOvdcNetworks }}
{{- if $.global.connectivity.network.extraOvdcNetworks }}
extraOvdcNetworks:
{{- range $.connectivity.network.extraOvdcNetworks }}
{{- range $.global.connectivity.network.extraOvdcNetworks }}
- {{ . }}
{{- end }}
{{- end -}}
Expand Down
4 changes: 2 additions & 2 deletions helm/cluster-cloud-director/templates/_ignition.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -59,9 +59,9 @@ ignition:
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/bin/bash -cv 'echo "$("$(find /usr/bin /usr/share/oem -name vmtoolsd -type f -executable 2>/dev/null | head -n 1)" --cmd "info-get guestinfo.ignition.network")" > /opt/set-networkd-units'
{{- if $.Values.connectivity.network.staticRoutes }}
{{- if $.Values.global.connectivity.network.staticRoutes }}
ExecStart=/usr/bin/bash -cv 'echo "sleep 3" >> /opt/set-networkd-units'
{{- range $.Values.connectivity.network.staticRoutes}}
{{- range $.Values.global.connectivity.network.staticRoutes}}
ExecStart=/usr/bin/bash -cv 'echo "sudo ip route add {{ .destination }} via {{ .via }}" >> /opt/set-networkd-units'
{{- end }}
{{- end }}
Expand Down
Loading

0 comments on commit e28a497

Please sign in to comment.