From 8e75faa941843f52b2fb15cbd5eec3ef5edd810e Mon Sep 17 00:00:00 2001 From: Yu Qi Zhang Date: Thu, 1 Jun 2023 21:05:55 -0400 Subject: [PATCH] Change to add --config-dir flag --- .../3983-drop-in-configuration/README.md | 21 ++++++++++--------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/keps/sig-node/3983-drop-in-configuration/README.md b/keps/sig-node/3983-drop-in-configuration/README.md index accc64fb65e7..f31961557eef 100644 --- a/keps/sig-node/3983-drop-in-configuration/README.md +++ b/keps/sig-node/3983-drop-in-configuration/README.md @@ -63,7 +63,7 @@ Items marked with (R) are required *prior to targeting to a milestone / release* ## Summary -Add support for a drop-in configuration directory for the Kubelet. This directory will be in `/etc/kubernetes/kubelet.conf.d` and configuration files will be processed in alphanumeric order. Establishment of conventions for configuration processing will be done, and further work can be done to add this support for other components. +Add support for a drop-in configuration directory for the Kubelet. This directory can be specified via a "--config-dir" flag, and configuration files will be processed in alphanumeric order. The flag will be empty by default and if not specified, drop-in support will not be enabled. Establishment of conventions for configuration processing will be done, and further work can be done to add this support for other components. ## Motivation @@ -75,28 +75,29 @@ Components in Kubernetes can similarly be configured by multiple entities and pr ### Goals -* Add support for a `/etc/kubernetes/kubelet.conf.d` directory, which will override the configuration for the Kubelet located at `/etc/kubernetes/kubelet.conf` +* Add support for a "--config-dir" flag to the kubelet to allow users to specify a drop-in directory, which will override the configuration for the Kubelet located at `/etc/kubernetes/kubelet.conf` * Extend API machinery code to reconcile multiple configuration files. * Define Kubernetes best-practices for configuration definitions, similarly to [API conventions](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md). ### Non-Goals * Add support for drop-in configuration for Kubernetes components other than the Kubelet. +* Dynamically reconfiguring running kubelets if drop-in contents change. ## Proposal -This proposal aims to add support for a drop-in configuration directory for the kubelet in `/etc/kubernetes/kubelet.conf.d`. Users are able to specify individually configurable kubelet config snippets in files, formatted in the same way as the existing kubelet.conf file. The kubelet will process the configuration provided in `/etc/kubernetes/kubelet.conf.d` in alphanumeric order: +This proposal aims to add support for a drop-in configuration directory for the kubelet via specifying a "--config-dir" flag (for example, `/etc/kubernetes/kubelet.conf.d`). Users are able to specify individually configurable kubelet config snippets in files, formatted in the same way as the existing kubelet.conf file. The kubelet will process the configuration provided in the drop-in directory in alphanumeric order: 1. If no other configuration for the subfield(s) exist, append to the base configuration -2. If the subfield(s) exists in the base configuration at `/etc/kubernetes/kubelet.conf` file or another file in `/etc/kubernetes/kubelet.conf.d` with lesser alphanumeric ordering, overwrite it +2. If the subfield(s) exists in the base configuration at `/etc/kubernetes/kubelet.conf` file or another file in the drop-in directory with lesser alphanumeric ordering, overwrite it * If the subfield(s) exist as a list, overwrite instead of attempting to merge. This makes it easier to delete items from lists defined in the base kubelet.conf or other drop-ins without having to modify other files. If there are any issues with the drop-ins (e.g. formatting errors), the error will be reported in the same way as a misconfigured kubelet.conf file -This drop-in directory is purely optional and if empty/does not exist, the base configuration is used and no behavior changes will be introduced. +This drop-in directory is purely optional and if empty, the base configuration is used and no behavior changes will be introduced. The "--config-dir" flag will be empty by default and if not specified, drop-in support will not be enabled. This aims to align with "--config" flag defaults. ### User Stories @@ -152,7 +153,7 @@ As a cluster admin, I would like to have cgroup management and log size manageme ### Upgrade / Downgrade Strategy -Upgrades and downgrades are safe as far as Kubelet stability is concerned. It’s possible a vendor may ship vital pieces of configuration within a drop-in directory. If the Kubelet downgrades to a version that doesn’t support reading the drop-in directory, it will only be configured with the original `/etc/kubelet/kubelet.conf`. However, assuming that vendor left that file in a valid state, the Kubelet should not be at risk of crashing. Other than vital configuration being missed, there’s no risk to the stability of the Kubelet on upgrades and downgrades. +Upgrades and downgrades are safe as far as Kubelet stability is concerned. It’s possible a vendor may ship vital pieces of configuration within a drop-in directory. If the Kubelet downgrades to a version that doesn’t support reading the drop-in directory, the kubelet will not recognize the "--config-dir" flag and risk failing. However, assuming that vendor left that the original `/etc/kubelet/kubelet.conf` is in a valid state, and the flag isn't specified, there should be no risk to the system. Any configuration that exists in a drop-in dir won't be applied, but that would not affect kubelet stability. ### Version Skew Strategy @@ -168,22 +169,22 @@ All behavior change is encapsulated within the Kubelet, so there is no version s ###### How can this feature be enabled / disabled in a live cluster? -There will be no feature gate. On a live cluster, admins can disable by not using the drop-in configuration directory. +There will be no feature gate. On a live cluster, admins can disable by removing the flag. ###### Does enabling the feature change any default behavior? -Yes, upgrading to a version of the Kubelet with this feature will enable the Kubelet to be configured with the drop-in directory +No, upgrading to a version of the Kubelet with this feature will not enable the Kubelet to be configured with the drop-in directory if no flag is specified. ###### Can the feature be disabled once it has been enabled (i.e. can we roll back the enablement)? -No, only a rollback of the Kubelet can roll back the enablement. +Yes. Roll back and remove the `--config-dir` flag from the kubelet's CLI. ###### What happens if we reenable the feature if it was previously rolled back? -This will be a Kubelet upgrade, which will re-enable the feature as expected. +This feature will be re-enabled via adding back the `--config-dir` flag to the CLI, as mentioned above. ###### Are there any tests for feature enablement/disablement?