diff --git a/OWNERS_ALIASES b/OWNERS_ALIASES index d5deda7cd1033..3548907406561 100644 --- a/OWNERS_ALIASES +++ b/OWNERS_ALIASES @@ -1,11 +1,13 @@ aliases: sig-docs-blog-owners: # Approvers for blog content - mrbobbytables + - natalisucks - nate-double-u - sftim sig-docs-blog-reviewers: # Reviewers for blog content - Gauravpadam - mrbobbytables + - natalisucks - nate-double-u - sftim sig-docs-website-owners: # Admins for overall website diff --git a/content/de/docs/concepts/workloads/pods/_index.md b/content/de/docs/concepts/workloads/pods/_index.md index bbb156efe6da9..a6d9333c895ab 100644 --- a/content/de/docs/concepts/workloads/pods/_index.md +++ b/content/de/docs/concepts/workloads/pods/_index.md @@ -364,6 +364,6 @@ oder {{< glossary_tooltip text="Deployments" term_id="deployment" >}} einbindet, kannst du Artikel zu früheren Technologien lesen, unter anderem: * [Aurora](https://aurora.apache.org/documentation/latest/reference/configuration/#job-schema) * [Borg](https://research.google.com/pubs/pub43438.html) - * [Marathon](https://mesosphere.github.io/marathon/docs/rest-api.html) + * [Marathon](https://github.com/d2iq-archive/marathon) * [Omega](https://research.google/pubs/pub41684/) * [Tupperware](https://engineering.fb.com/data-center-engineering/tupperware/). diff --git a/content/en/blog/_posts/2018-05-04-Announcing-Kubeflow-0-1.md b/content/en/blog/_posts/2018-05-04-Announcing-Kubeflow-0-1.md index 532e2727075a0..9a84a1be87340 100644 --- a/content/en/blog/_posts/2018-05-04-Announcing-Kubeflow-0-1.md +++ b/content/en/blog/_posts/2018-05-04-Announcing-Kubeflow-0-1.md @@ -42,7 +42,7 @@ NAMESPACE=kubeflow kubectl create namespace ${NAMESPACE} VERSION=v0.1.3 -# Initialize a ksonnet app. Set the namespace for it's default environment. +# Initialize a ksonnet app. Set the namespace for its default environment. APP_NAME=my-kubeflow ks init ${APP_NAME} cd ${APP_NAME} diff --git a/content/en/blog/_posts/2021-10-18-kpng-specialized-proxiers.md b/content/en/blog/_posts/2021-10-18-kpng-specialized-proxiers.md index 001123bf760bc..019b4b9f50c08 100644 --- a/content/en/blog/_posts/2021-10-18-kpng-specialized-proxiers.md +++ b/content/en/blog/_posts/2021-10-18-kpng-specialized-proxiers.md @@ -41,7 +41,7 @@ spec: If the `service.kubernetes.io/service-proxy-name` label is defined the `kube-proxy` will ignore the service. A custom controller can watch -services with the label set to it's own name, "kpng-example" in +services with the label set to its own name, "kpng-example" in this example, and setup specialized load-balancing. The `service.kubernetes.io/service-proxy-name` label is [not diff --git a/content/en/blog/_posts/2024-12-11-Kubernetes-v1-32-Release/index.md b/content/en/blog/_posts/2024-12-11-Kubernetes-v1-32-Release/index.md index 8bac1a8f79d5d..01fcbf5498047 100644 --- a/content/en/blog/_posts/2024-12-11-Kubernetes-v1-32-Release/index.md +++ b/content/en/blog/_posts/2024-12-11-Kubernetes-v1-32-Release/index.md @@ -1,16 +1,15 @@ --- layout: blog -title: 'Kubernetes v1.32: {release-name}' +title: 'Kubernetes v1.32: Penelope' date: 2024-12-11 slug: kubernetes-v1-32-release author: > [Kubernetes v1.32 Release Team](https://github.com/kubernetes/sig-release/blob/master/releases/release-1.32/release-team.md) -draft: true --- **Editors:** Matteo Bianchi, Edith Puclla, William Rizzo, Ryota Sawada, Rashan Smith -Announcing the release of Kubernetes v1.32: {release-name}! +Announcing the release of Kubernetes v1.32: Penelope! In line with previous releases, the release of Kubernetes v1.32 introduces new stable, beta, and alpha features. The consistent delivery of high-quality releases underscores the strength of our development cycle and the vibrant @@ -19,14 +18,22 @@ This release consists of 44 enhancements in total. Of those enhancements, 13 have graduated to Stable, 12 are entering Beta, and 19 have entered in Alpha. ## Release theme and logo -{{< figure src="/images/blog/2024-12-11-kubernetes-1.32-release/k8s-1.32.png" alt="Kubernetes v1.32 logo" -class="release-logo" >}} - +{{< figure src="k8s-1.32.png" alt="Kubernetes v1.32 logo: Penelope from the Odyssey, a helm and a purple geometric background" +class="release-logo" >}} -The Kubernetes v1.32 Release Theme is "{release-name}". +The Kubernetes v1.32 Release Theme is "Penelope". -Kubernetes v1.32's {release-story} +If Kubernetes is Ancient Greek for "pilot", in this release we start from that origin +and reflect on the last 10 years of Kubernetes and our accomplishments: +each release cycle is a journey, and just like Penelope, in "The Odyssey", +weaved for 10 years -- each night removing parts of what she had done during the day -- +so does each release add new features and removes others, albeit here with a much +clearer purpose of constantly improving Kubernetes. +With v1.32 being the last release in the year Kubernetes marks its first decade anniversary, +we wanted to honour all of those that have been part of the global Kubernetes crew +that roams the cloud-native seas through perils and challanges: +may we continue to weave the future of Kubernetes together. ## Updates to recent key features @@ -332,13 +339,6 @@ This removal will allow Kubernetes to handle new hardware requirements and resou the complexities of back and forth API calls to the kube-apiserver. See the enhancement issue [#3063](https://github.com/kubernetes/enhancements/issues/3063) to find out more. - -#### Deprecation of gitRepo volume types - -The [gitRepo](https://kubernetes.io/docs/concepts/storage/volumes/#gitrepo) volume type is deprecated and will be -removed in a future release, the deprecation has been executed in light of the security advisory encompassing the -[CVE-2024-10220](https://nvd.nist.gov/vuln/detail/CVE-2024-10220): Arbitrary command execution through gitRepo volume, -which was reported publicly in [this issue](https://github.com/kubernetes/kubernetes/issues/128885). #### API removals @@ -478,7 +478,7 @@ Antigua Guatemala, Guatemala ## Upcoming release webinar -Join members of the Kubernetes v1.32 release team on **Thursday, January 9th 2024 at 5:00 PM (UTC)**, to learn about the +Join members of the Kubernetes v1.32 release team on **Thursday, January 9th 2025 at 5:00 PM (UTC)**, to learn about the release highlights of this release, as well as deprecations and removals to help plan for upgrades. For more information and registration, visit the [event page](https://community.cncf.io/events/details/cncf-cncf-online-programs-presents-cncf-live-webinar-kubernetes-132-release/) diff --git a/content/en/blog/_posts/2024-12-11-Kubernetes-v1-32-Release/k8s-1.32.png b/content/en/blog/_posts/2024-12-11-Kubernetes-v1-32-Release/k8s-1.32.png new file mode 100644 index 0000000000000..e6a7d05c249c9 Binary files /dev/null and b/content/en/blog/_posts/2024-12-11-Kubernetes-v1-32-Release/k8s-1.32.png differ diff --git a/content/en/blog/_posts/2024-12-12-scheduler-queueinghint/index.md b/content/en/blog/_posts/2024-12-12-scheduler-queueinghint/index.md index dc7b358b7a2fe..e7a33473dc240 100644 --- a/content/en/blog/_posts/2024-12-12-scheduler-queueinghint/index.md +++ b/content/en/blog/_posts/2024-12-12-scheduler-queueinghint/index.md @@ -3,7 +3,6 @@ layout: blog title: "Kubernetes v1.32: QueueingHint Brings a New Possibility to Optimize Pod Scheduling" date: 2024-12-12 slug: scheduler-queueinghint -draft: true Author: > [Kensei Nakada](https://github.com/sanposhiho) (Tetrate.io) --- @@ -13,7 +12,7 @@ component that selects the nodes on which new Pods run. The scheduler processes these new Pods **one by one**. Therefore, the larger your clusters, the more important the throughput of the scheduler becomes. -Over the years, the Kubernetes project (and SIG Scheduling in particular) has improved the throughput +Over the years, Kubernetes SIG Scheduling has improved the throughput of the scheduler in multiple enhancements. This blog post describes a major improvement to the scheduler in Kubernetes v1.32: a [scheduling context element](/docs/concepts/scheduling-eviction/scheduling-framework/#extension-points) @@ -128,4 +127,4 @@ Please join us and share your feedback. ## How can I learn more? -- [KEP-4247: Per-plugin callback functions for efficient requeueing in the scheduling queue](https://github.com/kubernetes/enhancements/blob/master/keps/sig-scheduling/4247-queueinghint/README.md) \ No newline at end of file +- [KEP-4247: Per-plugin callback functions for efficient requeueing in the scheduling queue](https://github.com/kubernetes/enhancements/blob/master/keps/sig-scheduling/4247-queueinghint/README.md) diff --git a/content/en/blog/_posts/2024-11-11-memory-manager-moves-to-ga/index.md b/content/en/blog/_posts/2024-12-13-memory-manager-moves-to-ga/index.md similarity index 99% rename from content/en/blog/_posts/2024-11-11-memory-manager-moves-to-ga/index.md rename to content/en/blog/_posts/2024-12-13-memory-manager-moves-to-ga/index.md index 6b3bbb13ecee5..3542241029d9c 100644 --- a/content/en/blog/_posts/2024-11-11-memory-manager-moves-to-ga/index.md +++ b/content/en/blog/_posts/2024-12-13-memory-manager-moves-to-ga/index.md @@ -1,11 +1,10 @@ --- layout: blog title: "Kubernetes v1.32: Memory Manager Goes GA" -date: 2024-11-11 +date: 2024-12-13 slug: memory-manager-goes-ga author: > [Talor Itzhak](https://github.com/Tal-or) (Red Hat) -draft: true --- With Kubernetes 1.32, the memory manager has officially graduated to General Availability (GA), diff --git a/content/en/blog/_posts/2024-12-11-cpumanager-strict-cpu-reservation.md b/content/en/blog/_posts/2024-12-16-cpumanager-strict-cpu-reservation.md similarity index 99% rename from content/en/blog/_posts/2024-12-11-cpumanager-strict-cpu-reservation.md rename to content/en/blog/_posts/2024-12-16-cpumanager-strict-cpu-reservation.md index 88a60055593fe..557533e138b14 100644 --- a/content/en/blog/_posts/2024-12-11-cpumanager-strict-cpu-reservation.md +++ b/content/en/blog/_posts/2024-12-16-cpumanager-strict-cpu-reservation.md @@ -1,8 +1,7 @@ --- layout: blog title: 'Kubernetes v1.32 Adds A New CPU Manager Static Policy Option For Strict CPU Reservation' -draft: true -date: 2024-12-11 +date: 2024-12-16 slug: cpumanager-strict-cpu-reservation author: > [Jing Zhang](https://github.com/jingczhang) (Nokia) diff --git a/content/en/blog/_posts/2024-12-11-api-streaming/index.md b/content/en/blog/_posts/2024-12-17-api-streaming/index.md similarity index 99% rename from content/en/blog/_posts/2024-12-11-api-streaming/index.md rename to content/en/blog/_posts/2024-12-17-api-streaming/index.md index ad36102dfacf7..87224cc3a0601 100644 --- a/content/en/blog/_posts/2024-12-11-api-streaming/index.md +++ b/content/en/blog/_posts/2024-12-17-api-streaming/index.md @@ -1,8 +1,7 @@ --- layout: blog title: 'Enhancing Kubernetes API Server Efficiency with API Streaming' -date: 2024-12-11 -draft: true +date: 2024-12-17 slug: kube-apiserver-api-streaming author: > Stefan Schimanski (Upbound), diff --git a/content/en/blog/_posts/2024-12-11-api-streaming/kube-apiserver-memory_usage.png b/content/en/blog/_posts/2024-12-17-api-streaming/kube-apiserver-memory_usage.png similarity index 100% rename from content/en/blog/_posts/2024-12-11-api-streaming/kube-apiserver-memory_usage.png rename to content/en/blog/_posts/2024-12-17-api-streaming/kube-apiserver-memory_usage.png diff --git a/content/en/blog/_posts/2024-12-11-volume-group-snapshot-beta.md b/content/en/blog/_posts/2024-12-18-volume-group-snapshot-beta.md similarity index 99% rename from content/en/blog/_posts/2024-12-11-volume-group-snapshot-beta.md rename to content/en/blog/_posts/2024-12-18-volume-group-snapshot-beta.md index d72640eb0c0c8..a60046ba088f9 100644 --- a/content/en/blog/_posts/2024-12-11-volume-group-snapshot-beta.md +++ b/content/en/blog/_posts/2024-12-18-volume-group-snapshot-beta.md @@ -1,9 +1,8 @@ --- layout: blog title: "Kubernetes 1.32: Moving Volume Group Snapshots to Beta" -date: 2024-12-11 +date: 2024-12-18 slug: kubernetes-1-32-volume-group-snapshot-beta -draft: true author: > Xing Yang (VMware by Broadcom) --- diff --git a/content/en/docs/concepts/cluster-administration/compatibility-version.md b/content/en/docs/concepts/cluster-administration/compatibility-version.md new file mode 100644 index 0000000000000..951eaa61f2f6d --- /dev/null +++ b/content/en/docs/concepts/cluster-administration/compatibility-version.md @@ -0,0 +1,26 @@ +--- +title: Compatibility Version For Kubernetes Control Plane Components +reviewers: +- jpbetz +- siyuanfoundation +content_type: concept +weight: 70 +--- + + + +Since release v1.32, we introduced configurable version compatibility and emulation options to Kubernetes control plane components to make upgrades safer by providing more control and increasing the granularity of steps available to cluster administrators. + + + +## Emulated Version + +The emulation option is set by the `--emulated-version` flag of control plane components. It allows the component to emulate the behavior (APIs, features, ...) of an earlier version of Kubernetes. + +When used, the capabilities available will match the emulated version: +* Any capabilities present in the binary version that were introduced after the emulation version will be unavailable. +* Any capabilities removed after the emulation version will be available. + +This enables a binary from a particular Kubernetes release to emulate the behavior of a previous version with sufficient fidelity that interoperability with other system components can be defined in terms of the emulated version. + +The `--emulated-version` must be <= `binaryVersion`. See the help message of the `--emulated-version` flag for supported range of emulated versions. \ No newline at end of file diff --git a/content/en/docs/concepts/cluster-administration/logging.md b/content/en/docs/concepts/cluster-administration/logging.md index 5cc9429578bf0..e7a3724bb1cdc 100644 --- a/content/en/docs/concepts/cluster-administration/logging.md +++ b/content/en/docs/concepts/cluster-administration/logging.md @@ -75,6 +75,37 @@ appending a container name to the command, with a `-c` flag, like so: kubectl logs counter -c count ``` + +### Container log streams + +{{< feature-state feature_gate_name="PodLogsQuerySplitStreams" >}} + +As an alpha feature, the kubelet can split out the logs from the two standard streams produced +by a container: [standard output](https://en.wikipedia.org/wiki/Standard_streams#Standard_output_(stdout)) +and [standard error](https://en.wikipedia.org/wiki/Standard_streams#Standard_error_(stderr)). +To use this behavior, you must enable the `PodLogsQuerySplitStreams` +[feature gate](/docs/reference/command-line-tools-reference/feature-gates/). +With that feature gate enabled, Kubernetes {{< skew currentVersion >}} allows access to these +log streams directly via the Pod API. You can fetch a specific stream by specifying the stream name (either `Stdout` or `Stderr`), +using the `stream` query string. You must have access to read the `log` subresource of that Pod. + +To demonstrate this feature, you can create a Pod that periodically writes text to both the standard output and error stream. + +{{% code_sample file="debug/counter-pod-err.yaml" %}} + +To run this pod, use the following command: + +```shell +kubectl apply -f https://k8s.io/examples/debug/counter-pod-err.yaml +``` + +To fetch only the stderr log stream, you can run: + +```shell +kubectl get --raw "/api/v1/namespaces/default/pods/counter-err/log?stream=Stderr" +``` + + See the [`kubectl logs` documentation](/docs/reference/generated/kubectl/kubectl-commands#logs) for more details. diff --git a/content/en/docs/concepts/cluster-administration/node-shutdown.md b/content/en/docs/concepts/cluster-administration/node-shutdown.md index 275311b9ee3e7..f3a3711238855 100644 --- a/content/en/docs/concepts/cluster-administration/node-shutdown.md +++ b/content/en/docs/concepts/cluster-administration/node-shutdown.md @@ -217,9 +217,7 @@ these pods will be stuck in terminating status on the shutdown node forever. To mitigate the above situation, a user can manually add the taint `node.kubernetes.io/out-of-service` with either `NoExecute` or `NoSchedule` effect to a Node marking it out-of-service. -If the `NodeOutOfServiceVolumeDetach`[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) -is enabled on {{< glossary_tooltip text="kube-controller-manager" term_id="kube-controller-manager" >}}, -and a Node is marked out-of-service with this taint, the pods on the node will be forcefully deleted +If a Node is marked out-of-service with this taint, the pods on the node will be forcefully deleted if there are no matching tolerations on it and volume detach operations for the pods terminating on the node will happen immediately. This allows the Pods on the out-of-service node to recover quickly on a different node. @@ -267,6 +265,28 @@ via the [Non-Graceful Node Shutdown](#non-graceful-node-shutdown) procedure ment {{< /note >}} +## Windows Graceful node shutdown {#windows-graceful-node-shutdown} + +{{< feature-state feature_gate_name="WindowsGracefulNodeShutdown" >}} + +The Windows graceful node shutdown feature depends on kubelet running as a Windows service, +it will then have a registered [service control handler](https://learn.microsoft.com/en-us/windows/win32/services/service-control-handler-function) +to delay the presshutdown event with a given duration. + +Windows graceful node shutdown is controlled with the `WindowsGracefulNodeShutdown` +[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) +which is introduced in 1.32 as an alpha feature. + +Windows graceful node shutdown can not be cancelled. + +If Kubelet is not running as a Windows service, it will not be able to set and monitor +the [Preshutdown](https://learn.microsoft.com/en-us/windows/win32/api/winsvc/ns-winsvc-service_preshutdown_info) event, +the node will have to go through the [Non-Graceful Node Shutdown](#non-graceful-node-shutdown) procedure mentioned above. + +In the case where the Windows graceful node shutdown feature is enabled, but the kubelet is not +running as a Windows service, the kubelet will continue running instead of failing. However, +it will log an error indicating that it needs to be run as a Windows service. + ## {{% heading "whatsnext" %}} Learn more about the following: diff --git a/content/en/docs/concepts/configuration/manage-resources-containers.md b/content/en/docs/concepts/configuration/manage-resources-containers.md index 0432fc3968a92..9b358b2e241bc 100644 --- a/content/en/docs/concepts/configuration/manage-resources-containers.md +++ b/content/en/docs/concepts/configuration/manage-resources-containers.md @@ -109,6 +109,26 @@ a Pod. For a particular resource, a *Pod resource request/limit* is the sum of the resource requests/limits of that type for each container in the Pod. +## Pod-level resource specification + +{{< feature-state feature_gate_name="PodLevelResources" >}} + +Starting in Kubernetes 1.32, you can also specify resource requests and limits at +the Pod level. the Pod level. At Pod level, Kubernetes {{< skew currentVersion >}} +only supports resource requests or limits for specific resource types: `cpu` and / +or `memory`. This feature is currently in alpha and with the feature enabled, +Kubernetes allows you to declare an overall resource budget for the Pod, which is +especially helpful when dealing with a large number of containers where it can be +difficult to accurately gauge individual resource needs. Additionally, it enables +containers within a Pod to share idle resources with each other, improving resource +utilization. + +For a Pod, you can specify resource limits and requests for CPU and memory by including the following: +* `spec.resources.limits.cpu` +* `spec.resources.limits.memory` +* `spec.resources.requests.cpu` +* `spec.resources.requests.memory` + ## Resource units in Kubernetes ### CPU resource units {#meaning-of-cpu} @@ -192,6 +212,19 @@ spec: cpu: "500m" ``` +## Pod resources example {#example-2} + +{{< feature-state feature_gate_name="PodLevelResources" >}} + +The following Pod has an explicit request of 1 CPU and 100 MiB of memory, and an +explicit limit of 1 CPU and 200 MiB of memory. The `pod-resources-demo-ctr-1` +container has explicit requests and limits set. However, the +`pod-resources-demo-ctr-2` container will simply share the resources available +within the Pod resource boundaries, as it does not have explicit requests and limits +set. + +{{% code_sample file="pods/resource/pod-level-resources.yaml" %}} + ## How Pods with resource requests are scheduled When you create a Pod, the Kubernetes scheduler selects a node for the Pod to diff --git a/content/en/docs/concepts/configuration/secret.md b/content/en/docs/concepts/configuration/secret.md index 79598b84d82ca..1ecb0890aee7c 100644 --- a/content/en/docs/concepts/configuration/secret.md +++ b/content/en/docs/concepts/configuration/secret.md @@ -666,10 +666,7 @@ Therefore, one Pod does not have access to the Secrets of another Pod. ### Configure least-privilege access to Secrets -To enhance the security measures around Secrets, Kubernetes provides a mechanism: you can -annotate a ServiceAccount as `kubernetes.io/enforce-mountable-secrets: "true"`. - -For more information, you can refer to the [documentation about this annotation](/docs/concepts/security/service-accounts/#enforce-mountable-secrets). +To enhance the security measures around Secrets, use separate namespaces to isolate access to mounted secrets. {{< warning >}} Any containers that run with `privileged: true` on a node can access all diff --git a/content/en/docs/concepts/containers/container-lifecycle-hooks.md b/content/en/docs/concepts/containers/container-lifecycle-hooks.md index 9b6b37263cd6a..37b3171af643e 100644 --- a/content/en/docs/concepts/containers/container-lifecycle-hooks.md +++ b/content/en/docs/concepts/containers/container-lifecycle-hooks.md @@ -58,6 +58,10 @@ Resources consumed by the command are counted against the Container. * Sleep - Pauses the container for a specified duration. This is a beta-level feature default enabled by the `PodLifecycleSleepAction` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/). +{{< note >}} +Enable the `PodLifecycleSleepActionAllowZero` feature gate if you want to set a sleep duration of zero seconds (effectively a no-op) for your Sleep lifecycle hooks. +{{< /note >}} + ### Hook handler execution When a Container lifecycle management hook is called, diff --git a/content/en/docs/concepts/containers/images.md b/content/en/docs/concepts/containers/images.md index fb7113a3ce076..6ea5d0d62202f 100644 --- a/content/en/docs/concepts/containers/images.md +++ b/content/en/docs/concepts/containers/images.md @@ -214,7 +214,7 @@ behalf of the two different Pods, when parallel image pulls is enabled. ### Maximum parallel image pulls -{{< feature-state for_k8s_version="v1.27" state="alpha" >}} +{{< feature-state for_k8s_version="v1.32" state="beta" >}} When `serializeImagePulls` is set to false, the kubelet defaults to no limit on the maximum number of images being pulled at the same time. If you would like to diff --git a/content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md b/content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md index 9411e3c63e304..f8ecb35b4319c 100644 --- a/content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md +++ b/content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md @@ -316,9 +316,8 @@ may also be used with field selectors when included in the `spec.versions[*].sel {{< feature-state feature_gate_name="CustomResourceFieldSelectors" >}} The `spec.versions[*].selectableFields` field of a {{< glossary_tooltip term_id="CustomResourceDefinition" text="CustomResourceDefinition" >}} may be used to -declare which other fields in a custom resource may be used in field selectors -with the feature of `CustomResourceFieldSelectors` -[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) (This feature gate is enabled by default since Kubernetes v1.31). +declare which other fields in a custom resource may be used in field selectors. + The following example adds the `.spec.color` and `.spec.size` fields as selectable fields. diff --git a/content/en/docs/concepts/overview/working-with-objects/field-selectors.md b/content/en/docs/concepts/overview/working-with-objects/field-selectors.md index 25ecf6987afb9..b59ad835d303d 100644 --- a/content/en/docs/concepts/overview/working-with-objects/field-selectors.md +++ b/content/en/docs/concepts/overview/working-with-objects/field-selectors.md @@ -46,6 +46,14 @@ Error from server (BadRequest): Unable to find "ingresses" that match label sele | Node | `spec.unschedulable` | | CertificateSigningRequest | `spec.signerName` | +### Custom resources fields + +All custom resource types support the `metadata.name` and `metadata.namespace` fields. + +Additionally, the `spec.versions[*].selectableFields` field of a {{< glossary_tooltip term_id="CustomResourceDefinition" text="CustomResourceDefinition" >}} +declares which other fields in a custom resource may be used in field selectors. See [selectable fields for custom resources](/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#crd-selectable-fields) +for more information about how to use field selectors with CustomResourceDefinitions. + ## Supported operators You can use the `=`, `==`, and `!=` operators with field selectors (`=` and `==` mean the same thing). This `kubectl` command, for example, selects all Kubernetes Services that aren't in the `default` namespace: @@ -72,4 +80,4 @@ You can use field selectors across multiple resource types. This `kubectl` comma ```shell kubectl get statefulsets,services --all-namespaces --field-selector metadata.namespace!=default -``` \ No newline at end of file +``` diff --git a/content/en/docs/concepts/overview/working-with-objects/names.md b/content/en/docs/concepts/overview/working-with-objects/names.md index f8bcf56335667..2634f18d42741 100644 --- a/content/en/docs/concepts/overview/working-with-objects/names.md +++ b/content/en/docs/concepts/overview/working-with-objects/names.md @@ -32,6 +32,12 @@ of the same resource. API resources are distinguished by their API group, resour In cases when objects represent a physical entity, like a Node representing a physical host, when the host is re-created under the same name without deleting and re-creating the Node, Kubernetes treats the new host as the old one, which may lead to inconsistencies. {{< /note >}} +The server may generate a name when `generateName` is provided instead of `name` in a resource create request. +When `generateName` is used, the provided value is used as a name prefix, which server appends a generated suffix +to. Even though the name is generated, it may conflict with existing names resulting in a HTTP 409 resopnse. This +became far less likely to happen in Kubernetes v1.31 and later, since the server will make up to 8 attempt to generate a +unique name before returning a HTTP 409 response. + Below are four types of commonly used name constraints for resources. ### DNS Subdomain Names diff --git a/content/en/docs/concepts/policy/node-resource-managers.md b/content/en/docs/concepts/policy/node-resource-managers.md index 719e8b1151f0e..3c26ef3cecc55 100644 --- a/content/en/docs/concepts/policy/node-resource-managers.md +++ b/content/en/docs/concepts/policy/node-resource-managers.md @@ -13,10 +13,304 @@ In order to support latency-critical and high-throughput workloads, Kubernetes o -The main manager, the Topology Manager, is a Kubelet component that co-ordinates the overall resource management process through its [policy](/docs/tasks/administer-cluster/topology-manager/). +## Hardware topology alignment policies + +_Topology Manager_ is a kubelet component that aims to coordinate the set of components that are +responsible for these optimizations. The the overall resource management process is governed using +the policy you specify. +To learn more, read [Control Topology Management Policies on a Node](/docs/tasks/administer-cluster/topology-manager/). + +## Policies for assigning CPUs to Pods + +{{< feature-state feature_gate_name="CPUManager" >}} + +Once a Pod is bound to a Node, the kubelet on that node may need to either multiplex the existing +hardware (for example, sharing CPUs across multiple Pods) or allocate hardware by dedicating some +resource (for example, assigning one of more CPUs for a Pod's exclusive use). + +By default, the kubelet uses [CFS quota](https://en.wikipedia.org/wiki/Completely_Fair_Scheduler) +to enforce pod CPU limits.  When the node runs many CPU-bound pods, the workload can move to different CPU cores depending on +whether the pod is throttled and which CPU cores are available at scheduling time. Many workloads are not sensitive to this migration and thus +work fine without any intervention. + +However, in workloads where CPU cache affinity and scheduling latency significantly affect workload performance, the kubelet allows alternative CPU +management policies to determine some placement preferences on the node. +This is implemented using the _CPU Manager_ and its policy. +There are two available policies: + +- `none`: the `none` policy explicitly enables the existing default CPU +affinity scheme, providing no affinity beyond what the OS scheduler does +automatically.  Limits on CPU usage for +[Guaranteed pods](/docs/concepts/workloads/pods/pod-qos/) and +[Burstable pods](/docs/concepts/workloads/pods/pod-qos/) +are enforced using CFS quota. +- `static`: the `static` policy allows containers in `Guaranteed` pods with integer CPU +`requests` access to exclusive CPUs on the node. This exclusivity is enforced +using the [cpuset cgroup controller](https://www.kernel.org/doc/Documentation/cgroup-v2.txt). + +{{< note >}} +System services such as the container runtime and the kubelet itself can continue to run on these exclusive CPUs.  The exclusivity only extends to other pods. +{{< /note >}} + +CPU Manager doesn't support offlining and onlining of CPUs at runtime. + +### Static policy + +The static policy enables finer-grained CPU management and exclusive CPU assignment. +This policy manages a shared pool of CPUs that initially contains all CPUs in the +node. The amount of exclusively allocatable CPUs is equal to the total +number of CPUs in the node minus any CPU reservations set by the kubelet configuration. +CPUs reserved by these options are taken, in integer quantity, from the initial shared pool in ascending order by physical +core ID.  This shared pool is the set of CPUs on which any containers in +`BestEffort` and `Burstable` pods run. Containers in `Guaranteed` pods with fractional +CPU `requests` also run on CPUs in the shared pool. Only containers that are +both part of a `Guaranteed` pod and have integer CPU `requests` are assigned +exclusive CPUs. + +{{< note >}} +The kubelet requires a CPU reservation greater than zero when the static policy is enabled. +This is because zero CPU reservation would allow the shared pool to become empty. +{{< /note >}} + +As `Guaranteed` pods whose containers fit the requirements for being statically +assigned are scheduled to the node, CPUs are removed from the shared pool and +placed in the cpuset for the container. CFS quota is not used to bound +the CPU usage of these containers as their usage is bound by the scheduling domain +itself. In others words, the number of CPUs in the container cpuset is equal to the integer +CPU `limit` specified in the pod spec. This static assignment increases CPU +affinity and decreases context switches due to throttling for the CPU-bound +workload. + +Consider the containers in the following pod specs: + +```yaml +spec: + containers: + - name: nginx + image: nginx +``` + +The pod above runs in the `BestEffort` QoS class because no resource `requests` or +`limits` are specified. It runs in the shared pool. + +```yaml +spec: + containers: + - name: nginx + image: nginx + resources: + limits: + memory: "200Mi" + requests: + memory: "100Mi" +``` + +The pod above runs in the `Burstable` QoS class because resource `requests` do not +equal `limits` and the `cpu` quantity is not specified. It runs in the shared +pool. + +```yaml +spec: + containers: + - name: nginx + image: nginx + resources: + limits: + memory: "200Mi" + cpu: "2" + requests: + memory: "100Mi" + cpu: "1" +``` + +The pod above runs in the `Burstable` QoS class because resource `requests` do not +equal `limits`. It runs in the shared pool. + +```yaml +spec: + containers: + - name: nginx + image: nginx + resources: + limits: + memory: "200Mi" + cpu: "2" + requests: + memory: "200Mi" + cpu: "2" +``` + +The pod above runs in the `Guaranteed` QoS class because `requests` are equal to `limits`. +And the container's resource limit for the CPU resource is an integer greater than +or equal to one. The `nginx` container is granted 2 exclusive CPUs. + + +```yaml +spec: + containers: + - name: nginx + image: nginx + resources: + limits: + memory: "200Mi" + cpu: "1.5" + requests: + memory: "200Mi" + cpu: "1.5" +``` + +The pod above runs in the `Guaranteed` QoS class because `requests` are equal to `limits`. +But the container's resource limit for the CPU resource is a fraction. It runs in +the shared pool. + + +```yaml +spec: + containers: + - name: nginx + image: nginx + resources: + limits: + memory: "200Mi" + cpu: "2" +``` + +The pod above runs in the `Guaranteed` QoS class because only `limits` are specified +and `requests` are set equal to `limits` when not explicitly specified. And the +container's resource limit for the CPU resource is an integer greater than or +equal to one. The `nginx` container is granted 2 exclusive CPUs. + +#### Static policy options {#cpu-policy-static--options} + +The behavior of the static policy can be fine-tuned using the CPU Manager policy options. +The following policy options exist for the static CPU management policy: +{{/* options in alphabetical order */}} + +`align-by-socket` (alpha, hidden by default) +: Align CPUs by physical package / socket boundary, rather than logical NUMA boundaries (available since Kubernetes v1.25) +`distribute-cpus-across-cores` (alpha, hidden by default) +: Allocate virtual cores, sometimes called hardware threads, across different physical cores (available since Kubernetes v1.31) +`distribute-cpus-across-numa` (alpha, hidden by default) +: Spread CPUs across different NUMA domains, aiming for an even balance between the selected domains (available since Kubernetes v1.23) +`full-pcpus-only` (beta, visible by default) +: Always allocate full physical cores (available since Kubernetes v1.22) +`strict-cpu-reservation` (alpha, hidden by default) +: Prevent all the pods regardless of their Quality of Service class to run on reserved CPUs (available since Kubernetes v1.32) +`prefer-align-cpus-by-uncorecache` (alpha, hidden by default) +: Align CPUs by uncore (Last-Level) cache boundary on a best-effort way (available since Kubernetes v1.32) + +You can toggle groups of options on and off based upon their maturity level +using the following feature gates: +* `CPUManagerPolicyBetaOptions` (default enabled). Disable to hide beta-level options. +* `CPUManagerPolicyAlphaOptions` (default disabled). Enable to show alpha-level options. +You will still have to enable each option using the `cpuManagerPolicyOptions` field in the +kubelet configuration file. + +For more detail about the individual options you can configure, read on. + +##### `full-pcpus-only` + +If the `full-pcpus-only` policy option is specified, the static policy will always allocate full physical cores. +By default, without this option, the static policy allocates CPUs using a topology-aware best-fit allocation. +On SMT enabled systems, the policy can allocate individual virtual cores, which correspond to hardware threads. +This can lead to different containers sharing the same physical cores; this behaviour in turn contributes +to the [noisy neighbours problem](https://en.wikipedia.org/wiki/Cloud_computing_issues#Performance_interference_and_noisy_neighbors). +With the option enabled, the pod will be admitted by the kubelet only if the CPU request of all its containers +can be fulfilled by allocating full physical cores. +If the pod does not pass the admission, it will be put in Failed state with the message `SMTAlignmentError`. + +##### `distribute-cpus-across-numa` + +If the `distribute-cpus-across-numa`policy option is specified, the static +policy will evenly distribute CPUs across NUMA nodes in cases where more than +one NUMA node is required to satisfy the allocation. +By default, the `CPUManager` will pack CPUs onto one NUMA node until it is +filled, with any remaining CPUs simply spilling over to the next NUMA node. +This can cause undesired bottlenecks in parallel code relying on barriers (and +similar synchronization primitives), as this type of code tends to run only as +fast as its slowest worker (which is slowed down by the fact that fewer CPUs +are available on at least one NUMA node). +By distributing CPUs evenly across NUMA nodes, application developers can more +easily ensure that no single worker suffers from NUMA effects more than any +other, improving the overall performance of these types of applications. + +##### `align-by-socket` + +If the `align-by-socket` policy option is specified, CPUs will be considered +aligned at the socket boundary when deciding how to allocate CPUs to a +container. By default, the `CPUManager` aligns CPU allocations at the NUMA +boundary, which could result in performance degradation if CPUs need to be +pulled from more than one NUMA node to satisfy the allocation. Although it +tries to ensure that all CPUs are allocated from the _minimum_ number of NUMA +nodes, there is no guarantee that those NUMA nodes will be on the same socket. +By directing the `CPUManager` to explicitly align CPUs at the socket boundary +rather than the NUMA boundary, we are able to avoid such issues. Note, this +policy option is not compatible with `TopologyManager` `single-numa-node` +policy and does not apply to hardware where the number of sockets is greater +than number of NUMA nodes. + +##### `distribute-cpus-across-cores` + +If the `distribute-cpus-across-cores` policy option is specified, the static policy +will attempt to allocate virtual cores (hardware threads) across different physical cores. +By default, the `CPUManager` tends to pack cpus onto as few physical cores as possible, +which can lead to contention among cpus on the same physical core and result +in performance bottlenecks. By enabling the `distribute-cpus-across-cores` policy, +the static policy ensures that cpus are distributed across as many physical cores +as possible, reducing the contention on the same physical core and thereby +improving overall performance. However, it is important to note that this strategy +might be less effective when the system is heavily loaded. Under such conditions, +the benefit of reducing contention diminishes. Conversely, default behavior +can help in reducing inter-core communication overhead, potentially providing +better performance under high load conditions. + +##### `strict-cpu-reservation` + +The `reservedSystemCPUs` parameter in [KubeletConfiguration](/docs/reference/config-api/kubelet-config.v1beta1/), +or the deprecated kubelet command line option `--reserved-cpus`, defines an explicit CPU set for OS system daemons +and kubernetes system daemons. More details of this parameter can be found on the +[Explicitly Reserved CPU List](/docs/tasks/administer-cluster/reserve-compute-resources/#explicitly-reserved-cpu-list) page. +By default this isolation is implemented only for guaranteed pods with integer CPU requests not for burstable and best-effort pods +(and guaranteed pods with fractional CPU requests). Admission is only comparing the cpu requests against the allocatable cpus. +Since the cpu limit is higher than the request, the default behaviour allows burstable and best-effort pods to use up the capacity +of `reservedSystemCPUs` and cause host OS services to starve in real life deployments. +If the `strict-cpu-reservation` policy option is enabled, the static policy will not allow +any workload to use the CPU cores specified in `reservedSystemCPUs`. + +##### `prefer-align-cpus-by-uncorecache` + +If the `prefer-align-cpus-by-uncorecache` policy is specified, the static policy +will allocate CPU resources for individual containers such that all CPUs assigned +to a container share the same uncore cache block (also known as the Last-Level Cache +or LLC). By default, the `CPUManager` will tightly pack CPU assignments which can +result in containers being assigned CPUs from multiple uncore caches. This option +enables the `CPUManager` to allocate CPUs in a way that maximizes the efficient use +of the uncore cache. Allocation is performed on a best-effort basis, aiming to +affine as many CPUs as possible within the same uncore cache. If the container's +CPU requirement exceeds the CPU capacity of a single uncore cache, the `CPUManager` +minimizes the number of uncore caches used in order to maintain optimal uncore +cache alignment. Specific workloads can benefit in performance from the reduction +of inter-cache latency and noisy neighbors at the cache level. If the `CPUManager` +cannot align optimally while the node has sufficient resources, the container will +still be admitted using the default packed behavior. + + +## Memory Management Policies + +{{< feature-state feature_gate_name="MemoryManager" >}} + +The Kubernetes *Memory Manager* enables the feature of guaranteed memory (and hugepages) +allocation for pods in the `Guaranteed` {{< glossary_tooltip text="QoS class" term_id="qos-class" >}}. + +The Memory Manager employs hint generation protocol to yield the most suitable NUMA affinity for a pod. +The Memory Manager feeds the central manager (*Topology Manager*) with these affinity hints. +Based on both the hints and Topology Manager policy, the pod is rejected or admitted to the node. + +Moreover, the Memory Manager ensures that the memory which a pod requests +is allocated from a minimum number of NUMA nodes. + +## Other resource managers The configuration of individual managers is elaborated in dedicated documents: -- [CPU Manager Policies](/docs/tasks/administer-cluster/cpu-management-policies/) - [Device Manager](/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/#device-plugin-integration-with-the-topology-manager) -- [Memory Manager Policies](/docs/tasks/administer-cluster/memory-manager/) diff --git a/content/en/docs/concepts/scheduling-eviction/dynamic-resource-allocation.md b/content/en/docs/concepts/scheduling-eviction/dynamic-resource-allocation.md index a598eee3475c8..94d308475738f 100644 --- a/content/en/docs/concepts/scheduling-eviction/dynamic-resource-allocation.md +++ b/content/en/docs/concepts/scheduling-eviction/dynamic-resource-allocation.md @@ -5,18 +5,21 @@ reviewers: title: Dynamic Resource Allocation content_type: concept weight: 65 +api_metadata: +- apiVersion: "resource.k8s.io/v1beta1" + kind: "ResourceClaim" +- apiVersion: "resource.k8s.io/v1beta1" + kind: "ResourceClaimTemplate" +- apiVersion: "resource.k8s.io/v1beta1" + kind: "DeviceClass" +- apiVersion: "resource.k8s.io/v1beta1" + kind: "ResourceSlice" --- -Core Dynamic Resource Allocation with structured parameters: - {{< feature-state feature_gate_name="DynamicResourceAllocation" >}} -Dynamic Resource Allocation with control plane controller: - -{{< feature-state feature_gate_name="DRAControlPlaneController" >}} - Dynamic resource allocation is an API for requesting and sharing resources between pods and containers inside a pod. It is a generalization of the persistent volumes API for generic resources. Typically those resources @@ -28,8 +31,10 @@ resources handled by Kubernetes via _structured parameters_ (introduced in Kuber Different kinds of resources support arbitrary parameters for defining requirements and initialization. -When a driver provides a _control plane controller_, the driver itself -handles allocation in cooperation with the Kubernetes scheduler. +Kubernetes v1.26 through to 1.31 included an (alpha) implementation of _classic DRA_, +which is no longer supported. This documentation, which is for Kubernetes +v{{< skew currentVersion >}}, explains the current approach to dynamic resource +allocation within Kubernetes. ## {{% heading "prerequisites" %}} @@ -43,7 +48,7 @@ v{{< skew currentVersion>}}, check the documentation for that version of Kuberne ## API -The `resource.k8s.io/v1alpha3` +The `resource.k8s.io/v1beta1` {{< glossary_tooltip text="API group" term_id="api-group" >}} provides these types: ResourceClaim @@ -65,25 +70,14 @@ DeviceClass when installing a resource driver. Each request to allocate a device in a ResourceClaim must reference exactly one DeviceClass. -PodSchedulingContext -: Used internally by the control plane and resource drivers - to coordinate pod scheduling when ResourceClaims need to be allocated - for a Pod and those ResourceClaims use a control plane controller. - ResourceSlice -: Used with structured parameters to publish information about resources +: Used by DRA drivers to publish information about resources that are available in the cluster. -The developer of a resource driver decides whether they want to handle -allocation themselves with a control plane controller or instead rely on allocation -through Kubernetes with structured parameters. A -custom controller provides more flexibility, but cluster autoscaling is not -going to work reliably for node-local resources. Structured parameters enable -cluster autoscaling, but might not satisfy all use-cases. - -When a driver uses structured parameters, all parameters that select devices -are defined in the ResourceClaim and DeviceClass with in-tree types. Configuration -parameters can be embedded there as arbitrary JSON objects. +All parameters that select devices are defined in the ResourceClaim and +DeviceClass with in-tree types. Configuration parameters can be embedded there. +Which configuration parameters are valid depends on the DRA driver -- Kubernetes +only passes them through without interpreting them. The `core/v1` `PodSpec` defines ResourceClaims that are needed for a Pod in a `resourceClaims` field. Entries in that list reference either a ResourceClaim @@ -100,7 +94,7 @@ Here is an example for a fictional resource driver. Two ResourceClaim objects will get created for this Pod and each container gets access to one of them. ```yaml -apiVersion: resource.k8s.io/v1alpha3 +apiVersion: resource.k8s.io/v1beta1 kind: DeviceClass name: resource.example.com spec: @@ -108,7 +102,7 @@ spec: - cel: expression: device.driver == "resource-driver.example.com" --- -apiVersion: resource.k8s.io/v1alpha2 +apiVersion: resource.k8s.io/v1beta1 kind: ResourceClaimTemplate metadata: name: large-black-cat-claim-template @@ -151,51 +145,7 @@ spec: ## Scheduling -### With control plane controller - -In contrast to native resources (CPU, RAM) and extended resources (managed by a -device plugin, advertised by kubelet), without structured parameters -the scheduler has no knowledge of what -dynamic resources are available in a cluster or how they could be split up to -satisfy the requirements of a specific ResourceClaim. Resource drivers are -responsible for that. They mark ResourceClaims as "allocated" once resources -for it are reserved. This also then tells the scheduler where in the cluster a -ResourceClaim is available. - -When a pod gets scheduled, the scheduler checks all ResourceClaims needed by a Pod and -creates a PodScheduling object where it informs the resource drivers -responsible for those ResourceClaims about nodes that the scheduler considers -suitable for the Pod. The resource drivers respond by excluding nodes that -don't have enough of the driver's resources left. Once the scheduler has that -information, it selects one node and stores that choice in the PodScheduling -object. The resource drivers then allocate their ResourceClaims so that the -resources will be available on that node. Once that is complete, the Pod -gets scheduled. - -As part of this process, ResourceClaims also get reserved for the -Pod. Currently ResourceClaims can either be used exclusively by a single Pod or -an unlimited number of Pods. - -One key feature is that Pods do not get scheduled to a node unless all of -their resources are allocated and reserved. This avoids the scenario where a Pod -gets scheduled onto one node and then cannot run there, which is bad because -such a pending Pod also blocks all other resources like RAM or CPU that were -set aside for it. - -{{< note >}} - -Scheduling of pods which use ResourceClaims is going to be slower because of -the additional communication that is required. Beware that this may also impact -pods that don't use ResourceClaims because only one pod at a time gets -scheduled, blocking API calls are made while handling a pod with -ResourceClaims, and thus scheduling the next pod gets delayed. - -{{< /note >}} - -### With structured parameters - -When a driver uses structured parameters, the scheduler takes over the -responsibility of allocating resources to a ResourceClaim whenever a pod needs +The scheduler is responsible for allocating resources to a ResourceClaim whenever a pod needs them. It does so by retrieving the full list of available resources from ResourceSlice objects, tracking which of those resources have already been allocated to existing ResourceClaims, and then selecting from those resources @@ -235,14 +185,9 @@ later. Such a situation can also arise when support for dynamic resource allocation was not enabled in the scheduler at the time when the Pod got scheduled (version skew, configuration, feature gate, etc.). kube-controller-manager -detects this and tries to make the Pod runnable by triggering allocation and/or -reserving the required ResourceClaims. - -{{< note >}} - -This only works with resource drivers that don't use structured parameters. - -{{< /note >}} +detects this and tries to make the Pod runnable by reserving the required +ResourceClaims. However, this only works if those were allocated by +the scheduler for some other pod. It is better to avoid bypassing the scheduler because a Pod that is assigned to a node blocks normal resources (RAM, CPU) that then cannot be used for other Pods @@ -264,17 +209,132 @@ spec: You may also be able to mutate the incoming Pod, at admission time, to unset the `.spec.nodeName` field and to use a node selector instead. +## Admin access + +{{< feature-state feature_gate_name="DRAAdminAccess" >}} + +You can mark a request in a ResourceClaim or ResourceClaimTemplate as having privileged features. +A request with admin access grants access to devices which are in use and +may enable additional permissions when making the device available in a +container: + +```yaml +apiVersion: resource.k8s.io/v1beta1 +kind: ResourceClaimTemplate +metadata: + name: large-black-cat-claim-template +spec: + spec: + devices: + requests: + - name: req-0 + deviceClassName: resource.example.com + adminAccess: true +``` + +If this feature is disabled, the `adminAccess` field will be removed +automatically when creating such a ResourceClaim. + +Admin access is a privileged mode which should not be made available to normal +users in a multi-tenant cluster. Cluster administrators can restrict usage of +this feature by installing a validating admission policy similar to the following +example. Cluster administrators need to adapt at least the names and replace +"dra.example.com". + +```yaml +# Permission to use admin access is granted only in namespaces which have the +# "admin-access.dra.example.com" label. Other ways of making that decision are +# also possible. + +apiVersion: admissionregistration.k8s.io/v1 +kind: ValidatingAdmissionPolicy +metadata: + name: resourceclaim-policy.dra.example.com +spec: + failurePolicy: Fail + matchConstraints: + resourceRules: + - apiGroups: ["resource.k8s.io"] + apiVersions: ["v1alpha3", "v1beta1"] + operations: ["CREATE", "UPDATE"] + resources: ["resourceclaims"] + validations: + - expression: '! object.spec.devices.requests.exists(e, has(e.adminAccess) && e.adminAccess)' + reason: Forbidden + messageExpression: '"admin access to devices not enabled"' +--- +apiVersion: admissionregistration.k8s.io/v1 +kind: ValidatingAdmissionPolicyBinding +metadata: + name: resourceclaim-binding.dra.example.com +spec: + policyName: resourceclaim-policy.dra.example.com + validationActions: [Deny] + matchResources: + namespaceSelector: + matchExpressions: + - key: admin-access.dra.example.com + operator: DoesNotExist +--- +apiVersion: admissionregistration.k8s.io/v1 +kind: ValidatingAdmissionPolicy +metadata: + name: resourceclaimtemplate-policy.dra.example.com +spec: + failurePolicy: Fail + matchConstraints: + resourceRules: + - apiGroups: ["resource.k8s.io"] + apiVersions: ["v1alpha3", "v1beta1"] + operations: ["CREATE", "UPDATE"] + resources: ["resourceclaimtemplates"] + validations: + - expression: '! object.spec.spec.devices.requests.exists(e, has(e.adminAccess) && e.adminAccess)' + reason: Forbidden + messageExpression: '"admin access to devices not enabled"' +--- +apiVersion: admissionregistration.k8s.io/v1 +kind: ValidatingAdmissionPolicyBinding +metadata: + name: resourceclaimtemplate-binding.dra.example.com +spec: + policyName: resourceclaimtemplate-policy.dra.example.com + validationActions: [Deny] + matchResources: + namespaceSelector: + matchExpressions: + - key: admin-access.dra.example.com + operator: DoesNotExist +``` + +## ResourceClaim Device Status + +{{< feature-state feature_gate_name="DRAResourceClaimDeviceStatus" >}} + +The drivers can report driver-specific device status data for each allocated device +in a resource claim. For example, IPs assigned to a network interface device can be +reported in the ResourceClaim status. + +The drivers setting the status, the accuracy of the information depends on the implementation +of those DRA Drivers. Therefore, the reported status of the device may not always reflect the +real time changes of the state of the device. + +When the feature is disabled, that field automatically gets cleared when storing the ResourceClaim. + +A ResourceClaim device status is supported when it is possible, from a DRA driver, to update an +existing ResourceClaim where the `status.devices` field is set. + ## Enabling dynamic resource allocation -Dynamic resource allocation is an *alpha feature* and only enabled when the +Dynamic resource allocation is a *beta feature* which is off by default and only enabled when the `DynamicResourceAllocation` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) -and the `resource.k8s.io/v1alpha3` {{< glossary_tooltip text="API group" term_id="api-group" >}} +and the `resource.k8s.io/v1beta1` {{< glossary_tooltip text="API group" term_id="api-group" >}} are enabled. For details on that, see the `--feature-gates` and `--runtime-config` [kube-apiserver parameters](/docs/reference/command-line-tools-reference/kube-apiserver/). kube-scheduler, kube-controller-manager and kubelet also need the feature gate. -When a resource driver uses a control plane controller, then the -`DRAControlPlaneController` feature gate has to be enabled in addition to +When a resource driver reports the status of the devices, then the +`DRAResourceClaimDeviceStatus` feature gate has to be enabled in addition to `DynamicResourceAllocation`. A quick check whether a Kubernetes cluster supports the feature is to list @@ -297,11 +357,6 @@ If not supported, this error is printed instead: error: the server doesn't have a resource type "deviceclasses" ``` -A control plane controller is supported when it is possible to create a -ResourceClaim where the `spec.controller` field is set. When the -`DRAControlPlaneController` feature is disabled, that field automatically -gets cleared when storing the ResourceClaim. - The default configuration of kube-scheduler enables the "DynamicResources" plugin if and only if the feature gate is enabled and when using the v1 configuration API. Custom configurations may have to be modified to @@ -310,9 +365,21 @@ include it. In addition to enabling the feature in the cluster, a resource driver also has to be installed. Please refer to the driver's documentation for details. +### Enabling admin access + +[Admin access](#admin-access) is an *alpha feature* and only enabled when the +`DRAAdminAccess` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) +is enabled in the kube-apiserver and kube-scheduler. + +### Enabling Device Status + +[ResourceClaim Device Status](#resourceclaim-device-status) is an *alpha feature* +and only enabled when the `DRAResourceClaimDeviceStatus` +[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) +is enabled in the kube-apiserver. + ## {{% heading "whatsnext" %}} - For more information on the design, see the [Dynamic Resource Allocation with Structured Parameters](https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/4381-dra-structured-parameters) - and the - [Dynamic Resource Allocation with Control Plane Controller](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/3063-dynamic-resource-allocation/README.md) KEPs. + KEP. diff --git a/content/en/docs/concepts/scheduling-eviction/scheduling-framework.md b/content/en/docs/concepts/scheduling-eviction/scheduling-framework.md index 63a8c7d3e6fec..d76ad3e1d0809 100644 --- a/content/en/docs/concepts/scheduling-eviction/scheduling-framework.md +++ b/content/en/docs/concepts/scheduling-eviction/scheduling-framework.md @@ -74,7 +74,7 @@ Plugins that implement PreEnqueue, PreFilter, Filter, Reserve or Permit should i ### QueueingHint -{{< feature-state for_k8s_version="v1.28" state="beta" >}} +{{< feature-state for_k8s_version="v1.32" state="beta" >}} QueueingHint is a callback function for deciding whether a Pod can be requeued to the active queue or backoff queue. It's executed every time a certain kind of event or change happens in the cluster. @@ -84,11 +84,8 @@ so that the scheduler will retry the scheduling of the Pod. {{< note >}} QueueingHint evaluation during scheduling is a beta-level feature. -The v1.28 release series initially enabled the associated feature gate; however, after the -discovery of an excessive memory footprint, the Kubernetes project set that feature gate -to be disabled by default. In Kubernetes {{< skew currentVersion >}}, this feature gate is -disabled and you need to enable it manually. -You can enable it via the +In Kubernetes {{< skew currentVersion >}}, this feature gate is enabled by default, +and you can disable it via the `SchedulerQueueingHints` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/). {{< /note >}} diff --git a/content/en/docs/concepts/scheduling-eviction/topology-spread-constraints.md b/content/en/docs/concepts/scheduling-eviction/topology-spread-constraints.md index a20f15b9ce246..935ab1c7a9176 100644 --- a/content/en/docs/concepts/scheduling-eviction/topology-spread-constraints.md +++ b/content/en/docs/concepts/scheduling-eviction/topology-spread-constraints.md @@ -99,7 +99,7 @@ your cluster. Those fields are: {{< note >}} Before Kubernetes v1.30, the `minDomains` field was only available if the - `MinDomainsInPodTopologySpread` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) + `MinDomainsInPodTopologySpread` [feature gate](/docs/reference/command-line-tools-reference/feature-gates-removed/) was enabled (default since v1.28). In older Kubernetes clusters it might be explicitly disabled or the field might not be available. {{< /note >}} diff --git a/content/en/docs/concepts/security/secrets-good-practices.md b/content/en/docs/concepts/security/secrets-good-practices.md index 3e18929f90f3b..0075fa9ebec77 100644 --- a/content/en/docs/concepts/security/secrets-good-practices.md +++ b/content/en/docs/concepts/security/secrets-good-practices.md @@ -62,11 +62,8 @@ recommendations include: * Implement audit rules that alert on specific events, such as concurrent reading of multiple Secrets by a single user -#### Additional ServiceAccount annotations for Secret management - -You can also use the `kubernetes.io/enforce-mountable-secrets` annotation on -a ServiceAccount to enforce specific rules on how Secrets are used in a Pod. -For more details, see the [documentation on this annotation](/docs/reference/labels-annotations-taints/#enforce-mountable-secrets). +#### Restrict Access for Secrets +Use separate namespaces to isolate access to mounted secrets. ### Improve etcd management policies diff --git a/content/en/docs/concepts/security/service-accounts.md b/content/en/docs/concepts/security/service-accounts.md index 95fde2f28a81f..e1ed8ac958475 100644 --- a/content/en/docs/concepts/security/service-accounts.md +++ b/content/en/docs/concepts/security/service-accounts.md @@ -197,7 +197,13 @@ or using a custom mechanism such as an [authentication webhook](/docs/reference/ You can also use TokenRequest to obtain short-lived tokens for your external application. {{< /note >}} -### Restricting access to Secrets {#enforce-mountable-secrets} +### Restricting access to Secrets (deprecated) {#enforce-mountable-secrets} + +{{< feature-state for_k8s_version="v1.32" state="deprecated" >}} + +{{< note >}} +`kubernetes.io/enforce-mountable-secrets` is deprecated since Kubernetes v1.32. Use separate namespaces to isolate access to mounted secrets. +{{< /note >}} Kubernetes provides an annotation called `kubernetes.io/enforce-mountable-secrets` that you can add to your ServiceAccounts. When this annotation is applied, diff --git a/content/en/docs/concepts/services-networking/service.md b/content/en/docs/concepts/services-networking/service.md index 8681d74a53d9b..c14e4765c7429 100644 --- a/content/en/docs/concepts/services-networking/service.md +++ b/content/en/docs/concepts/services-networking/service.md @@ -681,14 +681,11 @@ The value of `spec.loadBalancerClass` must be a label-style identifier, with an optional prefix such as "`internal-vip`" or "`example.com/internal-vip`". Unprefixed names are reserved for end-users. -#### Specifying IPMode of load balancer status {#load-balancer-ip-mode} +#### Load balancer IP address mode {#load-balancer-ip-mode} {{< feature-state feature_gate_name="LoadBalancerIPMode" >}} -As a Beta feature in Kubernetes 1.30, -a [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) -named `LoadBalancerIPMode` allows you to set the `.status.loadBalancer.ingress.ipMode` -for a Service with `type` set to `LoadBalancer`. +For a Service of `type: LoadBalancer`, a controller can set `.status.loadBalancer.ingress.ipMode`. The `.status.loadBalancer.ingress.ipMode` specifies how the load-balancer IP behaves. It may be specified only when the `.status.loadBalancer.ingress.ip` field is also specified. diff --git a/content/en/docs/concepts/storage/persistent-volumes.md b/content/en/docs/concepts/storage/persistent-volumes.md index 39891d5e49959..90b79be3381f5 100644 --- a/content/en/docs/concepts/storage/persistent-volumes.md +++ b/content/en/docs/concepts/storage/persistent-volumes.md @@ -478,18 +478,17 @@ administrator intervention. {{% /tab %}} {{% tab name="By requesting expansion to smaller size" %}} -{{% feature-state for_k8s_version="v1.23" state="alpha" %}} +{{< feature-state feature_gate_name="RecoverVolumeExpansionFailure" >}} {{< note >}} -Recovery from failing PVC expansion by users is available as an alpha feature -since Kubernetes 1.23. The `RecoverVolumeExpansionFailure` feature must be -enabled for this feature to work. Refer to the +Recover from failing PVC expansion by users (`RecoverVolumeExpansionFailure`) is available as an beta feature +since Kubernetes 1.32 and should be enabled by default. Refer to the [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) documentation for more information. {{< /note >}} -If the feature gates `RecoverVolumeExpansionFailure` is -enabled in your cluster, and expansion has failed for a PVC, you can retry expansion with a + +When using `RecoverVolumeExpansionFailure` feature, if expansion has failed for a PVC, you can retry expansion with a smaller size than the previously requested value. To request a new expansion attempt with a smaller proposed size, edit `.spec.resources` for that PVC and choose a value that is less than the value you previously tried. diff --git a/content/en/docs/concepts/storage/volume-snapshots.md b/content/en/docs/concepts/storage/volume-snapshots.md index 3f1744e108751..f2cbc90db99f7 100644 --- a/content/en/docs/concepts/storage/volume-snapshots.md +++ b/content/en/docs/concepts/storage/volume-snapshots.md @@ -65,6 +65,9 @@ Users need to be aware of the following when using this feature: the csi-snapshotter. See [CSI Driver documentation](https://kubernetes-csi.github.io/docs/) for details. - The CRDs and snapshot controller installations are the responsibility of the Kubernetes distribution. +For advanced use cases, such as creating group snapshots of multiple volumes, see the external +[CSI Volume Group Snapshot documentation](https://kubernetes-csi.github.io/docs/group-snapshot-restore-feature.html). + ## Lifecycle of a volume snapshot and volume snapshot content `VolumeSnapshotContents` are resources in the cluster. `VolumeSnapshots` are requests diff --git a/content/en/docs/concepts/storage/volumes.md b/content/en/docs/concepts/storage/volumes.md index bd0f40b9556ba..b431f62eb236c 100644 --- a/content/en/docs/concepts/storage/volumes.md +++ b/content/en/docs/concepts/storage/volumes.md @@ -242,19 +242,12 @@ the `emptyDir.medium` field to `"Memory"`, Kubernetes mounts a tmpfs (RAM-backed filesystem) for you instead. While tmpfs is very fast be aware that, unlike disks, files you write count against the memory limit of the container that wrote them. - A size limit can be specified for the default medium, which limits the capacity of the `emptyDir` volume. The storage is allocated from [node ephemeral storage](/docs/concepts/configuration/manage-resources-containers/#setting-requests-and-limits-for-local-ephemeral-storage). If that is filled up from another source (for example, log files or image overlays), the `emptyDir` may run out of capacity before this limit. - -{{< note >}} -You can specify a size for memory backed volumes, provided that the `SizeMemoryBackedVolumes` -[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) -is enabled in your cluster (this has been beta, and active by default, since the Kubernetes 1.22 release). -If you don't specify a volume size, memory backed volumes are sized to node allocatable memory. -{{< /note>}} +If no size is specified, memory backed volumes are sized to node allocatable memory. {{< caution >}} Please check [here](/docs/concepts/configuration/manage-resources-containers/#memory-backed-emptydir) @@ -281,6 +274,27 @@ spec: sizeLimit: 500Mi ``` +#### emptyDir memory configuration example + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: test-pd +spec: + containers: + - image: registry.k8s.io/test-webserver + name: test-container + volumeMounts: + - mountPath: /cache + name: cache-volume + volumes: + - name: cache-volume + emptyDir: + sizeLimit: 500Mi + medium: Memory +``` + ### fc (fibre channel) {#fc} An `fc` volume type allows an existing fibre channel block storage volume diff --git a/content/en/docs/concepts/workloads/autoscaling.md b/content/en/docs/concepts/workloads/autoscaling.md index ff154d71599be..f2a5f1fbe0718 100644 --- a/content/en/docs/concepts/workloads/autoscaling.md +++ b/content/en/docs/concepts/workloads/autoscaling.md @@ -79,7 +79,7 @@ Mode | Description #### Requirements for in-place resizing -{{< feature-state for_k8s_version="v1.27" state="alpha" >}} +{{< feature-state feature_gate_name="InPlacePodVerticalScaling" >}} Resizing a workload in-place **without** restarting the {{< glossary_tooltip text="Pods" term_id="pod" >}} or its {{< glossary_tooltip text="Containers" term_id="container" >}} requires Kubernetes version 1.27 or later. diff --git a/content/en/docs/concepts/workloads/controllers/cron-jobs.md b/content/en/docs/concepts/workloads/controllers/cron-jobs.md index 0af2404a54eb6..397d7a4050beb 100644 --- a/content/en/docs/concepts/workloads/controllers/cron-jobs.md +++ b/content/en/docs/concepts/workloads/controllers/cron-jobs.md @@ -216,6 +216,10 @@ are certain circumstances where two Jobs might be created, or no Job might be cr Kubernetes tries to avoid those situations, but does not completely prevent them. Therefore, the Jobs that you define should be _idempotent_. +Starting with Kubernetes v1.32, CronJobs apply an annotation +`batch.kubernetes.io/cronjob-scheduled-timestamp` to their created Jobs. This annotation +indicates the originally scheduled creation time for the Job and is formatted in RFC3339. + If `startingDeadlineSeconds` is set to a large value or left unset (the default) and if `concurrencyPolicy` is set to `Allow`, the Jobs will always run at least once. diff --git a/content/en/docs/concepts/workloads/controllers/job.md b/content/en/docs/concepts/workloads/controllers/job.md index 7cd1f5b928fa1..901660fa305ec 100644 --- a/content/en/docs/concepts/workloads/controllers/job.md +++ b/content/en/docs/concepts/workloads/controllers/job.md @@ -695,8 +695,8 @@ triggered and all Pod finalizers were removed. However, some Pods would still be running or terminating at the moment that the terminal condition was added. In Kubernetes v1.31 and later, the controller only adds the Job terminal conditions -_after_ all of the Pods are terminated. You can enable this behavior by using the -`JobManagedBy` or the `JobPodReplacementPolicy` (enabled by default) +_after_ all of the Pods are terminated. You can control this behavior by using the +`JobManagedBy` and the `JobPodReplacementPolicy` (both enabled by default) [feature gates](/docs/reference/command-line-tools-reference/feature-gates/). ### Termination of Job pods @@ -1137,7 +1137,7 @@ status: {{< note >}} You can only set the `managedBy` field on Jobs if you enable the `JobManagedBy` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) -(disabled by default). +(enabled by default). {{< /note >}} This feature allows you to disable the built-in Job controller, for a specific diff --git a/content/en/docs/concepts/workloads/controllers/statefulset.md b/content/en/docs/concepts/workloads/controllers/statefulset.md index 177dd3b371abd..3a79abb5d6841 100644 --- a/content/en/docs/concepts/workloads/controllers/statefulset.md +++ b/content/en/docs/concepts/workloads/controllers/statefulset.md @@ -252,13 +252,13 @@ the StatefulSet. ### Pod index label -{{< feature-state for_k8s_version="v1.28" state="beta" >}} +{{< feature-state feature_gate_name="PodIndexLabel" >}} When the StatefulSet {{}} creates a Pod, the new Pod is labelled with `apps.kubernetes.io/pod-index`. The value of this label is the ordinal index of the Pod. This label allows you to route traffic to a particular pod index, filter logs/metrics -using the pod index label, and more. Note the feature gate `PodIndexLabel` must be enabled for this -feature, and it is enabled by default. +using the pod index label, and more. Note the feature gate `PodIndexLabel` is enabled and locked by default for this +feature, in order to disable it, users will have to use server emulated version v1.31. ## Deployment and Scaling Guarantees @@ -386,7 +386,7 @@ StatefulSet will then begin to recreate the Pods using the reverted template. ## PersistentVolumeClaim retention -{{< feature-state for_k8s_version="v1.27" state="beta" >}} +{{< feature-state feature_gate_name="StatefulSetAutoDeletePVC" >}} The optional `.spec.persistentVolumeClaimRetentionPolicy` field controls if and how PVCs are deleted during the lifecycle of a StatefulSet. You must enable the @@ -493,4 +493,4 @@ the `.spec.replicas` field automatically. Read the {{< api-reference page="workload-resources/stateful-set-v1" >}} object definition to understand the API for stateful sets. * Read about [PodDisruptionBudget](/docs/concepts/workloads/pods/disruptions/) and how - you can use it to manage application availability during disruptions. \ No newline at end of file + you can use it to manage application availability during disruptions. diff --git a/content/en/docs/concepts/workloads/pods/pod-lifecycle.md b/content/en/docs/concepts/workloads/pods/pod-lifecycle.md index eef73e8a26bd8..7d08a384fcdb0 100644 --- a/content/en/docs/concepts/workloads/pods/pod-lifecycle.md +++ b/content/en/docs/concepts/workloads/pods/pod-lifecycle.md @@ -260,6 +260,38 @@ problems, the kubelet resets the restart backoff timer for that container. [Sidecar containers and Pod lifecycle](/docs/concepts/workloads/pods/sidecar-containers/#sidecar-containers-and-pod-lifecycle) explains the behaviour of `init containers` when specify `restartpolicy` field on it. +### Configurable container restart delay + +{{< feature-state feature_gate_name="KubeletCrashLoopBackOffMax" >}} + +With the alpha feature gate `KubeletCrashLoopBackOffMax` enabled, you can +reconfigure the maximum delay between container start retries from the default +of 300s (5 minutes). This configuration is set per node using kubelet +configuration. In your [kubelet configuration](/docs/tasks/administer-cluster/kubelet-config-file/), +under `crashLoopBackOff` set the `maxContainerRestartPeriod` field between +`"1s"` and `"300s"`. As described above in [Container restart +policy](#restart-policy), delays on that node will still start at 10s and +increase exponentially by 2x each restart, but will now be capped at your +configured maximum. If the `maxContainerRestartPeriod` you configure is less +than the default initial value of 10s, the initial delay will instead be set to +the configured maximum. + +See the following kubelet configuration examples: + +```yaml +# container restart delays will start at 10s, increasing +# 2x each time they are restarted, to a maximum of 100s +kind: KubeletConfiguration +crashLoopBackOff: + maxContainerRestartPeriod: "100s" +``` + +```yaml +# delays between container restarts will always be 2s +kind: KubeletConfiguration +crashLoopBackOff: + maxContainerRestartPeriod: "2s" +``` ## Pod conditions @@ -677,8 +709,7 @@ Additionally, PodGC cleans up any Pods which satisfy any of the following condit 1. are orphan Pods - bound to a node which no longer exists, 1. are unscheduled terminating Pods, 1. are terminating Pods, bound to a non-ready node tainted with - [`node.kubernetes.io/out-of-service`](/docs/reference/labels-annotations-taints/#node-kubernetes-io-out-of-service), - when the `NodeOutOfServiceVolumeDetach` feature gate is enabled. + [`node.kubernetes.io/out-of-service`](/docs/reference/labels-annotations-taints/#node-kubernetes-io-out-of-service). Along with cleaning up the Pods, PodGC will also mark them as failed if they are in a non-terminal phase. Also, PodGC adds a Pod disruption condition when cleaning up an orphan Pod. diff --git a/content/en/docs/reference/access-authn-authz/admission-controllers.md b/content/en/docs/reference/access-authn-authz/admission-controllers.md index de2c6f447a216..fe41eb8351724 100644 --- a/content/en/docs/reference/access-authn-authz/admission-controllers.md +++ b/content/en/docs/reference/access-authn-authz/admission-controllers.md @@ -794,9 +794,7 @@ The Kubernetes project strongly recommends enabling this admission controller. You should enable this admission controller if you intend to make any use of Kubernetes `ServiceAccount` objects. -Regarding the annotation `kubernetes.io/enforce-mountable-secrets`: While the annotation's name suggests it only concerns the mounting of Secrets, -its enforcement also extends to other ways Secrets are used in the context of a Pod. -Therefore, it is crucial to ensure that all the referenced secrets are correctly specified in the ServiceAccount. +To enhance the security measures around Secrets, use separate namespaces to isolate access to mounted secrets. ### StorageObjectInUseProtection diff --git a/content/en/docs/reference/access-authn-authz/authentication.md b/content/en/docs/reference/access-authn-authz/authentication.md index f6371199b4510..1f52aea2799c4 100644 --- a/content/en/docs/reference/access-authn-authz/authentication.md +++ b/content/en/docs/reference/access-authn-authz/authentication.md @@ -466,6 +466,12 @@ jwt: expression: 'claims.sub' # extra attributes to be added to the UserInfo object. Keys must be domain-prefix path and must be unique. extra: + # key is a string to use as the extra attribute key. + # key must be a domain-prefix path (e.g. example.org/foo). All characters before the first "/" must be a valid + # subdomain as defined by RFC 1123. All characters trailing the first "/" must + # be valid HTTP Path characters as defined by RFC 3986. + # k8s.io, kubernetes.io and their subdomains are reserved for Kubernetes use and cannot be used. + # key must be lowercase and unique across all extra attributes. - key: 'example.com/tenant' # valueExpression is a CEL expression that evaluates to a string or a list of strings. valueExpression: 'claims.tenant' @@ -1087,9 +1093,8 @@ that grant access to the `*` user or `*` group do not include anonymous users. {{< feature-state feature_gate_name="AnonymousAuthConfigurableEndpoints" >}} The `AuthenticationConfiguration` can be used to configure the anonymous -authenticator. To enable configuring anonymous auth via the config file you need -enable the `AnonymousAuthConfigurableEndpoints` feature gate. When this feature -gate is enabled you cannot set the `--anonymous-auth` flag. +authenticator. If you set the anonymous field in the `AuthenticationConfiguration` +file then you cannot set the `--anonymous-auth` flag. The main advantage of configuring anonymous authenticator using the authentication configuration file is that in addition to enabling and disabling anonymous authentication diff --git a/content/en/docs/reference/access-authn-authz/authorization.md b/content/en/docs/reference/access-authn-authz/authorization.md index d68c39765cf17..292bf6f8ae4e8 100644 --- a/content/en/docs/reference/access-authn-authz/authorization.md +++ b/content/en/docs/reference/access-authn-authz/authorization.md @@ -165,41 +165,14 @@ to the built-in `cluster-admin` ClusterRole. ### Authorization mode configuration {#choice-of-authz-config} You can configure the Kubernetes API server's authorizer chain using either -[command line arguments](#using-flags-for-your-authorization-module) only or, as a beta feature, -using a [configuration file](#using-configuration-file-for-authorization). +a [configuration file](#using-configuration-file-for-authorization) only or +[command line arguments](#using-flags-for-your-authorization-module). You have to pick one of the two configuration approaches; setting both `--authorization-config` path and configuring an authorization webhook using the `--authorization-mode` and `--authorization-webhook-*` command line arguments is not allowed. If you try this, the API server reports an error message during startup, then exits immediately. -### Command line authorization mode configuration {#using-flags-for-your-authorization-module} - -{{< feature-state state="stable" for_k8s_version="v1.8" >}} - -You can use the following modes: - -* `--authorization-mode=ABAC` (Attribute-based access control mode) -* `--authorization-mode=RBAC` (Role-based access control mode) -* `--authorization-mode=Node` (Node authorizer) -* `--authorization-mode=Webhook` (Webhook authorization mode) -* `--authorization-mode=AlwaysAllow` (always allows requests; carries [security risks](#warning-always-allow)) -* `--authorization-mode=AlwaysDeny` (always denies requests) - -You can choose more than one authorization mode; for example: -`--authorization-mode=Node,Webhook` - -Kubernetes checks authorization modules based on the order that you specify them -on the API server's command line, so an earlier module has higher priority to allow -or deny a request. - -You cannot combine the `--authorization-mode` command line argument with the -`--authorization-config` command line argument used for -[configuring authorization using a local file](#using-configuration-file-for-authorization-mode). - -For more information on command line arguments to the API server, read the -[`kube-apiserver` reference](/docs/reference/command-line-tools-reference/kube-apiserver/). - @@ -207,7 +180,7 @@ For more information on command line arguments to the API server, read the {{< feature-state feature_gate_name="StructuredAuthorizationConfiguration" >}} -As a beta feature, Kubernetes lets you configure authorization chains that can include multiple +Kubernetes lets you configure authorization chains that can include multiple webhooks. The authorization items in that chain can have well-defined parameters that validate requests in a particular order, offering you fine-grained control, such as explicit Deny on failures. @@ -230,7 +203,7 @@ are only available if you use an authorization configuration file. # # DO NOT USE THE CONFIG AS IS. THIS IS AN EXAMPLE. # -apiVersion: apiserver.config.k8s.io/v1beta1 +apiVersion: apiserver.config.k8s.io/v1 kind: AuthorizationConfiguration authorizers: - type: Webhook @@ -347,6 +320,31 @@ A reload **must not** add or remove Node or RBAC authorizers (they can be reorde but cannot be added or removed). {{< /note >}} +### Command line authorization mode configuration {#using-flags-for-your-authorization-module} + +You can use the following modes: + +* `--authorization-mode=ABAC` (Attribute-based access control mode) +* `--authorization-mode=RBAC` (Role-based access control mode) +* `--authorization-mode=Node` (Node authorizer) +* `--authorization-mode=Webhook` (Webhook authorization mode) +* `--authorization-mode=AlwaysAllow` (always allows requests; carries [security risks](#warning-always-allow)) +* `--authorization-mode=AlwaysDeny` (always denies requests) + +You can choose more than one authorization mode; for example: +`--authorization-mode=Node,RBAC,Webhook` + +Kubernetes checks authorization modules based on the order that you specify them +on the API server's command line, so an earlier module has higher priority to allow +or deny a request. + +You cannot combine the `--authorization-mode` command line argument with the +`--authorization-config` command line argument used for +[configuring authorization using a local file](#using-configuration-file-for-authorization-mode). + +For more information on command line arguments to the API server, read the +[`kube-apiserver` reference](/docs/reference/command-line-tools-reference/kube-apiserver/). + ## Privilege escalation via workload creation or edits {#privilege-escalation-via-pod-creation} Users who can create/edit pods in a namespace, either directly or through an object that diff --git a/content/en/docs/reference/access-authn-authz/kubelet-authn-authz.md b/content/en/docs/reference/access-authn-authz/kubelet-authn-authz.md index 87c6dde16c7e8..7de116784f675 100644 --- a/content/en/docs/reference/access-authn-authz/kubelet-authn-authz.md +++ b/content/en/docs/reference/access-authn-authz/kubelet-authn-authz.md @@ -85,3 +85,38 @@ flags passed to the apiserver is authorized for the following attributes: * verb=\*, resource=nodes, subresource=log * verb=\*, resource=nodes, subresource=spec * verb=\*, resource=nodes, subresource=metrics + +### Fine-grained authorization + +{{< feature-state feature_gate_name="KubeletFineGrainedAuthz" >}} + +When the feature gate `KubeletFineGrainedAuthz` is enabled kubelet performs a +fine-grained check before falling back to the `proxy` subresource for the `/pods`, +`/runningPods`, `/configz` and `/healthz` endpoints. The resource and subresource +are determined from the incoming request's path: + +Kubelet API | resource | subresource +--------------|----------|------------ +/stats/\* | nodes | stats +/metrics/\* | nodes | metrics +/logs/\* | nodes | log +/spec/\* | nodes | spec +/pods | nodes | pods, proxy +/runningPods/ | nodes | pods, proxy +/healthz | nodes | healthz, proxy +/configz | nodes | configz, proxy +*all others* | nodes | proxy + + +When the feature-gate `KubeletFineGrainedAuthz` is enabled, ensure the user +identified by the `--kubelet-client-certificate` and `--kubelet-client-key` +flags passed to the API server is authorized for the following attributes: + +* verb=\*, resource=nodes, subresource=proxy +* verb=\*, resource=nodes, subresource=stats +* verb=\*, resource=nodes, subresource=log +* verb=\*, resource=nodes, subresource=spec +* verb=\*, resource=nodes, subresource=metrics +* verb=\*, resource=nodes, subresource=configz +* verb=\*, resource=nodes, subresource=healthz +* verb=\*, resource=nodes, subresource=pods diff --git a/content/en/docs/reference/access-authn-authz/mutating-admission-policy.md b/content/en/docs/reference/access-authn-authz/mutating-admission-policy.md new file mode 100644 index 0000000000000..14174d34296c2 --- /dev/null +++ b/content/en/docs/reference/access-authn-authz/mutating-admission-policy.md @@ -0,0 +1,216 @@ +--- +reviewers: +- deads2k +- sttts +- cici37 +title: Mutating Admission Policy +content_type: concept +--- + + + +{{< feature-state for_k8s_version="v1.32" state="alpha" >}} + + +This page provides an overview of _MutatingAdmissionPolicies_. + + + +## What are MutatingAdmissionPolicies? + +Mutating admission policies offer a declarative, in-process alternative to mutating admission webhooks. + +Mutating admission policies use the Common Expression Language (CEL) to declare mutations to resources. +Mutations can be defined either with an *apply configuration* that is merged using the +[server side apply merge strategy](/docs/reference/using-api/server-side-apply/#merge-strategy), +or a [JSON patch](https://jsonpatch.com/). + +Mutating admission policies are highly configurable, enabling policy authors to define policies +that can be parameterized and scoped to resources as needed by cluster administrators. + +## What resources make a policy + +A policy is generally made up of three resources: + +- The MutatingAdmissionPolicy describes the abstract logic of a policy + (think: "this policy sets a particular label to a particular value"). + +- A _parameter resource_ provides information to a MutatingAdmissionPolicy to make it a concrete + statement (think "set the `owner` label to something like `company.example.com`"). + Parameter resources refer to Kubernetes resources, available in the Kubernetes API. They can be built-in types or extensions, + such as a {{< glossary_tooltip term_id="CustomResourceDefinition" text="CustomResourceDefinition" >}} (CRD). For example, you can use a ConfigMap as a parameter. +- A MutatingAdmissionPolicyBinding links the above (MutatingAdmissionPolicy and parameter) resources together and provides scoping. + If you only want to set an `owner` label for `Pods`, and not other API kinds, the binding is where you + specify this mutation. + + + +At least a MutatingAdmissionPolicy and a corresponding MutatingAdmissionPolicyBinding +must be defined for a policy to have an effect. + +If a MutatingAdmissionPolicy does not need to be configured via parameters, simply leave +`spec.paramKind` in MutatingAdmissionPolicy not specified. + +## Getting Started with MutatingAdmissionPolicies + +Mutating admission policy is part of the cluster control-plane. You should write +and deploy them with great caution. The following describes how to quickly +experiment with Mutating admission policy. + +### Create a MutatingAdmissionPolicy + +The following is an example of a MutatingAdmissionPolicy. This policy mutates newly created Pods to have a sidecar container if it does not exist. + +{{% code_sample language="yaml" file="mutatingadmissionpolicy/applyconfiguration-example.yaml" %}} + +The `.spec.mutations` field consists of a list of expressions that evaluate to resource patches. +The emitted patches may be either [apply configurations](#patch-type-apply-configuration) or [JSON Patch](#patch-type-json-patch) +patches. You cannot specify an empty list of mutations. After evaluating all the +expressions, the API server applies those changes to the resource that is +passing through admission. + +To configure a mutating admission policy for use in a cluster, a binding is +required. The MutatingAdmissionPolicy will only be active if a corresponding +binding exists with the referenced `spec.policyName` matching the `spec.name` of +a policy. + +Once the binding and policy are created, any resource request that matches the +`spec.matchConditions` of a policy will trigger the set of mutations defined. + +In the example above, creating a Pod will add the `mesh-proxy` initContainer mutation: + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: myapp + namespace: default +spec: + ... + initContainers: + - name: mesh-proxy + image: mesh/proxy:v1.0.0 + args: ["proxy", "sidecar"] + restartPolicy: Always + - name: myapp-initializer + image: example/initializer:v1.0.0 + ... +``` + +#### Parameter resources + +Parameter resources allow a policy configuration to be separate from its +definition. A policy can define `paramKind`, which outlines GVK of the parameter +resource, and then a policy binding ties a policy by name (via `policyName`) to a +particular parameter resource via `paramRef`. + +Please refer to [parameter resources](/docs/reference/access-authn-authz/validating-admission-policy/#parameter-resources) for more information. + +#### `ApplyConfiguration` {#patch-type-apply-configuration} + +MutatingAdmissionPolicy expressions are always CEL. Each apply configuration +`expression` must evaluate to a CEL object (declared using `Object()` +initialization). + +Apply configurations may not modify atomic structs, maps or arrays due to the risk of accidental deletion of +values not included in the apply configuration. + +CEL expressions have access to the object types needed to create apply configurations: + +- `Object` - CEL type of the resource object. +- `Object.` - CEL type of object field (such as `Object.spec`) +- `Object.....` - CEL type of nested field (such as `Object.spec.containers`) + +CEL expressions have access to the contents of the API request, organized into CEL variables as well as some other useful variables: + +- `object` - The object from the incoming request. The value is null for DELETE requests. +- `oldObject` - The existing object. The value is null for CREATE requests. +- `request` - Attributes of the API request. +- `params` - Parameter resource referred to by the policy binding being evaluated. Only populated if the policy has a ParamKind. +- `namespaceObject` - The namespace object that the incoming object belongs to. The value is null for cluster-scoped resources. +- `variables` - Map of composited variables, from its name to its lazily evaluated value. + For example, a variable named `foo` can be accessed as `variables.foo`. +- `authorizer` - A CEL Authorizer. May be used to perform authorization checks for the principal (user or service account) of the request. + See https://pkg.go.dev/k8s.io/apiserver/pkg/cel/library#Authz +- `authorizer.requestResource` - A CEL ResourceCheck constructed from the `authorizer` and configured with the + request resource. + +The `apiVersion`, `kind`, `metadata.name`, `metadata.generateName` and `metadata.labels` are always accessible from the root of the +object. No other metadata properties are accessible. + +#### `JSONPatch` {#patch-type-json-patch} + +The same mutation can be written as a [JSON Patch](https://jsonpatch.com/) as follows: + +{{% code_sample language="yaml" file="mutatingadmissionpolicy/json-patch-example.yaml" %}} + +The expression will be evaluated by CEL to create a [JSON patch](https://jsonpatch.com/). +ref: https://github.com/google/cel-spec + +Each evaluated `expression` must return an array of `JSONPatch` values. The +`JSONPatch` type represents one operation from a JSON patch. + +For example, this CEL expression returns a JSON patch to conditionally modify a value: + +``` + [ + JSONPatch{op: "test", path: "/spec/example", value: "Red"}, + JSONPatch{op: "replace", path: "/spec/example", value: "Green"} + ] +``` + +To define a JSON object for the patch operation `value`, use CEL `Object` types. For example: + +``` + [ + JSONPatch{ + op: "add", + path: "/spec/selector", + value: Object.spec.selector{matchLabels: {"environment": "test"}} + } + ] +``` + +To use strings containing '/' and '~' as JSONPatch path keys, use `jsonpatch.escapeKey()`. For example: + +``` + [ + JSONPatch{ + op: "add", + path: "/metadata/labels/" + jsonpatch.escapeKey("example.com/environment"), + value: "test" + }, + ] +``` + +CEL expressions have access to the types needed to create JSON patches and objects: + +- `JSONPatch` - CEL type of JSON Patch operations. JSONPatch has the fields `op`, `from`, `path` and `value`. + See [JSON patch](https://jsonpatch.com/) for more details. The `value` field may be set to any of: string, + integer, array, map or object. If set, the `path` and `from` fields must be set to a + [JSON pointer](https://datatracker.ietf.org/doc/html/rfc6901/) string, where the `jsonpatch.escapeKey()` CEL + function may be used to escape path keys containing `/` and `~`. +- `Object` - CEL type of the resource object. +- `Object.` - CEL type of object field (such as `Object.spec`) +- `Object.....` - CEL type of nested field (such as `Object.spec.containers`) + +CEL expressions have access to the contents of the API request, organized into CEL variables as well as some other useful variables: + +- `object` - The object from the incoming request. The value is null for DELETE requests. +- `oldObject` - The existing object. The value is null for CREATE requests. +- `request` - Attributes of the API request. +- `params` - Parameter resource referred to by the policy binding being evaluated. Only populated if the policy has a ParamKind. +- `namespaceObject` - The namespace object that the incoming object belongs to. The value is null for cluster-scoped resources. +- `variables` - Map of composited variables, from its name to its lazily evaluated value. + For example, a variable named `foo` can be accessed as `variables.foo`. +- `authorizer` - A CEL Authorizer. May be used to perform authorization checks for the principal (user or service account) of the request. + See https://pkg.go.dev/k8s.io/apiserver/pkg/cel/library#Authz +- `authorizer.requestResource` - A CEL ResourceCheck constructed from the `authorizer` and configured with the + request resource. + +CEL expressions have access to [Kubernetes CEL function libraries](/docs/reference/using-api/cel/#cel-options-language-features-and-libraries) +as well as: + +- `jsonpatch.escapeKey` - Performs JSONPatch key escaping. `~` and `/` are escaped as `~0` and `~1` respectively. + +Only property names of the form `[a-zA-Z_.-/][a-zA-Z0-9_.-/]*` are accessible. \ No newline at end of file diff --git a/content/en/docs/reference/access-authn-authz/node.md b/content/en/docs/reference/access-authn-authz/node.md index d39b404c0c80b..dd144d7fa1403 100644 --- a/content/en/docs/reference/access-authn-authz/node.md +++ b/content/en/docs/reference/access-authn-authz/node.md @@ -69,7 +69,24 @@ the local `hostname` and the `--hostname-override` option. For specifics about how the kubelet determines the hostname, see the [kubelet options reference](/docs/reference/command-line-tools-reference/kubelet/). -To enable the Node authorizer, start the apiserver with `--authorization-mode=Node`. +To enable the Node authorizer, start the {{< glossary_tooltip text="API server" term_id="kube-apiserver" >}} +with the `--authorization-config` flag set to a file that includes the `Node` authorizer; for example: + +```yaml +apiVersion: apiserver.config.k8s.io/v1 +kind: AuthorizationConfiguration +authorizers: + ... + - type: Node + ... +``` + +Or, start the {{< glossary_tooltip text="API server" term_id="kube-apiserver" >}} with +the `--authorization-mode` flag set to a comma-separated list that includes `Node`; +for example: +```shell +kube-apiserver --authorization-mode=...,Node --other-options --more-options +``` To limit the API objects kubelets are able to write, enable the [NodeRestriction](/docs/reference/access-authn-authz/admission-controllers#noderestriction) diff --git a/content/en/docs/reference/access-authn-authz/rbac.md b/content/en/docs/reference/access-authn-authz/rbac.md index a18903db4cb2e..2608ba32284f2 100644 --- a/content/en/docs/reference/access-authn-authz/rbac.md +++ b/content/en/docs/reference/access-authn-authz/rbac.md @@ -20,10 +20,22 @@ RBAC authorization uses the `rbac.authorization.k8s.io` decisions, allowing you to dynamically configure policies through the Kubernetes API. To enable RBAC, start the {{< glossary_tooltip text="API server" term_id="kube-apiserver" >}} -with the `--authorization-mode` flag set to a comma-separated list that includes `RBAC`; +with the `--authorization-config` flag set to a file that includes the `RBAC` authorizer; for example: + +```yaml +apiVersion: apiserver.config.k8s.io/v1 +kind: AuthorizationConfiguration +authorizers: + ... + - type: RBAC + ... +``` + +Or, start the {{< glossary_tooltip text="API server" term_id="kube-apiserver" >}} with +the `--authorization-mode` flag set to a comma-separated list that includes `RBAC`; for example: ```shell -kube-apiserver --authorization-mode=Example,RBAC --other-options --more-options +kube-apiserver --authorization-mode=...,RBAC --other-options --more-options ``` ## API objects {#api-overview} diff --git a/content/en/docs/reference/access-authn-authz/service-accounts-admin.md b/content/en/docs/reference/access-authn-authz/service-accounts-admin.md index 791654393f51e..3dc41cd3f9227 100644 --- a/content/en/docs/reference/access-authn-authz/service-accounts-admin.md +++ b/content/en/docs/reference/access-authn-authz/service-accounts-admin.md @@ -18,7 +18,8 @@ For an introduction to service accounts, read [configure service accounts](/docs This task guide explains some of the concepts behind ServiceAccounts. The guide also explains how to obtain or revoke tokens that represent -ServiceAccounts. +ServiceAccounts, and how to (optionally) bind a ServiceAccount's validity to +the lifetime of an API object. @@ -68,7 +69,7 @@ Supported object types are as follows: * Pod (used for projected volume mounts, see below) * Secret (can be used to allow revoking a token by deleting the Secret) -* Node (in v1.30, creating new node-bound tokens is alpha, using existing node-bound tokens is beta) +* Node (in v1.32, creating new node-bound tokens is beta, using existing node-bound tokens is GA) When a token is bound to an object, the object's `metadata.name` and `metadata.uid` are stored as extra 'private claims' in the issued JWT. @@ -159,6 +160,70 @@ does not actually persist the TokenReview object into etcd. Hence `kubectl get tokenreview` is not a valid command. {{< /note >}} +#### Schema for service account private claims + +The schema for the Kubernetes-specific claims within JWT tokens is not currently documented, +however the relevant code area can be found in +[the serviceaccount package](https://github.com/kubernetes/kubernetes/blob/d8919343526597e0788a1efe133c70d9a0c07f69/pkg/serviceaccount/claims.go#L56-L68) +in the Kubernetes codebase. + +You can inspect a JWT using standard JWT decoding tool. Below is an example of a JWT for the +`my-serviceaccount` ServiceAccount, bound to a Pod object named `my-pod` which is scheduled +to the Node `my-node`, in the `my-namespace` namespace: + +```json +{ + "aud": [ + "https://my-audience.example.com" + ], + "exp": 1729605240, + "iat": 1729601640, + "iss": "https://my-cluster.example.com", + "jti": "aed34954-b33a-4142-b1ec-389d6bbb4936", + "kubernetes.io": { + "namespace": "my-namespace", + "node": { + "name": "my-node", + "uid": "646e7c5e-32d6-4d42-9dbd-e504e6cbe6b1" + }, + "pod": { + "name": "my-pod", + "uid": "5e0bd49b-f040-43b0-99b7-22765a53f7f3" + }, + "serviceaccount": { + "name": "my-serviceaccount", + "uid": "14ee3fa4-a7e2-420f-9f9a-dbc4507c3798" + } + }, + "nbf": 1729601640, + "sub": "system:serviceaccount:my-namespace:my-serviceaccount" +} +``` + +{{< note >}} +The `aud` and `iss` fields in this JWT may differ between different Kubernetes clusters depending +on your configuration. + +The presence of both the `pod` and `node` claim implies that this token is bound +to a *Pod* object. When verifying Pod bound ServiceAccount tokens, the API server **does not** +verify the existence of the referenced Node object. +{{< /note >}} + +Services that run outside of Kubernetes and want to perform offline validation of JWTs may +use this schema, along with a compliant JWT validator configured with OpenID Discovery information +from the API server, to verify presented JWTs without requiring use of the TokenReview API. + +Services that verify JWTs in this way **do not verify** the claims embedded in the JWT token to be +current and still valid. +This means if the token is bound to an object, and that object no longer exists, the token will still +be considered valid (until the configured token expires). + +Clients that require assurance that a token's bound claims are still valid **MUST** use the TokenReview +API to present the token to the `kube-apiserver` for it to verify and expand the embedded claims, using +similar steps to the [Verifying and inspecting private claims](#verifying-and-inspecting-private-claims) +section above, but with a [supported client library](/docs/reference/using-api/client-libraries/). +For more information on JWTs and their structure, see the [JSON Web Token RFC](https://datatracker.ietf.org/doc/html/rfc7519). + ## Bound service account token volume mechanism {#bound-service-account-token-volume} {{< feature-state feature_gate_name="BoundServiceAccountTokenVolume" >}} @@ -319,6 +384,12 @@ Similarly, you must pass the corresponding public key to the `kube-apiserver` using the `--service-account-key-file` flag. The public key will be used to verify the tokens during authentication. +{{< feature-state feature_gate_name="ExternalServiceAccountTokenSigner" >}} + +An alternate setup to setting `--service-account-private-key-file` and `--service-account-key-file` flags is +to configure an external JWT signer for [external ServiceAccount token signing and key management](#external-serviceaccount-token-signing-and-key-management). +Note that these setups are mutually exclusive and cannot be configured together. + ### ServiceAccount admission controller The modification of pods is implemented via a plugin @@ -544,6 +615,22 @@ Then, delete the Secret you now know the name of: kubectl -n examplens delete secret/example-automated-thing-token-zyxwv ``` +## External ServiceAccount token signing and key management + +{{< feature-state feature_gate_name="ExternalServiceAccountTokenSigner" >}} + +The kube-apiserver can be configured to use external signer for token signing and token verifying key management. +This feature enables kubernetes distributions to integrate with key management solutions of their choice (eg: HSMs, cloud KMSes) for service account credential signing and verification. +To configure kube-apiserver to use external-jwt-signer set the `--service-account-signing-endpoint` flag to the location of a Unix domain socket (UDS) on a filesystem, or be prefixed with an @ symbol and name a UDS in the abstract socket namespace. +At the configured UDS, shall be an RPC server which implements [ExternalJWTSigner](https://github.com/kubernetes/kubernetes/blob/release-1.32/staging/src/k8s.io/externaljwt/apis/v1alpha1/api.proto). +The external-jwt-signer must be healthy and be ready to serve supported service account keys for the kube-apiserver to start. + +Check out [KEP-740](https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/740-service-account-external-signing) for more details on ExternalJWTSigner. + +{{< note >}} +The kube-apiserver flags `--service-account-key-file` and `--service-account-signing-key-file` will continue to be used for reading from files unless `--service-account-signing-endpoint` is set; they are mutually exclusive ways of supporting JWT signing and authentication. +{{< /note >}} + ## Clean up If you created a namespace `examplens` to experiment with, you can remove it: diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/allow-unsafe-malformed-object-deletion.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/allow-unsafe-malformed-object-deletion.md new file mode 100644 index 0000000000000..83ee02bb33abc --- /dev/null +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/allow-unsafe-malformed-object-deletion.md @@ -0,0 +1,16 @@ +--- +title: AllowUnsafeMalformedObjectDeletion +content_type: feature_gate +_build: + list: never + render: false + +stages: + - stage: alpha + defaultValue: false + fromVersion: "1.32" +--- +Enables the cluster operator to identify corrupt resource(s) using the **list** +operation, and introduces an option `ignoreStoreReadErrorWithClusterBreakingPotential` +that the operator can set to perform unsafe and force **delete** operation of +such corrupt resource(s) using the Kubernetes API. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/anonymous-auth-configurable-endpoints.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/anonymous-auth-configurable-endpoints.md index 38c4f6666debf..f731f04625a33 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/anonymous-auth-configurable-endpoints.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/anonymous-auth-configurable-endpoints.md @@ -9,6 +9,10 @@ stages: - stage: alpha defaultValue: false fromVersion: "1.31" - + toVersion: "1.31" + - stage: beta + defaultValue: true + fromVersion: "1.32" --- -Enable configuring anonymous authentication / authorization for only certain API server endpoints. +Enable [configurable endpoints for anonymous auth](/docs/reference/access-authn-authz/authentication/#anonymous-authenticator-configuration) +for the API server. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/authorize-node-with-selectors.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/authorize-node-with-selectors.md index 9c09c59f97290..db45f7ec27dbd 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/authorize-node-with-selectors.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/authorize-node-with-selectors.md @@ -9,6 +9,10 @@ stages: - stage: alpha defaultValue: false fromVersion: "1.31" + toVersion: "1.31" + - stage: beta + defaultValue: true + fromVersion: "1.32" --- Make the [Node authorizer](/docs/reference/access-authn-authz/node/) use fine-grained selector authorization. Requires `AuthorizeWithSelectors` to be enabled. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/authorize-with-selectors.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/authorize-with-selectors.md index 4626d486b174c..d53ff8d6305ea 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/authorize-with-selectors.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/authorize-with-selectors.md @@ -9,6 +9,10 @@ stages: - stage: alpha defaultValue: false fromVersion: "1.31" + toVersion: "1.31" + - stage: beta + defaultValue: true + fromVersion: "1.32" --- Allows authorization to use field and label selectors. Enables `fieldSelector` and `labelSelector` fields in the [SubjectAccessReview API](/docs/reference/kubernetes-api/authorization-resources/subject-access-review-v1/), diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/cbor-serving-and-storage.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/cbor-serving-and-storage.md new file mode 100644 index 0000000000000..ac3e76422a41e --- /dev/null +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/cbor-serving-and-storage.md @@ -0,0 +1,15 @@ +--- +title: CBORServingAndStorage +content_type: feature_gate +_build: + list: never + render: false + +stages: + - stage: alpha + defaultValue: false + fromVersion: "1.32" +--- +Enables CBOR as a [supported encoding for requests and +responses](/docs/reference/using-api/api-concepts/#cbor-encoding), and as the preferred storage +encoding for custom resources. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/cloud-dual-stack-node-ips.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/cloud-dual-stack-node-ips.md index 970fc6ae8e3d6..f36aab7d5179d 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/cloud-dual-stack-node-ips.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/cloud-dual-stack-node-ips.md @@ -17,7 +17,9 @@ stages: - stage: stable defaultValue: true fromVersion: "1.30" + toVersion: "1.31" +removed: true --- Enables dual-stack `kubelet --node-ip` with external cloud providers. See [Configure IPv4/IPv6 dual-stack](/docs/concepts/services-networking/dual-stack/#configure-ipv4-ipv6-dual-stack) diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/component-flagz.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/component-flagz.md new file mode 100644 index 0000000000000..07854760d78c6 --- /dev/null +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/component-flagz.md @@ -0,0 +1,14 @@ +--- +title: ComponentFlagz +content_type: feature_gate +_build: + list: never + render: false + +stages: + - stage: alpha + defaultValue: false + fromVersion: "1.32" +--- +Enables the component's flagz endpoint. +See [zpages](/docs/reference/instrumentation/zpages/) for more information. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/component-statusz.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/component-statusz.md new file mode 100644 index 0000000000000..60c15d36a6671 --- /dev/null +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/component-statusz.md @@ -0,0 +1,14 @@ +--- +title: ComponentStatusz +content_type: feature_gate +_build: + list: never + render: false + +stages: + - stage: alpha + defaultValue: false + fromVersion: "1.32" +--- +Enables the component's statusz endpoint. +See [zpages](/docs/reference/instrumentation/zpages/) for more information. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/cron-jobs-scheduled-annotation.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/cron-jobs-scheduled-annotation.md index db6c450923338..32ba03a320fec 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/cron-jobs-scheduled-annotation.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/cron-jobs-scheduled-annotation.md @@ -9,6 +9,11 @@ stages: - stage: beta defaultValue: true fromVersion: "1.28" + toVersion: "1.31" + - stage: stable + defaultValue: true + fromVersion: "1.32" + --- Set the scheduled job time as an {{< glossary_tooltip text="annotation" term_id="annotation" >}} on Jobs that were created diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/custom-resource-field-selectors.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/custom-resource-field-selectors.md index 77c579c748951..d6e261e0aef57 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/custom-resource-field-selectors.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/custom-resource-field-selectors.md @@ -12,7 +12,11 @@ stages: toVersion: "1.30" - stage: beta defaultValue: true - fromVersion: "1.31" + fromVersion: "1.31" + toVersion: "1.31" + - stage: stable + defaultValue: true + fromVersion: "1.32" --- Enable `selectableFields` in the diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/dra-admin-access.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/dra-admin-access.md new file mode 100644 index 0000000000000..7720c3786d955 --- /dev/null +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/dra-admin-access.md @@ -0,0 +1,18 @@ +--- +title: DRAAdminAccess +content_type: feature_gate +_build: + list: never + render: false + +stages: + - stage: alpha + defaultValue: false + fromVersion: "1.32" +--- +Enables support for requesting [admin access](/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#admin-access) +in a ResourceClaim. A ResourceClaim +with admin access grants access to devices which are in use and may enable +additional access permissions when making the device available in a container. + +This feature gate has no effect unless you also enable the `DynamicResourceAllocation` feature gate. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/dra-control-plane-controller.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/dra-control-plane-controller.md index b5af438f14578..9b31d8f1c7915 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/dra-control-plane-controller.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/dra-control-plane-controller.md @@ -9,6 +9,9 @@ stages: - stage: alpha defaultValue: false fromVersion: "1.26" + toVersion: "1.31" + +removed: true --- Enables support for resources with custom parameters and a lifecycle that is independent of a Pod. Allocation of resources is handled diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/dra-resource-claim-device-status.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/dra-resource-claim-device-status.md new file mode 100644 index 0000000000000..ac33e12a3a59d --- /dev/null +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/dra-resource-claim-device-status.md @@ -0,0 +1,14 @@ +--- +title: DRAResourceClaimDeviceStatus +content_type: feature_gate +_build: + list: never + render: false + +stages: + - stage: alpha + defaultValue: false + fromVersion: "1.32" +--- +Enables support the ResourceClaim.status.devices field and for setting this +status from DRA drivers. \ No newline at end of file diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/dynamic-resource-allocation.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/dynamic-resource-allocation.md index 142a0e8c45bb6..adaad7cf6ae85 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/dynamic-resource-allocation.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/dynamic-resource-allocation.md @@ -8,7 +8,16 @@ _build: stages: - stage: alpha defaultValue: false - fromVersion: "1.26" + fromVersion: "1.30" + toVersion: "1.31" + - stage: beta + defaultValue: false + fromVersion: "1.32" + +# TODO: as soon as this is locked to "true" (= GA), comments about other DRA +# feature gate(s) like "unless you also enable the `DynamicResourceAllocation` feature gate" +# can be removed (for example, in dra-admin-access.md). + --- Enables support for resources with custom parameters and a lifecycle that is independent of a Pod. Allocation of resources is handled diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/external-service-account-token-signer.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/external-service-account-token-signer.md new file mode 100644 index 0000000000000..23e5e2a673eee --- /dev/null +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/external-service-account-token-signer.md @@ -0,0 +1,13 @@ +--- +title: ExternalServiceAccountTokenSigner +content_type: feature_gate +_build: + list: never + render: false + +stages: + - stage: alpha + defaultValue: false + fromVersion: "1.32" +--- +Enable setting `--service-account-signing-endpoint` to make the kube-apiserver use [external signer](/docs/reference/access-authn-authz/service-account-admin#external-serviceaccount-token-signing-and-key-management) for token signing and token verifying key management. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/hpa-container-metrics.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/hpa-container-metrics.md index 84d076cb4b6a6..ad5b945b9927d 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/hpa-container-metrics.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/hpa-container-metrics.md @@ -17,6 +17,9 @@ stages: - stage: stable defaultValue: true fromVersion: "1.30" + toVersion: "1.31" + +removed: true --- Allow {{< glossary_tooltip text="HorizontalPodAutoscalers" term_id="horizontal-pod-autoscaler" >}} to scale based on metrics from individual containers within target pods. \ No newline at end of file diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/job-managed-by.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/job-managed-by.md index 38733b6de66ff..cbf74738143a3 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/job-managed-by.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/job-managed-by.md @@ -10,5 +10,9 @@ stages: - stage: alpha defaultValue: false fromVersion: "1.30" + toVersion: "1.31" + - stage: beta + defaultValue: false + fromVersion: "1.32" --- Allows to delegate reconciliation of a Job object to an external controller. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/kmsv2-kdf.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/kmsv2-kdf.md index 213c9a664cb4d..96add97b1fac2 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/kmsv2-kdf.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/kmsv2-kdf.md @@ -12,7 +12,10 @@ stages: toVersion: "1.28" - stage: stable defaultValue: true - fromVersion: "1.29" + fromVersion: "1.29" + toVersion: "1.31" + +removed: true --- Enables KMS v2 to generate single use data encryption keys. See [Using a KMS Provider for data encryption](/docs/tasks/administer-cluster/kms-provider) for more details. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/kmsv2.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/kmsv2.md index 52ce6b7b6ab5e..d7890e0c2c3f3 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/kmsv2.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/kmsv2.md @@ -16,6 +16,9 @@ stages: toVersion: "1.28" - stage: stable defaultValue: true - fromVersion: "1.29" + fromVersion: "1.29" + toVersion: "1.31" + +removed: true --- Enables KMS v2 API for encryption at rest. See [Using a KMS Provider for data encryption](/docs/tasks/administer-cluster/kms-provider) for more details. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/kubelet-crash-loop-back-off-max.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/kubelet-crash-loop-back-off-max.md new file mode 100644 index 0000000000000..36d3493f1f396 --- /dev/null +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/kubelet-crash-loop-back-off-max.md @@ -0,0 +1,14 @@ +--- +title: KubeletCrashLoopBackOffMax +content_type: feature_gate +_build: + list: never + render: false + +stages: + - stage: alpha + defaultValue: false + fromVersion: "1.32" +--- +Enables support for configurable per-node backoff maximums for restarting +containers in the CrashLoopBackOff state. \ No newline at end of file diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/kubelet-finegrained-authz.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/kubelet-finegrained-authz.md new file mode 100644 index 0000000000000..0fb0afb8b4bdb --- /dev/null +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/kubelet-finegrained-authz.md @@ -0,0 +1,14 @@ +--- +title: KubeletFineGrainedAuthz +content_type: feature_gate +_build: + list: never + render: false + +stages: + - stage: alpha + defaultValue: false + fromVersion: "1.32" +--- +Enable [fine-grained authorization](/docs/reference/access-authn-authz/kubelet-authn-authz/#fine-grained-authorization) +for the kubelet's HTTP(s) API. \ No newline at end of file diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/legacy-service-account-token-clean-up.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/legacy-service-account-token-clean-up.md index f22aaae479dff..9adcaa0e74d57 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/legacy-service-account-token-clean-up.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/legacy-service-account-token-clean-up.md @@ -17,6 +17,9 @@ stages: - stage: stable defaultValue: true fromVersion: "1.30" + toVersion: "1.31" + +removed: true --- Enable cleaning up Secret-based [service account tokens](/docs/concepts/security/service-accounts/#get-a-token) diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/load-balancer-ip-mode.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/load-balancer-ip-mode.md index 6b87fd3abff38..9b692c6668ec0 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/load-balancer-ip-mode.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/load-balancer-ip-mode.md @@ -13,6 +13,10 @@ stages: - stage: beta defaultValue: true fromVersion: "1.30" + toVersion: "1.31" + - stage: stable + defaultValue: true + fromVersion: "1.32" --- Allows setting `ipMode` for Services where `type` is set to `LoadBalancer`. See [Specifying IPMode of load balancer status](/docs/concepts/services-networking/service/#load-balancer-ip-mode) diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/memory-manager.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/memory-manager.md index e9db5c4021155..87b2206e8c82d 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/memory-manager.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/memory-manager.md @@ -13,6 +13,10 @@ stages: - stage: beta defaultValue: true fromVersion: "1.22" + toVersion: "1.31" + - stage: stable + defaultValue: true + fromVersion: "1.32" --- Allows setting memory affinity for a container based on NUMA topology. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/min-domains-in-pod-topology-spread.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/min-domains-in-pod-topology-spread.md index ae8a3f7f383ad..4a09546d540a9 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/min-domains-in-pod-topology-spread.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/min-domains-in-pod-topology-spread.md @@ -21,6 +21,9 @@ stages: - stage: stable defaultValue: true fromVersion: "1.30" + toVersion: "1.31" + +removed: true --- Enable `minDomains` in [Pod topology spread constraints](/docs/concepts/scheduling-eviction/topology-spread-constraints/). diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/mutating-admission-policy.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/mutating-admission-policy.md index e25313e80a7e0..4da5395b3296c 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/mutating-admission-policy.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/mutating-admission-policy.md @@ -10,7 +10,7 @@ stages: defaultValue: false fromVersion: "1.30" --- -In Kubernetes {{< skew currentVersion >}}, this feature gate has no effect. -A future release of Kubernetes may use this feature gate to enable -the MutatingAdmissionPolicy in admission chain. +Enable [MutatingAdmissionPolicy](/docs/reference/access-authn-authz/mutating-admission-policy/) support for [CEL](https://kubernetes.io/docs/reference/using-api/cel/) mutations be used in admission control. + +For Kubernetes v1.30 and v1.31, this feature gate existed but had no effect. \ No newline at end of file diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/new-volume-manager-reconstruction.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/new-volume-manager-reconstruction.md index 6756213d92eb6..ff2a4a1d5f90d 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/new-volume-manager-reconstruction.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/new-volume-manager-reconstruction.md @@ -17,6 +17,9 @@ stages: - stage: stable defaultValue: true fromVersion: "1.30" + toVersion: "1.31" + +removed: true --- Enables improved discovery of mounted volumes during kubelet startup. Since the associated code had been significantly refactored, Kubernetes versions 1.25 to 1.29 diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/node-out-of-service-volume-detach.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/node-out-of-service-volume-detach.md index e3f0998d80799..ad5d5de72a3b3 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/node-out-of-service-volume-detach.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/node-out-of-service-volume-detach.md @@ -17,6 +17,9 @@ stages: - stage: stable defaultValue: true fromVersion: "1.28" + toVersion: "1.31" + +removed: true --- When a Node is marked out-of-service using the `node.kubernetes.io/out-of-service` taint, Pods on the node will be forcefully deleted diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/pod-host-ips.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/pod-host-ips.md index 0f39a10790f3c..c6ddd0084b1ee 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/pod-host-ips.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/pod-host-ips.md @@ -17,6 +17,9 @@ stages: - stage: stable defaultValue: true fromVersion: "1.30" + toVersion: "1.31" +removed: true + --- Enable the `status.hostIPs` field for pods and the {{< glossary_tooltip term_id="downward-api" text="downward API" >}}. The field lets you expose host IP addresses to workloads. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/pod-index-label.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/pod-index-label.md index be509292c662d..40118ee7499e9 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/pod-index-label.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/pod-index-label.md @@ -9,5 +9,9 @@ stages: - stage: beta defaultValue: true fromVersion: "1.28" + toVersion: "1.31" + - stage: stable + defaultValue: true + fromVersion: "1.32" --- Enables the Job controller and StatefulSet controller to add the pod index as a label when creating new pods. See [Job completion mode docs](/docs/concepts/workloads/controllers/job#completion-mode) and [StatefulSet pod index label docs](/docs/concepts/workloads/controllers/statefulset/#pod-index-label) for more details. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/pod-level-resources.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/pod-level-resources.md new file mode 100644 index 0000000000000..80cbc1027de4c --- /dev/null +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/pod-level-resources.md @@ -0,0 +1,14 @@ +--- +title: PodLevelResources +content_type: feature_gate +_build: + list: never + render: false + +stages: + - stage: alpha + defaultValue: false + fromVersion: "1.32" +--- +Enable _Pod level resources_: the ability to specify resource requests and limits +at the Pod level, rather than only for specific containers. \ No newline at end of file diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/pod-lifecycle-sleep-action-allow-zero.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/pod-lifecycle-sleep-action-allow-zero.md new file mode 100644 index 0000000000000..831f9555c1fc3 --- /dev/null +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/pod-lifecycle-sleep-action-allow-zero.md @@ -0,0 +1,13 @@ +--- +title: PodLifecycleSleepActionAllowZero +content_type: feature_gate +_build: + list: never + render: false + +stages: + - stage: alpha + defaultValue: false + fromVersion: "1.32" +--- +Enables setting zero value for the `sleep` action in [container lifecycle hooks](/docs/concepts/containers/container-lifecycle-hooks/). diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/pod-logs-query-split-streams.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/pod-logs-query-split-streams.md new file mode 100644 index 0000000000000..ba7a066d0a7b5 --- /dev/null +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/pod-logs-query-split-streams.md @@ -0,0 +1,13 @@ +--- +title: PodLogsQuerySplitStreams +content_type: feature_gate +_build: + list: never + render: false + +stages: + - stage: alpha + defaultValue: false + fromVersion: "1.32" +--- +Enable fetching specific log streams (either stdout or stderr) from a container's log streams, using the Pod API. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/prefer-align-cpus-by-uncorecache.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/prefer-align-cpus-by-uncorecache.md new file mode 100644 index 0000000000000..9914843dd8b05 --- /dev/null +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/prefer-align-cpus-by-uncorecache.md @@ -0,0 +1,13 @@ +--- +title: PreferAlignCpusByUncoreCache +content_type: feature_gate +_build: + list: never + render: false + +stages: + - stage: alpha + defaultValue: false + fromVersion: "1.32" +--- +When `PreferAlignCpusByUncoreCache` is enabled while the CPU Manager Policy is set to `static`, containers within a `Guaranteed` pod will individually be aligned to an uncore cache group at a best-effort policy. This feature can optimize performance for certain cache-sensitive workloads by minimizing the cpu allocation across uncore caches. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/recover-volume-expansion-failure.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/recover-volume-expansion-failure.md index 5c5afb86939cb..5184f9f3386f0 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/recover-volume-expansion-failure.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/recover-volume-expansion-failure.md @@ -9,6 +9,10 @@ stages: - stage: alpha defaultValue: false fromVersion: "1.23" + toVersion: "1.31" + - stage: beta + defaultValue: true + fromVersion: "1.32" --- Enables users to edit their PVCs to smaller sizes so as they can recover from previously issued volume expansion failures. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/relaxed-environment-variable-validation.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/relaxed-environment-variable-validation.md index 862ae57214bae..870553d0f9cfa 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/relaxed-environment-variable-validation.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/relaxed-environment-variable-validation.md @@ -9,5 +9,9 @@ stages: - stage: alpha defaultValue: false fromVersion: "1.30" + toVersion: "1.31" + - stage: beta + defaultValue: true + fromVersion: "1.32" --- Allow almost all printable ASCII characters in environment variables. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/retry-generate-name.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/retry-generate-name.md index 838951950bbb1..dd5a45854b08a 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/retry-generate-name.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/retry-generate-name.md @@ -13,7 +13,10 @@ stages: - stage: beta defaultValue: true fromVersion: "1.31" - + toVersion: "1.31" + - stage: stable + defaultValue: true + fromVersion: "1.32" --- Enables retrying of object creation when the {{< glossary_tooltip text="API server" term_id="kube-apiserver" >}} diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/scheduler-async-preemption.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/scheduler-async-preemption.md new file mode 100644 index 0000000000000..7c39bd39c0e80 --- /dev/null +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/scheduler-async-preemption.md @@ -0,0 +1,16 @@ +--- +title: SchedulerAsyncPreemption +content_type: feature_gate +_build: + list: never + render: false + +stages: + - stage: alpha + defaultValue: false + fromVersion: "1.32" +--- + +Enable running some expensive operations within the scheduler, associated with +[preemption](/docs/concepts/scheduling-eviction/pod-priority-preemption/), asynchronously. +Asynchronous processing of preemption improves overall Pod scheduling latency. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/scheduler-queueing-hints.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/scheduler-queueing-hints.md index b60c4ddc506c9..dba2cb95a9a40 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/scheduler-queueing-hints.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/scheduler-queueing-hints.md @@ -13,8 +13,12 @@ stages: - stage: beta defaultValue: false fromVersion: "1.29" + toVersion: "1.31" + - stage: beta + defaultValue: true + fromVersion: "1.32" --- -Enables [the scheduler's _queueing hints_ enhancement](https://github.com/kubernetes/enhancements/blob/master/keps/sig-scheduling/4247-queueinghint/README.md), +Enables [the scheduler's _queueing hints_ feature](/docs/concepts/scheduling-eviction/scheduling-framework/#queueinghint), which benefits to reduce the useless requeueing. The scheduler retries scheduling pods if something changes in the cluster that could make the pod scheduled. Queueing hints are internal signals that allow the scheduler to filter the changes in the cluster diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/selinux-change-policy.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/selinux-change-policy.md new file mode 100644 index 0000000000000..ed2e3d4e11cdc --- /dev/null +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/selinux-change-policy.md @@ -0,0 +1,20 @@ +--- +title: SELinuxChangePolicy +content_type: feature_gate +_build: + list: never + render: false + +stages: + - stage: alpha + defaultValue: false + fromVersion: "1.32" +--- +Enables `spec.securityContext.seLinuxChangePolicy` field. +This field can be used to opt-out from applying the SELinux label to the pod +volumes using mount options. This is required when a single volume that supports +mounting with SELinux mount option is shared between Pods that have different +SELinux labels, such as a privileged and unprivileged Pods. + +Enabling the `SELinuxChangePolicy` feature gate requires the feature gate `SELinuxMountReadWriteOncePod` to +be enabled. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/selinux-mount.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/selinux-mount.md index 124862976773c..6e786d50d8a9c 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/selinux-mount.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/selinux-mount.md @@ -16,5 +16,5 @@ recursively. It widens the performance improvements behind the `SELinuxMountReadWriteOncePod` feature gate by extending the implementation to all volumes. -Enabling the `SELinuxMount` feature gate requires the feature gate `SELinuxMountReadWriteOncePod` to -be enabled. +Enabling the `SELinuxMount` feature gate requires the feature gates `SELinuxMountReadWriteOncePod` +and `SELinuxChangePolicy` to be enabled. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/server-side-apply.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/server-side-apply.md index 8decb8b3c69a2..edeb86d7a4792 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/server-side-apply.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/server-side-apply.md @@ -17,6 +17,9 @@ stages: - stage: stable defaultValue: true fromVersion: "1.22" + toVersion: "1.31" + +removed: true --- Enables the [Sever Side Apply (SSA)](/docs/reference/using-api/server-side-apply/) feature on the API Server. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/server-side-field-validation.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/server-side-field-validation.md index 91849ecb76b49..3399e3e1af917 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/server-side-field-validation.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/server-side-field-validation.md @@ -17,6 +17,9 @@ stages: - stage: stable defaultValue: true fromVersion: "1.27" + toVersion: "1.31" + +removed: true --- Enables server-side field validation. This means the validation of resource schema is performed at the API server side rather than the client side diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/service-account-token-jti.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/service-account-token-jti.md index ab82953ada6da..3e63377aa1520 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/service-account-token-jti.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/service-account-token-jti.md @@ -13,6 +13,10 @@ stages: - stage: beta defaultValue: true fromVersion: "1.30" + toVersion: "1.31" + - stage: stable + defaultValue: true + fromVersion: "1.32" --- Controls whether JTIs (UUIDs) are embedded into generated service account tokens, and whether these JTIs are recorded into the Kubernetes audit log for future requests made by these tokens. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/service-account-token-node-binding-validation.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/service-account-token-node-binding-validation.md index 94021587aef52..023556884e344 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/service-account-token-node-binding-validation.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/service-account-token-node-binding-validation.md @@ -13,6 +13,10 @@ stages: - stage: beta defaultValue: true fromVersion: "1.30" + toVersion: "1.31" + - stage: stable + defaultValue: true + fromVersion: "1.32" --- Controls whether the apiserver will validate a Node reference in service account tokens. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/service-account-token-pod-node-info.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/service-account-token-pod-node-info.md index 86d8940b55ec2..624fd96242ac3 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/service-account-token-pod-node-info.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/service-account-token-pod-node-info.md @@ -13,6 +13,10 @@ stages: - stage: beta defaultValue: true fromVersion: "1.30" + toVersion: "1.31" + - stage: stable + defaultValue: true + fromVersion: "1.32" --- Controls whether the apiserver embeds the node name and uid for the associated node when issuing service account tokens bound to Pod objects. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/size-memory-backed-volumes.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/size-memory-backed-volumes.md index ea6ea9a4c5cce..69162233c6014 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/size-memory-backed-volumes.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/size-memory-backed-volumes.md @@ -13,6 +13,10 @@ stages: - stage: beta defaultValue: true fromVersion: "1.22" + toVersion: "1.31" + - stage: stable + defaultValue: true + fromVersion: "1.32" --- Enable kubelets to determine the size limit for memory-backed volumes (mainly `emptyDir` volumes). diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/stable-load-balancer-node-set.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/stable-load-balancer-node-set.md index e2968c340482d..943d90606cf86 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/stable-load-balancer-node-set.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/stable-load-balancer-node-set.md @@ -13,6 +13,9 @@ stages: - stage: stable defaultValue: true fromVersion: "1.30" + toVersion: "1.31" + +removed: true --- Enables less load balancer re-configurations by the service controller (KCCM) as an effect of changing node state. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/stateful-set-auto-delete-pvc.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/stateful-set-auto-delete-pvc.md index 99967d2059258..b04022452ea5d 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/stateful-set-auto-delete-pvc.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/stateful-set-auto-delete-pvc.md @@ -14,9 +14,13 @@ stages: - stage: beta defaultValue: true fromVersion: "1.27" + toVersion: "1.31" + - stage: stable + defaultValue: true + fromVersion: "1.32" --- Allows the use of the optional `.spec.persistentVolumeClaimRetentionPolicy` field, providing control over the deletion of PVCs in a StatefulSet's lifecycle. See [PersistentVolumeClaim retention](/docs/concepts/workloads/controllers/statefulset/#persistentvolumeclaim-retention) -for more details. \ No newline at end of file +for more details. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/structured-authorization-configuration.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/structured-authorization-configuration.md index d2f1a47283c6a..b40c8193298d0 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/structured-authorization-configuration.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/structured-authorization-configuration.md @@ -6,13 +6,17 @@ _build: render: false stages: - - stage: alpha + - stage: alpha defaultValue: false fromVersion: "1.29" toVersion: "1.29" - - stage: beta + - stage: beta defaultValue: true fromVersion: "1.30" + toVersion: "1.31" + - stage: stable + defaultValue: true + fromVersion: "1.32" --- Enable structured authorization configuration, so that cluster administrators can specify more than one [authorization webhook](/docs/reference/access-authn-authz/webhook/) diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/systemd-watchdog.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/systemd-watchdog.md new file mode 100644 index 0000000000000..830720dcca33b --- /dev/null +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/systemd-watchdog.md @@ -0,0 +1,15 @@ +--- +title: SystemdWatchdog +content_type: feature_gate +_build: + list: never + render: false + +stages: + - stage: beta + defaultValue: true + fromVersion: "1.32" +--- +Allow using systemd watchdog to monitor the health status of kubelet. +See [Kubelet Systemd Watchdog](/docs/reference/node/systemd-watchdog/) +for more details. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/topology-manager-policy-options.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/topology-manager-policy-options.md index 6a59d8f40bd4e..a2cbab3f385c4 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/topology-manager-policy-options.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/topology-manager-policy-options.md @@ -6,13 +6,17 @@ _build: render: false stages: - - stage: alpha + - stage: alpha defaultValue: false fromVersion: "1.26" toVersion: "1.27" - stage: beta defaultValue: true fromVersion: "1.28" + toVersion: "1.31" + - stage: stable + defaultValue: true + fromVersion: "1.32" --- Enable [fine-tuning](/docs/tasks/administer-cluster/topology-manager/#topology-manager-policy-options) -of topology manager policies. +of topology manager policies. \ No newline at end of file diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/watch-list-client.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/watch-list-client.md new file mode 100644 index 0000000000000..826e19712c81f --- /dev/null +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/watch-list-client.md @@ -0,0 +1,17 @@ +--- +title: WatchListClient +content_type: feature_gate +_build: + list: never + render: false + +stages: + - stage: beta + defaultValue: false + fromVersion: "1.30" +--- +Allows an API client to request a stream of data rather than fetching a full list. +This functionality is available in `client-go` and requires the +[WatchList](/docs/reference/command-line-tools-reference/feature-gates/) +feature to be enabled on the server. +If the `WatchList` is not supported on the server, the client will seamlessly fall back to a standard list request. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/watch-list.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/watch-list.md index cd258d1afd54a..1595cd251dce2 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/watch-list.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/watch-list.md @@ -9,5 +9,9 @@ stages: - stage: alpha defaultValue: false fromVersion: "1.27" + toVersion: "1.31" + - stage: beta + defaultValue: true + fromVersion: "1.32" --- Enable support for [streaming initial state of objects in watch requests](/docs/reference/using-api/api-concepts/#streaming-lists). diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/windows-cpu-and-memory-affinity.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/windows-cpu-and-memory-affinity.md new file mode 100644 index 0000000000000..00dfa1b4c4a3a --- /dev/null +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/windows-cpu-and-memory-affinity.md @@ -0,0 +1,17 @@ +--- +title: WindowsCPUAndMemoryAffinity +content_type: feature_gate + +_build: + list: never + render: false + +stages: + - stage: alpha + defaultValue: false + fromVersion: "1.32" +--- + +Add CPU and Memory Affinity support to Windows nodes with [CPUManager](/docs/tasks/administer-cluster/cpu-management-policies/#windows-support), +[MemoryManager](/docs/tasks/administer-cluster/memory-manager/#windows-support) +and topology manager. \ No newline at end of file diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/windows-graceful-node-shutdown.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/windows-graceful-node-shutdown.md new file mode 100644 index 0000000000000..278b5926ddb1e --- /dev/null +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/windows-graceful-node-shutdown.md @@ -0,0 +1,18 @@ +--- +title: WindowsGracefulNodeShutdown +content_type: feature_gate +_build: + list: never + render: false + +stages: + - stage: alpha + defaultValue: false + fromVersion: "1.32" + +--- +Enables support for windows node graceful shutdown in kubelet. +During a system shutdown, kubelet will attempt to detect the shutdown event +and gracefully terminate pods running on the node. See +[Graceful Node Shutdown](/docs/concepts/architecture/nodes/#graceful-node-shutdown) +for more details. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/zero-limited-nominal-concurrency-shares.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/zero-limited-nominal-concurrency-shares.md index c6fb5e8635090..3dcffd4521cdf 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates/zero-limited-nominal-concurrency-shares.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/zero-limited-nominal-concurrency-shares.md @@ -13,6 +13,9 @@ stages: - stage: stable defaultValue: true fromVersion: "1.30" + toVersion: "1.31" + +removed: true --- Allow [priority & fairness](/docs/concepts/cluster-administration/flow-control/) in the API server to use a zero value for the `nominalConcurrencyShares` field of diff --git a/content/en/docs/reference/command-line-tools-reference/kube-apiserver.md b/content/en/docs/reference/command-line-tools-reference/kube-apiserver.md index a283e8c6a40c3..f0869a3ad3ce0 100644 --- a/content/en/docs/reference/command-line-tools-reference/kube-apiserver.md +++ b/content/en/docs/reference/command-line-tools-reference/kube-apiserver.md @@ -316,7 +316,7 @@ kube-apiserver [flags] --authentication-config string -

File with Authentication Configuration to configure the JWT Token authenticator or the anonymous authenticator. Note: This feature is in Alpha since v1.29.--feature-gate=StructuredAuthenticationConfiguration=true needs to be set for enabling this feature.This feature is mutually exclusive with the oidc-* flags.To configure anonymous authenticator you need to enable --feature-gate=AnonymousAuthConfigurableEndpoints.When you configure anonymous authenticator in the authentication config you cannot use the --anonymous-auth flag.

+

File with Authentication Configuration to configure the JWT Token authenticator or the anonymous authenticator. Requires the StructuredAuthenticationConfiguration feature gate. Also requires the feature gate AnonymousAuthConfigurableEndpoints to configure the anonymous authenticator in the config file. This flag is mutually exclusive with the --oidc-* flags if the file configures the JWT Token authenticator. This flag is mutually exclusive with --anonymous-auth if the file configures the Anonymous authenticator.

@@ -344,7 +344,7 @@ kube-apiserver [flags] --authorization-config string -

File with Authorization Configuration to configure the authorizer chain.Note: This feature is in Alpha since v1.29.--feature-gate=StructuredAuthorizationConfiguration=true feature flag needs to be set to true for enabling the functionality.This feature is mutually exclusive with the other --authorization-mode and --authorization-webhook-* flags.

+

File with Authorization Configuration to configure the authorizer chain. Requires feature gate StructuredAuthorizationConfiguration. This flag is mutually exclusive with the other --authorization-mode and --authorization-webhook-* flags.

@@ -456,7 +456,7 @@ kube-apiserver [flags] --disable-admission-plugins strings -

admission plugins that should be disabled although they are in the default enabled plugins list (NamespaceLifecycle, LimitRanger, ServiceAccount, TaintNodesByCondition, PodSecurity, Priority, DefaultTolerationSeconds, DefaultStorageClass, StorageObjectInUseProtection, PersistentVolumeClaimResize, RuntimeClass, CertificateApproval, CertificateSigning, ClusterTrustBundleAttest, CertificateSubjectRestriction, DefaultIngressClass, MutatingAdmissionWebhook, ValidatingAdmissionPolicy, ValidatingAdmissionWebhook, ResourceQuota). Comma-delimited list of admission plugins: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, ClusterTrustBundleAttest, DefaultIngressClass, DefaultStorageClass, DefaultTolerationSeconds, DenyServiceExternalIPs, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PodNodeSelector, PodSecurity, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionPolicy, ValidatingAdmissionWebhook. The order of plugins in this flag does not matter.

+

admission plugins that should be disabled although they are in the default enabled plugins list (NamespaceLifecycle, LimitRanger, ServiceAccount, TaintNodesByCondition, PodSecurity, Priority, DefaultTolerationSeconds, DefaultStorageClass, StorageObjectInUseProtection, PersistentVolumeClaimResize, RuntimeClass, CertificateApproval, CertificateSigning, ClusterTrustBundleAttest, CertificateSubjectRestriction, DefaultIngressClass, MutatingAdmissionPolicy, MutatingAdmissionWebhook, ValidatingAdmissionPolicy, ValidatingAdmissionWebhook, ResourceQuota). Comma-delimited list of admission plugins: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, ClusterTrustBundleAttest, DefaultIngressClass, DefaultStorageClass, DefaultTolerationSeconds, DenyServiceExternalIPs, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionPolicy, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PodNodeSelector, PodSecurity, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionPolicy, ValidatingAdmissionWebhook. The order of plugins in this flag does not matter.

@@ -484,14 +484,14 @@ kube-apiserver [flags] --emulated-version strings -

The versions different components emulate their capabilities (APIs, features, ...) of.
If set, the component will emulate the behavior of this version instead of the underlying binary version.
Version format could only be major.minor, for example: '--emulated-version=wardle=1.2,kube=1.31'. Options are:
kube=1.31..1.31 (default=1.31)If the component is not specified, defaults to "kube"

+

The versions different components emulate their capabilities (APIs, features, ...) of.
If set, the component will emulate the behavior of this version instead of the underlying binary version.
Version format could only be major.minor, for example: '--emulated-version=wardle=1.2,kube=1.31'. Options are:
kube=1.32..1.32 (default=1.32)If the component is not specified, defaults to "kube"

--enable-admission-plugins strings -

admission plugins that should be enabled in addition to default enabled ones (NamespaceLifecycle, LimitRanger, ServiceAccount, TaintNodesByCondition, PodSecurity, Priority, DefaultTolerationSeconds, DefaultStorageClass, StorageObjectInUseProtection, PersistentVolumeClaimResize, RuntimeClass, CertificateApproval, CertificateSigning, ClusterTrustBundleAttest, CertificateSubjectRestriction, DefaultIngressClass, MutatingAdmissionWebhook, ValidatingAdmissionPolicy, ValidatingAdmissionWebhook, ResourceQuota). Comma-delimited list of admission plugins: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, ClusterTrustBundleAttest, DefaultIngressClass, DefaultStorageClass, DefaultTolerationSeconds, DenyServiceExternalIPs, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PodNodeSelector, PodSecurity, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionPolicy, ValidatingAdmissionWebhook. The order of plugins in this flag does not matter.

+

admission plugins that should be enabled in addition to default enabled ones (NamespaceLifecycle, LimitRanger, ServiceAccount, TaintNodesByCondition, PodSecurity, Priority, DefaultTolerationSeconds, DefaultStorageClass, StorageObjectInUseProtection, PersistentVolumeClaimResize, RuntimeClass, CertificateApproval, CertificateSigning, ClusterTrustBundleAttest, CertificateSubjectRestriction, DefaultIngressClass, MutatingAdmissionPolicy, MutatingAdmissionWebhook, ValidatingAdmissionPolicy, ValidatingAdmissionWebhook, ResourceQuota). Comma-delimited list of admission plugins: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, ClusterTrustBundleAttest, DefaultIngressClass, DefaultStorageClass, DefaultTolerationSeconds, DenyServiceExternalIPs, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionPolicy, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PodNodeSelector, PodSecurity, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionPolicy, ValidatingAdmissionWebhook. The order of plugins in this flag does not matter.

@@ -638,7 +638,7 @@ kube-apiserver [flags] --feature-gates colonSeparatedMultimapStringString -

Comma-separated list of component:key=value pairs that describe feature gates for alpha/experimental features of different components.
If the component is not specified, defaults to "kube". This flag can be repeatedly invoked. For example: --feature-gates 'wardle:featureA=true,wardle:featureB=false' --feature-gates 'kube:featureC=true'Options are:
kube:APIResponseCompression=true|false (BETA - default=true)
kube:APIServerIdentity=true|false (BETA - default=true)
kube:APIServerTracing=true|false (BETA - default=true)
kube:APIServingWithRoutine=true|false (ALPHA - default=false)
kube:AllAlpha=true|false (ALPHA - default=false)
kube:AllBeta=true|false (BETA - default=false)
kube:AnonymousAuthConfigurableEndpoints=true|false (ALPHA - default=false)
kube:AnyVolumeDataSource=true|false (BETA - default=true)
kube:AuthorizeNodeWithSelectors=true|false (ALPHA - default=false)
kube:AuthorizeWithSelectors=true|false (ALPHA - default=false)
kube:CPUManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
kube:CPUManagerPolicyBetaOptions=true|false (BETA - default=true)
kube:CPUManagerPolicyOptions=true|false (BETA - default=true)
kube:CRDValidationRatcheting=true|false (BETA - default=true)
kube:CSIMigrationPortworx=true|false (BETA - default=true)
kube:CSIVolumeHealth=true|false (ALPHA - default=false)
kube:CloudControllerManagerWebhook=true|false (ALPHA - default=false)
kube:ClusterTrustBundle=true|false (ALPHA - default=false)
kube:ClusterTrustBundleProjection=true|false (ALPHA - default=false)
kube:ComponentSLIs=true|false (BETA - default=true)
kube:ConcurrentWatchObjectDecode=true|false (BETA - default=false)
kube:ConsistentListFromCache=true|false (BETA - default=true)
kube:ContainerCheckpoint=true|false (BETA - default=true)
kube:ContextualLogging=true|false (BETA - default=true)
kube:CoordinatedLeaderElection=true|false (ALPHA - default=false)
kube:CronJobsScheduledAnnotation=true|false (BETA - default=true)
kube:CrossNamespaceVolumeDataSource=true|false (ALPHA - default=false)
kube:CustomCPUCFSQuotaPeriod=true|false (ALPHA - default=false)
kube:CustomResourceFieldSelectors=true|false (BETA - default=true)
kube:DRAControlPlaneController=true|false (ALPHA - default=false)
kube:DisableAllocatorDualWrite=true|false (ALPHA - default=false)
kube:DisableNodeKubeProxyVersion=true|false (BETA - default=true)
kube:DynamicResourceAllocation=true|false (ALPHA - default=false)
kube:EventedPLEG=true|false (ALPHA - default=false)
kube:GracefulNodeShutdown=true|false (BETA - default=true)
kube:GracefulNodeShutdownBasedOnPodPriority=true|false (BETA - default=true)
kube:HPAScaleToZero=true|false (ALPHA - default=false)
kube:HonorPVReclaimPolicy=true|false (BETA - default=true)
kube:ImageMaximumGCAge=true|false (BETA - default=true)
kube:ImageVolume=true|false (ALPHA - default=false)
kube:InPlacePodVerticalScaling=true|false (ALPHA - default=false)
kube:InTreePluginPortworxUnregister=true|false (ALPHA - default=false)
kube:InformerResourceVersion=true|false (ALPHA - default=false)
kube:JobBackoffLimitPerIndex=true|false (BETA - default=true)
kube:JobManagedBy=true|false (ALPHA - default=false)
kube:JobPodReplacementPolicy=true|false (BETA - default=true)
kube:JobSuccessPolicy=true|false (BETA - default=true)
kube:KubeletCgroupDriverFromCRI=true|false (BETA - default=true)
kube:KubeletInUserNamespace=true|false (ALPHA - default=false)
kube:KubeletPodResourcesDynamicResources=true|false (ALPHA - default=false)
kube:KubeletPodResourcesGet=true|false (ALPHA - default=false)
kube:KubeletSeparateDiskGC=true|false (BETA - default=true)
kube:KubeletTracing=true|false (BETA - default=true)
kube:LoadBalancerIPMode=true|false (BETA - default=true)
kube:LocalStorageCapacityIsolationFSQuotaMonitoring=true|false (BETA - default=false)
kube:LoggingAlphaOptions=true|false (ALPHA - default=false)
kube:LoggingBetaOptions=true|false (BETA - default=true)
kube:MatchLabelKeysInPodAffinity=true|false (BETA - default=true)
kube:MatchLabelKeysInPodTopologySpread=true|false (BETA - default=true)
kube:MaxUnavailableStatefulSet=true|false (ALPHA - default=false)
kube:MemoryManager=true|false (BETA - default=true)
kube:MemoryQoS=true|false (ALPHA - default=false)
kube:MultiCIDRServiceAllocator=true|false (BETA - default=false)
kube:MutatingAdmissionPolicy=true|false (ALPHA - default=false)
kube:NFTablesProxyMode=true|false (BETA - default=true)
kube:NodeInclusionPolicyInPodTopologySpread=true|false (BETA - default=true)
kube:NodeLogQuery=true|false (BETA - default=false)
kube:NodeSwap=true|false (BETA - default=true)
kube:OpenAPIEnums=true|false (BETA - default=true)
kube:PodAndContainerStatsFromCRI=true|false (ALPHA - default=false)
kube:PodDeletionCost=true|false (BETA - default=true)
kube:PodIndexLabel=true|false (BETA - default=true)
kube:PodLifecycleSleepAction=true|false (BETA - default=true)
kube:PodReadyToStartContainersCondition=true|false (BETA - default=true)
kube:PortForwardWebsockets=true|false (BETA - default=true)
kube:ProcMountType=true|false (BETA - default=false)
kube:QOSReserved=true|false (ALPHA - default=false)
kube:RecoverVolumeExpansionFailure=true|false (ALPHA - default=false)
kube:RecursiveReadOnlyMounts=true|false (BETA - default=true)
kube:RelaxedEnvironmentVariableValidation=true|false (ALPHA - default=false)
kube:ReloadKubeletServerCertificateFile=true|false (BETA - default=true)
kube:ResilientWatchCacheInitialization=true|false (BETA - default=true)
kube:ResourceHealthStatus=true|false (ALPHA - default=false)
kube:RetryGenerateName=true|false (BETA - default=true)
kube:RotateKubeletServerCertificate=true|false (BETA - default=true)
kube:RuntimeClassInImageCriApi=true|false (ALPHA - default=false)
kube:SELinuxMount=true|false (ALPHA - default=false)
kube:SELinuxMountReadWriteOncePod=true|false (BETA - default=true)
kube:SchedulerQueueingHints=true|false (BETA - default=false)
kube:SeparateCacheWatchRPC=true|false (BETA - default=true)
kube:SeparateTaintEvictionController=true|false (BETA - default=true)
kube:ServiceAccountTokenJTI=true|false (BETA - default=true)
kube:ServiceAccountTokenNodeBinding=true|false (BETA - default=true)
kube:ServiceAccountTokenNodeBindingValidation=true|false (BETA - default=true)
kube:ServiceAccountTokenPodNodeInfo=true|false (BETA - default=true)
kube:ServiceTrafficDistribution=true|false (BETA - default=true)
kube:SidecarContainers=true|false (BETA - default=true)
kube:SizeMemoryBackedVolumes=true|false (BETA - default=true)
kube:StatefulSetAutoDeletePVC=true|false (BETA - default=true)
kube:StorageNamespaceIndex=true|false (BETA - default=true)
kube:StorageVersionAPI=true|false (ALPHA - default=false)
kube:StorageVersionHash=true|false (BETA - default=true)
kube:StorageVersionMigrator=true|false (ALPHA - default=false)
kube:StrictCostEnforcementForVAP=true|false (BETA - default=false)
kube:StrictCostEnforcementForWebhooks=true|false (BETA - default=false)
kube:StructuredAuthenticationConfiguration=true|false (BETA - default=true)
kube:StructuredAuthorizationConfiguration=true|false (BETA - default=true)
kube:SupplementalGroupsPolicy=true|false (ALPHA - default=false)
kube:TopologyAwareHints=true|false (BETA - default=true)
kube:TopologyManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
kube:TopologyManagerPolicyBetaOptions=true|false (BETA - default=true)
kube:TopologyManagerPolicyOptions=true|false (BETA - default=true)
kube:TranslateStreamCloseWebsocketRequests=true|false (BETA - default=true)
kube:UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true)
kube:UnknownVersionInteroperabilityProxy=true|false (ALPHA - default=false)
kube:UserNamespacesPodSecurityStandards=true|false (ALPHA - default=false)
kube:UserNamespacesSupport=true|false (BETA - default=false)
kube:VolumeAttributesClass=true|false (BETA - default=false)
kube:VolumeCapacityPriority=true|false (ALPHA - default=false)
kube:WatchCacheInitializationPostStartHook=true|false (BETA - default=false)
kube:WatchFromStorageWithoutResourceVersion=true|false (BETA - default=false)
kube:WatchList=true|false (ALPHA - default=false)
kube:WatchListClient=true|false (BETA - default=false)
kube:WinDSR=true|false (ALPHA - default=false)
kube:WinOverlay=true|false (BETA - default=true)
kube:WindowsHostNetwork=true|false (ALPHA - default=true)

+

Comma-separated list of component:key=value pairs that describe feature gates for alpha/experimental features of different components.
If the component is not specified, defaults to "kube". This flag can be repeatedly invoked. For example: --feature-gates 'wardle:featureA=true,wardle:featureB=false' --feature-gates 'kube:featureC=true'Options are:
kube:APIResponseCompression=true|false (BETA - default=true)
kube:APIServerIdentity=true|false (BETA - default=true)
kube:APIServerTracing=true|false (BETA - default=true)
kube:APIServingWithRoutine=true|false (ALPHA - default=false)
kube:AllAlpha=true|false (ALPHA - default=false)
kube:AllBeta=true|false (BETA - default=false)
kube:AllowUnsafeMalformedObjectDeletion=true|false (ALPHA - default=false)
kube:AnonymousAuthConfigurableEndpoints=true|false (BETA - default=true)
kube:AnyVolumeDataSource=true|false (BETA - default=true)
kube:AuthorizeNodeWithSelectors=true|false (BETA - default=true)
kube:AuthorizeWithSelectors=true|false (BETA - default=true)
kube:BtreeWatchCache=true|false (BETA - default=true)
kube:CBORServingAndStorage=true|false (ALPHA - default=false)
kube:CPUManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
kube:CPUManagerPolicyBetaOptions=true|false (BETA - default=true)
kube:CPUManagerPolicyOptions=true|false (BETA - default=true)
kube:CRDValidationRatcheting=true|false (BETA - default=true)
kube:CSIMigrationPortworx=true|false (BETA - default=true)
kube:CSIVolumeHealth=true|false (ALPHA - default=false)
kube:ClientsAllowCBOR=true|false (ALPHA - default=false)
kube:ClientsPreferCBOR=true|false (ALPHA - default=false)
kube:CloudControllerManagerWebhook=true|false (ALPHA - default=false)
kube:ClusterTrustBundle=true|false (ALPHA - default=false)
kube:ClusterTrustBundleProjection=true|false (ALPHA - default=false)
kube:ComponentFlagz=true|false (ALPHA - default=false)
kube:ComponentStatusz=true|false (ALPHA - default=false)
kube:ConcurrentWatchObjectDecode=true|false (BETA - default=false)
kube:ConsistentListFromCache=true|false (BETA - default=true)
kube:ContainerCheckpoint=true|false (BETA - default=true)
kube:ContextualLogging=true|false (BETA - default=true)
kube:CoordinatedLeaderElection=true|false (ALPHA - default=false)
kube:CrossNamespaceVolumeDataSource=true|false (ALPHA - default=false)
kube:CustomCPUCFSQuotaPeriod=true|false (ALPHA - default=false)
kube:DRAAdminAccess=true|false (ALPHA - default=false)
kube:DRAResourceClaimDeviceStatus=true|false (ALPHA - default=false)
kube:DisableAllocatorDualWrite=true|false (ALPHA - default=false)
kube:DynamicResourceAllocation=true|false (BETA - default=false)
kube:EventedPLEG=true|false (ALPHA - default=false)
kube:ExternalServiceAccountTokenSigner=true|false (ALPHA - default=false)
kube:GracefulNodeShutdown=true|false (BETA - default=true)
kube:GracefulNodeShutdownBasedOnPodPriority=true|false (BETA - default=true)
kube:HPAScaleToZero=true|false (ALPHA - default=false)
kube:HonorPVReclaimPolicy=true|false (BETA - default=true)
kube:ImageMaximumGCAge=true|false (BETA - default=true)
kube:ImageVolume=true|false (ALPHA - default=false)
kube:InPlacePodVerticalScaling=true|false (ALPHA - default=false)
kube:InPlacePodVerticalScalingAllocatedStatus=true|false (ALPHA - default=false)
kube:InPlacePodVerticalScalingExclusiveCPUs=true|false (ALPHA - default=false)
kube:InTreePluginPortworxUnregister=true|false (ALPHA - default=false)
kube:InformerResourceVersion=true|false (ALPHA - default=false)
kube:JobBackoffLimitPerIndex=true|false (BETA - default=true)
kube:JobManagedBy=true|false (BETA - default=true)
kube:JobPodReplacementPolicy=true|false (BETA - default=true)
kube:JobSuccessPolicy=true|false (BETA - default=true)
kube:KubeletCgroupDriverFromCRI=true|false (BETA - default=true)
kube:KubeletCrashLoopBackOffMax=true|false (ALPHA - default=false)
kube:KubeletFineGrainedAuthz=true|false (ALPHA - default=false)
kube:KubeletInUserNamespace=true|false (ALPHA - default=false)
kube:KubeletPodResourcesDynamicResources=true|false (ALPHA - default=false)
kube:KubeletPodResourcesGet=true|false (ALPHA - default=false)
kube:KubeletSeparateDiskGC=true|false (BETA - default=true)
kube:KubeletTracing=true|false (BETA - default=true)
kube:LocalStorageCapacityIsolationFSQuotaMonitoring=true|false (BETA - default=false)
kube:LoggingAlphaOptions=true|false (ALPHA - default=false)
kube:LoggingBetaOptions=true|false (BETA - default=true)
kube:MatchLabelKeysInPodAffinity=true|false (BETA - default=true)
kube:MatchLabelKeysInPodTopologySpread=true|false (BETA - default=true)
kube:MaxUnavailableStatefulSet=true|false (ALPHA - default=false)
kube:MemoryQoS=true|false (ALPHA - default=false)
kube:MultiCIDRServiceAllocator=true|false (BETA - default=false)
kube:MutatingAdmissionPolicy=true|false (ALPHA - default=false)
kube:NFTablesProxyMode=true|false (BETA - default=true)
kube:NodeInclusionPolicyInPodTopologySpread=true|false (BETA - default=true)
kube:NodeLogQuery=true|false (BETA - default=false)
kube:NodeSwap=true|false (BETA - default=true)
kube:OpenAPIEnums=true|false (BETA - default=true)
kube:PodAndContainerStatsFromCRI=true|false (ALPHA - default=false)
kube:PodDeletionCost=true|false (BETA - default=true)
kube:PodLevelResources=true|false (ALPHA - default=false)
kube:PodLifecycleSleepAction=true|false (BETA - default=true)
kube:PodLifecycleSleepActionAllowZero=true|false (ALPHA - default=false)
kube:PodLogsQuerySplitStreams=true|false (ALPHA - default=false)
kube:PodReadyToStartContainersCondition=true|false (BETA - default=true)
kube:PortForwardWebsockets=true|false (BETA - default=true)
kube:ProcMountType=true|false (BETA - default=false)
kube:QOSReserved=true|false (ALPHA - default=false)
kube:RecoverVolumeExpansionFailure=true|false (BETA - default=true)
kube:RecursiveReadOnlyMounts=true|false (BETA - default=true)
kube:RelaxedDNSSearchValidation=true|false (ALPHA - default=false)
kube:RelaxedEnvironmentVariableValidation=true|false (BETA - default=true)
kube:ReloadKubeletServerCertificateFile=true|false (BETA - default=true)
kube:RemoteRequestHeaderUID=true|false (ALPHA - default=false)
kube:ResilientWatchCacheInitialization=true|false (BETA - default=true)
kube:ResourceHealthStatus=true|false (ALPHA - default=false)
kube:RotateKubeletServerCertificate=true|false (BETA - default=true)
kube:RuntimeClassInImageCriApi=true|false (ALPHA - default=false)
kube:SELinuxChangePolicy=true|false (ALPHA - default=false)
kube:SELinuxMount=true|false (ALPHA - default=false)
kube:SELinuxMountReadWriteOncePod=true|false (BETA - default=true)
kube:SchedulerAsyncPreemption=true|false (ALPHA - default=false)
kube:SchedulerQueueingHints=true|false (BETA - default=true)
kube:SeparateCacheWatchRPC=true|false (BETA - default=true)
kube:SeparateTaintEvictionController=true|false (BETA - default=true)
kube:ServiceAccountNodeAudienceRestriction=true|false (BETA - default=true)
kube:ServiceAccountTokenNodeBinding=true|false (BETA - default=true)
kube:ServiceTrafficDistribution=true|false (BETA - default=true)
kube:SidecarContainers=true|false (BETA - default=true)
kube:StorageNamespaceIndex=true|false (BETA - default=true)
kube:StorageVersionAPI=true|false (ALPHA - default=false)
kube:StorageVersionHash=true|false (BETA - default=true)
kube:StorageVersionMigrator=true|false (ALPHA - default=false)
kube:StructuredAuthenticationConfiguration=true|false (BETA - default=true)
kube:SupplementalGroupsPolicy=true|false (ALPHA - default=false)
kube:SystemdWatchdog=true|false (BETA - default=true)
kube:TopologyAwareHints=true|false (BETA - default=true)
kube:TopologyManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
kube:TopologyManagerPolicyBetaOptions=true|false (BETA - default=true)
kube:TranslateStreamCloseWebsocketRequests=true|false (BETA - default=true)
kube:UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true)
kube:UnknownVersionInteroperabilityProxy=true|false (ALPHA - default=false)
kube:UserNamespacesPodSecurityStandards=true|false (ALPHA - default=false)
kube:UserNamespacesSupport=true|false (BETA - default=false)
kube:VolumeAttributesClass=true|false (BETA - default=false)
kube:VolumeCapacityPriority=true|false (ALPHA - default=false)
kube:WatchCacheInitializationPostStartHook=true|false (BETA - default=false)
kube:WatchFromStorageWithoutResourceVersion=true|false (BETA - default=false)
kube:WatchList=true|false (BETA - default=true)
kube:WatchListClient=true|false (BETA - default=false)
kube:WinDSR=true|false (ALPHA - default=false)
kube:WinOverlay=true|false (BETA - default=true)
kube:WindowsCPUAndMemoryAffinity=true|false (ALPHA - default=false)
kube:WindowsGracefulNodeShutdown=true|false (ALPHA - default=false)
kube:WindowsHostNetwork=true|false (ALPHA - default=true)

@@ -928,6 +928,13 @@ kube-apiserver [flags]

List of request headers to inspect for groups. X-Remote-Group is suggested.

+ +--requestheader-uid-headers strings + + +

List of request headers to inspect for UIDs. X-Remote-Uid is suggested. Requires the RemoteRequestHeaderUID feature to be enabled.

+ + --requestheader-username-headers strings @@ -991,6 +998,13 @@ kube-apiserver [flags]

The maximum validity duration of a token created by the service account token issuer. If an otherwise valid TokenRequest with a validity duration larger than this value is requested, a token will be issued with a validity duration of this value.

+ +--service-account-signing-endpoint string + + +

Path to socket where a external JWT signer is listening. This flag is mutually exclusive with --service-account-signing-key-file and --service-account-key-file. Requires enabling feature gate (ExternalServiceAccountTokenSigner)

+ + --service-account-signing-key-file string diff --git a/content/en/docs/reference/command-line-tools-reference/kube-controller-manager.md b/content/en/docs/reference/command-line-tools-reference/kube-controller-manager.md index 5b915fdbd64f3..193dd25205af7 100644 --- a/content/en/docs/reference/command-line-tools-reference/kube-controller-manager.md +++ b/content/en/docs/reference/command-line-tools-reference/kube-controller-manager.md @@ -47,7 +47,7 @@ kube-controller-manager [flags] --allocate-node-cidrs -

Should CIDRs for Pods be allocated and set on the cloud provider.

+

Should CIDRs for Pods be allocated and set on the cloud provider. Requires --cluster-cidr.

@@ -173,7 +173,7 @@ kube-controller-manager [flags] --cluster-cidr string -

CIDR Range for Pods in cluster. Requires --allocate-node-cidrs to be true

+

CIDR Range for Pods in cluster. Only used when --allocate-node-cidrs=true; if false, this option will be ignored.

@@ -267,6 +267,13 @@ kube-controller-manager [flags]

The number of cron job objects that are allowed to sync concurrently. Larger number = more responsive jobs, but more CPU (and network) load

+ +--concurrent-daemonset-syncs int32     Default: 2 + + +

The number of daemonset objects that are allowed to sync concurrently. Larger number = more responsive daemonsets, but more CPU (and network) load

+ + --concurrent-deployment-syncs int32     Default: 5 @@ -404,7 +411,7 @@ kube-controller-manager [flags] --controllers strings     Default: "*" -

A list of controllers to enable. '*' enables all on-by-default controllers, 'foo' enables the controller named 'foo', '-foo' disables the controller named 'foo'.
All controllers: bootstrap-signer-controller, certificatesigningrequest-approving-controller, certificatesigningrequest-cleaner-controller, certificatesigningrequest-signing-controller, cloud-node-lifecycle-controller, clusterrole-aggregation-controller, cronjob-controller, daemonset-controller, deployment-controller, disruption-controller, endpoints-controller, endpointslice-controller, endpointslice-mirroring-controller, ephemeral-volume-controller, garbage-collector-controller, horizontal-pod-autoscaler-controller, job-controller, legacy-serviceaccount-token-cleaner-controller, namespace-controller, node-ipam-controller, node-lifecycle-controller, node-route-controller, persistentvolume-attach-detach-controller, persistentvolume-binder-controller, persistentvolume-expander-controller, persistentvolume-protection-controller, persistentvolumeclaim-protection-controller, pod-garbage-collector-controller, replicaset-controller, replicationcontroller-controller, resourceclaim-controller, resourcequota-controller, root-ca-certificate-publisher-controller, service-cidr-controller, service-lb-controller, serviceaccount-controller, serviceaccount-token-controller, statefulset-controller, storage-version-migrator-controller, storageversion-garbage-collector-controller, taint-eviction-controller, token-cleaner-controller, ttl-after-finished-controller, ttl-controller, validatingadmissionpolicy-status-controller
Disabled-by-default controllers: bootstrap-signer-controller, token-cleaner-controller

+

A list of controllers to enable. '*' enables all on-by-default controllers, 'foo' enables the controller named 'foo', '-foo' disables the controller named 'foo'.
All controllers: bootstrap-signer-controller, certificatesigningrequest-approving-controller, certificatesigningrequest-cleaner-controller, certificatesigningrequest-signing-controller, cloud-node-lifecycle-controller, clusterrole-aggregation-controller, cronjob-controller, daemonset-controller, deployment-controller, disruption-controller, endpoints-controller, endpointslice-controller, endpointslice-mirroring-controller, ephemeral-volume-controller, garbage-collector-controller, horizontal-pod-autoscaler-controller, job-controller, kube-apiserver-serving-clustertrustbundle-publisher-controller, legacy-serviceaccount-token-cleaner-controller, namespace-controller, node-ipam-controller, node-lifecycle-controller, node-route-controller, persistentvolume-attach-detach-controller, persistentvolume-binder-controller, persistentvolume-expander-controller, persistentvolume-protection-controller, persistentvolumeclaim-protection-controller, pod-garbage-collector-controller, replicaset-controller, replicationcontroller-controller, resourceclaim-controller, resourcequota-controller, root-ca-certificate-publisher-controller, selinux-warning-controller, service-cidr-controller, service-lb-controller, serviceaccount-controller, serviceaccount-token-controller, statefulset-controller, storage-version-migrator-controller, storageversion-garbage-collector-controller, taint-eviction-controller, token-cleaner-controller, ttl-after-finished-controller, ttl-controller, validatingadmissionpolicy-status-controller, volumeattributesclass-protection-controller
Disabled-by-default controllers: bootstrap-signer-controller, selinux-warning-controller, token-cleaner-controller

@@ -439,7 +446,7 @@ kube-controller-manager [flags] --emulated-version strings -

The versions different components emulate their capabilities (APIs, features, ...) of.
If set, the component will emulate the behavior of this version instead of the underlying binary version.
Version format could only be major.minor, for example: '--emulated-version=wardle=1.2,kube=1.31'. Options are:
kube=1.31..1.31 (default=1.31)If the component is not specified, defaults to "kube"

+

The versions different components emulate their capabilities (APIs, features, ...) of.
If set, the component will emulate the behavior of this version instead of the underlying binary version.
Version format could only be major.minor, for example: '--emulated-version=wardle=1.2,kube=1.31'. Options are:
kube=1.32..1.32 (default=1.32)If the component is not specified, defaults to "kube"

@@ -495,7 +502,7 @@ kube-controller-manager [flags] --feature-gates colonSeparatedMultimapStringString -

Comma-separated list of component:key=value pairs that describe feature gates for alpha/experimental features of different components.
If the component is not specified, defaults to "kube". This flag can be repeatedly invoked. For example: --feature-gates 'wardle:featureA=true,wardle:featureB=false' --feature-gates 'kube:featureC=true'Options are:
kube:APIResponseCompression=true|false (BETA - default=true)
kube:APIServerIdentity=true|false (BETA - default=true)
kube:APIServerTracing=true|false (BETA - default=true)
kube:APIServingWithRoutine=true|false (ALPHA - default=false)
kube:AllAlpha=true|false (ALPHA - default=false)
kube:AllBeta=true|false (BETA - default=false)
kube:AnonymousAuthConfigurableEndpoints=true|false (ALPHA - default=false)
kube:AnyVolumeDataSource=true|false (BETA - default=true)
kube:AuthorizeNodeWithSelectors=true|false (ALPHA - default=false)
kube:AuthorizeWithSelectors=true|false (ALPHA - default=false)
kube:CPUManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
kube:CPUManagerPolicyBetaOptions=true|false (BETA - default=true)
kube:CPUManagerPolicyOptions=true|false (BETA - default=true)
kube:CRDValidationRatcheting=true|false (BETA - default=true)
kube:CSIMigrationPortworx=true|false (BETA - default=true)
kube:CSIVolumeHealth=true|false (ALPHA - default=false)
kube:CloudControllerManagerWebhook=true|false (ALPHA - default=false)
kube:ClusterTrustBundle=true|false (ALPHA - default=false)
kube:ClusterTrustBundleProjection=true|false (ALPHA - default=false)
kube:ComponentSLIs=true|false (BETA - default=true)
kube:ConcurrentWatchObjectDecode=true|false (BETA - default=false)
kube:ConsistentListFromCache=true|false (BETA - default=true)
kube:ContainerCheckpoint=true|false (BETA - default=true)
kube:ContextualLogging=true|false (BETA - default=true)
kube:CoordinatedLeaderElection=true|false (ALPHA - default=false)
kube:CronJobsScheduledAnnotation=true|false (BETA - default=true)
kube:CrossNamespaceVolumeDataSource=true|false (ALPHA - default=false)
kube:CustomCPUCFSQuotaPeriod=true|false (ALPHA - default=false)
kube:CustomResourceFieldSelectors=true|false (BETA - default=true)
kube:DRAControlPlaneController=true|false (ALPHA - default=false)
kube:DisableAllocatorDualWrite=true|false (ALPHA - default=false)
kube:DisableNodeKubeProxyVersion=true|false (BETA - default=true)
kube:DynamicResourceAllocation=true|false (ALPHA - default=false)
kube:EventedPLEG=true|false (ALPHA - default=false)
kube:GracefulNodeShutdown=true|false (BETA - default=true)
kube:GracefulNodeShutdownBasedOnPodPriority=true|false (BETA - default=true)
kube:HPAScaleToZero=true|false (ALPHA - default=false)
kube:HonorPVReclaimPolicy=true|false (BETA - default=true)
kube:ImageMaximumGCAge=true|false (BETA - default=true)
kube:ImageVolume=true|false (ALPHA - default=false)
kube:InPlacePodVerticalScaling=true|false (ALPHA - default=false)
kube:InTreePluginPortworxUnregister=true|false (ALPHA - default=false)
kube:InformerResourceVersion=true|false (ALPHA - default=false)
kube:JobBackoffLimitPerIndex=true|false (BETA - default=true)
kube:JobManagedBy=true|false (ALPHA - default=false)
kube:JobPodReplacementPolicy=true|false (BETA - default=true)
kube:JobSuccessPolicy=true|false (BETA - default=true)
kube:KubeletCgroupDriverFromCRI=true|false (BETA - default=true)
kube:KubeletInUserNamespace=true|false (ALPHA - default=false)
kube:KubeletPodResourcesDynamicResources=true|false (ALPHA - default=false)
kube:KubeletPodResourcesGet=true|false (ALPHA - default=false)
kube:KubeletSeparateDiskGC=true|false (BETA - default=true)
kube:KubeletTracing=true|false (BETA - default=true)
kube:LoadBalancerIPMode=true|false (BETA - default=true)
kube:LocalStorageCapacityIsolationFSQuotaMonitoring=true|false (BETA - default=false)
kube:LoggingAlphaOptions=true|false (ALPHA - default=false)
kube:LoggingBetaOptions=true|false (BETA - default=true)
kube:MatchLabelKeysInPodAffinity=true|false (BETA - default=true)
kube:MatchLabelKeysInPodTopologySpread=true|false (BETA - default=true)
kube:MaxUnavailableStatefulSet=true|false (ALPHA - default=false)
kube:MemoryManager=true|false (BETA - default=true)
kube:MemoryQoS=true|false (ALPHA - default=false)
kube:MultiCIDRServiceAllocator=true|false (BETA - default=false)
kube:MutatingAdmissionPolicy=true|false (ALPHA - default=false)
kube:NFTablesProxyMode=true|false (BETA - default=true)
kube:NodeInclusionPolicyInPodTopologySpread=true|false (BETA - default=true)
kube:NodeLogQuery=true|false (BETA - default=false)
kube:NodeSwap=true|false (BETA - default=true)
kube:OpenAPIEnums=true|false (BETA - default=true)
kube:PodAndContainerStatsFromCRI=true|false (ALPHA - default=false)
kube:PodDeletionCost=true|false (BETA - default=true)
kube:PodIndexLabel=true|false (BETA - default=true)
kube:PodLifecycleSleepAction=true|false (BETA - default=true)
kube:PodReadyToStartContainersCondition=true|false (BETA - default=true)
kube:PortForwardWebsockets=true|false (BETA - default=true)
kube:ProcMountType=true|false (BETA - default=false)
kube:QOSReserved=true|false (ALPHA - default=false)
kube:RecoverVolumeExpansionFailure=true|false (ALPHA - default=false)
kube:RecursiveReadOnlyMounts=true|false (BETA - default=true)
kube:RelaxedEnvironmentVariableValidation=true|false (ALPHA - default=false)
kube:ReloadKubeletServerCertificateFile=true|false (BETA - default=true)
kube:ResilientWatchCacheInitialization=true|false (BETA - default=true)
kube:ResourceHealthStatus=true|false (ALPHA - default=false)
kube:RetryGenerateName=true|false (BETA - default=true)
kube:RotateKubeletServerCertificate=true|false (BETA - default=true)
kube:RuntimeClassInImageCriApi=true|false (ALPHA - default=false)
kube:SELinuxMount=true|false (ALPHA - default=false)
kube:SELinuxMountReadWriteOncePod=true|false (BETA - default=true)
kube:SchedulerQueueingHints=true|false (BETA - default=false)
kube:SeparateCacheWatchRPC=true|false (BETA - default=true)
kube:SeparateTaintEvictionController=true|false (BETA - default=true)
kube:ServiceAccountTokenJTI=true|false (BETA - default=true)
kube:ServiceAccountTokenNodeBinding=true|false (BETA - default=true)
kube:ServiceAccountTokenNodeBindingValidation=true|false (BETA - default=true)
kube:ServiceAccountTokenPodNodeInfo=true|false (BETA - default=true)
kube:ServiceTrafficDistribution=true|false (BETA - default=true)
kube:SidecarContainers=true|false (BETA - default=true)
kube:SizeMemoryBackedVolumes=true|false (BETA - default=true)
kube:StatefulSetAutoDeletePVC=true|false (BETA - default=true)
kube:StorageNamespaceIndex=true|false (BETA - default=true)
kube:StorageVersionAPI=true|false (ALPHA - default=false)
kube:StorageVersionHash=true|false (BETA - default=true)
kube:StorageVersionMigrator=true|false (ALPHA - default=false)
kube:StrictCostEnforcementForVAP=true|false (BETA - default=false)
kube:StrictCostEnforcementForWebhooks=true|false (BETA - default=false)
kube:StructuredAuthenticationConfiguration=true|false (BETA - default=true)
kube:StructuredAuthorizationConfiguration=true|false (BETA - default=true)
kube:SupplementalGroupsPolicy=true|false (ALPHA - default=false)
kube:TopologyAwareHints=true|false (BETA - default=true)
kube:TopologyManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
kube:TopologyManagerPolicyBetaOptions=true|false (BETA - default=true)
kube:TopologyManagerPolicyOptions=true|false (BETA - default=true)
kube:TranslateStreamCloseWebsocketRequests=true|false (BETA - default=true)
kube:UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true)
kube:UnknownVersionInteroperabilityProxy=true|false (ALPHA - default=false)
kube:UserNamespacesPodSecurityStandards=true|false (ALPHA - default=false)
kube:UserNamespacesSupport=true|false (BETA - default=false)
kube:VolumeAttributesClass=true|false (BETA - default=false)
kube:VolumeCapacityPriority=true|false (ALPHA - default=false)
kube:WatchCacheInitializationPostStartHook=true|false (BETA - default=false)
kube:WatchFromStorageWithoutResourceVersion=true|false (BETA - default=false)
kube:WatchList=true|false (ALPHA - default=false)
kube:WatchListClient=true|false (BETA - default=false)
kube:WinDSR=true|false (ALPHA - default=false)
kube:WinOverlay=true|false (BETA - default=true)
kube:WindowsHostNetwork=true|false (ALPHA - default=true)

+

Comma-separated list of component:key=value pairs that describe feature gates for alpha/experimental features of different components.
If the component is not specified, defaults to "kube". This flag can be repeatedly invoked. For example: --feature-gates 'wardle:featureA=true,wardle:featureB=false' --feature-gates 'kube:featureC=true'Options are:
kube:APIResponseCompression=true|false (BETA - default=true)
kube:APIServerIdentity=true|false (BETA - default=true)
kube:APIServerTracing=true|false (BETA - default=true)
kube:APIServingWithRoutine=true|false (ALPHA - default=false)
kube:AllAlpha=true|false (ALPHA - default=false)
kube:AllBeta=true|false (BETA - default=false)
kube:AllowUnsafeMalformedObjectDeletion=true|false (ALPHA - default=false)
kube:AnonymousAuthConfigurableEndpoints=true|false (BETA - default=true)
kube:AnyVolumeDataSource=true|false (BETA - default=true)
kube:AuthorizeNodeWithSelectors=true|false (BETA - default=true)
kube:AuthorizeWithSelectors=true|false (BETA - default=true)
kube:BtreeWatchCache=true|false (BETA - default=true)
kube:CBORServingAndStorage=true|false (ALPHA - default=false)
kube:CPUManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
kube:CPUManagerPolicyBetaOptions=true|false (BETA - default=true)
kube:CPUManagerPolicyOptions=true|false (BETA - default=true)
kube:CRDValidationRatcheting=true|false (BETA - default=true)
kube:CSIMigrationPortworx=true|false (BETA - default=true)
kube:CSIVolumeHealth=true|false (ALPHA - default=false)
kube:ClientsAllowCBOR=true|false (ALPHA - default=false)
kube:ClientsPreferCBOR=true|false (ALPHA - default=false)
kube:CloudControllerManagerWebhook=true|false (ALPHA - default=false)
kube:ClusterTrustBundle=true|false (ALPHA - default=false)
kube:ClusterTrustBundleProjection=true|false (ALPHA - default=false)
kube:ComponentFlagz=true|false (ALPHA - default=false)
kube:ComponentStatusz=true|false (ALPHA - default=false)
kube:ConcurrentWatchObjectDecode=true|false (BETA - default=false)
kube:ConsistentListFromCache=true|false (BETA - default=true)
kube:ContainerCheckpoint=true|false (BETA - default=true)
kube:ContextualLogging=true|false (BETA - default=true)
kube:CoordinatedLeaderElection=true|false (ALPHA - default=false)
kube:CrossNamespaceVolumeDataSource=true|false (ALPHA - default=false)
kube:CustomCPUCFSQuotaPeriod=true|false (ALPHA - default=false)
kube:DRAAdminAccess=true|false (ALPHA - default=false)
kube:DRAResourceClaimDeviceStatus=true|false (ALPHA - default=false)
kube:DisableAllocatorDualWrite=true|false (ALPHA - default=false)
kube:DynamicResourceAllocation=true|false (BETA - default=false)
kube:EventedPLEG=true|false (ALPHA - default=false)
kube:ExternalServiceAccountTokenSigner=true|false (ALPHA - default=false)
kube:GracefulNodeShutdown=true|false (BETA - default=true)
kube:GracefulNodeShutdownBasedOnPodPriority=true|false (BETA - default=true)
kube:HPAScaleToZero=true|false (ALPHA - default=false)
kube:HonorPVReclaimPolicy=true|false (BETA - default=true)
kube:ImageMaximumGCAge=true|false (BETA - default=true)
kube:ImageVolume=true|false (ALPHA - default=false)
kube:InPlacePodVerticalScaling=true|false (ALPHA - default=false)
kube:InPlacePodVerticalScalingAllocatedStatus=true|false (ALPHA - default=false)
kube:InPlacePodVerticalScalingExclusiveCPUs=true|false (ALPHA - default=false)
kube:InTreePluginPortworxUnregister=true|false (ALPHA - default=false)
kube:InformerResourceVersion=true|false (ALPHA - default=false)
kube:JobBackoffLimitPerIndex=true|false (BETA - default=true)
kube:JobManagedBy=true|false (BETA - default=true)
kube:JobPodReplacementPolicy=true|false (BETA - default=true)
kube:JobSuccessPolicy=true|false (BETA - default=true)
kube:KubeletCgroupDriverFromCRI=true|false (BETA - default=true)
kube:KubeletCrashLoopBackOffMax=true|false (ALPHA - default=false)
kube:KubeletFineGrainedAuthz=true|false (ALPHA - default=false)
kube:KubeletInUserNamespace=true|false (ALPHA - default=false)
kube:KubeletPodResourcesDynamicResources=true|false (ALPHA - default=false)
kube:KubeletPodResourcesGet=true|false (ALPHA - default=false)
kube:KubeletSeparateDiskGC=true|false (BETA - default=true)
kube:KubeletTracing=true|false (BETA - default=true)
kube:LocalStorageCapacityIsolationFSQuotaMonitoring=true|false (BETA - default=false)
kube:LoggingAlphaOptions=true|false (ALPHA - default=false)
kube:LoggingBetaOptions=true|false (BETA - default=true)
kube:MatchLabelKeysInPodAffinity=true|false (BETA - default=true)
kube:MatchLabelKeysInPodTopologySpread=true|false (BETA - default=true)
kube:MaxUnavailableStatefulSet=true|false (ALPHA - default=false)
kube:MemoryQoS=true|false (ALPHA - default=false)
kube:MultiCIDRServiceAllocator=true|false (BETA - default=false)
kube:MutatingAdmissionPolicy=true|false (ALPHA - default=false)
kube:NFTablesProxyMode=true|false (BETA - default=true)
kube:NodeInclusionPolicyInPodTopologySpread=true|false (BETA - default=true)
kube:NodeLogQuery=true|false (BETA - default=false)
kube:NodeSwap=true|false (BETA - default=true)
kube:OpenAPIEnums=true|false (BETA - default=true)
kube:PodAndContainerStatsFromCRI=true|false (ALPHA - default=false)
kube:PodDeletionCost=true|false (BETA - default=true)
kube:PodLevelResources=true|false (ALPHA - default=false)
kube:PodLifecycleSleepAction=true|false (BETA - default=true)
kube:PodLifecycleSleepActionAllowZero=true|false (ALPHA - default=false)
kube:PodLogsQuerySplitStreams=true|false (ALPHA - default=false)
kube:PodReadyToStartContainersCondition=true|false (BETA - default=true)
kube:PortForwardWebsockets=true|false (BETA - default=true)
kube:ProcMountType=true|false (BETA - default=false)
kube:QOSReserved=true|false (ALPHA - default=false)
kube:RecoverVolumeExpansionFailure=true|false (BETA - default=true)
kube:RecursiveReadOnlyMounts=true|false (BETA - default=true)
kube:RelaxedDNSSearchValidation=true|false (ALPHA - default=false)
kube:RelaxedEnvironmentVariableValidation=true|false (BETA - default=true)
kube:ReloadKubeletServerCertificateFile=true|false (BETA - default=true)
kube:RemoteRequestHeaderUID=true|false (ALPHA - default=false)
kube:ResilientWatchCacheInitialization=true|false (BETA - default=true)
kube:ResourceHealthStatus=true|false (ALPHA - default=false)
kube:RotateKubeletServerCertificate=true|false (BETA - default=true)
kube:RuntimeClassInImageCriApi=true|false (ALPHA - default=false)
kube:SELinuxChangePolicy=true|false (ALPHA - default=false)
kube:SELinuxMount=true|false (ALPHA - default=false)
kube:SELinuxMountReadWriteOncePod=true|false (BETA - default=true)
kube:SchedulerAsyncPreemption=true|false (ALPHA - default=false)
kube:SchedulerQueueingHints=true|false (BETA - default=true)
kube:SeparateCacheWatchRPC=true|false (BETA - default=true)
kube:SeparateTaintEvictionController=true|false (BETA - default=true)
kube:ServiceAccountNodeAudienceRestriction=true|false (BETA - default=true)
kube:ServiceAccountTokenNodeBinding=true|false (BETA - default=true)
kube:ServiceTrafficDistribution=true|false (BETA - default=true)
kube:SidecarContainers=true|false (BETA - default=true)
kube:StorageNamespaceIndex=true|false (BETA - default=true)
kube:StorageVersionAPI=true|false (ALPHA - default=false)
kube:StorageVersionHash=true|false (BETA - default=true)
kube:StorageVersionMigrator=true|false (ALPHA - default=false)
kube:StructuredAuthenticationConfiguration=true|false (BETA - default=true)
kube:SupplementalGroupsPolicy=true|false (ALPHA - default=false)
kube:SystemdWatchdog=true|false (BETA - default=true)
kube:TopologyAwareHints=true|false (BETA - default=true)
kube:TopologyManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
kube:TopologyManagerPolicyBetaOptions=true|false (BETA - default=true)
kube:TranslateStreamCloseWebsocketRequests=true|false (BETA - default=true)
kube:UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true)
kube:UnknownVersionInteroperabilityProxy=true|false (ALPHA - default=false)
kube:UserNamespacesPodSecurityStandards=true|false (ALPHA - default=false)
kube:UserNamespacesSupport=true|false (BETA - default=false)
kube:VolumeAttributesClass=true|false (BETA - default=false)
kube:VolumeCapacityPriority=true|false (ALPHA - default=false)
kube:WatchCacheInitializationPostStartHook=true|false (BETA - default=false)
kube:WatchFromStorageWithoutResourceVersion=true|false (BETA - default=false)
kube:WatchList=true|false (BETA - default=true)
kube:WatchListClient=true|false (BETA - default=true)
kube:WinDSR=true|false (ALPHA - default=false)
kube:WinOverlay=true|false (BETA - default=true)
kube:WindowsCPUAndMemoryAffinity=true|false (ALPHA - default=false)
kube:WindowsGracefulNodeShutdown=true|false (ALPHA - default=false)
kube:WindowsHostNetwork=true|false (ALPHA - default=true)

@@ -614,7 +621,7 @@ kube-controller-manager [flags] --leader-elect-resource-lock string     Default: "leases" -

The type of resource object that is used for locking during leader election. Supported options are 'leases', 'endpointsleases' and 'configmapsleases'.

+

The type of resource object that is used for locking during leader election. Supported options are 'leases'.

@@ -758,10 +765,10 @@ kube-controller-manager [flags] ---node-monitor-grace-period duration     Default: 40s +--node-monitor-grace-period duration     Default: 50s -

Amount of time which we allow running Node to be unresponsive before marking it unhealthy. Must be N times more than kubelet's nodeStatusUpdateFrequency, where N means number of retries allowed for kubelet to post node status.

+

Amount of time which we allow running Node to be unresponsive before marking it unhealthy. Must be N times more than kubelet's nodeStatusUpdateFrequency, where N means number of retries allowed for kubelet to post node status. This value should also be greater than the sum of HTTP2_PING_TIMEOUT_SECONDS and HTTP2_READ_IDLE_TIMEOUT_SECONDS

@@ -876,6 +883,13 @@ kube-controller-manager [flags]

List of request headers to inspect for groups. X-Remote-Group is suggested.

+ +--requestheader-uid-headers strings + + +

List of request headers to inspect for UIDs. X-Remote-Uid is suggested. Requires the RemoteRequestHeaderUID feature to be enabled.

+ + --requestheader-username-headers strings     Default: "x-remote-user" @@ -922,14 +936,14 @@ kube-controller-manager [flags] --service-account-private-key-file string -

Filename containing a PEM-encoded private RSA or ECDSA key used to sign service account tokens.

+

Enables legacy secret-based tokens when set. Filename containing a PEM-encoded private RSA or ECDSA key used to sign service account tokens.

--service-cluster-ip-range string -

CIDR Range for Services in cluster. Requires --allocate-node-cidrs to be true

+

CIDR Range for Services in cluster. Only used when --allocate-node-cidrs=true; if false, this option will be ignored.

diff --git a/content/en/docs/reference/command-line-tools-reference/kube-proxy.md b/content/en/docs/reference/command-line-tools-reference/kube-proxy.md index 92dd1488ba12c..0b34e4c637be4 100644 --- a/content/en/docs/reference/command-line-tools-reference/kube-proxy.md +++ b/content/en/docs/reference/command-line-tools-reference/kube-proxy.md @@ -158,7 +158,7 @@ kube-proxy [flags] --feature-gates <comma-separated 'key=True|False' pairs> -

A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
APIResponseCompression=true|false (BETA - default=true)
APIServerIdentity=true|false (BETA - default=true)
APIServerTracing=true|false (BETA - default=true)
APIServingWithRoutine=true|false (ALPHA - default=false)
AllAlpha=true|false (ALPHA - default=false)
AllBeta=true|false (BETA - default=false)
AnonymousAuthConfigurableEndpoints=true|false (ALPHA - default=false)
AnyVolumeDataSource=true|false (BETA - default=true)
AuthorizeNodeWithSelectors=true|false (ALPHA - default=false)
AuthorizeWithSelectors=true|false (ALPHA - default=false)
CPUManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
CPUManagerPolicyBetaOptions=true|false (BETA - default=true)
CPUManagerPolicyOptions=true|false (BETA - default=true)
CRDValidationRatcheting=true|false (BETA - default=true)
CSIMigrationPortworx=true|false (BETA - default=true)
CSIVolumeHealth=true|false (ALPHA - default=false)
CloudControllerManagerWebhook=true|false (ALPHA - default=false)
ClusterTrustBundle=true|false (ALPHA - default=false)
ClusterTrustBundleProjection=true|false (ALPHA - default=false)
ComponentSLIs=true|false (BETA - default=true)
ConcurrentWatchObjectDecode=true|false (BETA - default=false)
ConsistentListFromCache=true|false (BETA - default=true)
ContainerCheckpoint=true|false (BETA - default=true)
ContextualLogging=true|false (BETA - default=true)
CoordinatedLeaderElection=true|false (ALPHA - default=false)
CronJobsScheduledAnnotation=true|false (BETA - default=true)
CrossNamespaceVolumeDataSource=true|false (ALPHA - default=false)
CustomCPUCFSQuotaPeriod=true|false (ALPHA - default=false)
CustomResourceFieldSelectors=true|false (BETA - default=true)
DRAControlPlaneController=true|false (ALPHA - default=false)
DisableAllocatorDualWrite=true|false (ALPHA - default=false)
DisableNodeKubeProxyVersion=true|false (BETA - default=true)
DynamicResourceAllocation=true|false (ALPHA - default=false)
EventedPLEG=true|false (ALPHA - default=false)
GracefulNodeShutdown=true|false (BETA - default=true)
GracefulNodeShutdownBasedOnPodPriority=true|false (BETA - default=true)
HPAScaleToZero=true|false (ALPHA - default=false)
HonorPVReclaimPolicy=true|false (BETA - default=true)
ImageMaximumGCAge=true|false (BETA - default=true)
ImageVolume=true|false (ALPHA - default=false)
InPlacePodVerticalScaling=true|false (ALPHA - default=false)
InTreePluginPortworxUnregister=true|false (ALPHA - default=false)
InformerResourceVersion=true|false (ALPHA - default=false)
JobBackoffLimitPerIndex=true|false (BETA - default=true)
JobManagedBy=true|false (ALPHA - default=false)
JobPodReplacementPolicy=true|false (BETA - default=true)
JobSuccessPolicy=true|false (BETA - default=true)
KubeletCgroupDriverFromCRI=true|false (BETA - default=true)
KubeletInUserNamespace=true|false (ALPHA - default=false)
KubeletPodResourcesDynamicResources=true|false (ALPHA - default=false)
KubeletPodResourcesGet=true|false (ALPHA - default=false)
KubeletSeparateDiskGC=true|false (BETA - default=true)
KubeletTracing=true|false (BETA - default=true)
LoadBalancerIPMode=true|false (BETA - default=true)
LocalStorageCapacityIsolationFSQuotaMonitoring=true|false (BETA - default=false)
LoggingAlphaOptions=true|false (ALPHA - default=false)
LoggingBetaOptions=true|false (BETA - default=true)
MatchLabelKeysInPodAffinity=true|false (BETA - default=true)
MatchLabelKeysInPodTopologySpread=true|false (BETA - default=true)
MaxUnavailableStatefulSet=true|false (ALPHA - default=false)
MemoryManager=true|false (BETA - default=true)
MemoryQoS=true|false (ALPHA - default=false)
MultiCIDRServiceAllocator=true|false (BETA - default=false)
MutatingAdmissionPolicy=true|false (ALPHA - default=false)
NFTablesProxyMode=true|false (BETA - default=true)
NodeInclusionPolicyInPodTopologySpread=true|false (BETA - default=true)
NodeLogQuery=true|false (BETA - default=false)
NodeSwap=true|false (BETA - default=true)
OpenAPIEnums=true|false (BETA - default=true)
PodAndContainerStatsFromCRI=true|false (ALPHA - default=false)
PodDeletionCost=true|false (BETA - default=true)
PodIndexLabel=true|false (BETA - default=true)
PodLifecycleSleepAction=true|false (BETA - default=true)
PodReadyToStartContainersCondition=true|false (BETA - default=true)
PortForwardWebsockets=true|false (BETA - default=true)
ProcMountType=true|false (BETA - default=false)
QOSReserved=true|false (ALPHA - default=false)
RecoverVolumeExpansionFailure=true|false (ALPHA - default=false)
RecursiveReadOnlyMounts=true|false (BETA - default=true)
RelaxedEnvironmentVariableValidation=true|false (ALPHA - default=false)
ReloadKubeletServerCertificateFile=true|false (BETA - default=true)
ResilientWatchCacheInitialization=true|false (BETA - default=true)
ResourceHealthStatus=true|false (ALPHA - default=false)
RetryGenerateName=true|false (BETA - default=true)
RotateKubeletServerCertificate=true|false (BETA - default=true)
RuntimeClassInImageCriApi=true|false (ALPHA - default=false)
SELinuxMount=true|false (ALPHA - default=false)
SELinuxMountReadWriteOncePod=true|false (BETA - default=true)
SchedulerQueueingHints=true|false (BETA - default=false)
SeparateCacheWatchRPC=true|false (BETA - default=true)
SeparateTaintEvictionController=true|false (BETA - default=true)
ServiceAccountTokenJTI=true|false (BETA - default=true)
ServiceAccountTokenNodeBinding=true|false (BETA - default=true)
ServiceAccountTokenNodeBindingValidation=true|false (BETA - default=true)
ServiceAccountTokenPodNodeInfo=true|false (BETA - default=true)
ServiceTrafficDistribution=true|false (BETA - default=true)
SidecarContainers=true|false (BETA - default=true)
SizeMemoryBackedVolumes=true|false (BETA - default=true)
StatefulSetAutoDeletePVC=true|false (BETA - default=true)
StorageNamespaceIndex=true|false (BETA - default=true)
StorageVersionAPI=true|false (ALPHA - default=false)
StorageVersionHash=true|false (BETA - default=true)
StorageVersionMigrator=true|false (ALPHA - default=false)
StrictCostEnforcementForVAP=true|false (BETA - default=false)
StrictCostEnforcementForWebhooks=true|false (BETA - default=false)
StructuredAuthenticationConfiguration=true|false (BETA - default=true)
StructuredAuthorizationConfiguration=true|false (BETA - default=true)
SupplementalGroupsPolicy=true|false (ALPHA - default=false)
TopologyAwareHints=true|false (BETA - default=true)
TopologyManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
TopologyManagerPolicyBetaOptions=true|false (BETA - default=true)
TopologyManagerPolicyOptions=true|false (BETA - default=true)
TranslateStreamCloseWebsocketRequests=true|false (BETA - default=true)
UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true)
UnknownVersionInteroperabilityProxy=true|false (ALPHA - default=false)
UserNamespacesPodSecurityStandards=true|false (ALPHA - default=false)
UserNamespacesSupport=true|false (BETA - default=false)
VolumeAttributesClass=true|false (BETA - default=false)
VolumeCapacityPriority=true|false (ALPHA - default=false)
WatchCacheInitializationPostStartHook=true|false (BETA - default=false)
WatchFromStorageWithoutResourceVersion=true|false (BETA - default=false)
WatchList=true|false (ALPHA - default=false)
WatchListClient=true|false (BETA - default=false)
WinDSR=true|false (ALPHA - default=false)
WinOverlay=true|false (BETA - default=true)
WindowsHostNetwork=true|false (ALPHA - default=true)
This parameter is ignored if a config file is specified by --config.

+

A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
APIResponseCompression=true|false (BETA - default=true)
APIServerIdentity=true|false (BETA - default=true)
APIServerTracing=true|false (BETA - default=true)
APIServingWithRoutine=true|false (ALPHA - default=false)
AllAlpha=true|false (ALPHA - default=false)
AllBeta=true|false (BETA - default=false)
AllowUnsafeMalformedObjectDeletion=true|false (ALPHA - default=false)
AnonymousAuthConfigurableEndpoints=true|false (BETA - default=true)
AnyVolumeDataSource=true|false (BETA - default=true)
AuthorizeNodeWithSelectors=true|false (BETA - default=true)
AuthorizeWithSelectors=true|false (BETA - default=true)
BtreeWatchCache=true|false (BETA - default=true)
CBORServingAndStorage=true|false (ALPHA - default=false)
CPUManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
CPUManagerPolicyBetaOptions=true|false (BETA - default=true)
CPUManagerPolicyOptions=true|false (BETA - default=true)
CRDValidationRatcheting=true|false (BETA - default=true)
CSIMigrationPortworx=true|false (BETA - default=true)
CSIVolumeHealth=true|false (ALPHA - default=false)
ClientsAllowCBOR=true|false (ALPHA - default=false)
ClientsPreferCBOR=true|false (ALPHA - default=false)
CloudControllerManagerWebhook=true|false (ALPHA - default=false)
ClusterTrustBundle=true|false (ALPHA - default=false)
ClusterTrustBundleProjection=true|false (ALPHA - default=false)
ComponentFlagz=true|false (ALPHA - default=false)
ComponentStatusz=true|false (ALPHA - default=false)
ConcurrentWatchObjectDecode=true|false (BETA - default=false)
ConsistentListFromCache=true|false (BETA - default=true)
ContainerCheckpoint=true|false (BETA - default=true)
ContextualLogging=true|false (BETA - default=true)
CoordinatedLeaderElection=true|false (ALPHA - default=false)
CrossNamespaceVolumeDataSource=true|false (ALPHA - default=false)
CustomCPUCFSQuotaPeriod=true|false (ALPHA - default=false)
DRAAdminAccess=true|false (ALPHA - default=false)
DRAResourceClaimDeviceStatus=true|false (ALPHA - default=false)
DisableAllocatorDualWrite=true|false (ALPHA - default=false)
DynamicResourceAllocation=true|false (BETA - default=false)
EventedPLEG=true|false (ALPHA - default=false)
ExternalServiceAccountTokenSigner=true|false (ALPHA - default=false)
GracefulNodeShutdown=true|false (BETA - default=true)
GracefulNodeShutdownBasedOnPodPriority=true|false (BETA - default=true)
HPAScaleToZero=true|false (ALPHA - default=false)
HonorPVReclaimPolicy=true|false (BETA - default=true)
ImageMaximumGCAge=true|false (BETA - default=true)
ImageVolume=true|false (ALPHA - default=false)
InPlacePodVerticalScaling=true|false (ALPHA - default=false)
InPlacePodVerticalScalingAllocatedStatus=true|false (ALPHA - default=false)
InPlacePodVerticalScalingExclusiveCPUs=true|false (ALPHA - default=false)
InTreePluginPortworxUnregister=true|false (ALPHA - default=false)
InformerResourceVersion=true|false (ALPHA - default=false)
JobBackoffLimitPerIndex=true|false (BETA - default=true)
JobManagedBy=true|false (BETA - default=true)
JobPodReplacementPolicy=true|false (BETA - default=true)
JobSuccessPolicy=true|false (BETA - default=true)
KubeletCgroupDriverFromCRI=true|false (BETA - default=true)
KubeletCrashLoopBackOffMax=true|false (ALPHA - default=false)
KubeletFineGrainedAuthz=true|false (ALPHA - default=false)
KubeletInUserNamespace=true|false (ALPHA - default=false)
KubeletPodResourcesDynamicResources=true|false (ALPHA - default=false)
KubeletPodResourcesGet=true|false (ALPHA - default=false)
KubeletSeparateDiskGC=true|false (BETA - default=true)
KubeletTracing=true|false (BETA - default=true)
LocalStorageCapacityIsolationFSQuotaMonitoring=true|false (BETA - default=false)
LoggingAlphaOptions=true|false (ALPHA - default=false)
LoggingBetaOptions=true|false (BETA - default=true)
MatchLabelKeysInPodAffinity=true|false (BETA - default=true)
MatchLabelKeysInPodTopologySpread=true|false (BETA - default=true)
MaxUnavailableStatefulSet=true|false (ALPHA - default=false)
MemoryQoS=true|false (ALPHA - default=false)
MultiCIDRServiceAllocator=true|false (BETA - default=false)
MutatingAdmissionPolicy=true|false (ALPHA - default=false)
NFTablesProxyMode=true|false (BETA - default=true)
NodeInclusionPolicyInPodTopologySpread=true|false (BETA - default=true)
NodeLogQuery=true|false (BETA - default=false)
NodeSwap=true|false (BETA - default=true)
OpenAPIEnums=true|false (BETA - default=true)
PodAndContainerStatsFromCRI=true|false (ALPHA - default=false)
PodDeletionCost=true|false (BETA - default=true)
PodLevelResources=true|false (ALPHA - default=false)
PodLifecycleSleepAction=true|false (BETA - default=true)
PodLifecycleSleepActionAllowZero=true|false (ALPHA - default=false)
PodLogsQuerySplitStreams=true|false (ALPHA - default=false)
PodReadyToStartContainersCondition=true|false (BETA - default=true)
PortForwardWebsockets=true|false (BETA - default=true)
ProcMountType=true|false (BETA - default=false)
QOSReserved=true|false (ALPHA - default=false)
RecoverVolumeExpansionFailure=true|false (BETA - default=true)
RecursiveReadOnlyMounts=true|false (BETA - default=true)
RelaxedDNSSearchValidation=true|false (ALPHA - default=false)
RelaxedEnvironmentVariableValidation=true|false (BETA - default=true)
ReloadKubeletServerCertificateFile=true|false (BETA - default=true)
RemoteRequestHeaderUID=true|false (ALPHA - default=false)
ResilientWatchCacheInitialization=true|false (BETA - default=true)
ResourceHealthStatus=true|false (ALPHA - default=false)
RotateKubeletServerCertificate=true|false (BETA - default=true)
RuntimeClassInImageCriApi=true|false (ALPHA - default=false)
SELinuxChangePolicy=true|false (ALPHA - default=false)
SELinuxMount=true|false (ALPHA - default=false)
SELinuxMountReadWriteOncePod=true|false (BETA - default=true)
SchedulerAsyncPreemption=true|false (ALPHA - default=false)
SchedulerQueueingHints=true|false (BETA - default=true)
SeparateCacheWatchRPC=true|false (BETA - default=true)
SeparateTaintEvictionController=true|false (BETA - default=true)
ServiceAccountNodeAudienceRestriction=true|false (BETA - default=true)
ServiceAccountTokenNodeBinding=true|false (BETA - default=true)
ServiceTrafficDistribution=true|false (BETA - default=true)
SidecarContainers=true|false (BETA - default=true)
StorageNamespaceIndex=true|false (BETA - default=true)
StorageVersionAPI=true|false (ALPHA - default=false)
StorageVersionHash=true|false (BETA - default=true)
StorageVersionMigrator=true|false (ALPHA - default=false)
StructuredAuthenticationConfiguration=true|false (BETA - default=true)
SupplementalGroupsPolicy=true|false (ALPHA - default=false)
SystemdWatchdog=true|false (BETA - default=true)
TopologyAwareHints=true|false (BETA - default=true)
TopologyManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
TopologyManagerPolicyBetaOptions=true|false (BETA - default=true)
TranslateStreamCloseWebsocketRequests=true|false (BETA - default=true)
UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true)
UnknownVersionInteroperabilityProxy=true|false (ALPHA - default=false)
UserNamespacesPodSecurityStandards=true|false (ALPHA - default=false)
UserNamespacesSupport=true|false (BETA - default=false)
VolumeAttributesClass=true|false (BETA - default=false)
VolumeCapacityPriority=true|false (ALPHA - default=false)
WatchCacheInitializationPostStartHook=true|false (BETA - default=false)
WatchFromStorageWithoutResourceVersion=true|false (BETA - default=false)
WatchList=true|false (BETA - default=true)
WatchListClient=true|false (BETA - default=false)
WinDSR=true|false (ALPHA - default=false)
WinOverlay=true|false (BETA - default=true)
WindowsCPUAndMemoryAffinity=true|false (ALPHA - default=false)
WindowsGracefulNodeShutdown=true|false (ALPHA - default=false)
WindowsHostNetwork=true|false (ALPHA - default=true)
This parameter is ignored if a config file is specified by --config.

diff --git a/content/en/docs/reference/command-line-tools-reference/kube-scheduler.md b/content/en/docs/reference/command-line-tools-reference/kube-scheduler.md index ab374b1242362..fc557f1d99ffa 100644 --- a/content/en/docs/reference/command-line-tools-reference/kube-scheduler.md +++ b/content/en/docs/reference/command-line-tools-reference/kube-scheduler.md @@ -166,14 +166,14 @@ kube-scheduler [flags] --emulated-version strings -

The versions different components emulate their capabilities (APIs, features, ...) of.
If set, the component will emulate the behavior of this version instead of the underlying binary version.
Version format could only be major.minor, for example: '--emulated-version=wardle=1.2,kube=1.31'. Options are:
kube=1.31..1.31 (default=1.31)If the component is not specified, defaults to "kube"

+

The versions different components emulate their capabilities (APIs, features, ...) of.
If set, the component will emulate the behavior of this version instead of the underlying binary version.
Version format could only be major.minor, for example: '--emulated-version=wardle=1.2,kube=1.31'. Options are:
kube=1.32..1.32 (default=1.32)If the component is not specified, defaults to "kube"

--feature-gates colonSeparatedMultimapStringString -

Comma-separated list of component:key=value pairs that describe feature gates for alpha/experimental features of different components.
If the component is not specified, defaults to "kube". This flag can be repeatedly invoked. For example: --feature-gates 'wardle:featureA=true,wardle:featureB=false' --feature-gates 'kube:featureC=true'Options are:
kube:APIResponseCompression=true|false (BETA - default=true)
kube:APIServerIdentity=true|false (BETA - default=true)
kube:APIServerTracing=true|false (BETA - default=true)
kube:APIServingWithRoutine=true|false (ALPHA - default=false)
kube:AllAlpha=true|false (ALPHA - default=false)
kube:AllBeta=true|false (BETA - default=false)
kube:AnonymousAuthConfigurableEndpoints=true|false (ALPHA - default=false)
kube:AnyVolumeDataSource=true|false (BETA - default=true)
kube:AuthorizeNodeWithSelectors=true|false (ALPHA - default=false)
kube:AuthorizeWithSelectors=true|false (ALPHA - default=false)
kube:CPUManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
kube:CPUManagerPolicyBetaOptions=true|false (BETA - default=true)
kube:CPUManagerPolicyOptions=true|false (BETA - default=true)
kube:CRDValidationRatcheting=true|false (BETA - default=true)
kube:CSIMigrationPortworx=true|false (BETA - default=true)
kube:CSIVolumeHealth=true|false (ALPHA - default=false)
kube:CloudControllerManagerWebhook=true|false (ALPHA - default=false)
kube:ClusterTrustBundle=true|false (ALPHA - default=false)
kube:ClusterTrustBundleProjection=true|false (ALPHA - default=false)
kube:ComponentSLIs=true|false (BETA - default=true)
kube:ConcurrentWatchObjectDecode=true|false (BETA - default=false)
kube:ConsistentListFromCache=true|false (BETA - default=true)
kube:ContainerCheckpoint=true|false (BETA - default=true)
kube:ContextualLogging=true|false (BETA - default=true)
kube:CoordinatedLeaderElection=true|false (ALPHA - default=false)
kube:CronJobsScheduledAnnotation=true|false (BETA - default=true)
kube:CrossNamespaceVolumeDataSource=true|false (ALPHA - default=false)
kube:CustomCPUCFSQuotaPeriod=true|false (ALPHA - default=false)
kube:CustomResourceFieldSelectors=true|false (BETA - default=true)
kube:DRAControlPlaneController=true|false (ALPHA - default=false)
kube:DisableAllocatorDualWrite=true|false (ALPHA - default=false)
kube:DisableNodeKubeProxyVersion=true|false (BETA - default=true)
kube:DynamicResourceAllocation=true|false (ALPHA - default=false)
kube:EventedPLEG=true|false (ALPHA - default=false)
kube:GracefulNodeShutdown=true|false (BETA - default=true)
kube:GracefulNodeShutdownBasedOnPodPriority=true|false (BETA - default=true)
kube:HPAScaleToZero=true|false (ALPHA - default=false)
kube:HonorPVReclaimPolicy=true|false (BETA - default=true)
kube:ImageMaximumGCAge=true|false (BETA - default=true)
kube:ImageVolume=true|false (ALPHA - default=false)
kube:InPlacePodVerticalScaling=true|false (ALPHA - default=false)
kube:InTreePluginPortworxUnregister=true|false (ALPHA - default=false)
kube:InformerResourceVersion=true|false (ALPHA - default=false)
kube:JobBackoffLimitPerIndex=true|false (BETA - default=true)
kube:JobManagedBy=true|false (ALPHA - default=false)
kube:JobPodReplacementPolicy=true|false (BETA - default=true)
kube:JobSuccessPolicy=true|false (BETA - default=true)
kube:KubeletCgroupDriverFromCRI=true|false (BETA - default=true)
kube:KubeletInUserNamespace=true|false (ALPHA - default=false)
kube:KubeletPodResourcesDynamicResources=true|false (ALPHA - default=false)
kube:KubeletPodResourcesGet=true|false (ALPHA - default=false)
kube:KubeletSeparateDiskGC=true|false (BETA - default=true)
kube:KubeletTracing=true|false (BETA - default=true)
kube:LoadBalancerIPMode=true|false (BETA - default=true)
kube:LocalStorageCapacityIsolationFSQuotaMonitoring=true|false (BETA - default=false)
kube:LoggingAlphaOptions=true|false (ALPHA - default=false)
kube:LoggingBetaOptions=true|false (BETA - default=true)
kube:MatchLabelKeysInPodAffinity=true|false (BETA - default=true)
kube:MatchLabelKeysInPodTopologySpread=true|false (BETA - default=true)
kube:MaxUnavailableStatefulSet=true|false (ALPHA - default=false)
kube:MemoryManager=true|false (BETA - default=true)
kube:MemoryQoS=true|false (ALPHA - default=false)
kube:MultiCIDRServiceAllocator=true|false (BETA - default=false)
kube:MutatingAdmissionPolicy=true|false (ALPHA - default=false)
kube:NFTablesProxyMode=true|false (BETA - default=true)
kube:NodeInclusionPolicyInPodTopologySpread=true|false (BETA - default=true)
kube:NodeLogQuery=true|false (BETA - default=false)
kube:NodeSwap=true|false (BETA - default=true)
kube:OpenAPIEnums=true|false (BETA - default=true)
kube:PodAndContainerStatsFromCRI=true|false (ALPHA - default=false)
kube:PodDeletionCost=true|false (BETA - default=true)
kube:PodIndexLabel=true|false (BETA - default=true)
kube:PodLifecycleSleepAction=true|false (BETA - default=true)
kube:PodReadyToStartContainersCondition=true|false (BETA - default=true)
kube:PortForwardWebsockets=true|false (BETA - default=true)
kube:ProcMountType=true|false (BETA - default=false)
kube:QOSReserved=true|false (ALPHA - default=false)
kube:RecoverVolumeExpansionFailure=true|false (ALPHA - default=false)
kube:RecursiveReadOnlyMounts=true|false (BETA - default=true)
kube:RelaxedEnvironmentVariableValidation=true|false (ALPHA - default=false)
kube:ReloadKubeletServerCertificateFile=true|false (BETA - default=true)
kube:ResilientWatchCacheInitialization=true|false (BETA - default=true)
kube:ResourceHealthStatus=true|false (ALPHA - default=false)
kube:RetryGenerateName=true|false (BETA - default=true)
kube:RotateKubeletServerCertificate=true|false (BETA - default=true)
kube:RuntimeClassInImageCriApi=true|false (ALPHA - default=false)
kube:SELinuxMount=true|false (ALPHA - default=false)
kube:SELinuxMountReadWriteOncePod=true|false (BETA - default=true)
kube:SchedulerQueueingHints=true|false (BETA - default=false)
kube:SeparateCacheWatchRPC=true|false (BETA - default=true)
kube:SeparateTaintEvictionController=true|false (BETA - default=true)
kube:ServiceAccountTokenJTI=true|false (BETA - default=true)
kube:ServiceAccountTokenNodeBinding=true|false (BETA - default=true)
kube:ServiceAccountTokenNodeBindingValidation=true|false (BETA - default=true)
kube:ServiceAccountTokenPodNodeInfo=true|false (BETA - default=true)
kube:ServiceTrafficDistribution=true|false (BETA - default=true)
kube:SidecarContainers=true|false (BETA - default=true)
kube:SizeMemoryBackedVolumes=true|false (BETA - default=true)
kube:StatefulSetAutoDeletePVC=true|false (BETA - default=true)
kube:StorageNamespaceIndex=true|false (BETA - default=true)
kube:StorageVersionAPI=true|false (ALPHA - default=false)
kube:StorageVersionHash=true|false (BETA - default=true)
kube:StorageVersionMigrator=true|false (ALPHA - default=false)
kube:StrictCostEnforcementForVAP=true|false (BETA - default=false)
kube:StrictCostEnforcementForWebhooks=true|false (BETA - default=false)
kube:StructuredAuthenticationConfiguration=true|false (BETA - default=true)
kube:StructuredAuthorizationConfiguration=true|false (BETA - default=true)
kube:SupplementalGroupsPolicy=true|false (ALPHA - default=false)
kube:TopologyAwareHints=true|false (BETA - default=true)
kube:TopologyManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
kube:TopologyManagerPolicyBetaOptions=true|false (BETA - default=true)
kube:TopologyManagerPolicyOptions=true|false (BETA - default=true)
kube:TranslateStreamCloseWebsocketRequests=true|false (BETA - default=true)
kube:UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true)
kube:UnknownVersionInteroperabilityProxy=true|false (ALPHA - default=false)
kube:UserNamespacesPodSecurityStandards=true|false (ALPHA - default=false)
kube:UserNamespacesSupport=true|false (BETA - default=false)
kube:VolumeAttributesClass=true|false (BETA - default=false)
kube:VolumeCapacityPriority=true|false (ALPHA - default=false)
kube:WatchCacheInitializationPostStartHook=true|false (BETA - default=false)
kube:WatchFromStorageWithoutResourceVersion=true|false (BETA - default=false)
kube:WatchList=true|false (ALPHA - default=false)
kube:WatchListClient=true|false (BETA - default=false)
kube:WinDSR=true|false (ALPHA - default=false)
kube:WinOverlay=true|false (BETA - default=true)
kube:WindowsHostNetwork=true|false (ALPHA - default=true)

+

Comma-separated list of component:key=value pairs that describe feature gates for alpha/experimental features of different components.
If the component is not specified, defaults to "kube". This flag can be repeatedly invoked. For example: --feature-gates 'wardle:featureA=true,wardle:featureB=false' --feature-gates 'kube:featureC=true'Options are:
kube:APIResponseCompression=true|false (BETA - default=true)
kube:APIServerIdentity=true|false (BETA - default=true)
kube:APIServerTracing=true|false (BETA - default=true)
kube:APIServingWithRoutine=true|false (ALPHA - default=false)
kube:AllAlpha=true|false (ALPHA - default=false)
kube:AllBeta=true|false (BETA - default=false)
kube:AllowUnsafeMalformedObjectDeletion=true|false (ALPHA - default=false)
kube:AnonymousAuthConfigurableEndpoints=true|false (BETA - default=true)
kube:AnyVolumeDataSource=true|false (BETA - default=true)
kube:AuthorizeNodeWithSelectors=true|false (BETA - default=true)
kube:AuthorizeWithSelectors=true|false (BETA - default=true)
kube:BtreeWatchCache=true|false (BETA - default=true)
kube:CBORServingAndStorage=true|false (ALPHA - default=false)
kube:CPUManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
kube:CPUManagerPolicyBetaOptions=true|false (BETA - default=true)
kube:CPUManagerPolicyOptions=true|false (BETA - default=true)
kube:CRDValidationRatcheting=true|false (BETA - default=true)
kube:CSIMigrationPortworx=true|false (BETA - default=true)
kube:CSIVolumeHealth=true|false (ALPHA - default=false)
kube:ClientsAllowCBOR=true|false (ALPHA - default=false)
kube:ClientsPreferCBOR=true|false (ALPHA - default=false)
kube:CloudControllerManagerWebhook=true|false (ALPHA - default=false)
kube:ClusterTrustBundle=true|false (ALPHA - default=false)
kube:ClusterTrustBundleProjection=true|false (ALPHA - default=false)
kube:ComponentFlagz=true|false (ALPHA - default=false)
kube:ComponentStatusz=true|false (ALPHA - default=false)
kube:ConcurrentWatchObjectDecode=true|false (BETA - default=false)
kube:ConsistentListFromCache=true|false (BETA - default=true)
kube:ContainerCheckpoint=true|false (BETA - default=true)
kube:ContextualLogging=true|false (BETA - default=true)
kube:CoordinatedLeaderElection=true|false (ALPHA - default=false)
kube:CrossNamespaceVolumeDataSource=true|false (ALPHA - default=false)
kube:CustomCPUCFSQuotaPeriod=true|false (ALPHA - default=false)
kube:DRAAdminAccess=true|false (ALPHA - default=false)
kube:DRAResourceClaimDeviceStatus=true|false (ALPHA - default=false)
kube:DisableAllocatorDualWrite=true|false (ALPHA - default=false)
kube:DynamicResourceAllocation=true|false (BETA - default=false)
kube:EventedPLEG=true|false (ALPHA - default=false)
kube:ExternalServiceAccountTokenSigner=true|false (ALPHA - default=false)
kube:GracefulNodeShutdown=true|false (BETA - default=true)
kube:GracefulNodeShutdownBasedOnPodPriority=true|false (BETA - default=true)
kube:HPAScaleToZero=true|false (ALPHA - default=false)
kube:HonorPVReclaimPolicy=true|false (BETA - default=true)
kube:ImageMaximumGCAge=true|false (BETA - default=true)
kube:ImageVolume=true|false (ALPHA - default=false)
kube:InPlacePodVerticalScaling=true|false (ALPHA - default=false)
kube:InPlacePodVerticalScalingAllocatedStatus=true|false (ALPHA - default=false)
kube:InPlacePodVerticalScalingExclusiveCPUs=true|false (ALPHA - default=false)
kube:InTreePluginPortworxUnregister=true|false (ALPHA - default=false)
kube:InformerResourceVersion=true|false (ALPHA - default=false)
kube:JobBackoffLimitPerIndex=true|false (BETA - default=true)
kube:JobManagedBy=true|false (BETA - default=true)
kube:JobPodReplacementPolicy=true|false (BETA - default=true)
kube:JobSuccessPolicy=true|false (BETA - default=true)
kube:KubeletCgroupDriverFromCRI=true|false (BETA - default=true)
kube:KubeletCrashLoopBackOffMax=true|false (ALPHA - default=false)
kube:KubeletFineGrainedAuthz=true|false (ALPHA - default=false)
kube:KubeletInUserNamespace=true|false (ALPHA - default=false)
kube:KubeletPodResourcesDynamicResources=true|false (ALPHA - default=false)
kube:KubeletPodResourcesGet=true|false (ALPHA - default=false)
kube:KubeletSeparateDiskGC=true|false (BETA - default=true)
kube:KubeletTracing=true|false (BETA - default=true)
kube:LocalStorageCapacityIsolationFSQuotaMonitoring=true|false (BETA - default=false)
kube:LoggingAlphaOptions=true|false (ALPHA - default=false)
kube:LoggingBetaOptions=true|false (BETA - default=true)
kube:MatchLabelKeysInPodAffinity=true|false (BETA - default=true)
kube:MatchLabelKeysInPodTopologySpread=true|false (BETA - default=true)
kube:MaxUnavailableStatefulSet=true|false (ALPHA - default=false)
kube:MemoryQoS=true|false (ALPHA - default=false)
kube:MultiCIDRServiceAllocator=true|false (BETA - default=false)
kube:MutatingAdmissionPolicy=true|false (ALPHA - default=false)
kube:NFTablesProxyMode=true|false (BETA - default=true)
kube:NodeInclusionPolicyInPodTopologySpread=true|false (BETA - default=true)
kube:NodeLogQuery=true|false (BETA - default=false)
kube:NodeSwap=true|false (BETA - default=true)
kube:OpenAPIEnums=true|false (BETA - default=true)
kube:PodAndContainerStatsFromCRI=true|false (ALPHA - default=false)
kube:PodDeletionCost=true|false (BETA - default=true)
kube:PodLevelResources=true|false (ALPHA - default=false)
kube:PodLifecycleSleepAction=true|false (BETA - default=true)
kube:PodLifecycleSleepActionAllowZero=true|false (ALPHA - default=false)
kube:PodLogsQuerySplitStreams=true|false (ALPHA - default=false)
kube:PodReadyToStartContainersCondition=true|false (BETA - default=true)
kube:PortForwardWebsockets=true|false (BETA - default=true)
kube:ProcMountType=true|false (BETA - default=false)
kube:QOSReserved=true|false (ALPHA - default=false)
kube:RecoverVolumeExpansionFailure=true|false (BETA - default=true)
kube:RecursiveReadOnlyMounts=true|false (BETA - default=true)
kube:RelaxedDNSSearchValidation=true|false (ALPHA - default=false)
kube:RelaxedEnvironmentVariableValidation=true|false (BETA - default=true)
kube:ReloadKubeletServerCertificateFile=true|false (BETA - default=true)
kube:RemoteRequestHeaderUID=true|false (ALPHA - default=false)
kube:ResilientWatchCacheInitialization=true|false (BETA - default=true)
kube:ResourceHealthStatus=true|false (ALPHA - default=false)
kube:RotateKubeletServerCertificate=true|false (BETA - default=true)
kube:RuntimeClassInImageCriApi=true|false (ALPHA - default=false)
kube:SELinuxChangePolicy=true|false (ALPHA - default=false)
kube:SELinuxMount=true|false (ALPHA - default=false)
kube:SELinuxMountReadWriteOncePod=true|false (BETA - default=true)
kube:SchedulerAsyncPreemption=true|false (ALPHA - default=false)
kube:SchedulerQueueingHints=true|false (BETA - default=true)
kube:SeparateCacheWatchRPC=true|false (BETA - default=true)
kube:SeparateTaintEvictionController=true|false (BETA - default=true)
kube:ServiceAccountNodeAudienceRestriction=true|false (BETA - default=true)
kube:ServiceAccountTokenNodeBinding=true|false (BETA - default=true)
kube:ServiceTrafficDistribution=true|false (BETA - default=true)
kube:SidecarContainers=true|false (BETA - default=true)
kube:StorageNamespaceIndex=true|false (BETA - default=true)
kube:StorageVersionAPI=true|false (ALPHA - default=false)
kube:StorageVersionHash=true|false (BETA - default=true)
kube:StorageVersionMigrator=true|false (ALPHA - default=false)
kube:StructuredAuthenticationConfiguration=true|false (BETA - default=true)
kube:SupplementalGroupsPolicy=true|false (ALPHA - default=false)
kube:SystemdWatchdog=true|false (BETA - default=true)
kube:TopologyAwareHints=true|false (BETA - default=true)
kube:TopologyManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
kube:TopologyManagerPolicyBetaOptions=true|false (BETA - default=true)
kube:TranslateStreamCloseWebsocketRequests=true|false (BETA - default=true)
kube:UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true)
kube:UnknownVersionInteroperabilityProxy=true|false (ALPHA - default=false)
kube:UserNamespacesPodSecurityStandards=true|false (ALPHA - default=false)
kube:UserNamespacesSupport=true|false (BETA - default=false)
kube:VolumeAttributesClass=true|false (BETA - default=false)
kube:VolumeCapacityPriority=true|false (ALPHA - default=false)
kube:WatchCacheInitializationPostStartHook=true|false (BETA - default=false)
kube:WatchFromStorageWithoutResourceVersion=true|false (BETA - default=false)
kube:WatchList=true|false (BETA - default=true)
kube:WatchListClient=true|false (BETA - default=false)
kube:WinDSR=true|false (ALPHA - default=false)
kube:WinOverlay=true|false (BETA - default=true)
kube:WindowsCPUAndMemoryAffinity=true|false (ALPHA - default=false)
kube:WindowsGracefulNodeShutdown=true|false (ALPHA - default=false)
kube:WindowsHostNetwork=true|false (ALPHA - default=true)

@@ -243,7 +243,7 @@ kube-scheduler [flags] --leader-elect-resource-lock string     Default: "leases" -

The type of resource object that is used for locking during leader election. Supported options are 'leases', 'endpointsleases' and 'configmapsleases'.

+

The type of resource object that is used for locking during leader election. Supported options are 'leases'.

@@ -358,6 +358,13 @@ kube-scheduler [flags]

List of request headers to inspect for groups. X-Remote-Group is suggested.

+ +--requestheader-uid-headers strings + + +

List of request headers to inspect for UIDs. X-Remote-Uid is suggested. Requires the RemoteRequestHeaderUID feature to be enabled.

+ + --requestheader-username-headers strings     Default: "x-remote-user" diff --git a/content/en/docs/reference/command-line-tools-reference/kubelet.md b/content/en/docs/reference/command-line-tools-reference/kubelet.md index f410293c12e4a..af613679a0b71 100644 --- a/content/en/docs/reference/command-line-tools-reference/kubelet.md +++ b/content/en/docs/reference/command-line-tools-reference/kubelet.md @@ -346,6 +346,13 @@ kubelet [flags] [Experimental] Path of mounter binary. Leave empty to use the default mount. (DEPRECATED: will be removed in 1.24 or later, in favor of using CSI.) + +--fail-cgroupv1     Default: true + + +Prevent the kubelet from starting on the host using cgroup v1. + + --fail-swap-on     Default: true @@ -365,129 +372,132 @@ APIServerTracing=true|false (BETA - default=true)
APIServingWithRoutine=true|false (BETA - default=true)
AllAlpha=true|false (ALPHA - default=false)
AllBeta=true|false (BETA - default=false)
+AllowUnsafeMalformedObjectDeletion=true|false (ALPHA - default=false)
+AnonymousAuthConfigurableEndpoints=true|false (BETA - default=true)
AnyVolumeDataSource=true|false (BETA - default=true)
-AppArmor=true|false (BETA - default=true)
-AppArmorFields=true|false (BETA - default=true)
+AuthorizeNodeWithSelectors=true|false (BETA - default=true)
+AuthorizeWithSelectors=true|false (BETA - default=true)
+BtreeWatchCache=true|false (BETA - default=true)
+CBORServingAndStorage=true|false (ALPHA - default=false)
CPUManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
CPUManagerPolicyBetaOptions=true|false (BETA - default=true)
CPUManagerPolicyOptions=true|false (BETA - default=true)
CRDValidationRatcheting=true|false (BETA - default=true)
CSIMigrationPortworx=true|false (BETA - default=false)
CSIVolumeHealth=true|false (ALPHA - default=false)
+ClientsAllowCBOR=true|false (ALPHA - default=false)
+ClientsPreferCBOR=true|false (ALPHA - default=false)
CloudControllerManagerWebhook=true|false (ALPHA - default=false)
ClusterTrustBundle=true|false (ALPHA - default=false)
ClusterTrustBundleProjection=true|false (ALPHA - default=false)
-ComponentSLIs=true|false (BETA - default=true)
-ConsistentListFromCache=true|false (ALPHA - default=false)
+ComponentFlagz=true|false (ALPHA - default=false)
+ComponentStatusz=true|false (ALPHA - default=false)
+ConcurrentWatchObjectDecode=true|false (BETA - default=false)
+ConsistentListFromCache=true|false (BETA - default=true)
ContainerCheckpoint=true|false (BETA - default=true)
ContextualLogging=true|false (BETA - default=true)
-CronJobsScheduledAnnotation=true|false (BETA - default=true)
+CoordinatedLeaderElection=true|false (ALPHA - default=false)
CrossNamespaceVolumeDataSource=true|false (ALPHA - default=false)
CustomCPUCFSQuotaPeriod=true|false (ALPHA - default=false)
-CustomResourceFieldSelectors=true|false (ALPHA - default=false)
-DevicePluginCDIDevices=true|false (BETA - default=true)
-DisableCloudProviders=true|false (BETA - default=true)
-DisableKubeletCloudCredentialProviders=true|false (BETA - default=true)
-DisableNodeKubeProxyVersion=true|false (ALPHA - default=false)
-DynamicResourceAllocation=true|false (ALPHA - default=false)
-ElasticIndexedJob=true|false (BETA - default=true)
+DRAAdminAccess=true|false (ALPHA - default=false)
+DRAResourceClaimDeviceStatus=true|false (ALPHA - default=false)
+DisableAllocatorDualWrite=true|false (ALPHA - default=false) +DynamicResourceAllocation=true|false (BETA - default=false)
EventedPLEG=true|false (ALPHA - default=false)
+ExternalServiceAccountTokenSigner=true|false (ALPHA - default=false)
GracefulNodeShutdown=true|false (BETA - default=true)
GracefulNodeShutdownBasedOnPodPriority=true|false (BETA - default=true)
HPAScaleToZero=true|false (ALPHA - default=false)
-HonorPVReclaimPolicy=true|false (ALPHA - default=false)
+HonorPVReclaimPolicy=true|false (BETA - default=true)
ImageMaximumGCAge=true|false (BETA - default=true)
+ImageVolume=true|false (ALPHA - default=false)
InPlacePodVerticalScaling=true|false (ALPHA - default=false)
-InTreePluginAWSUnregister=true|false (ALPHA - default=false)
-InTreePluginAzureDiskUnregister=true|false (ALPHA - default=false)
-InTreePluginAzureFileUnregister=true|false (ALPHA - default=false)
-InTreePluginGCEUnregister=true|false (ALPHA - default=false)
-InTreePluginOpenStackUnregister=true|false (ALPHA - default=false)
+InPlacePodVerticalScalingAllocatedStatus=true|false (ALPHA - default=false)
+InPlacePodVerticalScalingExclusiveCPUs=true|false (ALPHA - default=false)
InTreePluginPortworxUnregister=true|false (ALPHA - default=false)
-InTreePluginvSphereUnregister=true|false (ALPHA - default=false)
InformerResourceVersion=true|false (ALPHA - default=false)
JobBackoffLimitPerIndex=true|false (BETA - default=true)
JobManagedBy=true|false (ALPHA - default=false)
-JobPodFailurePolicy=true|false (BETA - default=true)
JobPodReplacementPolicy=true|false (BETA - default=true)
-JobSuccessPolicy=true|false (ALPHA - default=false)
-KubeProxyDrainingTerminatingNodes=true|false (BETA - default=true)
-KubeletCgroupDriverFromCRI=true|false (ALPHA - default=false)
+JobSuccessPolicy=true|false (BETA - default=true)
+KubeletCgroupDriverFromCRI=true|false (BETA - default=true)
+KubeletCrashLoopBackOffMax=true|false (ALPHA - default=false)
+KubeletFineGrainedAuthz=true|false (ALPHA - default=false)
KubeletInUserNamespace=true|false (ALPHA - default=false)
KubeletPodResourcesDynamicResources=true|false (ALPHA - default=false)
KubeletPodResourcesGet=true|false (ALPHA - default=false)
-KubeletSeparateDiskGC=true|false (ALPHA - default=false)
+KubeletSeparateDiskGC=true|false (BETA - default=true)
KubeletTracing=true|false (BETA - default=true)
-LoadBalancerIPMode=true|false (BETA - default=true)
-LocalStorageCapacityIsolationFSQuotaMonitoring=true|false (ALPHA - default=false)
-LogarithmicScaleDown=true|false (BETA - default=true)
+LocalStorageCapacityIsolationFSQuotaMonitoring=true|false (BETA - default=true)
LoggingAlphaOptions=true|false (ALPHA - default=false)
LoggingBetaOptions=true|false (BETA - default=true)
-MatchLabelKeysInPodAffinity=true|false (ALPHA - default=false)
+MatchLabelKeysInPodAffinity=true|false (BETA - default=true)
MatchLabelKeysInPodTopologySpread=true|false (BETA - default=true)
MaxUnavailableStatefulSet=true|false (ALPHA - default=false)
-MemoryManager=true|false (BETA - default=true)
MemoryQoS=true|false (ALPHA - default=false)
-MultiCIDRServiceAllocator=true|false (ALPHA - default=false)
+MultiCIDRServiceAllocator=true|false (BETA - default=false)
MutatingAdmissionPolicy=true|false (ALPHA - default=false)
-NFTablesProxyMode=true|false (ALPHA - default=false)
+NFTablesProxyMode=true|false (BETA - default=true)
NodeInclusionPolicyInPodTopologySpread=true|false (BETA - default=true)
NodeLogQuery=true|false (BETA - default=false)
NodeSwap=true|false (BETA - default=true)
OpenAPIEnums=true|false (BETA - default=true)
-PDBUnhealthyPodEvictionPolicy=true|false (BETA - default=true)
-PersistentVolumeLastPhaseTransitionTime=true|false (BETA - default=true)
PodAndContainerStatsFromCRI=true|false (ALPHA - default=false)
PodDeletionCost=true|false (BETA - default=true)
-PodDisruptionConditions=true|false (BETA - default=true)
-PodIndexLabel=true|false (BETA - default=true)
+PodLevelResources=true|false (ALPHA - default=false)
PodLifecycleSleepAction=true|false (BETA - default=true)
+PodLifecycleSleepActionAllowZero=true|false (ALPHA - default=false)
+PodLogsQuerySplitStreams=true|false (ALPHA - default=false)
PodReadyToStartContainersCondition=true|false (BETA - default=true)
-PortForwardWebsockets=true|false (ALPHA - default=false)
-ProcMountType=true|false (ALPHA - default=false)
+PortForwardWebsockets=true|false (BETA - default=true)
+ProcMountType=true|false (BETA - default=true)
QOSReserved=true|false (ALPHA - default=false)
-RecoverVolumeExpansionFailure=true|false (ALPHA - default=false)
-RecursiveReadOnlyMounts=true|false (ALPHA - default=false)
-RelaxedEnvironmentVariableValidation=true|false (ALPHA - default=false)
-RetryGenerateName=true|false (ALPHA - default=false)
+RecoverVolumeExpansionFailure=true|false (BETA - default=true)
+RecursiveReadOnlyMounts=true|false (BETA - default=true)
+RelaxedDNSSearchValidation=true|false (ALPHA - default=false)
+RelaxedEnvironmentVariableValidation=true|false (BETA - default=true)
+ReloadKubeletServerCertificateFile=true|false (BETA - default=true)
+RemoteRequestHeaderUID=true|false (ALPHA - default=false)
+ResilientWatchCacheInitialization=true|false (BETA - default=true)
+ResourceHealthStatus=true|false (ALPHA - default=false)
RotateKubeletServerCertificate=true|false (BETA - default=true)
RuntimeClassInImageCriApi=true|false (ALPHA - default=false)
+SELinuxChangePolicy=true|false (ALPHA - default=false)
SELinuxMount=true|false (ALPHA - default=false)
SELinuxMountReadWriteOncePod=true|false (BETA - default=true)
-SchedulerQueueingHints=true|false (BETA - default=false)
+SchedulerAsyncPreemption=true|false (ALPHA - default=false)
+SchedulerQueueingHints=true|false (BETA - default=true)
SeparateCacheWatchRPC=true|false (BETA - default=true)
SeparateTaintEvictionController=true|false (BETA - default=true)
-ServiceAccountTokenJTI=true|false (BETA - default=true)
-ServiceAccountTokenNodeBinding=true|false (ALPHA - default=false)
-ServiceAccountTokenNodeBindingValidation=true|false (BETA - default=true)
-ServiceAccountTokenPodNodeInfo=true|false (BETA - default=true)
-ServiceTrafficDistribution=true|false (ALPHA - default=false)
+ServiceAccountNodeAudienceRestriction=true|false (BETA - default=true)
+ServiceAccountTokenNodeBinding=true|false (BETA - default=true)
+ServiceTrafficDistribution=true|false (BETA - default=true)
SidecarContainers=true|false (BETA - default=true)
-SizeMemoryBackedVolumes=true|false (BETA - default=true)
-StatefulSetAutoDeletePVC=true|false (BETA - default=true)
-StatefulSetStartOrdinal=true|false (BETA - default=true)
StorageNamespaceIndex=true|false (BETA - default=true)
StorageVersionAPI=true|false (ALPHA - default=false)
StorageVersionHash=true|false (BETA - default=true)
StorageVersionMigrator=true|false (ALPHA - default=false)
StructuredAuthenticationConfiguration=true|false (BETA - default=true)
-StructuredAuthorizationConfiguration=true|false (BETA - default=true)
+SupplementalGroupsPolicy=true|false (ALPHA - default=false)
+SystemdWatchdog=true|false (BETA - default=true)
TopologyAwareHints=true|false (BETA - default=true)
TopologyManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
TopologyManagerPolicyBetaOptions=true|false (BETA - default=true)
-TopologyManagerPolicyOptions=true|false (BETA - default=true)
TranslateStreamCloseWebsocketRequests=true|false (BETA - default=true)
UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true)
UnknownVersionInteroperabilityProxy=true|false (ALPHA - default=false)
UserNamespacesPodSecurityStandards=true|false (ALPHA - default=false)
UserNamespacesSupport=true|false (BETA - default=false)
-VolumeAttributesClass=true|false (ALPHA - default=false)
+VolumeAttributesClass=true|false (BETA - default=false)
VolumeCapacityPriority=true|false (ALPHA - default=false)
+WatchCacheInitializationPostStartHook=true|false (BETA - default=false)
WatchFromStorageWithoutResourceVersion=true|false (BETA - default=false)
-WatchList=true|false (ALPHA - default=false)
+WatchList=true|false (BETA - default=true)
WatchListClient=true|false (BETA - default=false)
WinDSR=true|false (ALPHA - default=false)
WinOverlay=true|false (BETA - default=true)
+WindowsCPUAndMemoryAffinity=true|false (ALPHA - default=false)
+WindowsGracefulNodeShutdown=true|false (ALPHA - default=false)
WindowsHostNetwork=true|false (ALPHA - default=true)
(DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.) @@ -576,13 +586,6 @@ WindowsHostNetwork=true|false (ALPHA - default=true)
The endpoint of remote image service. If not specified, it will be the same with --container-runtime-endpoint by default. UNIX domain socket are supported on Linux, while `npipe` and `tcp` endpoints are supported on Windows. Examples: unix:///path/to/runtime.sock, npipe:////./pipe/runtime. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.) - ---keep-terminated-pod-volumes - - -Keep terminated pod volumes mounted to the node after the pod terminates. Can be useful for debugging volume related issues. (DEPRECATED: will be removed in a future version) - - --kernel-memcg-notification @@ -809,7 +812,7 @@ WindowsHostNetwork=true|false (ALPHA - default=true)
---pod-infra-container-image string     Default: registry.k8s.io/pause:3.9 +--pod-infra-container-image string     Default: registry.k8s.io/pause:3.10 Specified image will not be pruned by the image garbage collector. CRI implementations have their own configuration to set this image. (DEPRECATED: will be removed in 1.27. Image garbage collector will get sandbox image information from CRI.) diff --git a/content/en/docs/reference/config-api/apiserver-admission.v1.md b/content/en/docs/reference/config-api/apiserver-admission.v1.md index ebcf70b3a1df6..25021cea9c834 100644 --- a/content/en/docs/reference/config-api/apiserver-admission.v1.md +++ b/content/en/docs/reference/config-api/apiserver-admission.v1.md @@ -152,7 +152,7 @@ requested. e.g. a patch can result in either a CREATE or UPDATE Operation.

userInfo [Required]
-authentication/v1.UserInfo +authentication/v1.UserInfo

UserInfo is information about the requesting user

@@ -226,7 +226,7 @@ This must be copied over from the corresponding AdmissionRequest.

status
-meta/v1.Status +meta/v1.Status

Result contains extra details into why an admission request was denied. diff --git a/content/en/docs/reference/config-api/apiserver-audit.v1.md b/content/en/docs/reference/config-api/apiserver-audit.v1.md index 454ed9c77b11f..64704c55ecc2c 100644 --- a/content/en/docs/reference/config-api/apiserver-audit.v1.md +++ b/content/en/docs/reference/config-api/apiserver-audit.v1.md @@ -71,14 +71,14 @@ For non-resource requests, this is the lower-cased HTTP method.

user [Required]
-authentication/v1.UserInfo +authentication/v1.UserInfo

Authenticated user information.

impersonatedUser
-authentication/v1.UserInfo +authentication/v1.UserInfo

Impersonated user information.

@@ -116,7 +116,7 @@ Does not apply for List-type requests, or non-resource requests.

responseStatus
-meta/v1.Status +meta/v1.Status

The response status, populated even when the ResponseObject is not a Status type. @@ -144,14 +144,14 @@ at Response Level.

requestReceivedTimestamp
-meta/v1.MicroTime +meta/v1.MicroTime

Time the request reached the apiserver.

stageTimestamp
-meta/v1.MicroTime +meta/v1.MicroTime

Time the request reached current audit stage.

@@ -188,7 +188,7 @@ should be short. Annotations are included in the Metadata level.

metadata
-meta/v1.ListMeta +meta/v1.ListMeta No description provided. @@ -223,7 +223,7 @@ categories are logged.

metadata
-meta/v1.ObjectMeta +meta/v1.ObjectMeta

ObjectMeta is included for interoperability with API infrastructure.

@@ -278,7 +278,7 @@ in a rule will override the global default.

metadata
-meta/v1.ListMeta +meta/v1.ListMeta No description provided. diff --git a/content/en/docs/reference/config-api/apiserver-config.v1.md b/content/en/docs/reference/config-api/apiserver-config.v1.md index f9fab1f3fd108..dfd6833eac8a9 100644 --- a/content/en/docs/reference/config-api/apiserver-config.v1.md +++ b/content/en/docs/reference/config-api/apiserver-config.v1.md @@ -11,6 +11,7 @@ auto_generated: true - [AdmissionConfiguration](#apiserver-config-k8s-io-v1-AdmissionConfiguration) +- [AuthorizationConfiguration](#apiserver-config-k8s-io-v1-AuthorizationConfiguration) - [EncryptionConfiguration](#apiserver-config-k8s-io-v1-EncryptionConfiguration) @@ -39,14 +40,40 @@ auto_generated: true +## `AuthorizationConfiguration` {#apiserver-config-k8s-io-v1-AuthorizationConfiguration} + + + + + + + + + + + + + + + + +
FieldDescription
apiVersion
string		apiserver.config.k8s.io/v1
kind
string		AuthorizationConfiguration
authorizers [Required]
+[]AuthorizerConfiguration +		 +

Authorizers is an ordered list of authorizers to +authorize requests against. +This is similar to the --authorization-modes kube-apiserver flag +Must be at least one.

+
+ ## `EncryptionConfiguration` {#apiserver-config-k8s-io-v1-EncryptionConfiguration}

EncryptionConfiguration stores the complete configuration for encryption providers. It also allows the use of wildcards to specify the resources that should be encrypted. -Use '*.<group>' to encrypt all resources within a group or '*.*' to encrypt all resources. -'*.' can be used to encrypt all resource in the core group. '*.*' will encrypt all +Use '.' to encrypt all resources within a group or '.' to encrypt all resources. +'.' can be used to encrypt all resource in the core group. '.' will encrypt all resources, even custom resources that are added after API server start. Use of wildcards that overlap within the same resource list or across multiple entries are not allowed since part of the configuration would be ineffective. @@ -81,7 +108,8 @@ resources: - aescbc: keys: - name: key3 - secret: c2VjcmV0IGlzIHNlY3VyZSwgSSB0aGluaw== + secret: c2VjcmV0IGlzIHNlY3VyZSwgSSB0aGluaw== + @@ -172,6 +200,53 @@ configuration. If present, it will be used instead of the path to the configurat
+## `AuthorizerConfiguration` {#apiserver-config-k8s-io-v1-AuthorizerConfiguration} + + +**Appears in:** + +- [AuthorizationConfiguration](#apiserver-config-k8s-io-v1-AuthorizationConfiguration) + + + + + + + + + + + + + + + + + + +
FieldDescription
type [Required]
+string +		 +

Type refers to the type of the authorizer +"Webhook" is supported in the generic API server +Other API servers may support additional authorizer +types like Node, RBAC, ABAC, etc.

+
name [Required]
+string +		 +

Name used to describe the webhook +This is explicitly used in monitoring machinery for metrics +Note: Names must be DNS1123 labels like myauthorizername or +subdomains like myauthorizer.example.domain +Required, with no default

+
webhook [Required]
+WebhookConfiguration +		 +

Webhook defines the configuration for a Webhook authorizer +Must be defined when Type=Webhook +Must not be defined when Type!=Webhook

+
+ ## `IdentityConfiguration` {#apiserver-config-k8s-io-v1-IdentityConfiguration} @@ -351,9 +426,9 @@ Set to a negative value to disable caching. This field is only allowed for KMS v

resources is a list of kubernetes resources which have to be encrypted. The resource names are derived from resource or resource.group of the group/version/resource. eg: pandas.awesome.bears.example is a custom resource with 'group': awesome.bears.example, 'resource': pandas. -Use '*.*' to encrypt all resources and '*.<group>' to encrypt all resources in a specific group. -eg: '*.awesome.bears.example' will encrypt all resources in the group 'awesome.bears.example'. -eg: '*.' will encrypt all resources in the core group (such as pods, configmaps, etc).

+Use '.' to encrypt all resources and '.' to encrypt all resources in a specific group. +eg: '.awesome.bears.example' will encrypt all resources in the group 'awesome.bears.example'. +eg: '*.' will encrypt all resources in the core group (such as pods, configmaps, etc).

providers [Required]
@@ -393,4 +468,190 @@ Each key has to be 32 bytes long.

+ +## `WebhookConfiguration` {#apiserver-config-k8s-io-v1-WebhookConfiguration} + + +**Appears in:** + +- [AuthorizerConfiguration](#apiserver-config-k8s-io-v1-AuthorizerConfiguration) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
authorizedTTL [Required]
+meta/v1.Duration +		 +

The duration to cache 'authorized' responses from the webhook +authorizer. +Same as setting --authorization-webhook-cache-authorized-ttl flag +Default: 5m0s

+
unauthorizedTTL [Required]
+meta/v1.Duration +		 +

The duration to cache 'unauthorized' responses from the webhook +authorizer. +Same as setting --authorization-webhook-cache-unauthorized-ttl flag +Default: 30s

+
timeout [Required]
+meta/v1.Duration +		 +

Timeout for the webhook request +Maximum allowed value is 30s. +Required, no default value.

+
subjectAccessReviewVersion [Required]
+string +		 +

The API version of the authorization.k8s.io SubjectAccessReview to +send to and expect from the webhook. +Same as setting --authorization-webhook-version flag +Valid values: v1beta1, v1 +Required, no default value

+
matchConditionSubjectAccessReviewVersion [Required]
+string +		 +

MatchConditionSubjectAccessReviewVersion specifies the SubjectAccessReview +version the CEL expressions are evaluated against +Valid values: v1 +Required, no default value

+
failurePolicy [Required]
+string +		 +

Controls the authorization decision when a webhook request fails to +complete or returns a malformed response or errors evaluating +matchConditions. +Valid values:

+
    +
  • NoOpinion: continue to subsequent authorizers to see if one of +them allows the request
    • +
  • Deny: reject the request without consulting subsequent authorizers +Required, with no default.
    • +
+
connectionInfo [Required]
+WebhookConnectionInfo +		 +

ConnectionInfo defines how we talk to the webhook

+
matchConditions [Required]
+[]WebhookMatchCondition +		 +

matchConditions is a list of conditions that must be met for a request to be sent to this +webhook. An empty list of matchConditions matches all requests. +There are a maximum of 64 match conditions allowed.

+

The exact matching logic is (in order):

+
    +
  1. If at least one matchCondition evaluates to FALSE, then the webhook is skipped.
    2. +
  2. If ALL matchConditions evaluate to TRUE, then the webhook is called.
    3. +
  3. If at least one matchCondition evaluates to an error (but none are FALSE): +
      +
    • If failurePolicy=Deny, then the webhook rejects the request
      • +
    • If failurePolicy=NoOpinion, then the error is ignored and the webhook is skipped
      • +
    +
    4. +
+
+ +## `WebhookConnectionInfo` {#apiserver-config-k8s-io-v1-WebhookConnectionInfo} + + +**Appears in:** + +- [WebhookConfiguration](#apiserver-config-k8s-io-v1-WebhookConfiguration) + + + + + + + + + + + + + + + +
FieldDescription
type [Required]
+string +		 +

Controls how the webhook should communicate with the server. +Valid values:

+
    +
  • KubeConfigFile: use the file specified in kubeConfigFile to locate the +server.
    • +
  • InClusterConfig: use the in-cluster configuration to call the +SubjectAccessReview API hosted by kube-apiserver. This mode is not +allowed for kube-apiserver.
    • +
+
kubeConfigFile [Required]
+string +		 +

Path to KubeConfigFile for connection info +Required, if connectionInfo.Type is KubeConfig

+
+ +## `WebhookMatchCondition` {#apiserver-config-k8s-io-v1-WebhookMatchCondition} + + +**Appears in:** + +- [WebhookConfiguration](#apiserver-config-k8s-io-v1-WebhookConfiguration) + + + + + + + + + + + + +
FieldDescription
expression [Required]
+string +		 +

expression represents the expression which will be evaluated by CEL. Must evaluate to bool. +CEL expressions have access to the contents of the SubjectAccessReview in v1 version. +If version specified by subjectAccessReviewVersion in the request variable is v1beta1, +the contents would be converted to the v1 version before evaluating the CEL expression.

+
    +
  • 'resourceAttributes' describes information for a resource access request and is unset for non-resource requests. e.g. has(request.resourceAttributes) && request.resourceAttributes.namespace == 'default'
    • +
  • 'nonResourceAttributes' describes information for a non-resource access request and is unset for resource requests. e.g. has(request.nonResourceAttributes) && request.nonResourceAttributes.path == '/healthz'.
    • +
  • 'user' is the user to test for. e.g. request.user == 'alice'
    • +
  • 'groups' is the groups to test for. e.g. ('group1' in request.groups)
    • +
  • 'extra' corresponds to the user.Info.GetExtra() method from the authenticator.
    • +
  • 'uid' is the information about the requesting user. e.g. request.uid == '1'
    • +
+

Documentation on CEL: https://kubernetes.io/docs/reference/using-api/cel/

+
+ \ No newline at end of file diff --git a/content/en/docs/reference/config-api/apiserver-config.v1alpha1.md b/content/en/docs/reference/config-api/apiserver-config.v1alpha1.md index fd70d7c4557ee..1f870d351bdd4 100644 --- a/content/en/docs/reference/config-api/apiserver-config.v1alpha1.md +++ b/content/en/docs/reference/config-api/apiserver-config.v1alpha1.md @@ -1264,6 +1264,14 @@ Required, if connectionInfo.Type is KubeConfig

CEL expressions have access to the contents of the SubjectAccessReview in v1 version. If version specified by subjectAccessReviewVersion in the request variable is v1beta1, the contents would be converted to the v1 version before evaluating the CEL expression.

+

Documentation on CEL: https://kubernetes.io/docs/reference/using-api/cel/

diff --git a/content/en/docs/reference/config-api/apiserver-config.v1beta1.md b/content/en/docs/reference/config-api/apiserver-config.v1beta1.md index 0e54df86dbf16..7a14460dd5775 100644 --- a/content/en/docs/reference/config-api/apiserver-config.v1beta1.md +++ b/content/en/docs/reference/config-api/apiserver-config.v1beta1.md @@ -1197,6 +1197,14 @@ Required, if connectionInfo.Type is KubeConfig

CEL expressions have access to the contents of the SubjectAccessReview in v1 version. If version specified by subjectAccessReviewVersion in the request variable is v1beta1, the contents would be converted to the v1 version before evaluating the CEL expression.

+

Documentation on CEL: https://kubernetes.io/docs/reference/using-api/cel/

diff --git a/content/en/docs/reference/config-api/client-authentication.v1.md b/content/en/docs/reference/config-api/client-authentication.v1.md index 6e31f697e5372..43d8fcd41204d 100644 --- a/content/en/docs/reference/config-api/client-authentication.v1.md +++ b/content/en/docs/reference/config-api/client-authentication.v1.md @@ -205,7 +205,7 @@ itself should at least be protected via file permissions.

expirationTimestamp
-meta/v1.Time +meta/v1.Time

ExpirationTimestamp indicates a time when the provided credentials expire.

diff --git a/content/en/docs/reference/config-api/client-authentication.v1beta1.md b/content/en/docs/reference/config-api/client-authentication.v1beta1.md index 542dd4acc7560..623a8cfa96478 100644 --- a/content/en/docs/reference/config-api/client-authentication.v1beta1.md +++ b/content/en/docs/reference/config-api/client-authentication.v1beta1.md @@ -205,7 +205,7 @@ itself should at least be protected via file permissions.

expirationTimestamp
-meta/v1.Time +meta/v1.Time

ExpirationTimestamp indicates a time when the provided credentials expire.

diff --git a/content/en/docs/reference/config-api/kube-controller-manager-config.v1alpha1.md b/content/en/docs/reference/config-api/kube-controller-manager-config.v1alpha1.md index 2cb2ddc94db6b..b993e25039181 100644 --- a/content/en/docs/reference/config-api/kube-controller-manager-config.v1alpha1.md +++ b/content/en/docs/reference/config-api/kube-controller-manager-config.v1alpha1.md @@ -16,6 +16,180 @@ auto_generated: true +## `ClientConnectionConfiguration` {#ClientConnectionConfiguration} + + +**Appears in:** + +- [GenericControllerManagerConfiguration](#controllermanager-config-k8s-io-v1alpha1-GenericControllerManagerConfiguration) + + +

ClientConnectionConfiguration contains details for constructing a client.

+ + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
kubeconfig [Required]
+string +		 +

kubeconfig is the path to a KubeConfig file.

+
acceptContentTypes [Required]
+string +		 +

acceptContentTypes defines the Accept header sent by clients when connecting to a server, overriding the +default value of 'application/json'. This field will control all connections to the server used by a particular +client.

+
contentType [Required]
+string +		 +

contentType is the content type used when sending data to the server from this client.

+
qps [Required]
+float32 +		 +

qps controls the number of queries per second allowed for this connection.

+
burst [Required]
+int32 +		 +

burst allows extra queries to accumulate when a client is exceeding its rate.

+
+ +## `DebuggingConfiguration` {#DebuggingConfiguration} + + +**Appears in:** + +- [GenericControllerManagerConfiguration](#controllermanager-config-k8s-io-v1alpha1-GenericControllerManagerConfiguration) + + +

DebuggingConfiguration holds configuration for Debugging related features.

+ + + + + + + + + + + + + + +
FieldDescription
enableProfiling [Required]
+bool +		 +

enableProfiling enables profiling via web interface host:port/debug/pprof/

+
enableContentionProfiling [Required]
+bool +		 +

enableContentionProfiling enables block profiling, if +enableProfiling is true.

+
+ +## `LeaderElectionConfiguration` {#LeaderElectionConfiguration} + + +**Appears in:** + +- [GenericControllerManagerConfiguration](#controllermanager-config-k8s-io-v1alpha1-GenericControllerManagerConfiguration) + + +

LeaderElectionConfiguration defines the configuration of leader election +clients for components that can run with leader election enabled.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
leaderElect [Required]
+bool +		 +

leaderElect enables a leader election client to gain leadership +before executing the main loop. Enable this when running replicated +components for high availability.

+
leaseDuration [Required]
+meta/v1.Duration +		 +

leaseDuration is the duration that non-leader candidates will wait +after observing a leadership renewal until attempting to acquire +leadership of a led but unrenewed leader slot. This is effectively the +maximum duration that a leader can be stopped before it is replaced +by another candidate. This is only applicable if leader election is +enabled.

+
renewDeadline [Required]
+meta/v1.Duration +		 +

renewDeadline is the interval between attempts by the acting master to +renew a leadership slot before it stops leading. This must be less +than or equal to the lease duration. This is only applicable if leader +election is enabled.

+
retryPeriod [Required]
+meta/v1.Duration +		 +

retryPeriod is the duration the clients should wait between attempting +acquisition and renewal of a leadership. This is only applicable if +leader election is enabled.

+
resourceLock [Required]
+string +		 +

resourceLock indicates the resource object type that will be used to lock +during leader election cycles.

+
resourceName [Required]
+string +		 +

resourceName indicates the name of resource object that will be used to lock +during leader election cycles.

+
resourceNamespace [Required]
+string +		 +

resourceName indicates the namespace of resource object that will be used to lock +during leader election cycles.

+
+ ## `NodeControllerConfiguration` {#NodeControllerConfiguration} @@ -1482,7 +1656,8 @@ be unresponsive before marking it unhealthy.

nodeMontiorGracePeriod is the amount of time which we allow a running node to be unresponsive before marking it unhealthy. Must be N times more than kubelet's nodeStatusUpdateFrequency, where N means number of retries allowed for kubelet -to post node status.

+to post node status. This value should also be greater than the sum of +HTTP2_PING_TIMEOUT_SECONDS and HTTP2_READ_IDLE_TIMEOUT_SECONDS.

PodEvictionTimeout [Required]
diff --git a/content/en/docs/reference/config-api/kube-proxy-config.v1alpha1.md b/content/en/docs/reference/config-api/kube-proxy-config.v1alpha1.md index a560024adc616..40ca1eaff5ac8 100644 --- a/content/en/docs/reference/config-api/kube-proxy-config.v1alpha1.md +++ b/content/en/docs/reference/config-api/kube-proxy-config.v1alpha1.md @@ -14,6 +14,305 @@ auto_generated: true +## `FormatOptions` {#FormatOptions} + + +**Appears in:** + +- [LoggingConfiguration](#LoggingConfiguration) + + +

FormatOptions contains options for the different logging formats.

+ + + + + + + + + + + + + + +
FieldDescription
text [Required]
+TextOptions +		 +

[Alpha] Text contains options for logging format "text". +Only available when the LoggingAlphaOptions feature gate is enabled.

+
json [Required]
+JSONOptions +		 +

[Alpha] JSON contains options for logging format "json". +Only available when the LoggingAlphaOptions feature gate is enabled.

+
+ +## `JSONOptions` {#JSONOptions} + + +**Appears in:** + +- [FormatOptions](#FormatOptions) + + +

JSONOptions contains options for logging format "json".

+ + + + + + + + + + + +
FieldDescription
OutputRoutingOptions [Required]
+OutputRoutingOptions +		(Members of OutputRoutingOptions are embedded into this type.) + No description provided.
+ +## `LogFormatFactory` {#LogFormatFactory} + + + +

LogFormatFactory provides support for a certain additional, +non-default log format.

+ + + + +## `LoggingConfiguration` {#LoggingConfiguration} + + +**Appears in:** + +- [KubeProxyConfiguration](#kubeproxy-config-k8s-io-v1alpha1-KubeProxyConfiguration) + +- [KubeletConfiguration](#kubelet-config-k8s-io-v1beta1-KubeletConfiguration) + + +

LoggingConfiguration contains logging options.

+ + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
format [Required]
+string +		 +

Format Flag specifies the structure of log messages. +default value of format is text

+
flushFrequency [Required]
+TimeOrMetaDuration +		 +

Maximum time between log flushes. +If a string, parsed as a duration (i.e. "1s") +If an int, the maximum number of nanoseconds (i.e. 1s = 1000000000). +Ignored if the selected logging backend writes log messages without buffering.

+
verbosity [Required]
+VerbosityLevel +		 +

Verbosity is the threshold that determines which log messages are +logged. Default is zero which logs only the most important +messages. Higher values enable additional messages. Error messages +are always logged.

+
vmodule [Required]
+VModuleConfiguration +		 +

VModule overrides the verbosity threshold for individual files. +Only supported for "text" log format.

+
options [Required]
+FormatOptions +		 +

[Alpha] Options holds additional parameters that are specific +to the different logging formats. Only the options for the selected +format get used, but all of them get validated. +Only available when the LoggingAlphaOptions feature gate is enabled.

+
+ +## `LoggingOptions` {#LoggingOptions} + + + +

LoggingOptions can be used with ValidateAndApplyWithOptions to override +certain global defaults.

+ + + + + + + + + + + + + + +
FieldDescription
ErrorStream [Required]
+io.Writer +		 +

ErrorStream can be used to override the os.Stderr default.

+
InfoStream [Required]
+io.Writer +		 +

InfoStream can be used to override the os.Stdout default.

+
+ +## `OutputRoutingOptions` {#OutputRoutingOptions} + + +**Appears in:** + +- [JSONOptions](#JSONOptions) + +- [TextOptions](#TextOptions) + + +

OutputRoutingOptions contains options that are supported by both "text" and "json".

+ + + + + + + + + + + + + + +
FieldDescription
splitStream [Required]
+bool +		 +

[Alpha] SplitStream redirects error messages to stderr while +info messages go to stdout, with buffering. The default is to write +both to stdout, without buffering. Only available when +the LoggingAlphaOptions feature gate is enabled.

+
infoBufferSize [Required]
+k8s.io/apimachinery/pkg/api/resource.QuantityValue +		 +

[Alpha] InfoBufferSize sets the size of the info stream when +using split streams. The default is zero, which disables buffering. +Only available when the LoggingAlphaOptions feature gate is enabled.

+
+ +## `TextOptions` {#TextOptions} + + +**Appears in:** + +- [FormatOptions](#FormatOptions) + + +

TextOptions contains options for logging format "text".

+ + + + + + + + + + + +
FieldDescription
OutputRoutingOptions [Required]
+OutputRoutingOptions +		(Members of OutputRoutingOptions are embedded into this type.) + No description provided.
+ +## `TimeOrMetaDuration` {#TimeOrMetaDuration} + + +**Appears in:** + +- [LoggingConfiguration](#LoggingConfiguration) + + +

TimeOrMetaDuration is present only for backwards compatibility for the +flushFrequency field, and new fields should use metav1.Duration.

+ + + + + + + + + + + + + + +
FieldDescription
Duration [Required]
+meta/v1.Duration +		 +

Duration holds the duration

+
- [Required]
+bool +		 +

SerializeAsString controls whether the value is serialized as a string or an integer

+
+ +## `VModuleConfiguration` {#VModuleConfiguration} + +(Alias of `[]k8s.io/component-base/logs/api/v1.VModuleItem`) + +**Appears in:** + +- [LoggingConfiguration](#LoggingConfiguration) + + +

VModuleConfiguration is a collection of individual file names or patterns +and the corresponding verbosity threshold.

+ + + + +## `VerbosityLevel` {#VerbosityLevel} + +(Alias of `uint32`) + +**Appears in:** + +- [LoggingConfiguration](#LoggingConfiguration) + + + +

VerbosityLevel represents a klog or logr verbosity threshold.

+ + + + + + + ## `ClientConnectionConfiguration` {#ClientConnectionConfiguration} diff --git a/content/en/docs/reference/config-api/kube-scheduler-config.v1.md b/content/en/docs/reference/config-api/kube-scheduler-config.v1.md index 5f6cd2e1a60c8..db57085dae757 100644 --- a/content/en/docs/reference/config-api/kube-scheduler-config.v1.md +++ b/content/en/docs/reference/config-api/kube-scheduler-config.v1.md @@ -396,7 +396,7 @@ Defaults to false.

addedAffinity
-core/v1.NodeAffinity +core/v1.NodeAffinity

AddedAffinity is applied to all Pods additionally to the NodeAffinity @@ -495,7 +495,7 @@ The default strategy is LeastAllocated with an equal "cpu" and "m defaultConstraints
-[]core/v1.TopologySpreadConstraint +[]core/v1.TopologySpreadConstraint

DefaultConstraints defines topology spread constraints to be applied to diff --git a/content/en/docs/reference/config-api/kubeadm-config.v1beta3.md b/content/en/docs/reference/config-api/kubeadm-config.v1beta3.md index 86531a85a88a2..ed8bc424291d1 100644 --- a/content/en/docs/reference/config-api/kubeadm-config.v1beta3.md +++ b/content/en/docs/reference/config-api/kubeadm-config.v1beta3.md @@ -307,7 +307,7 @@ for, so other administrators can know its purpose.

expires
-meta/v1.Time +meta/v1.Time

expires specifies the timestamp when this token expires. Defaults to being set @@ -1038,7 +1038,7 @@ file from which to load cluster information.

pathType
-core/v1.HostPathType +core/v1.HostPathType

pathType is the type of the hostPath.

@@ -1262,7 +1262,7 @@ This information will be annotated to the Node API object, for later re-use.

taints [Required]
-[]core/v1.Taint +[]core/v1.Taint

taints specifies the taints the Node API object should be registered with. @@ -1294,7 +1294,7 @@ Value all ignores errors from all checks.

imagePullPolicy
-core/v1.PullPolicy +core/v1.PullPolicy

imagePullPolicy specifies the policy for image pulling during kubeadm "init" and diff --git a/content/en/docs/reference/config-api/kubeadm-config.v1beta4.md b/content/en/docs/reference/config-api/kubeadm-config.v1beta4.md index 147bdb75bde61..85e4fe489ea5d 100644 --- a/content/en/docs/reference/config-api/kubeadm-config.v1beta4.md +++ b/content/en/docs/reference/config-api/kubeadm-config.v1beta4.md @@ -9,13 +9,12 @@ auto_generated: true This version improves on the v1beta3 format by fixing some minor issues and adding a few new fields.

A list of changes since v1beta3:

Basics

The preferred way to configure kubeadm is to pass an YAML configuration file with @@ -215,13 +215,13 @@ configuration types to be used during a kubeadm init run.

- name: SOME_VAR value: SOME_VALUE serverCertSANs: - - "ec2-10-100-0-1.compute-1.amazonaws.com" + - ec2-10-100-0-1.compute-1.amazonaws.com peerCertSANs: - - "10.100.0.1" + - 10.100.0.1 # external: # endpoints: - # - "10.100.0.1:2379" - # - "10.100.0.2:2379" + # - 10.100.0.1:2379 + # - 10.100.0.2:2379 # caFile: "/etcd/kubernetes/pki/etcd/etcd-ca.crt" # certFile: "/etcd/kubernetes/pki/etcd/etcd.crt" # keyFile: "/etcd/kubernetes/pki/etcd/etcd.key" @@ -235,7 +235,7 @@ configuration types to be used during a kubeadm init run.

apiServer: extraArgs: - name: authorization-mode - value: "Node,RBAC" + value: Node,RBAC extraEnvs: - name: SOME_VAR value: SOME_VALUE @@ -263,7 +263,7 @@ configuration types to be used during a kubeadm init run.

scheduler: extraArgs: - name: address - value: "10.100.0.1" + value: 10.100.0.1 extraVolumes: - name: "some-volume" hostPath: "/etc/some-path" @@ -388,7 +388,7 @@ for, so other administrators can know its purpose.

expires
-meta/v1.Time +meta/v1.Time

expires specifies the timestamp when this token expires. Defaults to being set @@ -725,7 +725,7 @@ node to the cluster

string -

caCertPath is the path to the SSL certificate authority used to secure comunications +

caCertPath is the path to the SSL certificate authority used to secure communications between node and control-plane. Defaults to "/etc/kubernetes/pki/ca.crt".

@@ -1238,7 +1238,7 @@ does not contain any other authentication information.

EnvVar [Required]
-core/v1.EnvVar +core/v1.EnvVar (Members of EnvVar are embedded into this type.) No description provided. @@ -1405,7 +1405,7 @@ file from which to load cluster information.

pathType
-core/v1.HostPathType +core/v1.HostPathType

pathType is the type of the hostPath.

@@ -1640,7 +1640,7 @@ This information will be annotated to the Node API object, for later re-use.

taints [Required]
-[]core/v1.Taint +[]core/v1.Taint

taints specifies the taints the Node API object should be registered with. @@ -1673,7 +1673,7 @@ Value 'all' ignores errors from all checks.

imagePullPolicy
-core/v1.PullPolicy +core/v1.PullPolicy

imagePullPolicy specifies the policy for image pulling during kubeadm init and @@ -1950,7 +1950,7 @@ NOTE: This field is currently ignored for kubeadm upgrade apply, bu imagePullPolicy
-core/v1.PullPolicy +core/v1.PullPolicy

imagePullPolicy specifies the policy for image pulling during kubeadm upgrade apply operations. @@ -2065,7 +2065,7 @@ The list of phases can be obtained with the kubeadm upgrade node phase --h imagePullPolicy
-core/v1.PullPolicy +core/v1.PullPolicy

imagePullPolicy specifies the policy for image pulling during kubeadm upgrade node operations. diff --git a/content/en/docs/reference/config-api/kubelet-config.v1beta1.md b/content/en/docs/reference/config-api/kubelet-config.v1beta1.md index bb69ed2a763e5..34fdcbbf91b9a 100644 --- a/content/en/docs/reference/config-api/kubelet-config.v1beta1.md +++ b/content/en/docs/reference/config-api/kubelet-config.v1beta1.md @@ -840,6 +840,19 @@ Requires the CPUManager feature gate to be enabled. Default: "None"

+singleProcessOOMKill
+bool + + +

singleProcessOOMKill, if true, will prevent the memory.oom.group flag from being set for container +cgroups in cgroups v2. This causes processes in the container to be OOM killed individually instead of as +a group. It means that if true, the behavior aligns with the behavior of cgroups v1. +The default value is determined automatically when you don't specify. +On non-linux such as windows, only null / absent is allowed. +On cgroup v1 linux, only null / absent and true are allowed. +On cgroup v2 linux, null / absent, true and false are allowed. The default value is false.

+ + cpuManagerPolicyOptions
map[string]string @@ -1125,9 +1138,6 @@ Default: "5m"

evictionMaxPodGracePeriod is the maximum allowed grace period (in seconds) to use when terminating pods in response to a soft eviction threshold being met. This value effectively caps the Pod's terminationGracePeriodSeconds value during soft evictions. -Note: Due to issue #64530, the behavior has a bug where this value currently just -overrides the grace period during soft eviction, which can increase the grace -period from what is set on the Pod. This bug will be fixed in a future release. Default: 0

@@ -1288,7 +1298,7 @@ managers are running. Valid values include:

systemReserved is a set of ResourceName=ResourceQuantity (e.g. cpu=200m,memory=150G) pairs that describe resources reserved for non-kubernetes components. Currently only cpu and memory are supported. -See http://kubernetes.io/docs/user-guide/compute-resources for more detail. +See https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources for more detail. Default: nil

@@ -1299,7 +1309,7 @@ Default: nil

kubeReserved is a set of ResourceName=ResourceQuantity (e.g. cpu=200m,memory=150G) pairs that describe resources reserved for kubernetes system components. Currently cpu, memory and local storage for root file system are supported. -See https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/ +See https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources for more details. Default: nil

@@ -1486,6 +1496,14 @@ This configuration must be empty if either ShutdownGracePeriod or ShutdownGraceP Default: nil

+crashLoopBackOff
+CrashLoopBackOffConfig + + +

CrashLoopBackOff contains config to modify node-level parameters for +container restart behavior

+ + reservedMemory
[]MemoryReservation @@ -1549,7 +1567,7 @@ Default: 0.9

registerWithTaints
-[]core/v1.Taint +[]core/v1.Taint

registerWithTaints are an array of taints to add to a node object when @@ -1640,7 +1658,7 @@ It exists in the kubeletconfig API group because it is classified as a versioned source
-core/v1.NodeConfigSource +core/v1.NodeConfigSource

source is the source that we are serializing.

@@ -1649,6 +1667,32 @@ It exists in the kubeletconfig API group because it is classified as a versioned +## `CrashLoopBackOffConfig` {#kubelet-config-k8s-io-v1beta1-CrashLoopBackOffConfig} + + +**Appears in:** + +- [KubeletConfiguration](#kubelet-config-k8s-io-v1beta1-KubeletConfiguration) + + + + + + + + + + + + +
FieldDescription
maxContainerRestartPeriod
+meta/v1.Duration +		 +

maxContainerRestartPeriod is the maximum duration the backoff delay can accrue +to for container restarts, minimum 1 second, maximum 300 seconds. If not set, +defaults to the internal crashloopbackoff maximum (300s).

+
+ ## `CredentialProvider` {#kubelet-config-k8s-io-v1beta1-CredentialProvider} @@ -2001,7 +2045,7 @@ and groups corresponding to the Organization in the client certificate.

No description provided. limits [Required]
-core/v1.ResourceList +core/v1.ResourceList No description provided. diff --git a/content/en/docs/reference/glossary/kubectl.md b/content/en/docs/reference/glossary/kubectl.md index 7963cd77b2ab4..fa745c4b2c677 100644 --- a/content/en/docs/reference/glossary/kubectl.md +++ b/content/en/docs/reference/glossary/kubectl.md @@ -20,3 +20,5 @@ using the Kubernetes API. You can use `kubectl` to create, inspect, update, and delete Kubernetes objects. + +In English, `kubectl` is (officially) pronounced /kjuːb/ /kənˈtɹəʊl/ (like "cube control"). diff --git a/content/en/docs/reference/glossary/rbac.md b/content/en/docs/reference/glossary/rbac.md index ff3dc7e7476bc..4090d03d2e4ee 100644 --- a/content/en/docs/reference/glossary/rbac.md +++ b/content/en/docs/reference/glossary/rbac.md @@ -15,5 +15,18 @@ tags: -RBAC utilizes *roles*, which contain permission rules, and *role bindings*, which grant the permissions defined in a role to a set of users. +RBAC utilizes four kinds of Kubernetes objects: +Role +: Defines permission rules in a specific namespace. + +ClusterRole +: Defines permission rules cluster-wide. + +RoleBinding +: Grants the permissions defined in a role to a set of users in a specific namespace. + +ClusterRoleBinding +: Grants the permissions defined in a role to a set of users cluster-wide. + +For more information, see [RBAC](/docs/reference/access-authn-authz/rbac/). diff --git a/content/en/docs/reference/instrumentation/metrics.md b/content/en/docs/reference/instrumentation/metrics.md index 6acaf5eecc6ca..bb64782fa535b 100644 --- a/content/en/docs/reference/instrumentation/metrics.md +++ b/content/en/docs/reference/instrumentation/metrics.md @@ -6,10 +6,10 @@ description: >- Details of the metric data that Kubernetes components export. --- -## Metrics (v1.31) +## Metrics (v1.32) - - + + This page details the metrics that different Kubernetes components export. You can query the metrics endpoint for these components using an HTTP scrape, and fetch the current metrics data in Prometheus format. @@ -495,7 +495,7 @@ Alpha metrics do not have any API guarantees. These metrics must be used at your
  • operationrejectedtype
    • apiserver_admission_webhook_fail_open_count
    -
    Admission webhook fail open count, identified by name and broken out for each admission type (validating or mutating).
    +
    Admission webhook fail open count, identified by name and broken out for each admission type (validating or admit).
    • ALPHA
    • Counter
      • @@ -509,7 +509,7 @@ Alpha metrics do not have any API guarantees. These metrics must be used at your
    • error_typenameoperationrejection_codetype
    apiserver_admission_webhook_request_total
    -
    Admission webhook request total, identified by name and broken out for each admission type (validating or mutating) and operation. Additional labels specify whether the request was rejected or not and an HTTP status code. Codes greater than 600 are truncated to 600, to keep the metrics cardinality bounded.
    +
    Admission webhook request total, identified by name and broken out for each admission type (validating or admit) and operation. Additional labels specify whether the request was rejected or not and an HTTP status code. Codes greater than 600 are truncated to 600, to keep the metrics cardinality bounded.
    • ALPHA
    • Counter
      • @@ -844,6 +844,41 @@ Alpha metrics do not have any API guarantees. These metrics must be used at your
    • Histogram
    • grpc_status_codemethod_nameprovider_name
    +
    apiserver_externaljwt_fetch_keys_data_timestamp
    +
    Unix Timestamp in seconds of the last successful FetchKeys data_timestamp value returned by the external signer
    +
      +
    • ALPHA
      • +
    • Gauge
      • +
    +
    +
    apiserver_externaljwt_fetch_keys_request_total
    +
    Total attempts at syncing supported JWKs
    +
      +
    • ALPHA
      • +
    • Counter
      • +
    • code
    +
    +
    apiserver_externaljwt_fetch_keys_success_timestamp
    +
    Unix Timestamp in seconds of the last successful FetchKeys request
    +
      +
    • ALPHA
      • +
    • Gauge
      • +
    +
    +
    apiserver_externaljwt_request_duration_seconds
    +
    Request duration and time for calls to external-jwt-signer
    +
      +
    • ALPHA
      • +
    • Histogram
      • +
    • codemethod
    +
    +
    apiserver_externaljwt_sign_request_total
    +
    Total attempts at signing JWT
    +
      +
    • ALPHA
      • +
    • Counter
      • +
    • code
    +
    apiserver_flowcontrol_current_inqueue_seats
    Number of seats currently pending in queues of the API Priority and Fairness subsystem
      @@ -1216,11 +1251,11 @@ Alpha metrics do not have any API guarantees. These metrics must be used at your
    • transformation_typetransformer_prefix
    apiserver_storage_transformation_operations_total
    -
    Total number of transformations. Successful transformation will have a status 'OK' and a varied status string when the transformation fails. This status and transformation_type fields may be used for alerting on encryption/decryption failure using transformation_type from_storage for decryption and to_storage for encryption
    +
    Total number of transformations. Successful transformation will have a status 'OK' and a varied status string when the transformation fails. The status, resource, and transformation_type fields can be used for alerting purposes. For example, you can monitor for encryption/decryption failures using the transformation_type (e.g., from_storage for decryption and to_storage for encryption). Additionally, these fields can be used to ensure that the correct transformers are applied to each resource.
    • ALPHA
    • Counter
      • -
    • statustransformation_typetransformer_prefix
    +
  • resourcestatustransformation_typetransformer_prefix
    • apiserver_stream_translator_requests_total
    Total number of requests that were handled by the StreamTranslatorProxy, which processes streaming RemoteCommand/V5
    @@ -1418,6 +1453,20 @@ Alpha metrics do not have any API guarantees. These metrics must be used at your
  • Counter
  • codewebhook
    • +
    clustertrustbundle_publisher_sync_duration_seconds
    +
    The time it took to sync a cluster trust bundle.
    +
      +
    • ALPHA
      • +
    • Histogram
      • +
    • code
    +
    +
    clustertrustbundle_publisher_sync_total
    +
    Number of syncs that occurred in cluster trust bundle publisher.
    +
      +
    • ALPHA
      • +
    • Counter
      • +
    • code
    +
    container_swap_usage_bytes
    Current amount of the container swap usage in bytes. Reported only on non-windows systems
      @@ -1432,6 +1481,20 @@ Alpha metrics do not have any API guarantees. These metrics must be used at your
    • Histogram
    • driver_namegrpc_status_codemethod_namemigrated
    +
    dra_grpc_operations_duration_seconds
    +
    Duration in seconds of the DRA gRPC operations
    +
      +
    • ALPHA
      • +
    • Histogram
      • +
    • driver_namegrpc_status_codemethod_name
    +
    +
    dra_operations_duration_seconds
    +
    Latency histogram in seconds for the duration of handling all ResourceClaims referenced by a pod when the pod starts or stops. Identified by the name of the operation (PrepareResources or UnprepareResources) and separated by the success of the operation. The number of failed operations is provided through the histogram's overall count.
    +
      +
    • ALPHA
      • +
    • Histogram
      • +
    • is_erroroperation_name
    +
    endpoint_slice_controller_changes
    Number of EndpointSlice changes
      @@ -1559,14 +1622,14 @@ Alpha metrics do not have any API guarantees. These metrics must be used at your
    ephemeral_volume_controller_create_failures_total
    -
    Number of PersistenVolumeClaims creation requests
    +
    Number of PersistentVolumeClaim creation requests
    • ALPHA
    • Counter
    ephemeral_volume_controller_create_total
    -
    Number of PersistenVolumeClaims creation requests
    +
    Number of PersistentVolumeClaim creation requests
    • ALPHA
    • Counter
      • @@ -1803,6 +1866,13 @@ Alpha metrics do not have any API guarantees. These metrics must be used at your
    • Gauge
    • static
    +
    kubelet_admission_rejections_total
    +
    Cumulative number pod admission rejections by the Kubelet.
    +
      +
    • ALPHA
      • +
    • Counter
      • +
    • reason
    +
    kubelet_certificate_manager_client_expiration_renew_errors
    Counter of certificate renewal errors.
      @@ -1845,6 +1915,13 @@ Alpha metrics do not have any API guarantees. These metrics must be used at your
    • Gauge
    +
    kubelet_container_aligned_compute_resources_count
    +
    Cumulative number of aligned compute resources allocated to containers by alignment type.
    +
      +
    • ALPHA
      • +
    • Counter
      • +
    • boundaryscope
    +
    kubelet_container_log_filesystem_used_bytes
    Bytes used by the container's logs on the filesystem.
      @@ -1859,6 +1936,13 @@ Alpha metrics do not have any API guarantees. These metrics must be used at your
    • Histogram
    +
    kubelet_cpu_manager_exclusive_cpu_allocation_count
    +
    The total number of CPUs exclusively allocated to containers running on this node
    +
      +
    • ALPHA
      • +
    • Gauge
      • +
    +
    kubelet_cpu_manager_pinning_errors_total
    The number of cpu core allocations which required pinning failed.
      @@ -1873,6 +1957,13 @@ Alpha metrics do not have any API guarantees. These metrics must be used at your
    • Counter
    +
    kubelet_cpu_manager_shared_pool_size_millicores
    +
    The size of the shared CPU pool for non-guaranteed QoS pods, in millicores.
    +
      +
    • ALPHA
      • +
    • Gauge
      • +
    +
    kubelet_credential_provider_plugin_duration
    Duration of execution in seconds for credential provider plugin
      @@ -1944,7 +2035,7 @@ Alpha metrics do not have any API guarantees. These metrics must be used at your
    • eviction_signal
    kubelet_graceful_shutdown_end_time_seconds
    -
    Last graceful shutdown start time since unix epoch in seconds
    +
    Last graceful shutdown end time since unix epoch in seconds
    • ALPHA
    • Gauge
      • @@ -2797,6 +2888,13 @@ Alpha metrics do not have any API guarantees. These metrics must be used at your
    • Histogram
    +
    resourceclaim_controller_allocated_resource_claims
    +
    Number of allocated ResourceClaims
    +
      +
    • ALPHA
      • +
    • Gauge
      • +
    +
    resourceclaim_controller_create_attempts_total
    Number of ResourceClaims creation requests
      @@ -2811,6 +2909,13 @@ Alpha metrics do not have any API guarantees. These metrics must be used at your
    • Counter
    +
    resourceclaim_controller_resource_claims
    +
    Number of ResourceClaims
    +
      +
    • ALPHA
      • +
    • Gauge
      • +
    +
    rest_client_dns_resolution_duration_seconds
    DNS resolver latency in seconds. Broken down by host.
      @@ -2944,6 +3049,13 @@ Alpha metrics do not have any API guarantees. These metrics must be used at your
    • Gauge
    • operation
    +
    scheduler_inflight_events
    +
    Number of events currently tracked in the scheduling queue.
    +
      +
    • ALPHA
      • +
    • Gauge
      • +
    • event
    +
    scheduler_permit_wait_duration_seconds
    Duration of waiting on permit.
      @@ -2965,6 +3077,20 @@ Alpha metrics do not have any API guarantees. These metrics must be used at your
    • Histogram
    • extension_pointpluginstatus
    +
    scheduler_preemption_goroutines_duration_seconds
    +
    Duration in seconds for running goroutines for the preemption.
    +
      +
    • ALPHA
      • +
    • Histogram
      • +
    • result
    +
    +
    scheduler_preemption_goroutines_execution_total
    +
    Number of preemption goroutines executed.
    +
      +
    • ALPHA
      • +
    • Counter
      • +
    • result
    +
    scheduler_queueing_hint_execution_duration_seconds
    Duration for running a queueing hint function of a plugin.
      @@ -3014,6 +3140,13 @@ Alpha metrics do not have any API guarantees. These metrics must be used at your
    • Custom
    • 1.29.0
    +
    selinux_warning_controller_selinux_volume_conflict
    +
    Conflict between two Pods using the same volume
    +
      +
    • ALPHA
      • +
    • Custom
      • +
    • propertypod1_namespacepod1_namepod1_valuepod2_namespacepod2_namepod2_value
    +
    service_controller_loadbalancer_sync_total
    A metric counting the amount of times any load balancer has been configured, as an effect of service/node changes on the cluster
      diff --git a/content/en/docs/reference/instrumentation/zpages.md b/content/en/docs/reference/instrumentation/zpages.md new file mode 100644 index 0000000000000..079f6fa08f3f0 --- /dev/null +++ b/content/en/docs/reference/instrumentation/zpages.md @@ -0,0 +1,70 @@ +--- +title: Kubernetes z-pages +content_type: reference +weight: 60 +reviewers: +- dashpole +--- + + + + +{{< feature-state for_k8s_version="v1.32" state="alpha" >}} + +Kubernetes core components can expose a suite of _z-endpoints_ to make it easier for users +to debug their cluster and its components. These endpoints are strictly to be used for human +inspection to gain real time debugging information of a component binary. +Avoid automated scraping of data returned by these endpoints; in Kubernetes {{< skew currentVersion >}} +these are an **alpha** feature and the response format may change in future releases. + + + +## z-pages + +Kubernetes v{{< skew currentVersion >}} allows you to enable _z-pages_ to help you troubleshoot +problems with its core control plane components. These special debugging endpoints provide internal +information about running components. For Kubernetes {{< skew currentVersion >}}, components +serve the following endpoints (when enabled): + +- [z-pages](#z-pages) + - [statusz](#statusz) + - [flagz](#flagz) + +### statusz + +Enabled using the `ComponentStatusz` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/), +the `/statusz` endpoint displays high level information about the component such as its Kubernetes version, emulation version, start time and more. + +The `/statusz` response from the API server is similar to: + +``` +kube-apiserver statusz +Warning: This endpoint is not meant to be machine parseable, has no formatting compatibility guarantees and is for debugging purposes only. + +Started: Wed Oct 16 21:03:43 UTC 2024 +Up: 0 hr 00 min 16 sec +Go version: go1.23.2 +Binary version: 1.32.0-alpha.0.1484+5eeac4f21a491b-dirty +Emulation version: 1.32.0-alpha.0.1484 +``` + +### flagz + +Enabled using the `ComponentFlagz` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/), the `/flagz` endpoint shows you the command line arguments that were used to start a component. + +The `/flagz` data for the API server looks something like: + +``` +kube-apiserver flags +Warning: This endpoint is not meant to be machine parseable, has no formatting compatibility guarantees and is for debugging purposes only. + +advertise-address=192.168.8.2 +contention-profiling=false +enable-priority-and-fairness=true +profiling=true +authorization-mode=[Node,RBAC] +authorization-webhook-cache-authorized-ttl=5m0s +authorization-webhook-cache-unauthorized-ttl=30s +authorization-webhook-version=v1beta1 +default-watch-cache-size=100 +``` \ No newline at end of file diff --git a/content/en/docs/reference/kubectl/generated/kubectl_apply/_index.md b/content/en/docs/reference/kubectl/generated/kubectl_apply/_index.md index 2b3024eeb6b80..5c8b533a3be9d 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_apply/_index.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_apply/_index.md @@ -204,6 +204,13 @@ kubectl apply (-f FILENAME | -k DIRECTORY)

      If true, keep the managedFields when printing objects in JSON or YAML format.

      + +--subresource string + + +

      If specified, apply will operate on the subresource of the requested object. Only allowed when using --server-side. This flag is beta and may change in the future.

      + + --template string @@ -222,7 +229,7 @@ kubectl apply (-f FILENAME | -k DIRECTORY) --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_apply/kubectl_apply_edit-last-applied.md b/content/en/docs/reference/kubectl/generated/kubectl_apply/kubectl_apply_edit-last-applied.md index 7feb1d8d67e99..198db73bb4527 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_apply/kubectl_apply_edit-last-applied.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_apply/kubectl_apply_edit-last-applied.md @@ -121,7 +121,7 @@ kubectl apply edit-last-applied (RESOURCE/NAME | -f FILENAME) --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_create/_index.md b/content/en/docs/reference/kubectl/generated/kubectl_create/_index.md index f5bda8c5f0f26..f38362cbc01e5 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_create/_index.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_create/_index.md @@ -154,7 +154,7 @@ kubectl create -f FILENAME --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_clusterrole.md b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_clusterrole.md index 8f210bb8e037a..27c009f92a839 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_clusterrole.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_clusterrole.md @@ -146,7 +146,7 @@ kubectl create clusterrole NAME --verb=verb --resource=resource.group [--resourc --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_clusterrolebinding.md b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_clusterrolebinding.md index e0219d2341294..8839f52c4418f 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_clusterrolebinding.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_clusterrolebinding.md @@ -131,7 +131,7 @@ kubectl create clusterrolebinding NAME --clusterrole=NAME [--user=username] [--g --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_configmap.md b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_configmap.md index c817c775a5f5b..9d626d4fd74f9 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_configmap.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_configmap.md @@ -149,7 +149,7 @@ kubectl create configmap NAME [--from-file=[key=]source] [--from-literal=key1=va --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_cronjob.md b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_cronjob.md index fd471895367a2..0affbe1e41334 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_cronjob.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_cronjob.md @@ -127,7 +127,7 @@ kubectl create cronjob NAME --image=image --schedule='0/5 * * * ?' -- [COMMAND] --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_deployment.md b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_deployment.md index 357efe7e461a4..49dd8231906aa 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_deployment.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_deployment.md @@ -136,7 +136,7 @@ kubectl create deployment NAME --image=image -- [COMMAND] [args...] --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_ingress.md b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_ingress.md index 9c7f8f8ead5a5..aefb16b3fbf8a 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_ingress.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_ingress.md @@ -164,7 +164,7 @@ kubectl create ingress NAME --rule=host/path=service:port[,tls[=secret]] --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_job.md b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_job.md index 4b0f0da738f3a..ff64214d7b1ab 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_job.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_job.md @@ -123,7 +123,7 @@ kubectl create job NAME --image=image [--from=cronjob/name] -- [COMMAND] [args.. --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_namespace.md b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_namespace.md index acc4fb17991c2..b9a1fd03699bd 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_namespace.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_namespace.md @@ -103,7 +103,7 @@ kubectl create namespace NAME [--dry-run=server|client|none] --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_poddisruptionbudget.md b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_poddisruptionbudget.md index 2e6fb2edd569e..67efaeb546507 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_poddisruptionbudget.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_poddisruptionbudget.md @@ -129,7 +129,7 @@ kubectl create poddisruptionbudget NAME --selector=SELECTOR --min-available=N [- --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_priorityclass.md b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_priorityclass.md index e3222fb882687..d1d681c4fb759 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_priorityclass.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_priorityclass.md @@ -130,7 +130,7 @@ kubectl create priorityclass NAME --value=VALUE --global-default=BOOL [--dry-run --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_quota.md b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_quota.md index 53d4230785bb6..258faa27075bb 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_quota.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_quota.md @@ -120,7 +120,7 @@ kubectl create quota NAME [--hard=key1=value1,key2=value2] [--scopes=Scope1,Scop --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_role.md b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_role.md index d4b357cff1d89..07258f7f1d130 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_role.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_role.md @@ -126,7 +126,7 @@ kubectl create role NAME --verb=verb --resource=resource.group/subresource [--re --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_rolebinding.md b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_rolebinding.md index ca3af6f4c09f7..7cc922915dba5 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_rolebinding.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_rolebinding.md @@ -141,7 +141,7 @@ kubectl create rolebinding NAME --clusterrole=NAME|--role=NAME [--user=username] --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_secret_docker-registry.md b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_secret_docker-registry.md index 1363ee05f52db..1d4934174f340 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_secret_docker-registry.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_secret_docker-registry.md @@ -159,7 +159,7 @@ kubectl create secret docker-registry NAME --docker-username=user --docker-passw --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_secret_generic.md b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_secret_generic.md index 456fcc88a9c68..2002e8ce4aa6b 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_secret_generic.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_secret_generic.md @@ -156,7 +156,7 @@ kubectl create secret generic NAME [--type=string] [--from-file=[key=]source] [- --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_secret_tls.md b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_secret_tls.md index afa2097ddf8d8..6f8f7625b51ed 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_secret_tls.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_secret_tls.md @@ -126,7 +126,7 @@ kubectl create secret tls NAME --cert=path/to/cert/file --key=path/to/key/file [ --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_service_clusterip.md b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_service_clusterip.md index 4208081bf90f6..8500d8d525681 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_service_clusterip.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_service_clusterip.md @@ -120,7 +120,7 @@ kubectl create service clusterip NAME [--tcp=:] [--dry-run=ser --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_service_externalname.md b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_service_externalname.md index 27f90c17d8231..6189459a64dc5 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_service_externalname.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_service_externalname.md @@ -119,7 +119,7 @@ kubectl create service externalname NAME --external-name external.name [--dry-ru --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_service_loadbalancer.md b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_service_loadbalancer.md index 843786054100f..dc2bca4ff5cc7 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_service_loadbalancer.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_service_loadbalancer.md @@ -110,7 +110,7 @@ kubectl create service loadbalancer NAME [--tcp=port:targetPort] [--dry-run=serv --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_service_nodeport.md b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_service_nodeport.md index 02f480f6ebc35..2c05120cf3280 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_service_nodeport.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_service_nodeport.md @@ -117,7 +117,7 @@ kubectl create service nodeport NAME [--tcp=port:targetPort] [--dry-run=server|c --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_serviceaccount.md b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_serviceaccount.md index 38571505b16a4..481c6943c4ce8 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_serviceaccount.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_create/kubectl_create_serviceaccount.md @@ -103,7 +103,7 @@ kubectl create serviceaccount NAME [--dry-run=server|client|none] --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_delete/_index.md b/content/en/docs/reference/kubectl/generated/kubectl_delete/_index.md index c23ddd5929857..482b6b059f8b2 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_delete/_index.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_delete/_index.md @@ -67,6 +67,9 @@ kubectl delete ([-f FILENAME] | [-k DIRECTORY] | TYPE [(NAME | -l label | --all) # Delete all pods kubectl delete pods --all + + # Delete all pods only if the user confirms the deletion + kubectl delete pods --all --interactive ``` ## {{% heading "options" %}} diff --git a/content/en/docs/reference/kubectl/generated/kubectl_edit/_index.md b/content/en/docs/reference/kubectl/generated/kubectl_edit/_index.md index 5ed5057abd830..8f82f3bb41589 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_edit/_index.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_edit/_index.md @@ -142,7 +142,7 @@ kubectl edit (RESOURCE/NAME | -f FILENAME) --subresource string -

      If specified, edit will operate on the subresource of the requested object. Must be one of [status]. This flag is beta and may change in the future.

      +

      If specified, edit will operate on the subresource of the requested object. This flag is beta and may change in the future.

      @@ -156,7 +156,7 @@ kubectl edit (RESOURCE/NAME | -f FILENAME) --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_explain/_index.md b/content/en/docs/reference/kubectl/generated/kubectl_explain/_index.md index ef164f0445c33..74b2c5a6b3079 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_explain/_index.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_explain/_index.md @@ -33,7 +33,7 @@ Describe fields and structure of various resources. Use "kubectl api-resources" for a complete list of supported resources. ``` -kubectl explain TYPE [--recursive=FALSE|TRUE] [--api-version=api-version-group] [--output=plaintext|plaintext-openapiv2] +kubectl explain TYPE [--recursive=FALSE|TRUE] [--api-version=api-version-group] [-o|--output=plaintext|plaintext-openapiv2] ``` ## {{% heading "examples" %}} @@ -79,7 +79,7 @@ kubectl explain TYPE [--recursive=FALSE|TRUE] [--api-version=api-version-group] ---output string     Default: "plaintext" +-o, --output string     Default: "plaintext"

      Format in which to render the schema. Valid values are: (plaintext, plaintext-openapiv2).

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_get/_index.md b/content/en/docs/reference/kubectl/generated/kubectl_get/_index.md index fa6625435e736..af5c0472aeaa0 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_get/_index.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_get/_index.md @@ -233,7 +233,7 @@ kubectl get [(-o|--output=)json|yaml|name|go-template|go-template-file|template| --subresource string -

      If specified, gets the subresource of the requested object. Must be one of [status scale]. This flag is beta and may change in the future.

      +

      If specified, gets the subresource of the requested object. This flag is beta and may change in the future.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_kustomize/_index.md b/content/en/docs/reference/kubectl/generated/kubectl_kustomize/_index.md index 5dd4d67311afa..c84625ed146bc 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_kustomize/_index.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_kustomize/_index.md @@ -92,6 +92,13 @@ kubectl kustomize DIR [flags]

      helm command (path to executable)

      + +--helm-debug + + +

      Enable debug output from the Helm chart inflator generator.

      + + --helm-kube-version string diff --git a/content/en/docs/reference/kubectl/generated/kubectl_logs/_index.md b/content/en/docs/reference/kubectl/generated/kubectl_logs/_index.md index 8aee08d91492d..4d798954c63bc 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_logs/_index.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_logs/_index.md @@ -34,15 +34,33 @@ kubectl logs [-f] [-p] (POD | TYPE/NAME) [-c CONTAINER] # Return snapshot logs from pod nginx with only one container kubectl logs nginx + # Return snapshot logs from pod nginx, prefixing each line with the source pod and container name + kubectl logs nginx --prefix + + # Return snapshot logs from pod nginx, limiting output to 500 bytes + kubectl logs nginx --limit-bytes=500 + + # Return snapshot logs from pod nginx, waiting up to 20 seconds for it to start running. + kubectl logs nginx --pod-running-timeout=20s + # Return snapshot logs from pod nginx with multi containers kubectl logs nginx --all-containers=true + # Return snapshot logs from all pods in the deployment nginx + kubectl logs deployment/nginx --all-pods=true + # Return snapshot logs from all containers in pods defined by label app=nginx kubectl logs -l app=nginx --all-containers=true + # Return snapshot logs from all pods defined by label app=nginx, limiting concurrent log requests to 10 pods + kubectl logs -l app=nginx --max-log-requests=10 + # Return snapshot of previous terminated ruby container logs from pod web-1 kubectl logs -p -c ruby web-1 + # Begin streaming the logs from pod nginx, continuing even if errors occur + kubectl logs nginx -f --ignore-errors=true + # Begin streaming the logs of the ruby container in pod web-1 kubectl logs -f -c ruby web-1 @@ -55,6 +73,9 @@ kubectl logs [-f] [-p] (POD | TYPE/NAME) [-c CONTAINER] # Show all logs from pod nginx written in the last hour kubectl logs --since=1h nginx + # Show all logs with timestamps from pod nginx starting from August 30, 2024, at 06:00:00 UTC + kubectl logs nginx --since-time=2024-08-30T06:00:00Z --timestamps=true + # Show logs from a kubelet with an expired serving certificate kubectl logs --insecure-skip-tls-verify-backend nginx diff --git a/content/en/docs/reference/kubectl/generated/kubectl_patch/_index.md b/content/en/docs/reference/kubectl/generated/kubectl_patch/_index.md index fdfff6f8425f1..376250d82b622 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_patch/_index.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_patch/_index.md @@ -151,7 +151,7 @@ kubectl patch (-f FILENAME | TYPE NAME) [-p PATCH|--patch-file FILE] --subresource string -

      If specified, patch will operate on the subresource of the requested object. Must be one of [status scale]. This flag is beta and may change in the future.

      +

      If specified, patch will operate on the subresource of the requested object. This flag is beta and may change in the future.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_replace/_index.md b/content/en/docs/reference/kubectl/generated/kubectl_replace/_index.md index 8df6c2cff94be..e39375137f478 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_replace/_index.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_replace/_index.md @@ -159,7 +159,7 @@ kubectl replace -f FILENAME --subresource string -

      If specified, replace will operate on the subresource of the requested object. Must be one of [status scale]. This flag is beta and may change in the future.

      +

      If specified, replace will operate on the subresource of the requested object. This flag is beta and may change in the future.

      @@ -180,7 +180,7 @@ kubectl replace -f FILENAME --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_taint/_index.md b/content/en/docs/reference/kubectl/generated/kubectl_taint/_index.md index c2b6c0fd1f02a..d557209caf626 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_taint/_index.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_taint/_index.md @@ -138,7 +138,7 @@ kubectl taint NODE NAME KEY_1=VAL_1:TAINT_EFFECT_1 ... KEY_N=VAL_N:TAINT_EFFECT_ --validate string[="strict"]     Default: "strict" -

      Must be one of: strict (or true), warn, ignore (or false).
      "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not.
      "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise.
      "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      +

      Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.

      diff --git a/content/en/docs/reference/kubectl/generated/kubectl_wait/_index.md b/content/en/docs/reference/kubectl/generated/kubectl_wait/_index.md index 75641a0fca0d2..93bb3be038281 100644 --- a/content/en/docs/reference/kubectl/generated/kubectl_wait/_index.md +++ b/content/en/docs/reference/kubectl/generated/kubectl_wait/_index.md @@ -26,12 +26,12 @@ Experimental: Wait for a specific condition on one or many resources. The command takes multiple resources and waits until the specified condition is seen in the Status field of every given resource. - Alternatively, the command can wait for the given set of resources to be deleted by providing the "delete" keyword as the value to the --for flag. + Alternatively, the command can wait for the given set of resources to be created or deleted by providing the "create" or "delete" keyword as the value to the --for flag. A successful message will be printed to stdout indicating when the specified condition has been met. You can use -o option to change to output destination. ``` -kubectl wait ([-f FILENAME] | resource.group/resource.name | resource.group [(-l label | --all)]) [--for=delete|--for condition=available|--for=jsonpath='{}'[=value]] +kubectl wait ([-f FILENAME] | resource.group/resource.name | resource.group [(-l label | --all)]) [--for=create|--for=delete|--for condition=available|--for=jsonpath='{}'[=value]] ``` ## {{% heading "examples" %}} @@ -52,6 +52,10 @@ kubectl wait ([-f FILENAME] | resource.group/resource.name | resource.group [(-l # Wait for the service "loadbalancer" to have ingress kubectl wait --for=jsonpath='{.status.loadBalancer.ingress}' service/loadbalancer + # Wait for the secret "busybox1" to be created, with a timeout of 30s + kubectl create secret generic busybox1 + kubectl wait --for=create secret/busybox1 --timeout=30s + # Wait for the pod "busybox1" to be deleted, with a timeout of 60s, after having issued the "delete" command kubectl delete pod/busybox1 kubectl wait --for=delete pod/busybox1 --timeout=60s @@ -105,7 +109,7 @@ kubectl wait ([-f FILENAME] | resource.group/resource.name | resource.group [(-l --for string -

      The condition to wait on: [delete|condition=condition-name[=condition-value]|jsonpath='{JSONPath expression}'=[JSONPath value]]. The default condition-value is true. Condition values are compared after Unicode simple case folding, which is a more general form of case-insensitivity.

      +

      The condition to wait on: [create|delete|condition=condition-name[=condition-value]|jsonpath='{JSONPath expression}'=[JSONPath value]]. The default condition-value is true. Condition values are compared after Unicode simple case folding, which is a more general form of case-insensitivity.

      @@ -161,7 +165,7 @@ kubectl wait ([-f FILENAME] | resource.group/resource.name | resource.group [(-l --timeout duration     Default: 30s -

      The length of time to wait before giving up. Zero means check once and don't wait, negative means wait for a week.

      +

      The length of time to wait before giving up. Zero means check once and don't wait, negative means wait for a week.

      diff --git a/content/en/docs/reference/kubectl/kubectl.md b/content/en/docs/reference/kubectl/kubectl.md index c09a05dfccc4b..e5a2f29c4d760 100644 --- a/content/en/docs/reference/kubectl/kubectl.md +++ b/content/en/docs/reference/kubectl/kubectl.md @@ -350,14 +350,6 @@ kubectl [flags] When set to false, turns off extra HTTP headers detailing invoked kubectl command (Kubernetes version v1.22 or later) - -KUBECTL_DEBUG_CUSTOM_PROFILE - - -When set to true, custom flag will be enabled in kubectl debug. This flag is used to customize the pre-defined profiles. - - - KUBECTL_EXPLAIN_OPENAPIV3 diff --git a/content/en/docs/reference/kubernetes-api/extend-resources/device-class-v1alpha3.md b/content/en/docs/reference/kubernetes-api/extend-resources/device-class-v1alpha3.md index 624c501ba0425..57082c3e25521 100644 --- a/content/en/docs/reference/kubernetes-api/extend-resources/device-class-v1alpha3.md +++ b/content/en/docs/reference/kubernetes-api/extend-resources/device-class-v1alpha3.md @@ -19,6 +19,8 @@ To update the reference content, please follow the [Contributing upstream](/docs/contribute/generate-ref-docs/contribute-upstream/) guide. You can file document formatting bugs against the [reference-docs](https://github.com/kubernetes-sigs/reference-docs/) project. + +TODO: this file should be under "workload-resources". --> `apiVersion: resource.k8s.io/v1alpha3` diff --git a/content/en/docs/reference/kubernetes-api/extend-resources/device-class-v1beta1.md b/content/en/docs/reference/kubernetes-api/extend-resources/device-class-v1beta1.md new file mode 100644 index 0000000000000..ffbda055f13d5 --- /dev/null +++ b/content/en/docs/reference/kubernetes-api/extend-resources/device-class-v1beta1.md @@ -0,0 +1,629 @@ +--- +api_metadata: + apiVersion: "resource.k8s.io/v1beta1" + import: "k8s.io/api/resource/v1beta1" + kind: "DeviceClass" +content_type: "api_reference" +description: "DeviceClass is a vendor- or admin-provided resource that contains device configuration and selectors." +title: "DeviceClass v1beta1" +weight: 2 +auto_generated: true +--- + + + +`apiVersion: resource.k8s.io/v1beta1` + +`import "k8s.io/api/resource/v1beta1"` + + +## DeviceClass {#DeviceClass} + +DeviceClass is a vendor- or admin-provided resource that contains device configuration and selectors. It can be referenced in the device requests of a claim to apply these presets. Cluster scoped. + +This is an alpha type and requires enabling the DynamicResourceAllocation feature gate. + +
      + +- **apiVersion**: resource.k8s.io/v1beta1 + + +- **kind**: DeviceClass + + +- **metadata** (}}">ObjectMeta) + + Standard object metadata + +- **spec** (}}">DeviceClassSpec), required + + Spec defines what can be allocated and how to configure it. + + This is mutable. Consumers have to be prepared for classes changing at any time, either because they get updated or replaced. Claim allocations are done once based on whatever was set in classes at the time of allocation. + + Changing the spec automatically increments the metadata.generation number. + + + + + +## DeviceClassSpec {#DeviceClassSpec} + +DeviceClassSpec is used in a [DeviceClass] to define what can be allocated and how to configure it. + +
      + +- **config** ([]DeviceClassConfiguration) + + *Atomic: will be replaced during a merge* + + Config defines configuration parameters that apply to each device that is claimed via this class. Some classses may potentially be satisfied by multiple drivers, so each instance of a vendor configuration applies to exactly one driver. + + They are passed to the driver, but are not considered while allocating the claim. + + + *DeviceClassConfiguration is used in DeviceClass.* + + - **config.opaque** (OpaqueDeviceConfiguration) + + Opaque provides driver-specific configuration parameters. + + + *OpaqueDeviceConfiguration contains configuration parameters for a driver in a format defined by the driver vendor.* + + - **config.opaque.driver** (string), required + + Driver is used to determine which kubelet plugin needs to be passed these configuration parameters. + + An admission policy provided by the driver developer could use this to decide whether it needs to validate them. + + Must be a DNS subdomain and should end with a DNS domain owned by the vendor of the driver. + + - **config.opaque.parameters** (RawExtension), required + + Parameters can contain arbitrary data. It is the responsibility of the driver developer to handle validation and versioning. Typically this includes self-identification and a version ("kind" + "apiVersion" for Kubernetes types), with conversion between different versions. + + + *RawExtension is used to hold extensions in external versions. + + To use this, make a field which has RawExtension as its type in your external, versioned struct, and Object in your internal struct. You also need to register your various plugin types. + + // Internal package: + + type MyAPIObject struct { + runtime.TypeMeta `json:",inline"` + MyPlugin runtime.Object `json:"myPlugin"` + } + + type PluginA struct { + AOption string `json:"aOption"` + } + + // External package: + + type MyAPIObject struct { + runtime.TypeMeta `json:",inline"` + MyPlugin runtime.RawExtension `json:"myPlugin"` + } + + type PluginA struct { + AOption string `json:"aOption"` + } + + // On the wire, the JSON will look something like this: + + { + "kind":"MyAPIObject", + "apiVersion":"v1", + "myPlugin": { + "kind":"PluginA", + "aOption":"foo", + }, + } + + So what happens? Decode first uses json or yaml to unmarshal the serialized data into your external MyAPIObject. That causes the raw JSON to be stored, but not unpacked. The next step is to copy (using pkg/conversion) into the internal struct. The runtime package's DefaultScheme has conversion functions installed which will unpack the JSON stored in RawExtension, turning it into the correct object type, and storing it in the Object. (TODO: In the case where the object is of an unknown type, a runtime.Unknown object will be created and stored.)* + +- **selectors** ([]DeviceSelector) + + *Atomic: will be replaced during a merge* + + Each selector must be satisfied by a device which is claimed via this class. + + + *DeviceSelector must have exactly one field set.* + + - **selectors.cel** (CELDeviceSelector) + + CEL contains a CEL expression for selecting a device. + + + *CELDeviceSelector contains a CEL expression for selecting a device.* + + - **selectors.cel.expression** (string), required + + Expression is a CEL expression which evaluates a single device. It must evaluate to true when the device under consideration satisfies the desired criteria, and false when it does not. Any other result is an error and causes allocation of devices to abort. + + The expression's input is an object named "device", which carries the following properties: + - driver (string): the name of the driver which defines this device. + - attributes (map[string]object): the device's attributes, grouped by prefix + (e.g. device.attributes["dra.example.com"] evaluates to an object with all + of the attributes which were prefixed by "dra.example.com". + - capacity (map[string]object): the device's capacities, grouped by prefix. + + Example: Consider a device with driver="dra.example.com", which exposes two attributes named "model" and "ext.example.com/family" and which exposes one capacity named "modules". This input to this expression would have the following fields: + + device.driver + device.attributes["dra.example.com"].model + device.attributes["ext.example.com"].family + device.capacity["dra.example.com"].modules + + The device.driver field can be used to check for a specific driver, either as a high-level precondition (i.e. you only want to consider devices from this driver) or as part of a multi-clause expression that is meant to consider devices from different drivers. + + The value type of each attribute is defined by the device definition, and users who write these expressions must consult the documentation for their specific drivers. The value type of each capacity is Quantity. + + If an unknown prefix is used as a lookup in either device.attributes or device.capacity, an empty map will be returned. Any reference to an unknown field will cause an evaluation error and allocation to abort. + + A robust expression should check for the existence of attributes before referencing them. + + For ease of use, the cel.bind() function is enabled, and can be used to simplify expressions that access multiple attributes with the same domain. For example: + + cel.bind(dra, device.attributes["dra.example.com"], dra.someBool && dra.anotherBool) + +- **suitableNodes** (NodeSelector) + + Only nodes matching the selector will be considered by the scheduler when trying to find a Node that fits a Pod when that Pod uses a claim that has not been allocated yet *and* that claim gets allocated through a control plane controller. It is ignored when the claim does not use a control plane controller for allocation. + + Setting this field is optional. If unset, all Nodes are candidates. + + This is an alpha field and requires enabling the DRAControlPlaneController feature gate. + + + *A node selector represents the union of the results of one or more label queries over a set of nodes; that is, it represents the OR of the selectors represented by the node selector terms.* + + - **suitableNodes.nodeSelectorTerms** ([]NodeSelectorTerm), required + + *Atomic: will be replaced during a merge* + + Required. A list of node selector terms. The terms are ORed. + + + *A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.* + + - **suitableNodes.nodeSelectorTerms.matchExpressions** ([]}}">NodeSelectorRequirement) + + *Atomic: will be replaced during a merge* + + A list of node selector requirements by node's labels. + + - **suitableNodes.nodeSelectorTerms.matchFields** ([]}}">NodeSelectorRequirement) + + *Atomic: will be replaced during a merge* + + A list of node selector requirements by node's fields. + + + + + +## DeviceClassList {#DeviceClassList} + +DeviceClassList is a collection of classes. + +
      + +- **apiVersion**: resource.k8s.io/v1beta1 + + +- **kind**: DeviceClassList + + +- **metadata** (}}">ListMeta) + + Standard list metadata + +- **items** ([]}}">DeviceClass), required + + Items is the list of resource classes. + + + + + +## Operations {#Operations} + + + +
      + + + + + + +### `get` read the specified DeviceClass + +#### HTTP Request + +GET /apis/resource.k8s.io/v1beta1/deviceclasses/{name} + +#### Parameters + + +- **name** (*in path*): string, required + + name of the DeviceClass + + +- **pretty** (*in query*): string + + }}">pretty + + + +#### Response + + +200 (}}">DeviceClass): OK + +401: Unauthorized + + +### `list` list or watch objects of kind DeviceClass + +#### HTTP Request + +GET /apis/resource.k8s.io/v1beta1/deviceclasses + +#### Parameters + + +- **allowWatchBookmarks** (*in query*): boolean + + }}">allowWatchBookmarks + + +- **continue** (*in query*): string + + }}">continue + + +- **fieldSelector** (*in query*): string + + }}">fieldSelector + + +- **labelSelector** (*in query*): string + + }}">labelSelector + + +- **limit** (*in query*): integer + + }}">limit + + +- **pretty** (*in query*): string + + }}">pretty + + +- **resourceVersion** (*in query*): string + + }}">resourceVersion + + +- **resourceVersionMatch** (*in query*): string + + }}">resourceVersionMatch + + +- **sendInitialEvents** (*in query*): boolean + + }}">sendInitialEvents + + +- **timeoutSeconds** (*in query*): integer + + }}">timeoutSeconds + + +- **watch** (*in query*): boolean + + }}">watch + + + +#### Response + + +200 (}}">DeviceClassList): OK + +401: Unauthorized + + +### `create` create a DeviceClass + +#### HTTP Request + +POST /apis/resource.k8s.io/v1beta1/deviceclasses + +#### Parameters + + +- **body**: }}">DeviceClass, required + + + + +- **dryRun** (*in query*): string + + }}">dryRun + + +- **fieldManager** (*in query*): string + + }}">fieldManager + + +- **fieldValidation** (*in query*): string + + }}">fieldValidation + + +- **pretty** (*in query*): string + + }}">pretty + + + +#### Response + + +200 (}}">DeviceClass): OK + +201 (}}">DeviceClass): Created + +202 (}}">DeviceClass): Accepted + +401: Unauthorized + + +### `update` replace the specified DeviceClass + +#### HTTP Request + +PUT /apis/resource.k8s.io/v1beta1/deviceclasses/{name} + +#### Parameters + + +- **name** (*in path*): string, required + + name of the DeviceClass + + +- **body**: }}">DeviceClass, required + + + + +- **dryRun** (*in query*): string + + }}">dryRun + + +- **fieldManager** (*in query*): string + + }}">fieldManager + + +- **fieldValidation** (*in query*): string + + }}">fieldValidation + + +- **pretty** (*in query*): string + + }}">pretty + + + +#### Response + + +200 (}}">DeviceClass): OK + +201 (}}">DeviceClass): Created + +401: Unauthorized + + +### `patch` partially update the specified DeviceClass + +#### HTTP Request + +PATCH /apis/resource.k8s.io/v1beta1/deviceclasses/{name} + +#### Parameters + + +- **name** (*in path*): string, required + + name of the DeviceClass + + +- **body**: }}">Patch, required + + + + +- **dryRun** (*in query*): string + + }}">dryRun + + +- **fieldManager** (*in query*): string + + }}">fieldManager + + +- **fieldValidation** (*in query*): string + + }}">fieldValidation + + +- **force** (*in query*): boolean + + }}">force + + +- **pretty** (*in query*): string + + }}">pretty + + + +#### Response + + +200 (}}">DeviceClass): OK + +201 (}}">DeviceClass): Created + +401: Unauthorized + + +### `delete` delete a DeviceClass + +#### HTTP Request + +DELETE /apis/resource.k8s.io/v1beta1/deviceclasses/{name} + +#### Parameters + + +- **name** (*in path*): string, required + + name of the DeviceClass + + +- **body**: }}">DeleteOptions + + + + +- **dryRun** (*in query*): string + + }}">dryRun + + +- **gracePeriodSeconds** (*in query*): integer + + }}">gracePeriodSeconds + + +- **pretty** (*in query*): string + + }}">pretty + + +- **propagationPolicy** (*in query*): string + + }}">propagationPolicy + + + +#### Response + + +200 (}}">DeviceClass): OK + +202 (}}">DeviceClass): Accepted + +401: Unauthorized + + +### `deletecollection` delete collection of DeviceClass + +#### HTTP Request + +DELETE /apis/resource.k8s.io/v1beta1/deviceclasses + +#### Parameters + + +- **body**: }}">DeleteOptions + + + + +- **continue** (*in query*): string + + }}">continue + + +- **dryRun** (*in query*): string + + }}">dryRun + + +- **fieldSelector** (*in query*): string + + }}">fieldSelector + + +- **gracePeriodSeconds** (*in query*): integer + + }}">gracePeriodSeconds + + +- **labelSelector** (*in query*): string + + }}">labelSelector + + +- **limit** (*in query*): integer + + }}">limit + + +- **pretty** (*in query*): string + + }}">pretty + + +- **propagationPolicy** (*in query*): string + + }}">propagationPolicy + + +- **resourceVersion** (*in query*): string + + }}">resourceVersion + + +- **resourceVersionMatch** (*in query*): string + + }}">resourceVersionMatch + + +- **sendInitialEvents** (*in query*): boolean + + }}">sendInitialEvents + + +- **timeoutSeconds** (*in query*): integer + + }}">timeoutSeconds + + + +#### Response + + +200 (}}">Status): OK + +401: Unauthorized + diff --git a/content/en/docs/reference/kubernetes-api/workload-resources/resource-claim-template-v1beta1.md b/content/en/docs/reference/kubernetes-api/workload-resources/resource-claim-template-v1beta1.md new file mode 100644 index 0000000000000..e98a3d918fb15 --- /dev/null +++ b/content/en/docs/reference/kubernetes-api/workload-resources/resource-claim-template-v1beta1.md @@ -0,0 +1,597 @@ +--- +api_metadata: + apiVersion: "resource.k8s.io/v1beta1" + import: "k8s.io/api/resource/v1beta1" + kind: "ResourceClaimTemplate" +content_type: "api_reference" +description: "ResourceClaimTemplate is used to produce ResourceClaim objects." +title: "ResourceClaimTemplate v1beta1" +weight: 17 +auto_generated: true +--- + + + +`apiVersion: resource.k8s.io/v1beta1` + +`import "k8s.io/api/resource/v1beta1"` + + +## ResourceClaimTemplate {#ResourceClaimTemplate} + +ResourceClaimTemplate is used to produce ResourceClaim objects. + +This is an alpha type and requires enabling the DynamicResourceAllocation feature gate. + +
      + +- **apiVersion**: resource.k8s.io/v1beta1 + + +- **kind**: ResourceClaimTemplate + + +- **metadata** (}}">ObjectMeta) + + Standard object metadata + +- **spec** (}}">ResourceClaimTemplateSpec), required + + Describes the ResourceClaim that is to be generated. + + This field is immutable. A ResourceClaim will get created by the control plane for a Pod when needed and then not get updated anymore. + + + + + +## ResourceClaimTemplateSpec {#ResourceClaimTemplateSpec} + +ResourceClaimTemplateSpec contains the metadata and fields for a ResourceClaim. + +
      + +- **spec** (}}">ResourceClaimSpec), required + + Spec for the ResourceClaim. The entire content is copied unchanged into the ResourceClaim that gets created from this template. The same fields as in a ResourceClaim are also valid here. + +- **metadata** (}}">ObjectMeta) + + ObjectMeta may contain labels and annotations that will be copied into the PVC when creating it. No other fields are allowed and will be rejected during validation. + + + + + +## ResourceClaimTemplateList {#ResourceClaimTemplateList} + +ResourceClaimTemplateList is a collection of claim templates. + +
      + +- **apiVersion**: resource.k8s.io/v1beta1 + + +- **kind**: ResourceClaimTemplateList + + +- **metadata** (}}">ListMeta) + + Standard list metadata + +- **items** ([]}}">ResourceClaimTemplate), required + + Items is the list of resource claim templates. + + + + + +## Operations {#Operations} + + + +
      + + + + + + +### `get` read the specified ResourceClaimTemplate + +#### HTTP Request + +GET /apis/resource.k8s.io/v1beta1/namespaces/{namespace}/resourceclaimtemplates/{name} + +#### Parameters + + +- **name** (*in path*): string, required + + name of the ResourceClaimTemplate + + +- **namespace** (*in path*): string, required + + }}">namespace + + +- **pretty** (*in query*): string + + }}">pretty + + + +#### Response + + +200 (}}">ResourceClaimTemplate): OK + +401: Unauthorized + + +### `list` list or watch objects of kind ResourceClaimTemplate + +#### HTTP Request + +GET /apis/resource.k8s.io/v1beta1/namespaces/{namespace}/resourceclaimtemplates + +#### Parameters + + +- **namespace** (*in path*): string, required + + }}">namespace + + +- **allowWatchBookmarks** (*in query*): boolean + + }}">allowWatchBookmarks + + +- **continue** (*in query*): string + + }}">continue + + +- **fieldSelector** (*in query*): string + + }}">fieldSelector + + +- **labelSelector** (*in query*): string + + }}">labelSelector + + +- **limit** (*in query*): integer + + }}">limit + + +- **pretty** (*in query*): string + + }}">pretty + + +- **resourceVersion** (*in query*): string + + }}">resourceVersion + + +- **resourceVersionMatch** (*in query*): string + + }}">resourceVersionMatch + + +- **sendInitialEvents** (*in query*): boolean + + }}">sendInitialEvents + + +- **timeoutSeconds** (*in query*): integer + + }}">timeoutSeconds + + +- **watch** (*in query*): boolean + + }}">watch + + + +#### Response + + +200 (}}">ResourceClaimTemplateList): OK + +401: Unauthorized + + +### `list` list or watch objects of kind ResourceClaimTemplate + +#### HTTP Request + +GET /apis/resource.k8s.io/v1beta1/resourceclaimtemplates + +#### Parameters + + +- **allowWatchBookmarks** (*in query*): boolean + + }}">allowWatchBookmarks + + +- **continue** (*in query*): string + + }}">continue + + +- **fieldSelector** (*in query*): string + + }}">fieldSelector + + +- **labelSelector** (*in query*): string + + }}">labelSelector + + +- **limit** (*in query*): integer + + }}">limit + + +- **pretty** (*in query*): string + + }}">pretty + + +- **resourceVersion** (*in query*): string + + }}">resourceVersion + + +- **resourceVersionMatch** (*in query*): string + + }}">resourceVersionMatch + + +- **sendInitialEvents** (*in query*): boolean + + }}">sendInitialEvents + + +- **timeoutSeconds** (*in query*): integer + + }}">timeoutSeconds + + +- **watch** (*in query*): boolean + + }}">watch + + + +#### Response + + +200 (}}">ResourceClaimTemplateList): OK + +401: Unauthorized + + +### `create` create a ResourceClaimTemplate + +#### HTTP Request + +POST /apis/resource.k8s.io/v1beta1/namespaces/{namespace}/resourceclaimtemplates + +#### Parameters + + +- **namespace** (*in path*): string, required + + }}">namespace + + +- **body**: }}">ResourceClaimTemplate, required + + + + +- **dryRun** (*in query*): string + + }}">dryRun + + +- **fieldManager** (*in query*): string + + }}">fieldManager + + +- **fieldValidation** (*in query*): string + + }}">fieldValidation + + +- **pretty** (*in query*): string + + }}">pretty + + + +#### Response + + +200 (}}">ResourceClaimTemplate): OK + +201 (}}">ResourceClaimTemplate): Created + +202 (}}">ResourceClaimTemplate): Accepted + +401: Unauthorized + + +### `update` replace the specified ResourceClaimTemplate + +#### HTTP Request + +PUT /apis/resource.k8s.io/v1beta1/namespaces/{namespace}/resourceclaimtemplates/{name} + +#### Parameters + + +- **name** (*in path*): string, required + + name of the ResourceClaimTemplate + + +- **namespace** (*in path*): string, required + + }}">namespace + + +- **body**: }}">ResourceClaimTemplate, required + + + + +- **dryRun** (*in query*): string + + }}">dryRun + + +- **fieldManager** (*in query*): string + + }}">fieldManager + + +- **fieldValidation** (*in query*): string + + }}">fieldValidation + + +- **pretty** (*in query*): string + + }}">pretty + + + +#### Response + + +200 (}}">ResourceClaimTemplate): OK + +201 (}}">ResourceClaimTemplate): Created + +401: Unauthorized + + +### `patch` partially update the specified ResourceClaimTemplate + +#### HTTP Request + +PATCH /apis/resource.k8s.io/v1beta1/namespaces/{namespace}/resourceclaimtemplates/{name} + +#### Parameters + + +- **name** (*in path*): string, required + + name of the ResourceClaimTemplate + + +- **namespace** (*in path*): string, required + + }}">namespace + + +- **body**: }}">Patch, required + + + + +- **dryRun** (*in query*): string + + }}">dryRun + + +- **fieldManager** (*in query*): string + + }}">fieldManager + + +- **fieldValidation** (*in query*): string + + }}">fieldValidation + + +- **force** (*in query*): boolean + + }}">force + + +- **pretty** (*in query*): string + + }}">pretty + + + +#### Response + + +200 (}}">ResourceClaimTemplate): OK + +201 (}}">ResourceClaimTemplate): Created + +401: Unauthorized + + +### `delete` delete a ResourceClaimTemplate + +#### HTTP Request + +DELETE /apis/resource.k8s.io/v1beta1/namespaces/{namespace}/resourceclaimtemplates/{name} + +#### Parameters + + +- **name** (*in path*): string, required + + name of the ResourceClaimTemplate + + +- **namespace** (*in path*): string, required + + }}">namespace + + +- **body**: }}">DeleteOptions + + + + +- **dryRun** (*in query*): string + + }}">dryRun + + +- **gracePeriodSeconds** (*in query*): integer + + }}">gracePeriodSeconds + + +- **pretty** (*in query*): string + + }}">pretty + + +- **propagationPolicy** (*in query*): string + + }}">propagationPolicy + + + +#### Response + + +200 (}}">ResourceClaimTemplate): OK + +202 (}}">ResourceClaimTemplate): Accepted + +401: Unauthorized + + +### `deletecollection` delete collection of ResourceClaimTemplate + +#### HTTP Request + +DELETE /apis/resource.k8s.io/v1beta1/namespaces/{namespace}/resourceclaimtemplates + +#### Parameters + + +- **namespace** (*in path*): string, required + + }}">namespace + + +- **body**: }}">DeleteOptions + + + + +- **continue** (*in query*): string + + }}">continue + + +- **dryRun** (*in query*): string + + }}">dryRun + + +- **fieldSelector** (*in query*): string + + }}">fieldSelector + + +- **gracePeriodSeconds** (*in query*): integer + + }}">gracePeriodSeconds + + +- **labelSelector** (*in query*): string + + }}">labelSelector + + +- **limit** (*in query*): integer + + }}">limit + + +- **pretty** (*in query*): string + + }}">pretty + + +- **propagationPolicy** (*in query*): string + + }}">propagationPolicy + + +- **resourceVersion** (*in query*): string + + }}">resourceVersion + + +- **resourceVersionMatch** (*in query*): string + + }}">resourceVersionMatch + + +- **sendInitialEvents** (*in query*): boolean + + }}">sendInitialEvents + + +- **timeoutSeconds** (*in query*): integer + + }}">timeoutSeconds + + + +#### Response + + +200 (}}">Status): OK + +401: Unauthorized + diff --git a/content/en/docs/reference/kubernetes-api/workload-resources/resource-claim-v1beta1.md b/content/en/docs/reference/kubernetes-api/workload-resources/resource-claim-v1beta1.md new file mode 100644 index 0000000000000..80767f613200e --- /dev/null +++ b/content/en/docs/reference/kubernetes-api/workload-resources/resource-claim-v1beta1.md @@ -0,0 +1,1156 @@ +--- +api_metadata: + apiVersion: "resource.k8s.io/v1beta1" + import: "k8s.io/api/resource/v1beta1" + kind: "ResourceClaim" +content_type: "api_reference" +description: "ResourceClaim describes a request for access to resources in the cluster, for use by workloads." +title: "ResourceClaim v1beta1" +weight: 16 +auto_generated: true +--- + + + +`apiVersion: resource.k8s.io/v1beta1` + +`import "k8s.io/api/resource/v1beta1"` + + +## ResourceClaim {#ResourceClaim} + +ResourceClaim describes a request for access to resources in the cluster, for use by workloads. For example, if a workload needs an accelerator device with specific properties, this is how that request is expressed. The status stanza tracks whether this claim has been satisfied and what specific resources have been allocated. + +This is an alpha type and requires enabling the DynamicResourceAllocation feature gate. + +
      + +- **apiVersion**: resource.k8s.io/v1beta1 + + +- **kind**: ResourceClaim + + +- **metadata** (}}">ObjectMeta) + + Standard object metadata + +- **spec** (}}">ResourceClaimSpec), required + + Spec describes what is being requested and how to configure it. The spec is immutable. + +- **status** (}}">ResourceClaimStatus) + + Status describes whether the claim is ready to use and what has been allocated. + + + + + +## ResourceClaimSpec {#ResourceClaimSpec} + +ResourceClaimSpec defines what is being requested in a ResourceClaim and how to configure it. + +
      + +- **controller** (string) + + Controller is the name of the DRA driver that is meant to handle allocation of this claim. If empty, allocation is handled by the scheduler while scheduling a pod. + + Must be a DNS subdomain and should end with a DNS domain owned by the vendor of the driver. + + This is an alpha field and requires enabling the DRAControlPlaneController feature gate. + +- **devices** (DeviceClaim) + + Devices defines how to request devices. + + + *DeviceClaim defines how to request devices with a ResourceClaim.* + + - **devices.config** ([]DeviceClaimConfiguration) + + *Atomic: will be replaced during a merge* + + This field holds configuration for multiple potential drivers which could satisfy requests in this claim. It is ignored while allocating the claim. + + + *DeviceClaimConfiguration is used for configuration parameters in DeviceClaim.* + + - **devices.config.opaque** (OpaqueDeviceConfiguration) + + Opaque provides driver-specific configuration parameters. + + + *OpaqueDeviceConfiguration contains configuration parameters for a driver in a format defined by the driver vendor.* + + - **devices.config.opaque.driver** (string), required + + Driver is used to determine which kubelet plugin needs to be passed these configuration parameters. + + An admission policy provided by the driver developer could use this to decide whether it needs to validate them. + + Must be a DNS subdomain and should end with a DNS domain owned by the vendor of the driver. + + - **devices.config.opaque.parameters** (RawExtension), required + + Parameters can contain arbitrary data. It is the responsibility of the driver developer to handle validation and versioning. Typically this includes self-identification and a version ("kind" + "apiVersion" for Kubernetes types), with conversion between different versions. + + + *RawExtension is used to hold extensions in external versions. + + To use this, make a field which has RawExtension as its type in your external, versioned struct, and Object in your internal struct. You also need to register your various plugin types. + + // Internal package: + + type MyAPIObject struct { + runtime.TypeMeta `json:",inline"` + MyPlugin runtime.Object `json:"myPlugin"` + } + + type PluginA struct { + AOption string `json:"aOption"` + } + + // External package: + + type MyAPIObject struct { + runtime.TypeMeta `json:",inline"` + MyPlugin runtime.RawExtension `json:"myPlugin"` + } + + type PluginA struct { + AOption string `json:"aOption"` + } + + // On the wire, the JSON will look something like this: + + { + "kind":"MyAPIObject", + "apiVersion":"v1", + "myPlugin": { + "kind":"PluginA", + "aOption":"foo", + }, + } + + So what happens? Decode first uses json or yaml to unmarshal the serialized data into your external MyAPIObject. That causes the raw JSON to be stored, but not unpacked. The next step is to copy (using pkg/conversion) into the internal struct. The runtime package's DefaultScheme has conversion functions installed which will unpack the JSON stored in RawExtension, turning it into the correct object type, and storing it in the Object. (TODO: In the case where the object is of an unknown type, a runtime.Unknown object will be created and stored.)* + + - **devices.config.requests** ([]string) + + *Atomic: will be replaced during a merge* + + Requests lists the names of requests where the configuration applies. If empty, it applies to all requests. + + - **devices.constraints** ([]DeviceConstraint) + + *Atomic: will be replaced during a merge* + + These constraints must be satisfied by the set of devices that get allocated for the claim. + + + *DeviceConstraint must have exactly one field set besides Requests.* + + - **devices.constraints.matchAttribute** (string) + + MatchAttribute requires that all devices in question have this attribute and that its type and value are the same across those devices. + + For example, if you specified "dra.example.com/numa" (a hypothetical example!), then only devices in the same NUMA node will be chosen. A device which does not have that attribute will not be chosen. All devices should use a value of the same type for this attribute because that is part of its specification, but if one device doesn't, then it also will not be chosen. + + Must include the domain qualifier. + + - **devices.constraints.requests** ([]string) + + *Atomic: will be replaced during a merge* + + Requests is a list of the one or more requests in this claim which must co-satisfy this constraint. If a request is fulfilled by multiple devices, then all of the devices must satisfy the constraint. If this is not specified, this constraint applies to all requests in this claim. + + - **devices.requests** ([]DeviceRequest) + + *Atomic: will be replaced during a merge* + + Requests represent individual requests for distinct devices which must all be satisfied. If empty, nothing needs to be allocated. + + + *DeviceRequest is a request for devices required for a claim. This is typically a request for a single resource like a device, but can also ask for several identical devices. + + A DeviceClassName is currently required. Clients must check that it is indeed set. It's absence indicates that something changed in a way that is not supported by the client yet, in which case it must refuse to handle the request.* + + - **devices.requests.deviceClassName** (string), required + + DeviceClassName references a specific DeviceClass, which can define additional configuration and selectors to be inherited by this request. + + A class is required. Which classes are available depends on the cluster. + + Administrators may use this to restrict which devices may get requested by only installing classes with selectors for permitted devices. If users are free to request anything without restrictions, then administrators can create an empty DeviceClass for users to reference. + + - **devices.requests.name** (string), required + + Name can be used to reference this request in a pod.spec.containers[].resources.claims entry and in a constraint of the claim. + + Must be a DNS label. + + - **devices.requests.adminAccess** (boolean) + + AdminAccess indicates that this is a claim for administrative access to the device(s). Claims with AdminAccess are expected to be used for monitoring or other management services for a device. They ignore all ordinary claims to the device with respect to access modes and any resource allocations. + + - **devices.requests.allocationMode** (string) + + AllocationMode and its related fields define how devices are allocated to satisfy this request. Supported values are: + + - ExactCount: This request is for a specific number of devices. + This is the default. The exact number is provided in the + count field. + + - All: This request is for all of the matching devices in a pool. + Allocation will fail if some devices are already allocated, + unless adminAccess is requested. + + If AlloctionMode is not specified, the default mode is ExactCount. If the mode is ExactCount and count is not specified, the default count is one. Any other requests must specify this field. + + More modes may get added in the future. Clients must refuse to handle requests with unknown modes. + + - **devices.requests.count** (int64) + + Count is used only when the count mode is "ExactCount". Must be greater than zero. If AllocationMode is ExactCount and this field is not specified, the default is one. + + - **devices.requests.selectors** ([]DeviceSelector) + + *Atomic: will be replaced during a merge* + + Selectors define criteria which must be satisfied by a specific device in order for that device to be considered for this request. All selectors must be satisfied for a device to be considered. + + + *DeviceSelector must have exactly one field set.* + + - **devices.requests.selectors.cel** (CELDeviceSelector) + + CEL contains a CEL expression for selecting a device. + + + *CELDeviceSelector contains a CEL expression for selecting a device.* + + - **devices.requests.selectors.cel.expression** (string), required + + Expression is a CEL expression which evaluates a single device. It must evaluate to true when the device under consideration satisfies the desired criteria, and false when it does not. Any other result is an error and causes allocation of devices to abort. + + The expression's input is an object named "device", which carries the following properties: + - driver (string): the name of the driver which defines this device. + - attributes (map[string]object): the device's attributes, grouped by prefix + (e.g. device.attributes["dra.example.com"] evaluates to an object with all + of the attributes which were prefixed by "dra.example.com". + - capacity (map[string]object): the device's capacities, grouped by prefix. + + Example: Consider a device with driver="dra.example.com", which exposes two attributes named "model" and "ext.example.com/family" and which exposes one capacity named "modules". This input to this expression would have the following fields: + + device.driver + device.attributes["dra.example.com"].model + device.attributes["ext.example.com"].family + device.capacity["dra.example.com"].modules + + The device.driver field can be used to check for a specific driver, either as a high-level precondition (i.e. you only want to consider devices from this driver) or as part of a multi-clause expression that is meant to consider devices from different drivers. + + The value type of each attribute is defined by the device definition, and users who write these expressions must consult the documentation for their specific drivers. The value type of each capacity is Quantity. + + If an unknown prefix is used as a lookup in either device.attributes or device.capacity, an empty map will be returned. Any reference to an unknown field will cause an evaluation error and allocation to abort. + + A robust expression should check for the existence of attributes before referencing them. + + For ease of use, the cel.bind() function is enabled, and can be used to simplify expressions that access multiple attributes with the same domain. For example: + + cel.bind(dra, device.attributes["dra.example.com"], dra.someBool && dra.anotherBool) + + + + + +## ResourceClaimStatus {#ResourceClaimStatus} + +ResourceClaimStatus tracks whether the resource has been allocated and what the result of that was. + +
      + +- **allocation** (AllocationResult) + + Allocation is set once the claim has been allocated successfully. + + + *AllocationResult contains attributes of an allocated resource.* + + - **allocation.controller** (string) + + Controller is the name of the DRA driver which handled the allocation. That driver is also responsible for deallocating the claim. It is empty when the claim can be deallocated without involving a driver. + + A driver may allocate devices provided by other drivers, so this driver name here can be different from the driver names listed for the results. + + This is an alpha field and requires enabling the DRAControlPlaneController feature gate. + + - **allocation.devices** (DeviceAllocationResult) + + Devices is the result of allocating devices. + + + *DeviceAllocationResult is the result of allocating devices.* + + - **allocation.devices.config** ([]DeviceAllocationConfiguration) + + *Atomic: will be replaced during a merge* + + This field is a combination of all the claim and class configuration parameters. Drivers can distinguish between those based on a flag. + + This includes configuration parameters for drivers which have no allocated devices in the result because it is up to the drivers which configuration parameters they support. They can silently ignore unknown configuration parameters. + + + *DeviceAllocationConfiguration gets embedded in an AllocationResult.* + + - **allocation.devices.config.source** (string), required + + Source records whether the configuration comes from a class and thus is not something that a normal user would have been able to set or from a claim. + + - **allocation.devices.config.opaque** (OpaqueDeviceConfiguration) + + Opaque provides driver-specific configuration parameters. + + + *OpaqueDeviceConfiguration contains configuration parameters for a driver in a format defined by the driver vendor.* + + - **allocation.devices.config.opaque.driver** (string), required + + Driver is used to determine which kubelet plugin needs to be passed these configuration parameters. + + An admission policy provided by the driver developer could use this to decide whether it needs to validate them. + + Must be a DNS subdomain and should end with a DNS domain owned by the vendor of the driver. + + - **allocation.devices.config.opaque.parameters** (RawExtension), required + + Parameters can contain arbitrary data. It is the responsibility of the driver developer to handle validation and versioning. Typically this includes self-identification and a version ("kind" + "apiVersion" for Kubernetes types), with conversion between different versions. + + + *RawExtension is used to hold extensions in external versions. + + To use this, make a field which has RawExtension as its type in your external, versioned struct, and Object in your internal struct. You also need to register your various plugin types. + + // Internal package: + + type MyAPIObject struct { + runtime.TypeMeta `json:",inline"` + MyPlugin runtime.Object `json:"myPlugin"` + } + + type PluginA struct { + AOption string `json:"aOption"` + } + + // External package: + + type MyAPIObject struct { + runtime.TypeMeta `json:",inline"` + MyPlugin runtime.RawExtension `json:"myPlugin"` + } + + type PluginA struct { + AOption string `json:"aOption"` + } + + // On the wire, the JSON will look something like this: + + { + "kind":"MyAPIObject", + "apiVersion":"v1", + "myPlugin": { + "kind":"PluginA", + "aOption":"foo", + }, + } + + So what happens? Decode first uses json or yaml to unmarshal the serialized data into your external MyAPIObject. That causes the raw JSON to be stored, but not unpacked. The next step is to copy (using pkg/conversion) into the internal struct. The runtime package's DefaultScheme has conversion functions installed which will unpack the JSON stored in RawExtension, turning it into the correct object type, and storing it in the Object. (TODO: In the case where the object is of an unknown type, a runtime.Unknown object will be created and stored.)* + + - **allocation.devices.config.requests** ([]string) + + *Atomic: will be replaced during a merge* + + Requests lists the names of requests where the configuration applies. If empty, its applies to all requests. + + - **allocation.devices.results** ([]DeviceRequestAllocationResult) + + *Atomic: will be replaced during a merge* + + Results lists all allocated devices. + + + *DeviceRequestAllocationResult contains the allocation result for one request.* + + - **allocation.devices.results.device** (string), required + + Device references one device instance via its name in the driver's resource pool. It must be a DNS label. + + - **allocation.devices.results.driver** (string), required + + Driver specifies the name of the DRA driver whose kubelet plugin should be invoked to process the allocation once the claim is needed on a node. + + Must be a DNS subdomain and should end with a DNS domain owned by the vendor of the driver. + + - **allocation.devices.results.pool** (string), required + + This name together with the driver name and the device name field identify which device was allocated (`\/\/\`). + + Must not be longer than 253 characters and may contain one or more DNS sub-domains separated by slashes. + + - **allocation.devices.results.request** (string), required + + Request is the name of the request in the claim which caused this device to be allocated. Multiple devices may have been allocated per request. + + - **allocation.nodeSelector** (NodeSelector) + + NodeSelector defines where the allocated resources are available. If unset, they are available everywhere. + + + *A node selector represents the union of the results of one or more label queries over a set of nodes; that is, it represents the OR of the selectors represented by the node selector terms.* + + - **allocation.nodeSelector.nodeSelectorTerms** ([]NodeSelectorTerm), required + + *Atomic: will be replaced during a merge* + + Required. A list of node selector terms. The terms are ORed. + + + *A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.* + + - **allocation.nodeSelector.nodeSelectorTerms.matchExpressions** ([]}}">NodeSelectorRequirement) + + *Atomic: will be replaced during a merge* + + A list of node selector requirements by node's labels. + + - **allocation.nodeSelector.nodeSelectorTerms.matchFields** ([]}}">NodeSelectorRequirement) + + *Atomic: will be replaced during a merge* + + A list of node selector requirements by node's fields. + +- **deallocationRequested** (boolean) + + Indicates that a claim is to be deallocated. While this is set, no new consumers may be added to ReservedFor. + + This is only used if the claim needs to be deallocated by a DRA driver. That driver then must deallocate this claim and reset the field together with clearing the Allocation field. + + This is an alpha field and requires enabling the DRAControlPlaneController feature gate. + +- **reservedFor** ([]ResourceClaimConsumerReference) + + *Patch strategy: merge on key `uid`* + + *Map: unique values on key uid will be kept during a merge* + + ReservedFor indicates which entities are currently allowed to use the claim. A Pod which references a ResourceClaim which is not reserved for that Pod will not be started. A claim that is in use or might be in use because it has been reserved must not get deallocated. + + In a cluster with multiple scheduler instances, two pods might get scheduled concurrently by different schedulers. When they reference the same ResourceClaim which already has reached its maximum number of consumers, only one pod can be scheduled. + + Both schedulers try to add their pod to the claim.status.reservedFor field, but only the update that reaches the API server first gets stored. The other one fails with an error and the scheduler which issued it knows that it must put the pod back into the queue, waiting for the ResourceClaim to become usable again. + + There can be at most 32 such reservations. This may get increased in the future, but not reduced. + + + *ResourceClaimConsumerReference contains enough information to let you locate the consumer of a ResourceClaim. The user must be a resource in the same namespace as the ResourceClaim.* + + - **reservedFor.name** (string), required + + Name is the name of resource being referenced. + + - **reservedFor.resource** (string), required + + Resource is the type of resource being referenced, for example "pods". + + - **reservedFor.uid** (string), required + + UID identifies exactly one incarnation of the resource. + + - **reservedFor.apiGroup** (string) + + APIGroup is the group for the resource being referenced. It is empty for the core API. This matches the group in the APIVersion that is used when creating the resources. + + + + + +## ResourceClaimList {#ResourceClaimList} + +ResourceClaimList is a collection of claims. + +
      + +- **apiVersion**: resource.k8s.io/v1beta1 + + +- **kind**: ResourceClaimList + + +- **metadata** (}}">ListMeta) + + Standard list metadata + +- **items** ([]}}">ResourceClaim), required + + Items is the list of resource claims. + + + + + +## Operations {#Operations} + + + +
      + + + + + + +### `get` read the specified ResourceClaim + +#### HTTP Request + +GET /apis/resource.k8s.io/v1beta1/namespaces/{namespace}/resourceclaims/{name} + +#### Parameters + + +- **name** (*in path*): string, required + + name of the ResourceClaim + + +- **namespace** (*in path*): string, required + + }}">namespace + + +- **pretty** (*in query*): string + + }}">pretty + + + +#### Response + + +200 (}}">ResourceClaim): OK + +401: Unauthorized + + +### `get` read status of the specified ResourceClaim + +#### HTTP Request + +GET /apis/resource.k8s.io/v1beta1/namespaces/{namespace}/resourceclaims/{name}/status + +#### Parameters + + +- **name** (*in path*): string, required + + name of the ResourceClaim + + +- **namespace** (*in path*): string, required + + }}">namespace + + +- **pretty** (*in query*): string + + }}">pretty + + + +#### Response + + +200 (}}">ResourceClaim): OK + +401: Unauthorized + + +### `list` list or watch objects of kind ResourceClaim + +#### HTTP Request + +GET /apis/resource.k8s.io/v1beta1/namespaces/{namespace}/resourceclaims + +#### Parameters + + +- **namespace** (*in path*): string, required + + }}">namespace + + +- **allowWatchBookmarks** (*in query*): boolean + + }}">allowWatchBookmarks + + +- **continue** (*in query*): string + + }}">continue + + +- **fieldSelector** (*in query*): string + + }}">fieldSelector + + +- **labelSelector** (*in query*): string + + }}">labelSelector + + +- **limit** (*in query*): integer + + }}">limit + + +- **pretty** (*in query*): string + + }}">pretty + + +- **resourceVersion** (*in query*): string + + }}">resourceVersion + + +- **resourceVersionMatch** (*in query*): string + + }}">resourceVersionMatch + + +- **sendInitialEvents** (*in query*): boolean + + }}">sendInitialEvents + + +- **timeoutSeconds** (*in query*): integer + + }}">timeoutSeconds + + +- **watch** (*in query*): boolean + + }}">watch + + + +#### Response + + +200 (}}">ResourceClaimList): OK + +401: Unauthorized + + +### `list` list or watch objects of kind ResourceClaim + +#### HTTP Request + +GET /apis/resource.k8s.io/v1beta1/resourceclaims + +#### Parameters + + +- **allowWatchBookmarks** (*in query*): boolean + + }}">allowWatchBookmarks + + +- **continue** (*in query*): string + + }}">continue + + +- **fieldSelector** (*in query*): string + + }}">fieldSelector + + +- **labelSelector** (*in query*): string + + }}">labelSelector + + +- **limit** (*in query*): integer + + }}">limit + + +- **pretty** (*in query*): string + + }}">pretty + + +- **resourceVersion** (*in query*): string + + }}">resourceVersion + + +- **resourceVersionMatch** (*in query*): string + + }}">resourceVersionMatch + + +- **sendInitialEvents** (*in query*): boolean + + }}">sendInitialEvents + + +- **timeoutSeconds** (*in query*): integer + + }}">timeoutSeconds + + +- **watch** (*in query*): boolean + + }}">watch + + + +#### Response + + +200 (}}">ResourceClaimList): OK + +401: Unauthorized + + +### `create` create a ResourceClaim + +#### HTTP Request + +POST /apis/resource.k8s.io/v1beta1/namespaces/{namespace}/resourceclaims + +#### Parameters + + +- **namespace** (*in path*): string, required + + }}">namespace + + +- **body**: }}">ResourceClaim, required + + + + +- **dryRun** (*in query*): string + + }}">dryRun + + +- **fieldManager** (*in query*): string + + }}">fieldManager + + +- **fieldValidation** (*in query*): string + + }}">fieldValidation + + +- **pretty** (*in query*): string + + }}">pretty + + + +#### Response + + +200 (}}">ResourceClaim): OK + +201 (}}">ResourceClaim): Created + +202 (}}">ResourceClaim): Accepted + +401: Unauthorized + + +### `update` replace the specified ResourceClaim + +#### HTTP Request + +PUT /apis/resource.k8s.io/v1beta1/namespaces/{namespace}/resourceclaims/{name} + +#### Parameters + + +- **name** (*in path*): string, required + + name of the ResourceClaim + + +- **namespace** (*in path*): string, required + + }}">namespace + + +- **body**: }}">ResourceClaim, required + + + + +- **dryRun** (*in query*): string + + }}">dryRun + + +- **fieldManager** (*in query*): string + + }}">fieldManager + + +- **fieldValidation** (*in query*): string + + }}">fieldValidation + + +- **pretty** (*in query*): string + + }}">pretty + + + +#### Response + + +200 (}}">ResourceClaim): OK + +201 (}}">ResourceClaim): Created + +401: Unauthorized + + +### `update` replace status of the specified ResourceClaim + +#### HTTP Request + +PUT /apis/resource.k8s.io/v1beta1/namespaces/{namespace}/resourceclaims/{name}/status + +#### Parameters + + +- **name** (*in path*): string, required + + name of the ResourceClaim + + +- **namespace** (*in path*): string, required + + }}">namespace + + +- **body**: }}">ResourceClaim, required + + + + +- **dryRun** (*in query*): string + + }}">dryRun + + +- **fieldManager** (*in query*): string + + }}">fieldManager + + +- **fieldValidation** (*in query*): string + + }}">fieldValidation + + +- **pretty** (*in query*): string + + }}">pretty + + + +#### Response + + +200 (}}">ResourceClaim): OK + +201 (}}">ResourceClaim): Created + +401: Unauthorized + + +### `patch` partially update the specified ResourceClaim + +#### HTTP Request + +PATCH /apis/resource.k8s.io/v1beta1/namespaces/{namespace}/resourceclaims/{name} + +#### Parameters + + +- **name** (*in path*): string, required + + name of the ResourceClaim + + +- **namespace** (*in path*): string, required + + }}">namespace + + +- **body**: }}">Patch, required + + + + +- **dryRun** (*in query*): string + + }}">dryRun + + +- **fieldManager** (*in query*): string + + }}">fieldManager + + +- **fieldValidation** (*in query*): string + + }}">fieldValidation + + +- **force** (*in query*): boolean + + }}">force + + +- **pretty** (*in query*): string + + }}">pretty + + + +#### Response + + +200 (}}">ResourceClaim): OK + +201 (}}">ResourceClaim): Created + +401: Unauthorized + + +### `patch` partially update status of the specified ResourceClaim + +#### HTTP Request + +PATCH /apis/resource.k8s.io/v1beta1/namespaces/{namespace}/resourceclaims/{name}/status + +#### Parameters + + +- **name** (*in path*): string, required + + name of the ResourceClaim + + +- **namespace** (*in path*): string, required + + }}">namespace + + +- **body**: }}">Patch, required + + + + +- **dryRun** (*in query*): string + + }}">dryRun + + +- **fieldManager** (*in query*): string + + }}">fieldManager + + +- **fieldValidation** (*in query*): string + + }}">fieldValidation + + +- **force** (*in query*): boolean + + }}">force + + +- **pretty** (*in query*): string + + }}">pretty + + + +#### Response + + +200 (}}">ResourceClaim): OK + +201 (}}">ResourceClaim): Created + +401: Unauthorized + + +### `delete` delete a ResourceClaim + +#### HTTP Request + +DELETE /apis/resource.k8s.io/v1beta1/namespaces/{namespace}/resourceclaims/{name} + +#### Parameters + + +- **name** (*in path*): string, required + + name of the ResourceClaim + + +- **namespace** (*in path*): string, required + + }}">namespace + + +- **body**: }}">DeleteOptions + + + + +- **dryRun** (*in query*): string + + }}">dryRun + + +- **gracePeriodSeconds** (*in query*): integer + + }}">gracePeriodSeconds + + +- **pretty** (*in query*): string + + }}">pretty + + +- **propagationPolicy** (*in query*): string + + }}">propagationPolicy + + + +#### Response + + +200 (}}">ResourceClaim): OK + +202 (}}">ResourceClaim): Accepted + +401: Unauthorized + + +### `deletecollection` delete collection of ResourceClaim + +#### HTTP Request + +DELETE /apis/resource.k8s.io/v1beta1/namespaces/{namespace}/resourceclaims + +#### Parameters + + +- **namespace** (*in path*): string, required + + }}">namespace + + +- **body**: }}">DeleteOptions + + + + +- **continue** (*in query*): string + + }}">continue + + +- **dryRun** (*in query*): string + + }}">dryRun + + +- **fieldSelector** (*in query*): string + + }}">fieldSelector + + +- **gracePeriodSeconds** (*in query*): integer + + }}">gracePeriodSeconds + + +- **labelSelector** (*in query*): string + + }}">labelSelector + + +- **limit** (*in query*): integer + + }}">limit + + +- **pretty** (*in query*): string + + }}">pretty + + +- **propagationPolicy** (*in query*): string + + }}">propagationPolicy + + +- **resourceVersion** (*in query*): string + + }}">resourceVersion + + +- **resourceVersionMatch** (*in query*): string + + }}">resourceVersionMatch + + +- **sendInitialEvents** (*in query*): boolean + + }}">sendInitialEvents + + +- **timeoutSeconds** (*in query*): integer + + }}">timeoutSeconds + + + +#### Response + + +200 (}}">Status): OK + +401: Unauthorized + diff --git a/content/en/docs/reference/kubernetes-api/workload-resources/resource-slice-v1beta1.md b/content/en/docs/reference/kubernetes-api/workload-resources/resource-slice-v1beta1.md new file mode 100644 index 0000000000000..00a1a2e0b2cbd --- /dev/null +++ b/content/en/docs/reference/kubernetes-api/workload-resources/resource-slice-v1beta1.md @@ -0,0 +1,617 @@ +--- +api_metadata: + apiVersion: "resource.k8s.io/v1beta1" + import: "k8s.io/api/resource/v1beta1" + kind: "ResourceSlice" +content_type: "api_reference" +description: "ResourceSlice represents one or more resources in a pool of similar resources, managed by a common driver." +title: "ResourceSlice v1beta1" +weight: 18 +auto_generated: true +--- + + + +`apiVersion: resource.k8s.io/v1beta1` + +`import "k8s.io/api/resource/v1beta1"` + + +## ResourceSlice {#ResourceSlice} + +ResourceSlice represents one or more resources in a pool of similar resources, managed by a common driver. A pool may span more than one ResourceSlice, and exactly how many ResourceSlices comprise a pool is determined by the driver. + +At the moment, the only supported resources are devices with attributes and capacities. Each device in a given pool, regardless of how many ResourceSlices, must have a unique name. The ResourceSlice in which a device gets published may change over time. The unique identifier for a device is the tuple \, \, \. + +Whenever a driver needs to update a pool, it increments the pool.Spec.Pool.Generation number and updates all ResourceSlices with that new number and new resource definitions. A consumer must only use ResourceSlices with the highest generation number and ignore all others. + +When allocating all resources in a pool matching certain criteria or when looking for the best solution among several different alternatives, a consumer should check the number of ResourceSlices in a pool (included in each ResourceSlice) to determine whether its view of a pool is complete and if not, should wait until the driver has completed updating the pool. + +For resources that are not local to a node, the node name is not set. Instead, the driver may use a node selector to specify where the devices are available. + +This is an alpha type and requires enabling the DynamicResourceAllocation feature gate. + +
      + +- **apiVersion**: resource.k8s.io/v1beta1 + + +- **kind**: ResourceSlice + + +- **metadata** (}}">ObjectMeta) + + Standard object metadata + +- **spec** (}}">ResourceSliceSpec), required + + Contains the information published by the driver. + + Changing the spec automatically increments the metadata.generation number. + + + + + +## ResourceSliceSpec {#ResourceSliceSpec} + +ResourceSliceSpec contains the information published by the driver in one ResourceSlice. + +
      + +- **driver** (string), required + + Driver identifies the DRA driver providing the capacity information. A field selector can be used to list only ResourceSlice objects with a certain driver name. + + Must be a DNS subdomain and should end with a DNS domain owned by the vendor of the driver. This field is immutable. + +- **pool** (ResourcePool), required + + Pool describes the pool that this ResourceSlice belongs to. + + + *ResourcePool describes the pool that ResourceSlices belong to.* + + - **pool.generation** (int64), required + + Generation tracks the change in a pool over time. Whenever a driver changes something about one or more of the resources in a pool, it must change the generation in all ResourceSlices which are part of that pool. Consumers of ResourceSlices should only consider resources from the pool with the highest generation number. The generation may be reset by drivers, which should be fine for consumers, assuming that all ResourceSlices in a pool are updated to match or deleted. + + Combined with ResourceSliceCount, this mechanism enables consumers to detect pools which are comprised of multiple ResourceSlices and are in an incomplete state. + + - **pool.name** (string), required + + Name is used to identify the pool. For node-local devices, this is often the node name, but this is not required. + + It must not be longer than 253 characters and must consist of one or more DNS sub-domains separated by slashes. This field is immutable. + + - **pool.resourceSliceCount** (int64), required + + ResourceSliceCount is the total number of ResourceSlices in the pool at this generation number. Must be greater than zero. + + Consumers can use this to check whether they have seen all ResourceSlices belonging to the same pool. + +- **allNodes** (boolean) + + AllNodes indicates that all nodes have access to the resources in the pool. + + Exactly one of NodeName, NodeSelector and AllNodes must be set. + +- **devices** ([]Device) + + *Atomic: will be replaced during a merge* + + Devices lists some or all of the devices in this pool. + + Must not have more than 128 entries. + + + *Device represents one individual hardware instance that can be selected based on its attributes. Besides the name, exactly one field must be set.* + + - **devices.name** (string), required + + Name is unique identifier among all devices managed by the driver in the pool. It must be a DNS label. + + - **devices.basic** (BasicDevice) + + Basic defines one device instance. + + + *BasicDevice defines one device instance.* + + - **devices.basic.attributes** (map[string]DeviceAttribute) + + Attributes defines the set of attributes for this device. The name of each attribute must be unique in that set. + + The maximum number of attributes and capacities combined is 32. + + + *DeviceAttribute must have exactly one field set.* + + - **devices.basic.attributes.bool** (boolean) + + BoolValue is a true/false value. + + - **devices.basic.attributes.int** (int64) + + IntValue is a number. + + - **devices.basic.attributes.string** (string) + + StringValue is a string. Must not be longer than 64 characters. + + - **devices.basic.attributes.version** (string) + + VersionValue is a semantic version according to semver.org spec 2.0.0. Must not be longer than 64 characters. + + - **devices.basic.capacity** (map[string]}}">Quantity) + + Capacity defines the set of capacities for this device. The name of each capacity must be unique in that set. + + The maximum number of attributes and capacities combined is 32. + +- **nodeName** (string) + + NodeName identifies the node which provides the resources in this pool. A field selector can be used to list only ResourceSlice objects belonging to a certain node. + + This field can be used to limit access from nodes to ResourceSlices with the same node name. It also indicates to autoscalers that adding new nodes of the same type as some old node might also make new resources available. + + Exactly one of NodeName, NodeSelector and AllNodes must be set. This field is immutable. + +- **nodeSelector** (NodeSelector) + + NodeSelector defines which nodes have access to the resources in the pool, when that pool is not limited to a single node. + + Must use exactly one term. + + Exactly one of NodeName, NodeSelector and AllNodes must be set. + + + *A node selector represents the union of the results of one or more label queries over a set of nodes; that is, it represents the OR of the selectors represented by the node selector terms.* + + - **nodeSelector.nodeSelectorTerms** ([]NodeSelectorTerm), required + + *Atomic: will be replaced during a merge* + + Required. A list of node selector terms. The terms are ORed. + + + *A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.* + + - **nodeSelector.nodeSelectorTerms.matchExpressions** ([]}}">NodeSelectorRequirement) + + *Atomic: will be replaced during a merge* + + A list of node selector requirements by node's labels. + + - **nodeSelector.nodeSelectorTerms.matchFields** ([]}}">NodeSelectorRequirement) + + *Atomic: will be replaced during a merge* + + A list of node selector requirements by node's fields. + + + + + +## ResourceSliceList {#ResourceSliceList} + +ResourceSliceList is a collection of ResourceSlices. + +
      + +- **apiVersion**: resource.k8s.io/v1beta1 + + +- **kind**: ResourceSliceList + + +- **items** ([]}}">ResourceSlice), required + + Items is the list of resource ResourceSlices. + +- **metadata** (}}">ListMeta) + + Standard list metadata + + + + + +## Operations {#Operations} + + + +
      + + + + + + +### `get` read the specified ResourceSlice + +#### HTTP Request + +GET /apis/resource.k8s.io/v1beta1/resourceslices/{name} + +#### Parameters + + +- **name** (*in path*): string, required + + name of the ResourceSlice + + +- **pretty** (*in query*): string + + }}">pretty + + + +#### Response + + +200 (}}">ResourceSlice): OK + +401: Unauthorized + + +### `list` list or watch objects of kind ResourceSlice + +#### HTTP Request + +GET /apis/resource.k8s.io/v1beta1/resourceslices + +#### Parameters + + +- **allowWatchBookmarks** (*in query*): boolean + + }}">allowWatchBookmarks + + +- **continue** (*in query*): string + + }}">continue + + +- **fieldSelector** (*in query*): string + + }}">fieldSelector + + +- **labelSelector** (*in query*): string + + }}">labelSelector + + +- **limit** (*in query*): integer + + }}">limit + + +- **pretty** (*in query*): string + + }}">pretty + + +- **resourceVersion** (*in query*): string + + }}">resourceVersion + + +- **resourceVersionMatch** (*in query*): string + + }}">resourceVersionMatch + + +- **sendInitialEvents** (*in query*): boolean + + }}">sendInitialEvents + + +- **timeoutSeconds** (*in query*): integer + + }}">timeoutSeconds + + +- **watch** (*in query*): boolean + + }}">watch + + + +#### Response + + +200 (}}">ResourceSliceList): OK + +401: Unauthorized + + +### `create` create a ResourceSlice + +#### HTTP Request + +POST /apis/resource.k8s.io/v1beta1/resourceslices + +#### Parameters + + +- **body**: }}">ResourceSlice, required + + + + +- **dryRun** (*in query*): string + + }}">dryRun + + +- **fieldManager** (*in query*): string + + }}">fieldManager + + +- **fieldValidation** (*in query*): string + + }}">fieldValidation + + +- **pretty** (*in query*): string + + }}">pretty + + + +#### Response + + +200 (}}">ResourceSlice): OK + +201 (}}">ResourceSlice): Created + +202 (}}">ResourceSlice): Accepted + +401: Unauthorized + + +### `update` replace the specified ResourceSlice + +#### HTTP Request + +PUT /apis/resource.k8s.io/v1beta1/resourceslices/{name} + +#### Parameters + + +- **name** (*in path*): string, required + + name of the ResourceSlice + + +- **body**: }}">ResourceSlice, required + + + + +- **dryRun** (*in query*): string + + }}">dryRun + + +- **fieldManager** (*in query*): string + + }}">fieldManager + + +- **fieldValidation** (*in query*): string + + }}">fieldValidation + + +- **pretty** (*in query*): string + + }}">pretty + + + +#### Response + + +200 (}}">ResourceSlice): OK + +201 (}}">ResourceSlice): Created + +401: Unauthorized + + +### `patch` partially update the specified ResourceSlice + +#### HTTP Request + +PATCH /apis/resource.k8s.io/v1beta1/resourceslices/{name} + +#### Parameters + + +- **name** (*in path*): string, required + + name of the ResourceSlice + + +- **body**: }}">Patch, required + + + + +- **dryRun** (*in query*): string + + }}">dryRun + + +- **fieldManager** (*in query*): string + + }}">fieldManager + + +- **fieldValidation** (*in query*): string + + }}">fieldValidation + + +- **force** (*in query*): boolean + + }}">force + + +- **pretty** (*in query*): string + + }}">pretty + + + +#### Response + + +200 (}}">ResourceSlice): OK + +201 (}}">ResourceSlice): Created + +401: Unauthorized + + +### `delete` delete a ResourceSlice + +#### HTTP Request + +DELETE /apis/resource.k8s.io/v1beta1/resourceslices/{name} + +#### Parameters + + +- **name** (*in path*): string, required + + name of the ResourceSlice + + +- **body**: }}">DeleteOptions + + + + +- **dryRun** (*in query*): string + + }}">dryRun + + +- **gracePeriodSeconds** (*in query*): integer + + }}">gracePeriodSeconds + + +- **pretty** (*in query*): string + + }}">pretty + + +- **propagationPolicy** (*in query*): string + + }}">propagationPolicy + + + +#### Response + + +200 (}}">ResourceSlice): OK + +202 (}}">ResourceSlice): Accepted + +401: Unauthorized + + +### `deletecollection` delete collection of ResourceSlice + +#### HTTP Request + +DELETE /apis/resource.k8s.io/v1beta1/resourceslices + +#### Parameters + + +- **body**: }}">DeleteOptions + + + + +- **continue** (*in query*): string + + }}">continue + + +- **dryRun** (*in query*): string + + }}">dryRun + + +- **fieldSelector** (*in query*): string + + }}">fieldSelector + + +- **gracePeriodSeconds** (*in query*): integer + + }}">gracePeriodSeconds + + +- **labelSelector** (*in query*): string + + }}">labelSelector + + +- **limit** (*in query*): integer + + }}">limit + + +- **pretty** (*in query*): string + + }}">pretty + + +- **propagationPolicy** (*in query*): string + + }}">propagationPolicy + + +- **resourceVersion** (*in query*): string + + }}">resourceVersion + + +- **resourceVersionMatch** (*in query*): string + + }}">resourceVersionMatch + + +- **sendInitialEvents** (*in query*): boolean + + }}">sendInitialEvents + + +- **timeoutSeconds** (*in query*): integer + + }}">timeoutSeconds + + + +#### Response + + +200 (}}">Status): OK + +401: Unauthorized + diff --git a/content/en/docs/reference/labels-annotations-taints/_index.md b/content/en/docs/reference/labels-annotations-taints/_index.md index 2ad10c1160a1d..af0fef648a4a6 100644 --- a/content/en/docs/reference/labels-annotations-taints/_index.md +++ b/content/en/docs/reference/labels-annotations-taints/_index.md @@ -804,7 +804,7 @@ Used on: All Objects This annotation is used for describing specific behaviour of given object. -### kubernetes.io/enforce-mountable-secrets {#enforce-mountable-secrets} +### kubernetes.io/enforce-mountable-secrets (deprecated) {#enforce-mountable-secrets} Type: Annotation @@ -812,6 +812,10 @@ Example: `kubernetes.io/enforce-mountable-secrets: "true"` Used on: ServiceAccount +{{< note >}} +`kubernetes.io/enforce-mountable-secrets` is deprecated since Kubernetes v1.32. Use separate namespaces to isolate access to mounted secrets. +{{< /note >}} + The value for this annotation must be **true** to take effect. When you set this annotation to "true", Kubernetes enforces the following rules for Pods running as this ServiceAccount: @@ -1423,6 +1427,19 @@ Used on: Ingress Starting in v1.18, this annotation is deprecated in favor of `spec.ingressClassName`. {{< /note >}} +### kubernetes.io/cluster-service (deprecated) {#kubernetes-io-cluster-service} + +Type: Label + +Example: `kubernetes.io/cluster-service: "true"` + +Used on: Service + +This label indicates that the Service provides a service to the cluster, if the value is set to true. +When you run `kubectl cluster-info`, the tool queries for Services with this label set to true. + +However, setting this label on any Service is deprecated. + ### storageclass.kubernetes.io/is-default-class Type: Annotation @@ -1778,10 +1795,8 @@ Example: `node.kubernetes.io/out-of-service:NoExecute` Used on: Node A user can manually add the taint to a Node marking it out-of-service. -If the `NodeOutOfServiceVolumeDetach` -[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) -is enabled on `kube-controller-manager`, and a Node is marked out-of-service with this taint, -the Pods on the node will be forcefully deleted if there are no matching tolerations on it and +If a Node is marked out-of-service with this taint, the Pods on the node +will be forcefully deleted if there are no matching tolerations on it and volume detach operations for the Pods terminating on the node will happen immediately. This allows the Pods on the out-of-service node to recover quickly on a different node. diff --git a/content/en/docs/reference/node/kernel-version-requirements.md b/content/en/docs/reference/node/kernel-version-requirements.md index 84765fc478a49..d972cfd82e2db 100644 --- a/content/en/docs/reference/node/kernel-version-requirements.md +++ b/content/en/docs/reference/node/kernel-version-requirements.md @@ -32,6 +32,8 @@ Code: https://github.com/kubernetes/kubernetes/blob/00236ae0d73d2455a2470469ed10 - `net.ipv4.tcp_keepalive_intvl` (since Kubernetes 1.29, needs kernel 4.5+); - `net.ipv4.tcp_keepalive_probes` (since Kubernetes 1.29, needs kernel 4.5+); - `net.ipv4.tcp_syncookies` (namespaced since kernel 4.6+). +- `net.ipv4.tcp_rmem` (since Kubernetes 1.32, needs kernel 4.15+). +- `net.ipv4.tcp_wmem` (since Kubernetes 1.32, needs kernel 4.15+). - `net.ipv4.vs.conn_reuse_mode` (used in `ipvs` proxy mode, needs kernel 4.1+); ### kube proxy `nftables` proxy mode diff --git a/content/en/docs/reference/node/kubelet-files.md b/content/en/docs/reference/node/kubelet-files.md index d50b395294782..321ffb65483c8 100644 --- a/content/en/docs/reference/node/kubelet-files.md +++ b/content/en/docs/reference/node/kubelet-files.md @@ -94,7 +94,7 @@ The name of a checkpoint file is `kubelet_internal_checkpoint` for [Device Manag If your cluster has [in-place Pod vertical scaling](/docs/concepts/workloads/autoscaling/#in-place-resizing) enabled ([feature gate](/docs/reference/command-line-tools-reference/feature-gates/) -name `InPlacePodVerticalScaling`), then the kubelet stores a local record of Pod status. +name `InPlacePodVerticalScaling`), then the kubelet stores a local record of allocated Pod resources. The file name is `pod_status_manager_state` within the kubelet base directory (`/var/lib/kubelet` by default on Linux; configurable using `--root-dir`). diff --git a/content/en/docs/reference/node/node-status.md b/content/en/docs/reference/node/node-status.md index 772bd2a57524f..4c4f729c33fc2 100644 --- a/content/en/docs/reference/node/node-status.md +++ b/content/en/docs/reference/node/node-status.md @@ -43,7 +43,7 @@ The `conditions` field describes the status of all `Running` nodes. Examples of {{< table caption = "Node conditions, and a description of when each condition applies." >}} | Node Condition | Description | |----------------------|-------------| -| `Ready` | `True` if the node is healthy and ready to accept pods, `False` if the node is not healthy and is not accepting pods, and `Unknown` if the node controller has not heard from the node in the last `node-monitor-grace-period` (default is 40 seconds) | +| `Ready` | `True` if the node is healthy and ready to accept pods, `False` if the node is not healthy and is not accepting pods, and `Unknown` if the node controller has not heard from the node in the last `node-monitor-grace-period` (default is 50 seconds) | | `DiskPressure` | `True` if pressure exists on the disk size—that is, if the disk capacity is low; otherwise `False` | | `MemoryPressure` | `True` if pressure exists on the node memory—that is, if the node memory is low; otherwise `False` | | `PIDPressure` | `True` if pressure exists on the processes—that is, if there are too many processes on the node; otherwise `False` | @@ -76,7 +76,7 @@ When problems occur on nodes, the Kubernetes control plane automatically creates [taints](/docs/concepts/scheduling-eviction/taint-and-toleration/) that match the conditions affecting the node. An example of this is when the `status` of the Ready condition remains `Unknown` or `False` for longer than the kube-controller-manager's `NodeMonitorGracePeriod`, -which defaults to 40 seconds. This will cause either an `node.kubernetes.io/unreachable` taint, for an `Unknown` status, +which defaults to 50 seconds. This will cause either an `node.kubernetes.io/unreachable` taint, for an `Unknown` status, or a `node.kubernetes.io/not-ready` taint, for a `False` status, to be added to the Node. These taints affect pending pods as the scheduler takes the Node's taints into consideration when diff --git a/content/en/docs/reference/node/systemd-watchdog.md b/content/en/docs/reference/node/systemd-watchdog.md new file mode 100644 index 0000000000000..ab74988b3f90b --- /dev/null +++ b/content/en/docs/reference/node/systemd-watchdog.md @@ -0,0 +1,70 @@ +--- +content_type: "reference" +title: Kubelet Systemd Watchdog +weight: 80 +--- + +{{< feature-state feature_gate_name="SystemdWatchdog" >}} + +On Linux nodes, Kubernetes {{< skew currentVersion >}} supports integrating with +[systemd](https://systemd.io/) to allow the operating system supervisor to recover +a failed kubelet. This integration is not enabled by default. +It can be used as an alternative to periodically requesting +the kubelet's `/healthz` endpoint for health checks. If the kubelet +does not respond to the watchdog within the timeout period, the watchdog +will kill the kubelet. + +The systemd watchdog works by requiring the service to periodically send +a _keep-alive_ signal to the systemd process. If the signal is not received +within a specified timeout period, the service is considered unresponsive +and is terminated. The service can then be restarted according to the configuration. + +## Configuration + +Using the systemd watchdog requires configuring the `WatchdogSec` parameter +in the `[Service]` section of the kubelet service unit file: +``` +[Service] +WatchdogSec=30s +``` + +Setting `WatchdogSec=30s` indicates a service watchdog timeout of 30 seconds. +Within the kubelet, the `sd_notify()` function is invoked, at intervals of `WatchdogSec` ÷ 2, to send +`WATCHDOG=1` (a keep-alive message). If the watchdog is not fed +within the timeout period, the kubelet will be killed. Setting `Restart` +to "always", "on-failure", "on-watchdog", or "on-abnormal" will ensure that the service +is automatically restarted. + +Some details about the systemd configuration: + +1. If you set the systemd value for `WatchdogSec` to 0, or omit setting it, the systemd watchdog is not + enabled for this unit. +2. The kubelet supports a minimum watchdog period of 1.0 seconds; this is to prevent the kubelet + from being killed unexpectedly. You can set the value of `WatchdogSec` in a systemd unit definition + to a period shorter than 1 second, but Kubernetes does not support any shorter interval. + The timeout does not have to be a whole integer number of seconds. +3. The Kubernetes project suggests setting `WatchdogSec` to approximately a 15s period. + Periods longer than 10 minutes are supported but explicitly **not** recommended. + +### Example Configuration +```systemd +[Unit] +Description=kubelet: The Kubernetes Node Agent +Documentation=https://kubernetes.io/docs/home/ +Wants=network-online.target +After=network-online.target + +[Service] +ExecStart=/usr/bin/kubelet +# Configures the watchdog timeout +WatchdogSec=30s +Restart=on-failure +StartLimitInterval=0 +RestartSec=10 + +[Install] +WantedBy=multi-user.target +``` +## {{% heading "whatsnext" %}} +For more details about systemd configuration, refer to the +[systemd documentation](https://www.freedesktop.org/software/systemd/man/latest/systemd.service.html#WatchdogSec=) diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_certs/_index.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_certs/_index.md index d73a49816cfa5..91a6d4d46836d 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_certs/_index.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_certs/_index.md @@ -10,12 +10,12 @@ guide. You can file document formatting bugs against the --> -Commands related to handling kubernetes certificates +Commands related to handling Kubernetes certificates ### Synopsis -Commands related to handling kubernetes certificates +Commands related to handling Kubernetes certificates ``` kubeadm certs [flags] diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_certs/kubeadm_certs_renew.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_certs/kubeadm_certs_renew.md index 7b4a30d807dfb..9cfb0598d427e 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_certs/kubeadm_certs_renew.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_certs/kubeadm_certs_renew.md @@ -15,7 +15,7 @@ Renew certificates for a Kubernetes cluster ### Synopsis -This command is not meant to be run on its own. See list of available subcommands. +Renew certificates for a Kubernetes cluster ``` kubeadm certs renew [flags] diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_config/_index.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_config/_index.md index c2006abb08ea7..6f1af509404bc 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_config/_index.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_config/_index.md @@ -18,7 +18,7 @@ Manage configuration for a kubeadm cluster persisted in a ConfigMap in the clust There is a ConfigMap in the kube-system namespace called "kubeadm-config" that kubeadm uses to store internal configuration about the cluster. kubeadm CLI v1.8.0+ automatically creates this ConfigMap with the config used with 'kubeadm init', but if you -initialized your cluster using kubeadm v1.7.x or lower, you must use the 'kubeadm init phase upload-config' command to +initialized your cluster using kubeadm v1.7.x or lower, you must use the 'kubeadm init phase upload-config' command to create this ConfigMap. This is required so that 'kubeadm upgrade' can configure your upgraded cluster correctly. diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_config/kubeadm_config_images_list.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_config/kubeadm_config_images_list.md index 8b8cb3abfbe4c..dae173036efff 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_config/kubeadm_config_images_list.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_config/kubeadm_config_images_list.md @@ -48,7 +48,7 @@ kubeadm config images list [flags] --feature-gates string -

      A set of key=value pairs that describe feature gates for various features. Options are:
      ControlPlaneKubeletLocalMode=true|false (ALPHA - default=false)
      EtcdLearnerMode=true|false (BETA - default=true)
      PublicKeysECDSA=true|false (DEPRECATED - default=false)
      RootlessControlPlane=true|false (ALPHA - default=false)
      WaitForAllControlPlaneComponents=true|false (ALPHA - default=false)

      +

      A set of key=value pairs that describe feature gates for various features. Options are:
      ControlPlaneKubeletLocalMode=true|false (ALPHA - default=false)
      EtcdLearnerMode=true|false (default=true)
      NodeLocalCRISocket=true|false (ALPHA - default=false)
      PublicKeysECDSA=true|false (DEPRECATED - default=false)
      RootlessControlPlane=true|false (ALPHA - default=false)
      WaitForAllControlPlaneComponents=true|false (ALPHA - default=false)

      diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_config/kubeadm_config_images_pull.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_config/kubeadm_config_images_pull.md index fec80d3e65313..4d2e55311fa0f 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_config/kubeadm_config_images_pull.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_config/kubeadm_config_images_pull.md @@ -48,7 +48,7 @@ kubeadm config images pull [flags] --feature-gates string -

      A set of key=value pairs that describe feature gates for various features. Options are:
      ControlPlaneKubeletLocalMode=true|false (ALPHA - default=false)
      EtcdLearnerMode=true|false (BETA - default=true)
      PublicKeysECDSA=true|false (DEPRECATED - default=false)
      RootlessControlPlane=true|false (ALPHA - default=false)
      WaitForAllControlPlaneComponents=true|false (ALPHA - default=false)

      +

      A set of key=value pairs that describe feature gates for various features. Options are:
      ControlPlaneKubeletLocalMode=true|false (ALPHA - default=false)
      EtcdLearnerMode=true|false (default=true)
      NodeLocalCRISocket=true|false (ALPHA - default=false)
      PublicKeysECDSA=true|false (DEPRECATED - default=false)
      RootlessControlPlane=true|false (ALPHA - default=false)
      WaitForAllControlPlaneComponents=true|false (ALPHA - default=false)

      diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/_index.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/_index.md index aa6dee8fd27d6..c505bb6e9f33d 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/_index.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/_index.md @@ -53,7 +53,6 @@ mark-control-plane Mark a node as a control-plane bootstrap-token Generates bootstrap tokens used to join a node to a cluster kubelet-finalize Updates settings relevant to the kubelet after TLS bootstrap /enable-client-cert-rotation Enable kubelet client certificate rotation - /experimental-cert-rotation Enable kubelet client certificate rotation (DEPRECATED: use 'enable-client-cert-rotation' instead) addon Install required addons for passing conformance tests /coredns Install the CoreDNS addon to a Kubernetes cluster /kube-proxy Install the kube-proxy addon to a Kubernetes cluster @@ -141,7 +140,7 @@ kubeadm init [flags] --feature-gates string -

      A set of key=value pairs that describe feature gates for various features. Options are:
      ControlPlaneKubeletLocalMode=true|false (ALPHA - default=false)
      EtcdLearnerMode=true|false (BETA - default=true)
      PublicKeysECDSA=true|false (DEPRECATED - default=false)
      RootlessControlPlane=true|false (ALPHA - default=false)
      WaitForAllControlPlaneComponents=true|false (ALPHA - default=false)

      +

      A set of key=value pairs that describe feature gates for various features. Options are:
      ControlPlaneKubeletLocalMode=true|false (ALPHA - default=false)
      EtcdLearnerMode=true|false (default=true)
      NodeLocalCRISocket=true|false (ALPHA - default=false)
      PublicKeysECDSA=true|false (DEPRECATED - default=false)
      RootlessControlPlane=true|false (ALPHA - default=false)
      WaitForAllControlPlaneComponents=true|false (ALPHA - default=false)

      diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase.md index e5c16f10cea0a..4cdcb20b0dbab 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase.md @@ -10,12 +10,16 @@ guide. You can file document formatting bugs against the --> -Use this command to invoke single phase of the init workflow +Use this command to invoke single phase of the "init" workflow ### Synopsis -Use this command to invoke single phase of the init workflow +Use this command to invoke single phase of the "init" workflow + +``` +kubeadm init phase [flags] +``` ### Options diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_addon.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_addon.md index e4af694fa579a..fbd41859b2cfc 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_addon.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_addon.md @@ -15,7 +15,7 @@ Install required addons for passing conformance tests ### Synopsis -This command is not meant to be run on its own. See list of available subcommands. +Install required addons for passing conformance tests ``` kubeadm init phase addon [flags] diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_addon_all.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_addon_all.md index 1cba797ebdbe5..f156bdfe9ad29 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_addon_all.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_addon_all.md @@ -69,7 +69,7 @@ kubeadm init phase addon all [flags] --feature-gates string -

      A set of key=value pairs that describe feature gates for various features. Options are:
      ControlPlaneKubeletLocalMode=true|false (ALPHA - default=false)
      EtcdLearnerMode=true|false (BETA - default=true)
      PublicKeysECDSA=true|false (DEPRECATED - default=false)
      RootlessControlPlane=true|false (ALPHA - default=false)
      WaitForAllControlPlaneComponents=true|false (ALPHA - default=false)

      +

      A set of key=value pairs that describe feature gates for various features. Options are:
      ControlPlaneKubeletLocalMode=true|false (ALPHA - default=false)
      EtcdLearnerMode=true|false (default=true)
      NodeLocalCRISocket=true|false (ALPHA - default=false)
      PublicKeysECDSA=true|false (DEPRECATED - default=false)
      RootlessControlPlane=true|false (ALPHA - default=false)
      WaitForAllControlPlaneComponents=true|false (ALPHA - default=false)

      diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_addon_coredns.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_addon_coredns.md index ae940f5fa506a..c3a9f7b4f3880 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_addon_coredns.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_addon_coredns.md @@ -48,7 +48,7 @@ kubeadm init phase addon coredns [flags] --feature-gates string -

      A set of key=value pairs that describe feature gates for various features. Options are:
      ControlPlaneKubeletLocalMode=true|false (ALPHA - default=false)
      EtcdLearnerMode=true|false (BETA - default=true)
      PublicKeysECDSA=true|false (DEPRECATED - default=false)
      RootlessControlPlane=true|false (ALPHA - default=false)
      WaitForAllControlPlaneComponents=true|false (ALPHA - default=false)

      +

      A set of key=value pairs that describe feature gates for various features. Options are:
      ControlPlaneKubeletLocalMode=true|false (ALPHA - default=false)
      EtcdLearnerMode=true|false (default=true)
      NodeLocalCRISocket=true|false (ALPHA - default=false)
      PublicKeysECDSA=true|false (DEPRECATED - default=false)
      RootlessControlPlane=true|false (ALPHA - default=false)
      WaitForAllControlPlaneComponents=true|false (ALPHA - default=false)

      diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_certs.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_certs.md index 0841881fe62b2..0a1498f364fbd 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_certs.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_certs.md @@ -15,7 +15,7 @@ Certificate generation ### Synopsis -This command is not meant to be run on its own. See list of available subcommands. +Certificate generation ``` kubeadm init phase certs [flags] diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_control-plane.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_control-plane.md index 1682a87a16bc9..49ad05f21a3d4 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_control-plane.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_control-plane.md @@ -15,7 +15,7 @@ Generate all static Pod manifest files necessary to establish the control plane ### Synopsis -This command is not meant to be run on its own. See list of available subcommands. +Generate all static Pod manifest files necessary to establish the control plane ``` kubeadm init phase control-plane [flags] diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_control-plane_all.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_control-plane_all.md index f2e74a805aa6e..a2a55a5285212 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_control-plane_all.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_control-plane_all.md @@ -87,7 +87,7 @@ kubeadm init phase control-plane all [flags] --feature-gates string -

      A set of key=value pairs that describe feature gates for various features. Options are:
      ControlPlaneKubeletLocalMode=true|false (ALPHA - default=false)
      EtcdLearnerMode=true|false (BETA - default=true)
      PublicKeysECDSA=true|false (DEPRECATED - default=false)
      RootlessControlPlane=true|false (ALPHA - default=false)
      WaitForAllControlPlaneComponents=true|false (ALPHA - default=false)

      +

      A set of key=value pairs that describe feature gates for various features. Options are:
      ControlPlaneKubeletLocalMode=true|false (ALPHA - default=false)
      EtcdLearnerMode=true|false (default=true)
      NodeLocalCRISocket=true|false (ALPHA - default=false)
      PublicKeysECDSA=true|false (DEPRECATED - default=false)
      RootlessControlPlane=true|false (ALPHA - default=false)
      WaitForAllControlPlaneComponents=true|false (ALPHA - default=false)

      diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_control-plane_apiserver.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_control-plane_apiserver.md index cb73c5cd5ed4a..c023fe07510e9 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_control-plane_apiserver.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_control-plane_apiserver.md @@ -76,7 +76,7 @@ kubeadm init phase control-plane apiserver [flags] --feature-gates string -

      A set of key=value pairs that describe feature gates for various features. Options are:
      ControlPlaneKubeletLocalMode=true|false (ALPHA - default=false)
      EtcdLearnerMode=true|false (BETA - default=true)
      PublicKeysECDSA=true|false (DEPRECATED - default=false)
      RootlessControlPlane=true|false (ALPHA - default=false)
      WaitForAllControlPlaneComponents=true|false (ALPHA - default=false)

      +

      A set of key=value pairs that describe feature gates for various features. Options are:
      ControlPlaneKubeletLocalMode=true|false (ALPHA - default=false)
      EtcdLearnerMode=true|false (default=true)
      NodeLocalCRISocket=true|false (ALPHA - default=false)
      PublicKeysECDSA=true|false (DEPRECATED - default=false)
      RootlessControlPlane=true|false (ALPHA - default=false)
      WaitForAllControlPlaneComponents=true|false (ALPHA - default=false)

      diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_etcd.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_etcd.md index 2947236f0ccb4..90622d995b648 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_etcd.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_etcd.md @@ -15,7 +15,7 @@ Generate static Pod manifest file for local etcd ### Synopsis -This command is not meant to be run on its own. See list of available subcommands. +Generate static Pod manifest file for local etcd ``` kubeadm init phase etcd [flags] diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_kubeconfig.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_kubeconfig.md index 4b0add3db89a6..b789ccade2525 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_kubeconfig.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_kubeconfig.md @@ -15,7 +15,7 @@ Generate all kubeconfig files necessary to establish the control plane and the a ### Synopsis -This command is not meant to be run on its own. See list of available subcommands. +Generate all kubeconfig files necessary to establish the control plane and the admin kubeconfig file ``` kubeadm init phase kubeconfig [flags] diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_upload-config.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_upload-config.md index 3183ac3cd8ce2..551f7b27b8387 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_upload-config.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_upload-config.md @@ -15,7 +15,7 @@ Upload the kubeadm and kubelet configuration to a ConfigMap ### Synopsis -This command is not meant to be run on its own. See list of available subcommands. +Upload the kubeadm and kubelet configuration to a ConfigMap ``` kubeadm init phase upload-config [flags] diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_upload-config_kubeadm.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_upload-config_kubeadm.md index 2d401db08c3e0..865abc919897b 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_upload-config_kubeadm.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_upload-config_kubeadm.md @@ -27,7 +27,7 @@ kubeadm init phase upload-config kubeadm [flags] ``` # upload the configuration of your cluster - kubeadm init phase upload-config --config=myConfig.yaml + kubeadm init phase upload-config kubeadm --config=myConfig.yaml ``` ### Options diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_join/kubeadm_join_phase.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_join/kubeadm_join_phase.md index 328e8e7cd428d..2835d863b5735 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_join/kubeadm_join_phase.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_join/kubeadm_join_phase.md @@ -10,12 +10,16 @@ guide. You can file document formatting bugs against the --> -Use this command to invoke single phase of the join workflow +Use this command to invoke single phase of the "join" workflow ### Synopsis -Use this command to invoke single phase of the join workflow +Use this command to invoke single phase of the "join" workflow + +``` +kubeadm join phase [flags] +``` ### Options diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_reset/kubeadm_reset_phase.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_reset/kubeadm_reset_phase.md index ee2ad4fb3d00d..0c1473b64a95b 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_reset/kubeadm_reset_phase.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_reset/kubeadm_reset_phase.md @@ -10,12 +10,16 @@ guide. You can file document formatting bugs against the --> -Use this command to invoke single phase of the reset workflow +Use this command to invoke single phase of the "reset" workflow ### Synopsis -Use this command to invoke single phase of the reset workflow +Use this command to invoke single phase of the "reset" workflow + +``` +kubeadm reset phase [flags] +``` ### Options diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply.md index 2c0e24732f1b9..a7e424670b43e 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply.md @@ -17,6 +17,22 @@ Upgrade your Kubernetes cluster to the specified version Upgrade your Kubernetes cluster to the specified version +The "apply [version]" command executes the following phases: +``` +preflight Run preflight checks before upgrade +control-plane Upgrade the control plane +upload-config Upload the kubeadm and kubelet configurations to ConfigMaps + /kubeadm Upload the kubeadm ClusterConfiguration to a ConfigMap + /kubelet Upload the kubelet configuration to a ConfigMap +kubelet-config Upgrade the kubelet configuration for this node +bootstrap-token Configures bootstrap token and cluster-info RBAC rules +addon Upgrade the default kubeadm addons + /coredns Upgrade the CoreDNS addon + /kube-proxy Upgrade the kube-proxy addon +post-upgrade Run post upgrade tasks +``` + + ``` kubeadm upgrade apply [version] ``` @@ -114,6 +130,13 @@ kubeadm upgrade apply [version]

      Specifies whether the configuration file that will be used in the upgrade should be printed or not.

      + +--skip-phases strings + + +

      List of phases to be skipped

      + + -y, --yes diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase.md new file mode 100644 index 0000000000000..74c3b4bcaf74e --- /dev/null +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase.md @@ -0,0 +1,65 @@ + + + +Use this command to invoke single phase of the "apply" workflow + +### Synopsis + + +Use this command to invoke single phase of the "apply" workflow + +``` +kubeadm upgrade apply phase [flags] +``` + +### Options + + ++++ + + + + + + + + + + +
      -h, --help

      help for phase

      + + + +### Options inherited from parent commands + + ++++ + + + + + + + + + + +
      --rootfs string

      The path to the 'real' host root filesystem. This will cause kubeadm to chroot into the provided path.

      + + + diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_addon.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_addon.md new file mode 100644 index 0000000000000..916265a9f1058 --- /dev/null +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_addon.md @@ -0,0 +1,65 @@ + + + +Upgrade the default kubeadm addons + +### Synopsis + + +Upgrade the default kubeadm addons + +``` +kubeadm upgrade apply phase addon [flags] +``` + +### Options + + ++++ + + + + + + + + + + +
      -h, --help

      help for addon

      + + + +### Options inherited from parent commands + + ++++ + + + + + + + + + + +
      --rootfs string

      The path to the 'real' host root filesystem. This will cause kubeadm to chroot into the provided path.

      + + + diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_addon_all.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_addon_all.md new file mode 100644 index 0000000000000..ff8fcb2bc2f2b --- /dev/null +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_addon_all.md @@ -0,0 +1,93 @@ + + + +Upgrade all the addons + +### Synopsis + + +Upgrade all the addons + +``` +kubeadm upgrade apply phase addon all [flags] +``` + +### Options + + ++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
      --config string

      Path to a kubeadm configuration file.

      --dry-run

      Do not change any state, just output what actions would be performed.

      -h, --help

      help for all

      --kubeconfig string     Default: "/etc/kubernetes/admin.conf"

      The kubeconfig file to use when talking to the cluster. If the flag is not set, a set of standard locations can be searched for an existing kubeconfig file.

      --patches string

      Path to a directory that contains files named "target[suffix][+patchtype].extension". For example, "kube-apiserver0+merge.yaml" or just "etcd.json". "target" can be one of "kube-apiserver", "kube-controller-manager", "kube-scheduler", "etcd", "kubeletconfiguration", "corednsdeployment". "patchtype" can be one of "strategic", "merge" or "json" and they match the patch formats supported by kubectl. The default "patchtype" is "strategic". "extension" must be either "json" or "yaml". "suffix" is an optional string that can be used to determine which patches are applied first alpha-numerically.

      + + + +### Options inherited from parent commands + + ++++ + + + + + + + + + + +
      --rootfs string

      The path to the 'real' host root filesystem. This will cause kubeadm to chroot into the provided path.

      + + + diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_addon_coredns.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_addon_coredns.md new file mode 100644 index 0000000000000..7129fe1180238 --- /dev/null +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_addon_coredns.md @@ -0,0 +1,93 @@ + + + +Upgrade the CoreDNS addon + +### Synopsis + + +Upgrade the CoreDNS addon + +``` +kubeadm upgrade apply phase addon coredns [flags] +``` + +### Options + + ++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
      --config string

      Path to a kubeadm configuration file.

      --dry-run

      Do not change any state, just output what actions would be performed.

      -h, --help

      help for coredns

      --kubeconfig string     Default: "/etc/kubernetes/admin.conf"

      The kubeconfig file to use when talking to the cluster. If the flag is not set, a set of standard locations can be searched for an existing kubeconfig file.

      --patches string

      Path to a directory that contains files named "target[suffix][+patchtype].extension". For example, "kube-apiserver0+merge.yaml" or just "etcd.json". "target" can be one of "kube-apiserver", "kube-controller-manager", "kube-scheduler", "etcd", "kubeletconfiguration", "corednsdeployment". "patchtype" can be one of "strategic", "merge" or "json" and they match the patch formats supported by kubectl. The default "patchtype" is "strategic". "extension" must be either "json" or "yaml". "suffix" is an optional string that can be used to determine which patches are applied first alpha-numerically.

      + + + +### Options inherited from parent commands + + ++++ + + + + + + + + + + +
      --rootfs string

      The path to the 'real' host root filesystem. This will cause kubeadm to chroot into the provided path.

      + + + diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_kubelet-finalize_experimental-cert-rotation.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_addon_kube-proxy.md similarity index 75% rename from content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_kubelet-finalize_experimental-cert-rotation.md rename to content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_addon_kube-proxy.md index 959c1aaf84460..571aee7359e5e 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/kubeadm_init_phase_kubelet-finalize_experimental-cert-rotation.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_addon_kube-proxy.md @@ -10,15 +10,15 @@ guide. You can file document formatting bugs against the --> -Enable kubelet client certificate rotation (DEPRECATED: use 'enable-client-cert-rotation' instead) +Upgrade the kube-proxy addon ### Synopsis -Enable kubelet client certificate rotation (DEPRECATED: use 'enable-client-cert-rotation' instead) +Upgrade the kube-proxy addon ``` -kubeadm init phase kubelet-finalize experimental-cert-rotation [flags] +kubeadm upgrade apply phase addon kube-proxy [flags] ``` ### Options @@ -31,31 +31,31 @@ kubeadm init phase kubelet-finalize experimental-cert-rotation [flags] ---cert-dir string     Default: "/etc/kubernetes/pki" +--config string -

      The path where to save and store the certificates.

      +

      Path to a kubeadm configuration file.

      ---config string +--dry-run -

      Path to a kubeadm configuration file.

      +

      Do not change any state, just output what actions would be performed.

      ---dry-run +-h, --help -

      Don't apply any changes; just output what would be done.

      +

      help for kube-proxy

      --h, --help +--kubeconfig string     Default: "/etc/kubernetes/admin.conf" -

      help for experimental-cert-rotation

      +

      The kubeconfig file to use when talking to the cluster. If the flag is not set, a set of standard locations can be searched for an existing kubeconfig file.

      diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_bootstrap-token.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_bootstrap-token.md new file mode 100644 index 0000000000000..2891a9b8cbc74 --- /dev/null +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_bootstrap-token.md @@ -0,0 +1,86 @@ + + + +Configures bootstrap token and cluster-info RBAC rules + +### Synopsis + + +Configures bootstrap token and cluster-info RBAC rules + +``` +kubeadm upgrade apply phase bootstrap-token [flags] +``` + +### Options + + ++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
      --config string

      Path to a kubeadm configuration file.

      --dry-run

      Do not change any state, just output what actions would be performed.

      -h, --help

      help for bootstrap-token

      --kubeconfig string     Default: "/etc/kubernetes/admin.conf"

      The kubeconfig file to use when talking to the cluster. If the flag is not set, a set of standard locations can be searched for an existing kubeconfig file.

      + + + +### Options inherited from parent commands + + ++++ + + + + + + + + + + +
      --rootfs string

      The path to the 'real' host root filesystem. This will cause kubeadm to chroot into the provided path.

      + + + diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_control-plane.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_control-plane.md new file mode 100644 index 0000000000000..ee3c9ab03b465 --- /dev/null +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_control-plane.md @@ -0,0 +1,107 @@ + + + +Upgrade the control plane + +### Synopsis + + +Upgrade the control plane + +``` +kubeadm upgrade apply phase control-plane [flags] +``` + +### Options + + ++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
      --certificate-renewal     Default: true

      Perform the renewal of certificates used by component changed during upgrades.

      --config string

      Path to a kubeadm configuration file.

      --dry-run

      Do not change any state, just output what actions would be performed.

      --etcd-upgrade     Default: true

      Perform the upgrade of etcd.

      -h, --help

      help for control-plane

      --kubeconfig string     Default: "/etc/kubernetes/admin.conf"

      The kubeconfig file to use when talking to the cluster. If the flag is not set, a set of standard locations can be searched for an existing kubeconfig file.

      --patches string

      Path to a directory that contains files named "target[suffix][+patchtype].extension". For example, "kube-apiserver0+merge.yaml" or just "etcd.json". "target" can be one of "kube-apiserver", "kube-controller-manager", "kube-scheduler", "etcd", "kubeletconfiguration", "corednsdeployment". "patchtype" can be one of "strategic", "merge" or "json" and they match the patch formats supported by kubectl. The default "patchtype" is "strategic". "extension" must be either "json" or "yaml". "suffix" is an optional string that can be used to determine which patches are applied first alpha-numerically.

      + + + +### Options inherited from parent commands + + ++++ + + + + + + + + + + +
      --rootfs string

      The path to the 'real' host root filesystem. This will cause kubeadm to chroot into the provided path.

      + + + diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_kubelet-config.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_kubelet-config.md new file mode 100644 index 0000000000000..c9588bb5cc7ac --- /dev/null +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_kubelet-config.md @@ -0,0 +1,93 @@ + + + +Upgrade the kubelet configuration for this node + +### Synopsis + + +Upgrade the kubelet configuration for this node by downloading it from the kubelet-config ConfigMap stored in the cluster + +``` +kubeadm upgrade apply phase kubelet-config [flags] +``` + +### Options + + ++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
      --config string

      Path to a kubeadm configuration file.

      --dry-run

      Do not change any state, just output what actions would be performed.

      -h, --help

      help for kubelet-config

      --kubeconfig string     Default: "/etc/kubernetes/admin.conf"

      The kubeconfig file to use when talking to the cluster. If the flag is not set, a set of standard locations can be searched for an existing kubeconfig file.

      --patches string

      Path to a directory that contains files named "target[suffix][+patchtype].extension". For example, "kube-apiserver0+merge.yaml" or just "etcd.json". "target" can be one of "kube-apiserver", "kube-controller-manager", "kube-scheduler", "etcd", "kubeletconfiguration", "corednsdeployment". "patchtype" can be one of "strategic", "merge" or "json" and they match the patch formats supported by kubectl. The default "patchtype" is "strategic". "extension" must be either "json" or "yaml". "suffix" is an optional string that can be used to determine which patches are applied first alpha-numerically.

      + + + +### Options inherited from parent commands + + ++++ + + + + + + + + + + +
      --rootfs string

      The path to the 'real' host root filesystem. This will cause kubeadm to chroot into the provided path.

      + + + diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_post-upgrade.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_post-upgrade.md new file mode 100644 index 0000000000000..10e52ac6f8738 --- /dev/null +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_post-upgrade.md @@ -0,0 +1,86 @@ + + + +Run post upgrade tasks + +### Synopsis + + +Run post upgrade tasks + +``` +kubeadm upgrade apply phase post-upgrade [flags] +``` + +### Options + + ++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
      --config string

      Path to a kubeadm configuration file.

      --dry-run

      Do not change any state, just output what actions would be performed.

      -h, --help

      help for post-upgrade

      --kubeconfig string     Default: "/etc/kubernetes/admin.conf"

      The kubeconfig file to use when talking to the cluster. If the flag is not set, a set of standard locations can be searched for an existing kubeconfig file.

      + + + +### Options inherited from parent commands + + ++++ + + + + + + + + + + +
      --rootfs string

      The path to the 'real' host root filesystem. This will cause kubeadm to chroot into the provided path.

      + + + diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_preflight.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_preflight.md new file mode 100644 index 0000000000000..08988b5e590ac --- /dev/null +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_preflight.md @@ -0,0 +1,121 @@ + + + +Run preflight checks before upgrade + +### Synopsis + + +Run preflight checks before upgrade + +``` +kubeadm upgrade apply phase preflight [flags] +``` + +### Options + + ++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
      --allow-experimental-upgrades

      Show unstable versions of Kubernetes as an upgrade alternative and allow upgrading to an alpha/beta/release candidate versions of Kubernetes.

      --allow-release-candidate-upgrades

      Show release candidate versions of Kubernetes as an upgrade alternative and allow upgrading to a release candidate versions of Kubernetes.

      --config string

      Path to a kubeadm configuration file.

      --dry-run

      Do not change any state, just output what actions would be performed.

      -f, --force

      Force upgrading although some requirements might not be met. This also implies non-interactive mode.

      -h, --help

      help for preflight

      --ignore-preflight-errors strings

      A list of checks whose errors will be shown as warnings. Example: 'IsPrivilegedUser,Swap'. Value 'all' ignores errors from all checks.

      --kubeconfig string     Default: "/etc/kubernetes/admin.conf"

      The kubeconfig file to use when talking to the cluster. If the flag is not set, a set of standard locations can be searched for an existing kubeconfig file.

      -y, --yes

      Perform the upgrade and do not prompt for confirmation (non-interactive mode).

      + + + +### Options inherited from parent commands + + ++++ + + + + + + + + + + +
      --rootfs string

      The path to the 'real' host root filesystem. This will cause kubeadm to chroot into the provided path.

      + + + diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_upload-config.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_upload-config.md new file mode 100644 index 0000000000000..c842fd94b8213 --- /dev/null +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_upload-config.md @@ -0,0 +1,65 @@ + + + +Upload the kubeadm and kubelet configurations to ConfigMaps + +### Synopsis + + +Upload the kubeadm and kubelet configurations to ConfigMaps + +``` +kubeadm upgrade apply phase upload-config [flags] +``` + +### Options + + ++++ + + + + + + + + + + +
      -h, --help

      help for upload-config

      + + + +### Options inherited from parent commands + + ++++ + + + + + + + + + + +
      --rootfs string

      The path to the 'real' host root filesystem. This will cause kubeadm to chroot into the provided path.

      + + + diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_upload-config_all.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_upload-config_all.md new file mode 100644 index 0000000000000..392b1d09f79f4 --- /dev/null +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_upload-config_all.md @@ -0,0 +1,86 @@ + + + +Upload all the configurations to ConfigMaps + +### Synopsis + + +Upload all the configurations to ConfigMaps + +``` +kubeadm upgrade apply phase upload-config all [flags] +``` + +### Options + + ++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
      --config string

      Path to a kubeadm configuration file.

      --dry-run

      Do not change any state, just output what actions would be performed.

      -h, --help

      help for all

      --kubeconfig string     Default: "/etc/kubernetes/admin.conf"

      The kubeconfig file to use when talking to the cluster. If the flag is not set, a set of standard locations can be searched for an existing kubeconfig file.

      + + + +### Options inherited from parent commands + + ++++ + + + + + + + + + + +
      --rootfs string

      The path to the 'real' host root filesystem. This will cause kubeadm to chroot into the provided path.

      + + + diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_upload-config_kubeadm.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_upload-config_kubeadm.md new file mode 100644 index 0000000000000..5b7cf3728619a --- /dev/null +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_upload-config_kubeadm.md @@ -0,0 +1,86 @@ + + + +Upload the kubeadm ClusterConfiguration to a ConfigMap + +### Synopsis + + +Upload the kubeadm ClusterConfiguration to a ConfigMap + +``` +kubeadm upgrade apply phase upload-config kubeadm [flags] +``` + +### Options + + ++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
      --config string

      Path to a kubeadm configuration file.

      --dry-run

      Do not change any state, just output what actions would be performed.

      -h, --help

      help for kubeadm

      --kubeconfig string     Default: "/etc/kubernetes/admin.conf"

      The kubeconfig file to use when talking to the cluster. If the flag is not set, a set of standard locations can be searched for an existing kubeconfig file.

      + + + +### Options inherited from parent commands + + ++++ + + + + + + + + + + +
      --rootfs string

      The path to the 'real' host root filesystem. This will cause kubeadm to chroot into the provided path.

      + + + diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_upload-config_kubelet.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_upload-config_kubelet.md new file mode 100644 index 0000000000000..c0531d5ed6f24 --- /dev/null +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_upload-config_kubelet.md @@ -0,0 +1,86 @@ + + + +Upload the kubelet configuration to a ConfigMap + +### Synopsis + + +Upload the kubelet configuration to a ConfigMap + +``` +kubeadm upgrade apply phase upload-config kubelet [flags] +``` + +### Options + + ++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
      --config string

      Path to a kubeadm configuration file.

      --dry-run

      Do not change any state, just output what actions would be performed.

      -h, --help

      help for kubelet

      --kubeconfig string     Default: "/etc/kubernetes/admin.conf"

      The kubeconfig file to use when talking to the cluster. If the flag is not set, a set of standard locations can be searched for an existing kubeconfig file.

      + + + +### Options inherited from parent commands + + ++++ + + + + + + + + + + +
      --rootfs string

      The path to the 'real' host root filesystem. This will cause kubeadm to chroot into the provided path.

      + + + diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node.md index b239b460de8d0..fc47d4b8e8c29 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node.md @@ -22,6 +22,10 @@ The "node" command executes the following phases: preflight Run upgrade node pre-flight checks control-plane Upgrade the control plane instance deployed on this node, if any kubelet-config Upgrade the kubelet configuration for this node +addon Upgrade the default kubeadm addons + /coredns Upgrade the CoreDNS addon + /kube-proxy Upgrade the kube-proxy addon +post-upgrade Run post upgrade tasks ``` diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase.md index b38f1b431db81..469318f1eef1c 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase.md @@ -10,12 +10,16 @@ guide. You can file document formatting bugs against the --> -Use this command to invoke single phase of the node workflow +Use this command to invoke single phase of the "node" workflow ### Synopsis -Use this command to invoke single phase of the node workflow +Use this command to invoke single phase of the "node" workflow + +``` +kubeadm upgrade node phase [flags] +``` ### Options diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_addon.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_addon.md new file mode 100644 index 0000000000000..8fd5d267bb49b --- /dev/null +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_addon.md @@ -0,0 +1,65 @@ + + + +Upgrade the default kubeadm addons + +### Synopsis + + +Upgrade the default kubeadm addons + +``` +kubeadm upgrade node phase addon [flags] +``` + +### Options + + ++++ + + + + + + + + + + +
      -h, --help

      help for addon

      + + + +### Options inherited from parent commands + + ++++ + + + + + + + + + + +
      --rootfs string

      The path to the 'real' host root filesystem. This will cause kubeadm to chroot into the provided path.

      + + + diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_addon_all.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_addon_all.md new file mode 100644 index 0000000000000..466eef2b9f694 --- /dev/null +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_addon_all.md @@ -0,0 +1,93 @@ + + + +Upgrade all the addons + +### Synopsis + + +Upgrade all the addons + +``` +kubeadm upgrade node phase addon all [flags] +``` + +### Options + + ++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
      --config string

      Path to a kubeadm configuration file.

      --dry-run

      Do not change any state, just output the actions that would be performed.

      -h, --help

      help for all

      --kubeconfig string     Default: "/etc/kubernetes/admin.conf"

      The kubeconfig file to use when talking to the cluster. If the flag is not set, a set of standard locations can be searched for an existing kubeconfig file.

      --patches string

      Path to a directory that contains files named "target[suffix][+patchtype].extension". For example, "kube-apiserver0+merge.yaml" or just "etcd.json". "target" can be one of "kube-apiserver", "kube-controller-manager", "kube-scheduler", "etcd", "kubeletconfiguration", "corednsdeployment". "patchtype" can be one of "strategic", "merge" or "json" and they match the patch formats supported by kubectl. The default "patchtype" is "strategic". "extension" must be either "json" or "yaml". "suffix" is an optional string that can be used to determine which patches are applied first alpha-numerically.

      + + + +### Options inherited from parent commands + + ++++ + + + + + + + + + + +
      --rootfs string

      The path to the 'real' host root filesystem. This will cause kubeadm to chroot into the provided path.

      + + + diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_addon_coredns.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_addon_coredns.md new file mode 100644 index 0000000000000..a4bfcfc754b4e --- /dev/null +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_addon_coredns.md @@ -0,0 +1,93 @@ + + + +Upgrade the CoreDNS addon + +### Synopsis + + +Upgrade the CoreDNS addon + +``` +kubeadm upgrade node phase addon coredns [flags] +``` + +### Options + + ++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
      --config string

      Path to a kubeadm configuration file.

      --dry-run

      Do not change any state, just output the actions that would be performed.

      -h, --help

      help for coredns

      --kubeconfig string     Default: "/etc/kubernetes/admin.conf"

      The kubeconfig file to use when talking to the cluster. If the flag is not set, a set of standard locations can be searched for an existing kubeconfig file.

      --patches string

      Path to a directory that contains files named "target[suffix][+patchtype].extension". For example, "kube-apiserver0+merge.yaml" or just "etcd.json". "target" can be one of "kube-apiserver", "kube-controller-manager", "kube-scheduler", "etcd", "kubeletconfiguration", "corednsdeployment". "patchtype" can be one of "strategic", "merge" or "json" and they match the patch formats supported by kubectl. The default "patchtype" is "strategic". "extension" must be either "json" or "yaml". "suffix" is an optional string that can be used to determine which patches are applied first alpha-numerically.

      + + + +### Options inherited from parent commands + + ++++ + + + + + + + + + + +
      --rootfs string

      The path to the 'real' host root filesystem. This will cause kubeadm to chroot into the provided path.

      + + + diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_addon_kube-proxy.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_addon_kube-proxy.md new file mode 100644 index 0000000000000..5c6624504f618 --- /dev/null +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_addon_kube-proxy.md @@ -0,0 +1,86 @@ + + + +Upgrade the kube-proxy addon + +### Synopsis + + +Upgrade the kube-proxy addon + +``` +kubeadm upgrade node phase addon kube-proxy [flags] +``` + +### Options + + ++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
      --config string

      Path to a kubeadm configuration file.

      --dry-run

      Do not change any state, just output the actions that would be performed.

      -h, --help

      help for kube-proxy

      --kubeconfig string     Default: "/etc/kubernetes/admin.conf"

      The kubeconfig file to use when talking to the cluster. If the flag is not set, a set of standard locations can be searched for an existing kubeconfig file.

      + + + +### Options inherited from parent commands + + ++++ + + + + + + + + + + +
      --rootfs string

      The path to the 'real' host root filesystem. This will cause kubeadm to chroot into the provided path.

      + + + diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_control-plane.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_control-plane.md index cc798cada6925..705aca5ef8730 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_control-plane.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_control-plane.md @@ -37,6 +37,13 @@ kubeadm upgrade node phase control-plane [flags]

      Perform the renewal of certificates used by component changed during upgrades.

      + +--config string + + +

      Path to a kubeadm configuration file.

      + + --dry-run diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_kubelet-config.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_kubelet-config.md index 091260a6f122b..6a0ff0b53c42e 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_kubelet-config.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_kubelet-config.md @@ -15,7 +15,7 @@ Upgrade the kubelet configuration for this node ### Synopsis -Download the kubelet configuration from the kubelet-config ConfigMap stored in the cluster +Upgrade the kubelet configuration for this node by downloading it from the kubelet-config ConfigMap stored in the cluster ``` kubeadm upgrade node phase kubelet-config [flags] @@ -30,6 +30,13 @@ kubeadm upgrade node phase kubelet-config [flags] + +--config string + + +

      Path to a kubeadm configuration file.

      + + --dry-run diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_post-upgrade.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_post-upgrade.md new file mode 100644 index 0000000000000..af2b5840e8dee --- /dev/null +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_post-upgrade.md @@ -0,0 +1,86 @@ + + + +Run post upgrade tasks + +### Synopsis + + +Run post upgrade tasks + +``` +kubeadm upgrade node phase post-upgrade [flags] +``` + +### Options + + ++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
      --config string

      Path to a kubeadm configuration file.

      --dry-run

      Do not change any state, just output the actions that would be performed.

      -h, --help

      help for post-upgrade

      --kubeconfig string     Default: "/etc/kubernetes/admin.conf"

      The kubeconfig file to use when talking to the cluster. If the flag is not set, a set of standard locations can be searched for an existing kubeconfig file.

      + + + +### Options inherited from parent commands + + ++++ + + + + + + + + + + +
      --rootfs string

      The path to the 'real' host root filesystem. This will cause kubeadm to chroot into the provided path.

      + + + diff --git a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_preflight.md b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_preflight.md index e61de50418fd6..73671da374fe9 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_preflight.md +++ b/content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_preflight.md @@ -30,6 +30,13 @@ kubeadm upgrade node phase preflight [flags] + +--config string + + +

      Path to a kubeadm configuration file.

      + + -h, --help diff --git a/content/en/docs/reference/setup-tools/kubeadm/implementation-details.md b/content/en/docs/reference/setup-tools/kubeadm/implementation-details.md index 9fa9abd5c3ed4..354f66b4f03ab 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/implementation-details.md +++ b/content/en/docs/reference/setup-tools/kubeadm/implementation-details.md @@ -35,7 +35,7 @@ The cluster that `kubeadm init` and `kubeadm join` set up should be: - **User-friendly**: The user should not have to run anything more than a couple of commands: - `kubeadm init` - `export KUBECONFIG=/etc/kubernetes/admin.conf` - - `kubectl apply -f ` + - `kubectl apply -f ` - `kubeadm join --token :` - **Extendable**: - It should _not_ favor any particular network provider. Configuring the cluster network is out-of-scope @@ -76,6 +76,23 @@ in a majority of cases, and the most intuitive location; other constant paths an - `front-proxy-ca.crt`, `front-proxy-ca.key` for the front proxy certificate authority - `front-proxy-client.crt`, `front-proxy-client.key` for the front proxy client +## The kubeadm configuration file format + +Most kubeadm commands support a `--config` flag which allows passing a configuration file from +disk. The configuration file format follows the common Kubernetes API `apiVersion` / `kind` scheme, +but is considered a component configuration format. Several Kubernetes components, such as the kubelet, +also support file-based configuration. + +Different kubeadm subcommands require a different `kind` of configuration file. +For example, `InitConfiguration` for `kubeadm init`, `JoinConfiguration` for `kubeadm join`, `UpgradeConfiguration` for `kubeadm upgrade` and `ResetConfiguration` +for `kubeadm reset`. + +The command `kubeadm config migrate` can be used to migrate an older format configuration +file to a newer (current) configuration format. The kubeadm tool only supports migrating from +deprecated configuration formats to the current format. + +See the [kubeadm configuration reference](/docs/reference/config-api/kubeadm-config.v1beta4/) page for more details. + ## kubeadm init workflow internal design The `kubeadm init` consists of a sequence of atomic work tasks to perform, @@ -109,8 +126,8 @@ The user can skip specific preflight checks or all of them with the `--ignore-pr - [Error] if API server bindPort or ports 10250/10251/10252 are used - [Error] if `/etc/kubernetes/manifest` folder already exists and it is not empty - [Error] if swap is on -- [Error] if `conntrack`, `ip`, `iptables`, `mount`, `nsenter` commands are not present in the command path -- [Warning] if `ebtables`, `ethtool`, `socat`, `tc`, `touch`, `crictl` commands are not present in the command path +- [Error] if `ip`, `iptables`, `mount`, `nsenter` commands are not present in the command path +- [Warning] if `ethtool`, `tc`, `touch` commands are not present in the command path - [Warning] if extra arg flags for API server, controller manager, scheduler contains some invalid options - [Warning] if connection to https://API.AdvertiseAddress:API.BindPort goes through proxy - [Warning] if connection to services subnet goes through proxy (only first address checked) @@ -188,7 +205,7 @@ Please note that: Kubeadm generates kubeconfig files with identities for control plane components: - A kubeconfig file for the kubelet to use during TLS bootstrap - - /etc/kubernetes/bootstrap-kubelet.conf. Inside this file, there is a bootstrap-token or embedded + `/etc/kubernetes/bootstrap-kubelet.conf`. Inside this file, there is a bootstrap-token or embedded client certificates for authenticating this node with the cluster. This client certificate should: @@ -223,7 +240,15 @@ The `super-admin.conf` file must be stored in a safe location and should not be See [RBAC user facing role bindings](/docs/reference/access-authn-authz/rbac/#user-facing-roles) for additional information on RBAC and built-in ClusterRoles and groups. -Please note that: +You can run [`kubeadm kubeconfig user`](/docs/reference/setup-tools/kubeadm/kubeadm-kubeconfig/#cmd-kubeconfig-user) +to generate kubeconfig files for additional users. + +{{< caution >}} +The generated configuration files include an embedded authentication key, and you should treat +them as confidential. +{{< /caution >}} + +Also note that: 1. `ca.crt` certificate is embedded in all the kubeconfig files. 1. If a given kubeconfig file exists, and its content is evaluated as compliant with the above specs, @@ -253,7 +278,7 @@ Static Pod manifests share a set of common properties: - Leader election is enabled for both the controller-manager and the scheduler - Controller-manager and the scheduler will reference kubeconfig files with their respective, unique identities -- All static Pods get any extra flags specified by the user as described in +- All static Pods get any extra flags or patches that you specify, as described in [passing custom arguments to control plane components](/docs/setup/production-environment/tools/kubeadm/control-plane-flags/) - All static Pods get any extra Volumes specified by the user (Host path) @@ -344,10 +369,6 @@ the users: - `--allocate-node-cidrs=true` - `--cluster-cidr` and `--node-cidr-mask-size` flags according to the given CIDR -- If a cloud provider is specified, the corresponding `--cloud-provider` is specified together - with the `--cloud-config` path if such configuration file exists (this is experimental, alpha - and will be removed in a future version) - Other flags that are set unconditionally are: - `--controllers` enabling all the default controllers plus `BootstrapSigner` and `TokenCleaner` @@ -365,7 +386,7 @@ Other flags that are set unconditionally are: #### Scheduler -The static Pod manifest for the scheduler is not affected by parameters provided by the users. +The static Pod manifest for the scheduler is not affected by parameters provided by the user. ### Generate static Pod manifest for local etcd @@ -389,12 +410,10 @@ Please note that: ### Wait for the control plane to come up -kubeadm waits (upto 4m0s) until `localhost:6443/healthz` (kube-apiserver liveness) returns `ok`. -However, in order to detect deadlock conditions, kubeadm fails fast if `localhost:10255/healthz` -(kubelet liveness) or `localhost:10255/healthz/syncloop` (kubelet readiness) don't return `ok` -within 40s and 60s respectively. +On control plane nodes, kubeadm waits up to 4 minutes for the control plane components +and the kubelet to be available. It does that by performing a health check on the respective +component `/healthz` or `/livez` endpoints. -kubeadm relies on the kubelet to pull the control plane images and run them properly as static Pods. After the control plane is up, kubeadm completes the tasks described in following paragraphs. ### Save the kubeadm ClusterConfiguration in a ConfigMap for later reference @@ -518,9 +537,8 @@ deployed as a DaemonSet: #### DNS -- The CoreDNS service is named `kube-dns`. This is done to prevent any interruption - in service when the user is switching the cluster DNS from kube-dns to CoreDNS - through the `--config` method described [here](/docs/reference/setup-tools/kubeadm/kubeadm-init-phase/#cmd-phase-addon). +- The CoreDNS service is named `kube-dns` for compatibility reasons with the legacy `kube-dns` +addon. - A ServiceAccount for CoreDNS is created in the `kube-system` namespace. @@ -534,8 +552,8 @@ You can use CoreDNS with kubeadm even when the related Service is named `kube-dn Similarly to `kubeadm init`, also `kubeadm join` internal workflow consists of a sequence of atomic work tasks to perform. -This is split into discovery (having the Node trust the Kubernetes Master) and TLS bootstrap -(having the Kubernetes Master trust the Node). +This is split into discovery (having the Node trust the Kubernetes API Server) and TLS bootstrap +(having the Kubernetes API Server trust the Node). see [Authenticating with Bootstrap Tokens](/docs/reference/access-authn-authz/bootstrap-tokens/) or the corresponding [design proposal](https://git.k8s.io/design-proposals-archive/cluster-lifecycle/bootstrap-discovery.md). @@ -545,12 +563,10 @@ or the corresponding [design proposal](https://git.k8s.io/design-proposals-archi `kubeadm` executes a set of preflight checks before starting the join, with the aim to verify preconditions and avoid common cluster startup problems. -Please note that: +Also note that: 1. `kubeadm join` preflight checks are basically a subset of `kubeadm init` preflight checks -1. Starting from 1.24, kubeadm uses crictl to communicate to all known CRI endpoints. -1. Starting from 1.9, kubeadm provides support for joining nodes running on Windows; in that case, - linux specific controls are skipped. +1. If you are joining a Windows node, Linux specific controls are skipped. 1. In any case the user can skip specific preflight checks (or eventually all preflight checks) with the `--ignore-preflight-errors` option. @@ -582,9 +598,8 @@ In order to prevent "man in the middle" attacks, several steps are taken: compared with the CA retrieved initially {{< note >}} - -Pub key validation can be skipped passing `--discovery-token-unsafe-skip-ca-verification` flag; -This weakens the kubeadm security model since others can potentially impersonate the Kubernetes Master. +You can skip CA validation by passing the `--discovery-token-unsafe-skip-ca-verification` flag on the command line. +This weakens the kubeadm security model since others can potentially impersonate the Kubernetes API server. {{< /note >}} #### File/https discovery @@ -620,3 +635,88 @@ is deleted. - The automatic CSR approval is managed by the csrapprover controller, according to the configuration present in the `kubeadm init` process {{< /note >}} + +## kubeadm upgrade workflow internal design + +`kubeadm upgrade` has sub-commands for handling the upgrade of the Kubernets cluster created by kubeadm. +You must run `kubeadm upgrade apply` on a control plane node (you can choose which one); +this starts the upgrade process. You then run `kubeadm upgrade node` on all remaining +nodes (both worker nodes and control plane nodes). + +Both `kubeadm upgrade apply` and `kubeadm upgrade node` have a `phase` subcommand which provides access +to the internal phases of the upgrade process. +See [`kubeadm upgrade phase`](/docs/reference/setup-tools/kubeadm/kubeadm-upgrade-phase/) for more details. + +Additional utility upgrade commands are `kubeadm upgrade plan` and `kubeadm upgrade diff`. + +All upgrade sub-commands support passing a configuration file. + +### kubeadm upgrade plan + +You can optionally run `kubeadm upgrade plan` before you run `kubeadm upgrade apply`. +The `plan` subcommand checks which versions are available to upgrade +to and validates whether your current cluster is upgradeable. + +### kubeadm upgrade diff + +This shows what differences would be applied to existing static pod manifests for control plane nodes. +A more verbose way to do the same thing is running `kubeadm upgrade apply --dry-run` or +`kubeadm upgrade node --dry-run`. + +### kubeadm upgrade apply + +`kubeadm upgrade apply` prepares the cluster for the upgrade of all nodes, and also +upgrades the control plane node where it's run. The steps it performs are: + +- Runs preflight checks similarly to `kubeadm init` and `kubeadm join`, ensuring container images are downloaded + and the cluster is in a good state to be upgraded. +- Upgrades the control plane manifest files on disk in `/etc/kubernetes/manifests` and waits + for the kubelet to restart the components if the files have changed. +- Uploads the updated kubeadm and kubelet configurations to the cluster in the `kubeadm-config` + and the `kubelet-config` ConfigMaps (both in the `kube-system` namespace). +- Writes updated kubelet configuration for this node in `/var/lib/kubelet/config.yaml`. +- Configures bootstrap token and the `cluster-info` ConfigMap for RBAC rules. This is the same as + in the `kubeadm init` stage and ensures that the cluster continues to support nodes joining with bootstrap tokens. +- Upgrades the kube-proxy and CoreDNS addons conditionally if all existing kube-apiservers in the cluster + have already been upgraded to the target version. +- Performs any post-upgrade tasks, such as, cleaning up deprecated features which are release specific. + +### kubeadm upgrade node + +`kubeadm upgrade node` upgrades a single control plane or worker node after the cluster upgrade has +started (by running `kubeadm upgrade apply`). The command detects if the node is a control plane node by checking +if the file `/etc/kubernetes/manifests/kube-apiserver.yaml` exists. On finding that file, the kubeadm tool +infers that there is a running kube-apiserver Pod on this node. + +- Runs preflight checks similarly to `kubeadm upgrade apply`. +- For control plane nodes, upgrades the control plane manifest files on disk in `/etc/kubernetes/manifests` + and waits for the kubelet to restart the components if the files have changed. +- Writes the updated kubelet configuration for this node in `/var/lib/kubelet/config.yaml`. +- (For control plane nodes) upgrades the kube-proxy and CoreDNS + {{< glossary_tooltip text="addons" term_id="addons" >}} conditionally, provided that all existing + API servers in the cluster have already been upgraded to the target version. +- Performs any post-upgrade tasks, such as cleaning up deprecated features which are release specific. + +## kubeadm reset workflow internal design + +You can use the `kubeadm reset` subcommand on a node where kubeadm commands previously executed. +This subcommand performs a **best-effort** cleanup of the node. +If certain actions fail you must intervene and perform manual cleanup. + +The command supports phases. +See [`kubeadm reset phase`](/docs/reference/setup-tools/kubeadm/kubeadm-reset-phase/) for more details. + +The command supports a configuration file. + +Additionally: +- IPVS, iptables and nftables rules are **not** cleaned up. +- CNI (network plugin) configuration is **not** cleaned up. +- `.kube/` in the user's home directory is **not** cleaned up. + +The command has the following stages: +- Runs preflight checks on the node to determine if its healthy. +- For control plane nodes, removes any local etcd member data. +- Stops the kubelet. +- Stops running containers. +- Unmounts any mounted directories in `/var/lib/kubelet`. +- Deletes any files and directories managed by kubeadm in `/var/lib/kubelet` and `/etc/kubernetes`. diff --git a/content/en/docs/reference/setup-tools/kubeadm/kubeadm-init-phase.md b/content/en/docs/reference/setup-tools/kubeadm/kubeadm-init-phase.md index 37841f4d4c861..a9fcb403e2722 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/kubeadm-init-phase.md +++ b/content/en/docs/reference/setup-tools/kubeadm/kubeadm-init-phase.md @@ -129,7 +129,7 @@ phases. {{< tabs name="tab-kubelet-finalize" >}} {{< tab name="kubelet-finalize" include="generated/kubeadm_init/kubeadm_init_phase_kubelet-finalize.md" />}} {{< tab name="kubelet-finalize-all" include="generated/kubeadm_init/kubeadm_init_phase_kubelet-finalize_all.md" />}} -{{< tab name="kubelet-finalize-cert-rotation" include="generated/kubeadm_init/kubeadm_init_phase_kubelet-finalize_experimental-cert-rotation.md" />}} +{{< tab name="kubelet-finalize-enable-client-cert-rotation" include="generated/kubeadm_init/kubeadm_init_phase_kubelet-finalize_enable-client-cert-rotation.md" />}} {{< /tabs >}} ## kubeadm init phase addon {#cmd-phase-addon} diff --git a/content/en/docs/reference/setup-tools/kubeadm/kubeadm-init.md b/content/en/docs/reference/setup-tools/kubeadm/kubeadm-init.md index 0838c9bb48401..a9dfab8da6092 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/kubeadm-init.md +++ b/content/en/docs/reference/setup-tools/kubeadm/kubeadm-init.md @@ -6,7 +6,7 @@ weight: 20 -This command initializes a Kubernetes control-plane node. +This command initializes a Kubernetes control plane node. @@ -14,7 +14,7 @@ This command initializes a Kubernetes control-plane node. ### Init workflow {#init-workflow} -`kubeadm init` bootstraps a Kubernetes control-plane node by executing the +`kubeadm init` bootstraps a Kubernetes control plane node by executing the following steps: 1. Runs a series of pre-flight checks to validate the system state @@ -25,11 +25,11 @@ following steps: 1. Generates a self-signed CA to set up identities for each component in the cluster. The user can provide their own CA cert and/or key by dropping it in the cert directory configured via `--cert-dir` (`/etc/kubernetes/pki` by default). - The APIServer certs will have additional SAN entries for any `--apiserver-cert-extra-sans` + The API server certs will have additional SAN entries for any `--apiserver-cert-extra-sans` arguments, lowercased if necessary. -1. Writes kubeconfig files in `/etc/kubernetes/` for the kubelet, the controller-manager and the - scheduler to use to connect to the API server, each with its own identity. Also +1. Writes kubeconfig files in `/etc/kubernetes/` for the kubelet, the controller-manager, and the + scheduler to connect to the API server, each with its own identity. Also additional kubeconfig files are written, for kubeadm as administrative entity (`admin.conf`) and for a super admin user that can bypass RBAC (`super-admin.conf`). @@ -42,13 +42,13 @@ following steps: Once control plane Pods are up and running, the `kubeadm init` sequence can continue. -1. Apply labels and taints to the control-plane node so that no additional workloads will +1. Apply labels and taints to the control plane node so that no additional workloads will run there. 1. Generates the token that additional nodes can use to register - themselves with a control-plane in the future. Optionally, the user can provide a + themselves with a control plane in the future. Optionally, the user can provide a token via `--token`, as described in the - [kubeadm token](/docs/reference/setup-tools/kubeadm/kubeadm-token/) docs. + [kubeadm token](/docs/reference/setup-tools/kubeadm/kubeadm-token/) documents. 1. Makes all the necessary configurations for allowing node joining with the [Bootstrap Tokens](/docs/reference/access-authn-authz/bootstrap-tokens/) and @@ -62,7 +62,7 @@ following steps: - Configure auto-approval for new CSR requests. - See [kubeadm join](/docs/reference/setup-tools/kubeadm/kubeadm-join/) for additional info. + See [kubeadm join](/docs/reference/setup-tools/kubeadm/kubeadm-join/) for additional information. 1. Installs a DNS server (CoreDNS) and the kube-proxy addon components via the API server. In Kubernetes version 1.11 and later CoreDNS is the default DNS server. @@ -74,7 +74,7 @@ following steps: ### Using init phases with kubeadm {#init-phases} -Kubeadm allows you to create a control-plane node in phases using the `kubeadm init phase` command. +kubeadm allows you to create a control plane node in phases using the `kubeadm init phase` command. To view the ordered list of phases and sub-phases you can call `kubeadm init --help`. The list will be located at the top of the help screen and each phase will have a description next to it. @@ -117,13 +117,13 @@ Alternatively, you can use the `skipPhases` field under `InitConfiguration`. ### Using kubeadm init with a configuration file {#config-file} {{< caution >}} -The config file is still considered beta and may change in future versions. +The configuration file is still considered beta and may change in future versions. {{< /caution >}} It's possible to configure `kubeadm init` with a configuration file instead of command line flags, and some more advanced features may only be available as configuration file options. This file is passed using the `--config` flag and it must -contain a `ClusterConfiguration` structure and optionally more structures separated by `---\n` +contain a `ClusterConfiguration` structure and optionally more structures separated by `---\n`. Mixing `--config` with others flags may not be allowed in some cases. The default configuration can be printed out using the @@ -137,7 +137,7 @@ For more information on the fields and usage of the configuration you can naviga ### Using kubeadm init with feature gates {#feature-gates} -Kubeadm supports a set of feature gates that are unique to kubeadm and can only be applied +kubeadm supports a set of feature gates that are unique to kubeadm and can only be applied during cluster creation with `kubeadm init`. These features can control the behavior of the cluster. Feature gates are removed after a feature graduates to GA. @@ -156,8 +156,8 @@ List of feature gates: Feature | Default | Alpha | Beta | GA :-------|:--------|:------|:-----|:---- `ControlPlaneKubeletLocalMode` | `false` | 1.31 | - | - -`EtcdLearnerMode` | `true` | 1.27 | 1.29 | - -`PublicKeysECDSA` | `false` | 1.19 | - | - +`EtcdLearnerMode` | `true` | 1.27 | 1.29 | 1.32 +`NodeLocalCRISocket` | `false` | 1.32 | - | - `WaitForAllControlPlaneComponents` | `false` | 1.30 | - | - {{< /table >}} @@ -169,46 +169,63 @@ Feature gate descriptions: `ControlPlaneKubeletLocalMode` : With this feature gate enabled, when joining a new control plane node, kubeadm will configure the kubelet -to connect to the local kube-apiserver. This ensures that there will not be a violation of the version skew -policy during rolling upgrades. + to connect to the local kube-apiserver. This ensures that there will not be a violation of the version skew + policy during rolling upgrades. `EtcdLearnerMode` : With this feature gate enabled, when joining a new control plane node, a new etcd member will be created -as a learner and promoted to a voting member only after the etcd data are fully aligned. - -`PublicKeysECDSA` -: Can be used to create a cluster that uses ECDSA certificates instead of the default RSA algorithm. -Renewal of existing ECDSA certificates is also supported using `kubeadm certs renew`, but you cannot -switch between the RSA and ECDSA algorithms on the fly or during upgrades. Kubernetes -{{< skew currentVersion >}} has a bug where keys in generated kubeconfig files are set use RSA -despite the feature gate being enabled. Kubernetes versions before v1.31 had a bug where keys in generated kubeconfig files -were set use RSA, even when you had enabled the `PublicKeysECDSA` feature gate. + as a learner and promoted to a voting member only after the etcd data are fully aligned. + +`NodeLocalCRISocket` +: With this feature gate enabled, kubeadm will read/write the CRI socket for each node from/to the file + `/var/lib/kubelet/instance-config.yaml` instead of reading/writing it from/to the annotation + `kubeadm.alpha.kubernetes.io/cri-socket` on the Node object. The new file is applied as an instance + configuration patch, before any other user managed patches are applied when the `--patches` flag + is used. It contains a single field `containerRuntimeEndpoint` from the + [KubeletConfiguration file format](/docs/reference/config-api/kubelet-config.v1beta1/). If the feature gate + is enabled during upgrade, but the file `/var/lib/kubelet/instance-config.yaml` does not exist yet, + kubeadm will attempt to read the CRI socket value from the file `/var/lib/kubelet/kubeadm-flags.env`. `WaitForAllControlPlaneComponents` -: With this feature gate enabled kubeadm will wait for all control plane components (kube-apiserver, -kube-controller-manager, kube-scheduler) on a control plane node to report status 200 on their `/healthz` -endpoints. These checks are performed on `https://127.0.0.1:PORT/healthz`, where `PORT` is taken from -`--secure-port` of a component. If you specify custom `--secure-port` values in the kubeadm configuration -they will be respected. Without the feature gate enabled, kubeadm will only wait for the kube-apiserver -on a control plane node to become ready. The wait process starts right after the kubelet on the host -is started by kubeadm. You are advised to enable this feature gate in case you wish to observe a ready -state from all control plane components during the `kubeadm init` or `kubeadm join` command execution. +: With this feature gate enabled, kubeadm will wait for all control plane components (kube-apiserver, + kube-controller-manager, kube-scheduler) on a control plane node to report status 200 on their `/livez` + or `/healthz` endpoints. These checks are performed on `https://ADDRESS:PORT/ENDPOINT`. + + - `PORT` is taken from `--secure-port` of a component. + - `ADDRESS` is `--advertise-address` for kube-apiserver and `--bind-address` for the + kube-controller-manager and kube-scheduler. + - `ENDPOINT` is only `/healthz` for kube-controller-manager until it supports `/livez` as well. + + If you specify custom `ADDRESS` or `PORT` in the kubeadm configuration they will be respected. + Without the feature gate enabled, kubeadm will only wait for the kube-apiserver + on a control plane node to become ready. The wait process starts right after the kubelet on the host + is started by kubeadm. You are advised to enable this feature gate in case you wish to observe a ready + state from all control plane components during the `kubeadm init` or `kubeadm join` command execution. List of deprecated feature gates: {{< table caption="kubeadm deprecated feature gates" >}} Feature | Default | Alpha | Beta | GA | Deprecated :-------|:--------|:------|:-----|:---|:---------- +`PublicKeysECDSA` | `false` | 1.19 | - | - | 1.31 `RootlessControlPlane` | `false` | 1.22 | - | - | 1.31 {{< /table >}} Feature gate descriptions: +`PublicKeysECDSA` +: Can be used to create a cluster that uses ECDSA certificates instead of the default RSA algorithm. + Renewal of existing ECDSA certificates is also supported using `kubeadm certs renew`, but you cannot + switch between the RSA and ECDSA algorithms on the fly or during upgrades. Kubernetes versions before v1.31 + had a bug where keys in generated kubeconfig files were set use RSA, even when you had enabled the + `PublicKeysECDSA` feature gate. This feature gate is deprecated in favor of the `encryptionAlgorithm` + functionality available in kubeadm v1beta4. + `RootlessControlPlane` : Setting this flag configures the kubeadm deployed control plane component static Pod containers -for `kube-apiserver`, `kube-controller-manager`, `kube-scheduler` and `etcd` to run as non-root users. -If the flag is not set, those components run as root. You can change the value of this feature gate before -you upgrade to a newer version of Kubernetes. + for `kube-apiserver`, `kube-controller-manager`, `kube-scheduler` and `etcd` to run as non-root users. + If the flag is not set, those components run as root. You can change the value of this feature gate before + you upgrade to a newer version of Kubernetes. List of removed feature gates: @@ -224,37 +241,41 @@ Feature gate descriptions: `IPv6DualStack` : This flag helps to configure components dual stack when the feature is in progress. For more details on Kubernetes -dual-stack support see [Dual-stack support with kubeadm](/docs/setup/production-environment/tools/kubeadm/dual-stack-support/). + dual-stack support see [Dual-stack support with kubeadm](/docs/setup/production-environment/tools/kubeadm/dual-stack-support/). `UnversionedKubeletConfigMap` : This flag controls the name of the {{< glossary_tooltip text="ConfigMap" term_id="configmap" >}} where kubeadm stores -kubelet configuration data. With this flag not specified or set to `true`, the ConfigMap is named `kubelet-config`. -If you set this flag to `false`, the name of the ConfigMap includes the major and minor version for Kubernetes -(for example: `kubelet-config-{{< skew currentVersion >}}`). Kubeadm ensures that RBAC rules for reading and writing -that ConfigMap are appropriate for the value you set. When kubeadm writes this ConfigMap (during `kubeadm init` -or `kubeadm upgrade apply`), kubeadm respects the value of `UnversionedKubeletConfigMap`. When reading that ConfigMap -(during `kubeadm join`, `kubeadm reset`, `kubeadm upgrade ...`), kubeadm attempts to use unversioned ConfigMap name first; -if that does not succeed, kubeadm falls back to using the legacy (versioned) name for that ConfigMap. + kubelet configuration data. With this flag not specified or set to `true`, the ConfigMap is named `kubelet-config`. + If you set this flag to `false`, the name of the ConfigMap includes the major and minor version for Kubernetes + (for example: `kubelet-config-{{< skew currentVersion >}}`). Kubeadm ensures that RBAC rules for reading and writing + that ConfigMap are appropriate for the value you set. When kubeadm writes this ConfigMap (during `kubeadm init` + or `kubeadm upgrade apply`), kubeadm respects the value of `UnversionedKubeletConfigMap`. When reading that ConfigMap + (during `kubeadm join`, `kubeadm reset`, `kubeadm upgrade`...), kubeadm attempts to use unversioned ConfigMap name first. + If that does not succeed, kubeadm falls back to using the legacy (versioned) name for that ConfigMap. `UpgradeAddonsBeforeControlPlane` -: This feature gate has been removed. It was introduced in v1.28 as a deprecated feature and then removed in v1.31. For documentation on older versions, please switch to the corresponding website version. +: This feature gate has been removed. It was introduced in v1.28 as a deprecated feature and then removed in v1.31. + For documentation on older versions, please switch to the corresponding website version. ### Adding kube-proxy parameters {#kube-proxy} For information about kube-proxy parameters in the kubeadm configuration see: + - [kube-proxy reference](/docs/reference/config-api/kube-proxy-config.v1alpha1/) For information about enabling IPVS mode with kubeadm see: + - [IPVS](https://github.com/kubernetes/kubernetes/blob/master/pkg/proxy/ipvs/README.md) ### Passing custom flags to control plane components {#control-plane-flags} For information about passing flags to control plane components see: + - [control-plane-flags](/docs/setup/production-environment/tools/kubeadm/control-plane-flags/) ### Running kubeadm without an Internet connection {#without-internet-connection} -For running kubeadm without an Internet connection you have to pre-pull the required control-plane images. +For running kubeadm without an Internet connection you have to pre-pull the required control plane images. You can list and pull the images using the `kubeadm config images` sub-command: @@ -292,26 +313,24 @@ can consume, you must: * Pull images from the defaults paths at `registry.k8s.io` using `kubeadm config images {list|pull}`. * Push images to the paths from `kubeadm config images list --config=config.yaml`, -where `config.yaml` contains the custom `imageRepository`, and/or `imageTag` -for etcd and CoreDNS. + where `config.yaml` contains the custom `imageRepository`, and/or `imageTag` for etcd and CoreDNS. * Pass the same `config.yaml` to `kubeadm init`. #### Custom sandbox (pause) images {#custom-pause-image} To set a custom image for these you need to configure this in your -{{< glossary_tooltip text="container runtime" term_id="container-runtime" >}} -to use the image. +{{< glossary_tooltip text="container runtime" term_id="container-runtime" >}} to use the image. Consult the documentation for your container runtime to find out how to change this setting; for selected container runtimes, you can also find advice within the [Container Runtimes](/docs/setup/production-environment/container-runtimes/) topic. -### Uploading control-plane certificates to the cluster +### Uploading control plane certificates to the cluster By adding the flag `--upload-certs` to `kubeadm init` you can temporary upload -the control-plane certificates to a Secret in the cluster. Please note that this Secret +the control plane certificates to a Secret in the cluster. Please note that this Secret will expire automatically after 2 hours. The certificates are encrypted using a 32byte key that can be specified using `--certificate-key`. The same key can be used -to download the certificates when additional control-plane nodes are joining, by passing +to download the certificates when additional control plane nodes are joining, by passing `--control-plane` and `--certificate-key` to `kubeadm join`. The following phase command can be used to re-upload the certificates after expiration: @@ -319,6 +338,7 @@ The following phase command can be used to re-upload the certificates after expi ```shell kubeadm init phase upload-certs --upload-certs --config=SOME_YAML_FILE ``` + {{< note >}} A predefined `certificateKey` can be provided in `InitConfiguration` when passing the [configuration file](/docs/reference/config-api/kubeadm-config.v1beta4/) with `--config`. @@ -351,12 +371,12 @@ For further information, see ### Use kubeadm with CRI runtimes -By default kubeadm attempts to detect your container runtime. For more details on this detection, +By default, kubeadm attempts to detect your container runtime. For more details on this detection, see the [kubeadm CRI installation guide](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#installing-runtime). ### Setting the node name -By default, `kubeadm` assigns a node name based on a machine's host address. +By default, kubeadm assigns a node name based on a machine's host address. You can override this setting with the `--node-name` flag. The flag passes the appropriate [`--hostname-override`](/docs/reference/command-line-tools-reference/kubelet/#options) value to the kubelet. @@ -369,24 +389,23 @@ Be aware that overriding the hostname can Rather than copying the token you obtained from `kubeadm init` to each node, as in the [basic kubeadm tutorial](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/), you can parallelize the token distribution for easier automation. To implement this automation, -you must know the IP address that the control-plane node will have after it is started, or use a +you must know the IP address that the control plane node will have after it is started, or use a DNS name or an address of a load balancer. -1. Generate a token. This token must have the form `<6 character string>.<16 - character string>`. More formally, it must match the regex: - `[a-z0-9]{6}\.[a-z0-9]{16}`. +1. Generate a token. This token must have the form `<6 character string>.<16 character string>`. + More formally, it must match the regex: `[a-z0-9]{6}\.[a-z0-9]{16}`. kubeadm can generate a token for you: ```shell - kubeadm token generate + kubeadm token generate ``` -1. Start both the control-plane node and the worker nodes concurrently with this token. - As they come up they should find each other and form the cluster. The same +1. Start both the control plane node and the worker nodes concurrently with this token. + As they come up they should find each other and form the cluster. The same `--token` argument can be used on both `kubeadm init` and `kubeadm join`. -1. Similar can be done for `--certificate-key` when joining additional control-plane +1. Similar can be done for `--certificate-key` when joining additional control plane nodes. The key can be generated using: ```shell @@ -394,13 +413,13 @@ DNS name or an address of a load balancer. ``` Once the cluster is up, you can use the `/etc/kubernetes/admin.conf` file from -a control-plane node to talk to the cluster with administrator credentials or +a control plane node to talk to the cluster with administrator credentials or [Generating kubeconfig files for additional users](/docs/tasks/administer-cluster/kubeadm/kubeadm-certs#kubeconfig-additional-users). Note that this style of bootstrap has some relaxed security guarantees because it does not allow the root CA hash to be validated with -`--discovery-token-ca-cert-hash` (since it's not generated when the nodes are -provisioned). For details, see the [kubeadm join](/docs/reference/setup-tools/kubeadm/kubeadm-join/). +`--discovery-token-ca-cert-hash` (since it's not generated when the nodes are provisioned). +For details, see the [kubeadm join](/docs/reference/setup-tools/kubeadm/kubeadm-join/). ## {{% heading "whatsnext" %}} @@ -412,4 +431,3 @@ provisioned). For details, see the [kubeadm join](/docs/reference/setup-tools/ku cluster to a newer version * [kubeadm reset](/docs/reference/setup-tools/kubeadm/kubeadm-reset/) to revert any changes made to this host by `kubeadm init` or `kubeadm join` - diff --git a/content/en/docs/reference/setup-tools/kubeadm/kubeadm-upgrade-phase.md b/content/en/docs/reference/setup-tools/kubeadm/kubeadm-upgrade-phase.md index 54e2428b8493a..80ba1cb0f8791 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/kubeadm-upgrade-phase.md +++ b/content/en/docs/reference/setup-tools/kubeadm/kubeadm-upgrade-phase.md @@ -1,23 +1,37 @@ --- -title: kubeadm upgrade phase -weight: 90 +title: kubeadm upgrade phases +weight: 40 content_type: concept --- -In v1.15.0, kubeadm introduced preliminary support for `kubeadm upgrade node` phases. -Phases for other `kubeadm upgrade` sub-commands such as `apply`, could be added in the -following releases. + +## kubeadm upgrade apply phase {#cmd-apply-phase} + +Using the phases of `kubeadm upgrade apply`, you can choose to execute the separate steps of the initial upgrade +of a control plane node. + +{{< tabs name="tab-phase" >}} +{{< tab name="phase" include="generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase.md" />}} +{{< tab name="preflight" include="generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_preflight.md" />}} +{{< tab name="control-plane" include="generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_control-plane.md" />}} +{{< tab name="upload-config" include="generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_upload-config.md" />}} +{{< tab name="kubelet-config" include="generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_kubelet-config.md" />}} +{{< tab name="bootstrap-token" include="generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_bootstrap-token.md" />}} +{{< tab name="addon" include="generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_addon.md" />}} +{{< tab name="post-upgrade" include="generated/kubeadm_upgrade/kubeadm_upgrade_apply_phase_post-upgrade.md" />}} +{{< /tabs >}} ## kubeadm upgrade node phase {#cmd-node-phase} -Using this phase you can choose to execute the separate steps of the upgrade of -secondary control-plane or worker nodes. Please note that `kubeadm upgrade apply` still has to -be called on a primary control-plane node. +Using the phases of `kubeadm upgrade node` you can choose to execute the separate steps of the upgrade of +secondary control-plane or worker nodes. {{< tabs name="tab-phase" >}} {{< tab name="phase" include="generated/kubeadm_upgrade/kubeadm_upgrade_node_phase.md" />}} {{< tab name="preflight" include="generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_preflight.md" />}} {{< tab name="control-plane" include="generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_control-plane.md" />}} {{< tab name="kubelet-config" include="generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_kubelet-config.md" />}} +{{< tab name="addon" include="generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_addon.md" />}} +{{< tab name="post-upgrade" include="generated/kubeadm_upgrade/kubeadm_upgrade_node_phase_post-upgrade.md" />}} {{< /tabs >}} ## {{% heading "whatsnext" %}} diff --git a/content/en/docs/reference/using-api/api-concepts.md b/content/en/docs/reference/using-api/api-concepts.md index c3d37b967ca66..6cbcbca585ee1 100644 --- a/content/en/docs/reference/using-api/api-concepts.md +++ b/content/en/docs/reference/using-api/api-concepts.md @@ -331,6 +331,36 @@ For example: Accept: application/vnd.kubernetes.protobuf, application/json ``` +### CBOR resource encoding {#cbor-encoding} + +{{< feature-state feature_gate_name="CBORServingAndStorage" >}} + +With the `CBORServingAndStorage` [feature +gate](/docs/reference/command-line-tools-reference/feature-gates/) enabled, request and response +bodies for all built-in resource types and all resources defined by a {{< glossary_tooltip +term_id="CustomResourceDefinition" text="CustomResourceDefinition" >}} may be encoded to the +[CBOR](https://www.rfc-editor.org/rfc/rfc8949) binary data format. CBOR is also supported at the {{< +glossary_tooltip text="aggregation layer" term_id="aggregation-layer" >}} if it is enabled in +individual aggregated API servers. + +Clients should indicate the IANA media type `application/cbor` in the `Content-Type` HTTP request +header when the request body contains a single CBOR [encoded data +item](https://www.rfc-editor.org/rfc/rfc8949.html#section-1.2-4.2), and in the `Accept` HTTP request +header when prepared to accept a CBOR encoded data item in the response. API servers will use +`application/cbor` in the `Content-Type` HTTP response header when the response body contains a +CBOR-encoded object. + +If an API server encodes its response to a [watch request](#efficient-detection-of-changes) using +CBOR, the response body will be a [CBOR Sequence](https://www.rfc-editor.org/rfc/rfc8742) and the +`Content-Type` HTTP response header will use the IANA media type `application/cbor-seq`. Each entry +of the sequence (if any) is a single CBOR-encoded watch event. + +In addition to the existing `application/apply-patch+yaml` media type for YAML-encoded [server-side +apply configurations](#patch-and-apply), API servers that enable CBOR will accept the +`application/apply-patch+cbor` media type for CBOR-encoded server-side apply configurations. There +is no supported CBOR equivalent for `application/json-patch+json` or `application/merge-patch+json`, +or `application/strategic-merge-patch+json`. + ## Efficient detection of changes The Kubernetes API allows clients to make an initial request for an object or a @@ -443,13 +473,11 @@ the API server will send any `BOOKMARK` event even when requested. On large clusters, retrieving the collection of some resource types may result in a significant increase of resource usage (primarily RAM) on the control plane. -In order to alleviate its impact and simplify the user experience of the **list** + **watch** -pattern, Kubernetes v1.27 introduces as an alpha feature the support -for requesting the initial state (previously requested via the **list** request) as part of -the **watch** request. +To alleviate the impact and simplify the user experience of the **list** + **watch** +pattern, Kubernetes v1.32 promotes to beta the feature that allows requesting the initial state +(previously requested via the **list** request) as part of the **watch** request. -Provided that the `WatchList` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) -is enabled, this can be achieved by specifying `sendInitialEvents=true` as query string parameter +On the client-side the initial state can be requested by specifying `sendInitialEvents=true` as query string parameter in a **watch** request. If set, the API server starts the watch stream with synthetic init events (of type `ADDED`) to build the whole state of all existing objects followed by a [`BOOKMARK` event](/docs/reference/using-api/api-concepts/#watch-bookmarks) @@ -847,6 +875,41 @@ not vulnerable to ordering changes in the list. Once the last finalizer is removed, the resource is actually removed from etcd. +### Force deletion + +{{< feature-state feature_gate_name="AllowUnsafeMalformedObjectDeletion" >}} + +{{< caution >}} +This may break the workload associated with the resource being force deleted, if it +relies on the normal deletion flow, so cluster breaking consequences may apply. +{{< /caution >}} + +By enabling the delete option `ignoreStoreReadErrorWithClusterBreakingPotential`, the +user can perform an unsafe force **delete** operation of an undecryptable/corrupt +resource. This option is behind an ALPHA feature gate, and it is disabled by +default. In order to use this option, the cluster operator must enable the feature by +setting the command line option `--feature-gates=AllowUnsafeMalformedObjectDeletion=true`. + +{{< note >}} +The user performing the force **delete** operation must have the privileges to do both +the **delete** and **unsafe-delete-ignore-read-errors** verbs on the given resource. +{{< /note >}} + +A resource is considered corrupt if it can not be successfully retrieved from the +storage due to a) transformation error (for example: decryption failure), or b) the object +failed to decode. The API server first attempts a normal deletion, and if it fails with +a _corrupt resource_ error then it triggers the force delete. A force **delete** operation +is unsafe because it ignores finalizer constraints, and skips precondition checks. + +The default value for this option is `false`, this maintains backward compatibility. +For a **delete** request with `ignoreStoreReadErrorWithClusterBreakingPotential` +set to `true`, the fields `dryRun`, `gracePeriodSeconds`, `orphanDependents`, +`preconditions`, and `propagationPolicy` must be left unset. + +{{< note >}} +If the user issues a **delete** request with `ignoreStoreReadErrorWithClusterBreakingPotential` +set to `true` on an otherwise readable resource, the API server aborts the request with an error. +{{< /note >}} ## Single resource API diff --git a/content/en/docs/setup/production-environment/_index.md b/content/en/docs/setup/production-environment/_index.md index a7972e093091f..775c1d21dba23 100644 --- a/content/en/docs/setup/production-environment/_index.md +++ b/content/en/docs/setup/production-environment/_index.md @@ -232,7 +232,7 @@ As someone setting up authentication and authorization on your production Kubern - *Set the authorization mode*: When the Kubernetes API server ([kube-apiserver](/docs/reference/command-line-tools-reference/kube-apiserver/)) - starts, the supported authentication modes must be set using the *--authorization-mode* + starts, supported authorization modes must be set using an *--authorization-config* file or the *--authorization-mode* flag. For example, that flag in the *kube-adminserver.yaml* file (in */etc/kubernetes/manifests*) could be set to Node,RBAC. This would allow Node and RBAC authorization for authenticated requests. - *Create user certificates and role bindings (RBAC)*: If you are using RBAC diff --git a/content/en/docs/tasks/administer-cluster/cpu-management-policies.md b/content/en/docs/tasks/administer-cluster/cpu-management-policies.md index 287c2ef8d8f45..95a257f2818c6 100644 --- a/content/en/docs/tasks/administer-cluster/cpu-management-policies.md +++ b/content/en/docs/tasks/administer-cluster/cpu-management-policies.md @@ -25,6 +25,8 @@ For detailed information on resource management, please refer to the [Resource Management for Pods and Containers](/docs/concepts/configuration/manage-resources-containers) documentation. +For detailed information on how the kubelet implements resource management, please refer to the +[Node ResourceManagers](/docs/concepts/policy/node-resource-managers) documentation. ## {{% heading "prerequisites" %}} @@ -36,7 +38,7 @@ If you are running an older version of Kubernetes, please look at the documentat -## CPU Management Policies +## Configuring CPU management policies By default, the kubelet uses [CFS quota](https://en.wikipedia.org/wiki/Completely_Fair_Scheduler) to enforce pod CPU limits.  When the node runs many CPU-bound pods, @@ -49,6 +51,14 @@ However, in workloads where CPU cache affinity and scheduling latency significantly affect workload performance, the kubelet allows alternative CPU management policies to determine some placement preferences on the node. +## Windows Support + +{{< feature-state feature_gate_name="WindowsCPUAndMemoryAffinity" >}} + +CPU Manager support can be enabled on Windows by using the `WindowsCPUAndMemoryAffinity` feature gate +and it requires support in the container runtime. +Once the feature gate is enabled, follow the steps below to configure the [CPU manager policy](#configuration). + ### Configuration The CPU Manager policy is set with the `--cpu-manager-policy` kubelet @@ -100,32 +110,17 @@ process will result in kubelet crashlooping with the following error: could not restore state from checkpoint: configured policy "static" differs from state checkpoint policy "none", please drain this node and delete the CPU manager checkpoint file "/var/lib/kubelet/cpu_manager_state" before restarting Kubelet ``` -### None policy - -The `none` policy explicitly enables the existing default CPU -affinity scheme, providing no affinity beyond what the OS scheduler does -automatically.  Limits on CPU usage for -[Guaranteed pods](/docs/tasks/configure-pod-container/quality-service-pod/) and -[Burstable pods](/docs/tasks/configure-pod-container/quality-service-pod/) -are enforced using CFS quota. - -### Static policy - -The `static` policy allows containers in `Guaranteed` pods with integer CPU -`requests` access to exclusive CPUs on the node. This exclusivity is enforced -using the [cpuset cgroup controller](https://www.kernel.org/doc/Documentation/cgroup-v1/cpusets.txt). - {{< note >}} -System services such as the container runtime and the kubelet itself can continue to run on these exclusive CPUs.  The exclusivity only extends to other pods. -{{< /note >}} - -{{< note >}} -CPU Manager doesn't support offlining and onlining of -CPUs at runtime. Also, if the set of online CPUs changes on the node, -the node must be drained and CPU manager manually reset by deleting the +if the set of online CPUs changes on the node, the node must be drained and CPU manager manually reset by deleting the state file `cpu_manager_state` in the kubelet root directory. {{< /note >}} +#### `none` policy configuration + +This policy has no extra configuration items. + +#### `static` policy configuration + This policy manages a shared pool of CPUs that initially contains all CPUs in the node. The amount of exclusively allocatable CPUs is equal to the total number of CPUs in the node minus any CPU reservations by the kubelet `--kube-reserved` or @@ -147,115 +142,7 @@ the static policy is enabled. This is because zero CPU reservation would allow t pool to become empty. {{< /note >}} -As `Guaranteed` pods whose containers fit the requirements for being statically -assigned are scheduled to the node, CPUs are removed from the shared pool and -placed in the cpuset for the container. CFS quota is not used to bound -the CPU usage of these containers as their usage is bound by the scheduling domain -itself. In others words, the number of CPUs in the container cpuset is equal to the integer -CPU `limit` specified in the pod spec. This static assignment increases CPU -affinity and decreases context switches due to throttling for the CPU-bound -workload. - -Consider the containers in the following pod specs: - -```yaml -spec: - containers: - - name: nginx - image: nginx -``` - -The pod above runs in the `BestEffort` QoS class because no resource `requests` or -`limits` are specified. It runs in the shared pool. - -```yaml -spec: - containers: - - name: nginx - image: nginx - resources: - limits: - memory: "200Mi" - requests: - memory: "100Mi" -``` - -The pod above runs in the `Burstable` QoS class because resource `requests` do not -equal `limits` and the `cpu` quantity is not specified. It runs in the shared -pool. - -```yaml -spec: - containers: - - name: nginx - image: nginx - resources: - limits: - memory: "200Mi" - cpu: "2" - requests: - memory: "100Mi" - cpu: "1" -``` - -The pod above runs in the `Burstable` QoS class because resource `requests` do not -equal `limits`. It runs in the shared pool. - -```yaml -spec: - containers: - - name: nginx - image: nginx - resources: - limits: - memory: "200Mi" - cpu: "2" - requests: - memory: "200Mi" - cpu: "2" -``` - -The pod above runs in the `Guaranteed` QoS class because `requests` are equal to `limits`. -And the container's resource limit for the CPU resource is an integer greater than -or equal to one. The `nginx` container is granted 2 exclusive CPUs. - - -```yaml -spec: - containers: - - name: nginx - image: nginx - resources: - limits: - memory: "200Mi" - cpu: "1.5" - requests: - memory: "200Mi" - cpu: "1.5" -``` - -The pod above runs in the `Guaranteed` QoS class because `requests` are equal to `limits`. -But the container's resource limit for the CPU resource is a fraction. It runs in -the shared pool. - - -```yaml -spec: - containers: - - name: nginx - image: nginx - resources: - limits: - memory: "200Mi" - cpu: "2" -``` - -The pod above runs in the `Guaranteed` QoS class because only `limits` are specified -and `requests` are set equal to `limits` when not explicitly specified. And the -container's resource limit for the CPU resource is an integer greater than or -equal to one. The `nginx` container is granted 2 exclusive CPUs. - -#### Static policy options +#### Static policy options {#cpu-policy-static--options} You can toggle groups of options on and off based upon their maturity level using the following feature gates: @@ -268,56 +155,8 @@ The following policy options exist for the static `CPUManager` policy: * `distribute-cpus-across-numa` (alpha, hidden by default) (1.23 or higher) * `align-by-socket` (alpha, hidden by default) (1.25 or higher) * `distribute-cpus-across-cores` (alpha, hidden by default) (1.31 or higher) - -If the `full-pcpus-only` policy option is specified, the static policy will always allocate full physical cores. -By default, without this option, the static policy allocates CPUs using a topology-aware best-fit allocation. -On SMT enabled systems, the policy can allocate individual virtual cores, which correspond to hardware threads. -This can lead to different containers sharing the same physical cores; this behaviour in turn contributes -to the [noisy neighbours problem](https://en.wikipedia.org/wiki/Cloud_computing_issues#Performance_interference_and_noisy_neighbors). -With the option enabled, the pod will be admitted by the kubelet only if the CPU request of all its containers -can be fulfilled by allocating full physical cores. -If the pod does not pass the admission, it will be put in Failed state with the message `SMTAlignmentError`. - -If the `distribute-cpus-across-numa`policy option is specified, the static -policy will evenly distribute CPUs across NUMA nodes in cases where more than -one NUMA node is required to satisfy the allocation. -By default, the `CPUManager` will pack CPUs onto one NUMA node until it is -filled, with any remaining CPUs simply spilling over to the next NUMA node. -This can cause undesired bottlenecks in parallel code relying on barriers (and -similar synchronization primitives), as this type of code tends to run only as -fast as its slowest worker (which is slowed down by the fact that fewer CPUs -are available on at least one NUMA node). -By distributing CPUs evenly across NUMA nodes, application developers can more -easily ensure that no single worker suffers from NUMA effects more than any -other, improving the overall performance of these types of applications. - -If the `align-by-socket` policy option is specified, CPUs will be considered -aligned at the socket boundary when deciding how to allocate CPUs to a -container. By default, the `CPUManager` aligns CPU allocations at the NUMA -boundary, which could result in performance degradation if CPUs need to be -pulled from more than one NUMA node to satisfy the allocation. Although it -tries to ensure that all CPUs are allocated from the _minimum_ number of NUMA -nodes, there is no guarantee that those NUMA nodes will be on the same socket. -By directing the `CPUManager` to explicitly align CPUs at the socket boundary -rather than the NUMA boundary, we are able to avoid such issues. Note, this -policy option is not compatible with `TopologyManager` `single-numa-node` -policy and does not apply to hardware where the number of sockets is greater -than number of NUMA nodes. - - -If the `distribute-cpus-across-cores` policy option is specified, the static policy -will attempt to allocate virtual cores (hardware threads) across different physical cores. -By default, the `CPUManager` tends to pack cpus onto as few physical cores as possible, -which can lead to contention among cpus on the same physical core and result -in performance bottlenecks. By enabling the `distribute-cpus-across-cores` policy, -the static policy ensures that cpus are distributed across as many physical cores -as possible, reducing the contention on the same physical core and thereby -improving overall performance. However, it is important to note that this strategy -might be less effective when the system is heavily loaded. Under such conditions, -the benefit of reducing contention diminishes. Conversely, default behavior -can help in reducing inter-core communication overhead, potentially providing -better performance under high load conditions. - +* `strict-cpu-reservation` (alpha, hidden by default) (1.32 or higher) +* `prefer-align-cpus-by-uncorecache` (alpha, hidden by default) (1.32 or higher) The `full-pcpus-only` option can be enabled by adding `full-pcpus-only=true` to the CPUManager policy options. @@ -334,3 +173,14 @@ The `distribute-cpus-across-cores` option can be enabled by adding `distribute-cpus-across-cores=true` to the `CPUManager` policy options. It cannot be used with `full-pcpus-only` or `distribute-cpus-across-numa` policy options together at this moment. + +The `strict-cpu-reservation` option can be enabled by adding `strict-cpu-reservation=true` to +the CPUManager policy options followed by removing the `/var/lib/kubelet/cpu_manager_state` file and restart kubelet. + +The `prefer-align-cpus-by-uncorecache` option can be enabled by adding the +`prefer-align-cpus-by-uncorecache` to the `CPUManager` policy options. If +incompatible options are used, the kubelet will fail to start with the error +explained in the logs. + +For mode detail about the behavior of the individual options you can configure, please refer to the +[Node ResourceManagers](/docs/concepts/policy/node-resource-managers) documentation. diff --git a/content/en/docs/tasks/administer-cluster/kubeadm/kubeadm-certs.md b/content/en/docs/tasks/administer-cluster/kubeadm/kubeadm-certs.md index ab362ef8fbdcc..21d711462008c 100644 --- a/content/en/docs/tasks/administer-cluster/kubeadm/kubeadm-certs.md +++ b/content/en/docs/tasks/administer-cluster/kubeadm/kubeadm-certs.md @@ -24,6 +24,8 @@ Following this recommendation helps you to to stay secure. You should be familiar with [PKI certificates and requirements in Kubernetes](/docs/setup/best-practices/certificates/). +You should be familiar with how to pass a [configuration](/docs/reference/config-api/kubeadm-config.v1beta4/) file to the kubeadm commands. + This guide covers the usage of the `openssl` command (used for manual certificate signing, if you choose that approach), but you can use your preferred tools. @@ -45,6 +47,37 @@ kubeadm does not overwrite them. This means you can, for example, copy an existi CA into `/etc/kubernetes/pki/ca.crt` and `/etc/kubernetes/pki/ca.key`, and kubeadm will use this CA for signing the rest of the certificates. +## Choosing an encryption algorithm {#choosing-encryption-algorithm} + +kubeadm allows you to choose an encryption algorithm that is used for creating +public and private keys. That can be done by using the `encryptionAlgorithm` field of the +kubeadm configuration: + +```yaml +apiVersion: kubeadm.k8s.io/v1beta4 +kind: ClusterConfiguration +encryptionAlgorithm: +``` + +`` can be one of `RSA-2048` (default), `RSA-3072`, `RSA-4096` or `ECDSA-P256`. + +## Choosing certificate validity period {#choosing-cert-validity-period} + +kubeadm allows you to choose the validity period of CA and leaf certificates. +That can be done by using the `certificateValidityPeriod` and `caCertificateValidityPeriod` +fields of the kubeadm configuration: + +```yaml +apiVersion: kubeadm.k8s.io/v1beta4 +kind: ClusterConfiguration +certificateValidityPeriod: 8760h # Default: 365 days × 24 hours = 1 year +caCertificateValidityPeriod: 87600h # Default: 365 days × 24 hours * 10 = 10 years +``` + +The values of the fields follow the accepted format for +[Go's `time.Duration` values](https://pkg.go.dev/time#ParseDuration), with the longest supported +unit being `h` (hours). + ## External CA mode {#external-ca-mode} It is also possible to provide only the `ca.crt` file and not the diff --git a/content/en/docs/tasks/administer-cluster/manage-resources/cpu-constraint-namespace.md b/content/en/docs/tasks/administer-cluster/manage-resources/cpu-constraint-namespace.md index f4a35c9e6ea1d..e2c74b7704b07 100644 --- a/content/en/docs/tasks/administer-cluster/manage-resources/cpu-constraint-namespace.md +++ b/content/en/docs/tasks/administer-cluster/manage-resources/cpu-constraint-namespace.md @@ -277,6 +277,8 @@ kubectl delete namespace constraints-cpu-example * [Assign CPU Resources to Containers and Pods](/docs/tasks/configure-pod-container/assign-cpu-resource/) +* [Assign Pod-level CPU and memory resources](/docs/tasks/configure-pod-container/assign-pod-level-resources/) + * [Configure Quality of Service for Pods](/docs/tasks/configure-pod-container/quality-service-pod/) diff --git a/content/en/docs/tasks/administer-cluster/manage-resources/cpu-default-namespace.md b/content/en/docs/tasks/administer-cluster/manage-resources/cpu-default-namespace.md index 4bda1d192cf24..7ce1e82278569 100644 --- a/content/en/docs/tasks/administer-cluster/manage-resources/cpu-default-namespace.md +++ b/content/en/docs/tasks/administer-cluster/manage-resources/cpu-default-namespace.md @@ -211,6 +211,8 @@ kubectl delete namespace default-cpu-example * [Assign CPU Resources to Containers and Pods](/docs/tasks/configure-pod-container/assign-cpu-resource/) +* [Assign Pod-level CPU and memory resources](/docs/tasks/configure-pod-container/assign-pod-level-resources/) + * [Configure Quality of Service for Pods](/docs/tasks/configure-pod-container/quality-service-pod/) diff --git a/content/en/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/en/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index 2bd7dce1d70c9..1539e3c1f5e03 100644 --- a/content/en/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/en/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -275,6 +275,8 @@ kubectl delete namespace constraints-mem-example * [Assign CPU Resources to Containers and Pods](/docs/tasks/configure-pod-container/assign-cpu-resource/) +* [Assign Pod-level CPU and memory resources](/docs/tasks/configure-pod-container/assign-pod-level-resources/) + * [Configure Quality of Service for Pods](/docs/tasks/configure-pod-container/quality-service-pod/) diff --git a/content/en/docs/tasks/administer-cluster/manage-resources/memory-default-namespace.md b/content/en/docs/tasks/administer-cluster/manage-resources/memory-default-namespace.md index bba2a7a8badd0..6fe2053f51a59 100644 --- a/content/en/docs/tasks/administer-cluster/manage-resources/memory-default-namespace.md +++ b/content/en/docs/tasks/administer-cluster/manage-resources/memory-default-namespace.md @@ -228,6 +228,8 @@ kubectl delete namespace default-mem-example * [Assign CPU Resources to Containers and Pods](/docs/tasks/configure-pod-container/assign-cpu-resource/) +* [Assign Pod-level CPU and memory resources](/docs/tasks/configure-pod-container/assign-pod-level-resources/) + * [Configure Quality of Service for Pods](/docs/tasks/configure-pod-container/quality-service-pod/) diff --git a/content/en/docs/tasks/administer-cluster/manage-resources/quota-memory-cpu-namespace.md b/content/en/docs/tasks/administer-cluster/manage-resources/quota-memory-cpu-namespace.md index b335012f80974..8aca1b82c0bac 100644 --- a/content/en/docs/tasks/administer-cluster/manage-resources/quota-memory-cpu-namespace.md +++ b/content/en/docs/tasks/administer-cluster/manage-resources/quota-memory-cpu-namespace.md @@ -185,6 +185,8 @@ kubectl delete namespace quota-mem-cpu-example * [Assign CPU Resources to Containers and Pods](/docs/tasks/configure-pod-container/assign-cpu-resource/) +* [Assign Pod-level CPU and memory resources](/docs/tasks/configure-pod-container/assign-pod-level-resources/) + * [Configure Quality of Service for Pods](/docs/tasks/configure-pod-container/quality-service-pod/) diff --git a/content/en/docs/tasks/administer-cluster/manage-resources/quota-pod-namespace.md b/content/en/docs/tasks/administer-cluster/manage-resources/quota-pod-namespace.md index 510bb7df5e5d2..6d6a0fb838303 100644 --- a/content/en/docs/tasks/administer-cluster/manage-resources/quota-pod-namespace.md +++ b/content/en/docs/tasks/administer-cluster/manage-resources/quota-pod-namespace.md @@ -142,6 +142,8 @@ kubectl delete namespace quota-pod-example * [Assign CPU Resources to Containers and Pods](/docs/tasks/configure-pod-container/assign-cpu-resource/) +* [Assign Pod-level CPU and memory resources](/docs/tasks/configure-pod-container/assign-pod-level-resources/) + * [Configure Quality of Service for Pods](/docs/tasks/configure-pod-container/quality-service-pod/) diff --git a/content/en/docs/tasks/administer-cluster/memory-manager.md b/content/en/docs/tasks/administer-cluster/memory-manager.md index 459acb54112b7..0f133d7db9fc5 100644 --- a/content/en/docs/tasks/administer-cluster/memory-manager.md +++ b/content/en/docs/tasks/administer-cluster/memory-manager.md @@ -6,7 +6,7 @@ reviewers: - derekwaynecarr content_type: task -min-kubernetes-server-version: v1.21 +min-kubernetes-server-version: v1.32 weight: 410 --- @@ -46,7 +46,7 @@ Preceding v1.22, the `kubelet` must be started with the following flag: in order to enable the Memory Manager feature. -## How Memory Manager Operates? +## How does the Memory Manager Operate? The Memory Manager currently offers the guaranteed memory (and hugepages) allocation for Pods in Guaranteed QoS class. @@ -57,7 +57,7 @@ prepare and deploy a `Guaranteed` pod as illustrated in the section The Memory Manager is a Hint Provider, and it provides topology hints for the Topology Manager which then aligns the requested resources according to these topology hints. -It also enforces `cgroups` (i.e. `cpuset.mems`) for pods. +On Linux, it also enforces `cgroups` (i.e. `cpuset.mems`) for pods. The complete flow diagram concerning pod admission and deployment process is illustrated in [Memory Manager KEP: Design Overview][4] and below: @@ -91,6 +91,14 @@ The problem has been solved as elaborated in Also, reference [Memory Manager KEP: Simulation - how the Memory Manager works? (by examples)][1] illustrates how the management of groups occurs. +### Windows Support + +{{< feature-state feature_gate_name="WindowsCPUAndMemoryAffinity" >}} + +Windows support can be enabled via the `WindowsCPUAndMemoryAffinity` feature gate +and it requires support in the container runtime. +Only the [BestEffort Policy](#policy-best-effort) is supported on Windows. + ## Memory Manager configuration Other Managers should be first pre-configured. Next, the Memory Manager feature should be enabled @@ -103,7 +111,8 @@ node stability (section [Reserved memory flag](#reserved-memory-flag)). Memory Manager supports two policies. You can select a policy via a `kubelet` flag `--memory-manager-policy`: * `None` (default) -* `Static` +* `Static` (Linux only) +* `BestEffort` (Windows Only) #### None policy {#policy-none} @@ -123,6 +132,24 @@ In the case of the `BestEffort` or `Burstable` pod, the `Static` Memory Manager the default topology hint as there is no request for the guaranteed memory, and does not reserve the memory in the internal [NodeMap][2] object. +This policy is only supported on Linux. + +#### BestEffort policy {#policy-best-effort} + +{{< feature-state feature_gate_name="WindowsCPUAndMemoryAffinity" >}} + +This policy is only supported on Windows. + +On Windows, NUMA node assignment works differently than Linux. +There is no mechanism to ensure that Memory access only comes from a specific NUMA node. +Instead the Windows scheduler will select the most optimal NUMA node based on the CPU(s) assignments. +It is possible that Windows might use other NUMA nodes if deemed optimal by the Windows scheduler. + +The policy does track the amount of memory available and requested through the internal [NodeMap][2]. +The memory manager will make a best effort at ensuring that enough memory is available on +a NUMA node before making the assignment. +This means that in most cases memory assignment should function as expected. + ### Reserved memory flag The [Node Allocatable](/docs/tasks/administer-cluster/reserve-compute-resources/) mechanism @@ -217,13 +244,17 @@ display an error. Here is an example of a correct configuration: ```shell ---feature-gates=MemoryManager=true --kube-reserved=cpu=4,memory=4Gi --system-reserved=cpu=1,memory=1Gi --memory-manager-policy=Static --reserved-memory '0:memory=3Gi;1:memory=2148Mi' ``` +Prior to Kubernetes 1.32, you also need to add +```shell +--feature-gates=MemoryManager=true +``` + Let us validate the configuration above: 1. `kube-reserved + system-reserved + eviction-hard(default) = reserved-memory(0) + reserved-memory(1)` diff --git a/content/en/docs/tasks/administer-cluster/quota-api-object.md b/content/en/docs/tasks/administer-cluster/quota-api-object.md index f1fd715402a9b..9af4aefe0654a 100644 --- a/content/en/docs/tasks/administer-cluster/quota-api-object.md +++ b/content/en/docs/tasks/administer-cluster/quota-api-object.md @@ -167,6 +167,8 @@ kubectl delete namespace quota-object-example * [Assign CPU Resources to Containers and Pods](/docs/tasks/configure-pod-container/assign-cpu-resource/) +* [Assign Pod-level CPU and memory resources](/docs/tasks/configure-pod-container/assign-pod-level-resources/) + * [Configure Quality of Service for Pods](/docs/tasks/configure-pod-container/quality-service-pod/) diff --git a/content/en/docs/tasks/administer-cluster/sysctl-cluster.md b/content/en/docs/tasks/administer-cluster/sysctl-cluster.md index fa22ffd1c5cc2..c638a54823400 100644 --- a/content/en/docs/tasks/administer-cluster/sysctl-cluster.md +++ b/content/en/docs/tasks/administer-cluster/sysctl-cluster.md @@ -81,6 +81,8 @@ The following sysctls are supported in the _safe_ set: - `net.ipv4.tcp_fin_timeout` (since Kubernetes 1.29, needs kernel 4.6+); - `net.ipv4.tcp_keepalive_intvl` (since Kubernetes 1.29, needs kernel 4.5+); - `net.ipv4.tcp_keepalive_probes` (since Kubernetes 1.29, needs kernel 4.5+). +- `net.ipv4.tcp_rmem` (since Kubernetes 1.32, needs kernel 4.15+). +- `net.ipv4.tcp_wmem` (since Kubernetes 1.32, needs kernel 4.15+). {{< note >}} There are some exceptions to the set of safe sysctls: diff --git a/content/en/docs/tasks/administer-cluster/topology-manager.md b/content/en/docs/tasks/administer-cluster/topology-manager.md index 21935b0bd4282..69972a0b7e9b6 100644 --- a/content/en/docs/tasks/administer-cluster/topology-manager.md +++ b/content/en/docs/tasks/administer-cluster/topology-manager.md @@ -58,6 +58,13 @@ the pod can be accepted or rejected from the node based on the selected hint. The hint is then stored in the Topology Manager for use by the *Hint Providers* when making the resource allocation decisions. +## Windows Support + +{{< feature-state feature_gate_name="WindowsCPUAndMemoryAffinity" >}} + +The Topology Manager support can be enabled on Windows by using the `WindowsCPUAndMemoryAffinity` feature gate and +it requires support in the container runtime. + ## Topology manager scopes and policies The Topology Manager currently: @@ -223,12 +230,11 @@ You can toggle groups of options on and off based upon their maturity level usin You will still have to enable each option using the `TopologyManagerPolicyOptions` kubelet option. -### `prefer-closest-numa-nodes` (beta) {#policy-option-prefer-closest-numa-nodes} +### `prefer-closest-numa-nodes` {#policy-option-prefer-closest-numa-nodes} -The `prefer-closest-numa-nodes` option is beta since Kubernetes 1.28. In Kubernetes {{< skew currentVersion >}} -this policy option is visible by default provided that the `TopologyManagerPolicyOptions` and -`TopologyManagerPolicyBetaOptions` [feature gates](/docs/reference/command-line-tools-reference/feature-gates/) -are enabled. +The `prefer-closest-numa-nodes` option is GA since Kubernetes 1.32. In Kubernetes {{< skew currentVersion >}} +this policy option is visible by default provided that the `TopologyManagerPolicyOptions` +[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) is enabled. The Topology Manager is not aware by default of NUMA distances, and does not take them into account when making Pod admission decisions. This limitation surfaces in multi-socket, as well as single-socket multi NUMA systems, diff --git a/content/en/docs/tasks/configure-pod-container/assign-cpu-resource.md b/content/en/docs/tasks/configure-pod-container/assign-cpu-resource.md index 27393f9efbe43..eff43f94ca69a 100644 --- a/content/en/docs/tasks/configure-pod-container/assign-cpu-resource.md +++ b/content/en/docs/tasks/configure-pod-container/assign-cpu-resource.md @@ -257,6 +257,8 @@ kubectl delete namespace cpu-example * [Assign Memory Resources to Containers and Pods](/docs/tasks/configure-pod-container/assign-memory-resource/) +* [Assign Pod-level CPU and memory resources](/docs/tasks/configure-pod-container/assign-pod-level-resources/) + * [Configure Quality of Service for Pods](/docs/tasks/configure-pod-container/quality-service-pod/) ### For cluster administrators diff --git a/content/en/docs/tasks/configure-pod-container/assign-memory-resource.md b/content/en/docs/tasks/configure-pod-container/assign-memory-resource.md index de4ba2c30edc7..bae19375cf430 100644 --- a/content/en/docs/tasks/configure-pod-container/assign-memory-resource.md +++ b/content/en/docs/tasks/configure-pod-container/assign-memory-resource.md @@ -340,6 +340,8 @@ kubectl delete namespace mem-example * [Assign CPU Resources to Containers and Pods](/docs/tasks/configure-pod-container/assign-cpu-resource/) +* [Assign Pod-level CPU and memory resources](/docs/tasks/configure-pod-container/assign-pod-level-resources/) + * [Configure Quality of Service for Pods](/docs/tasks/configure-pod-container/quality-service-pod/) ### For cluster administrators diff --git a/content/en/docs/tasks/configure-pod-container/assign-pod-level-resources.md b/content/en/docs/tasks/configure-pod-container/assign-pod-level-resources.md new file mode 100644 index 0000000000000..653e64944168a --- /dev/null +++ b/content/en/docs/tasks/configure-pod-container/assign-pod-level-resources.md @@ -0,0 +1,280 @@ +--- +title: Assign Pod-level CPU and memory resources +content_type: task +weight: 30 +min-kubernetes-server-version: 1.32 +--- + + + + +{{< feature-state feature_gate_name="PodLevelResources" >}} + +This page shows how to specify CPU and memory resources for a Pod at pod-level in +addition to container-level resource specifications. A Kubernetes node allocates +resources to a pod based on the pod's resource requests. These requests can be +defined at the pod level or individually for containers within the pod. When +both are present, the pod-level requests take precedence. + +Similarly, a pod's resource usage is restricted by limits, which can also be set at +the pod-level or individually for containers within the pod. Again, +pod-level limits are prioritized when both are present. This allows for flexible +resource management, enabling you to control resource allocation at both the pod and +container levels. + +In order to specify the resources at pod-level, it is required to enable +`PodLevelResources` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/). + +For Pod Level Resources: +* Priority: When both pod-level and container-level resources are specified, + pod-level resources take precedence. +* QoS: Pod-level resources take precedence in influencing the QoS class of the pod. +* OOM Score: The OOM score adjustment calculation considers both pod-level and + container-level resources. +* Compatibility: Pod-level resources are designed to be compatible with existing + features. + +## {{% heading "prerequisites" %}} + + +{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}} + +The `PodLevelResources` [feature +gate](/docs/reference/command-line-tools-reference/feature-gates/) must be enabled +for your control plane and for all nodes in your cluster. + + + +## Create a namespace + +Create a namespace so that the resources you create in this exercise are +isolated from the rest of your cluster. + +```shell +kubectl create namespace pod-resources-example +``` + +## Create a pod with memory requests and limits at pod-level + +To specify memory requests for a Pod at pod-level, include the `resources.requests.memory` +field in the Pod spec manifest. To specify a memory limit, include `resources.limits.memory`. + +In this exercise, you create a Pod that has one Container. The Pod has a +memory request of 100 MiB and a memory limit of 200 MiB. Here's the configuration +file for the Pod: + +{{% code_sample file="pods/resource/pod-level-memory-request-limit.yaml" %}} + +The `args` section in the manifest provides arguments for the container when it starts. +The `"--vm-bytes", "150M"` arguments tell the Container to attempt to allocate 150 MiB of memory. + +Create the Pod: + +```shell +kubectl apply -f https://k8s.io/examples/pods/resource/pod-level-memory-request-limit.yaml --namespace=pod-resources-example +``` + +Verify that the Pod is running: + +```shell +kubectl get pod memory-demo --namespace=pod-resources-example +``` + +View detailed information about the Pod: + +```shell +kubectl get pod memory-demo --output=yaml --namespace=pod-resources-example +``` + +The output shows that the Pod has a memory request of 100 MiB +and a memory limit of 200 MiB. + + +```yaml +... +spec: + containers: + ... + resources: + requests: + memory: 100Mi + limits: + memory: 200Mi +... +``` + +Run `kubectl top` to fetch the metrics for the pod: + +```shell +kubectl top pod memory-demo --namespace=pod-resources-example +``` + +The output shows that the Pod is using about 162,900,000 bytes of memory, which +is about 150 MiB. This is greater than the Pod's 100 MiB request, but within the +Pod's 200 MiB limit. + +``` +NAME CPU(cores) MEMORY(bytes) +memory-demo 162856960 +``` + +## Create a pod with CPU requests and limits at pod-level +To specify a CPU request for a Pod, include the `resources.requests.cpu` field +in the Pod spec manifest. To specify a CPU limit, include `resources.limits.cpu`. + +In this exercise, you create a Pod that has one container. The Pod has a request +of 0.5 CPU and a limit of 1 CPU. Here is the configuration file for the Pod: + +{{% code_sample file="pods/resource/pod-level-cpu-request-limit.yaml" %}} + +The `args` section of the configuration file provides arguments for the container when it starts. +The `-cpus "2"` argument tells the Container to attempt to use 2 CPUs. + +Create the Pod: + +```shell +kubectl apply -f https://k8s.io/examples/pods/resource/pod-level-cpu-request-limit.yaml --namespace=pod-resources-example +``` + +Verify that the Pod is running: + +```shell +kubectl get pod cpu-demo --namespace=pod-resources-example +``` + +View detailed information about the Pod: + +```shell +kubectl get pod cpu-demo --output=yaml --namespace=pod-resources-example +``` + +The output shows that the Pod has a CPU request of 500 milliCPU +and a CPU limit of 1 CPU. + +```yaml +spec: + containers: + ... + resources: + limits: + cpu: "1" + requests: + cpu: 500m +``` + +Use `kubectl top` to fetch the metrics for the Pod: + +```shell +kubectl top pod cpu-demo --namespace=pod-resources-example +``` + +This example output shows that the Pod is using 974 milliCPU, which is +slightly less than the limit of 1 CPU specified in the Pod configuration. + +``` +NAME CPU(cores) MEMORY(bytes) +cpu-demo 974m +``` + +Recall that by setting `-cpu "2"`, you configured the Container to attempt to use 2 +CPUs, but the Container is only being allowed to use about 1 CPU. The container's +CPU use is being throttled, because the container is attempting to use more CPU +resources than the Pod CPU limit. + +## Create a pod with resource requests and limits at both pod-level and container-level + +To assign CPU and memory resources to a Pod, you can specify them at both the pod +level and the container level. Include the `resources` field in the Pod spec to +define resources for the entire Pod. Additionally, include the `resources` field +within container's specification in the Pod's manifest to set container-specific +resource requirements. + +In this exercise, you'll create a Pod with two containers to explore the interaction +of pod-level and container-level resource specifications. The Pod itself will have +defined CPU requests and limits, while only one of the containers will have its own +explicit resource requests and limits. The other container will inherit the resource +constraints from the pod-level settings. Here's the configuration file for the Pod: + +{{% code_sample file="pods/resource/pod-level-resources.yaml" %}} + +Create the Pod: + +```shell +kubectl apply -f https://k8s.io/examples/pods/resource/pod-level-resources.yaml --namespace=pod-resources-example +``` + +Verify that the Pod Container is running: + +```shell +kubectl get pod-resources-demo --namespace=pod-resources-example +``` + +View detailed information about the Pod: + +```shell +kubectl get pod memory-demo --output=yaml --namespace=pod-resources-example +``` + +The output shows that one container in the Pod has a memory request of 50 MiB and a +CPU request of 0.5 cores, with a memory limit of 100 MiB and a CPU limit of 0.5 +cores. The Pod itself has a memory request of 100 MiB and a CPU request of +1 core, and a memory limit of 200 MiB and a CPU limit of 1 core. + +```yaml +... +containers: + name: pod-resources-demo-ctr-1 + resources: + requests: + cpu: 500m + memory: 50Mi + limits: + cpu: 500m + memory: 100Mi + ... + name: pod-resources-demo-ctr-2 + resources: {} +resources: + limits: + cpu: 1 + memory: 200Mi + requests: + cpu: 1 + memory: 100Mi +... +``` + +Since pod-level requests and limits are specified, the request guarantees for both +containers in the pod will be equal 1 core or CPU and 100Mi of memory. Additionally, +both containers together won't be able to use more resources than specified in the +pod-level limits, ensuring they cannot exceed a combined total of 200 MiB of memory +and 1 core of CPU. + +## Clean up + +Delete your namespace: + +```shell +kubectl delete namespace pod-resources-example +``` + +## {{% heading "whatsnext" %}} + + +### For application developers + +* [Assign Memory Resources to Containers and Pods](/docs/tasks/configure-pod-container/assign-memory-resource/) + +* [Assign CPU Resources to Containers and Pods](/docs/tasks/configure-pod-container/assign-cpu-resource/) + +### For cluster administrators + +* [Configure Default Memory Requests and Limits for a Namespace](/docs/tasks/administer-cluster/manage-resources/memory-default-namespace/) + +* [Configure Default CPU Requests and Limits for a Namespace](/docs/tasks/administer-cluster/manage-resources/cpu-default-namespace/) + +* [Configure Minimum and Maximum Memory Constraints for a Namespace](/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace/) + +* [Configure Minimum and Maximum CPU Constraints for a Namespace](/docs/tasks/administer-cluster/manage-resources/cpu-constraint-namespace/) + +* [Configure Memory and CPU Quotas for a Namespace](/docs/tasks/administer-cluster/manage-resources/quota-memory-cpu-namespace/) diff --git a/content/en/docs/tasks/configure-pod-container/configure-service-account.md b/content/en/docs/tasks/configure-pod-container/configure-service-account.md index 987136f343121..ee8ce93e51455 100644 --- a/content/en/docs/tasks/configure-pod-container/configure-service-account.md +++ b/content/en/docs/tasks/configure-pod-container/configure-service-account.md @@ -192,8 +192,7 @@ token might be shorter, or could even be longer). {{< feature-state feature_gate_name="ServiceAccountTokenNodeBinding" >}} -When the `ServiceAccountTokenNodeBinding` and `ServiceAccountTokenNodeBindingValidation` -features are enabled, and using `kubectl` v1.31 or later, it is possible to create a service +Using `kubectl` v1.31 or later, it is possible to create a service account token that is directly bound to a Node: ```shell @@ -437,10 +436,10 @@ The JSON payload of this token follows a well defined schema - an example payloa "exp": 1731613413, "iat": 1700077413, "iss": "https://kubernetes.default.svc", # matches the first value passed to the --service-account-issuer flag - "jti": "ea28ed49-2e11-4280-9ec5-bc3d1d84661a", # ServiceAccountTokenJTI feature must be enabled for the claim to be present + "jti": "ea28ed49-2e11-4280-9ec5-bc3d1d84661a", "kubernetes.io": { "namespace": "kube-system", - "node": { # ServiceAccountTokenPodNodeInfo feature must be enabled for the API server to add this node reference claim + "node": { "name": "127.0.0.1", "uid": "58456cb0-dd00-45ed-b797-5578fdceaced" }, diff --git a/content/en/docs/tasks/configure-pod-container/resize-container-resources.md b/content/en/docs/tasks/configure-pod-container/resize-container-resources.md index 8390605fd267d..b8e45a92a1392 100644 --- a/content/en/docs/tasks/configure-pod-container/resize-container-resources.md +++ b/content/en/docs/tasks/configure-pod-container/resize-container-resources.md @@ -24,26 +24,33 @@ to be enabled. The alternative is to delete the Pod and let the [workload controller](/docs/concepts/workloads/controllers/) make a replacement Pod that has a different resource requirement. +A resize request is made through the pod `/resize` subresource, which takes the full updated pod for +an update request, or a patch on the pod object for a patch request. + For in-place resize of pod resources: -- Container's resource `requests` and `limits` are _mutable_ for CPU - and memory resources. -- `allocatedResources` field in `containerStatuses` of the Pod's status reflects - the resources allocated to the pod's containers. -- `resources` field in `containerStatuses` of the Pod's status reflects the - actual resource `requests` and `limits` that are configured on the running - containers as reported by the container runtime. -- `resize` field in the Pod's status shows the status of the last requested +- A container's resource `requests` and `limits` are _mutable_ for CPU + and memory resources. These fields represent the _desired_ resources for the container. +- The `resources` field in `containerStatuses` of the Pod's status reflects the resources + _allocated_ to the pod's containers. For running containers, this reflects the actual resource + `requests` and `limits` that are configured as reported by the container runtime. For non-running + containers, these are the resources allocated for the container when it starts. +- The `resize` field in the Pod's status shows the status of the last requested pending resize. It can have the following values: - - `Proposed`: This value indicates an acknowledgement of the requested resize - and that the request was validated and recorded. + - `Proposed`: This value indicates that a pod was resized, but the Kubelet has not yet processed + the resize. - `InProgress`: This value indicates that the node has accepted the resize request and is in the process of applying it to the pod's containers. - `Deferred`: This value means that the requested resize cannot be granted at this time, and the node will keep retrying. The resize may be granted when - other pods leave and free up node resources. + other pods are removed and free up node resources. - `Infeasible`: is a signal that the node cannot accommodate the requested resize. This can happen if the requested resize exceeds the maximum resources the node can ever allocate for a pod. + - `""`: An empty or unset value indicates that the last resize completed. This should only be the + case if the resources in the container spec match the resources in the container status. + +If a node has pods with an incomplete resize, the scheduler will compute the pod requests from the +maximum of a container's desired resource requests, and it's actual requests reported in the status. ## {{% heading "prerequisites" %}} @@ -107,6 +114,21 @@ have changed, the container will be restarted in order to resize its memory. +## Limitations + +In-place resize of pod resources currently has the following limitations: + +- Only CPU and memory resources can be changed. +- Pod QoS Class cannot change. This means that requests must continue to equal limits for Guaranteed + pods, Burstable pods cannot set requests and limits to be equal for both CPU & memory, and you + cannot add resource requirements to Best Effort pods. +- Init containers and Ephemeral Containers cannot be resized. +- Resource requests and limits cannot be removed once set. +- A container's memory limit may not be reduced below its usage. If a request puts a container in + this state, the resize status will remain in `InProgress` until the desired memory limit becomes + feasible. +- Windows pods cannot be resized. + ## Create a pod with resource requests and limits @@ -159,9 +181,6 @@ spec: name: qos-demo-ctr-5 ready: true ... - allocatedResources: - cpu: 700m - memory: 200Mi resources: limits: cpu: 700m @@ -190,7 +209,7 @@ resources, you cannot change the QoS class in which the Pod was created. Now, patch the Pod's Container with CPU requests & limits both set to `800m`: ```shell -kubectl -n qos-example patch pod qos-demo-5 --patch '{"spec":{"containers":[{"name":"qos-demo-ctr-5", "resources":{"requests":{"cpu":"800m"}, "limits":{"cpu":"800m"}}}]}}' +kubectl -n qos-example patch pod qos-demo-5 --subresource resize --patch '{"spec":{"containers":[{"name":"qos-demo-ctr-5", "resources":{"requests":{"cpu":"800m"}, "limits":{"cpu":"800m"}}}]}}' ``` Query the Pod's detailed information after the Pod has been patched. @@ -215,9 +234,6 @@ spec: ... containerStatuses: ... - allocatedResources: - cpu: 800m - memory: 200Mi resources: limits: cpu: 800m @@ -229,12 +245,9 @@ spec: started: true ``` -Observe that the `allocatedResources` values have been updated to reflect the new -desired CPU requests. This indicates that node was able to accommodate the -increased CPU resource needs. - -In the Container's status, updated CPU resource values shows that new CPU -resources have been applied. The Container's `restartCount` remains unchanged, +Observe that the `resources` in the `containerStatuses` have been updated to reflect the new desired +CPU requests. This indicates that node was able to accommodate the increased CPU resource needs, +and the new CPU resources have been applied. The Container's `restartCount` remains unchanged, indicating that container's CPU resources were resized without restarting the container. @@ -256,6 +269,8 @@ kubectl delete namespace qos-example * [Assign CPU Resources to Containers and Pods](/docs/tasks/configure-pod-container/assign-cpu-resource/) +* [Assign Pod-level CPU and memory resources](/docs/tasks/configure-pod-container/assign-pod-level-resources/) + ### For cluster administrators * [Configure Default Memory Requests and Limits for a Namespace](/docs/tasks/administer-cluster/manage-resources/memory-default-namespace/) diff --git a/content/en/docs/tasks/configure-pod-container/security-context.md b/content/en/docs/tasks/configure-pod-container/security-context.md index 142f3a63fd490..8bd070f9c39e7 100644 --- a/content/en/docs/tasks/configure-pod-container/security-context.md +++ b/content/en/docs/tasks/configure-pod-container/security-context.md @@ -677,8 +677,8 @@ To assign SELinux labels, the SELinux security module must be loaded on the host Kubernetes v1.27 introduced an early limited form of this behavior that was only applicable to volumes (and PersistentVolumeClaims) using the `ReadWriteOncePod` access mode. -As an alpha feature, you can enable the `SELinuxMount` -[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) to widen that +As an alpha feature, you can enable the `SELinuxMount` and `SELinuxChangePolicy` +[feature gates](/docs/reference/command-line-tools-reference/feature-gates/) to widen that performance improvement to other kinds of PersistentVolumeClaims, as explained in detail below. {{< /note >}} @@ -694,7 +694,9 @@ To benefit from this speedup, all these conditions must be met: and `SELinuxMountReadWriteOncePod` must be enabled. * Pod must use PersistentVolumeClaim with applicable `accessModes` and [feature gates](/docs/reference/command-line-tools-reference/feature-gates/): * Either the volume has `accessModes: ["ReadWriteOncePod"]`, and feature gate `SELinuxMountReadWriteOncePod` is enabled. - * Or the volume can use any other access modes and both feature gates `SELinuxMountReadWriteOncePod` and `SELinuxMount` must be enabled. + * Or the volume can use any other access modes and both feature gates + `SELinuxMountReadWriteOncePod`, `SELinuxChangePolicy` and `SELinuxMount` must be enabled + and the Pod has `spec.securityContext.seLinuxChangePolicy` either nil (default) or `MountOption`. * Pod (or all its Containers that use the PersistentVolumeClaim) must have `seLinuxOptions` set. * The corresponding PersistentVolume must be either: @@ -706,7 +708,52 @@ To benefit from this speedup, all these conditions must be met: For any other volume types, SELinux relabelling happens another way: the container runtime recursively changes the SELinux label for all inodes (files and directories) in the volume. -The more files and directories in the volume, the longer that relabelling takes. + +{{< feature-state feature_gate_name="SELinuxChangePolicy" >}} +For Pods that want to opt-out from relabeling using mount options, they can set +`spec.securityContext.seLinuxChangePolicy` to `Recursive`. This is required +when multiple pods share a single volume on the same node, but they run with +different SELinux labels that allows simultaneous access to the volume. For example, a privileged pod +running with label `spc_t` and an unprivileged pod running with the default label `container_file_t`. +With unset `spec.securityContext.seLinuxChangePolicy` (or with the default value `MountOption`), +only one of such pods is able to run on a node, the other one gets ContainerCreating with error +`conflicting SELinux labels of volume :