Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,22 @@ documentation_complete: true

title: 'Ensure Control Plane / Master Nodes are Distributed Across Three Failure Zones'

description: |-
description: |-
Distributing Kubernetes control plane nodes across failure zones enhances security by mitigating
the risk of a single point of failure and reducing the impact of API inavailability or attacks
targeting a specific zone. Maintaining three instances of etcd across different failure zones
targeting a specific zone. Maintaining three instances of etcd across different failure zones
ensures fault tolerance and enables the system to reach quorum, thereby safeguarding critical data
integrity and ensuring continued operation even in the event of zone failures.

rationale: |-
Distributing Kubernetes control plane nodes across failure zones is crucial for enhancing overall
system resilience and security. By spreading control plane components across different zones,
the system becomes more fault-tolerant, reducing the risk of widespread outages due to failures or
attacks in a single zone. Having multiple instances of etcd spread across these zones ensures data
Distributing Kubernetes control plane nodes across failure zones is crucial for enhancing overall
system resilience and security. By spreading control plane components across different zones,
the system becomes more fault-tolerant, reducing the risk of widespread outages due to failures or
attacks in a single zone. Having multiple instances of etcd spread across these zones ensures data
integrity and availability, as it requires a quorum of nodes to reach consensus.
With three zones, Kubernetes can achieve a quorum with a simple majority (i.e., two out of three)
for critical components like etcd, ensuring system stability even if one zone fails.
Failure zones are marked on nodes using a well-known label called "topology.kubernetes.io/zone".
With three zones, Kubernetes can achieve a quorum with a simple majority (i.e., two out of three)
for critical components like etcd, ensuring system stability even if one zone fails.
Failure zones are marked on nodes using a well-known label called "topology.kubernetes.io/zone".
This label is automatically assigned to each node by cloud providers but might need to be managed
manually in other environments

Expand All @@ -26,6 +26,8 @@ identifiers:

severity: medium

platform: not ocp4-on-hypershift-hosted

ocil_clause: 'Kubernetes control plane not distributed across three failure zones'

ocil: |-
Expand Down
18 changes: 10 additions & 8 deletions applications/openshift/master/master_taint_noschedule/rule.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ documentation_complete: true

title: Verify that Control Plane Nodes are not schedulable for workloads

{{% set jqfilter = '[ .items[] | select(.spec.taints[]?.key == "node-role.kubernetes.io/master" and .spec.taints[]?.effect == "NoSchedule") | .metadata.name ]' %}}

description: -|
<p>
User workloads should not be colocated with control plane workloads. To ensure that the scheduler won't
Expand All @@ -25,22 +27,22 @@ rationale: -|
In some setups it might be necessary to make the control plane schedulable for workloads i.e.
Single Node Openshift (SNO) or Compact Cluster (Three Node Cluster) setups.

{{% set jqfilter = '.items[] | select(.metadata.labels."node-role.kubernetes.io/master" == "" or .metadata.labels."node-role.kubernetes.io/control-plane" == "" ) | .spec.taints[] | select(.key == "node-role.kubernetes.io/master" and .effect == "NoSchedule")' %}}

identifiers:
cce@ocp4: CCE-88731-5

severity: medium

platform: not ocp4-on-hypershift-hosted

ocil_clause: 'Control Plane is schedulable'

ocil: |-
Run the following command to see if control planes are schedulable
<pre>$oc get --raw /api/v1/nodes | jq '.items[] | select(.metadata.labels."node-role.kubernetes.io/master" == "" or .metadata.labels."node-role.kubernetes.io/control-plane" == "" ) | .spec.taints[] | select(.key == "node-role.kubernetes.io/master" and .effect == "NoSchedule" )'</pre>
for each master node, there should be an output of a key with the NoSchedule effect.
<pre>$oc get --raw /api/v1/nodes | jq '[ .items[] | select(.spec.taints[]?.key == "node-role.kubernetes.io/master" and .spec.taints[]?.effect == "NoSchedule") | .metadata.name ]'</pre>
for each non-schedulable master node, there should be the name in the output.

By editing the cluster scheduler you can centrally configure the masters as schedulable or not
by setting .spec.mastersSchedulable to true.
By editing the cluster scheduler you can centrally configure the masters as not schedulable
by setting .spec.mastersSchedulable to false.
Use <pre>$oc edit schedulers.config.openshift.io cluster</pre> to configure the scheduling.

warnings:
Expand All @@ -53,9 +55,9 @@ template:
ocp_data: "true"
filepath: |-
{{{ openshift_filtered_path('/api/v1/nodes', jqfilter) }}}
yamlpath: ".effect"
yamlpath: "[:]"
check_existence: "at_least_one_exists"
entity_check: "at least one"
values:
- value: "NoSchedule"
- value: "(.*?)"
operation: "pattern match"
Loading