DaemonSet, Add Priority class (#172)

As part of https://bugzilla.redhat.com/show_bug.cgi?id=1953482
We are adding Priority Class [1] to each network component.

The motivation is to make the control plane pods less sensitive to preemption
than user workloads.

Pods that are node specific will have the higher build-in priority,
since preempting them from a specific node, makes them unavailable
until they are rescheduled on that specific node.
Pods that are network control plane, but are not node specific
will have system-cluster-critical, which would still make them more important
than user and custom workloads, but less than system-node-critical.

Since ovs-cni pod should run on each node,
assign system-node-critical pc to it.

[1] https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/

Signed-off-by: Or Shoval <[email protected]>

examples/ovs-cni.yml

@@ -36,6 +36,7 @@ spec:
         volumeMounts:
         - name: cnibin
           mountPath: /host/opt/cni/bin
+      priorityClassName: system-node-critical
       containers:
       - name: ovs-cni-marker
         image: quay.io/kubevirt/ovs-cni-marker:latest

manifests/ovs-cni.yml.in

@@ -36,6 +36,7 @@ spec:
         volumeMounts:
         - name: cnibin
           mountPath: /host${CNI_MOUNT_PATH}
+      priorityClassName: system-node-critical
       containers:
       - name: ovs-cni-marker
         image: ${OVS_CNI_MARKER_IMAGE_REPO}/${OVS_CNI_MARKER_IMAGE_NAME}:${OVS_CNI_MARKER_IMAGE_VERSION}