Support draining DaemonSet pods using sriov devices #840
Conversation
|
Thanks for your PR,
To skip the vendors CIs, Maintainers can use one of:
|
Pull Request Test Coverage Report for Build 13370907002Details
💛 - Coveralls |
pkg/drain/drainer.go
Outdated
| // remove pods that are owned by a DaemonSet and use SR-IOV devices | ||
| dsPodsList := getDsPodsToRemove(podList) | ||
| for _, pod := range dsPodsList { | ||
| err = d.kubeClient.CoreV1().Pods(pod.Namespace).Delete(ctx, pod.Name, metav1.DeleteOptions{}) |
There was a problem hiding this comment.
Shouldn't we block? Waiting to ensure that the pod is fully removed before continuing? A pod that's slow to delete might cause a race condition
There was a problem hiding this comment.
yes I switch to use the drain from the kubernetes drain helper it does that :)
SchSeba
force-pushed
the
drain_daemon
branch
2 times, most recently
from
2140f89 to
c20fba0
Compare
pkg/drain/drainer.go
Outdated
|
|
||
| // on full drain there is no need to try and remove pods that are owned by DaemonSets | ||
| // as we are going to reboot the node in any case. | ||
| if fullNodeDrain { |
There was a problem hiding this comment.
do we care ? (just thinking how to simplify) why not always remove DS pods from node IF they have sriov resources ?
with this commit we also take care of removing daemonset owned pods using sriov devices. we only do it when drain is requested we don't do it for reboot requests Signed-off-by: Sebastian Sch <[email protected]>
SchSeba
force-pushed
the
drain_daemon
branch
from
c20fba0 to
cd2fb78
Compare
| @@ -2376,6 +2538,18 @@ func waitForPodRunning(p *corev1.Pod) *corev1.Pod { | |||
| return ret | |||
| } | |||
|
|
|||
| func waitForDaemonReady(d *appsv1.DaemonSet) *appsv1.DaemonSet { | |||
There was a problem hiding this comment.
nit: waitForDaemonSetReady
| reqLogger.Info("drainNode(): Draining failed, retrying", "error", err) | ||
| return false, nil | ||
|
|
||
| err = d.removeDaemonSetsFromNode(ctx, node.Name) |
There was a problem hiding this comment.
I'm not entirely clear on how this works. I understand the process of selecting and removing DS pods, but I'm unsure how we prevent the DS controller from restarting pods on the node we intend to drain. Could you clarify?
with this commit we also take care of removing DaemonSet owned pods using sriov devices.
we only do it when drain is requested we don't do it for reboot requests