Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Operator doesn't delete Evicted Pods. #464

Closed
Yagyansh opened this issue Aug 2, 2022 · 2 comments · Fixed by #465
Closed

Operator doesn't delete Evicted Pods. #464

Yagyansh opened this issue Aug 2, 2022 · 2 comments · Fixed by #465
Labels
question Further information is requested

Comments

@Yagyansh
Copy link
Contributor

Yagyansh commented Aug 2, 2022

Operator Version - v0.10.0

I'm using GHE.

When pods for a runner pool evict, the pods are successfully unregistered from Github but the pods aren't deleted and new pods aren't spun which leads to jobs being queued.
If I have 1 runner pod which gets evicted, then the CURRENTPOOLSIZE for the runner pool remains 1 which is why no new pod is spun.

I've already seen #232. This solves the evicted runners being unregistered, which is happening successfully but pods aren't being deleted.

@davidkarlsen
Copy link
Collaborator

@Yagyansh Evicted pods are a sign of something malfunctioning, hence k8s does not remove them automatically (in order to let the user detect the fact and understand why).

What does the logs of the operator say?

@davidkarlsen davidkarlsen added the question Further information is requested label Aug 2, 2022
@Yagyansh
Copy link
Contributor Author

Yagyansh commented Aug 2, 2022

@davidkarlsen

I understand that, but when the operator has successfully de-registered the pod why isn't a new pod(i.e runner) being created?

I don't see any errors when the operator unregister the evicted pod.

Shouldn't the evicted pods be handled gracefully by the operator after #232 fix?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants