Skip to content

Commit

Permalink
Account that k8s pods may be uncounted yet.
Browse files Browse the repository at this point in the history
  • Loading branch information
mapk-amazon committed Oct 15, 2024
1 parent 4ce5bca commit f8c8001
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion lib/galaxy/jobs/runners/kubernetes.py
Original file line number Diff line number Diff line change
Expand Up @@ -772,7 +772,10 @@ def check_watched_item(self, job_state):
# as probably this means that the k8s API server hasn't
# had time to fill in the object status since the
# job was created only too recently.
if len(job.obj["status"]) == 0:
# It is possible that k8s didn't account for the status of the pods
# and they are in the uncountedTerminatedPods status. In this
# case we also need to wait a moment
if len(job.obj["status"]) == 0 or 'uncountedTerminatedPods' in job.obj["status"]:
return job_state
if "succeeded" in job.obj["status"]:
succeeded = job.obj["status"]["succeeded"]
Expand Down

0 comments on commit f8c8001

Please sign in to comment.