Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Job run failed to shutdown #311

Open
jhamman opened this issue Dec 9, 2024 · 2 comments
Open

Job run failed to shutdown #311

jhamman opened this issue Dec 9, 2024 · 2 comments

Comments

@jhamman
Copy link

jhamman commented Dec 9, 2024

I ran a job over the weekend (run-1b450464 in the earthmover-devs workspace). This job was started with --interactive and was terminated locally but kept running on coiled. Fortunately, I checked the dashboard on and saw that it was still running.

Is there someway to put a max lifetime or idle cull behavior on jobs like this?

@jrbourbeau
Copy link
Member

Thanks for the issue @jhamman (sorry for the slow response here).

was terminated

Joe mentioned offline that this was due to a blip in his network

@ntabris I'm wondering if you have any thoughts on this one?

@ntabris
Copy link
Member

ntabris commented Dec 16, 2024

Hm, we could have something on the VM handle when then the SSH session exits. Not sure how trivial/hard this would be.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants