Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add warming code to the control plane #610

Open
philwinder opened this issue Nov 22, 2024 · 0 comments
Open

Add warming code to the control plane #610

philwinder opened this issue Nov 22, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@philwinder
Copy link
Contributor

Ideally the control plane should be responsible for warming. This means that runners can connect quickly, but marked as warming. This way you don't get into a state where runners are running but not communicating with the control plane because they are too busy pulling shit.

That way the control plane really knows who is ready for scheduling.

Could also assign slots for common models. So they're warm and ready to go as soon as they've stopped warming.

@philwinder philwinder added the enhancement New feature or request label Nov 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant