Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Out of band communication to report slot status #606

Open
philwinder opened this issue Nov 22, 2024 · 0 comments
Open

Out of band communication to report slot status #606

philwinder opened this issue Nov 22, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@philwinder
Copy link
Contributor

At the moment the controlplane has to hack into the various pubsub receivers to know that a slot has actually started work. E.g. a slot only knows it has started when the first byte is received in the openai chat handler on the control plane.

This isn't ideal because we've had instances where this doesn't happen for some reason. E.g. a runner was shut down. So the slot just times out.

We should have a better mechanism for reporting the actual state of not only slots, but also work on those slots. This might be considered a part of #605.

@philwinder philwinder added the enhancement New feature or request label Nov 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant