You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
At the moment the controlplane has to hack into the various pubsub receivers to know that a slot has actually started work. E.g. a slot only knows it has started when the first byte is received in the openai chat handler on the control plane.
This isn't ideal because we've had instances where this doesn't happen for some reason. E.g. a runner was shut down. So the slot just times out.
We should have a better mechanism for reporting the actual state of not only slots, but also work on those slots. This might be considered a part of #605.
The text was updated successfully, but these errors were encountered:
At the moment the controlplane has to hack into the various pubsub receivers to know that a slot has actually started work. E.g. a slot only knows it has started when the first byte is received in the openai chat handler on the control plane.
This isn't ideal because we've had instances where this doesn't happen for some reason. E.g. a runner was shut down. So the slot just times out.
We should have a better mechanism for reporting the actual state of not only slots, but also work on those slots. This might be considered a part of #605.
The text was updated successfully, but these errors were encountered: