You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Some HTTP requests may be expensive to compute the answer to. Consider LLM inferencing as an example. If a client disconnects or sends an HTTP/2 RST_STREAM cancellation message whilst a request is being processed, there doesn't seem to be any way to detect that this has happened whilst the controller method is executing.
Although it may be possible to do this using reactive streams, that isn't appropriate for many kinds of code that aren't reactive in nature. For example, llama3.java does CPU inferencing in a tight inner loop. There's no obvious way to make that 'reactive' and no reason why it should be. On the other hand, doing a callback or cancellation check during the inner loop would be easy.
Java has Thread.interrupt() to signal to a thread that it should cancel in a controlled manner. When Micronaut is combined with virtual threads, this would be a natural thing to do in response to an HTTP client timing out or disconnecting.
The text was updated successfully, but these errors were encountered:
Feature description
Some HTTP requests may be expensive to compute the answer to. Consider LLM inferencing as an example. If a client disconnects or sends an HTTP/2
RST_STREAM
cancellation message whilst a request is being processed, there doesn't seem to be any way to detect that this has happened whilst the controller method is executing.Although it may be possible to do this using reactive streams, that isn't appropriate for many kinds of code that aren't reactive in nature. For example, llama3.java does CPU inferencing in a tight inner loop. There's no obvious way to make that 'reactive' and no reason why it should be. On the other hand, doing a callback or cancellation check during the inner loop would be easy.
Java has
Thread.interrupt()
to signal to a thread that it should cancel in a controlled manner. When Micronaut is combined with virtual threads, this would be a natural thing to do in response to an HTTP client timing out or disconnecting.The text was updated successfully, but these errors were encountered: