Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to detect if an HttpRequest has been aborted/cancelled/timed out in synchronous code #11012

Open
mikehearn opened this issue Jul 24, 2024 · 0 comments
Labels
type: enhancement New feature or request

Comments

@mikehearn
Copy link
Contributor

Feature description

Some HTTP requests may be expensive to compute the answer to. Consider LLM inferencing as an example. If a client disconnects or sends an HTTP/2 RST_STREAM cancellation message whilst a request is being processed, there doesn't seem to be any way to detect that this has happened whilst the controller method is executing.

Although it may be possible to do this using reactive streams, that isn't appropriate for many kinds of code that aren't reactive in nature. For example, llama3.java does CPU inferencing in a tight inner loop. There's no obvious way to make that 'reactive' and no reason why it should be. On the other hand, doing a callback or cancellation check during the inner loop would be easy.

Java has Thread.interrupt() to signal to a thread that it should cancel in a controlled manner. When Micronaut is combined with virtual threads, this would be a natural thing to do in response to an HTTP client timing out or disconnecting.

@graemerocher graemerocher added the type: enhancement New feature or request label Aug 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants