Usage tracking lost when streaming fails mid-request

[habema]

Summary

When RunResultStreaming raises an error (API errors, connection drops, context window exceeded, etc.), result.context_wrapper.usage stays at 0 tokens even though tokens were consumed.

Root Cause

Usage is only accumulated when ResponseCompletedEvent arrives:

# src/agents/run.py, line 1245
async for event in model.stream_response(...):
    if isinstance(event, ResponseCompletedEvent):
        usage = Usage(...)
        context_wrapper.usage.add(usage)  # line 1263

If the model provider raises an exception before yielding ResponseCompletedEvent (which happens for context window errors, mid-stream connection failures, rate limits, etc.), the loop exits without ever updating usage.

Fix Ideas

1. Estimate input tokens on error: When streaming fails, estimate input tokens from the request we sent (we have filtered.input + filtered.instructions). Mark it as estimated via a flag. Output tokens are lost but at least we track what went in.

I'd love to hear about whether this fix idea is valid and welcome before going ahead with implementing it. If you can think of any workarounds temporarily, that would also be great.

[seratch]

Thanks for sharing this issue. Estimating the token usage from request data could be an interesting idea, but it won't be accurate for many situations. I think the ideal solution would be receiving current usage data from the OpenAI server periodically. I am unable to guarantee this enhancement will come, but I will share this feedback with the server engineering team.

[habema]

The server limitation here is a pretty complex one. It boils down to the fact that that usage blocks are only exposed in repsonse.completed events. I don't think this is something that could feasibly change at the API level.

Maybe the word estimate is not the best way to describe it. Until any point of failure that could happen before ResponseCompletedEvent, we can track input tokens dependably. As for output tokens, there are two options:

• Not track the failed turn's output tokens.
• Locally tokenize and keep count of the number of output tokens between every ResponseCompletedEvent and the next.

Let me know if either of these are valid and welcome contributions. If not, I'm afraid I will have to make those changes in my own fork. This is a pretty severe limitation in a production usecase.

[seratch]

Thanks for clarifying this. Yeah, if the implementation is relevant enough, the first approach should be good to go.