Fixes #1582: Handling of structured CompletionUsage response values for token usage #1583
+17
−4
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR resolves a compatibility issue caused by recent changes in the Python SDK. Previously, token usage data (
completion_tokens
,prompt_tokens
,total_tokens
) was returned as integers. Recent API updates introduced structuredCompletionUsage
objects, resulting in type errors after eval execution.Final checklist 👀
Submission agreement
pip install pre-commit; pre-commit install
and have verified thatmypy
,black
,isort
,autoflake
andruff
are running when I commit and push