Add support for extended thinking in Anthropic's models via Bedrock's Converse API

[lcfyi]

Title

Add support for extended thinking in Anthropic's models via Bedrock's Converse API

Relevant issues

I'm hesitant to say this PR fixes that issue, because the original reporter is using a bedrock/ model which should route to the Invoke API which handles the thinking blocks properly. Specifically, I wasn't able to reproduce the issue they were seeing when using bedrock/us.anthropic.claude-sonnet-4-20250514-v1:0 and bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0 as the model.

However, I was able to reproduce their issue with bedrock/converse/us.anthropic.claude-sonnet-4-20250514-v1:0 and bedrock/converse/us.anthropic.claude-sonnet-4-5-20250929-v1:0, where fixing that was the main motivation for this PR. When running the litellm proxy on b348a26, I get the error that reporter saw:

➜ poetry run python3 repro.py

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
...
litellm.exceptions.InternalServerError: litellm.InternalServerError: AnthropicException - {"error":{"message":"Error calling litellm.acompletion for non-Anthropic model: litellm.BadRequestError: BedrockException - {\"message\":\"The model returned the following errors: messages.1.content.0.type: Expected `thinking` or `redacted_thinking`, but found `text`. When `thinking` is enabled, a final `assistant` message must start with a thinking block (preceeding the lastmost set of `tool_use` and `tool_result` blocks). We recommend you include thinking blocks from previous turns. To avoid this requirement, disable `thinking`. Please consult our documentation at https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking\"}","type":"None","param":"None","code":"500"}}. Handle with `litellm.InternalServerError`.

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• I have added a screenshot of my new test passing locally

• My PR passes all unit tests on make test-unit

This is not passing. However, the change is unrelated to my code:

=============================================================================== short test summary info ================================================================================
FAILED tests/test_litellm/llms/gemini/test_gemini_common_utils.py::TestGoogleAIStudioTokenCounter::test_clean_contents_for_gemini_api_preserves_other_fields - ModuleNotFoundError: No module named 'google.genai'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! xdist.dsession.Interrupted: stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
========================================================= 1 failed, 1160 passed, 12 skipped, 82 warnings in 694.44s (0:11:34) ==========================================================

• My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🐛 Bug Fix
✅ Test

Changes

This PR adds support for extended thinking in Anthropic's models, building upon the thinking_blocks that's already parsed out from the Converse response here:

litellm/litellm/llms/bedrock/chat/converse_transformation.py

Lines 1321 to 1330 in b348a26

	if reasoningContentBlocks is not None:
	chat_completion_message["provider_specific_fields"] = {
	"reasoningContentBlocks": reasoningContentBlocks,
	}
	chat_completion_message["reasoning_content"] = (
	self._transform_reasoning_content(reasoningContentBlocks)
	)
	chat_completion_message["thinking_blocks"] = (
	self._transform_thinking_blocks(reasoningContentBlocks)
	)

This is mostly a mechanical parsing change, and is intended for downstream providers that faithfully return thinking_blocks that Anthropic clients expect. As such, it ignores reasoning_content because it doesn't return a signature (which many Anthropic clients, such as Claude Code, validate and require).

Extended test plan

Validation with the reproduction script in #14194

I also confirmed that I can no longer reproduce the issue with the repro script in the linked issue:

➜ poetry run python3 repro.py
ModelResponse(id='chatcmpl-54a7744a-9746-4679-a2c2-4fd735b46f3d', created=1759658928, model='us.anthropic.claude-sonnet-4-20250514-v1:0', object='chat.completion', system_fingerprint=None, choices=[Choices(finish_reason='tool_calls', index=0, message=Message(content=None, role='assistant', tool_calls=[ChatCompletionMessageToolCall(index=0, function=Function(arguments='{"thought": "The user wants me to help implement necessary changes to make the AnnotationBbox gid work in SVG output. I\'ve been working on this for a while and tried several approaches.\\n\\nLooking at the problem:\\n1. The issue is that when setting a gid on an AnnotationBbox, it doesn\'t appear in the SVG output\\n2. The fix involves two parts:\\n   - Making AnnotationBbox.draw pass its gid to the underlying OffsetImage and BboxImage\\n   - Making the SVG backend respect the gid when drawing images\\n\\nFrom my analysis, I\'ve identified that:\\n1. AnnotationBbox has a draw method that calls self.offsetbox.draw(renderer)\\n2. OffsetImage (which is the offsetbox) has a draw method that calls self.image.draw(renderer) \\n3. BboxImage (which is the image) has a draw method that sets gc.set_gid(self.get_gid()) and calls renderer.draw_image(gc, ...)\\n4. The SVG backend\'s draw_image method gets the gid from gc.get_gid() and uses it as the image ID\\n\\nThe problem is that the gid is set on AnnotationBbox but never propagated down to the BboxImage.\\n\\nI\'ve made changes to:\\n1. AnnotationBbox.draw to pass the gid to offsetbox and offsetbox.image\\n2. The SVG backend to properly use the gid when provided\\n\\nHowever, when I test the changes, they don\'t seem to be taking effect. This might be because:\\n1. The changes aren\'t being applied correctly\\n2. There\'s some caching or import issue\\n3. The methods are being wrapped by decorators\\n\\nLet me create a final test that hot-patches the methods at runtime to verify the fix works."}', name='think'), id='tooluse_rBOm_l8jQeWaZDo0WrLyfg', type='function')], function_call=None, provider_specific_fields={'citations': None, 'thinking_blocks': None}))], usage=Usage(completion_tokens=460, prompt_tokens=74772, total_tokens=75232, completion_tokens_details=None, prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=None, cached_tokens=0, text_tokens=None, image_tokens=None, cache_creation_tokens=0), cache_creation_input_tokens=0, cache_read_input_tokens=0))

Validation with a tool (Claude Code)

With this litellm config:

model_list:
  - model_name: claude-sonnet-4-20250514
    litellm_params:
      model: bedrock/converse/us.anthropic.claude-sonnet-4-20250514-v1:0
      aws_region_name: "us-west-2"

Before

 ▐▛███▜▌   Claude Code v2.0.8
▝▜█████▛▘  Sonnet 4 · API Usage Billing
  ▘▘ ▝▝    /Users/lcfyi/Development/misc/litellm-server

> what files are in my curent directory?

⏺ Bash(ls -la)
  ⎿  total 8
     drwxr-xr-x   4 lcfyi  staff   128 Oct  4 20:21 .
     drwxr-xr-x  77 lcfyi  staff  2464 Oct  4 20:03 ..
     drwxr-xr-x   7 lcfyi  staff   224 Oct  4 20:04 .venv
     -rw-r--r--   1 lcfyi  staff   862 Oct  5 03:03 config.yaml
  ⎿  500 {"error":{"message":"Error calling litellm.acompletion for non-Anthropic model:
     litellm.BadRequestError: BedrockException - {\"message\":\"The model returned the
     following errors: messages.1.content.0.type: Expected `thinking` or
     `redacted_thinking`, but found `text`. When `thinking` is enabled, a final `assistant`
      message must start with a thinking block (preceeding the lastmost set of `tool_use`
     and `tool_result` blocks). We recommend you include thinking blocks from previous
     turns. To avoid this requirement, disable `thinking`. Please consult our documentation
      at https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking\"}","type":
     "None","param":"None","code":"500"}}
     Retrying in 18 seconds… (attempt 6/10)

──────────────────────────────────────────────────────────────────────────────────────────
  Showing detailed transcript · Ctrl+O to toggle

The tool call succeeds, but the follow-up fails because we aren't passing in thinking blocks that would've been generated with the tool call.

After

 ▐▛███▜▌   Claude Code v2.0.8
▝▜█████▛▘  Sonnet 4 · API Usage Billing
  ▘▘ ▝▝    /Users/lcfyi/Development/misc/litellm-server


> what files are in my current directory?

∴ Thinking…

  The user is asking what files are in their current directory. This is a simple task to
  list files in the current directory. I should use the Bash tool to run ls to list the
  files.

  This is a straightforward request that doesn't require a todo list since it's just a
  single simple command to list directory contents.

⏺ Bash(ls)
  ⎿  config.yaml

⏺ There is one file  03:27 AM  converse/us.anthropic.claude-sonnet-4-20250514-v1:0
  in your current
  directory:
  config.yaml

──────────────────────────────────────────────────────────────────────────────────────────
  Showing detailed transcript · Ctrl+O to toggle

[vercel]

@lcfyi is attempting to deploy a commit to the CLERKIEAI Team on Vercel.

A member of the Team first needs to authorize it.

[krrishdholakia]

your code modifies the anthropic v1 messages api.

The error - is on the bedrock/converse_transformation for /v1/chat/completions AFAIK

[lcfyi]

Yeah, that was the API I was looking to fix initially (such that litellm works for Claude Code), so it's not fully related to the linked issue.

This change should fix models that are explicitly routing to converse though.

[Sameerlite]

Hi @krrishdholakia , This PR partially fixes #14194. This only takes care of streaming cases. The one problem people in that issue are facing is when they use Claude code with http://localhost:4004/v1/messages they don't get back the thinking blocks in the streaming response. Which in turn makes Claude send a request without thinking block and it hence throws an error. So this is taking care of this case.

I am raising a PR which will take care of the other issues. But this LGTM

[Sameerlite]

LGTM