Skip to content

Include reasoning tokens and token streaming for anthropic LLMs #3079

@damithsenanayake

Description

@damithsenanayake

Missing Thoughts and Unable to Token Stream with Claude Models
To leverage models put forward by Anthropic through model garden, we utilise the anthropic_llm.py where Claude models are defined. However, this does not allow for token streaming or forwarding reasoning tokens to the UI.

Solution
Implement:

  • parsing Google GenAI's Thinking Config into Anthropic's Thinking Config to be passed along to anthropic API
  • processing the anthropic API messages into response parts compatible with ADK
  • allow for custom configurations (e.g., beta headers) in Anthropic API

Alternatives

Proposed Fix
PR created #3070

Metadata

Metadata

Assignees

No one assigned

    Labels

    bot triaged[Bot] This issue is triaged by ADK botmodels[Component] Issues related to model support

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions