[Feature]: OpenAI Responses API Support

[krrishdholakia]

The Feature

Parent ticket to track support for new OpenAI responses API

Based on discord, this will be supported as a separate api spec, instead of trying to translate into /chat/completions. https://discord.com/channels/1123360753068540065/1139937429588021289/1349086515665305671

Checklist

Checklist for create responses endpoint

• Non-streaming Async
• Non-streaming Sync
• Streaming Async
• Streaming Sync
• Non-streaming logging + cost tracking
• Streaming logging + cost tracking
• litellm.router support

Proxy Checklist

• post /responses OpenAI non-streaming
• post /responses OpenAI non-streaming - logging + cost tracking
• post /responses OpenAI streaming
• post /responses OpenAI streaming - logging + cost tracking
• Get Response get https://api.openai.com/v1/responses/{response_id}
• Delete Response delete https://api.openai.com/v1/responses/{response_id}
• List Input items get https://api.openai.com/v1/responses/{response_id}/input_items

Motivation, pitch

Make it easy to give devs llm access

Are you a ML Ops Team?

No

Twitter / LinkedIn details

No response

[mvrodrig]

Great news!

[ishaan-jaff]

Initial PR here: #9155

[taralika]

would be great to get this for Azure as well given: https://azure.microsoft.com/en-us/blog/announcing-the-responses-api-and-computer-using-agent-in-azure-ai-foundry/

[krrishdholakia]

Hey @taralika our account doesn't have access. Does yours?

[ishaan-jaff]

OpenaI responses API on LiteLLM Proxy here: #9183

[ishaan-jaff]

@taralika - can you share how you deploy Responses API on Azure AI foundry ? I don't see it

[taralika]

yeah I don't see it either.. I missed that the article says "available in few weeks" 🤦‍♂️

[ishaan-jaff]

ok, will wait for support from Azure API. Will add Day-0 support for it once out

[ishaan-jaff]

Responses API live here, can users on this issue thread help Beta test it (@mvrodrig @blairhudson, @jskalant)

Release: https://github.com/BerriAI/litellm/releases/tag/v1.63.8-nightly

Doc: https://docs.litellm.ai/docs/response_api

[yigitkonur]

you guys are crazy! big props!

[marty-sullivan]

@ishaan-jaff it looks like Responses is available in Azure now with the 2025-03-01-preview API. We're hoping to see this added to LiteLLM soon :)

https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/responses?tabs=python-secure

[OriNachum]

Does it support Bedrock as well? I saw on OpenAI that this opens Computer Use, and Bedrock also adds it.
So it will be a must to allow Computer Use with bedrock, for instance.

[ishaan-jaff]

@OriNachum adding support for Bedrock on this endpoint is on our roadmap too