Amazon Bedrock support

[sandrich]

It would be amazing to enable Amazon Bedrock support, rather than OpenAI. This would allow to access many other models including Anthropic Claude.

I tried to setup Bedrock Access Gateway which essentially is an OpenAI API proxy which translates into Bedrock API calls

https://github.com/aws-samples/bedrock-access-gateway

It requires the env variables OPENAI_BASE_URL and OPENAI_API_KEY

Example query using curl against the proxy

curl $OPENAI_BASE_URL/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "us.anthropic.claude-3-5-sonnet-20241022-v2:0",
    "messages": [
      {
        "role": "user",
        "content": "Hello!"
      }
    ]
  }'
{"id":"chatcmpl-fb7f646e","created":1738607696,"model":"us.anthropic.claude-3-5-sonnet-20241022-v2:0","system_fingerprint":"fp","choices":[{"index":0,"finish_reason":"stop","logprobs":null,"message":{"role":"assistant","content":"Hi! How can I help you today?"}}],"object":"chat.completion","usage":{"prompt_tokens":9,"completion_tokens":12,"total_tokens":21}}