You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When using AWS Bedrock inference profiles, which are formatted as: arn:aws:bedrock:us-east-1:<account_id>:application-inference-profile/<profile_id>, kong is unable to route the request.
I have experience two behaviors:
if the ai-proxy is configured with the unescaped ARN, Kong fails with:
{"error":{"message":"transformation failed from type bedrock://llm/v1/chat: no generation candidates received from Bedrock, or max_tokens too short"}}
when configured with the escaped one (arn%3Aaws%3Abedrock%3Aus-east-1%3A<account_id>%3Aapplication-inference-profile%2F<profile_id>), Kong fails with:
{"message":"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method."}
Expected Behavior
Kong should correctly format and escape the ARN when generating the SigV4 signature and constructing the request URL to match AWS’s expected format.
The Converse API requires the URL to be formatted as:
/model/<model-id/inference-profile>/converse
The AWS CLI (in debug mode) shows that this URL is then encoded when making the request to Bedrock.
Steps To Reproduce
Create the plugin with the unescaped model_name ARN
I did check the aws cli with the inference profile to rule out authentication errors with the inference profile, and the profile works using the converse endpoint.
I verified the URL behavior by enabling the File Log plugin on /dev/stdout.
The text was updated successfully, but these errors were encountered:
While going through the source code I noticed that the plugin allows for setting and overriding the upstream_path. I did try that, but the overridden path is then used for the sigv4 authentication.
Another possible way would be to allow the inference profile by recommending the model.options.upstream_path edits, and by adding ngx.unescape_uri(parsed_url.path) in the authentication process.
muscionig
changed the title
Bug: AI-Proxy Does Not Correctly Escape AWS Bedrock Inference Profile ARN in Signature
Bug: ai-proxy does not correctly escape AWS Bedrock inference profile ARN in signature and request
Feb 25, 2025
Hey @muscionig thanks so much for this, I actually hadn't seen that you can just specify the model ARN directly for Bedrock-converse, I thought it was just for InvokeModel, so I didn't check it. But yeah I had this same problem with "upstream_path" which is:
fixed but
getting removed soon, the field is not needed with the next release
I will either ask if your fix can be merged directly, or I will bundle it into the next 3.10 main and tag you as contributor (because we have a giant PR of fixes coming already, it's quicker).
Hi @tysoekong, thanks for the update and for considering my changes! I’m totally fine with the fix being integrated into a different PR and with the collaborator approach.
Is the PR public? I’d love to test it in the meantime to ensure everything works as expected.
Is there an existing issue for this?
Kong version (
$ kong version
)Kong 3.9.0
Current Behavior
When using AWS Bedrock inference profiles, which are formatted as:
arn:aws:bedrock:us-east-1:<account_id>:application-inference-profile/<profile_id>
,kong
is unable to route the request.I have experience two behaviors:
ai-proxy
is configured with the unescaped ARN, Kong fails with:arn%3Aaws%3Abedrock%3Aus-east-1%3A<account_id>%3Aapplication-inference-profile%2F<profile_id>
), Kong fails with:Expected Behavior
Kong should correctly format and escape the ARN when generating the SigV4 signature and constructing the request URL to match AWS’s expected format.
Converse
API requires the URL to be formatted as:Steps To Reproduce
Create the plugin with the unescaped
model_name
ARNCreate the plugin with the escaped
model_name
ARNsigv4
error.Anything else?
aws cli
with the inference profile to rule out authentication errors with the inference profile, and the profile works using the converse endpoint./dev/stdout
.The text was updated successfully, but these errors were encountered: