feat: set up gen ai inference attributes for foundational text models in java v1 sdk #19
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of changes:
Adding auto-instrumentation support for GenAI inference parameters for Java V1 AWS SDK.
The following foundational text models are supported:
Full list can be found here.
Note: Removed support for old Cohere Command models since they already throw a 404 response for Java V1.
Note: There is a backwards compatibility issue with the existing tests as they rely on older versions of the AWS SDK. Since Bedrock is newer resource we are forced to introduce a newer SDK version which implicitly adds in behavior that is not backwards compatible with the existing tests. It seems these dependency issues in the tests are a known issue in upstream which is why they rely on such old AWS resource versions in the first place. Synced with @mxiamxia and we decided to temporarily disable these tests for now as we figure out a long-term solution. We rely on the contract tests to verify the SDK quality for the time being.
New inference parameter attributes added according to OpenTelemetry Semantic Conventions for GenAI attributes:
gen_ai.request.max_tokens
gen_ai.request.temperature
gen_ai.request.top_p
gen_ai.response.finish_reasons
gen_ai.usage.input_tokens
gen_ai.usage.output_tokens
Test Plan:
Set up sample app to make Bedrock Runtime
InvokeModel
API calls to the supported foundational models and verified the auto-instrumentation attributes.AI21 Jamba
Amazon Titan
Anthropic Claude
Cohere Command
Meta Llama
Mistral AI