Skip to content

Conversation

sonianuj287
Copy link

@sonianuj287 sonianuj287 commented Oct 7, 2025

This PR updates the DefaultLlmImpl class to properly parse responses from the Claude V3 family of Bedrock models. Previously, only the V2 response format (completion field) was supported. With this change, the parser now supports the V3 format (content -> text) while maintaining backward compatibility with V2.

Changes Made

  • Updated buildChatCompletionOutput():
    Detects "content" field for Claude V3 responses.
    Extracts text from the first element of the content list.
    Adds extracted text to answers.
    Added a new unit test in DefaultLlmImplTests.java:
  • testChatCompletionApiForBedrockClaudeV3()
    Mocks a V3-style Bedrock response and validates output parsing.
    Reformatted test files to satisfy Spotless formatting rules.

Why this change is needed
Claude V2 models are deprecated; V3 is the current standard.
Without this fix, requests to "bedrock/anthropic-claude" with V3 models fail to extract answers, causing RAG pipelines to break.
Ensures users don’t need to pass llmResponseField manually for default Bedrock Claude models.

How to test
Run all existing unit tests:
./gradlew clean build

resolves #4168 (comment)

@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 7, 2025 15:01 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 7, 2025 15:01 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 7, 2025 15:01 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 7, 2025 15:01 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 force-pushed the fix/claude-v3-bedrock-support branch from aa64626 to 540a52e Compare October 7, 2025 15:02
@sonianuj287 sonianuj287 had a problem deploying to ml-commons-cicd-env-require-approval October 7, 2025 15:03 — with GitHub Actions Failure
@sonianuj287 sonianuj287 had a problem deploying to ml-commons-cicd-env-require-approval October 7, 2025 15:03 — with GitHub Actions Error
@sonianuj287 sonianuj287 had a problem deploying to ml-commons-cicd-env-require-approval October 7, 2025 15:03 — with GitHub Actions Failure
@sonianuj287 sonianuj287 had a problem deploying to ml-commons-cicd-env-require-approval October 7, 2025 15:03 — with GitHub Actions Error
@sonianuj287
Copy link
Author

Hi @mingshl @rithin-pullela-aws , Could you please review my PR and suggest changes.

Thanks :)

@mingshl
Copy link
Collaborator

mingshl commented Oct 7, 2025

oh great, this is going to use the same interface string, and it will auto detect these two formats and parse accordingly. Can you also add in ITs? this is the best way we can monitor and keep track when the model interface failed.

I ran your CIs and also I added the resolved issues and link the issue in the PR descriptions.

@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 7, 2025 20:02 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 7, 2025 20:02 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 7, 2025 20:02 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 7, 2025 20:02 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 11, 2025 06:53 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 11, 2025 06:53 — with GitHub Actions Waiting
@sonianuj287
Copy link
Author

Hi @rithin-pullela-aws , sorry for bothering you again and again. But please if you have time to review them once and suggest me the changes, it'll keep me occupied by working on it.
Thanks :)

@sonianuj287 sonianuj287 temporarily deployed to ml-commons-cicd-env-require-approval October 13, 2025 18:14 — with GitHub Actions Inactive
@sonianuj287 sonianuj287 temporarily deployed to ml-commons-cicd-env-require-approval October 13, 2025 18:14 — with GitHub Actions Inactive
@sonianuj287 sonianuj287 temporarily deployed to ml-commons-cicd-env-require-approval October 13, 2025 18:14 — with GitHub Actions Inactive
@sonianuj287 sonianuj287 temporarily deployed to ml-commons-cicd-env-require-approval October 13, 2025 18:14 — with GitHub Actions Inactive
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 13, 2025 20:00 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 13, 2025 20:00 — with GitHub Actions Waiting
Copy link
Contributor

@rithin-pullela-aws rithin-pullela-aws left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small Comment about the IT.
It should simple to have the IT fixed instead of writing a new one.

We can remove the BM25_SEARCH_REQUEST_WITH_CONVO_WITH_LLM_RESPONSE_TEMPLATE from plugin/src/test/java/org/opensearch/ml/rest/RestMLRAGSearchProcessorIT.java to have the IT hit the code change

@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 15, 2025 04:32 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 15, 2025 04:32 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 15, 2025 04:32 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 15, 2025 04:32 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 force-pushed the fix/claude-v3-bedrock-support branch 2 times, most recently from 0cea7a7 to 19380cb Compare October 15, 2025 10:05
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 15, 2025 10:07 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 15, 2025 10:07 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 15, 2025 10:07 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 15, 2025 10:07 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 force-pushed the fix/claude-v3-bedrock-support branch from 19380cb to 6006e30 Compare October 15, 2025 10:24
@sonianuj287 sonianuj287 force-pushed the fix/claude-v3-bedrock-support branch from 6006e30 to cef3064 Compare October 15, 2025 10:25
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 15, 2025 10:26 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 15, 2025 10:26 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 15, 2025 10:26 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 15, 2025 10:26 — with GitHub Actions Waiting
@sonianuj287
Copy link
Author

Small Comment about the IT. It should simple to have the IT fixed instead of writing a new one.

We can remove the BM25_SEARCH_REQUEST_WITH_CONVO_WITH_LLM_RESPONSE_TEMPLATE from plugin/src/test/java/org/opensearch/ml/rest/RestMLRAGSearchProcessorIT.java to have the IT hit the code change

Hi @rithin-pullela-aws , I understood your point. Made some changes, please review.
Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Outdated Default Claude model parsing in Generative QA Processor

4 participants