Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add standard integration tests for Amazon, AI21, Cohere and Meta models #237

Open
3coins opened this issue Oct 10, 2024 · 1 comment
Open

Comments

@3coins
Copy link
Collaborator

3coins commented Oct 10, 2024

Add standard integration test suite for Amazon, AI21, Cohere, and Meta models for the ChatBedrock and ChatBedrockConverse classes. For example, here is the standard test for Anthropic models.

class TestBedrockStandard(ChatModelIntegrationTests):
    @property
    def chat_model_class(self) -> Type[BaseChatModel]:
        return ChatBedrockConverse

    @property
    def chat_model_params(self) -> dict:
        return {"model": "anthropic.claude-3-sonnet-20240229-v1:0"}

    @property
    def standard_chat_model_params(self) -> dict:
        return {"temperature": 0, "max_tokens": 100, "stop": []}

    @property
    def supports_image_inputs(self) -> bool:
        return True

Note: The converse API does not support all capabilities for all providers defined within the standard test integration suite, some of which has to be skipped. For more details on compatibility of converse API, see the Bedrock documentation here.
https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-supported-models-features.html

@3coins
Copy link
Collaborator Author

3coins commented Dec 9, 2024

As of 12/9/2024, integration tests for Cohere and Meta has been added.

@3coins 3coins added the todo label Dec 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant