Add baseten integration #389

philipkiely-baseten · 2025-07-09T04:36:17Z

Description

Adds Baseten as a model provider

Related Issues

Documentation PR

strands-agents/docs#124

Type of Change

New feature

Testing

How have you tested the change? Verify that the changes do not break functionality or introduce warnings in consuming repositories: agents-docs, agents-tools, agents-cli

I ran hatch run prepare

Checklist

I have read the CONTRIBUTING document
I have added any necessary tests that prove my fix is effective or my feature works
I have updated the documentation accordingly
I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
My changes generate no new warnings
Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

yonib05 · 2025-07-10T22:50:09Z

Hey @philipkiely-baseten,
Thank you so much for this PR and your contribution! We really appreciate the work you've put into this, and Baseten looks like a fantastic platform.
While we'd love to include every great model provider, we've reached a point where we need to be selective about adding new providers directly to the core SDK to keep it maintainable and focused.

That said, we absolutely want to support your integration! We'd recommend one of these approaches:

Option 1: Publish your model provider implementation as a standalone package on PyPI. This way, customers can easily install it and use it with our SDK. Here's a great example of how this works: https://pypi.org/project/strands-nvidia-nim/
Option 2: Include it as a submodule in your official Baseten SDK https://pypi.org/project/baseten/, which would give your users direct access through your existing package.

Either way, we'd be happy to feature you on our documentation page as a supported model provider, giving you visibility to our community.
Would either of these approaches work for your use case? We're excited to see Baseten integrated with our ecosystem!

JackYPCOnline · 2025-07-13T18:20:39Z

src/strands/models/baseten.py

+from typing_extensions import Unpack, override
+
+from ..types.content import Messages
+from ..types.models import OpenAIModel


This import path is out of date.

JackYPCOnline · 2025-07-13T18:27:52Z

src/strands/models/baseten.py

+        return cast(BasetenModel.BasetenConfig, self.config)
+
+    @override
+    def stream(self, request: dict[str, Any]) -> Iterable[dict[str, Any]]:


JackYPCOnline · 2025-07-13T18:30:33Z

src/strands/models/baseten.py

+        elif "base_url" in self.config:
+            client_args["base_url"] = self.config["base_url"]
+
+        self.client = openai.OpenAI(**client_args)


We've migrated to AsyncOpenAI in our implementation. Please verify this change is properly reflected throughout the codebase in your PR. Also, ensure you've pulled the most recent code before proceeding with your review.

JackYPCOnline · 2025-07-13T18:32:26Z

src/strands/models/baseten.py

+        Returns:
+            An iterable of response events from the Baseten model.
+        """
+        response = self.client.chat.completions.create(**request)


async happens also here ^^

JackYPCOnline · 2025-07-13T18:33:46Z

src/strands/models/baseten.py

+        yield {"chunk_type": "metadata", "data": event.usage}
+
+    @override
+    def structured_output(


you might want to update async here

JackYPCOnline · 2025-07-15T02:56:34Z

tests_integ/models/providers.py

@@ -69,6 +70,16 @@ def __init__(self):
        max_tokens=512,
    ),
 )
+baseten = ProviderInfo(


If you setup baseten providerInfo, you could reference other integration tests code style, using that pytestmark instead of pytest.skip()

JackYPCOnline · 2025-07-15T03:13:48Z

src/strands/models/baseten.py

+            "content": [cls.format_request_message_content(content) for content in contents],
+        }
+
+    def format_request_messages(self, messages: Messages, system_prompt: Optional[str] = None) -> list[dict[str, Any]]:


You are defining this method to be an instance method but use it as static method in your tests.

JackYPCOnline · 2025-07-15T03:14:52Z

tests/strands/models/test_baseten.py

+def test_format_request_messages_simple():
+    """Test formatting simple messages."""
+    messages = [{"role": "user", "content": [{"text": "Hello"}]}]
+    result = BasetenModel.format_request_messages(messages)


format_request_messages as static method here. This test fails.

JackYPCOnline · 2025-07-15T03:33:26Z

tests/strands/models/test_baseten.py

+    with unittest.mock.patch.object(strands.models.baseten.openai, "AsyncOpenAI") as mock_client_cls:
+        yield mock_client_cls
+
+


Please double check your tests, 13 tests failed.

JackYPCOnline · 2025-07-15T03:34:04Z

tests_integ/models/test_model_baseten.py

+            "api_key": os.getenv("BASETEN_API_KEY"),
+        },
+    )
+


5 integration tests failed also.

JackYPCOnline · 2025-07-16T18:41:01Z

src/strands/models/baseten.py

+        ...
+
+
+class BasetenModel(Model):


You can also reference https://strandsagents.com/latest/documentation/docs/user-guide/concepts/model-providers/cohere/
I think your previous revision was extending OpenAI model provider.

philipkiely-baseten requested a deployment to manual-approval July 9, 2025 04:36 — with GitHub Actions Waiting

JackYPCOnline reviewed Jul 13, 2025

View reviewed changes

philipkiely-baseten requested a deployment to manual-approval July 14, 2025 23:08 — with GitHub Actions Waiting

philipkiely-baseten force-pushed the main branch from cb5560d to 2676b19 Compare July 14, 2025 23:08

philipkiely-baseten requested a deployment to manual-approval July 14, 2025 23:08 — with GitHub Actions Waiting

Add baseten integration

5dfb0e0

philipkiely-baseten force-pushed the main branch from 2676b19 to 5dfb0e0 Compare July 14, 2025 23:35

philipkiely-baseten requested a deployment to manual-approval July 14, 2025 23:35 — with GitHub Actions Waiting

Update Strands with async and fixed bug with dedicated

3152819

philipkiely-baseten had a problem deploying to manual-approval July 15, 2025 00:49 — with GitHub Actions Error

JackYPCOnline reviewed Jul 15, 2025

View reviewed changes

philipkiely-baseten requested a deployment to manual-approval July 15, 2025 03:06 — with GitHub Actions Waiting

JackYPCOnline reviewed Jul 15, 2025

View reviewed changes

tests_integ/models/test_model_baseten.py

"api_key": os.getenv("BASETEN_API_KEY"),

},

)

Copy link

Contributor

JackYPCOnline Jul 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5 integration tests failed also.

JackYPCOnline reviewed Jul 16, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add baseten integration #389

Add baseten integration #389

Uh oh!

philipkiely-baseten commented Jul 9, 2025

Uh oh!

yonib05 commented Jul 10, 2025

Uh oh!

JackYPCOnline Jul 13, 2025

Uh oh!

JackYPCOnline Jul 13, 2025 •

edited

Loading

Uh oh!

JackYPCOnline Jul 13, 2025 •

edited

Loading

Uh oh!

JackYPCOnline Jul 13, 2025

Uh oh!

JackYPCOnline Jul 13, 2025

Uh oh!

JackYPCOnline Jul 15, 2025

Uh oh!

JackYPCOnline Jul 15, 2025

Uh oh!

JackYPCOnline Jul 15, 2025

Uh oh!

JackYPCOnline Jul 15, 2025

Uh oh!

JackYPCOnline Jul 15, 2025

Uh oh!

JackYPCOnline Jul 16, 2025

Uh oh!

Uh oh!

		with unittest.mock.patch.object(strands.models.baseten.openai, "AsyncOpenAI") as mock_client_cls:
		yield mock_client_cls

Add baseten integration #389

Are you sure you want to change the base?

Add baseten integration #389

Uh oh!

Conversation

philipkiely-baseten commented Jul 9, 2025

Description

Related Issues

Documentation PR

Type of Change

Testing

Checklist

Uh oh!

yonib05 commented Jul 10, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JackYPCOnline Jul 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JackYPCOnline Jul 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

JackYPCOnline Jul 13, 2025 •

edited

Loading

JackYPCOnline Jul 13, 2025 •

edited

Loading