Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes a problem with tools not working with the new converse endpoint using anthropic models #76

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

userlerueda
Copy link

@userlerueda userlerueda commented Jun 15, 2024

This PR addresses the following problems found while testing the new converse endpoint with anthropic tools:

  • Found some fields that might have been typos (e.g., toolUseID vs toolUseId) boto3 docs
  • Found an issue where bedrock returns the input values for the tool as string instead of a JSON (dict), added a function to attempt to convert it and on JSON decode error, return the original value.
  • Found another issue where the current _snake_to_camel_keys function will attempt to convert the input values for the tool. The problem seems to be that we are calling the _format_tools twice. An easy way to fix it is to add a property to the ChatBedrockConverse so we keep the formatted_tools, that way we only format them once. I have done that and marked the property as excluded from dumping to avoid any compatibility problems.

With these changes, I was able to run the https://python.langchain.com/v0.1/docs/use_cases/tool_use/multiple_tools/ example with bedrock, using anthropic.

{
  "input": "Take 3 to the fifth power and multiply that by the sum of twelve and three, then square the whole result",
  "conversation_id": null,
  "chat_history": [],
  "output": [
    {
      "type": "text",
      "text": "So the final result of taking 3 to the 5th power (243), multiplying by 12 + 3 (15) to get 3645, and then squaring that is 13286025.",
      "index": 0
    }
  ]
}

@userlerueda userlerueda force-pushed the userlerueda/bedrock_converse-mod-init branch from 29478c3 to f1dbef2 Compare June 17, 2024 15:36
@userlerueda userlerueda changed the base branch from bagatur/bedrock_converse to main June 22, 2024 16:11
Anthropic returns the input dictionary for the tool as a string.
Adding a function to attempt to convert it to a dict if it is a string, if conversion fails, returns the string
@userlerueda userlerueda force-pushed the userlerueda/bedrock_converse-mod-init branch 4 times, most recently from 9997c92 to 60e3ba9 Compare June 22, 2024 16:14
It seems like the first time we are calling _format_tools everything works fine, but on a second run, the _snake_case_to_camel_case converts the input parameters to camelCase and breaks the tools.
@userlerueda userlerueda force-pushed the userlerueda/bedrock_converse-mod-init branch from 60e3ba9 to b6d235c Compare June 22, 2024 16:15
@userlerueda userlerueda changed the title Updates __init__.py files for an easier import of ChatBedrockConverse model Fixes a problem with tools not working with the new converse endpoint using anthropic models Jun 22, 2024
@3coins
Copy link
Collaborator

3coins commented Jul 12, 2024

@userlerueda
Can you provide some sample code that showcases the problem. The _format_tools duplicate calls that you have called out are in 2 different methods.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants