Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Anthropic integration #375

Open
rlouf opened this issue Nov 16, 2023 · 7 comments · May be fixed by #1244
Open

Add Anthropic integration #375

rlouf opened this issue Nov 16, 2023 · 7 comments · May be fixed by #1244

Comments

@rlouf
Copy link
Member

rlouf commented Nov 16, 2023

No description provided.

@jakaline-dev
Copy link

Correct me if I'm wrong - for what I know, the anthropic/claude API does not have an API for outputting token probabilities yet

@rlouf
Copy link
Member Author

rlouf commented Nov 19, 2023

That's correct, but we try to add other providers so people can compare their output to guided generation in outlines. I often need to do this.

@sztanko
Copy link

sztanko commented Dec 15, 2023

Anthropic is now available via AWS Bedrock: https://aws.amazon.com/bedrock/

I think it would be great to add support for Bedrock alltogether!

@rlouf rlouf closed this as not planned Won't fix, can't repro, duplicate, stale Jan 26, 2024
@shabie
Copy link

shabie commented Feb 18, 2024

why was this idea of support for Bedrock dropped?

@rlouf
Copy link
Member Author

rlouf commented Feb 19, 2024

The team has very limited resources

@shabie
Copy link

shabie commented Feb 19, 2024

What if I volunteered to do this?

@jeremyallenjacobson
Copy link

Correct me if I'm wrong - for what I know, the anthropic/claude API does not have an API for outputting token probabilities yet

Yes, with Bedrock, what is returned from the API request varies from one model to the next.
For instance, the Anthropic models response does not contain token probabilities in the response.
On the other hand, the Cohere Command models response will contain an array of per token likelihoods if you specify the return_likelihoods input parameter. I am not familiar enough with Outlines to know if that is enough.

@rlouf rlouf reopened this Jun 11, 2024
@rlouf rlouf linked a pull request Nov 5, 2024 that will close this issue
@rlouf rlouf added this to the 1.0 milestone Nov 5, 2024
@rlouf rlouf self-assigned this Nov 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants