Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support for claude #38

Open
wants to merge 16 commits into
base: main
Choose a base branch
from

Conversation

TwistingTwists
Copy link
Contributor

@TwistingTwists TwistingTwists commented Mar 25, 2024

Done so far:

  • basic md_json mode support
  • basic text support
  • do_streaming_chat_completion/2 in anthropic.ex
  • to_openai_response/1 needs to be more robust

In progress:

Sorry, something went wrong.

@TwistingTwists TwistingTwists mentioned this pull request Mar 25, 2024
@TwistingTwists TwistingTwists marked this pull request as draft March 25, 2024 05:36
@TwistingTwists
Copy link
Contributor Author

I am taking time off for week or so. Might not be actively able to actively contibute to this PR until 4 april.

Can review changes, comments. :)

@TwistingTwists
Copy link
Contributor Author

@TwistingTwists
Copy link
Contributor Author

Sax parser for supporting claude native XML format.

https://github.com/qcam/saxy

@thbar
Copy link
Collaborator

thbar commented Mar 28, 2024

Examples of using Sax parsing with Saxy (which I used recently), if that's useful!

@TwistingTwists
Copy link
Contributor Author

Examples of using Sax parsing with Saxy (which I used recently), if that's useful!

Super useful. Gem. 🌸🌸

Will continue working on this pr after holidays finish. :)

@TwistingTwists
Copy link
Contributor Author

https://twitter.com/jxnlco/status/1773536962403533187

Maybe don't need to do XML stuff. Json mode has been reported to work reliably

@noozo
Copy link

noozo commented Apr 5, 2024

{"anthropic-beta", "tools-2024-04-04"} this header is required to use tools now

@noozo
Copy link

noozo commented Apr 5, 2024

The function to conform to openai structure should be:

  # %{
  #   "content" => [
  #     %{
  #       "text" => "ACTUAL RESPONSE ",
  #       "type" => "text"
  #     }
  #   ],
  #   "id" => "msg_01W35ZRGGrPaL4fXqgB5fHDs",
  #   "model" => "claude-3-haiku-20240307",
  #   "role" => "assistant",
  #   "stop_reason" => "end_turn",
  #   "stop_sequence" => nil,
  #   "type" => "message",
  #   "usage" => %{"input_tokens" => 243, "output_tokens" => 132}
  # }
  defp to_openai_response(%{"content" => [%{"text" => text}]} = _params) do
    %{
      "choices" => [
        %{
          "message" => %{"content" => text}
        }
      ]
    }
  end

@noozo
Copy link

noozo commented Apr 5, 2024

Added PR but removed my fork so i could submit something else to main repo, but it's still here: https://github.com/TwistingTwists/instructor_ex/pull/1/files

@TwistingTwists
Copy link
Contributor Author

Will work on it over the weekend. Thanks for this @noozo !

@TwistingTwists TwistingTwists marked this pull request as ready for review June 24, 2024 09:36
@TwistingTwists
Copy link
Contributor Author

@cigrainger : Streaming response works.

Will add an example in a few minutes. Feel free to checkout this branch.

This might not be mergable, at the moment, because it includes lots of goodies from other branches as well. But it works.

@TwistingTwists
Copy link
Contributor Author

Big thanks to @noozo , @thbar for the tips ! :)

@TwistingTwists
Copy link
Contributor Author

Also, since I was testing Gemini as well, I rebased Andres Alejos 's work and put this in the same PR.

can remove this if needed.

@TwistingTwists
Copy link
Contributor Author

almost done with #33

@cigrainger
Copy link
Contributor

Gave it a try and got the following error:

** (MatchError) no match of right hand side value: {:error, :adapter_error, %{"error" => %{"message" => "tool_choice: Input tag 'function' found using 'type' does not match any of the expected tags: 'auto', 'any', 'tool'", "type" => "invalid_request_error"}, "type" => "error"}}

@TwistingTwists
Copy link
Contributor Author

TwistingTwists commented Jun 27, 2024

@cigrainger ,{:error, error} is always a tuple of two. Also updated the signature of the chat_completion function.

Andres Alejos and others added 10 commits June 27, 2024 18:56

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
done:
* basic md_json mode support
* basic text support
* vision support

todo:
* do_streaming_chat_completion/2 in anthropic.ex
* to_openai_response/1 needs to be more robust
* tool mode for anthropic https://docs.anthropic.com/claude/docs/functions-external-tools#function-calling-format = handle formatting request and parsing response
*

unresolved bugs:
* Jason.decode on Anthropic Response does not work
* until this is merged, using my branch for running anthropic examples
@ice-cap0
Copy link

ice-cap0 commented Jul 9, 2024

eagerly waiting for this 🎉

@TwistingTwists
Copy link
Contributor Author

@frozencap : feel free to try it out today from git directly. Open to feedback.

{:instructor_ex, git: "https://github.com/TwistingTwists/instructor_ex/", branch: "claude_json"}

@southkeys
Copy link

breaks with ecto 3.12

@TwistingTwists
Copy link
Contributor Author

breaks with ecto 3.12

Thanks for reporting. will do it over weekend

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants