Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

I cooked, idk what to say #74

Merged
merged 5 commits into from
Sep 24, 2024
Merged

I cooked, idk what to say #74

merged 5 commits into from
Sep 24, 2024

Conversation

thmsmlr
Copy link
Owner

@thmsmlr thmsmlr commented Sep 24, 2024

I'm not going to lie, I coded locally for seven days, and this is the result.

I'm going to try my best to describe the changes, but, c'est la vie.

  • I improved streaming reliability. Previously, sometimes the chunks weren’t full lines, which would break the parsing. That issue is now fixed.
  • I refactored the internal API between Instructor and the adapters so that it is the adapter's responsibility to parse the JSON. Instructor seamlessly works with that. This change makes it considerably easier to implement any OpenAI-like adapter. For non-OpenAI style adapters like Google Gemini, some work is required to convert the OpenAI message format into the Gemini API format. However, you do not need to map the responses.
  • I added the Anthropic and Gemini adapters. I also changed the Llama CPP adapter to use its built-in OpenAI compliant server.
  • I also refactored the tests a bit to make it easier to validate new adapters as you write them. I hope to eventually write some stronger evaluations, but this should work for now.

@thmsmlr thmsmlr merged commit 0db54d0 into main Sep 24, 2024
1 check failed
@thmsmlr thmsmlr deleted the refactor branch September 24, 2024 00:43
@samrat samrat mentioned this pull request Nov 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant