Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean Up Dead outlines.integrations Code #1054

Closed
lapp0 opened this issue Jul 20, 2024 · 1 comment · Fixed by #1061
Closed

Clean Up Dead outlines.integrations Code #1054

lapp0 opened this issue Jul 20, 2024 · 1 comment · Fixed by #1061
Milestone

Comments

@lapp0
Copy link
Contributor

lapp0 commented Jul 20, 2024

What behavior of the library made you think about the improvement?

With all models other than exllamav2 having a single integration implementation via outlines.generate.api.SequenceGeneratorAdapter and outlines.processors, we don't need outlines.integrations.

  • outlines.integrations.transformers has RegexPrefixAllowedTokens and JSONPrefixAllowedTokens which are now handled by outlines.processors
  • outlines.integrations.llamacpp has logits processors which are redundant with outlines.processors
  • outlines.integrations.vllm has logits processors which are redundant with outlines.processors
  • outlines.integrations.utils has adapt_tokenizer, useful for FSM construction, and convert_json_schema_to_str, used for the three model integrations listed above

After the vLLM integration, we no longer need outlines.integrations

How would you like it to behave?

Get rid of outlines/integrations/ and tests/generate/test_integration_*.py

  • Remove outlines.integrations.transformers, outlines.integrations.llamacpp, and outlines.integrations.vllm
  • Remove outlines.integrations.utils, but move def adapt_tokenizer(...) to outlines.models.tokenizer
@lapp0 lapp0 changed the title Clean Up Dead Integrations Code Clean Up Dead outlines.integrations Code Jul 20, 2024
@rlouf
Copy link
Member

rlouf commented Jul 20, 2024

We should do this once #1010 has been merged and then release 0.1.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants