You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
What behavior of the library made you think about the improvement?
With all models other than exllamav2 having a single integration implementation via outlines.generate.api.SequenceGeneratorAdapter and outlines.processors, we don't need outlines.integrations.
outlines.integrations.transformers has RegexPrefixAllowedTokens and JSONPrefixAllowedTokens which are now handled by outlines.processors
outlines.integrations.llamacpp has logits processors which are redundant with outlines.processors
outlines.integrations.vllm has logits processors which are redundant with outlines.processors
outlines.integrations.utils has adapt_tokenizer, useful for FSM construction, and convert_json_schema_to_str, used for the three model integrations listed above
After the vLLM integration, we no longer need outlines.integrations
How would you like it to behave?
Get rid of outlines/integrations/ and tests/generate/test_integration_*.py
Remove outlines.integrations.transformers, outlines.integrations.llamacpp, and outlines.integrations.vllm
Remove outlines.integrations.utils, but move def adapt_tokenizer(...) to outlines.models.tokenizer
The text was updated successfully, but these errors were encountered:
What behavior of the library made you think about the improvement?
With all models other than
exllamav2
having a single integration implementation viaoutlines.generate.api.SequenceGeneratorAdapter
andoutlines.processors
, we don't needoutlines.integrations
.outlines.integrations.transformers
hasRegexPrefixAllowedTokens
andJSONPrefixAllowedTokens
which are now handled byoutlines.processors
outlines.integrations.llamacpp
has logits processors which are redundant withoutlines.processors
outlines.integrations.vllm
has logits processors which are redundant withoutlines.processors
outlines.integrations.utils
hasadapt_tokenizer
, useful for FSM construction, andconvert_json_schema_to_str
, used for the three model integrations listed aboveAfter the vLLM integration, we no longer need
outlines.integrations
How would you like it to behave?
Get rid of
outlines/integrations/
andtests/generate/test_integration_*.py
outlines.integrations.transformers
,outlines.integrations.llamacpp
, andoutlines.integrations.vllm
outlines.integrations.utils
, but movedef adapt_tokenizer(...)
tooutlines.models.tokenizer
The text was updated successfully, but these errors were encountered: