Way to sponsor the project

[viktor-ferenczi]

What behavior of the library made you think about the improvement?

No sponsorship link provided.

How would you like it to behave?

Please add a way to sponsor the project. A PayPal link would do, for example.

My work depends on getting the constrained generation working well with vLLM. This includes tensor parallel mode, JSON, regex and Lark grammar. Today I got it working (except of the Lark grammar) by cherry-picking changes from PRs and making a fix myself. So it is coming together nicely and I would like to keep the project going.

Thanks.

[SupreethRao99]

@viktor-ferenczi could you tell us more about how you got contained generation working with tensor parallel mode? a lot of my work also involves a lot of constrained generation.

Thanks!

[viktor-ferenczi]

@rlouf Could you please review this? Thanks!

[viktor-ferenczi]

@SupreethRao99

Take a look into this ticket: #524

Found a solution in lapp0's PR: #539

Fixed the problem with that PR, so it works now. Waiting on the PR's author (lapp0) to take the fix and get his PR reviewed.

Until then I keep a working outlines branch here: https://github.com/viktor-ferenczi/outlines/tree/dev

Please note, that my branch includes code from my open PR to change the vLLM request parameters, but you can cherry-pick only the fix to the RegexLogitsProcessor, that's a separate commit.

I use it with 2x4090 GPUs and the TheBloke/deepseek-coder-33B-instruct-AWQ model as I described it in #524. Constrained generation is a huge efficiency boost in many cases, at least for me.

[rlouf]

@viktor-ferenczi Thank you so much, it means a lot to us that you're appreciating our work! No need for sponsorship, but we're thinking about ways to get the community more involved in our journey.

[simon-mo]

From vLLM perspective, we are big fan of Outlines! From our side, we will make sure this is also a priority, please don't hesitate to reach out and let us know what's the blocker.