Build Index from regex
#125
Draft
+1,056
−2,521
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Supersedes #38? |
|
Yes, it will |
torymur
force-pushed
the
index-from-regex-97
branch
2 times, most recently
from
fa36be9 to
5b8808d
Compare
torymur
force-pushed
the
index-from-regex-97
branch
3 times, most recently
from
88d609b to
52edc6d
Compare
torymur
force-pushed
the
index-from-regex-97
branch
from
1a8831a to
bf6e8a6
Compare
torymur
commented
Jan 9, 2025
| mask: List[int] = [1 if s in allowed else 0 for s in range(1, n_tokens + 1)] | ||
| tokens = model(tokens, mask=mask) | ||
| state = fsm.get_next_state(state, tokens[-1]) | ||
| allowed = guide.read_next_token(tokens[-1]) |
There was a problem hiding this comment.
@dpsimpson Could you, please, take a look at this failing test: https://github.com/dottxt-ai/outlines-core/actions/runs/12695987037/job/35389081344
Interface has changed and I updated it here somewhat accordingly, but it needs to be checked, for example I added third value to Vocabulary (instead of eos token before that) just for the sake of keeping the dimensions right, I suspect that could be incorrect 😅
But overall, since its statistical, it goes over my head to fully understand the intentions and adjust properly expected testing values or construction logic, so I would really appreciate your help here 🙏
torymur
force-pushed
the
index-from-regex-97
branch
from
5ea7e10 to
3fef1d8
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR drops all intermediate python and related rust structures and builds an
Indexfrom regex purely in rust. Also stabilizing python bindings forVocabulary,IndexandGuideand rust interfaces forVocabularyandIndex.indexfrom regex #97interegularfunctionality #38 and closes Port necessaryinteregularfunctionality #8walk_fsmtests to Rust #68create_fsm_index_end_to_endtests to Rust #69test_guidetests totest_regex#71Rust
Indexfrom regex withregex-automataTokentype fromStringtoVec<u8>Vocabulary'seos_token_idfromOption<TokenId>toTokenIdIndexinterfaceIndexpython binding interfaceVocabularyinterfaceVocabularypython binding interfaceinterregularworkflowGuidepython bindingPython
interregularworkflowGuide,Index&VocabularyBenchmarks
Benchmarks reveal x10+ times on average improvement for index compilation comparing to the current main branch.
On my machine to the degree, where a showcase with setting longer GIL switch interval for multithreaded case doesn't exhibit difference with one thread (because it's so fast in the first place, that there is a little sense to give it more time to execute without interruption), but it depends on machine.
