Allow initializing lexers with a char iterator #42
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This fixes #41 in an almost backwards compatible way. Generated lexers now have
an extra constructor:
API of the generated lexers are exactly the same, however, if a lexer is
constructed with
new_from_iter
instead ofnew
ornew_with_state
, thenmatch_
method will panic in runtime. This is because in lexers constructedwith
new_from_iter
we don't have the input string, so cannot return a sliceto it. Instead use
match_loc
to get the start and end locations of thecurrent match.
Only breaking change is the generated types now have one more generic argument,
for the iterator type. So for a lexer like:
Instead of
we now generate
So any code that refers to the lexer type will break.
Other than this the changes should be backwards compatible.
Fixes #41