Skip to content

Commit

Permalink
Specify line_breaks on tokens with custom patterns
Browse files Browse the repository at this point in the history
A warning was printed by Chevrotain for tokens using a custom matcher
function for the keyword pattern.
While this doesn't happen in the default implementation, it is possible
to override `buildKeywordPattern` and use a custom matcher function
  • Loading branch information
aabounegm committed Sep 17, 2024
1 parent b2a6fe9 commit 3983f24
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion packages/langium/src/parser/token-builder.ts
Original file line number Diff line number Diff line change
Expand Up @@ -130,9 +130,12 @@ export class DefaultTokenBuilder implements TokenBuilder {
}

protected buildKeywordToken(keyword: Keyword, terminalTokens: TokenType[], caseInsensitive: boolean): TokenType {
const keywordPattern = this.buildKeywordPattern(keyword, caseInsensitive);

return {
name: keyword.value,
PATTERN: this.buildKeywordPattern(keyword, caseInsensitive),
PATTERN: keywordPattern,
LINE_BREAKS: typeof keywordPattern === 'function',
LONGER_ALT: this.findLongerAlt(keyword, terminalTokens)
};
}
Expand Down

0 comments on commit 3983f24

Please sign in to comment.