How to match negated tokens?

[raylras]

I have a rule defined as follows:

BracketExpression:
    '<' content = (/* whatever, just not '>' */)+ '>'
;

However, it seems that this syntax is not permitted in Langium 3.1.1:

BracketExpression:
    '<' content = (!'>')+ '>'
;

While I could technically enumerate all the tokens, this approach is clearly not ideal.

[msujew]

Hey @raylras,

parser generators (or parser engines in general) generally don't support not parsing something. You instruct what to parse, and they will do it. For your case in particular, you might be able to get away with defining a terminal that matches everything until it sees the > character:

BracketExpression:
    '<' content = BracketContent '>'
;

terminal BracketContent: /[\s\S]+?(?=[>])/; // use positive lookahead to not consume the `>` character

Note that this is slightly hacky - this isn't a language that is well parsable by a parser with a separate tokenization and parsing phase. Often times, those languages require a custom lexer/tokenizer that is aware of some of the context of the text.

[raylras]

terminal BracketContent: /[\s\S]+?(?=[>])/; // use positive lookahead to not consume the > character

Even if I place this terminal rule at the end of the .langium file, it still matches anything and overrides other terminal rules. So it is basically unusable.

this isn't a language that is well parsable by a parser with a separate tokenization and parsing phase.

I'm not sure. This syntax comes from a handwritten LL parser originally.
To my surprise, ANTLR works correctly when I translated it to ANTLR v4.

[msujew]

Even if I place this terminal rule at the end of the .langium file, it still matches anything and overrides other terminal rules. So it is basically unusable.

I see, I can confirm that this doesn't seem to work - though I have no clue why. I'll have some time after my current conference to look into this.

To my surprise, ANTLR works correctly when I translated it to ANTLR v4.

Huh, I wasn't aware that ANTLR supports that. That's fairly interesting.