Parser changes questions

[Arcnor]

Hi there!

First of all, thank you for making Ink available for all, I really love this system and I try to find any excuse to use it on all my projects :D.

I've been working on a project lately that requires me to recreate the Ink after adding some modifications to it (we're adding automatic ids to every line for voiceover / I18N) and I've found it difficult to recreate the Ink exactly from an AST. Basically, I'm trying to write a "Ink to Ink" parser 🙂 .

I was wondering if changes that make this process easier would be accepted upstream: As an example, one of the changes I had to do was create a BooleanNumber class and return that instead of a normal Number when parsing the true/false case. That one was innocuous but something more "controversial" like whitespace should also be saved to help with this task (the Roslyn compiler also does this, they call it "Syntax trivia"), maybe with an option so the cost is not paid when not needed.

I'll appreciate any thoughts on this, and thanks again!

[joethephish]

Hey! That’s really interesting. Storing a specific Boolean type would be totally fine, and makes sense.

Parsing and storing whitespace in the AST could work. The one that might be a bit harder is comments since they’re eliminated separately in a pre-pass right now.

So, that would probably be the hardest part, since we’d need to reconsider how comments are parsed from the ground up, and check for them somewhere where they’re caught absolutely everywhere...

[Arcnor]

Thanks for your answer!

And as you say, comments are eliminated right now, which is not ideal, so I'll also need to change that. Maybe the preprocessing can be removed, and when encountering them in the parsing phase they can either be added as "trivia" or removed, but I'm assuming there is a reason why it's done in this way right now.