docs(core): extend the grammar definition

[lukeribchester]

This pull request extends the Pact grammar definition documented in Semantics.md.

Draft: The current definition is approximately complete based on the current Pact 5 parser implementation, but contains a few minor issues which are visible when testing against the Pact test files. These issues will be documented, discussed, and fixed here before merging this pull request.

To Do

• Finalise operators.

Pull Request Checklist

• Any changes that could be relevant to users have been recorded in the changelog

[jmcardon]

Hey @lukeribchester, thanks for the PR! Looks good so far, let me know what issues come up, because the current parser spec works against all the test files.

[lukeribchester]

Hey @lukeribchester, thanks for the PR! Looks good so far, let me know what issues come up, because the current parser spec works against all the test files.

Hey @jmcardon, thank you and apologies for my delayed reply. I've further refined the grammar definition based on my experiences working with it for the IntelliJ Platform plugin release, and I've separated most special characters from the identifier token definition into their respective definitions and categories. My parser now functions correctly against all test files except for fv-syntax-regression.repl — where individual % and @ symbols raise syntax errors. My understanding is that neither of these are operators in Pact nor Lisp. I could classify them as identifiers but that feels weird. What do you recommend?

[lukeribchester]

This is currently in use in version 0.2.0 of the Pact plugin for the IntelliJ Platform if you wish to check it out — it's the first release with native language support.

[jmcardon]

Hey @lukeribchester, thanks for the PR! Looks good so far, let me know what issues come up, because the current parser spec works against all the test files.

Hey @jmcardon, thank you and apologies for my delayed reply. I've further refined the grammar definition based on my experiences working with it for the IntelliJ Platform plugin release, and I've separated most special characters from the identifier token definition into their respective definitions and categories. My parser now functions correctly against all test files except for fv-syntax-regression.repl — where individual % and @ symbols raise syntax errors. My understanding is that neither of these are operators in Pact nor Lisp. I could classify them as identifiers but that feels weird. What do you recommend?

Hi I'm sorry for the delayed reply!

Pact-5 is just parsing this atm, and we amended the parse tree to work with this regression.