Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

updates to grammar to successfully generate lexer/parser/visitor with Antlr4.7.1 #428

Merged
merged 15 commits into from
Sep 28, 2018

Conversation

clinssen
Copy link
Contributor

No description provided.

@clinssen
Copy link
Contributor Author

OK, so this is not exactly the latest version of Antlr4, but the one from the Ubuntu package repository--the actually latest Antlr4 (4.7.1) fails to parse the grammar, and complains about "custom channels are not supported in combined grammars". This might be a separate issue that we have to figure out.

@clinssen
Copy link
Contributor Author

A slight change in the Antlr4 grammar (PyNestML.g4) seems to have fixed the issue.

@clinssen clinssen changed the title bug fix in the visitor, perhaps due to variable renaming in the past updates to grammar to successfully generate lexer/parser/visitor with Antlr4.7.1 Sep 26, 2018
@clinssen
Copy link
Contributor Author

I filed this PR because I couldn't generate the lexer/parser/visitor.py from the PyNESTML grammar with antlr4.7.1.

In general, Antlr recommends that you separate your lexer grammar and your parser grammar. The lexer grammar is the old Tokens.g4, but another (new?) requirement of Antlr is that you explicitly define all your literals. That is, you cannot use '=' in the parser, but have to use, say, EQUALS, with EQUALS : '=' defined in the lexer. See e.g. antlr/antlr4#37

This PR implements the necessary changes. Note that the target language is set to Python2; this is also Python3 compatible (but not vice versa).

I would still like to include lexer/parser/visitor generation as part of the Travis CI build script.

@clinssen
Copy link
Contributor Author

With the latest commit, lexer/parser/visitor generation has been included in the Travis CI build script. This means that during installation, the pynestml/generated directory is cleared, and antlr4 is invoked to generate lexer/parser/visitor.py, which are then used by pynestml in the further tests.

Copy link
Contributor

@jougs jougs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many thanks for this update. I did not check any single line of the change and trust that @clinssen knows what he is doing.

@jougs jougs merged commit a538671 into nest:master Sep 28, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants