Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a lexer, tests and examples for COBOL #2067

Merged
merged 21 commits into from
Oct 8, 2024
Merged

Conversation

bartbroere
Copy link
Contributor

In this PR I add a lexer for COBOL, some tests and an example sourced from a Creative Commons repository.

Hopefully this PR is complete enough. Let me know if I should add or fix some things.

Resolves #1569

@bartbroere
Copy link
Contributor Author

The CI fails for 2.7 because of this:

/home/runner/work/rouge/rouge/lib/rouge/lexers/mojo.rb:6:in `<module:Lexers>': uninitialized constant Rouge::Lexers::Python (NameError)
	from /home/runner/work/rouge/rouge/lib/rouge/lexers/mojo.rb:[5](https://github.com/rouge-ruby/rouge/actions/runs/10759257587/job/29835709356?pr=2067#step:4:6):in `<module:Rouge>'
	from /home/runner/work/rouge/rouge/lib/rouge/lexers/mojo.rb:4:in `<top (required)>'
	from /home/runner/work/rouge/rouge/lib/rouge/lexer.rb:532:in `load'
	from /home/runner/work/rouge/rouge/lib/rouge/lexer.rb:532:in `load_lexer'
	from /home/runner/work/rouge/rouge/lib/rouge.rb:52:in `block in load_lexers'
	from /home/runner/work/rouge/rouge/lib/rouge.rb:51:in `each'
	from /home/runner/work/rouge/rouge/lib/rouge.rb:51:in `load_lexers'
	from /home/runner/work/rouge/rouge/lib/rouge.rb:[6](https://github.com/rouge-ruby/rouge/actions/runs/10759257587/job/29835709356?pr=2067#step:4:7)7:in `<top (required)>'
	from /opt/hostedtoolcache/Ruby/2.7.8/x64/lib/ruby/gems/2.7.0/gems/bundler-2.4.22/lib/bundler/runtime.rb:60:in `require'
	from /opt/hostedtoolcache/Ruby/2.[7](https://github.com/rouge-ruby/rouge/actions/runs/10759257587/job/29835709356?pr=2067#step:4:8).8/x64/lib/ruby/gems/2.7.0/gems/bundler-2.4.22/lib/bundler/runtime.rb:60:in `block (2 levels) in require'
	from /opt/hostedtoolcache/Ruby/2.7.[8](https://github.com/rouge-ruby/rouge/actions/runs/10759257587/job/29835709356?pr=2067#step:4:9)/x64/lib/ruby/gems/2.7.0/gems/bundler-2.4.22/lib/bundler/runtime.rb:55:in `each'
	from /opt/hostedtoolcache/Ruby/2.7.8/x64/lib/ruby/gems/2.7.0/gems/bundler-2.4.22/lib/bundler/runtime.rb:55:in `block in require'
	from /opt/hostedtoolcache/Ruby/2.7.8/x64/lib/ruby/gems/2.7.0/gems/bundler-2.4.22/lib/bundler/runtime.rb:44:in `each'
	from /opt/hostedtoolcache/Ruby/2.7.8/x64/lib/ruby/gems/2.7.0/gems/bundler-2.4.22/lib/bundler/runtime.rb:44:in `require'
	from /opt/hostedtoolcache/Ruby/2.7.8/x64/lib/ruby/gems/2.7.0/gems/bundler-2.4.22/lib/bundler.rb:187:in `require'
	from /home/runner/work/rouge/rouge/spec/spec_helper.rb:6:in `<top (required)>'
	from /opt/hostedtoolcache/Ruby/2.7.8/x64/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require'
	from /opt/hostedtoolcache/Ruby/2.7.8/x64/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require'
rake aborted!
Command failed with status (1)
/home/runner/work/rouge/rouge/vendor/bundle/ruby/2.7.0/gems/rake-[13](https://github.com/rouge-ruby/rouge/actions/runs/10759257587/job/29835709356?pr=2067#step:4:14).2.1/exe/rake:27:in `<top (required)>'
/opt/hostedtoolcache/Ruby/2.7.8/x64/bin/bundle:23:in `load'
/opt/hostedtoolcache/Ruby/2.7.8/x64/bin/bundle:23:in `<main>'
Tasks: TOP => check:specs
(See full trace by running task with --trace)

Since it doesn't seem related to my changes, I opened the PR #2068 to fix this.


# List of COBOL keywords
# sourced from https://www.ibm.com/docs/en/cobol-zos/6.4?topic=appendixes-reserved-words
KEYWORDS = %w[
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Could we convert this into a class variable to take advantage of lazy loading? Constants are eagerly loaded when the class is defined, which can slow down startup time. Also, since the collection is rather big, Sets can be more memory-efficient due to their hash-based implementation.

def keywords
  @keywords ||= Set.new(%w(ACCEPT ACCESS ...))
end

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I applied this, and two of your other suggestions. Hope it is what you meant, Ruby is a relatively new language to me so I had to do some learning about class variables and lazy loading.

lib/rouge/lexers/cobol.rb Outdated Show resolved Hide resolved
lib/rouge/lexers/cobol.rb Outdated Show resolved Hide resolved
@tancnle
Copy link
Collaborator

tancnle commented Sep 27, 2024

Thank you for your work on this PR, @bartbroere 🙇🏼‍♂️. The changes look good overall. I've left some suggestions. Please let me know what you think 🙏🏼

@tancnle
Copy link
Collaborator

tancnle commented Oct 4, 2024

@bartbroere What do you think about 5b32755? I have extracted the keywords, divisions, etc. to class methods and rejig the rule for better performant.

@bartbroere
Copy link
Contributor Author

@bartbroere What do you think about 5b32755? I have extracted the keywords, divisions, etc. to class methods and rejig the rule for better performant.

Thanks for that! I can follow what happens, and if it positively impacts (startup) performance that's even better

@tancnle tancnle added this pull request to the merge queue Oct 8, 2024
Merged via the queue into rouge-ruby:master with commit 8272f9f Oct 8, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

COBOL
2 participants