Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

experimental HiFi tree diff algorithm for use with quick-fixes and refactoring commands in the IDE #2031

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

jurgenvinju
Copy link
Member

@jurgenvinju jurgenvinju commented Sep 12, 2024

This transforms a pair of parse Trees <original, rewritten> to a list[TextEdit]s. The resulting
TextEdits are ready for use in VScode extensions via LSP features in util::LanguageServer and
util::IDEServices.

The single pass parse tree recursion maps sub-tree and sub-list differences to textual differences in a special way.
It lifts on the semantics of special parse tree non-terminals (literals, lexicals, separators) to ignore certain
superfluous changes made in the rewritten tree. As a result the edited source text retains more of
its original layout, including indentation and comments, as compared to yielding the
rewritten parse tree to a string and replacing the entire file.

Especially with separated lists this algorithm does amazing work. Suppose you have a target pattern:
<Element e>, <{Element ","}* newElems, <Element f> and newElems happens to be empty, then:

  1. Rascal will remove the superfluous separators and layout around the empty list.
  2. HifiTreeDiff will try and keep indentation and source code comments after the e and before the f even
    though these have been replaced by the concrete target pattern.
  3. If the list is non-empty, also the layout before the new sublist and after the new sublist will be taken from the
    original list where possible.

The smaller diffs are not only good for high-fidelity in general, but also in particular smaller diffs are essential for providing interactive preview and undo features in the IDE. This PR enables language engineers to use parse tree rewriting rather than collecting the text edits themselves.

TODO's:

  • write test, and fix initial issues triggered by the tests
  • test the example in the documentation
  • shorten the documentation
  • document the TextEdits module
  • short-circuit lexical identifiers (do not go deeper)
  • write indentation inheritance algorithm

Copy link

codecov bot commented Sep 12, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 49%. Comparing base (05358f7) to head (374a8a2).
Report is 25 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##              main   #2031   +/-   ##
=======================================
- Coverage       49%     49%   -1%     
+ Complexity    6305    6302    -3     
=======================================
  Files          664     664           
  Lines        59626   59635    +9     
  Branches      8646    8647    +1     
=======================================
  Hits         29482   29482           
- Misses       27935   27936    +1     
- Partials      2209    2217    +8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jurgenvinju
Copy link
Member Author

@tvdstorm @DavyLandman I don't have the energy to finish this now, so I'm parking it here until I do. I wanted you to know it exists, because low-fidelity rewrites are a common issue we have to solve and because refactoring and quick-fixes in VScode are now under our fingertips.

@DavyLandman
Copy link
Member

Cool stuff 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants