Use AST

[pherrymason]

Motivation

Treesitter is used to obtain a structured representation of source code, which has been utilized to extract symbol declarations.

So far, Treesitter has been used to parse the source code to analyze and obtain a structured representation of the code: Context Syntax Tree (CST). The CST, also known as the Parse Tree, contains all the elements that are part of the text (spaces, brackets, etc.).
Although the CST can be navigated to analyze the code, it contains too much information about the language's syntax, making certain tasks more challenging.

On the other hand, using an AST (Abstract Syntax Tree) would allow working with a simpler structure that more accurately reflects the program's logic.

Strategy

Continuing to use Treesitter as a parser simplifies a large part of the work, as there is currently no other ready-to-use parser available.
One option is to convert the CST generated by Treesitter into an AST.
Subsequently, by using the Visitor pattern, it would be possible to implement different analyzers:

• Symbol table generator
• Diagnostics
• Etc.

TODO

• Create an Abstract Syntax Tree from treesitter's CST.
• Registering symbols from AST
  • Import statements.
  • Global variables.
  • Global constants
  • Enums
  • Faults
  • Interfaces
  • Structs
  • Bitstructs
  • Define
  • Function declarations
    • local variables
  • Macros
  • Distincts
• LSP Features
  • Migrate stdlib indexer
  • load whole workspace on workspace open.
  • go to definition: Mostly done, generics left.
    • Solve generics
  • hover: Mostly done, needs to review every type of symbol to complete descriptions left.
  • autocomplete: entry point created, started to migrate code
  • function signature: entry point created, started to migrate code
• Port Bump C3 Treesitter-parser to 0.2.3: Bump C3 tree-sitter parser to 0.2.3 #99
• Port CPU improvements: Greatly optimize type search #100
• Port Show documentation on hover and completion: Show documentation on hover and completion #101 (Hover done, completion missing)
• Port Improve syntax highlighting in function information on hovering: Slightly improve function hover #102
• Port Improve macro handling: Improve macro handling #103
• Port Fix parsing of non-type alias: Fix parsing of non-type alias def #105
• Port Add completion detail: Add completion detail #106
• Port Support distinct types (Support distinct types #107).
• Port Fix completion of enum and fault methods: Fix completion of enum and fault methods #111
• Port improvements to stdlib indexer: Improve types in the stdlib indexer #113

[pherrymason]

@PgBiel , I'm interested in the design of the ast nodes and how symbols are now indexed.
You'll see there's a single struct type to store all different symbols instead one different struct for each type of symbol. For now I think it is working.

[PgBiel]

Thanks. I'll probably stay off for several days now, but I'll try to take a look at a later date and give you some feedback. 👍