parser.md

Parser

The sourcecode for the TypeScript parser is located entirely in parser.ts. Scanner is controlled internally by the Parser to convert the source code to an AST. Here is a review of what the desired outcome is.

SourceCode ~~ scanner ~~> Token Stream ~~ parser ~~> AST

The parser is implemented as a singleton (similar reasons to scanner, don't want to recreate it if we can reinit it). It is actually implemented as namespace Parser which contains state variables for the Parser as well as a singleton scanner. As mentioned before it contains a const scanner. The parser functions manage this scanner.

Usage by program

Parser is driven indirectly by Program (indirectly as its actually by CompilerHost which we mentioned previously). Basically this is the simplified call stack:

Program ->
    CompilerHost.getSourceFile ->
        (global function parser.ts).createSourceFile ->
            Parser.parseSourceFile

The parseSourceFile not only primes the state for the Parser but also primes the state for the scanner by calling initializeState. It then goes on to parse the source file using parseSourceFileWorker.

Sample Usage

Before we dig too deep into the parser internals, here is a sample code that uses the TypeScript's parser to get the AST of a source file (using ts.createSourceFile), and then print it.

code/compiler/parser/runParser.ts

import * as ts from "ntypescript";

function printAllChildren(node: ts.Node, depth = 0) {
    console.log(new Array(depth + 1).join('----'), ts.syntaxKindToName(node.kind), node.pos, node.end);
    depth++;
    node.getChildren().forEach(c=> printAllChildren(c, depth));
}

var sourceCode = `
var foo = 123;
`.trim();

var sourceFile = ts.createSourceFile('foo.ts', sourceCode, ts.ScriptTarget.ES5, true);
printAllChildren(sourceFile);

This will print out the following:

SourceFile 0 14
---- SyntaxList 0 14
-------- VariableStatement 0 14
------------ VariableDeclarationList 0 13
---------------- VarKeyword 0 3
---------------- SyntaxList 3 13
-------------------- VariableDeclaration 3 13
------------------------ Identifier 3 7
------------------------ FirstAssignment 7 9
------------------------ FirstLiteralToken 9 13
------------ SemicolonToken 13 14
---- EndOfFileToken 14 14

This looks like a (very right sided) tree if you tilt your head to the left.