Testing

Testing

A new test-suite is in development. See issue #3.

This page documents the proposed architecture of the test-suite and the level of assurance for the correctness of the library.

Parser

We will use test262 (on GitHub) for the parser test cases. Run the parser on every test-case and compare the result to the expected parse tree. However, test262 containts a lot of cases, and encoding their parse trees by hand is going to be tedious. Instead, we are going to leverage an existing parser implementation, Mozilla SpiderMonkey, which has a convenient API. We have a parser and an AST for the SpiderMonkey source representation, mozilla-js-parser-api, as well as a converter from that representation to our AST representation language-ecmascript-testgen. (Both are still being tested.)

We don't want to depend on SpiderMonkey for test-suite execution because it's slow, and the test-suite can be run in environments that don't have it (e.g. Travis CI). Instead, we will write a separate program to generate expected language-ecmascript parse trees from the SpiderMonkey parse trees. We will need to rerun this program regularly and, at least, before every Hackage release.

The current work plan is:

• Include test262 and mozilla-js-parser-api as Git submodules
• Write a test-case generator that goes through all the cases in test262, invokes SpiderMonkey on the case, parses the result using mozilla-js-parser-api, translates it into the language-ecmascript AST and saves the expected result in either a separate file (using binary serialization) or in a Haskell source file (like it's done in test/Test/StatementTests.hs right now).
• Modify the parser test-suite to read both the test-cases and expected parse trees, run the parser on the every test-case and compare it to the corresponding parse tree.

Pretty-printer

We test the pretty-printer using QuickCheck. The property we test is: \forall js. parse(pretty(js)) == js. The test is already implemented.