Skip to content

Commit

Permalink
Document maintanance of tc39 (#3926)
Browse files Browse the repository at this point in the history
* Document maintenance of tc39

---------

Co-authored-by: Mihail Stoykov <[email protected]>
  • Loading branch information
olegbespalov and mstoykov authored Aug 30, 2024
1 parent 67b8135 commit b3f18b2
Showing 1 changed file with 11 additions and 5 deletions.
16 changes: 11 additions & 5 deletions js/tc39/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,21 @@ Ways to use it:
that was tested with this module
2. Run `go test &> out.log`

The full list of failing tests, and the error, is in `breaking_test_errors-*.json`. All errors list there with the corresponding error will *not* be counted as errors - this is what the test expects, those specific errors.
Due to changes to Sobek it's common for the error to change, or there to be now a new error on the previously passing test, or (hopefully) a test that was not passing but now is.
In all of those cases `breaking_test_errors-*.json` needs to be updated. Run the test with `-update` flag to update: `go test -update`

NOTE: some text editors/IDEs will try to parse files ending in `json` as JSON, which given the size of `breaking_test_errors-*.json` might be a problem when it's not actually a JSON (before the edit). So it might be a better idea to name it something different if editing by hand and fix it later.
The full list of failing tests, and the error, is in `breaking_test_errors-*.json`. All errors list there with the corresponding error will *not* be counted as errors - this is what the test expects, those specific errors. See reasons for this at the end of this document.

This is a modified version of [the code in the original goja
repo](https://github.com/dop251/goja/blob/master/tc39_test.go) that Sobek was forked from.

## Maintaining the tests

There are few things to keep in mind when maintaining the tests.

* For most of the time, we aim to have the same version (commit hash) of the test suite as the one that [uses Sobek](https://github.com/grafana/sobek/blob/main/.tc39_test262_checkout.sh#L3). So if Sobek brings a new version of the test suite, we should update the version of the test suite in this package as well.
* The new version of Sobek could bring a new functionality, so you might need to re-evaluate the `featuresBlockList`, `skipList` and any other list in the `tc39_test.go` file. The way to go is also keeping this lists closer to the Sobek, however they are not the same, so you might need to adjust them.
* Due to changes to Sobek it's common for the error to change, or there to be now a new error on the previously passing test, or (hopefully) a test that was not passing but now is.
In all of those cases `breaking_test_errors-*.json` needs to be updated. Run the test with `-update` flag to update: `go test -update`.
* Important to mention that there should be a balance between just updating the list of the errors and actually fixing errors. So it's recommended to check the diff of the `breaking_test_errors-*.json` and case by case decide if the error should be updated or the test should be fixed.


## Reasons for recording breaking_test_errors.json

Expand Down

0 comments on commit b3f18b2

Please sign in to comment.