From b3f18b2b8cd82da87f2b2bc5593d9939338bc614 Mon Sep 17 00:00:00 2001 From: Oleg Bespalov Date: Fri, 30 Aug 2024 10:38:35 +0200 Subject: [PATCH] Document maintanance of tc39 (#3926) * Document maintenance of tc39 --------- Co-authored-by: Mihail Stoykov <312246+mstoykov@users.noreply.github.com> --- js/tc39/README.md | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/js/tc39/README.md b/js/tc39/README.md index 5d317841dfb..9919d3951ef 100644 --- a/js/tc39/README.md +++ b/js/tc39/README.md @@ -7,15 +7,21 @@ Ways to use it: that was tested with this module 2. Run `go test &> out.log` -The full list of failing tests, and the error, is in `breaking_test_errors-*.json`. All errors list there with the corresponding error will *not* be counted as errors - this is what the test expects, those specific errors. -Due to changes to Sobek it's common for the error to change, or there to be now a new error on the previously passing test, or (hopefully) a test that was not passing but now is. -In all of those cases `breaking_test_errors-*.json` needs to be updated. Run the test with `-update` flag to update: `go test -update` - -NOTE: some text editors/IDEs will try to parse files ending in `json` as JSON, which given the size of `breaking_test_errors-*.json` might be a problem when it's not actually a JSON (before the edit). So it might be a better idea to name it something different if editing by hand and fix it later. +The full list of failing tests, and the error, is in `breaking_test_errors-*.json`. All errors list there with the corresponding error will *not* be counted as errors - this is what the test expects, those specific errors. See reasons for this at the end of this document. This is a modified version of [the code in the original goja repo](https://github.com/dop251/goja/blob/master/tc39_test.go) that Sobek was forked from. +## Maintaining the tests + +There are few things to keep in mind when maintaining the tests. + +* For most of the time, we aim to have the same version (commit hash) of the test suite as the one that [uses Sobek](https://github.com/grafana/sobek/blob/main/.tc39_test262_checkout.sh#L3). So if Sobek brings a new version of the test suite, we should update the version of the test suite in this package as well. +* The new version of Sobek could bring a new functionality, so you might need to re-evaluate the `featuresBlockList`, `skipList` and any other list in the `tc39_test.go` file. The way to go is also keeping this lists closer to the Sobek, however they are not the same, so you might need to adjust them. +* Due to changes to Sobek it's common for the error to change, or there to be now a new error on the previously passing test, or (hopefully) a test that was not passing but now is. +In all of those cases `breaking_test_errors-*.json` needs to be updated. Run the test with `-update` flag to update: `go test -update`. +* Important to mention that there should be a balance between just updating the list of the errors and actually fixing errors. So it's recommended to check the diff of the `breaking_test_errors-*.json` and case by case decide if the error should be updated or the test should be fixed. + ## Reasons for recording breaking_test_errors.json