-
Notifications
You must be signed in to change notification settings - Fork 0
/
tester
executable file
·43 lines (37 loc) · 1.87 KB
/
tester
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
# Write the actual tester here. This should be an executable that takes test settings as a json string sent as an argument to the `tester` script,
# and prints test results to stdout as json in the following format:
{
"test_name": # name of the test,
"output": # any additional information provided by the test run (test result output for example),
"marks_earned": # the number of marks earned by running this test,
"marks_total": # the number of total possible marks that can be earned from running this test,
"status": # a status code. One of: "pass", "fail", "partial", "error",
"time": # the amount of time in milliseconds it took to run the test (optional)
}
If this tester generates annotations for the files under test, they should be reported by printing the annotations
to stdout as json in the following format:
{
"annotations": [
{
"filename": # a relative path (from the root of the test directory) to the file to annotate,
"content": # a string containing the content/message of the annotation,
"line_start": # an integer indicating which line the annotation starts at (1-indexed),
"line_end": # an integer indicating which line the annotation ends at (1-indexed),
"column_start": # an integer indicating which character/column the annotation starts at (0-indexed),
"column_end", # an integer indicating which character/column the annotation ends at (0-indexed)
},
...
]
}
# This tester can run multiple tests and each test should print a json string formatted as above to stdout.
# Do not print the json strings in an array, just print them one by one to stdout. For example, a tester should
# print:
{ "test_name": ... }
{ "test_name": ... }
{ "test_name": ... }
# as opposed to:
[
{ "test_name": ... },
{ "test_name": ... },
{ "test_name": ... }
]