Support serialization tests when generating ci.json

[generalmimon]

See kaitai-io/kaitai_struct#1060 (comment)

[generalmimon]

Works only in Python so far. I took the path of least resistance, which turned out to be having separate targets python and python-write (at least from the perspective of the kst-adoption-report and aggregate/convert_to_json scripts). This means that Python test results will be in two folders: the traditional parse tests would remain in test_out/python (and to my knowledge, the ci.json stays exactly the same) and the serialization tests would go to test_out/python-write.

The test names in test_out/python-write/ci.json are special in that they include both the class name and the method name (eventually with stripped test prefixes / snake_case -> camelCase conversions), as discussed in kaitai-io/kaitai_struct#1060 (comment). For example:

{
  "$meta": {
    "lang": "python-write",
    "timestamp": "2023-07-31T10:02:40Z"
  },
  "BcdUserTypeBe.ReadWriteRoundtrip": {
    "status": "passed",
    "elapsed": 0.008,
    "is_kst": true
  },
  "BcdUserTypeLe.ReadWriteRoundtrip": {
    "status": "passed",
    "elapsed": 0.0,
    "is_kst": true
  },
  "BitsByteAligned.ReadWriteRoundtrip": {
    "status": "passed",
    "elapsed": 0.0,
    "is_kst": true
  },

KST adoption reporting is implemented too - if the test class is included in the kst_adoption.log file, all tests in that class will have "is_kst": true. If a class was in kst_adoption.log but there was no test method in it, a plain object { "is_kst": true } itself is added to ci.json under the pure class name as a key (this may be a bit unnecessary because I think such "test result" wouldn't show in our CI dashboard anyway, but the existing mode worked in a similar way).

[GreyCat]

Seems to work, so looks good to me!

Potentially I can outline 2 larger avenues of exploration:

1. How we want to store/present tests for reading and writing parts altogether — e.g. for now "spec" + "specwrite" and "python" + "python-write" seems to be ok, but generally moving forward I really think we want to have them as first class citizens on the same level.

2. Potentially looking into reworking Python invocation per se to report everything in one XML file (as many other languages do), i.e. https://github.com/xmlrunner/unittest-xml-reporting#reporting-to-a-single-file

[generalmimon]

@GreyCat:

2. Potentially looking into reworking Python invocation per se to report everything in one XML file (as many other languages do), i.e. https://github.com/xmlrunner/unittest-xml-reporting#reporting-to-a-single-file

I agree that a single XML report would be better, but separate XML files work for now. And note that Java (and maybe other languages, haven't checked) uses separate XML files too.

[generalmimon]

@GreyCat I've just added Java support in a similar way.