Report warnings

[jyutzler]

In a few places, we want to report warnings. It is unclear to me how to do this through the interface.

• Broken RTree trigger
• If accepted, require extents for feature tables

[lgoltz]

Currently it is only possible to pass, fail or skip a test.

Warnings can be logged only in the application log.

[jyutzler]

I'd like to escalate this. The inability to post a warning has a significant impact in our ability to inform the users of these tests.

[ghobona]

@keshav-nangare @dstenger There is a log.xml file that is generated by each ETS. Could we use that to log warnings into?

[dstenger]

If we write the warning into the log file it is probably hard for non-technical users to find them.

@jyutzler What is the exact purpose of the warnings? Shall they be displayed in the HTML report?

[jyutzler]

Yes, I would certainly like for warnings to be displayed in the HTML report. I wouldn't want a GeoPackage flagged with one or more warnings to fail certification, but I also would want responsible organizations to strive to achieve GeoPackages that don't have warnings.

[dstenger]

I propose to use a skip result for warnings. A skipped test does not cause an implementation to fail the test suite but a skip is also not a successfully passed test.
The message of the skip might be something like: "WARNING: Some text."

[dstenger]

@jyutzler What do you think of this proposal?

[jyutzler]

While I do think it would be useful to report on the reason a test was skipped (not requested, no relevant data, etc.), I think this is semantically quite different from a warning. This approach will not allow us to express the message we want to express.

[dstenger]

Currently, it is not possible to configure a text when a test passes. Would it help you if we enable that? Then you could configure something like "WARNING: Some text." for your specific case.

Can you please give us an exact example including the actual message?

[jyutzler]

RTree:

WARNING: the trigger rtree_<t>_<c>_update3 is invalid. Please see http://www.geopackage.org/spec121/#_define_triggers_to_maintain_spatial_index_values for more information.

Once we have proven the concept, we will provide additional instances.

[dstenger]

@keshav-nangare Can you please check if such messages (#72 (comment)) can be printed by the HTML report of TEAM Engine?
I propose to rename field "Reason of Failure:" to "Details:" and to print the message there. This should be discussed with @ghobona.

[ghobona]

@dstenger @keshav-nangare Yes, I am happy for you to rename the field to "Details".

[dstenger]

@keshav-nangare

Can you please check if SoftAssert of TestNG framework can be used?

Steps for implementation:

1. Enhance TEAM Engine to be capable to print warnings/ SoftAssert and rename column "Reason of Failure:" to "Details:".
2. Enhance this test suite to throw a warning (demo).