diff --git a/docs/src/docs/doc-versions.adoc b/docs/src/docs/doc-versions.adoc index 7206d7540..ccd436e90 100644 --- a/docs/src/docs/doc-versions.adoc +++ b/docs/src/docs/doc-versions.adoc @@ -3,37 +3,21 @@ * Latest Testerra 1 version: https://docs.testerra.io/testerra/1-latest/index.html * Latest Testerra 2 version: https://docs.testerra.io/testerra/2-latest/index.html - [cols="1,4"] |=== |Version|Link - -|1.10 -|https://docs.testerra.io/testerra/1.10/index.html - -|1.11 -|https://docs.testerra.io/testerra/1.11/index.html - -|1.12 -|https://docs.testerra.io/testerra/1.12/index.html - -|1.13 -|https://docs.testerra.io/testerra/1.13/index.html - -|2.0 -|https://docs.testerra.io/testerra/2.0/index.html - -|2.1 -|https://docs.testerra.io/testerra/2.1/index.html - -|2.2 -|https://docs.testerra.io/testerra/2.2/index.html - -|2.4 -|https://docs.testerra.io/testerra/2.4/index.html - -|2.7 -|https://docs.testerra.io/testerra/2.7/index.html +|1.10 | https://docs.testerra.io/testerra/1.10/index.html +|1.11 | https://docs.testerra.io/testerra/1.11/index.html +|1.12 | https://docs.testerra.io/testerra/1.12/index.html +|1.13 | https://docs.testerra.io/testerra/1.13/index.html +|2.0 | https://docs.testerra.io/testerra/2.0/index.html +|2.1 | https://docs.testerra.io/testerra/2.1/index.html +|2.2 | https://docs.testerra.io/testerra/2.2/index.html +|2.4 | https://docs.testerra.io/testerra/2.4/index.html +|2.6 | https://docs.testerra.io/testerra/2.5/index.html +|2.7 | https://docs.testerra.io/testerra/2.7/index.html +|2.8 | https://docs.testerra.io/testerra/2.8/index.html +|2.9 | https://docs.testerra.io/testerra/2.9/index.html |=== You will find detailed release notes at https://github.com/telekom/testerra/releases. diff --git a/docs/src/docs/reports/dashboard.adoc b/docs/src/docs/reports/dashboard.adoc index 54655136c..4e7d8ab99 100644 --- a/docs/src/docs/reports/dashboard.adoc +++ b/docs/src/docs/reports/dashboard.adoc @@ -1,22 +1,60 @@ = Dashboard View +== Overview + The dashboard view gives an overview of the last test execution. It shows the total number of executed tests and of passed, skipped or failed tests. image::report-ng-05.png[align="center", alt="Dashboard"] On the Breakdown panel is a pie chart that shows the proportions, displayed with different colors according to the test status. -Test status colors: -[cols="2,^1,4",options="header"] +Right beside the Breakdown pie chart you find another panel with the Top 3 failure types (aspects). +There are two more panels below, one displaying detailed information about test execution duration, start and end times and another panel with a list of test classes that reports how many tests did pass, fail or were skipped in the individual test classes. + +== Status definitions + +Testerra defines the following *final 4 statuses* which describe the final test result: + +[cols="1a,4",options="header"] +|=== +| Test Status | Description + +| image::status_passed.png[] +| The test was successful. + +| image::status_failed.png[] +| The test was failed. + +| image::status_skipped.png[] +| The test was not executed because a precondition failed like a configuration method, a data provider or an another test. + +| image::status_expected_failed.png[] +| The test failed as expected and the test method is annotated with <<#Fails, @Fails>> because of a known bug or problem. + |=== -| Color | Test Status | Description -| Green | Passed | The test was passed. -| Yellow | Skipped | The test was not executed, because a test it depends on failed. -| Red | Failed | The test failed. -| Dark Red | Expected Fail | The test failed, as expected e.g. because of a known bug or problem. + +Additional to the statuses above Testerra defines some more _sub statuses_: + +[cols="1a,4",options="header"] |=== +| Test Status | Description + +| image::status_retried.png[] +| The test failed at it's first try. Testerra executed that test again if the <<#_retry_analyzer, RetryAnalyzer>> was active. + +| image::status_recovered.png[] +| The test was passed at it's second (third...) execution if the <<#_retry_analyzer, RetryAnalyzer>> was active. + +| image::status_repaired.png[] +| The test was passed but it is still annotated with `@Fails`. The annotation can be removed now. + +|=== + +[NOTE] +==== +* The dashboard only shows the final statuses in the 'Breakdown' and 'Test Classes' charts. +* The 'Tests' table summarize all executed tests inclusive retried tests. Recovered and repaired tests belong to passed tests. +* The tests view shows all tests with their detailed status. +==== -Check out the <<#Fails, @Fails>> Annotation to see how to mark a test as expected to fail. -Right beside the Breakdown pie chart you find another panel with the Top 3 failure types (aspects). -There are two more panels below, one displaying detailed information about test execution duration, start and end times and another panel with a list of test classes that reports how many tests did pass, fail or were skipped in the individual test classes. diff --git a/docs/src/docs/reports/print-preview.adoc b/docs/src/docs/reports/print-preview.adoc index c262cb574..b7557b3f8 100644 --- a/docs/src/docs/reports/print-preview.adoc +++ b/docs/src/docs/reports/print-preview.adoc @@ -1,4 +1,4 @@ -= Print Preview += Print preview If you want to consolidate all report information into a single PDF file or print it directly, the "Print Report" feature can be accessed by clicking the button located at the bottom of the navigation drawer. diff --git a/docs/src/images/status_expected_failed.png b/docs/src/images/status_expected_failed.png new file mode 100644 index 000000000..d62bae376 Binary files /dev/null and b/docs/src/images/status_expected_failed.png differ diff --git a/docs/src/images/status_failed.png b/docs/src/images/status_failed.png new file mode 100644 index 000000000..1ffc4a114 Binary files /dev/null and b/docs/src/images/status_failed.png differ diff --git a/docs/src/images/status_passed.png b/docs/src/images/status_passed.png new file mode 100644 index 000000000..379c0c3ea Binary files /dev/null and b/docs/src/images/status_passed.png differ diff --git a/docs/src/images/status_recovered.png b/docs/src/images/status_recovered.png new file mode 100644 index 000000000..87ab89a80 Binary files /dev/null and b/docs/src/images/status_recovered.png differ diff --git a/docs/src/images/status_repaired.png b/docs/src/images/status_repaired.png new file mode 100644 index 000000000..6b2fadb68 Binary files /dev/null and b/docs/src/images/status_repaired.png differ diff --git a/docs/src/images/status_retried.png b/docs/src/images/status_retried.png new file mode 100644 index 000000000..4e71ea951 Binary files /dev/null and b/docs/src/images/status_retried.png differ diff --git a/docs/src/images/status_skipped.png b/docs/src/images/status_skipped.png new file mode 100644 index 000000000..c8a76949b Binary files /dev/null and b/docs/src/images/status_skipped.png differ