Test organization

Tests have a highly organized structure. The key concepts are:

• Test Collection
• Test
• Section
• Step

Test Collection

A test collection is a list of test IDs. Test collections are created for many reasons and they are mostly hidden from the user's view. Any time you see a grouping of tests in the user interface there is a test collection behind it. The most visible test collection is the grouping of tests for an actor or option.

Test collections are managed indirectly by indexing tests with the collections they belong to. Each test definition has a file collections.txt which holds the name of all collections that test is part of. The master index is created each time Toolkit is started or when the Reindex Test Kits button is pressed in the Toolkit Configuration tool.

Test

A test is an identifiable unit of testing that is indexed in Gazelle and assigned to a known actor and option. A test can be run using the Conformance Test Tool. Sometimes tests depend on other tests. When they do they reference the log files produced by the other tests. Running a test when the dependency test has not been run leads to an error.

Section

A test is comprised of one or more sections. A section is the smallest run-able element of testing. A section can be run through the Conformance Test Tool. Sometimes sections depend on other sections within the test. The running of a section generates a single log file in the External Cache.