Config Checker

Setup

Clone the repository from https://github.com/ISISComputingGroup/ConfigChecker.

To set up the config checker to run locally on all instruments:

1. Run run_tests.bat

To set up the config checker to run on a single specified instrument:

1. Edit the call to run_tests.py in run_tests.bat to include (for example) --instrument DEMO
2. Run run_tests.bat

The first time the config checker runs, it will take a short while to clone the configurations and gui repositories locally. These are "working copies" for the config checker script.

Important: Don't point the config checker script at any repository where you have changes you want to keep - it git cleans and git resets it's working repositories whenever it switches branches!

Skipped tests

The config checker works by comparing and making assertions about the contents of:

1. The settings repository of an instrument
2. The GUI version that the instrument is using (this is determined by the contents of configurations/config_version.txt
3. The PVs that the instrument is publishing.

In cases where the above information is not accessible, some tests will be skipped. This is so that a single offline instrument would not cause the jenkins build to fail. However, it also means that an instrument needs to be up (and it's blockserver accessible over channel access) for all the tests to run.

If an instrument does not have configurations/config_version.txt, any test relying on knowing the GUI version will be skipped. Instead you will get a single failing test saying that the above file did not exist. It is possible that fixing that failure will cause a new set of failures to appear, because the tests that depend on that file can now be run as opposed to skipping.

Fixing config checker issues

Fixing config checker issues should generally be done in agreement with the instrument scientist.

Here are some possible resolutions to some failures. Not all possible failures are listed below:

Failure	Possible resolution
Config version is wrong	Check why config version and server version are different. Run upgrade script.
There should not be more than one 'config_version.txt' file in the repository.	Remove extra version.txt file.
Component contained unexpected files in its directory	Check with the scientist whether we can move these files to a data/old directory or similar.
Component did not contain the required config file	Check if this config actually loads in IBEX. If not, remove or move to data/old.
Configuration contained multiple instances of ioc	Check this configuration and remove any duplicated instances of an IOC.

Output

The config checker runs in Python's unittest framework and provides XML test reports, which can be found in test-reports/<instrument>