Running Tests and Continuous Integration

How to Run Tests

All IDAES tests are written using pytest (documentation), which automatically searches through the project folders to find any files marked as test files. The following terminal command can be used from the idaes-pse directory to run all the tests in the IDAES project (Warning: this will take around an hour):

>>> pytest

Running Specific Tests

As the full suite of IDAES tests takes a long time to run, it is generally a good idea to only run those tests that are important for what you are currently working on. pytest allows you to run specific sub-sets of the test suite, such as by indicating specific sub-folders to test or by specifying pytest.marks. These approaches can be combined, to allow running tests in specific marks in sub-folder.

Running tests by path:

>>> pytest PATH

Running test by pytest.mark

>>> pytest -m unit  # Only run tests with the "unit" mark
>>> pytest -m "not integration"  # Run all tests that do not have the "integration" mark

When to Run Tests

Developers should run the unit and component tests at a minimum before creating a Pull Request in order to avoid test failures on the PR (this will take ~20 minutes). A PR must pass all tests (including integration tests) before it will be merged, thus it is best to identify these early. It is also a good idea to run the integration tests unless you are certain your changes will not affect the examples - these take a long time to run however, so be judicious when running them.

Developers are also encouraged to run tests of the code they are working on more frequently, and to even develop tests in tandem with the code.

Automated Testing (Continuous Integration)

IDAES uses automated testing via GitHub Actions to run the test suite on all Pull Requests to the main repository, as well as to run regularly scheduled tests of the current code.

A document describing the IDAES check suite in the context of Continuous Integration is available here.