Testing

Mantid Imaging uses unit tests, static analysis and GUI approval testing

The test suite can be run using our makefile target. Note this does not include the system and screenshot tests:

make check

On Windows, you may need to install a Windows version of GNU Make.:

mamba install make

Tests are run automatically on pull requests on GitHub using actions, but should also be run during development. The full suite of unit tests can run very slowly on a Windows machine. We recommend selecting only relevant tests to run locally when developing on Windows.

See the Makefile for the underlying commands if you need to run them with other options.

Pre-commit

Mantid Imaging uses pre-commit to run pre-commit hooks. This must be manually enabled by running:

pre-commit install

Once this has been completed, the hooks should execute with every commit.

Unit testing

Unit tests can be run using pytest, e.g.:

make pytest

For options such as running a subset of tests, see PyTest Docs

Static analysis

Mantid Imaging uses mypy, ruff and yapf for static analysis and formatting. They are run by make check, or can be run individually, e.g. make mypy.

GUI screenshot testing

Mantid Imaging uses Applitools Eyes for GUI approval testing. Screenshots of windows are uploaded and compared to known good baseline images. This is run in the github action on pull requests.

The tests can be run locally, where the screenshots will be writen to a directory:

make test-screenshots

See the Makefile for the underlying command used. You may want to run it with a specific output directory by changing APPLITOOLS_IMAGE_DIR, or to upload to the Applitools API by setting a APPLITOOLS_API_KEY (can be found in the Applitools web interface).

When these tests are run from the automated tests differences between uploaded and baseline images can be examined and approved or rejected from the Applitools web interface.

GUI system tests

GUI system tests run work flows in Mantid Imaging in a ‘realistic’ way, where possible by using QTest methods to emulate mouse and keyboard actions. They use the same data files as the GUI screenshot tests. These take several minutes to run (longer on Windows) and so must be explicitly requested.:

make test-system

Logging

Logging can be controlled using the QSettings configuration file .config/mantidproject/Mantid Imaging.conf on Linux or the equivalent registry keys on windows (See QSettings). For example:

[logging]
log_level=DEBUG
log_dir=/tmp/mantid_imaging_logs
performance_log=true

Benchmarking and profiling

Mantid imaging has some utilities to help with benchmarking and profiling.

The context managers ExecutionTimer and ExecutionProfiler can be used to wrap some lines of code to record and log its run time. For example:

with ExecutionProfiler(msg="a_slow_function()"):
    a_slow_function()

will record and log a profile of function calls and times within a_slow_function().