Integrated performance tests into the CI system
Issue/feature addressed
Implemented performance testing (!1614 (merged)) into the CI system. This merge request should be stacked with !1614 (merged).
Proposed solution
Average execution time has been added as a test metric that can be used. One performance test -- IncNavierStokesSolver_Perf_ChanFlow_3DH1D_pRef -- would become active in this solution.
This test would run 15 times on the relatively quiet runner 42 (stpancras) and fail if the average execution time was outside ~5% of the baseline.
Implementation
- New debian-bullseye-performance-build-env job -- based on default-build-env but with NEKTAR_BUILD_PERFORMANCE_TESTS_ON and other tests OFF. Runs on runner 42 (stpancras)
- New debian-bullseye-performance-build-and-test job
- When docker images are run, the hostname is now manually specified as CI_RUNNER_ID.CI_JOB_NAME (e.g. 42.debian-bullseye-performance-build-and-test). This ensures that performance tests will only run if they are on the right runner.
Tests
IncNavierStokesSolver_Perf_ChanFlow_3DH1D_pRef
Suggested reviewers
@ccantwel (UROP supervisor)
Notes
Please add any other information that could be useful for reviewers.
Checklist
-
Functions and classes, or changes to them, are documented. -
User guide/documentation is updated. -
Changelog is updated. -
Suitable tests added for new functionality. -
Contributed code is correctly formatted. (See the contributing guidelines). -
License added to any new files. -
No extraneous files have been added (e.g. compiler output or test data files).
Edited by Chris Cantwell