Integrated performance tests into the CI system

Merged Tom Gordon requested to merge twg21/nektar:feature/CI-perf-monitoring into master

Issue/feature addressed

Implemented performance testing (!1614 (merged)) into the CI system. This merge request should be stacked with !1614 (merged).

Proposed solution

Average execution time has been added as a test metric that can be used. One performance test -- IncNavierStokesSolver_Perf_ChanFlow_3DH1D_pRef -- would become active in this solution.

This test would run 15 times on the relatively quiet runner 42 (stpancras) and fail if the average execution time was outside ~5% of the baseline.

Implementation

  • New debian-bullseye-performance-build-env job -- based on default-build-env but with NEKTAR_BUILD_PERFORMANCE_TESTS_ON and other tests OFF. Runs on runner 42 (stpancras)
  • New debian-bullseye-performance-build-and-test job
  • When docker images are run, the hostname is now manually specified as CI_RUNNER_ID.CI_JOB_NAME (e.g. 42.debian-bullseye-performance-build-and-test). This ensures that performance tests will only run if they are on the right runner.

Tests

IncNavierStokesSolver_Perf_ChanFlow_3DH1D_pRef

Suggested reviewers

@ccantwel (UROP supervisor)

Notes

Please add any other information that could be useful for reviewers.

Checklist

  • Functions and classes, or changes to them, are documented.
  • User guide/documentation is updated.
  • Changelog is updated.
  • Suitable tests added for new functionality.
  • Contributed code is correctly formatted. (See the contributing guidelines).
  • License added to any new files.
  • No extraneous files have been added (e.g. compiler output or test data files).
Edited by Chris Cantwell

Merge request reports