Bug 1813054 - Add a test for our regression detection system. r=perftest-reviewers...
commite3fee0fc8e0ac1493357ac0e456895a22f2fd3d2
authorGreg Mierzwinski <gmierz2@outlook.com>
Tue, 31 Jan 2023 15:16:27 +0000 (31 15:16 +0000)
committerGreg Mierzwinski <gmierz2@outlook.com>
Tue, 31 Jan 2023 15:16:27 +0000 (31 15:16 +0000)
tree01415f1f2ee7b52a3b8727294dc8b62fc2a63ce3
parent172617b27f0df471cc4d0b3b8230627d2dbedf97
Bug 1813054 - Add a test for our regression detection system. r=perftest-reviewers,AlexandruIonescu

This patch adds a new test for our regression detection system. It produces a single constant value that we can easily change in the test manifest to trigger an alert. This is being done to in preparation for work on the entire alerting system, and to test if some unimplemented fields in the harness work (min_back_window, max_back_window, and fore_window). They all exist in the performance artifact schemas in mozilla-central, and the treeherder repo. I've also disabled the push/backstop optimizations for the regression tests so that they run on each push. It's limited to running on linux machines.

Differential Revision: https://phabricator.services.mozilla.com/D168116
taskcluster/ci/test/browsertime-desktop.yml
taskcluster/ci/test/test-sets.yml
taskcluster/gecko_taskgraph/transforms/test/other.py
testing/perfdocs/generated/raptor.rst
testing/raptor/browsertime/constant_regression_test.js [new file with mode: 0644]
testing/raptor/raptor/output.py
testing/raptor/raptor/perfdocs/config.yml
testing/raptor/raptor/raptor.ini
testing/raptor/raptor/results.py
testing/raptor/raptor/tests/custom/browsertime-regression-test.ini [new file with mode: 0644]