dom/docs/workersAndStorage/PerformanceTesting.rst

   1 ================================
   2 Service Worker Performance Tests
   3 ================================
   4
   5 Our performance tests are mochitests running in the `mozperftest
   6 <https://firefox-source-docs.mozilla.org/testing/perfdocs/mozperftest.html>`_
   7 harness.  Tests reside under `dom/serviceworkers/test/performance
   8 <https://searchfox.org/mozilla-central/source/dom/serviceworkers/test/performance>`_,
   9 itemized in perftest.toml.  Beyond the standard mochitest machinery,
  10 performance tests define a ``perfMetadata`` variable at the top level, and call:
  11
  12 ``info("perfMetrics", JSON.stringify(metrics));``
  13
  14 to report their results, where *metrics* is a map from testpoint name to scalar
  15 numeric value.  See the `performance scripts documentation
  16 <https://firefox-source-docs.mozilla.org/testing/perfdocs/writing.html#mochitest>`_
  17 for more.
  18
  19 They can be run via mach perftest, or as normal mochitests via mach test.
  20 (Currently we can’t run the full manifest, see `bug 1865852
  21 <https://bugzilla.mozilla.org/show_bug.cgi?id=1865852>`_.)
  22
  23 Adding new tests
  24 ================
  25
  26 Add files to `perftest.toml
  27 <https://searchfox.org/mozilla-central/source/dom/serviceworkers/test/performance/perftest.toml>`_
  28 as usual for mochitests.
  29
  30 Modify linux.yml, macosx.yml, and windows.yml under `taskcluster/ci/perftest
  31 <https://searchfox.org/mozilla-central/source/taskcluster/ci/perftest>`_.
  32 Currently, each test needs to be added individually to the run command (`here
  33 <https://searchfox.org/mozilla-central/rev/91cc8848427fdbbeb324e6ca56a0d08d32d3c308/taskcluster/ci/perftest/linux.yml#121-149>`_,
  34 for example).  kind.yml can be ignored–it provides some defaults.
  35
  36 Modify the documentation using:
  37
  38 ``$ ./mach lint -l perfdocs . --fix --warnings --outgoing``
  39
  40 There's currently a `bug <https://bugzilla.mozilla.org/show_bug.cgi?id=1872613>`_
  41 which will likely cause the command to fail.  Running it a second time should
  42 succeed.
  43
  44 Staging tests in try jobs
  45 =========================
  46
  47 ``$ ./mach try fuzzy --full``
  48
  49 Look for ``‘service-worker`` to find things like:
  50
  51 | >perftest-windows-service-worker
  52 | >perftest-macosx-service-worker
  53 | >perftest-linux-service-worker
  54 |
  55
  56 Results
  57 =======
  58
  59 Results can be found in treeherder on `mozilla-central
  60 <https://treeherder.mozilla.org/jobs?repo=mozilla-central&searchStr=perftest>`_
  61 and `autoland
  62 <https://treeherder.mozilla.org/jobs?repo=autoland&searchStr=perftest>`_.  Look
  63 for linux-sw, macosx-sw, and win-sw (`example
  64 <https://treeherder.mozilla.org/perfherder/graphs?series=mozilla-central,4967140,1,15&selected=4967140,1814245176>`_).
  65 These symbol names are defined in the .yml files under taskcluster/ci/perftest.
  66
  67 Contacts
  68 ========
  69 | `Joshua Marshall <https://people.mozilla.org/p/jmarshall>`_   (DOM LWS)
  70 | `Gregory Mierzwinski <https://people.mozilla.org/p/sparky>`_  (Performance Tools)