4 Taskgraph builds a directed acyclic graph, where each node in the graph
5 represents a task, and each edge represents the dependency relationships
8 See Taskgraph's `graph generation documentation`_ for more details.
13 The decision task is the first task created when a new graph begins. It is
14 responsible for creating the rest of the task graph.
16 The decision task for pushes is defined in-tree, in ``.taskcluster.yml``. That
17 task description invokes ``mach taskcluster decision`` with some metadata about
18 the push. That mach command determines the required graph of tasks to submit,
19 then calls the Taskcluster API to create the tasks.
23 ``mach taskgraph decision`` is *not* meant to be invoked by humans.
24 Instead, follow `this guide`_ (prepending ``mach`` to all commands) to
25 invoke Taskgraph locally without submitting anything to Taskcluster.
32 Generation starts with "kinds". These are yaml files that denote groupings of
33 tasks that are loosely related to one another. For example, in Gecko there are
34 ``build`` and ``test`` kinds. Each kind has its own directory under
35 `taskcluster/ci`_ which contains a ``kind.yml`` file.
37 For more information on kinds, see Taskgraph's `kind documentation`_. For a
38 list of available kinds in ``mozilla-central``, see the :doc:`kinds reference
39 </taskcluster/kinds>`.
44 Next, a "loader" is responsible for parsing each ``kind.yml`` file and turning
45 it into an initial set of tasks. Loaders will always parse kinds in an ordering
46 satisfying the ``kind-dependencies`` key.
48 See Taskgraph's `loader documentation`_ for more details.
55 After the initial set of tasks are loaded, transformations are applied to each
56 task. Transforms are Python functions that take a generator of tasks as input,
57 and yields a generator of tasks as output. Possibly modifying, adding or removing
60 See Taskgrpah's `transforms documentation`_ for more details on transforms, or
61 the :doc:`transforms section </taskcluster/transforms/index>` for information
62 on some of the transforms available in ``mozilla-central``.
67 After transformation is finished, the `target_tasks`_ module filters out any
68 tasks that aren't applicable to the current :doc:`parameter set
69 </taskcluster/parameters>`.
74 After the "target graph" is generated, an optimization process looks to remove
75 or replace unnecessary tasks in the graph. For instance, a task may be removed
76 if the push doesn't modify any of the files that could affect it.
78 See Taskgraph's `optimization documentation`_ for more details, or the
79 :doc:`optimization strategies <optimization/index>` available in
86 A few components of tasks are only known at the very end of the decision task
87 -- just before the ``queue.createTask`` call is made. These are specified
88 using simple parameterized values, as follows:
90 ``{"relative-datestamp": "certain number of seconds/hours/days/years"}``
91 Objects of this form will be replaced with an offset from the current time
92 just before the ``queue.createTask`` call is made. For example, an
93 artifact expiration might be specified as ``{"relative-datestamp": "1
96 ``{"task-reference": "string containing <dep-name>"}``
97 The task definition may contain "task references" of this form. These will
98 be replaced during the optimization step, with the appropriate taskId for
99 the named dependency substituted for ``<dep-name>`` in the string.
100 Additionally, `decision` and `self` can be used a dependency names to refer
101 to the decision task, and the task itself. Multiple labels may be
102 substituted in a single string, and ``<<>`` can be used to escape a literal
105 ``{"artifact-reference": "..<dep-name/artifact/name>.."}``
106 Similar to a ``task-reference``, but this substitutes a URL to the queue's
107 ``getLatestArtifact`` API method (for which a GET will redirect to the
110 .. _taskgraph-graph-config:
115 There are several configuration settings that are pertain to the entire
116 taskgraph. These are specified in :file:`config.yml` at the root of the
117 taskgraph configuration (typically :file:`taskcluster/ci/`). The available
118 settings are documented inline in `taskcluster/gecko_taskgraph/config.py
119 <https://searchfox.org/mozilla-central/source/taskcluster/gecko_taskgraph/config.py>`_.
121 .. _taskgraph-trust-domain:
126 Action Tasks are tasks which perform an action based on a manual trigger (e.g
127 clicking a button in Treeherder). Actions are how it is possible to retrigger
130 For more information, see Taskgraph's `actions documentation`_.
132 .. _graph generation documentation: https://taskcluster-taskgraph.readthedocs.io/en/latest/concepts/task-graphs.html
133 .. _this guide: https://taskcluster-taskgraph.readthedocs.io/en/latest/howto/run-locally.html
134 .. _taskcluster/ci: https://searchfox.org/mozilla-central/source/taskcluster/ci
135 .. _kind documentation: https://taskcluster-taskgraph.readthedocs.io/en/latest/concepts/kind.html
136 .. _loader documentation: https://taskcluster-taskgraph.readthedocs.io/en/latest/concepts/loading.html
137 .. _transforms documentation: https://taskcluster-taskgraph.readthedocs.io/en/latest/concepts/transforms.html
139 .. _optimization documentation: https://taskcluster-taskgraph.readthedocs.io/en/latest/concepts/optimization.html
140 .. _actions documentation: https://taskcluster-taskgraph.readthedocs.io/en/latest/howto/create-actions.html