1 A guide to searching crash reports
2 ==================================
6 Please read the :ref:`documentation about individual crash
7 reports <Understanding crash reports>` before reading
10 The Mozilla `crash-stats <https://crash-stats.mozilla.org/>`__ site
11 provides facilities for investigating large numbers of Firefox `crash
12 reports <Understanding crash reports>`__. This guide to
13 searching through crash reports may help you locate the crash reports
14 that will help you find and fix the Firefox bug you're working on.
16 Specifically, crash-stats offers two basic functions:
19 You can search the crash reports database by over 100 criteria: crash
20 signature, date, platform, product, version, etc.
22 You can cluster the results of each search into groups using the same
25 To achieve full power and flexibility requires a good understanding of
26 both of these functions. Search is easy to understand, but the grouping
27 capabilities are easy to overlook.
35 You can get to the `search
36 page <https://crash-stats.mozilla.org/search/?product=Firefox&_dont_run=1>`__
37 by clicking on the "Super Search" link near the toolbar at the top right
38 of any page in crash-stats. This brings up a search form like the one in
39 the following screenshot.
41 |Search in crash-stats|
43 Fields are provided for four common search criteria: product, version,
44 platform, and process type. The product field is pre-populated with
45 "Firefox" because that is a common case. As the fine print says, the
46 default date range is the past week.
48 The default search: Signature facet
49 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
51 If you click on the "Search" button, you will get
52 `results <https://crash-stats.mozilla.org/search/?product=Firefox&_sort=-date&_facets=signature&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#facet-signature>`__
53 like the ones in the following screenshot.
55 |Results of a default search in crash-stats|
57 By default, the "Signature facet" tab is selected. ("Facet" is a term
58 that means "group".) In these results, the found crash reports are
59 grouped according to crash signature and ranked by group size. The
60 columns show each group's rank, signature, size (both a count and a
61 proportion of matching crash reports), and finally a list of bugs that
62 have been marked as relating to this signature.
64 The numbers are large because this search matched all Firefox crash
65 reports from the past seven days. The first group has over 100,000 crash
66 reports, which accounts for 7.77% of all matching crashes. This
67 indicates there are over 1.3 million crash reports matching this search.
69 You can reorder the groups in various ways by clicking on the column
70 headers. The links within the results do the following things.
72 - The first link in each "Signature" column cell links to a signature
73 report, which contains additional details about crash reports with
75 - The "Add term" link in each "Signature" column cell lets you perform
76 a narrower subsequent search among crash reports with that signature.
77 - The links in each "Bugs" column cell link to bug reports in Bugzilla.
79 The default search: Crash reports
80 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
82 If you switch to the "Crash Reports" tab you will see
83 `results <https://crash-stats.mozilla.org/search/?product=Firefox&_sort=-date&_facets=signature&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#crash-reports>`__
84 like the ones in the following screenshot.
86 |Results of a default search in crash-stats (crash reports tab)|
88 This is a list of all the individual crash reports that match the search
89 criteria. If the number of matches is large -- in this case it exceeds
90 1.3 million, just as we saw in the "Signature facet" tab -- the results
91 will be spread across multiple pages, which you can visit by clicking
92 the links at the top right of the tab.
94 The links within the results do the following things.
96 - The link in each "Crash ID" column cell links to an individual crash
98 - The links in each "Signature" column cell have the same effect that
99 they did in the "Signature facet" tab.
100 - The links in the remaining column cells also let you perform a
101 narrower subsequent search with that link's value added to the search
107 You can add criteria to perform a narrower search. For example, to
108 perform a search for all Mac crash reports that occurred while
109 JavaScript garbage collection was running, do the following.
111 - Add "Mac OS X" to the "Platform" field.
112 - Select "New line", and then choose a field ("is garbage collecting")
113 and an operator ("is true"). The operators available for each field
116 With these criteria added the search form looks like the following
119 |crash-stats Super Search form with additional criteria|
121 After clicking on "Search" we get
122 `results <https://crash-stats.mozilla.org/search/?is_garbage_collecting=__true__&product=Firefox&platform=Mac%20OS%20X&_sort=-date&_facets=signature&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform>`__
123 like those in the following screenshot.
125 |Results of a narrower search in crash-stats|
127 The number of crash reports matching this search is in the thousands,
128 i.e. much smaller than the previous search.
133 The "proto signature" field is just the raw unprocessed crash stack
134 concatenated together.
136 You can do things like:
138 - Search for crashes where the signature is Foo, and the proto
139 signature contains Bar. This is helpful if you have a fairly generic
140 signature and you want to see how many of them are a particular case
141 of it that you've come across. Or instead of a signature Foo, a moz
142 crash reason or something else.
143 - Use it as a facet. This lets you skim the full signatures of crashes
144 at a glance, bucketed together a bit. Note that because the proto
145 signature includes the entire signature, things aren't grouped all
151 In the previous section we saw one example of grouping, in the
152 "Signature facet" tab that is shown by default. But there are many other
153 interesting ways to group searches.
155 Facets in the search form
156 ~~~~~~~~~~~~~~~~~~~~~~~~~
158 To do a search with non-signature grouping first click on the "More
159 options..." text, which reveals the additional fields shown in the
160 following screenshot.
162 |crash-stats Super Search form with different facets|
164 (The "Show columns" and "Sort by" fields are straightforward. They apply
165 to the "Crash reports" tab of any search results, and are not related to
168 The "Facet on" field is the one that controls grouping. By default, it
169 contains the value "signature", which explains why we saw a "Signature
170 facet" tab in the earlier search results. But we can change the values
171 in this field and get different facet tabs in the search results.
176 For example, if we start with a default search for all Firefox crashes
177 in the past week, but then replace the "signature" facet with "platform"
178 and "moz crash reason", we get search results with two facet tabs. The
179 first of these is a "Platform facet" tab, with
180 `results <https://crash-stats.mozilla.org/search/?product=Firefox&_sort=-date&_facets=platform&_facets=moz_crash_reason&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#facet-platform>`__
181 like those shown in the following screenshot.
183 |Results of a faceted search in crash-stats|
185 This has the same columns as the "Signature facet" tab we saw earlier,
186 except for the "Bugs" column, because that is a special column that only
187 applies to the signature facet. This tab shows the distribution of crash
188 reports across the various platforms. Crash reports always include a
189 platform field (though it may be empty if something has gone wrong) and
190 so the percentages add up to 100.
192 Grouping by "moz crash reason"
193 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
195 The second facet tab is a "Moz crash reason facet" tab, with
196 `results <https://crash-stats.mozilla.org/search/?product=Firefox&_sort=-date&_facets=platform&_facets=moz_crash_reason&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#facet-moz_crash_reason>`__
197 like those shown in the following screenshot.
199 |Results of a faceted search in crash-stats (moz crash reason tab)|
201 This immediately shows which ``MOZ_CRASH`` calls are being hit
202 frequently by users. Only a subset of crash reports have the "moz crash
203 reason" field -- those that crashed due to hitting a ``MOZ_CRASH`` call
204 -- so all crashes that lack that field are omitted from this tab. For
205 that reason, the percentages do not add up to 100.
207 An example of less useful grouping
208 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
210 The usefulness of grouping varies from field to field. In particular,
211 fields that can have many possible values (such as numeric fields) often
212 don't group well. For example, if we do a default search grouped by
214 `results <https://crash-stats.mozilla.org/search/?product=Firefox&_sort=-date&_facets=uptime&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#facet-uptime>`__
215 like those in the following screenshot.
217 |Results of a faceted search in crash-stats (uptime)|
219 In this example the top 10 groups account for less than 12% of all
220 crashes, and there is an extremely long tail. These results would be
221 improved by using numeric ranges instead of individual values, but
222 unfortunately that isn't supported.
227 The combination of searching and grouping is powerful. Searches find
228 crash reports that match particular criteria, and grouping organizes
229 those crash reports into interesting groups.
231 When a search is performed, the page's URL is updated to include the
232 search parameters. This means that the results of any search can be
233 easily shared by copying and pasting the page's URL.
235 To become an expert at searching and grouping requires understanding the
236 full range of the 100+ fields available for searching and grouping. One
237 way to learn about them is to read lots of individual crash reports;
238 note that all fields shown in the Details tab of an individual crash
239 report have a tool-tip that indicates its key for search. Alternatively,
240 you can browse the `complete
241 list <https://crash-stats.mozilla.org/documentation/supersearch/api/#section-filters>`__.
243 There is also an API through which searches can be performed
244 programmatically. See the `API
245 documentation <https://crash-stats.mozilla.org/documentation/supersearch/>`__
246 for full details; note that it uses the term "aggregation" for
249 .. |Search in crash-stats| image:: img/super-search-form.png
250 .. |Results of a default search in crash-stats| image:: img/default-search-results.png
251 .. |Results of a default search in crash-stats (crash reports tab)| image:: img/default-search-results2.png
252 .. |crash-stats Super Search form with additional criteria| image:: img/super-search-form2.png
253 .. |Results of a narrower search in crash-stats| image:: img/narrower-search-results.png
254 .. |crash-stats Super Search form with different facets| image:: img/super-search-form3.png
255 .. |Results of a faceted search in crash-stats| image:: img/facet-search-results.png
256 .. |Results of a faceted search in crash-stats (moz crash reason tab)| image:: img/facet-search-results2.png
257 .. |Results of a faceted search in crash-stats (uptime)| image:: img/facet-search-results3.png