Backed out changeset bd51857879db (bug 1862957) for causing WakeLock related failures...
[gecko.git] / xpcom / base / nsCycleCollector.cpp
blob001b1c9e9187bea2ab6c6d5b69aeed6e9bd9ef05
1 /* -*- Mode: C++; tab-width: 8; indent-tabs-mode: nil; c-basic-offset: 2 -*- */
2 /* vim: set ts=8 sts=2 et sw=2 tw=80: */
3 /* This Source Code Form is subject to the terms of the Mozilla Public
4 * License, v. 2.0. If a copy of the MPL was not distributed with this
5 * file, You can obtain one at http://mozilla.org/MPL/2.0/. */
7 //
8 // This file implements a garbage-cycle collector based on the paper
9 //
10 // Concurrent Cycle Collection in Reference Counted Systems
11 // Bacon & Rajan (2001), ECOOP 2001 / Springer LNCS vol 2072
13 // We are not using the concurrent or acyclic cases of that paper; so
14 // the green, red and orange colors are not used.
16 // The collector is based on tracking pointers of four colors:
18 // Black nodes are definitely live. If we ever determine a node is
19 // black, it's ok to forget about, drop from our records.
21 // White nodes are definitely garbage cycles. Once we finish with our
22 // scanning, we unlink all the white nodes and expect that by
23 // unlinking them they will self-destruct (since a garbage cycle is
24 // only keeping itself alive with internal links, by definition).
26 // Snow-white is an addition to the original algorithm. A snow-white node
27 // has reference count zero and is just waiting for deletion.
29 // Grey nodes are being scanned. Nodes that turn grey will turn
30 // either black if we determine that they're live, or white if we
31 // determine that they're a garbage cycle. After the main collection
32 // algorithm there should be no grey nodes.
34 // Purple nodes are *candidates* for being scanned. They are nodes we
35 // haven't begun scanning yet because they're not old enough, or we're
36 // still partway through the algorithm.
38 // XPCOM objects participating in garbage-cycle collection are obliged
39 // to inform us when they ought to turn purple; that is, when their
40 // refcount transitions from N+1 -> N, for nonzero N. Furthermore we
41 // require that *after* an XPCOM object has informed us of turning
42 // purple, they will tell us when they either transition back to being
43 // black (incremented refcount) or are ultimately deleted.
45 // Incremental cycle collection
47 // Beyond the simple state machine required to implement incremental
48 // collection, the CC needs to be able to compensate for things the browser
49 // is doing during the collection. There are two kinds of problems. For each
50 // of these, there are two cases to deal with: purple-buffered C++ objects
51 // and JS objects.
53 // The first problem is that an object in the CC's graph can become garbage.
54 // This is bad because the CC touches the objects in its graph at every
55 // stage of its operation.
57 // All cycle collected C++ objects that die during a cycle collection
58 // will end up actually getting deleted by the SnowWhiteKiller. Before
59 // the SWK deletes an object, it checks if an ICC is running, and if so,
60 // if the object is in the graph. If it is, the CC clears mPointer and
61 // mParticipant so it does not point to the raw object any more. Because
62 // objects could die any time the CC returns to the mutator, any time the CC
63 // accesses a PtrInfo it must perform a null check on mParticipant to
64 // ensure the object has not gone away.
66 // JS objects don't always run finalizers, so the CC can't remove them from
67 // the graph when they die. Fortunately, JS objects can only die during a GC,
68 // so if a GC is begun during an ICC, the browser synchronously finishes off
69 // the ICC, which clears the entire CC graph. If the GC and CC are scheduled
70 // properly, this should be rare.
72 // The second problem is that objects in the graph can be changed, say by
73 // being addrefed or released, or by having a field updated, after the object
74 // has been added to the graph. The problem is that ICC can miss a newly
75 // created reference to an object, and end up unlinking an object that is
76 // actually alive.
78 // The basic idea of the solution, from "An on-the-fly Reference Counting
79 // Garbage Collector for Java" by Levanoni and Petrank, is to notice if an
80 // object has had an additional reference to it created during the collection,
81 // and if so, don't collect it during the current collection. This avoids having
82 // to rerun the scan as in Bacon & Rajan 2001.
84 // For cycle collected C++ objects, we modify AddRef to place the object in
85 // the purple buffer, in addition to Release. Then, in the CC, we treat any
86 // objects in the purple buffer as being alive, after graph building has
87 // completed. Because they are in the purple buffer, they will be suspected
88 // in the next CC, so there's no danger of leaks. This is imprecise, because
89 // we will treat as live an object that has been Released but not AddRefed
90 // during graph building, but that's probably rare enough that the additional
91 // bookkeeping overhead is not worthwhile.
93 // For JS objects, the cycle collector is only looking at gray objects. If a
94 // gray object is touched during ICC, it will be made black by UnmarkGray.
95 // Thus, if a JS object has become black during the ICC, we treat it as live.
96 // Merged JS zones have to be handled specially: we scan all zone globals.
97 // If any are black, we treat the zone as being black.
99 // Safety
101 // An XPCOM object is either scan-safe or scan-unsafe, purple-safe or
102 // purple-unsafe.
104 // An nsISupports object is scan-safe if:
106 // - It can be QI'ed to |nsXPCOMCycleCollectionParticipant|, though
107 // this operation loses ISupports identity (like nsIClassInfo).
108 // - Additionally, the operation |traverse| on the resulting
109 // nsXPCOMCycleCollectionParticipant does not cause *any* refcount
110 // adjustment to occur (no AddRef / Release calls).
112 // A non-nsISupports ("native") object is scan-safe by explicitly
113 // providing its nsCycleCollectionParticipant.
115 // An object is purple-safe if it satisfies the following properties:
117 // - The object is scan-safe.
119 // When we receive a pointer |ptr| via
120 // |nsCycleCollector::suspect(ptr)|, we assume it is purple-safe. We
121 // can check the scan-safety, but have no way to ensure the
122 // purple-safety; objects must obey, or else the entire system falls
123 // apart. Don't involve an object in this scheme if you can't
124 // guarantee its purple-safety. The easiest way to ensure that an
125 // object is purple-safe is to use nsCycleCollectingAutoRefCnt.
127 // When we have a scannable set of purple nodes ready, we begin
128 // our walks. During the walks, the nodes we |traverse| should only
129 // feed us more scan-safe nodes, and should not adjust the refcounts
130 // of those nodes.
132 // We do not |AddRef| or |Release| any objects during scanning. We
133 // rely on the purple-safety of the roots that call |suspect| to
134 // hold, such that we will clear the pointer from the purple buffer
135 // entry to the object before it is destroyed. The pointers that are
136 // merely scan-safe we hold only for the duration of scanning, and
137 // there should be no objects released from the scan-safe set during
138 // the scan.
140 // We *do* call |Root| and |Unroot| on every white object, on
141 // either side of the calls to |Unlink|. This keeps the set of white
142 // objects alive during the unlinking.
145 #if !defined(__MINGW32__)
146 # ifdef WIN32
147 # include <crtdbg.h>
148 # include <errno.h>
149 # endif
150 #endif
152 #include "base/process_util.h"
154 #include "mozilla/ArrayUtils.h"
155 #include "mozilla/AutoRestore.h"
156 #include "mozilla/CycleCollectedJSContext.h"
157 #include "mozilla/CycleCollectedJSRuntime.h"
158 #include "mozilla/DebugOnly.h"
159 #include "mozilla/HashFunctions.h"
160 #include "mozilla/HashTable.h"
161 #include "mozilla/HoldDropJSObjects.h"
162 /* This must occur *after* base/process_util.h to avoid typedefs conflicts. */
163 #include <stdint.h>
164 #include <stdio.h>
166 #include <utility>
168 #include "js/SliceBudget.h"
169 #include "mozilla/Attributes.h"
170 #include "mozilla/Likely.h"
171 #include "mozilla/LinkedList.h"
172 #include "mozilla/MemoryReporting.h"
173 #include "mozilla/MruCache.h"
174 #include "mozilla/PoisonIOInterposer.h"
175 #include "mozilla/ProfilerLabels.h"
176 #include "mozilla/SegmentedVector.h"
177 #include "mozilla/Telemetry.h"
178 #include "mozilla/ThreadLocal.h"
179 #include "mozilla/UniquePtr.h"
180 #include "nsCycleCollectionNoteRootCallback.h"
181 #include "nsCycleCollectionParticipant.h"
182 #include "nsCycleCollector.h"
183 #include "nsDeque.h"
184 #include "nsDumpUtils.h"
185 #include "nsExceptionHandler.h"
186 #include "nsIConsoleService.h"
187 #include "nsICycleCollectorListener.h"
188 #include "nsIFile.h"
189 #include "nsIMemoryReporter.h"
190 #include "nsISerialEventTarget.h"
191 #include "nsPrintfCString.h"
192 #include "nsTArray.h"
193 #include "nsThreadUtils.h"
194 #include "nsXULAppAPI.h"
195 #include "prenv.h"
196 #include "xpcpublic.h"
198 using namespace mozilla;
200 struct NurseryPurpleBufferEntry {
201 void* mPtr;
202 nsCycleCollectionParticipant* mParticipant;
203 nsCycleCollectingAutoRefCnt* mRefCnt;
206 #define NURSERY_PURPLE_BUFFER_SIZE 2048
207 bool gNurseryPurpleBufferEnabled = true;
208 NurseryPurpleBufferEntry gNurseryPurpleBufferEntry[NURSERY_PURPLE_BUFFER_SIZE];
209 uint32_t gNurseryPurpleBufferEntryCount = 0;
211 void ClearNurseryPurpleBuffer();
213 static void SuspectUsingNurseryPurpleBuffer(
214 void* aPtr, nsCycleCollectionParticipant* aCp,
215 nsCycleCollectingAutoRefCnt* aRefCnt) {
216 MOZ_ASSERT(NS_IsMainThread(), "Wrong thread!");
217 MOZ_ASSERT(gNurseryPurpleBufferEnabled);
218 if (gNurseryPurpleBufferEntryCount == NURSERY_PURPLE_BUFFER_SIZE) {
219 ClearNurseryPurpleBuffer();
222 gNurseryPurpleBufferEntry[gNurseryPurpleBufferEntryCount] = {aPtr, aCp,
223 aRefCnt};
224 ++gNurseryPurpleBufferEntryCount;
227 // #define COLLECT_TIME_DEBUG
229 // Enable assertions that are useful for diagnosing errors in graph
230 // construction.
231 // #define DEBUG_CC_GRAPH
233 #define DEFAULT_SHUTDOWN_COLLECTIONS 5
235 // One to do the freeing, then another to detect there is no more work to do.
236 #define NORMAL_SHUTDOWN_COLLECTIONS 2
238 // Cycle collector environment variables
240 // MOZ_CC_LOG_ALL: If defined, always log cycle collector heaps.
242 // MOZ_CC_LOG_SHUTDOWN: If defined, log cycle collector heaps at shutdown.
244 // MOZ_CC_LOG_SHUTDOWN_SKIP: If set to a non-negative integer value n, then
245 // skip logging for the first n shutdown CCs. This implies MOZ_CC_LOG_SHUTDOWN.
246 // The first log or two are much larger than the rest, so it can be useful to
247 // reduce the total size of logs if you know already that the initial logs
248 // aren't interesting.
250 // MOZ_CC_LOG_THREAD: If set to "main", only automatically log main thread
251 // CCs. If set to "worker", only automatically log worker CCs. If set to "all",
252 // log either. The default value is "all". This must be used with either
253 // MOZ_CC_LOG_ALL or MOZ_CC_LOG_SHUTDOWN for it to do anything.
255 // MOZ_CC_LOG_PROCESS: If set to "main", only automatically log main process
256 // CCs. If set to "content", only automatically log tab CCs. If set to "all",
257 // log everything. The default value is "all". This must be used with either
258 // MOZ_CC_LOG_ALL or MOZ_CC_LOG_SHUTDOWN for it to do anything.
260 // MOZ_CC_ALL_TRACES: If set to "all", any cycle collector
261 // logging done will be WantAllTraces, which disables
262 // various cycle collector optimizations to give a fuller picture of
263 // the heap. If set to "shutdown", only shutdown logging will be WantAllTraces.
264 // The default is none.
266 // MOZ_CC_RUN_DURING_SHUTDOWN: In non-DEBUG or builds, if this is set,
267 // run cycle collections at shutdown.
269 // MOZ_CC_LOG_DIRECTORY: The directory in which logs are placed (such as
270 // logs from MOZ_CC_LOG_ALL and MOZ_CC_LOG_SHUTDOWN, or other uses
271 // of nsICycleCollectorListener)
273 // Various parameters of this collector can be tuned using environment
274 // variables.
276 struct nsCycleCollectorParams {
277 bool mLogAll;
278 bool mLogShutdown;
279 bool mAllTracesAll;
280 bool mAllTracesShutdown;
281 bool mLogThisThread;
282 int32_t mLogShutdownSkip = 0;
284 nsCycleCollectorParams()
285 : mLogAll(PR_GetEnv("MOZ_CC_LOG_ALL") != nullptr),
286 mLogShutdown(PR_GetEnv("MOZ_CC_LOG_SHUTDOWN") != nullptr),
287 mAllTracesAll(false),
288 mAllTracesShutdown(false) {
289 if (const char* lssEnv = PR_GetEnv("MOZ_CC_LOG_SHUTDOWN_SKIP")) {
290 mLogShutdown = true;
291 nsDependentCString lssString(lssEnv);
292 nsresult rv;
293 int32_t lss = lssString.ToInteger(&rv);
294 if (NS_SUCCEEDED(rv) && lss >= 0) {
295 mLogShutdownSkip = lss;
299 const char* logThreadEnv = PR_GetEnv("MOZ_CC_LOG_THREAD");
300 bool threadLogging = true;
301 if (logThreadEnv && !!strcmp(logThreadEnv, "all")) {
302 if (NS_IsMainThread()) {
303 threadLogging = !strcmp(logThreadEnv, "main");
304 } else {
305 threadLogging = !strcmp(logThreadEnv, "worker");
309 const char* logProcessEnv = PR_GetEnv("MOZ_CC_LOG_PROCESS");
310 bool processLogging = true;
311 if (logProcessEnv && !!strcmp(logProcessEnv, "all")) {
312 switch (XRE_GetProcessType()) {
313 case GeckoProcessType_Default:
314 processLogging = !strcmp(logProcessEnv, "main");
315 break;
316 case GeckoProcessType_Content:
317 processLogging = !strcmp(logProcessEnv, "content");
318 break;
319 default:
320 processLogging = false;
321 break;
324 mLogThisThread = threadLogging && processLogging;
326 const char* allTracesEnv = PR_GetEnv("MOZ_CC_ALL_TRACES");
327 if (allTracesEnv) {
328 if (!strcmp(allTracesEnv, "all")) {
329 mAllTracesAll = true;
330 } else if (!strcmp(allTracesEnv, "shutdown")) {
331 mAllTracesShutdown = true;
336 // aShutdownCount is how many shutdown CCs we've started.
337 // For non-shutdown CCs, we'll pass in 0.
338 // For the first shutdown CC, we'll pass in 1.
339 bool LogThisCC(int32_t aShutdownCount) {
340 if (mLogAll) {
341 return mLogThisThread;
343 if (aShutdownCount == 0 || !mLogShutdown) {
344 return false;
346 if (aShutdownCount <= mLogShutdownSkip) {
347 return false;
349 return mLogThisThread;
352 bool AllTracesThisCC(bool aIsShutdown) {
353 return mAllTracesAll || (aIsShutdown && mAllTracesShutdown);
357 #ifdef COLLECT_TIME_DEBUG
358 class TimeLog {
359 public:
360 TimeLog() : mLastCheckpoint(TimeStamp::Now()) {}
362 void Checkpoint(const char* aEvent) {
363 TimeStamp now = TimeStamp::Now();
364 double dur = (now - mLastCheckpoint).ToMilliseconds();
365 if (dur >= 0.5) {
366 printf("cc: %s took %.1fms\n", aEvent, dur);
368 mLastCheckpoint = now;
371 private:
372 TimeStamp mLastCheckpoint;
374 #else
375 class TimeLog {
376 public:
377 TimeLog() = default;
378 void Checkpoint(const char* aEvent) {}
380 #endif
382 ////////////////////////////////////////////////////////////////////////
383 // Base types
384 ////////////////////////////////////////////////////////////////////////
386 class PtrInfo;
388 class EdgePool {
389 public:
390 // EdgePool allocates arrays of void*, primarily to hold PtrInfo*.
391 // However, at the end of a block, the last two pointers are a null
392 // and then a void** pointing to the next block. This allows
393 // EdgePool::Iterators to be a single word but still capable of crossing
394 // block boundaries.
396 EdgePool() {
397 mSentinelAndBlocks[0].block = nullptr;
398 mSentinelAndBlocks[1].block = nullptr;
401 ~EdgePool() {
402 MOZ_ASSERT(!mSentinelAndBlocks[0].block && !mSentinelAndBlocks[1].block,
403 "Didn't call Clear()?");
406 void Clear() {
407 EdgeBlock* b = EdgeBlocks();
408 while (b) {
409 EdgeBlock* next = b->Next();
410 delete b;
411 b = next;
414 mSentinelAndBlocks[0].block = nullptr;
415 mSentinelAndBlocks[1].block = nullptr;
418 #ifdef DEBUG
419 bool IsEmpty() {
420 return !mSentinelAndBlocks[0].block && !mSentinelAndBlocks[1].block;
422 #endif
424 private:
425 struct EdgeBlock;
426 union PtrInfoOrBlock {
427 // Use a union to avoid reinterpret_cast and the ensuing
428 // potential aliasing bugs.
429 PtrInfo* ptrInfo;
430 EdgeBlock* block;
432 struct EdgeBlock {
433 enum { EdgeBlockSize = 16 * 1024 };
435 PtrInfoOrBlock mPointers[EdgeBlockSize];
436 EdgeBlock() {
437 mPointers[EdgeBlockSize - 2].block = nullptr; // sentinel
438 mPointers[EdgeBlockSize - 1].block = nullptr; // next block pointer
440 EdgeBlock*& Next() { return mPointers[EdgeBlockSize - 1].block; }
441 PtrInfoOrBlock* Start() { return &mPointers[0]; }
442 PtrInfoOrBlock* End() { return &mPointers[EdgeBlockSize - 2]; }
445 // Store the null sentinel so that we can have valid iterators
446 // before adding any edges and without adding any blocks.
447 PtrInfoOrBlock mSentinelAndBlocks[2];
449 EdgeBlock*& EdgeBlocks() { return mSentinelAndBlocks[1].block; }
450 EdgeBlock* EdgeBlocks() const { return mSentinelAndBlocks[1].block; }
452 public:
453 class Iterator {
454 public:
455 Iterator() : mPointer(nullptr) {}
456 explicit Iterator(PtrInfoOrBlock* aPointer) : mPointer(aPointer) {}
457 Iterator(const Iterator& aOther) = default;
459 Iterator& operator++() {
460 if (!mPointer->ptrInfo) {
461 // Null pointer is a sentinel for link to the next block.
462 mPointer = (mPointer + 1)->block->mPointers;
464 ++mPointer;
465 return *this;
468 PtrInfo* operator*() const {
469 if (!mPointer->ptrInfo) {
470 // Null pointer is a sentinel for link to the next block.
471 return (mPointer + 1)->block->mPointers->ptrInfo;
473 return mPointer->ptrInfo;
475 bool operator==(const Iterator& aOther) const {
476 return mPointer == aOther.mPointer;
478 bool operator!=(const Iterator& aOther) const {
479 return mPointer != aOther.mPointer;
482 #ifdef DEBUG_CC_GRAPH
483 bool Initialized() const { return mPointer != nullptr; }
484 #endif
486 private:
487 PtrInfoOrBlock* mPointer;
490 class Builder;
491 friend class Builder;
492 class Builder {
493 public:
494 explicit Builder(EdgePool& aPool)
495 : mCurrent(&aPool.mSentinelAndBlocks[0]),
496 mBlockEnd(&aPool.mSentinelAndBlocks[0]),
497 mNextBlockPtr(&aPool.EdgeBlocks()) {}
499 Iterator Mark() { return Iterator(mCurrent); }
501 void Add(PtrInfo* aEdge) {
502 if (mCurrent == mBlockEnd) {
503 EdgeBlock* b = new EdgeBlock();
504 *mNextBlockPtr = b;
505 mCurrent = b->Start();
506 mBlockEnd = b->End();
507 mNextBlockPtr = &b->Next();
509 (mCurrent++)->ptrInfo = aEdge;
512 private:
513 // mBlockEnd points to space for null sentinel
514 PtrInfoOrBlock* mCurrent;
515 PtrInfoOrBlock* mBlockEnd;
516 EdgeBlock** mNextBlockPtr;
519 size_t SizeOfExcludingThis(MallocSizeOf aMallocSizeOf) const {
520 size_t n = 0;
521 EdgeBlock* b = EdgeBlocks();
522 while (b) {
523 n += aMallocSizeOf(b);
524 b = b->Next();
526 return n;
530 #ifdef DEBUG_CC_GRAPH
531 # define CC_GRAPH_ASSERT(b) MOZ_ASSERT(b)
532 #else
533 # define CC_GRAPH_ASSERT(b)
534 #endif
536 #define CC_TELEMETRY(_name, _value) \
537 do { \
538 if (NS_IsMainThread()) { \
539 Telemetry::Accumulate(Telemetry::CYCLE_COLLECTOR##_name, _value); \
540 } else { \
541 Telemetry::Accumulate(Telemetry::CYCLE_COLLECTOR_WORKER##_name, _value); \
543 } while (0)
545 enum NodeColor { black, white, grey };
547 // This structure should be kept as small as possible; we may expect
548 // hundreds of thousands of them to be allocated and touched
549 // repeatedly during each cycle collection.
550 class PtrInfo final {
551 public:
552 // mParticipant knows a more concrete type.
553 void* mPointer;
554 nsCycleCollectionParticipant* mParticipant;
555 uint32_t mColor : 2;
556 uint32_t mInternalRefs : 30;
557 uint32_t mRefCount;
559 private:
560 EdgePool::Iterator mFirstChild;
562 static const uint32_t kInitialRefCount = UINT32_MAX - 1;
564 public:
565 PtrInfo(void* aPointer, nsCycleCollectionParticipant* aParticipant)
566 : mPointer(aPointer),
567 mParticipant(aParticipant),
568 mColor(grey),
569 mInternalRefs(0),
570 mRefCount(kInitialRefCount) {
571 MOZ_ASSERT(aParticipant);
573 // We initialize mRefCount to a large non-zero value so
574 // that it doesn't look like a JS object to the cycle collector
575 // in the case where the object dies before being traversed.
576 MOZ_ASSERT(!IsGrayJS() && !IsBlackJS());
579 // Allow NodePool::NodeBlock's constructor to compile.
580 PtrInfo()
581 : mPointer{nullptr},
582 mParticipant{nullptr},
583 mColor{0},
584 mInternalRefs{0},
585 mRefCount{0} {
586 MOZ_ASSERT_UNREACHABLE("should never be called");
589 bool IsGrayJS() const { return mRefCount == 0; }
591 bool IsBlackJS() const { return mRefCount == UINT32_MAX; }
593 bool WasTraversed() const { return mRefCount != kInitialRefCount; }
595 EdgePool::Iterator FirstChild() const {
596 CC_GRAPH_ASSERT(mFirstChild.Initialized());
597 return mFirstChild;
600 // this PtrInfo must be part of a NodePool
601 EdgePool::Iterator LastChild() const {
602 CC_GRAPH_ASSERT((this + 1)->mFirstChild.Initialized());
603 return (this + 1)->mFirstChild;
606 void SetFirstChild(EdgePool::Iterator aFirstChild) {
607 CC_GRAPH_ASSERT(aFirstChild.Initialized());
608 mFirstChild = aFirstChild;
611 // this PtrInfo must be part of a NodePool
612 void SetLastChild(EdgePool::Iterator aLastChild) {
613 CC_GRAPH_ASSERT(aLastChild.Initialized());
614 (this + 1)->mFirstChild = aLastChild;
617 void AnnotatedReleaseAssert(bool aCondition, const char* aMessage);
620 void PtrInfo::AnnotatedReleaseAssert(bool aCondition, const char* aMessage) {
621 if (aCondition) {
622 return;
625 const char* piName = "Unknown";
626 if (mParticipant) {
627 piName = mParticipant->ClassName();
629 nsPrintfCString msg("%s, for class %s", aMessage, piName);
630 NS_WARNING(msg.get());
631 CrashReporter::AnnotateCrashReport(CrashReporter::Annotation::CycleCollector,
632 msg);
634 MOZ_CRASH();
638 * A structure designed to be used like a linked list of PtrInfo, except
639 * it allocates many PtrInfos at a time.
641 class NodePool {
642 private:
643 // The -2 allows us to use |NodeBlockSize + 1| for |mEntries|, and fit
644 // |mNext|, all without causing slop.
645 enum { NodeBlockSize = 4 * 1024 - 2 };
647 struct NodeBlock {
648 // We create and destroy NodeBlock using moz_xmalloc/free rather than new
649 // and delete to avoid calling its constructor and destructor.
650 NodeBlock() : mNext{nullptr} {
651 MOZ_ASSERT_UNREACHABLE("should never be called");
653 // Ensure NodeBlock is the right size (see the comment on NodeBlockSize
654 // above).
655 static_assert(
656 sizeof(NodeBlock) == 81904 || // 32-bit; equals 19.996 x 4 KiB pages
657 sizeof(NodeBlock) ==
658 131048, // 64-bit; equals 31.994 x 4 KiB pages
659 "ill-sized NodeBlock");
661 ~NodeBlock() { MOZ_ASSERT_UNREACHABLE("should never be called"); }
663 NodeBlock* mNext;
664 PtrInfo mEntries[NodeBlockSize + 1]; // +1 to store last child of last node
667 public:
668 NodePool() : mBlocks(nullptr), mLast(nullptr) {}
670 ~NodePool() { MOZ_ASSERT(!mBlocks, "Didn't call Clear()?"); }
672 void Clear() {
673 NodeBlock* b = mBlocks;
674 while (b) {
675 NodeBlock* n = b->mNext;
676 free(b);
677 b = n;
680 mBlocks = nullptr;
681 mLast = nullptr;
684 #ifdef DEBUG
685 bool IsEmpty() { return !mBlocks && !mLast; }
686 #endif
688 class Builder;
689 friend class Builder;
690 class Builder {
691 public:
692 explicit Builder(NodePool& aPool)
693 : mNextBlock(&aPool.mBlocks), mNext(aPool.mLast), mBlockEnd(nullptr) {
694 MOZ_ASSERT(!aPool.mBlocks && !aPool.mLast, "pool not empty");
696 PtrInfo* Add(void* aPointer, nsCycleCollectionParticipant* aParticipant) {
697 if (mNext == mBlockEnd) {
698 NodeBlock* block = static_cast<NodeBlock*>(malloc(sizeof(NodeBlock)));
699 if (!block) {
700 return nullptr;
703 *mNextBlock = block;
704 mNext = block->mEntries;
705 mBlockEnd = block->mEntries + NodeBlockSize;
706 block->mNext = nullptr;
707 mNextBlock = &block->mNext;
709 return new (mozilla::KnownNotNull, mNext++)
710 PtrInfo(aPointer, aParticipant);
713 private:
714 NodeBlock** mNextBlock;
715 PtrInfo*& mNext;
716 PtrInfo* mBlockEnd;
719 class Enumerator;
720 friend class Enumerator;
721 class Enumerator {
722 public:
723 explicit Enumerator(NodePool& aPool)
724 : mFirstBlock(aPool.mBlocks),
725 mCurBlock(nullptr),
726 mNext(nullptr),
727 mBlockEnd(nullptr),
728 mLast(aPool.mLast) {}
730 bool IsDone() const { return mNext == mLast; }
732 bool AtBlockEnd() const { return mNext == mBlockEnd; }
734 PtrInfo* GetNext() {
735 MOZ_ASSERT(!IsDone(), "calling GetNext when done");
736 if (mNext == mBlockEnd) {
737 NodeBlock* nextBlock = mCurBlock ? mCurBlock->mNext : mFirstBlock;
738 mNext = nextBlock->mEntries;
739 mBlockEnd = mNext + NodeBlockSize;
740 mCurBlock = nextBlock;
742 return mNext++;
745 private:
746 // mFirstBlock is a reference to allow an Enumerator to be constructed
747 // for an empty graph.
748 NodeBlock*& mFirstBlock;
749 NodeBlock* mCurBlock;
750 // mNext is the next value we want to return, unless mNext == mBlockEnd
751 // NB: mLast is a reference to allow enumerating while building!
752 PtrInfo* mNext;
753 PtrInfo* mBlockEnd;
754 PtrInfo*& mLast;
757 size_t SizeOfExcludingThis(MallocSizeOf aMallocSizeOf) const {
758 // We don't measure the things pointed to by mEntries[] because those
759 // pointers are non-owning.
760 size_t n = 0;
761 NodeBlock* b = mBlocks;
762 while (b) {
763 n += aMallocSizeOf(b);
764 b = b->mNext;
766 return n;
769 private:
770 NodeBlock* mBlocks;
771 PtrInfo* mLast;
774 struct PtrToNodeHashPolicy {
775 using Key = PtrInfo*;
776 using Lookup = void*;
778 static js::HashNumber hash(const Lookup& aLookup) {
779 return mozilla::HashGeneric(aLookup);
782 static bool match(const Key& aKey, const Lookup& aLookup) {
783 return aKey->mPointer == aLookup;
787 struct WeakMapping {
788 // map and key will be null if the corresponding objects are GC marked
789 PtrInfo* mMap;
790 PtrInfo* mKey;
791 PtrInfo* mKeyDelegate;
792 PtrInfo* mVal;
795 class CCGraphBuilder;
797 struct CCGraph {
798 NodePool mNodes;
799 EdgePool mEdges;
800 nsTArray<WeakMapping> mWeakMaps;
801 uint32_t mRootCount;
803 private:
804 friend CCGraphBuilder;
806 mozilla::HashSet<PtrInfo*, PtrToNodeHashPolicy> mPtrInfoMap;
808 bool mOutOfMemory;
810 static const uint32_t kInitialMapLength = 16384;
812 public:
813 CCGraph()
814 : mRootCount(0), mPtrInfoMap(kInitialMapLength), mOutOfMemory(false) {}
816 ~CCGraph() = default;
818 void Init() { MOZ_ASSERT(IsEmpty(), "Failed to call CCGraph::Clear"); }
820 void Clear() {
821 mNodes.Clear();
822 mEdges.Clear();
823 mWeakMaps.Clear();
824 mRootCount = 0;
825 mPtrInfoMap.clearAndCompact();
826 mOutOfMemory = false;
829 #ifdef DEBUG
830 bool IsEmpty() {
831 return mNodes.IsEmpty() && mEdges.IsEmpty() && mWeakMaps.IsEmpty() &&
832 mRootCount == 0 && mPtrInfoMap.empty();
834 #endif
836 PtrInfo* FindNode(void* aPtr);
837 void RemoveObjectFromMap(void* aObject);
839 uint32_t MapCount() const { return mPtrInfoMap.count(); }
841 size_t SizeOfExcludingThis(MallocSizeOf aMallocSizeOf) const {
842 size_t n = 0;
844 n += mNodes.SizeOfExcludingThis(aMallocSizeOf);
845 n += mEdges.SizeOfExcludingThis(aMallocSizeOf);
847 // We don't measure what the WeakMappings point to, because the
848 // pointers are non-owning.
849 n += mWeakMaps.ShallowSizeOfExcludingThis(aMallocSizeOf);
851 n += mPtrInfoMap.shallowSizeOfExcludingThis(aMallocSizeOf);
853 return n;
857 PtrInfo* CCGraph::FindNode(void* aPtr) {
858 auto p = mPtrInfoMap.lookup(aPtr);
859 return p ? *p : nullptr;
862 void CCGraph::RemoveObjectFromMap(void* aObj) {
863 auto p = mPtrInfoMap.lookup(aObj);
864 if (p) {
865 PtrInfo* pinfo = *p;
866 pinfo->mPointer = nullptr;
867 pinfo->mParticipant = nullptr;
868 mPtrInfoMap.remove(p);
872 static nsISupports* CanonicalizeXPCOMParticipant(nsISupports* aIn) {
873 nsISupports* out = nullptr;
874 aIn->QueryInterface(NS_GET_IID(nsCycleCollectionISupports),
875 reinterpret_cast<void**>(&out));
876 return out;
879 struct nsPurpleBufferEntry {
880 nsPurpleBufferEntry(void* aObject, nsCycleCollectingAutoRefCnt* aRefCnt,
881 nsCycleCollectionParticipant* aParticipant)
882 : mObject(aObject), mRefCnt(aRefCnt), mParticipant(aParticipant) {}
884 nsPurpleBufferEntry(nsPurpleBufferEntry&& aOther)
885 : mObject(nullptr), mRefCnt(nullptr), mParticipant(nullptr) {
886 Swap(aOther);
889 void Swap(nsPurpleBufferEntry& aOther) {
890 std::swap(mObject, aOther.mObject);
891 std::swap(mRefCnt, aOther.mRefCnt);
892 std::swap(mParticipant, aOther.mParticipant);
895 void Clear() {
896 mRefCnt->RemoveFromPurpleBuffer();
897 mRefCnt = nullptr;
898 mObject = nullptr;
899 mParticipant = nullptr;
902 ~nsPurpleBufferEntry() {
903 if (mRefCnt) {
904 mRefCnt->RemoveFromPurpleBuffer();
908 void* mObject;
909 nsCycleCollectingAutoRefCnt* mRefCnt;
910 nsCycleCollectionParticipant* mParticipant; // nullptr for nsISupports
913 class nsCycleCollector;
915 struct nsPurpleBuffer {
916 private:
917 uint32_t mCount;
919 // Try to match the size of a jemalloc bucket, to minimize slop bytes.
920 // - On 32-bit platforms sizeof(nsPurpleBufferEntry) is 12, so mEntries'
921 // Segment is 16,372 bytes.
922 // - On 64-bit platforms sizeof(nsPurpleBufferEntry) is 24, so mEntries'
923 // Segment is 32,760 bytes.
924 static const uint32_t kEntriesPerSegment = 1365;
925 static const size_t kSegmentSize =
926 sizeof(nsPurpleBufferEntry) * kEntriesPerSegment;
927 typedef SegmentedVector<nsPurpleBufferEntry, kSegmentSize,
928 InfallibleAllocPolicy>
929 PurpleBufferVector;
930 PurpleBufferVector mEntries;
932 public:
933 nsPurpleBuffer() : mCount(0) {
934 static_assert(
935 sizeof(PurpleBufferVector::Segment) == 16372 || // 32-bit
936 sizeof(PurpleBufferVector::Segment) == 32760 || // 64-bit
937 sizeof(PurpleBufferVector::Segment) == 32744, // 64-bit Windows
938 "ill-sized nsPurpleBuffer::mEntries");
941 ~nsPurpleBuffer() = default;
943 // This method compacts mEntries.
944 template <class PurpleVisitor>
945 void VisitEntries(PurpleVisitor& aVisitor) {
946 Maybe<AutoRestore<bool>> ar;
947 if (NS_IsMainThread()) {
948 ar.emplace(gNurseryPurpleBufferEnabled);
949 gNurseryPurpleBufferEnabled = false;
950 ClearNurseryPurpleBuffer();
953 if (mEntries.IsEmpty()) {
954 return;
957 uint32_t oldLength = mEntries.Length();
958 uint32_t keptLength = 0;
959 auto revIter = mEntries.IterFromLast();
960 auto iter = mEntries.Iter();
961 // After iteration this points to the first empty entry.
962 auto firstEmptyIter = mEntries.Iter();
963 auto iterFromLastEntry = mEntries.IterFromLast();
964 for (; !iter.Done(); iter.Next()) {
965 nsPurpleBufferEntry& e = iter.Get();
966 if (e.mObject) {
967 if (!aVisitor.Visit(*this, &e)) {
968 return;
972 // Visit call above may have cleared the entry, or the entry was empty
973 // already.
974 if (!e.mObject) {
975 // Try to find a non-empty entry from the end of the vector.
976 for (; !revIter.Done(); revIter.Prev()) {
977 nsPurpleBufferEntry& otherEntry = revIter.Get();
978 if (&e == &otherEntry) {
979 break;
981 if (otherEntry.mObject) {
982 if (!aVisitor.Visit(*this, &otherEntry)) {
983 return;
985 // Visit may have cleared otherEntry.
986 if (otherEntry.mObject) {
987 e.Swap(otherEntry);
988 revIter.Prev(); // We've swapped this now empty entry.
989 break;
995 // Entry is non-empty even after the Visit call, ensure it is kept
996 // in mEntries.
997 if (e.mObject) {
998 firstEmptyIter.Next();
999 ++keptLength;
1002 if (&e == &revIter.Get()) {
1003 break;
1007 // There were some empty entries.
1008 if (oldLength != keptLength) {
1009 // While visiting entries, some new ones were possibly added. This can
1010 // happen during CanSkip. Move all such new entries to be after other
1011 // entries. Note, we don't call Visit on newly added entries!
1012 if (&iterFromLastEntry.Get() != &mEntries.GetLast()) {
1013 iterFromLastEntry.Next(); // Now pointing to the first added entry.
1014 auto& iterForNewEntries = iterFromLastEntry;
1015 while (!iterForNewEntries.Done()) {
1016 MOZ_ASSERT(!firstEmptyIter.Done());
1017 MOZ_ASSERT(!firstEmptyIter.Get().mObject);
1018 firstEmptyIter.Get().Swap(iterForNewEntries.Get());
1019 firstEmptyIter.Next();
1020 iterForNewEntries.Next();
1024 mEntries.PopLastN(oldLength - keptLength);
1028 void FreeBlocks() {
1029 mCount = 0;
1030 mEntries.Clear();
1033 void SelectPointers(CCGraphBuilder& aBuilder);
1035 // RemoveSkippable removes entries from the purple buffer synchronously
1036 // (1) if !aAsyncSnowWhiteFreeing and nsPurpleBufferEntry::mRefCnt is 0 or
1037 // (2) if nsXPCOMCycleCollectionParticipant::CanSkip() for the obj or
1038 // (3) if nsPurpleBufferEntry::mRefCnt->IsPurple() is false.
1039 // (4) If aRemoveChildlessNodes is true, then any nodes in the purple buffer
1040 // that will have no children in the cycle collector graph will also be
1041 // removed. CanSkip() may be run on these children.
1042 void RemoveSkippable(nsCycleCollector* aCollector, js::SliceBudget& aBudget,
1043 bool aRemoveChildlessNodes, bool aAsyncSnowWhiteFreeing,
1044 CC_ForgetSkippableCallback aCb);
1046 MOZ_ALWAYS_INLINE void Put(void* aObject, nsCycleCollectionParticipant* aCp,
1047 nsCycleCollectingAutoRefCnt* aRefCnt) {
1048 nsPurpleBufferEntry entry(aObject, aRefCnt, aCp);
1049 Unused << mEntries.Append(std::move(entry));
1050 MOZ_ASSERT(!entry.mRefCnt, "Move didn't work!");
1051 ++mCount;
1054 void Remove(nsPurpleBufferEntry* aEntry) {
1055 MOZ_ASSERT(mCount != 0, "must have entries");
1056 --mCount;
1057 aEntry->Clear();
1060 uint32_t Count() const { return mCount; }
1062 size_t SizeOfExcludingThis(MallocSizeOf aMallocSizeOf) const {
1063 return mEntries.SizeOfExcludingThis(aMallocSizeOf);
1067 static bool AddPurpleRoot(CCGraphBuilder& aBuilder, void* aRoot,
1068 nsCycleCollectionParticipant* aParti);
1070 struct SelectPointersVisitor {
1071 explicit SelectPointersVisitor(CCGraphBuilder& aBuilder)
1072 : mBuilder(aBuilder) {}
1074 bool Visit(nsPurpleBuffer& aBuffer, nsPurpleBufferEntry* aEntry) {
1075 MOZ_ASSERT(aEntry->mObject, "Null object in purple buffer");
1076 MOZ_ASSERT(aEntry->mRefCnt->get() != 0,
1077 "SelectPointersVisitor: snow-white object in the purple buffer");
1078 if (!aEntry->mRefCnt->IsPurple() ||
1079 AddPurpleRoot(mBuilder, aEntry->mObject, aEntry->mParticipant)) {
1080 aBuffer.Remove(aEntry);
1082 return true;
1085 private:
1086 CCGraphBuilder& mBuilder;
1089 void nsPurpleBuffer::SelectPointers(CCGraphBuilder& aBuilder) {
1090 SelectPointersVisitor visitor(aBuilder);
1091 VisitEntries(visitor);
1093 MOZ_ASSERT(mCount == 0, "AddPurpleRoot failed");
1094 if (mCount == 0) {
1095 FreeBlocks();
1099 enum ccPhase {
1100 IdlePhase,
1101 GraphBuildingPhase,
1102 ScanAndCollectWhitePhase,
1103 CleanupPhase
1106 enum ccIsManual { CCIsNotManual = false, CCIsManual = true };
1108 ////////////////////////////////////////////////////////////////////////
1109 // Top level structure for the cycle collector.
1110 ////////////////////////////////////////////////////////////////////////
1112 using js::SliceBudget;
1114 class JSPurpleBuffer;
1116 class nsCycleCollector : public nsIMemoryReporter {
1117 public:
1118 NS_DECL_ISUPPORTS
1119 NS_DECL_NSIMEMORYREPORTER
1121 private:
1122 bool mActivelyCollecting;
1123 bool mFreeingSnowWhite;
1124 // mScanInProgress should be false when we're collecting white objects.
1125 bool mScanInProgress;
1126 CycleCollectorResults mResults;
1127 TimeStamp mCollectionStart;
1129 CycleCollectedJSRuntime* mCCJSRuntime;
1131 ccPhase mIncrementalPhase;
1132 int32_t mShutdownCount = 0;
1133 CCGraph mGraph;
1134 UniquePtr<CCGraphBuilder> mBuilder;
1135 RefPtr<nsCycleCollectorLogger> mLogger;
1137 #ifdef DEBUG
1138 nsISerialEventTarget* mEventTarget;
1139 #endif
1141 nsCycleCollectorParams mParams;
1143 uint32_t mWhiteNodeCount;
1145 CC_BeforeUnlinkCallback mBeforeUnlinkCB;
1146 CC_ForgetSkippableCallback mForgetSkippableCB;
1148 nsPurpleBuffer mPurpleBuf;
1150 uint32_t mUnmergedNeeded;
1151 uint32_t mMergedInARow;
1153 RefPtr<JSPurpleBuffer> mJSPurpleBuffer;
1155 private:
1156 virtual ~nsCycleCollector();
1158 public:
1159 nsCycleCollector();
1161 void SetCCJSRuntime(CycleCollectedJSRuntime* aCCRuntime);
1162 void ClearCCJSRuntime();
1164 void SetBeforeUnlinkCallback(CC_BeforeUnlinkCallback aBeforeUnlinkCB) {
1165 CheckThreadSafety();
1166 mBeforeUnlinkCB = aBeforeUnlinkCB;
1169 void SetForgetSkippableCallback(
1170 CC_ForgetSkippableCallback aForgetSkippableCB) {
1171 CheckThreadSafety();
1172 mForgetSkippableCB = aForgetSkippableCB;
1175 void Suspect(void* aPtr, nsCycleCollectionParticipant* aCp,
1176 nsCycleCollectingAutoRefCnt* aRefCnt);
1177 void SuspectNurseryEntries();
1178 uint32_t SuspectedCount();
1179 void ForgetSkippable(js::SliceBudget& aBudget, bool aRemoveChildlessNodes,
1180 bool aAsyncSnowWhiteFreeing);
1181 bool FreeSnowWhite(bool aUntilNoSWInPurpleBuffer);
1182 bool FreeSnowWhiteWithBudget(js::SliceBudget& aBudget);
1184 // This method assumes its argument is already canonicalized.
1185 void RemoveObjectFromGraph(void* aPtr);
1187 void PrepareForGarbageCollection();
1188 void FinishAnyCurrentCollection(CCReason aReason);
1190 bool Collect(CCReason aReason, ccIsManual aIsManual, SliceBudget& aBudget,
1191 nsICycleCollectorListener* aManualListener,
1192 bool aPreferShorterSlices = false);
1193 MOZ_CAN_RUN_SCRIPT
1194 void Shutdown(bool aDoCollect);
1196 bool IsIdle() const { return mIncrementalPhase == IdlePhase; }
1198 void SizeOfIncludingThis(mozilla::MallocSizeOf aMallocSizeOf,
1199 size_t* aObjectSize, size_t* aGraphSize,
1200 size_t* aPurpleBufferSize) const;
1202 JSPurpleBuffer* GetJSPurpleBuffer();
1204 CycleCollectedJSRuntime* Runtime() { return mCCJSRuntime; }
1206 private:
1207 void CheckThreadSafety();
1208 MOZ_CAN_RUN_SCRIPT
1209 void ShutdownCollect();
1211 void FixGrayBits(bool aIsShutdown, TimeLog& aTimeLog);
1212 bool IsIncrementalGCInProgress();
1213 void FinishAnyIncrementalGCInProgress();
1214 bool ShouldMergeZones(ccIsManual aIsManual);
1216 void BeginCollection(CCReason aReason, ccIsManual aIsManual,
1217 nsICycleCollectorListener* aManualListener);
1218 void MarkRoots(SliceBudget& aBudget);
1219 void ScanRoots(bool aFullySynchGraphBuild);
1220 void ScanIncrementalRoots();
1221 void ScanWhiteNodes(bool aFullySynchGraphBuild);
1222 void ScanBlackNodes();
1223 void ScanWeakMaps();
1225 // returns whether anything was collected
1226 bool CollectWhite();
1228 void CleanupAfterCollection();
1231 NS_IMPL_ISUPPORTS(nsCycleCollector, nsIMemoryReporter)
1234 * GraphWalker is templatized over a Visitor class that must provide
1235 * the following two methods:
1237 * bool ShouldVisitNode(PtrInfo const *pi);
1238 * void VisitNode(PtrInfo *pi);
1240 template <class Visitor>
1241 class GraphWalker {
1242 private:
1243 Visitor mVisitor;
1245 void DoWalk(nsDeque<PtrInfo>& aQueue);
1247 void CheckedPush(nsDeque<PtrInfo>& aQueue, PtrInfo* aPi) {
1248 if (!aPi) {
1249 MOZ_CRASH();
1251 if (!aQueue.Push(aPi, fallible)) {
1252 mVisitor.Failed();
1256 public:
1257 void Walk(PtrInfo* aPi);
1258 void WalkFromRoots(CCGraph& aGraph);
1259 // copy-constructing the visitor should be cheap, and less
1260 // indirection than using a reference
1261 explicit GraphWalker(const Visitor aVisitor) : mVisitor(aVisitor) {}
1264 ////////////////////////////////////////////////////////////////////////
1265 // The static collector struct
1266 ////////////////////////////////////////////////////////////////////////
1268 struct CollectorData {
1269 RefPtr<nsCycleCollector> mCollector;
1270 CycleCollectedJSContext* mContext;
1273 static MOZ_THREAD_LOCAL(CollectorData*) sCollectorData;
1275 ////////////////////////////////////////////////////////////////////////
1276 // Utility functions
1277 ////////////////////////////////////////////////////////////////////////
1279 static inline void ToParticipant(nsISupports* aPtr,
1280 nsXPCOMCycleCollectionParticipant** aCp) {
1281 // We use QI to move from an nsISupports to an
1282 // nsXPCOMCycleCollectionParticipant, which is a per-class singleton helper
1283 // object that implements traversal and unlinking logic for the nsISupports
1284 // in question.
1285 *aCp = nullptr;
1286 CallQueryInterface(aPtr, aCp);
1289 static void ToParticipant(void* aParti, nsCycleCollectionParticipant** aCp) {
1290 // If the participant is null, this is an nsISupports participant,
1291 // so we must QI to get the real participant.
1293 if (!*aCp) {
1294 nsISupports* nsparti = static_cast<nsISupports*>(aParti);
1295 MOZ_ASSERT(CanonicalizeXPCOMParticipant(nsparti) == nsparti);
1296 nsXPCOMCycleCollectionParticipant* xcp;
1297 ToParticipant(nsparti, &xcp);
1298 *aCp = xcp;
1302 template <class Visitor>
1303 MOZ_NEVER_INLINE void GraphWalker<Visitor>::Walk(PtrInfo* aPi) {
1304 nsDeque<PtrInfo> queue;
1305 CheckedPush(queue, aPi);
1306 DoWalk(queue);
1309 template <class Visitor>
1310 MOZ_NEVER_INLINE void GraphWalker<Visitor>::WalkFromRoots(CCGraph& aGraph) {
1311 nsDeque<PtrInfo> queue;
1312 NodePool::Enumerator etor(aGraph.mNodes);
1313 for (uint32_t i = 0; i < aGraph.mRootCount; ++i) {
1314 CheckedPush(queue, etor.GetNext());
1316 DoWalk(queue);
1319 template <class Visitor>
1320 MOZ_NEVER_INLINE void GraphWalker<Visitor>::DoWalk(nsDeque<PtrInfo>& aQueue) {
1321 // Use a aQueue to match the breadth-first traversal used when we
1322 // built the graph, for hopefully-better locality.
1323 while (aQueue.GetSize() > 0) {
1324 PtrInfo* pi = aQueue.PopFront();
1326 if (pi->WasTraversed() && mVisitor.ShouldVisitNode(pi)) {
1327 mVisitor.VisitNode(pi);
1328 for (EdgePool::Iterator child = pi->FirstChild(),
1329 child_end = pi->LastChild();
1330 child != child_end; ++child) {
1331 CheckedPush(aQueue, *child);
1337 struct CCGraphDescriber : public LinkedListElement<CCGraphDescriber> {
1338 CCGraphDescriber() : mAddress("0x"), mCnt(0), mType(eUnknown) {}
1340 enum Type {
1341 eRefCountedObject,
1342 eGCedObject,
1343 eGCMarkedObject,
1344 eEdge,
1345 eRoot,
1346 eGarbage,
1347 eUnknown
1350 nsCString mAddress;
1351 nsCString mName;
1352 nsCString mCompartmentOrToAddress;
1353 uint32_t mCnt;
1354 Type mType;
1357 class LogStringMessageAsync : public DiscardableRunnable {
1358 public:
1359 explicit LogStringMessageAsync(const nsAString& aMsg)
1360 : mozilla::DiscardableRunnable("LogStringMessageAsync"), mMsg(aMsg) {}
1362 NS_IMETHOD Run() override {
1363 nsCOMPtr<nsIConsoleService> cs =
1364 do_GetService(NS_CONSOLESERVICE_CONTRACTID);
1365 if (cs) {
1366 cs->LogStringMessage(mMsg.get());
1368 return NS_OK;
1371 private:
1372 nsString mMsg;
1375 class nsCycleCollectorLogSinkToFile final : public nsICycleCollectorLogSink {
1376 public:
1377 NS_DECL_ISUPPORTS
1379 nsCycleCollectorLogSinkToFile()
1380 : mProcessIdentifier(base::GetCurrentProcId()),
1381 mGCLog("gc-edges"),
1382 mCCLog("cc-edges") {}
1384 NS_IMETHOD GetFilenameIdentifier(nsAString& aIdentifier) override {
1385 aIdentifier = mFilenameIdentifier;
1386 return NS_OK;
1389 NS_IMETHOD SetFilenameIdentifier(const nsAString& aIdentifier) override {
1390 mFilenameIdentifier = aIdentifier;
1391 return NS_OK;
1394 NS_IMETHOD GetProcessIdentifier(int32_t* aIdentifier) override {
1395 *aIdentifier = mProcessIdentifier;
1396 return NS_OK;
1399 NS_IMETHOD SetProcessIdentifier(int32_t aIdentifier) override {
1400 mProcessIdentifier = aIdentifier;
1401 return NS_OK;
1404 NS_IMETHOD GetGcLog(nsIFile** aPath) override {
1405 NS_IF_ADDREF(*aPath = mGCLog.mFile);
1406 return NS_OK;
1409 NS_IMETHOD GetCcLog(nsIFile** aPath) override {
1410 NS_IF_ADDREF(*aPath = mCCLog.mFile);
1411 return NS_OK;
1414 NS_IMETHOD Open(FILE** aGCLog, FILE** aCCLog) override {
1415 nsresult rv;
1417 if (mGCLog.mStream || mCCLog.mStream) {
1418 return NS_ERROR_UNEXPECTED;
1421 rv = OpenLog(&mGCLog);
1422 NS_ENSURE_SUCCESS(rv, rv);
1423 *aGCLog = mGCLog.mStream;
1425 rv = OpenLog(&mCCLog);
1426 NS_ENSURE_SUCCESS(rv, rv);
1427 *aCCLog = mCCLog.mStream;
1429 return NS_OK;
1432 NS_IMETHOD CloseGCLog() override {
1433 if (!mGCLog.mStream) {
1434 return NS_ERROR_UNEXPECTED;
1436 CloseLog(&mGCLog, u"Garbage"_ns);
1437 return NS_OK;
1440 NS_IMETHOD CloseCCLog() override {
1441 if (!mCCLog.mStream) {
1442 return NS_ERROR_UNEXPECTED;
1444 CloseLog(&mCCLog, u"Cycle"_ns);
1445 return NS_OK;
1448 private:
1449 ~nsCycleCollectorLogSinkToFile() {
1450 if (mGCLog.mStream) {
1451 MozillaUnRegisterDebugFILE(mGCLog.mStream);
1452 fclose(mGCLog.mStream);
1454 if (mCCLog.mStream) {
1455 MozillaUnRegisterDebugFILE(mCCLog.mStream);
1456 fclose(mCCLog.mStream);
1460 struct FileInfo {
1461 const char* const mPrefix;
1462 nsCOMPtr<nsIFile> mFile;
1463 FILE* mStream;
1465 explicit FileInfo(const char* aPrefix)
1466 : mPrefix(aPrefix), mStream(nullptr) {}
1470 * Create a new file named something like aPrefix.$PID.$IDENTIFIER.log in
1471 * $MOZ_CC_LOG_DIRECTORY or in the system's temp directory. No existing
1472 * file will be overwritten; if aPrefix.$PID.$IDENTIFIER.log exists, we'll
1473 * try a file named something like aPrefix.$PID.$IDENTIFIER-1.log, and so
1474 * on.
1476 already_AddRefed<nsIFile> CreateTempFile(const char* aPrefix) {
1477 nsPrintfCString filename("%s.%d%s%s.log", aPrefix, mProcessIdentifier,
1478 mFilenameIdentifier.IsEmpty() ? "" : ".",
1479 NS_ConvertUTF16toUTF8(mFilenameIdentifier).get());
1481 // Get the log directory either from $MOZ_CC_LOG_DIRECTORY or from
1482 // the fallback directories in OpenTempFile. We don't use an nsCOMPtr
1483 // here because OpenTempFile uses an in/out param and getter_AddRefs
1484 // wouldn't work.
1485 nsIFile* logFile = nullptr;
1486 if (char* env = PR_GetEnv("MOZ_CC_LOG_DIRECTORY")) {
1487 NS_NewNativeLocalFile(nsCString(env), /* followLinks = */ true, &logFile);
1490 // On Android or B2G, this function will open a file named
1491 // aFilename under a memory-reporting-specific folder
1492 // (/data/local/tmp/memory-reports). Otherwise, it will open a
1493 // file named aFilename under "NS_OS_TEMP_DIR".
1494 nsresult rv =
1495 nsDumpUtils::OpenTempFile(filename, &logFile, "memory-reports"_ns);
1496 if (NS_FAILED(rv)) {
1497 NS_IF_RELEASE(logFile);
1498 return nullptr;
1501 return dont_AddRef(logFile);
1504 nsresult OpenLog(FileInfo* aLog) {
1505 // Initially create the log in a file starting with "incomplete-".
1506 // We'll move the file and strip off the "incomplete-" once the dump
1507 // completes. (We do this because we don't want scripts which poll
1508 // the filesystem looking for GC/CC dumps to grab a file before we're
1509 // finished writing to it.)
1510 nsAutoCString incomplete;
1511 incomplete += "incomplete-";
1512 incomplete += aLog->mPrefix;
1513 MOZ_ASSERT(!aLog->mFile);
1514 aLog->mFile = CreateTempFile(incomplete.get());
1515 if (NS_WARN_IF(!aLog->mFile)) {
1516 return NS_ERROR_UNEXPECTED;
1519 MOZ_ASSERT(!aLog->mStream);
1520 nsresult rv = aLog->mFile->OpenANSIFileDesc("w", &aLog->mStream);
1521 if (NS_WARN_IF(NS_FAILED(rv))) {
1522 return NS_ERROR_UNEXPECTED;
1524 MozillaRegisterDebugFILE(aLog->mStream);
1525 return NS_OK;
1528 nsresult CloseLog(FileInfo* aLog, const nsAString& aCollectorKind) {
1529 MOZ_ASSERT(aLog->mStream);
1530 MOZ_ASSERT(aLog->mFile);
1532 MozillaUnRegisterDebugFILE(aLog->mStream);
1533 fclose(aLog->mStream);
1534 aLog->mStream = nullptr;
1536 // Strip off "incomplete-".
1537 nsCOMPtr<nsIFile> logFileFinalDestination = CreateTempFile(aLog->mPrefix);
1538 if (NS_WARN_IF(!logFileFinalDestination)) {
1539 return NS_ERROR_UNEXPECTED;
1542 nsAutoString logFileFinalDestinationName;
1543 logFileFinalDestination->GetLeafName(logFileFinalDestinationName);
1544 if (NS_WARN_IF(logFileFinalDestinationName.IsEmpty())) {
1545 return NS_ERROR_UNEXPECTED;
1548 aLog->mFile->MoveTo(/* directory */ nullptr, logFileFinalDestinationName);
1550 // Save the file path.
1551 aLog->mFile = logFileFinalDestination;
1553 // Log to the error console.
1554 nsAutoString logPath;
1555 logFileFinalDestination->GetPath(logPath);
1556 nsAutoString msg =
1557 aCollectorKind + u" Collector log dumped to "_ns + logPath;
1559 // We don't want any JS to run between ScanRoots and CollectWhite calls,
1560 // and since ScanRoots calls this method, better to log the message
1561 // asynchronously.
1562 RefPtr<LogStringMessageAsync> log = new LogStringMessageAsync(msg);
1563 NS_DispatchToCurrentThread(log);
1564 return NS_OK;
1567 int32_t mProcessIdentifier;
1568 nsString mFilenameIdentifier;
1569 FileInfo mGCLog;
1570 FileInfo mCCLog;
1573 NS_IMPL_ISUPPORTS(nsCycleCollectorLogSinkToFile, nsICycleCollectorLogSink)
1575 class nsCycleCollectorLogger final : public nsICycleCollectorListener {
1576 ~nsCycleCollectorLogger() { ClearDescribers(); }
1578 public:
1579 nsCycleCollectorLogger()
1580 : mLogSink(nsCycleCollector_createLogSink()),
1581 mWantAllTraces(false),
1582 mDisableLog(false),
1583 mWantAfterProcessing(false),
1584 mCCLog(nullptr) {}
1586 NS_DECL_ISUPPORTS
1588 void SetAllTraces() { mWantAllTraces = true; }
1590 bool IsAllTraces() { return mWantAllTraces; }
1592 NS_IMETHOD AllTraces(nsICycleCollectorListener** aListener) override {
1593 SetAllTraces();
1594 NS_ADDREF(*aListener = this);
1595 return NS_OK;
1598 NS_IMETHOD GetWantAllTraces(bool* aAllTraces) override {
1599 *aAllTraces = mWantAllTraces;
1600 return NS_OK;
1603 NS_IMETHOD GetDisableLog(bool* aDisableLog) override {
1604 *aDisableLog = mDisableLog;
1605 return NS_OK;
1608 NS_IMETHOD SetDisableLog(bool aDisableLog) override {
1609 mDisableLog = aDisableLog;
1610 return NS_OK;
1613 NS_IMETHOD GetWantAfterProcessing(bool* aWantAfterProcessing) override {
1614 *aWantAfterProcessing = mWantAfterProcessing;
1615 return NS_OK;
1618 NS_IMETHOD SetWantAfterProcessing(bool aWantAfterProcessing) override {
1619 mWantAfterProcessing = aWantAfterProcessing;
1620 return NS_OK;
1623 NS_IMETHOD GetLogSink(nsICycleCollectorLogSink** aLogSink) override {
1624 NS_ADDREF(*aLogSink = mLogSink);
1625 return NS_OK;
1628 NS_IMETHOD SetLogSink(nsICycleCollectorLogSink* aLogSink) override {
1629 if (!aLogSink) {
1630 return NS_ERROR_INVALID_ARG;
1632 mLogSink = aLogSink;
1633 return NS_OK;
1636 nsresult Begin() {
1637 nsresult rv;
1639 mCurrentAddress.AssignLiteral("0x");
1640 ClearDescribers();
1641 if (mDisableLog) {
1642 return NS_OK;
1645 FILE* gcLog;
1646 rv = mLogSink->Open(&gcLog, &mCCLog);
1647 NS_ENSURE_SUCCESS(rv, rv);
1648 // Dump the JS heap.
1649 CollectorData* data = sCollectorData.get();
1650 if (data && data->mContext) {
1651 data->mContext->Runtime()->DumpJSHeap(gcLog);
1653 rv = mLogSink->CloseGCLog();
1654 NS_ENSURE_SUCCESS(rv, rv);
1656 fprintf(mCCLog, "# WantAllTraces=%s\n", mWantAllTraces ? "true" : "false");
1657 return NS_OK;
1659 void NoteRefCountedObject(uint64_t aAddress, uint32_t aRefCount,
1660 const char* aObjectDescription) {
1661 if (!mDisableLog) {
1662 fprintf(mCCLog, "%p [rc=%u] %s\n", (void*)aAddress, aRefCount,
1663 aObjectDescription);
1665 if (mWantAfterProcessing) {
1666 CCGraphDescriber* d = new CCGraphDescriber();
1667 mDescribers.insertBack(d);
1668 mCurrentAddress.AssignLiteral("0x");
1669 mCurrentAddress.AppendInt(aAddress, 16);
1670 d->mType = CCGraphDescriber::eRefCountedObject;
1671 d->mAddress = mCurrentAddress;
1672 d->mCnt = aRefCount;
1673 d->mName.Append(aObjectDescription);
1676 void NoteGCedObject(uint64_t aAddress, bool aMarked,
1677 const char* aObjectDescription,
1678 uint64_t aCompartmentAddress) {
1679 if (!mDisableLog) {
1680 fprintf(mCCLog, "%p [gc%s] %s\n", (void*)aAddress,
1681 aMarked ? ".marked" : "", aObjectDescription);
1683 if (mWantAfterProcessing) {
1684 CCGraphDescriber* d = new CCGraphDescriber();
1685 mDescribers.insertBack(d);
1686 mCurrentAddress.AssignLiteral("0x");
1687 mCurrentAddress.AppendInt(aAddress, 16);
1688 d->mType = aMarked ? CCGraphDescriber::eGCMarkedObject
1689 : CCGraphDescriber::eGCedObject;
1690 d->mAddress = mCurrentAddress;
1691 d->mName.Append(aObjectDescription);
1692 if (aCompartmentAddress) {
1693 d->mCompartmentOrToAddress.AssignLiteral("0x");
1694 d->mCompartmentOrToAddress.AppendInt(aCompartmentAddress, 16);
1695 } else {
1696 d->mCompartmentOrToAddress.SetIsVoid(true);
1700 void NoteEdge(uint64_t aToAddress, const char* aEdgeName) {
1701 if (!mDisableLog) {
1702 fprintf(mCCLog, "> %p %s\n", (void*)aToAddress, aEdgeName);
1704 if (mWantAfterProcessing) {
1705 CCGraphDescriber* d = new CCGraphDescriber();
1706 mDescribers.insertBack(d);
1707 d->mType = CCGraphDescriber::eEdge;
1708 d->mAddress = mCurrentAddress;
1709 d->mCompartmentOrToAddress.AssignLiteral("0x");
1710 d->mCompartmentOrToAddress.AppendInt(aToAddress, 16);
1711 d->mName.Append(aEdgeName);
1714 void NoteWeakMapEntry(uint64_t aMap, uint64_t aKey, uint64_t aKeyDelegate,
1715 uint64_t aValue) {
1716 if (!mDisableLog) {
1717 fprintf(mCCLog, "WeakMapEntry map=%p key=%p keyDelegate=%p value=%p\n",
1718 (void*)aMap, (void*)aKey, (void*)aKeyDelegate, (void*)aValue);
1720 // We don't support after-processing for weak map entries.
1722 void NoteIncrementalRoot(uint64_t aAddress) {
1723 if (!mDisableLog) {
1724 fprintf(mCCLog, "IncrementalRoot %p\n", (void*)aAddress);
1726 // We don't support after-processing for incremental roots.
1728 void BeginResults() {
1729 if (!mDisableLog) {
1730 fputs("==========\n", mCCLog);
1733 void DescribeRoot(uint64_t aAddress, uint32_t aKnownEdges) {
1734 if (!mDisableLog) {
1735 fprintf(mCCLog, "%p [known=%u]\n", (void*)aAddress, aKnownEdges);
1737 if (mWantAfterProcessing) {
1738 CCGraphDescriber* d = new CCGraphDescriber();
1739 mDescribers.insertBack(d);
1740 d->mType = CCGraphDescriber::eRoot;
1741 d->mAddress.AppendInt(aAddress, 16);
1742 d->mCnt = aKnownEdges;
1745 void DescribeGarbage(uint64_t aAddress) {
1746 if (!mDisableLog) {
1747 fprintf(mCCLog, "%p [garbage]\n", (void*)aAddress);
1749 if (mWantAfterProcessing) {
1750 CCGraphDescriber* d = new CCGraphDescriber();
1751 mDescribers.insertBack(d);
1752 d->mType = CCGraphDescriber::eGarbage;
1753 d->mAddress.AppendInt(aAddress, 16);
1756 void End() {
1757 if (!mDisableLog) {
1758 mCCLog = nullptr;
1759 Unused << NS_WARN_IF(NS_FAILED(mLogSink->CloseCCLog()));
1762 NS_IMETHOD ProcessNext(nsICycleCollectorHandler* aHandler,
1763 bool* aCanContinue) override {
1764 if (NS_WARN_IF(!aHandler) || NS_WARN_IF(!mWantAfterProcessing)) {
1765 return NS_ERROR_UNEXPECTED;
1767 CCGraphDescriber* d = mDescribers.popFirst();
1768 if (d) {
1769 switch (d->mType) {
1770 case CCGraphDescriber::eRefCountedObject:
1771 aHandler->NoteRefCountedObject(d->mAddress, d->mCnt, d->mName);
1772 break;
1773 case CCGraphDescriber::eGCedObject:
1774 case CCGraphDescriber::eGCMarkedObject:
1775 aHandler->NoteGCedObject(
1776 d->mAddress, d->mType == CCGraphDescriber::eGCMarkedObject,
1777 d->mName, d->mCompartmentOrToAddress);
1778 break;
1779 case CCGraphDescriber::eEdge:
1780 aHandler->NoteEdge(d->mAddress, d->mCompartmentOrToAddress, d->mName);
1781 break;
1782 case CCGraphDescriber::eRoot:
1783 aHandler->DescribeRoot(d->mAddress, d->mCnt);
1784 break;
1785 case CCGraphDescriber::eGarbage:
1786 aHandler->DescribeGarbage(d->mAddress);
1787 break;
1788 case CCGraphDescriber::eUnknown:
1789 MOZ_ASSERT_UNREACHABLE("CCGraphDescriber::eUnknown");
1790 break;
1792 delete d;
1794 if (!(*aCanContinue = !mDescribers.isEmpty())) {
1795 mCurrentAddress.AssignLiteral("0x");
1797 return NS_OK;
1799 NS_IMETHOD AsLogger(nsCycleCollectorLogger** aRetVal) override {
1800 RefPtr<nsCycleCollectorLogger> rval = this;
1801 rval.forget(aRetVal);
1802 return NS_OK;
1805 private:
1806 void ClearDescribers() {
1807 CCGraphDescriber* d;
1808 while ((d = mDescribers.popFirst())) {
1809 delete d;
1813 nsCOMPtr<nsICycleCollectorLogSink> mLogSink;
1814 bool mWantAllTraces;
1815 bool mDisableLog;
1816 bool mWantAfterProcessing;
1817 nsCString mCurrentAddress;
1818 mozilla::LinkedList<CCGraphDescriber> mDescribers;
1819 FILE* mCCLog;
1822 NS_IMPL_ISUPPORTS(nsCycleCollectorLogger, nsICycleCollectorListener)
1824 already_AddRefed<nsICycleCollectorListener> nsCycleCollector_createLogger() {
1825 nsCOMPtr<nsICycleCollectorListener> logger = new nsCycleCollectorLogger();
1826 return logger.forget();
1829 static bool GCThingIsGrayCCThing(JS::GCCellPtr thing) {
1830 return JS::IsCCTraceKind(thing.kind()) && JS::GCThingIsMarkedGrayInCC(thing);
1833 static bool ValueIsGrayCCThing(const JS::Value& value) {
1834 return JS::IsCCTraceKind(value.traceKind()) &&
1835 JS::GCThingIsMarkedGray(value.toGCCellPtr());
1838 ////////////////////////////////////////////////////////////////////////
1839 // Bacon & Rajan's |MarkRoots| routine.
1840 ////////////////////////////////////////////////////////////////////////
1842 class CCGraphBuilder final : public nsCycleCollectionTraversalCallback,
1843 public nsCycleCollectionNoteRootCallback {
1844 private:
1845 CCGraph& mGraph;
1846 CycleCollectorResults& mResults;
1847 NodePool::Builder mNodeBuilder;
1848 EdgePool::Builder mEdgeBuilder;
1849 MOZ_INIT_OUTSIDE_CTOR PtrInfo* mCurrPi;
1850 nsCycleCollectionParticipant* mJSParticipant;
1851 nsCycleCollectionParticipant* mJSZoneParticipant;
1852 nsCString mNextEdgeName;
1853 RefPtr<nsCycleCollectorLogger> mLogger;
1854 bool mMergeZones;
1855 UniquePtr<NodePool::Enumerator> mCurrNode;
1856 uint32_t mNoteChildCount;
1858 struct PtrInfoCache : public MruCache<void*, PtrInfo*, PtrInfoCache, 491> {
1859 static HashNumber Hash(const void* aKey) { return HashGeneric(aKey); }
1860 static bool Match(const void* aKey, const PtrInfo* aVal) {
1861 return aVal->mPointer == aKey;
1865 PtrInfoCache mGraphCache;
1867 public:
1868 CCGraphBuilder(CCGraph& aGraph, CycleCollectorResults& aResults,
1869 CycleCollectedJSRuntime* aCCRuntime,
1870 nsCycleCollectorLogger* aLogger, bool aMergeZones);
1871 virtual ~CCGraphBuilder();
1873 bool WantAllTraces() const {
1874 return nsCycleCollectionNoteRootCallback::WantAllTraces();
1877 bool AddPurpleRoot(void* aRoot, nsCycleCollectionParticipant* aParti);
1879 // This is called when all roots have been added to the graph, to prepare for
1880 // BuildGraph().
1881 void DoneAddingRoots();
1883 // Do some work traversing nodes in the graph. Returns true if this graph
1884 // building is finished.
1885 bool BuildGraph(SliceBudget& aBudget);
1887 void RemoveCachedEntry(void* aPtr) { mGraphCache.Remove(aPtr); }
1889 private:
1890 PtrInfo* AddNode(void* aPtr, nsCycleCollectionParticipant* aParticipant);
1891 PtrInfo* AddWeakMapNode(JS::GCCellPtr aThing);
1892 PtrInfo* AddWeakMapNode(JSObject* aObject);
1894 void SetFirstChild() { mCurrPi->SetFirstChild(mEdgeBuilder.Mark()); }
1896 void SetLastChild() { mCurrPi->SetLastChild(mEdgeBuilder.Mark()); }
1898 public:
1899 // nsCycleCollectionNoteRootCallback methods.
1900 NS_IMETHOD_(void)
1901 NoteXPCOMRoot(nsISupports* aRoot,
1902 nsCycleCollectionParticipant* aParticipant) override;
1903 NS_IMETHOD_(void) NoteJSRoot(JSObject* aRoot) override;
1904 NS_IMETHOD_(void)
1905 NoteNativeRoot(void* aRoot,
1906 nsCycleCollectionParticipant* aParticipant) override;
1907 NS_IMETHOD_(void)
1908 NoteWeakMapping(JSObject* aMap, JS::GCCellPtr aKey, JSObject* aKdelegate,
1909 JS::GCCellPtr aVal) override;
1910 // This is used to create synthetic non-refcounted references to
1911 // nsXPCWrappedJS from their wrapped JS objects. No map is needed, because
1912 // the SubjectToFinalization list is like a known-black weak map, and
1913 // no delegate is needed because the keys are all unwrapped objects.
1914 NS_IMETHOD_(void)
1915 NoteWeakMapping(JSObject* aKey, nsISupports* aVal,
1916 nsCycleCollectionParticipant* aValParticipant) override;
1918 // nsCycleCollectionTraversalCallback methods.
1919 NS_IMETHOD_(void)
1920 DescribeRefCountedNode(nsrefcnt aRefCount, const char* aObjName) override;
1921 NS_IMETHOD_(void)
1922 DescribeGCedNode(bool aIsMarked, const char* aObjName,
1923 uint64_t aCompartmentAddress) override;
1925 NS_IMETHOD_(void) NoteXPCOMChild(nsISupports* aChild) override;
1926 NS_IMETHOD_(void) NoteJSChild(JS::GCCellPtr aThing) override;
1927 NS_IMETHOD_(void)
1928 NoteNativeChild(void* aChild,
1929 nsCycleCollectionParticipant* aParticipant) override;
1930 NS_IMETHOD_(void) NoteNextEdgeName(const char* aName) override;
1932 private:
1933 NS_IMETHOD_(void)
1934 NoteRoot(void* aRoot, nsCycleCollectionParticipant* aParticipant) {
1935 MOZ_ASSERT(aRoot);
1936 MOZ_ASSERT(aParticipant);
1938 if (!aParticipant->CanSkipInCC(aRoot) || MOZ_UNLIKELY(WantAllTraces())) {
1939 AddNode(aRoot, aParticipant);
1943 NS_IMETHOD_(void)
1944 NoteChild(void* aChild, nsCycleCollectionParticipant* aCp,
1945 nsCString& aEdgeName) {
1946 PtrInfo* childPi = AddNode(aChild, aCp);
1947 if (!childPi) {
1948 return;
1950 mEdgeBuilder.Add(childPi);
1951 if (mLogger) {
1952 mLogger->NoteEdge((uint64_t)aChild, aEdgeName.get());
1954 ++childPi->mInternalRefs;
1957 JS::Zone* MergeZone(JS::GCCellPtr aGcthing) {
1958 if (!mMergeZones) {
1959 return nullptr;
1961 JS::Zone* zone = JS::GetTenuredGCThingZone(aGcthing);
1962 if (js::IsSystemZone(zone)) {
1963 return nullptr;
1965 return zone;
1969 CCGraphBuilder::CCGraphBuilder(CCGraph& aGraph, CycleCollectorResults& aResults,
1970 CycleCollectedJSRuntime* aCCRuntime,
1971 nsCycleCollectorLogger* aLogger,
1972 bool aMergeZones)
1973 : mGraph(aGraph),
1974 mResults(aResults),
1975 mNodeBuilder(aGraph.mNodes),
1976 mEdgeBuilder(aGraph.mEdges),
1977 mJSParticipant(nullptr),
1978 mJSZoneParticipant(nullptr),
1979 mLogger(aLogger),
1980 mMergeZones(aMergeZones),
1981 mNoteChildCount(0) {
1982 // 4096 is an allocation bucket size.
1983 static_assert(sizeof(CCGraphBuilder) <= 4096,
1984 "Don't create too large CCGraphBuilder objects");
1986 if (aCCRuntime) {
1987 mJSParticipant = aCCRuntime->GCThingParticipant();
1988 mJSZoneParticipant = aCCRuntime->ZoneParticipant();
1991 if (mLogger) {
1992 mFlags |= nsCycleCollectionTraversalCallback::WANT_DEBUG_INFO;
1993 if (mLogger->IsAllTraces()) {
1994 mFlags |= nsCycleCollectionTraversalCallback::WANT_ALL_TRACES;
1995 mWantAllTraces = true; // for nsCycleCollectionNoteRootCallback
1999 mMergeZones = mMergeZones && MOZ_LIKELY(!WantAllTraces());
2001 MOZ_ASSERT(nsCycleCollectionNoteRootCallback::WantAllTraces() ==
2002 nsCycleCollectionTraversalCallback::WantAllTraces());
2005 CCGraphBuilder::~CCGraphBuilder() = default;
2007 PtrInfo* CCGraphBuilder::AddNode(void* aPtr,
2008 nsCycleCollectionParticipant* aParticipant) {
2009 if (mGraph.mOutOfMemory) {
2010 return nullptr;
2013 PtrInfoCache::Entry cached = mGraphCache.Lookup(aPtr);
2014 if (cached) {
2015 #ifdef DEBUG
2016 if (cached.Data()->mParticipant != aParticipant) {
2017 auto* parti1 = cached.Data()->mParticipant;
2018 auto* parti2 = aParticipant;
2019 NS_WARNING(
2020 nsPrintfCString("cached participant: %s; AddNode participant: %s\n",
2021 parti1 ? parti1->ClassName() : "null",
2022 parti2 ? parti2->ClassName() : "null")
2023 .get());
2025 #endif
2026 MOZ_ASSERT(cached.Data()->mParticipant == aParticipant,
2027 "nsCycleCollectionParticipant shouldn't change!");
2028 return cached.Data();
2031 PtrInfo* result;
2032 auto p = mGraph.mPtrInfoMap.lookupForAdd(aPtr);
2033 if (!p) {
2034 // New entry
2035 result = mNodeBuilder.Add(aPtr, aParticipant);
2036 if (!result) {
2037 return nullptr;
2040 if (!mGraph.mPtrInfoMap.add(p, result)) {
2041 // `result` leaks here, but we can't free it because it's
2042 // pool-allocated within NodePool.
2043 mGraph.mOutOfMemory = true;
2044 MOZ_ASSERT(false, "OOM while building cycle collector graph");
2045 return nullptr;
2048 } else {
2049 result = *p;
2050 MOZ_ASSERT(result->mParticipant == aParticipant,
2051 "nsCycleCollectionParticipant shouldn't change!");
2054 cached.Set(result);
2056 return result;
2059 bool CCGraphBuilder::AddPurpleRoot(void* aRoot,
2060 nsCycleCollectionParticipant* aParti) {
2061 ToParticipant(aRoot, &aParti);
2063 if (WantAllTraces() || !aParti->CanSkipInCC(aRoot)) {
2064 PtrInfo* pinfo = AddNode(aRoot, aParti);
2065 if (!pinfo) {
2066 return false;
2070 return true;
2073 void CCGraphBuilder::DoneAddingRoots() {
2074 // We've finished adding roots, and everything in the graph is a root.
2075 mGraph.mRootCount = mGraph.MapCount();
2077 mCurrNode = MakeUnique<NodePool::Enumerator>(mGraph.mNodes);
2080 MOZ_NEVER_INLINE bool CCGraphBuilder::BuildGraph(SliceBudget& aBudget) {
2081 MOZ_ASSERT(mCurrNode);
2083 while (!aBudget.isOverBudget() && !mCurrNode->IsDone()) {
2084 mNoteChildCount = 0;
2086 PtrInfo* pi = mCurrNode->GetNext();
2087 if (!pi) {
2088 MOZ_CRASH();
2091 mCurrPi = pi;
2093 // We need to call SetFirstChild() even on deleted nodes, to set their
2094 // firstChild() that may be read by a prior non-deleted neighbor.
2095 SetFirstChild();
2097 if (pi->mParticipant) {
2098 nsresult rv = pi->mParticipant->TraverseNativeAndJS(pi->mPointer, *this);
2099 MOZ_RELEASE_ASSERT(!NS_FAILED(rv),
2100 "Cycle collector Traverse method failed");
2103 if (mCurrNode->AtBlockEnd()) {
2104 SetLastChild();
2107 aBudget.step(mNoteChildCount + 1);
2110 if (!mCurrNode->IsDone()) {
2111 return false;
2114 if (mGraph.mRootCount > 0) {
2115 SetLastChild();
2118 mCurrNode = nullptr;
2120 return true;
2123 NS_IMETHODIMP_(void)
2124 CCGraphBuilder::NoteXPCOMRoot(nsISupports* aRoot,
2125 nsCycleCollectionParticipant* aParticipant) {
2126 MOZ_ASSERT(aRoot == CanonicalizeXPCOMParticipant(aRoot));
2128 #ifdef DEBUG
2129 nsXPCOMCycleCollectionParticipant* cp;
2130 ToParticipant(aRoot, &cp);
2131 MOZ_ASSERT(aParticipant == cp);
2132 #endif
2134 NoteRoot(aRoot, aParticipant);
2137 NS_IMETHODIMP_(void)
2138 CCGraphBuilder::NoteJSRoot(JSObject* aRoot) {
2139 if (JS::Zone* zone = MergeZone(JS::GCCellPtr(aRoot))) {
2140 NoteRoot(zone, mJSZoneParticipant);
2141 } else {
2142 NoteRoot(aRoot, mJSParticipant);
2146 NS_IMETHODIMP_(void)
2147 CCGraphBuilder::NoteNativeRoot(void* aRoot,
2148 nsCycleCollectionParticipant* aParticipant) {
2149 NoteRoot(aRoot, aParticipant);
2152 NS_IMETHODIMP_(void)
2153 CCGraphBuilder::DescribeRefCountedNode(nsrefcnt aRefCount,
2154 const char* aObjName) {
2155 mCurrPi->AnnotatedReleaseAssert(aRefCount != 0,
2156 "CCed refcounted object has zero refcount");
2157 mCurrPi->AnnotatedReleaseAssert(
2158 aRefCount != UINT32_MAX,
2159 "CCed refcounted object has overflowing refcount");
2161 mResults.mVisitedRefCounted++;
2163 if (mLogger) {
2164 mLogger->NoteRefCountedObject((uint64_t)mCurrPi->mPointer, aRefCount,
2165 aObjName);
2168 mCurrPi->mRefCount = aRefCount;
2171 NS_IMETHODIMP_(void)
2172 CCGraphBuilder::DescribeGCedNode(bool aIsMarked, const char* aObjName,
2173 uint64_t aCompartmentAddress) {
2174 uint32_t refCount = aIsMarked ? UINT32_MAX : 0;
2175 mResults.mVisitedGCed++;
2177 if (mLogger) {
2178 mLogger->NoteGCedObject((uint64_t)mCurrPi->mPointer, aIsMarked, aObjName,
2179 aCompartmentAddress);
2182 mCurrPi->mRefCount = refCount;
2185 NS_IMETHODIMP_(void)
2186 CCGraphBuilder::NoteXPCOMChild(nsISupports* aChild) {
2187 nsCString edgeName;
2188 if (WantDebugInfo()) {
2189 edgeName.Assign(mNextEdgeName);
2190 mNextEdgeName.Truncate();
2192 if (!aChild || !(aChild = CanonicalizeXPCOMParticipant(aChild))) {
2193 return;
2196 ++mNoteChildCount;
2198 nsXPCOMCycleCollectionParticipant* cp;
2199 ToParticipant(aChild, &cp);
2200 if (cp && (!cp->CanSkipThis(aChild) || WantAllTraces())) {
2201 NoteChild(aChild, cp, edgeName);
2205 NS_IMETHODIMP_(void)
2206 CCGraphBuilder::NoteNativeChild(void* aChild,
2207 nsCycleCollectionParticipant* aParticipant) {
2208 nsCString edgeName;
2209 if (WantDebugInfo()) {
2210 edgeName.Assign(mNextEdgeName);
2211 mNextEdgeName.Truncate();
2213 if (!aChild) {
2214 return;
2217 ++mNoteChildCount;
2219 MOZ_ASSERT(aParticipant, "Need a nsCycleCollectionParticipant!");
2220 if (!aParticipant->CanSkipThis(aChild) || WantAllTraces()) {
2221 NoteChild(aChild, aParticipant, edgeName);
2225 NS_IMETHODIMP_(void)
2226 CCGraphBuilder::NoteJSChild(JS::GCCellPtr aChild) {
2227 if (!aChild) {
2228 return;
2231 ++mNoteChildCount;
2233 nsCString edgeName;
2234 if (MOZ_UNLIKELY(WantDebugInfo())) {
2235 edgeName.Assign(mNextEdgeName);
2236 mNextEdgeName.Truncate();
2239 if (GCThingIsGrayCCThing(aChild) || MOZ_UNLIKELY(WantAllTraces())) {
2240 if (JS::Zone* zone = MergeZone(aChild)) {
2241 NoteChild(zone, mJSZoneParticipant, edgeName);
2242 } else {
2243 NoteChild(aChild.asCell(), mJSParticipant, edgeName);
2248 NS_IMETHODIMP_(void)
2249 CCGraphBuilder::NoteNextEdgeName(const char* aName) {
2250 if (WantDebugInfo()) {
2251 mNextEdgeName = aName;
2255 PtrInfo* CCGraphBuilder::AddWeakMapNode(JS::GCCellPtr aNode) {
2256 MOZ_ASSERT(aNode, "Weak map node should be non-null.");
2258 if (!GCThingIsGrayCCThing(aNode) && !WantAllTraces()) {
2259 return nullptr;
2262 if (JS::Zone* zone = MergeZone(aNode)) {
2263 return AddNode(zone, mJSZoneParticipant);
2265 return AddNode(aNode.asCell(), mJSParticipant);
2268 PtrInfo* CCGraphBuilder::AddWeakMapNode(JSObject* aObject) {
2269 return AddWeakMapNode(JS::GCCellPtr(aObject));
2272 NS_IMETHODIMP_(void)
2273 CCGraphBuilder::NoteWeakMapping(JSObject* aMap, JS::GCCellPtr aKey,
2274 JSObject* aKdelegate, JS::GCCellPtr aVal) {
2275 // Don't try to optimize away the entry here, as we've already attempted to
2276 // do that in TraceWeakMapping in nsXPConnect.
2277 WeakMapping* mapping = mGraph.mWeakMaps.AppendElement();
2278 mapping->mMap = aMap ? AddWeakMapNode(aMap) : nullptr;
2279 mapping->mKey = aKey ? AddWeakMapNode(aKey) : nullptr;
2280 mapping->mKeyDelegate =
2281 aKdelegate ? AddWeakMapNode(aKdelegate) : mapping->mKey;
2282 mapping->mVal = aVal ? AddWeakMapNode(aVal) : nullptr;
2284 if (mLogger) {
2285 mLogger->NoteWeakMapEntry((uint64_t)aMap, aKey ? aKey.unsafeAsInteger() : 0,
2286 (uint64_t)aKdelegate,
2287 aVal ? aVal.unsafeAsInteger() : 0);
2291 NS_IMETHODIMP_(void)
2292 CCGraphBuilder::NoteWeakMapping(JSObject* aKey, nsISupports* aVal,
2293 nsCycleCollectionParticipant* aValParticipant) {
2294 MOZ_ASSERT(aKey, "Don't call NoteWeakMapping with a null key");
2295 MOZ_ASSERT(aVal, "Don't call NoteWeakMapping with a null value");
2296 WeakMapping* mapping = mGraph.mWeakMaps.AppendElement();
2297 mapping->mMap = nullptr;
2298 mapping->mKey = AddWeakMapNode(aKey);
2299 mapping->mKeyDelegate = mapping->mKey;
2300 MOZ_ASSERT(js::UncheckedUnwrapWithoutExpose(aKey) == aKey);
2301 mapping->mVal = AddNode(aVal, aValParticipant);
2303 if (mLogger) {
2304 mLogger->NoteWeakMapEntry(0, (uint64_t)aKey, 0, (uint64_t)aVal);
2308 static bool AddPurpleRoot(CCGraphBuilder& aBuilder, void* aRoot,
2309 nsCycleCollectionParticipant* aParti) {
2310 return aBuilder.AddPurpleRoot(aRoot, aParti);
2313 // MayHaveChild() will be false after a Traverse if the object does
2314 // not have any children the CC will visit.
2315 class ChildFinder : public nsCycleCollectionTraversalCallback {
2316 public:
2317 ChildFinder() : mMayHaveChild(false) {}
2319 // The logic of the Note*Child functions must mirror that of their
2320 // respective functions in CCGraphBuilder.
2321 NS_IMETHOD_(void) NoteXPCOMChild(nsISupports* aChild) override;
2322 NS_IMETHOD_(void)
2323 NoteNativeChild(void* aChild, nsCycleCollectionParticipant* aHelper) override;
2324 NS_IMETHOD_(void) NoteJSChild(JS::GCCellPtr aThing) override;
2326 NS_IMETHOD_(void)
2327 NoteWeakMapping(JSObject* aKey, nsISupports* aVal,
2328 nsCycleCollectionParticipant* aValParticipant) override {}
2330 NS_IMETHOD_(void)
2331 DescribeRefCountedNode(nsrefcnt aRefcount, const char* aObjname) override {}
2332 NS_IMETHOD_(void)
2333 DescribeGCedNode(bool aIsMarked, const char* aObjname,
2334 uint64_t aCompartmentAddress) override {}
2335 NS_IMETHOD_(void) NoteNextEdgeName(const char* aName) override {}
2336 bool MayHaveChild() { return mMayHaveChild; }
2338 private:
2339 bool mMayHaveChild;
2342 NS_IMETHODIMP_(void)
2343 ChildFinder::NoteXPCOMChild(nsISupports* aChild) {
2344 if (!aChild || !(aChild = CanonicalizeXPCOMParticipant(aChild))) {
2345 return;
2347 nsXPCOMCycleCollectionParticipant* cp;
2348 ToParticipant(aChild, &cp);
2349 if (cp && !cp->CanSkip(aChild, true)) {
2350 mMayHaveChild = true;
2354 NS_IMETHODIMP_(void)
2355 ChildFinder::NoteNativeChild(void* aChild,
2356 nsCycleCollectionParticipant* aHelper) {
2357 if (!aChild) {
2358 return;
2360 MOZ_ASSERT(aHelper, "Native child must have a participant");
2361 if (!aHelper->CanSkip(aChild, true)) {
2362 mMayHaveChild = true;
2366 NS_IMETHODIMP_(void)
2367 ChildFinder::NoteJSChild(JS::GCCellPtr aChild) {
2368 if (aChild && JS::GCThingIsMarkedGray(aChild)) {
2369 mMayHaveChild = true;
2373 static bool MayHaveChild(void* aObj, nsCycleCollectionParticipant* aCp) {
2374 ChildFinder cf;
2375 aCp->TraverseNativeAndJS(aObj, cf);
2376 return cf.MayHaveChild();
2379 // JSPurpleBuffer keeps references to GCThings which might affect the
2380 // next cycle collection. It is owned only by itself and during unlink its
2381 // self reference is broken down and the object ends up killing itself.
2382 // If GC happens before CC, references to GCthings and the self reference are
2383 // removed.
2384 class JSPurpleBuffer {
2385 ~JSPurpleBuffer() {
2386 MOZ_ASSERT(mValues.IsEmpty());
2387 MOZ_ASSERT(mObjects.IsEmpty());
2390 public:
2391 explicit JSPurpleBuffer(RefPtr<JSPurpleBuffer>& aReferenceToThis)
2392 : mReferenceToThis(aReferenceToThis),
2393 mValues(kSegmentSize),
2394 mObjects(kSegmentSize) {
2395 mReferenceToThis = this;
2396 mozilla::HoldJSObjects(this);
2399 void Destroy() {
2400 RefPtr<JSPurpleBuffer> referenceToThis;
2401 mReferenceToThis.swap(referenceToThis);
2402 mValues.Clear();
2403 mObjects.Clear();
2404 mozilla::DropJSObjects(this);
2407 NS_INLINE_DECL_CYCLE_COLLECTING_NATIVE_REFCOUNTING(JSPurpleBuffer)
2408 NS_DECL_CYCLE_COLLECTION_SCRIPT_HOLDER_NATIVE_CLASS(JSPurpleBuffer)
2410 RefPtr<JSPurpleBuffer>& mReferenceToThis;
2412 // These are raw pointers instead of Heap<T> because we only need Heap<T> for
2413 // pointers which may point into the nursery. The purple buffer never contains
2414 // pointers to the nursery because nursery gcthings can never be gray and only
2415 // gray things can be inserted into the purple buffer.
2416 static const size_t kSegmentSize = 512;
2417 SegmentedVector<JS::Value, kSegmentSize, InfallibleAllocPolicy> mValues;
2418 SegmentedVector<JSObject*, kSegmentSize, InfallibleAllocPolicy> mObjects;
2421 NS_IMPL_CYCLE_COLLECTION_CLASS(JSPurpleBuffer)
2423 NS_IMPL_CYCLE_COLLECTION_UNLINK_BEGIN(JSPurpleBuffer)
2424 tmp->Destroy();
2425 NS_IMPL_CYCLE_COLLECTION_UNLINK_END
2427 NS_IMPL_CYCLE_COLLECTION_TRAVERSE_BEGIN(JSPurpleBuffer)
2428 CycleCollectionNoteChild(cb, tmp, "self");
2429 NS_IMPL_CYCLE_COLLECTION_TRAVERSE_END
2431 #define NS_TRACE_SEGMENTED_ARRAY(_field, _type) \
2433 for (auto iter = tmp->_field.Iter(); !iter.Done(); iter.Next()) { \
2434 js::gc::CallTraceCallbackOnNonHeap<_type, TraceCallbacks>( \
2435 &iter.Get(), aCallbacks, #_field, aClosure); \
2439 NS_IMPL_CYCLE_COLLECTION_TRACE_BEGIN(JSPurpleBuffer)
2440 NS_TRACE_SEGMENTED_ARRAY(mValues, JS::Value)
2441 NS_TRACE_SEGMENTED_ARRAY(mObjects, JSObject*)
2442 NS_IMPL_CYCLE_COLLECTION_TRACE_END
2444 class SnowWhiteKiller : public TraceCallbacks {
2445 struct SnowWhiteObject {
2446 void* mPointer;
2447 nsCycleCollectionParticipant* mParticipant;
2448 nsCycleCollectingAutoRefCnt* mRefCnt;
2451 // Segments are 4 KiB on 32-bit and 8 KiB on 64-bit.
2452 static const size_t kSegmentSize = sizeof(void*) * 1024;
2453 typedef SegmentedVector<SnowWhiteObject, kSegmentSize, InfallibleAllocPolicy>
2454 ObjectsVector;
2456 public:
2457 SnowWhiteKiller(nsCycleCollector* aCollector, js::SliceBudget* aBudget)
2458 : mCollector(aCollector),
2459 mObjects(kSegmentSize),
2460 mBudget(aBudget),
2461 mSawSnowWhiteObjects(false) {
2462 MOZ_ASSERT(mCollector, "Calling SnowWhiteKiller after nsCC went away");
2465 explicit SnowWhiteKiller(nsCycleCollector* aCollector)
2466 : SnowWhiteKiller(aCollector, nullptr) {}
2468 ~SnowWhiteKiller() {
2469 for (auto iter = mObjects.Iter(); !iter.Done(); iter.Next()) {
2470 SnowWhiteObject& o = iter.Get();
2471 MaybeKillObject(o);
2475 private:
2476 void MaybeKillObject(SnowWhiteObject& aObject) {
2477 if (!aObject.mRefCnt->get() && !aObject.mRefCnt->IsInPurpleBuffer()) {
2478 mCollector->RemoveObjectFromGraph(aObject.mPointer);
2479 aObject.mRefCnt->stabilizeForDeletion();
2481 JS::AutoEnterCycleCollection autocc(mCollector->Runtime()->Runtime());
2482 aObject.mParticipant->Trace(aObject.mPointer, *this, nullptr);
2484 aObject.mParticipant->DeleteCycleCollectable(aObject.mPointer);
2488 public:
2489 bool Visit(nsPurpleBuffer& aBuffer, nsPurpleBufferEntry* aEntry) {
2490 if (mBudget) {
2491 if (mBudget->isOverBudget()) {
2492 return false;
2494 mBudget->step();
2497 MOZ_ASSERT(aEntry->mObject, "Null object in purple buffer");
2498 if (!aEntry->mRefCnt->get()) {
2499 mSawSnowWhiteObjects = true;
2500 void* o = aEntry->mObject;
2501 nsCycleCollectionParticipant* cp = aEntry->mParticipant;
2502 ToParticipant(o, &cp);
2503 SnowWhiteObject swo = {o, cp, aEntry->mRefCnt};
2504 if (!mBudget) {
2505 mObjects.InfallibleAppend(swo);
2507 aBuffer.Remove(aEntry);
2508 if (mBudget) {
2509 MaybeKillObject(swo);
2512 return true;
2515 bool HasSnowWhiteObjects() const { return !mObjects.IsEmpty(); }
2517 bool SawSnowWhiteObjects() const { return mSawSnowWhiteObjects; }
2519 virtual void Trace(JS::Heap<JS::Value>* aValue, const char* aName,
2520 void* aClosure) const override {
2521 const JS::Value& val = aValue->unbarrieredGet();
2522 if (val.isGCThing() && ValueIsGrayCCThing(val)) {
2523 MOZ_ASSERT(!js::gc::IsInsideNursery(val.toGCThing()));
2524 mCollector->GetJSPurpleBuffer()->mValues.InfallibleAppend(val);
2528 virtual void Trace(JS::Heap<jsid>* aId, const char* aName,
2529 void* aClosure) const override {}
2531 void AppendJSObjectToPurpleBuffer(JSObject* obj) const {
2532 if (obj && JS::ObjectIsMarkedGray(obj)) {
2533 MOZ_ASSERT(JS::ObjectIsTenured(obj));
2534 mCollector->GetJSPurpleBuffer()->mObjects.InfallibleAppend(obj);
2538 virtual void Trace(JS::Heap<JSObject*>* aObject, const char* aName,
2539 void* aClosure) const override {
2540 AppendJSObjectToPurpleBuffer(aObject->unbarrieredGet());
2543 virtual void Trace(nsWrapperCache* aWrapperCache, const char* aName,
2544 void* aClosure) const override {
2545 AppendJSObjectToPurpleBuffer(aWrapperCache->GetWrapperPreserveColor());
2548 virtual void Trace(JS::TenuredHeap<JSObject*>* aObject, const char* aName,
2549 void* aClosure) const override {
2550 AppendJSObjectToPurpleBuffer(aObject->unbarrieredGetPtr());
2553 virtual void Trace(JS::Heap<JSString*>* aString, const char* aName,
2554 void* aClosure) const override {}
2556 virtual void Trace(JS::Heap<JSScript*>* aScript, const char* aName,
2557 void* aClosure) const override {}
2559 virtual void Trace(JS::Heap<JSFunction*>* aFunction, const char* aName,
2560 void* aClosure) const override {}
2562 private:
2563 RefPtr<nsCycleCollector> mCollector;
2564 ObjectsVector mObjects;
2565 js::SliceBudget* mBudget;
2566 bool mSawSnowWhiteObjects;
2569 class RemoveSkippableVisitor : public SnowWhiteKiller {
2570 public:
2571 RemoveSkippableVisitor(nsCycleCollector* aCollector, js::SliceBudget& aBudget,
2572 bool aRemoveChildlessNodes,
2573 bool aAsyncSnowWhiteFreeing,
2574 CC_ForgetSkippableCallback aCb)
2575 : SnowWhiteKiller(aCollector),
2576 mBudget(aBudget),
2577 mRemoveChildlessNodes(aRemoveChildlessNodes),
2578 mAsyncSnowWhiteFreeing(aAsyncSnowWhiteFreeing),
2579 mDispatchedDeferredDeletion(false),
2580 mCallback(aCb) {}
2582 ~RemoveSkippableVisitor() {
2583 // Note, we must call the callback before SnowWhiteKiller calls
2584 // DeleteCycleCollectable!
2585 if (mCallback) {
2586 mCallback();
2588 if (HasSnowWhiteObjects()) {
2589 // Effectively a continuation.
2590 nsCycleCollector_dispatchDeferredDeletion(true);
2594 bool Visit(nsPurpleBuffer& aBuffer, nsPurpleBufferEntry* aEntry) {
2595 if (mBudget.isOverBudget()) {
2596 return false;
2599 // CanSkip calls can be a bit slow, so increase the likelihood that
2600 // isOverBudget actually checks whether we're over the time budget.
2601 mBudget.step(5);
2602 MOZ_ASSERT(aEntry->mObject, "null mObject in purple buffer");
2603 if (!aEntry->mRefCnt->get()) {
2604 if (!mAsyncSnowWhiteFreeing) {
2605 SnowWhiteKiller::Visit(aBuffer, aEntry);
2606 } else if (!mDispatchedDeferredDeletion) {
2607 mDispatchedDeferredDeletion = true;
2608 nsCycleCollector_dispatchDeferredDeletion(false);
2610 return true;
2612 void* o = aEntry->mObject;
2613 nsCycleCollectionParticipant* cp = aEntry->mParticipant;
2614 ToParticipant(o, &cp);
2615 if (aEntry->mRefCnt->IsPurple() && !cp->CanSkip(o, false) &&
2616 (!mRemoveChildlessNodes || MayHaveChild(o, cp))) {
2617 return true;
2619 aBuffer.Remove(aEntry);
2620 return true;
2623 private:
2624 js::SliceBudget& mBudget;
2625 bool mRemoveChildlessNodes;
2626 bool mAsyncSnowWhiteFreeing;
2627 bool mDispatchedDeferredDeletion;
2628 CC_ForgetSkippableCallback mCallback;
2631 void nsPurpleBuffer::RemoveSkippable(nsCycleCollector* aCollector,
2632 js::SliceBudget& aBudget,
2633 bool aRemoveChildlessNodes,
2634 bool aAsyncSnowWhiteFreeing,
2635 CC_ForgetSkippableCallback aCb) {
2636 RemoveSkippableVisitor visitor(aCollector, aBudget, aRemoveChildlessNodes,
2637 aAsyncSnowWhiteFreeing, aCb);
2638 VisitEntries(visitor);
2641 bool nsCycleCollector::FreeSnowWhite(bool aUntilNoSWInPurpleBuffer) {
2642 CheckThreadSafety();
2644 if (mFreeingSnowWhite) {
2645 return false;
2648 AUTO_PROFILER_LABEL_CATEGORY_PAIR(GCCC_FreeSnowWhite);
2650 AutoRestore<bool> ar(mFreeingSnowWhite);
2651 mFreeingSnowWhite = true;
2653 bool hadSnowWhiteObjects = false;
2654 do {
2655 SnowWhiteKiller visitor(this);
2656 mPurpleBuf.VisitEntries(visitor);
2657 hadSnowWhiteObjects = hadSnowWhiteObjects || visitor.HasSnowWhiteObjects();
2658 if (!visitor.HasSnowWhiteObjects()) {
2659 break;
2661 } while (aUntilNoSWInPurpleBuffer);
2662 return hadSnowWhiteObjects;
2665 bool nsCycleCollector::FreeSnowWhiteWithBudget(js::SliceBudget& aBudget) {
2666 CheckThreadSafety();
2668 if (mFreeingSnowWhite) {
2669 return false;
2672 AUTO_PROFILER_LABEL_CATEGORY_PAIR(GCCC_FreeSnowWhite);
2673 AutoRestore<bool> ar(mFreeingSnowWhite);
2674 mFreeingSnowWhite = true;
2676 SnowWhiteKiller visitor(this, &aBudget);
2677 mPurpleBuf.VisitEntries(visitor);
2678 return visitor.SawSnowWhiteObjects();
2682 void nsCycleCollector::ForgetSkippable(js::SliceBudget& aBudget,
2683 bool aRemoveChildlessNodes,
2684 bool aAsyncSnowWhiteFreeing) {
2685 CheckThreadSafety();
2687 if (mFreeingSnowWhite) {
2688 return;
2691 // If we remove things from the purple buffer during graph building, we may
2692 // lose track of an object that was mutated during graph building.
2693 MOZ_ASSERT(IsIdle());
2695 if (mCCJSRuntime) {
2696 mCCJSRuntime->PrepareForForgetSkippable();
2698 MOZ_ASSERT(
2699 !mScanInProgress,
2700 "Don't forget skippable or free snow-white while scan is in progress.");
2701 mPurpleBuf.RemoveSkippable(this, aBudget, aRemoveChildlessNodes,
2702 aAsyncSnowWhiteFreeing, mForgetSkippableCB);
2705 MOZ_NEVER_INLINE void nsCycleCollector::MarkRoots(SliceBudget& aBudget) {
2706 JS::AutoAssertNoGC nogc;
2707 TimeLog timeLog;
2708 AutoRestore<bool> ar(mScanInProgress);
2709 MOZ_RELEASE_ASSERT(!mScanInProgress);
2710 mScanInProgress = true;
2711 MOZ_ASSERT(mIncrementalPhase == GraphBuildingPhase);
2713 AUTO_PROFILER_LABEL_CATEGORY_PAIR(GCCC_BuildGraph);
2714 JS::AutoEnterCycleCollection autocc(Runtime()->Runtime());
2715 bool doneBuilding = mBuilder->BuildGraph(aBudget);
2717 if (!doneBuilding) {
2718 timeLog.Checkpoint("MarkRoots()");
2719 return;
2722 mBuilder = nullptr;
2723 mIncrementalPhase = ScanAndCollectWhitePhase;
2724 timeLog.Checkpoint("MarkRoots()");
2727 ////////////////////////////////////////////////////////////////////////
2728 // Bacon & Rajan's |ScanRoots| routine.
2729 ////////////////////////////////////////////////////////////////////////
2731 struct ScanBlackVisitor {
2732 ScanBlackVisitor(uint32_t& aWhiteNodeCount, bool& aFailed)
2733 : mWhiteNodeCount(aWhiteNodeCount), mFailed(aFailed) {}
2735 bool ShouldVisitNode(PtrInfo const* aPi) { return aPi->mColor != black; }
2737 MOZ_NEVER_INLINE void VisitNode(PtrInfo* aPi) {
2738 if (aPi->mColor == white) {
2739 --mWhiteNodeCount;
2741 aPi->mColor = black;
2744 void Failed() { mFailed = true; }
2746 private:
2747 uint32_t& mWhiteNodeCount;
2748 bool& mFailed;
2751 static void FloodBlackNode(uint32_t& aWhiteNodeCount, bool& aFailed,
2752 PtrInfo* aPi) {
2753 GraphWalker<ScanBlackVisitor>(ScanBlackVisitor(aWhiteNodeCount, aFailed))
2754 .Walk(aPi);
2755 MOZ_ASSERT(aPi->mColor == black || !aPi->WasTraversed(),
2756 "FloodBlackNode should make aPi black");
2759 // Iterate over the WeakMaps. If we mark anything while iterating
2760 // over the WeakMaps, we must iterate over all of the WeakMaps again.
2761 void nsCycleCollector::ScanWeakMaps() {
2762 bool anyChanged;
2763 bool failed = false;
2764 do {
2765 anyChanged = false;
2766 for (uint32_t i = 0; i < mGraph.mWeakMaps.Length(); i++) {
2767 WeakMapping* wm = &mGraph.mWeakMaps[i];
2769 // If any of these are null, the original object was marked black.
2770 uint32_t mColor = wm->mMap ? wm->mMap->mColor : black;
2771 uint32_t kColor = wm->mKey ? wm->mKey->mColor : black;
2772 uint32_t kdColor = wm->mKeyDelegate ? wm->mKeyDelegate->mColor : black;
2773 uint32_t vColor = wm->mVal ? wm->mVal->mColor : black;
2775 MOZ_ASSERT(mColor != grey, "Uncolored weak map");
2776 MOZ_ASSERT(kColor != grey, "Uncolored weak map key");
2777 MOZ_ASSERT(kdColor != grey, "Uncolored weak map key delegate");
2778 MOZ_ASSERT(vColor != grey, "Uncolored weak map value");
2780 if (mColor == black && kColor != black && kdColor == black) {
2781 FloodBlackNode(mWhiteNodeCount, failed, wm->mKey);
2782 anyChanged = true;
2785 if (mColor == black && kColor == black && vColor != black) {
2786 FloodBlackNode(mWhiteNodeCount, failed, wm->mVal);
2787 anyChanged = true;
2790 } while (anyChanged);
2792 if (failed) {
2793 MOZ_ASSERT(false, "Ran out of memory in ScanWeakMaps");
2794 CC_TELEMETRY(_OOM, true);
2798 // Flood black from any objects in the purple buffer that are in the CC graph.
2799 class PurpleScanBlackVisitor {
2800 public:
2801 PurpleScanBlackVisitor(CCGraph& aGraph, nsCycleCollectorLogger* aLogger,
2802 uint32_t& aCount, bool& aFailed)
2803 : mGraph(aGraph), mLogger(aLogger), mCount(aCount), mFailed(aFailed) {}
2805 bool Visit(nsPurpleBuffer& aBuffer, nsPurpleBufferEntry* aEntry) {
2806 MOZ_ASSERT(aEntry->mObject,
2807 "Entries with null mObject shouldn't be in the purple buffer.");
2808 MOZ_ASSERT(aEntry->mRefCnt->get() != 0,
2809 "Snow-white objects shouldn't be in the purple buffer.");
2811 void* obj = aEntry->mObject;
2813 MOZ_ASSERT(
2814 aEntry->mParticipant ||
2815 CanonicalizeXPCOMParticipant(static_cast<nsISupports*>(obj)) == obj,
2816 "Suspect nsISupports pointer must be canonical");
2818 PtrInfo* pi = mGraph.FindNode(obj);
2819 if (!pi) {
2820 return true;
2822 MOZ_ASSERT(pi->mParticipant,
2823 "No dead objects should be in the purple buffer.");
2824 if (MOZ_UNLIKELY(mLogger)) {
2825 mLogger->NoteIncrementalRoot((uint64_t)pi->mPointer);
2827 if (pi->mColor == black) {
2828 return true;
2830 FloodBlackNode(mCount, mFailed, pi);
2831 return true;
2834 private:
2835 CCGraph& mGraph;
2836 RefPtr<nsCycleCollectorLogger> mLogger;
2837 uint32_t& mCount;
2838 bool& mFailed;
2841 // Objects that have been stored somewhere since the start of incremental graph
2842 // building must be treated as live for this cycle collection, because we may
2843 // not have accurate information about who holds references to them.
2844 void nsCycleCollector::ScanIncrementalRoots() {
2845 TimeLog timeLog;
2847 // Reference counted objects:
2848 // We cleared the purple buffer at the start of the current ICC, so if a
2849 // refcounted object is purple, it may have been AddRef'd during the current
2850 // ICC. (It may also have only been released.) If that is the case, we cannot
2851 // be sure that the set of things pointing to the object in the CC graph
2852 // is accurate. Therefore, for safety, we treat any purple objects as being
2853 // live during the current CC. We don't remove anything from the purple
2854 // buffer here, so these objects will be suspected and freed in the next CC
2855 // if they are garbage.
2856 bool failed = false;
2857 PurpleScanBlackVisitor purpleScanBlackVisitor(mGraph, mLogger,
2858 mWhiteNodeCount, failed);
2859 mPurpleBuf.VisitEntries(purpleScanBlackVisitor);
2860 timeLog.Checkpoint("ScanIncrementalRoots::fix purple");
2862 bool hasJSRuntime = !!mCCJSRuntime;
2863 nsCycleCollectionParticipant* jsParticipant =
2864 hasJSRuntime ? mCCJSRuntime->GCThingParticipant() : nullptr;
2865 nsCycleCollectionParticipant* zoneParticipant =
2866 hasJSRuntime ? mCCJSRuntime->ZoneParticipant() : nullptr;
2867 bool hasLogger = !!mLogger;
2869 NodePool::Enumerator etor(mGraph.mNodes);
2870 while (!etor.IsDone()) {
2871 PtrInfo* pi = etor.GetNext();
2873 // As an optimization, if an object has already been determined to be live,
2874 // don't consider it further. We can't do this if there is a listener,
2875 // because the listener wants to know the complete set of incremental roots.
2876 if (pi->mColor == black && MOZ_LIKELY(!hasLogger)) {
2877 continue;
2880 // Garbage collected objects:
2881 // If a GCed object was added to the graph with a refcount of zero, and is
2882 // now marked black by the GC, it was probably gray before and was exposed
2883 // to active JS, so it may have been stored somewhere, so it needs to be
2884 // treated as live.
2885 if (pi->IsGrayJS() && MOZ_LIKELY(hasJSRuntime)) {
2886 // If the object is still marked gray by the GC, nothing could have gotten
2887 // hold of it, so it isn't an incremental root.
2888 if (pi->mParticipant == jsParticipant) {
2889 JS::GCCellPtr ptr(pi->mPointer, JS::GCThingTraceKind(pi->mPointer));
2890 if (GCThingIsGrayCCThing(ptr)) {
2891 continue;
2893 } else if (pi->mParticipant == zoneParticipant) {
2894 JS::Zone* zone = static_cast<JS::Zone*>(pi->mPointer);
2895 if (js::ZoneGlobalsAreAllGray(zone)) {
2896 continue;
2898 } else {
2899 MOZ_ASSERT(false, "Non-JS thing with 0 refcount? Treating as live.");
2901 } else if (!pi->mParticipant && pi->WasTraversed()) {
2902 // Dead traversed refcounted objects:
2903 // If the object was traversed, it must have been alive at the start of
2904 // the CC, and thus had a positive refcount. It is dead now, so its
2905 // refcount must have decreased at some point during the CC. Therefore,
2906 // it would be in the purple buffer if it wasn't dead, so treat it as an
2907 // incremental root.
2909 // This should not cause leaks because as the object died it should have
2910 // released anything it held onto, which will add them to the purple
2911 // buffer, which will cause them to be considered in the next CC.
2912 } else {
2913 continue;
2916 // At this point, pi must be an incremental root.
2918 // If there's a listener, tell it about this root. We don't bother with the
2919 // optimization of skipping the Walk() if pi is black: it will just return
2920 // without doing anything and there's no need to make this case faster.
2921 if (MOZ_UNLIKELY(hasLogger) && pi->mPointer) {
2922 // Dead objects aren't logged. See bug 1031370.
2923 mLogger->NoteIncrementalRoot((uint64_t)pi->mPointer);
2926 FloodBlackNode(mWhiteNodeCount, failed, pi);
2929 timeLog.Checkpoint("ScanIncrementalRoots::fix nodes");
2931 if (failed) {
2932 NS_ASSERTION(false, "Ran out of memory in ScanIncrementalRoots");
2933 CC_TELEMETRY(_OOM, true);
2937 // Mark nodes white and make sure their refcounts are ok.
2938 // No nodes are marked black during this pass to ensure that refcount
2939 // checking is run on all nodes not marked black by ScanIncrementalRoots.
2940 void nsCycleCollector::ScanWhiteNodes(bool aFullySynchGraphBuild) {
2941 NodePool::Enumerator nodeEnum(mGraph.mNodes);
2942 while (!nodeEnum.IsDone()) {
2943 PtrInfo* pi = nodeEnum.GetNext();
2944 if (pi->mColor == black) {
2945 // Incremental roots can be in a nonsensical state, so don't
2946 // check them. This will miss checking nodes that are merely
2947 // reachable from incremental roots.
2948 MOZ_ASSERT(!aFullySynchGraphBuild,
2949 "In a synch CC, no nodes should be marked black early on.");
2950 continue;
2952 MOZ_ASSERT(pi->mColor == grey);
2954 if (!pi->WasTraversed()) {
2955 // This node was deleted before it was traversed, so there's no reason
2956 // to look at it.
2957 MOZ_ASSERT(!pi->mParticipant,
2958 "Live nodes should all have been traversed");
2959 continue;
2962 if (pi->mInternalRefs == pi->mRefCount || pi->IsGrayJS()) {
2963 pi->mColor = white;
2964 ++mWhiteNodeCount;
2965 continue;
2968 pi->AnnotatedReleaseAssert(
2969 pi->mInternalRefs <= pi->mRefCount,
2970 "More references to an object than its refcount");
2972 // This node will get marked black in the next pass.
2976 // Any remaining grey nodes that haven't already been deleted must be alive,
2977 // so mark them and their children black. Any nodes that are black must have
2978 // already had their children marked black, so there's no need to look at them
2979 // again. This pass may turn some white nodes to black.
2980 void nsCycleCollector::ScanBlackNodes() {
2981 bool failed = false;
2982 NodePool::Enumerator nodeEnum(mGraph.mNodes);
2983 while (!nodeEnum.IsDone()) {
2984 PtrInfo* pi = nodeEnum.GetNext();
2985 if (pi->mColor == grey && pi->WasTraversed()) {
2986 FloodBlackNode(mWhiteNodeCount, failed, pi);
2990 if (failed) {
2991 NS_ASSERTION(false, "Ran out of memory in ScanBlackNodes");
2992 CC_TELEMETRY(_OOM, true);
2996 void nsCycleCollector::ScanRoots(bool aFullySynchGraphBuild) {
2997 JS::AutoAssertNoGC nogc;
2998 AutoRestore<bool> ar(mScanInProgress);
2999 MOZ_RELEASE_ASSERT(!mScanInProgress);
3000 mScanInProgress = true;
3001 mWhiteNodeCount = 0;
3002 MOZ_ASSERT(mIncrementalPhase == ScanAndCollectWhitePhase);
3004 JS::AutoEnterCycleCollection autocc(Runtime()->Runtime());
3006 if (!aFullySynchGraphBuild) {
3007 ScanIncrementalRoots();
3010 TimeLog timeLog;
3011 ScanWhiteNodes(aFullySynchGraphBuild);
3012 timeLog.Checkpoint("ScanRoots::ScanWhiteNodes");
3014 ScanBlackNodes();
3015 timeLog.Checkpoint("ScanRoots::ScanBlackNodes");
3017 // Scanning weak maps must be done last.
3018 ScanWeakMaps();
3019 timeLog.Checkpoint("ScanRoots::ScanWeakMaps");
3021 if (mLogger) {
3022 mLogger->BeginResults();
3024 NodePool::Enumerator etor(mGraph.mNodes);
3025 while (!etor.IsDone()) {
3026 PtrInfo* pi = etor.GetNext();
3027 if (!pi->WasTraversed()) {
3028 continue;
3030 switch (pi->mColor) {
3031 case black:
3032 if (!pi->IsGrayJS() && !pi->IsBlackJS() &&
3033 pi->mInternalRefs != pi->mRefCount) {
3034 mLogger->DescribeRoot((uint64_t)pi->mPointer, pi->mInternalRefs);
3036 break;
3037 case white:
3038 mLogger->DescribeGarbage((uint64_t)pi->mPointer);
3039 break;
3040 case grey:
3041 MOZ_ASSERT(false, "All traversed objects should be black or white");
3042 break;
3046 mLogger->End();
3047 mLogger = nullptr;
3048 timeLog.Checkpoint("ScanRoots::listener");
3052 ////////////////////////////////////////////////////////////////////////
3053 // Bacon & Rajan's |CollectWhite| routine, somewhat modified.
3054 ////////////////////////////////////////////////////////////////////////
3056 bool nsCycleCollector::CollectWhite() {
3057 // Explanation of "somewhat modified": we have no way to collect the
3058 // set of whites "all at once", we have to ask each of them to drop
3059 // their outgoing links and assume this will cause the garbage cycle
3060 // to *mostly* self-destruct (except for the reference we continue
3061 // to hold).
3063 // To do this "safely" we must make sure that the white nodes we're
3064 // operating on are stable for the duration of our operation. So we
3065 // make 3 sets of calls to language runtimes:
3067 // - Root(whites), which should pin the whites in memory.
3068 // - Unlink(whites), which drops outgoing links on each white.
3069 // - Unroot(whites), which returns the whites to normal GC.
3071 // Segments are 4 KiB on 32-bit and 8 KiB on 64-bit.
3072 static const size_t kSegmentSize = sizeof(void*) * 1024;
3073 SegmentedVector<PtrInfo*, kSegmentSize, InfallibleAllocPolicy> whiteNodes(
3074 kSegmentSize);
3075 TimeLog timeLog;
3077 MOZ_ASSERT(mIncrementalPhase == ScanAndCollectWhitePhase);
3079 uint32_t numWhiteNodes = 0;
3080 uint32_t numWhiteGCed = 0;
3081 uint32_t numWhiteJSZones = 0;
3084 JS::AutoAssertNoGC nogc;
3085 bool hasJSRuntime = !!mCCJSRuntime;
3086 nsCycleCollectionParticipant* zoneParticipant =
3087 hasJSRuntime ? mCCJSRuntime->ZoneParticipant() : nullptr;
3089 NodePool::Enumerator etor(mGraph.mNodes);
3090 while (!etor.IsDone()) {
3091 PtrInfo* pinfo = etor.GetNext();
3092 if (pinfo->mColor == white && pinfo->mParticipant) {
3093 if (pinfo->IsGrayJS()) {
3094 MOZ_ASSERT(mCCJSRuntime);
3095 ++numWhiteGCed;
3096 JS::Zone* zone;
3097 if (MOZ_UNLIKELY(pinfo->mParticipant == zoneParticipant)) {
3098 ++numWhiteJSZones;
3099 zone = static_cast<JS::Zone*>(pinfo->mPointer);
3100 } else {
3101 JS::GCCellPtr ptr(pinfo->mPointer,
3102 JS::GCThingTraceKind(pinfo->mPointer));
3103 zone = JS::GetTenuredGCThingZone(ptr);
3105 mCCJSRuntime->AddZoneWaitingForGC(zone);
3106 } else {
3107 whiteNodes.InfallibleAppend(pinfo);
3108 pinfo->mParticipant->Root(pinfo->mPointer);
3109 ++numWhiteNodes;
3115 mResults.mFreedRefCounted += numWhiteNodes;
3116 mResults.mFreedGCed += numWhiteGCed;
3117 mResults.mFreedJSZones += numWhiteJSZones;
3119 timeLog.Checkpoint("CollectWhite::Root");
3121 if (mBeforeUnlinkCB) {
3122 mBeforeUnlinkCB();
3123 timeLog.Checkpoint("CollectWhite::BeforeUnlinkCB");
3126 // Unlink() can trigger a GC, so do not touch any JS or anything
3127 // else not in whiteNodes after here.
3129 for (auto iter = whiteNodes.Iter(); !iter.Done(); iter.Next()) {
3130 PtrInfo* pinfo = iter.Get();
3131 MOZ_ASSERT(pinfo->mParticipant,
3132 "Unlink shouldn't see objects removed from graph.");
3133 pinfo->mParticipant->Unlink(pinfo->mPointer);
3134 #ifdef DEBUG
3135 if (mCCJSRuntime) {
3136 mCCJSRuntime->AssertNoObjectsToTrace(pinfo->mPointer);
3138 #endif
3140 timeLog.Checkpoint("CollectWhite::Unlink");
3142 JS::AutoAssertNoGC nogc;
3143 for (auto iter = whiteNodes.Iter(); !iter.Done(); iter.Next()) {
3144 PtrInfo* pinfo = iter.Get();
3145 MOZ_ASSERT(pinfo->mParticipant,
3146 "Unroot shouldn't see objects removed from graph.");
3147 pinfo->mParticipant->Unroot(pinfo->mPointer);
3149 timeLog.Checkpoint("CollectWhite::Unroot");
3151 nsCycleCollector_dispatchDeferredDeletion(false, true);
3152 timeLog.Checkpoint("CollectWhite::dispatchDeferredDeletion");
3154 mIncrementalPhase = CleanupPhase;
3156 return numWhiteNodes > 0 || numWhiteGCed > 0 || numWhiteJSZones > 0;
3159 ////////////////////////
3160 // Memory reporting
3161 ////////////////////////
3163 MOZ_DEFINE_MALLOC_SIZE_OF(CycleCollectorMallocSizeOf)
3165 NS_IMETHODIMP
3166 nsCycleCollector::CollectReports(nsIHandleReportCallback* aHandleReport,
3167 nsISupports* aData, bool aAnonymize) {
3168 size_t objectSize, graphSize, purpleBufferSize;
3169 SizeOfIncludingThis(CycleCollectorMallocSizeOf, &objectSize, &graphSize,
3170 &purpleBufferSize);
3172 if (objectSize > 0) {
3173 MOZ_COLLECT_REPORT("explicit/cycle-collector/collector-object", KIND_HEAP,
3174 UNITS_BYTES, objectSize,
3175 "Memory used for the cycle collector object itself.");
3178 if (graphSize > 0) {
3179 MOZ_COLLECT_REPORT(
3180 "explicit/cycle-collector/graph", KIND_HEAP, UNITS_BYTES, graphSize,
3181 "Memory used for the cycle collector's graph. This should be zero when "
3182 "the collector is idle.");
3185 if (purpleBufferSize > 0) {
3186 MOZ_COLLECT_REPORT("explicit/cycle-collector/purple-buffer", KIND_HEAP,
3187 UNITS_BYTES, purpleBufferSize,
3188 "Memory used for the cycle collector's purple buffer.");
3191 return NS_OK;
3194 ////////////////////////////////////////////////////////////////////////
3195 // Collector implementation
3196 ////////////////////////////////////////////////////////////////////////
3198 nsCycleCollector::nsCycleCollector()
3199 : mActivelyCollecting(false),
3200 mFreeingSnowWhite(false),
3201 mScanInProgress(false),
3202 mCCJSRuntime(nullptr),
3203 mIncrementalPhase(IdlePhase),
3204 #ifdef DEBUG
3205 mEventTarget(GetCurrentSerialEventTarget()),
3206 #endif
3207 mWhiteNodeCount(0),
3208 mBeforeUnlinkCB(nullptr),
3209 mForgetSkippableCB(nullptr),
3210 mUnmergedNeeded(0),
3211 mMergedInARow(0) {
3214 nsCycleCollector::~nsCycleCollector() {
3215 MOZ_ASSERT(!mJSPurpleBuffer, "Didn't call JSPurpleBuffer::Destroy?");
3217 UnregisterWeakMemoryReporter(this);
3220 void nsCycleCollector::SetCCJSRuntime(CycleCollectedJSRuntime* aCCRuntime) {
3221 MOZ_RELEASE_ASSERT(
3222 !mCCJSRuntime,
3223 "Multiple registrations of CycleCollectedJSRuntime in cycle collector");
3224 mCCJSRuntime = aCCRuntime;
3226 if (!NS_IsMainThread()) {
3227 return;
3230 // We can't register as a reporter in nsCycleCollector() because that runs
3231 // before the memory reporter manager is initialized. So we do it here
3232 // instead.
3233 RegisterWeakMemoryReporter(this);
3236 void nsCycleCollector::ClearCCJSRuntime() {
3237 MOZ_RELEASE_ASSERT(mCCJSRuntime,
3238 "Clearing CycleCollectedJSRuntime in cycle collector "
3239 "before a runtime was registered");
3240 mCCJSRuntime = nullptr;
3243 #ifdef DEBUG
3244 static bool HasParticipant(void* aPtr, nsCycleCollectionParticipant* aParti) {
3245 if (aParti) {
3246 return true;
3249 nsXPCOMCycleCollectionParticipant* xcp;
3250 ToParticipant(static_cast<nsISupports*>(aPtr), &xcp);
3251 return xcp != nullptr;
3253 #endif
3255 MOZ_ALWAYS_INLINE void nsCycleCollector::Suspect(
3256 void* aPtr, nsCycleCollectionParticipant* aParti,
3257 nsCycleCollectingAutoRefCnt* aRefCnt) {
3258 CheckThreadSafety();
3260 // Don't call AddRef or Release of a CCed object in a Traverse() method.
3261 MOZ_ASSERT(!mScanInProgress,
3262 "Attempted to call Suspect() while a scan was in progress");
3264 if (MOZ_UNLIKELY(mScanInProgress)) {
3265 return;
3268 MOZ_ASSERT(aPtr, "Don't suspect null pointers");
3270 MOZ_ASSERT(HasParticipant(aPtr, aParti),
3271 "Suspected nsISupports pointer must QI to "
3272 "nsXPCOMCycleCollectionParticipant");
3274 MOZ_ASSERT(aParti || CanonicalizeXPCOMParticipant(
3275 static_cast<nsISupports*>(aPtr)) == aPtr,
3276 "Suspect nsISupports pointer must be canonical");
3278 mPurpleBuf.Put(aPtr, aParti, aRefCnt);
3281 void nsCycleCollector::SuspectNurseryEntries() {
3282 MOZ_ASSERT(NS_IsMainThread(), "Wrong thread!");
3283 while (gNurseryPurpleBufferEntryCount) {
3284 NurseryPurpleBufferEntry& entry =
3285 gNurseryPurpleBufferEntry[--gNurseryPurpleBufferEntryCount];
3286 mPurpleBuf.Put(entry.mPtr, entry.mParticipant, entry.mRefCnt);
3290 void nsCycleCollector::CheckThreadSafety() {
3291 #ifdef DEBUG
3292 MOZ_ASSERT(mEventTarget->IsOnCurrentThread());
3293 #endif
3296 // The cycle collector uses the mark bitmap to discover what JS objects are
3297 // reachable only from XPConnect roots that might participate in cycles. We ask
3298 // the JS runtime whether we need to force a GC before this CC. It should only
3299 // be true when UnmarkGray has run out of stack. We also force GCs on shutdown
3300 // to collect cycles involving both DOM and JS, and in WantAllTraces CCs to
3301 // prevent hijinks from ForgetSkippable and compartmental GCs.
3302 void nsCycleCollector::FixGrayBits(bool aIsShutdown, TimeLog& aTimeLog) {
3303 CheckThreadSafety();
3305 if (!mCCJSRuntime) {
3306 return;
3309 // If we're not forcing a GC anyways due to shutdown or an all traces CC,
3310 // check to see if we still need to do one to fix the gray bits.
3311 if (!(aIsShutdown || (mLogger && mLogger->IsAllTraces()))) {
3312 mCCJSRuntime->FixWeakMappingGrayBits();
3313 aTimeLog.Checkpoint("FixWeakMappingGrayBits");
3315 bool needGC = !mCCJSRuntime->AreGCGrayBitsValid();
3316 // Only do a telemetry ping for non-shutdown CCs.
3317 CC_TELEMETRY(_NEED_GC, needGC);
3318 if (!needGC) {
3319 return;
3323 mResults.mForcedGC = true;
3325 uint32_t count = 0;
3326 do {
3327 if (aIsShutdown) {
3328 mCCJSRuntime->GarbageCollect(JS::GCOptions::Shutdown,
3329 JS::GCReason::SHUTDOWN_CC);
3330 } else {
3331 mCCJSRuntime->GarbageCollect(JS::GCOptions::Normal,
3332 JS::GCReason::CC_FORCED);
3335 mCCJSRuntime->FixWeakMappingGrayBits();
3337 // It's possible that FixWeakMappingGrayBits will hit OOM when unmarking
3338 // gray and we will have to go round again. The second time there should not
3339 // be any weak mappings to fix up so the loop body should run at most twice.
3340 MOZ_RELEASE_ASSERT(count < 2);
3341 count++;
3342 } while (!mCCJSRuntime->AreGCGrayBitsValid());
3344 aTimeLog.Checkpoint("FixGrayBits");
3347 bool nsCycleCollector::IsIncrementalGCInProgress() {
3348 return mCCJSRuntime && JS::IsIncrementalGCInProgress(mCCJSRuntime->Runtime());
3351 void nsCycleCollector::FinishAnyIncrementalGCInProgress() {
3352 if (IsIncrementalGCInProgress()) {
3353 NS_WARNING("Finishing incremental GC in progress during CC");
3354 JSContext* cx = CycleCollectedJSContext::Get()->Context();
3355 JS::PrepareForIncrementalGC(cx);
3356 JS::FinishIncrementalGC(cx, JS::GCReason::CC_FORCED);
3360 void nsCycleCollector::CleanupAfterCollection() {
3361 TimeLog timeLog;
3362 MOZ_ASSERT(mIncrementalPhase == CleanupPhase);
3363 MOZ_RELEASE_ASSERT(!mScanInProgress);
3364 mGraph.Clear();
3365 timeLog.Checkpoint("CleanupAfterCollection::mGraph.Clear()");
3367 uint32_t interval =
3368 (uint32_t)((TimeStamp::Now() - mCollectionStart).ToMilliseconds());
3369 #ifdef COLLECT_TIME_DEBUG
3370 printf("cc: total cycle collector time was %ums in %u slices\n", interval,
3371 mResults.mNumSlices);
3372 printf(
3373 "cc: visited %u ref counted and %u GCed objects, freed %d ref counted "
3374 "and %d GCed objects",
3375 mResults.mVisitedRefCounted, mResults.mVisitedGCed,
3376 mResults.mFreedRefCounted, mResults.mFreedGCed);
3377 uint32_t numVisited = mResults.mVisitedRefCounted + mResults.mVisitedGCed;
3378 if (numVisited > 1000) {
3379 uint32_t numFreed = mResults.mFreedRefCounted + mResults.mFreedGCed;
3380 printf(" (%d%%)", 100 * numFreed / numVisited);
3382 printf(".\ncc: \n");
3383 #endif
3385 CC_TELEMETRY(, interval);
3386 CC_TELEMETRY(_VISITED_REF_COUNTED, mResults.mVisitedRefCounted);
3387 CC_TELEMETRY(_VISITED_GCED, mResults.mVisitedGCed);
3388 CC_TELEMETRY(_COLLECTED, mWhiteNodeCount);
3389 timeLog.Checkpoint("CleanupAfterCollection::telemetry");
3391 if (mCCJSRuntime) {
3392 mCCJSRuntime->FinalizeDeferredThings(
3393 mResults.mAnyManual ? CycleCollectedJSRuntime::FinalizeNow
3394 : CycleCollectedJSRuntime::FinalizeIncrementally);
3395 mCCJSRuntime->EndCycleCollectionCallback(mResults);
3396 timeLog.Checkpoint("CleanupAfterCollection::EndCycleCollectionCallback()");
3398 mIncrementalPhase = IdlePhase;
3401 void nsCycleCollector::ShutdownCollect() {
3402 FinishAnyIncrementalGCInProgress();
3403 CycleCollectedJSContext* ccJSContext = CycleCollectedJSContext::Get();
3404 JS::ShutdownAsyncTasks(ccJSContext->Context());
3406 SliceBudget unlimitedBudget = SliceBudget::unlimited();
3407 uint32_t i;
3408 bool collectedAny = true;
3409 for (i = 0; i < DEFAULT_SHUTDOWN_COLLECTIONS && collectedAny; ++i) {
3410 collectedAny = Collect(CCReason::SHUTDOWN, ccIsManual::CCIsManual,
3411 unlimitedBudget, nullptr);
3412 // Run any remaining tasks that may have been enqueued via RunInStableState
3413 // or DispatchToMicroTask. These can hold alive CCed objects, and we want to
3414 // clear them out before we run the CC again or finish shutting down.
3415 ccJSContext->PerformMicroTaskCheckPoint(true);
3416 ccJSContext->ProcessStableStateQueue();
3419 // This warning would happen very frequently, so don't do it unless we're
3420 // logging this CC, so we might care about how many CCs there are.
3421 NS_WARNING_ASSERTION(
3422 !mParams.LogThisCC(mShutdownCount) || i < NORMAL_SHUTDOWN_COLLECTIONS,
3423 "Extra shutdown CC");
3426 static void PrintPhase(const char* aPhase) {
3427 #ifdef DEBUG_PHASES
3428 printf("cc: begin %s on %s\n", aPhase,
3429 NS_IsMainThread() ? "mainthread" : "worker");
3430 #endif
3433 bool nsCycleCollector::Collect(CCReason aReason, ccIsManual aIsManual,
3434 SliceBudget& aBudget,
3435 nsICycleCollectorListener* aManualListener,
3436 bool aPreferShorterSlices) {
3437 AUTO_PROFILER_LABEL_RELEVANT_FOR_JS("Incremental CC", GCCC);
3439 CheckThreadSafety();
3441 // This can legitimately happen in a few cases. See bug 383651.
3442 if (mActivelyCollecting || mFreeingSnowWhite) {
3443 return false;
3445 mActivelyCollecting = true;
3447 MOZ_ASSERT(!IsIncrementalGCInProgress());
3449 bool startedIdle = IsIdle();
3450 bool collectedAny = false;
3452 // If the CC started idle, it will call BeginCollection, which
3453 // will do FreeSnowWhite, so it doesn't need to be done here.
3454 if (!startedIdle) {
3455 TimeLog timeLog;
3456 FreeSnowWhite(true);
3457 timeLog.Checkpoint("Collect::FreeSnowWhite");
3460 if (aIsManual == ccIsManual::CCIsManual) {
3461 mResults.mAnyManual = true;
3464 ++mResults.mNumSlices;
3466 bool continueSlice = aBudget.isUnlimited() || !aPreferShorterSlices;
3467 do {
3468 switch (mIncrementalPhase) {
3469 case IdlePhase:
3470 PrintPhase("BeginCollection");
3471 BeginCollection(aReason, aIsManual, aManualListener);
3472 break;
3473 case GraphBuildingPhase:
3474 PrintPhase("MarkRoots");
3475 MarkRoots(aBudget);
3477 // Only continue this slice if we're running synchronously or the
3478 // next phase will probably be short, to reduce the max pause for this
3479 // collection.
3480 // (There's no need to check if we've finished graph building, because
3481 // if we haven't, we've already exceeded our budget, and will finish
3482 // this slice anyways.)
3483 continueSlice = aBudget.isUnlimited() ||
3484 (mResults.mNumSlices < 3 && !aPreferShorterSlices);
3485 break;
3486 case ScanAndCollectWhitePhase:
3487 // We do ScanRoots and CollectWhite in a single slice to ensure
3488 // that we won't unlink a live object if a weak reference is
3489 // promoted to a strong reference after ScanRoots has finished.
3490 // See bug 926533.
3492 AUTO_PROFILER_LABEL_CATEGORY_PAIR(GCCC_ScanRoots);
3493 PrintPhase("ScanRoots");
3494 ScanRoots(startedIdle);
3497 AUTO_PROFILER_LABEL_CATEGORY_PAIR(GCCC_CollectWhite);
3498 PrintPhase("CollectWhite");
3499 collectedAny = CollectWhite();
3501 break;
3502 case CleanupPhase:
3503 PrintPhase("CleanupAfterCollection");
3504 CleanupAfterCollection();
3505 continueSlice = false;
3506 break;
3508 if (continueSlice) {
3509 aBudget.stepAndForceCheck();
3510 continueSlice = !aBudget.isOverBudget();
3512 } while (continueSlice);
3514 // Clear mActivelyCollecting here to ensure that a recursive call to
3515 // Collect() does something.
3516 mActivelyCollecting = false;
3518 if (aIsManual && !startedIdle) {
3519 // We were in the middle of an incremental CC (using its own listener).
3520 // Somebody has forced a CC, so after having finished out the current CC,
3521 // run the CC again using the new listener.
3522 MOZ_ASSERT(IsIdle());
3523 if (Collect(aReason, ccIsManual::CCIsManual, aBudget, aManualListener)) {
3524 collectedAny = true;
3528 MOZ_ASSERT_IF(aIsManual == CCIsManual, IsIdle());
3530 return collectedAny;
3533 // Any JS objects we have in the graph could die when we GC, but we
3534 // don't want to abandon the current CC, because the graph contains
3535 // information about purple roots. So we synchronously finish off
3536 // the current CC.
3537 void nsCycleCollector::PrepareForGarbageCollection() {
3538 if (IsIdle()) {
3539 MOZ_ASSERT(mGraph.IsEmpty(), "Non-empty graph when idle");
3540 MOZ_ASSERT(!mBuilder, "Non-null builder when idle");
3541 if (mJSPurpleBuffer) {
3542 mJSPurpleBuffer->Destroy();
3544 return;
3547 FinishAnyCurrentCollection(CCReason::GC_WAITING);
3550 void nsCycleCollector::FinishAnyCurrentCollection(CCReason aReason) {
3551 if (IsIdle()) {
3552 return;
3555 SliceBudget unlimitedBudget = SliceBudget::unlimited();
3556 PrintPhase("FinishAnyCurrentCollection");
3557 // Use CCIsNotManual because we only want to finish the CC in progress.
3558 Collect(aReason, ccIsManual::CCIsNotManual, unlimitedBudget, nullptr);
3560 // It is only okay for Collect() to have failed to finish the
3561 // current CC if we're reentering the CC at some point past
3562 // graph building. We need to be past the point where the CC will
3563 // look at JS objects so that it is safe to GC.
3564 MOZ_ASSERT(IsIdle() || (mActivelyCollecting &&
3565 mIncrementalPhase != GraphBuildingPhase),
3566 "Reentered CC during graph building");
3569 // Don't merge too many times in a row, and do at least a minimum
3570 // number of unmerged CCs in a row.
3571 static const uint32_t kMinConsecutiveUnmerged = 3;
3572 static const uint32_t kMaxConsecutiveMerged = 3;
3574 bool nsCycleCollector::ShouldMergeZones(ccIsManual aIsManual) {
3575 if (!mCCJSRuntime) {
3576 return false;
3579 MOZ_ASSERT(mUnmergedNeeded <= kMinConsecutiveUnmerged);
3580 MOZ_ASSERT(mMergedInARow <= kMaxConsecutiveMerged);
3582 if (mMergedInARow == kMaxConsecutiveMerged) {
3583 MOZ_ASSERT(mUnmergedNeeded == 0);
3584 mUnmergedNeeded = kMinConsecutiveUnmerged;
3587 if (mUnmergedNeeded > 0) {
3588 mUnmergedNeeded--;
3589 mMergedInARow = 0;
3590 return false;
3593 if (aIsManual == CCIsNotManual && mCCJSRuntime->UsefulToMergeZones()) {
3594 mMergedInARow++;
3595 return true;
3596 } else {
3597 mMergedInARow = 0;
3598 return false;
3602 void nsCycleCollector::BeginCollection(
3603 CCReason aReason, ccIsManual aIsManual,
3604 nsICycleCollectorListener* aManualListener) {
3605 TimeLog timeLog;
3606 MOZ_ASSERT(IsIdle());
3607 MOZ_RELEASE_ASSERT(!mScanInProgress);
3609 mCollectionStart = TimeStamp::Now();
3611 if (mCCJSRuntime) {
3612 mCCJSRuntime->BeginCycleCollectionCallback(aReason);
3613 timeLog.Checkpoint("BeginCycleCollectionCallback()");
3616 bool isShutdown = (aReason == CCReason::SHUTDOWN);
3617 if (isShutdown) {
3618 mShutdownCount += 1;
3621 // Set up the listener for this CC.
3622 MOZ_ASSERT_IF(isShutdown, !aManualListener);
3623 MOZ_ASSERT(!mLogger, "Forgot to clear a previous listener?");
3625 if (aManualListener) {
3626 aManualListener->AsLogger(getter_AddRefs(mLogger));
3629 aManualListener = nullptr;
3630 if (!mLogger && mParams.LogThisCC(mShutdownCount)) {
3631 mLogger = new nsCycleCollectorLogger();
3632 if (mParams.AllTracesThisCC(isShutdown)) {
3633 mLogger->SetAllTraces();
3637 // BeginCycleCollectionCallback() might have started an IGC, and we need
3638 // to finish it before we run FixGrayBits.
3639 FinishAnyIncrementalGCInProgress();
3640 timeLog.Checkpoint("Pre-FixGrayBits finish IGC");
3642 FixGrayBits(isShutdown, timeLog);
3643 if (mCCJSRuntime) {
3644 mCCJSRuntime->CheckGrayBits();
3647 FreeSnowWhite(true);
3648 timeLog.Checkpoint("BeginCollection FreeSnowWhite");
3650 if (mLogger && NS_FAILED(mLogger->Begin())) {
3651 mLogger = nullptr;
3654 // FreeSnowWhite could potentially have started an IGC, which we need
3655 // to finish before we look at any JS roots.
3656 FinishAnyIncrementalGCInProgress();
3657 timeLog.Checkpoint("Post-FreeSnowWhite finish IGC");
3659 // Set up the data structures for building the graph.
3660 JS::AutoAssertNoGC nogc;
3661 JS::AutoEnterCycleCollection autocc(mCCJSRuntime->Runtime());
3662 mGraph.Init();
3663 mResults.Init();
3664 mResults.mSuspectedAtCCStart = SuspectedCount();
3665 mResults.mAnyManual = aIsManual;
3666 bool mergeZones = ShouldMergeZones(aIsManual);
3667 mResults.mMergedZones = mergeZones;
3669 MOZ_ASSERT(!mBuilder, "Forgot to clear mBuilder");
3670 mBuilder = MakeUnique<CCGraphBuilder>(mGraph, mResults, mCCJSRuntime, mLogger,
3671 mergeZones);
3672 timeLog.Checkpoint("BeginCollection prepare graph builder");
3674 if (mCCJSRuntime) {
3675 mCCJSRuntime->TraverseRoots(*mBuilder);
3676 timeLog.Checkpoint("mJSContext->TraverseRoots()");
3679 AutoRestore<bool> ar(mScanInProgress);
3680 MOZ_RELEASE_ASSERT(!mScanInProgress);
3681 mScanInProgress = true;
3682 mPurpleBuf.SelectPointers(*mBuilder);
3683 timeLog.Checkpoint("SelectPointers()");
3685 mBuilder->DoneAddingRoots();
3686 mIncrementalPhase = GraphBuildingPhase;
3689 uint32_t nsCycleCollector::SuspectedCount() {
3690 CheckThreadSafety();
3691 if (NS_IsMainThread()) {
3692 return gNurseryPurpleBufferEntryCount + mPurpleBuf.Count();
3695 return mPurpleBuf.Count();
3698 void nsCycleCollector::Shutdown(bool aDoCollect) {
3699 CheckThreadSafety();
3701 if (NS_IsMainThread()) {
3702 gNurseryPurpleBufferEnabled = false;
3705 // Always delete snow white objects.
3706 FreeSnowWhite(true);
3708 if (aDoCollect) {
3709 ShutdownCollect();
3712 if (mJSPurpleBuffer) {
3713 mJSPurpleBuffer->Destroy();
3717 void nsCycleCollector::RemoveObjectFromGraph(void* aObj) {
3718 if (IsIdle()) {
3719 return;
3722 mGraph.RemoveObjectFromMap(aObj);
3723 if (mBuilder) {
3724 mBuilder->RemoveCachedEntry(aObj);
3728 void nsCycleCollector::SizeOfIncludingThis(mozilla::MallocSizeOf aMallocSizeOf,
3729 size_t* aObjectSize,
3730 size_t* aGraphSize,
3731 size_t* aPurpleBufferSize) const {
3732 *aObjectSize = aMallocSizeOf(this);
3734 *aGraphSize = mGraph.SizeOfExcludingThis(aMallocSizeOf);
3736 *aPurpleBufferSize = mPurpleBuf.SizeOfExcludingThis(aMallocSizeOf);
3738 // These fields are deliberately not measured:
3739 // - mCCJSRuntime: because it's non-owning and measured by JS reporters.
3740 // - mParams: because it only contains scalars.
3743 JSPurpleBuffer* nsCycleCollector::GetJSPurpleBuffer() {
3744 if (!mJSPurpleBuffer) {
3745 // The Release call here confuses the GC analysis.
3746 JS::AutoSuppressGCAnalysis nogc;
3747 // JSPurpleBuffer keeps itself alive, but we need to create it in such way
3748 // that it ends up in the normal purple buffer. That happens when
3749 // nsRefPtr goes out of the scope and calls Release.
3750 RefPtr<JSPurpleBuffer> pb = new JSPurpleBuffer(mJSPurpleBuffer);
3752 return mJSPurpleBuffer;
3755 ////////////////////////////////////////////////////////////////////////
3756 // Module public API (exported in nsCycleCollector.h)
3757 // Just functions that redirect into the singleton, once it's built.
3758 ////////////////////////////////////////////////////////////////////////
3760 void nsCycleCollector_registerJSContext(CycleCollectedJSContext* aCx) {
3761 CollectorData* data = sCollectorData.get();
3763 // We should have started the cycle collector by now.
3764 MOZ_ASSERT(data);
3765 MOZ_ASSERT(data->mCollector);
3766 // But we shouldn't already have a context.
3767 MOZ_ASSERT(!data->mContext);
3769 data->mContext = aCx;
3770 data->mCollector->SetCCJSRuntime(aCx->Runtime());
3773 void nsCycleCollector_forgetJSContext() {
3774 CollectorData* data = sCollectorData.get();
3776 // We should have started the cycle collector by now.
3777 MOZ_ASSERT(data);
3778 // And we shouldn't have already forgotten our context.
3779 MOZ_ASSERT(data->mContext);
3781 // But it may have shutdown already.
3782 if (data->mCollector) {
3783 data->mCollector->ClearCCJSRuntime();
3784 data->mContext = nullptr;
3785 } else {
3786 data->mContext = nullptr;
3787 delete data;
3788 sCollectorData.set(nullptr);
3792 /* static */
3793 CycleCollectedJSContext* CycleCollectedJSContext::Get() {
3794 CollectorData* data = sCollectorData.get();
3795 if (data) {
3796 return data->mContext;
3798 return nullptr;
3801 MOZ_NEVER_INLINE static void SuspectAfterShutdown(
3802 void* aPtr, nsCycleCollectionParticipant* aCp,
3803 nsCycleCollectingAutoRefCnt* aRefCnt, bool* aShouldDelete) {
3804 if (aRefCnt->get() == 0) {
3805 if (!aShouldDelete) {
3806 // The CC is shut down, so we can't be in the middle of an ICC.
3807 ToParticipant(aPtr, &aCp);
3808 aRefCnt->stabilizeForDeletion();
3809 aCp->DeleteCycleCollectable(aPtr);
3810 } else {
3811 *aShouldDelete = true;
3813 } else {
3814 // Make sure we'll get called again.
3815 aRefCnt->RemoveFromPurpleBuffer();
3819 void NS_CycleCollectorSuspect3(void* aPtr, nsCycleCollectionParticipant* aCp,
3820 nsCycleCollectingAutoRefCnt* aRefCnt,
3821 bool* aShouldDelete) {
3822 if ((
3823 #ifdef HAVE_64BIT_BUILD
3824 aRefCnt->IsOnMainThread() ||
3825 #endif
3826 NS_IsMainThread()) &&
3827 gNurseryPurpleBufferEnabled) {
3828 // The next time the object is passed to the purple buffer, we can do faster
3829 // IsOnMainThread() check.
3830 aRefCnt->SetIsOnMainThread();
3831 SuspectUsingNurseryPurpleBuffer(aPtr, aCp, aRefCnt);
3832 return;
3835 CollectorData* data = sCollectorData.get();
3837 // This assertion will happen if you AddRef or Release a cycle collected
3838 // object on a thread that does not have an active cycle collector.
3839 // This can happen in a few situations:
3840 // 1. We never cycle collect on this thread. (The cycle collector is only
3841 // run on the main thread and DOM worker threads.)
3842 // 2. The cycle collector hasn't been initialized on this thread yet.
3843 // 3. The cycle collector has already been shut down on this thread.
3844 MOZ_DIAGNOSTIC_ASSERT(
3845 data,
3846 "Cycle collected object used on a thread without a cycle collector.");
3848 if (MOZ_LIKELY(data->mCollector)) {
3849 data->mCollector->Suspect(aPtr, aCp, aRefCnt);
3850 return;
3852 SuspectAfterShutdown(aPtr, aCp, aRefCnt, aShouldDelete);
3855 void ClearNurseryPurpleBuffer() {
3856 MOZ_ASSERT(NS_IsMainThread(), "Wrong thread!");
3857 CollectorData* data = sCollectorData.get();
3858 MOZ_ASSERT(data);
3859 MOZ_ASSERT(data->mCollector);
3860 data->mCollector->SuspectNurseryEntries();
3863 uint32_t nsCycleCollector_suspectedCount() {
3864 CollectorData* data = sCollectorData.get();
3866 // We should have started the cycle collector by now.
3867 MOZ_ASSERT(data);
3869 if (!data->mCollector) {
3870 return 0;
3873 return data->mCollector->SuspectedCount();
3876 bool nsCycleCollector_init() {
3877 #ifdef DEBUG
3878 static bool sInitialized;
3880 MOZ_ASSERT(NS_IsMainThread(), "Wrong thread!");
3881 MOZ_ASSERT(!sInitialized, "Called twice!?");
3882 sInitialized = true;
3883 #endif
3885 return sCollectorData.init();
3888 void nsCycleCollector_startup() {
3889 if (sCollectorData.get()) {
3890 MOZ_CRASH();
3893 CollectorData* data = new CollectorData;
3894 data->mCollector = new nsCycleCollector();
3895 data->mContext = nullptr;
3897 sCollectorData.set(data);
3900 void nsCycleCollector_setBeforeUnlinkCallback(CC_BeforeUnlinkCallback aCB) {
3901 CollectorData* data = sCollectorData.get();
3903 // We should have started the cycle collector by now.
3904 MOZ_ASSERT(data);
3905 MOZ_ASSERT(data->mCollector);
3907 data->mCollector->SetBeforeUnlinkCallback(aCB);
3910 void nsCycleCollector_setForgetSkippableCallback(
3911 CC_ForgetSkippableCallback aCB) {
3912 CollectorData* data = sCollectorData.get();
3914 // We should have started the cycle collector by now.
3915 MOZ_ASSERT(data);
3916 MOZ_ASSERT(data->mCollector);
3918 data->mCollector->SetForgetSkippableCallback(aCB);
3921 void nsCycleCollector_forgetSkippable(js::SliceBudget& aBudget,
3922 bool aRemoveChildlessNodes,
3923 bool aAsyncSnowWhiteFreeing) {
3924 CollectorData* data = sCollectorData.get();
3926 // We should have started the cycle collector by now.
3927 MOZ_ASSERT(data);
3928 MOZ_ASSERT(data->mCollector);
3930 TimeLog timeLog;
3931 data->mCollector->ForgetSkippable(aBudget, aRemoveChildlessNodes,
3932 aAsyncSnowWhiteFreeing);
3933 timeLog.Checkpoint("ForgetSkippable()");
3936 void nsCycleCollector_dispatchDeferredDeletion(bool aContinuation,
3937 bool aPurge) {
3938 CycleCollectedJSRuntime* rt = CycleCollectedJSRuntime::Get();
3939 if (rt) {
3940 rt->DispatchDeferredDeletion(aContinuation, aPurge);
3944 bool nsCycleCollector_doDeferredDeletion() {
3945 CollectorData* data = sCollectorData.get();
3947 // We should have started the cycle collector by now.
3948 MOZ_ASSERT(data);
3949 MOZ_ASSERT(data->mCollector);
3950 MOZ_ASSERT(data->mContext);
3952 return data->mCollector->FreeSnowWhite(false);
3955 bool nsCycleCollector_doDeferredDeletionWithBudget(js::SliceBudget& aBudget) {
3956 CollectorData* data = sCollectorData.get();
3958 // We should have started the cycle collector by now.
3959 MOZ_ASSERT(data);
3960 MOZ_ASSERT(data->mCollector);
3961 MOZ_ASSERT(data->mContext);
3963 return data->mCollector->FreeSnowWhiteWithBudget(aBudget);
3966 already_AddRefed<nsICycleCollectorLogSink> nsCycleCollector_createLogSink() {
3967 nsCOMPtr<nsICycleCollectorLogSink> sink = new nsCycleCollectorLogSinkToFile();
3968 return sink.forget();
3971 bool nsCycleCollector_collect(CCReason aReason,
3972 nsICycleCollectorListener* aManualListener) {
3973 CollectorData* data = sCollectorData.get();
3975 // We should have started the cycle collector by now.
3976 MOZ_ASSERT(data);
3977 MOZ_ASSERT(data->mCollector);
3979 AUTO_PROFILER_LABEL("nsCycleCollector_collect", GCCC);
3981 SliceBudget unlimitedBudget = SliceBudget::unlimited();
3982 return data->mCollector->Collect(aReason, ccIsManual::CCIsManual,
3983 unlimitedBudget, aManualListener);
3986 void nsCycleCollector_collectSlice(SliceBudget& budget, CCReason aReason,
3987 bool aPreferShorterSlices) {
3988 CollectorData* data = sCollectorData.get();
3990 // We should have started the cycle collector by now.
3991 MOZ_ASSERT(data);
3992 MOZ_ASSERT(data->mCollector);
3994 AUTO_PROFILER_LABEL("nsCycleCollector_collectSlice", GCCC);
3996 data->mCollector->Collect(aReason, ccIsManual::CCIsNotManual, budget, nullptr,
3997 aPreferShorterSlices);
4000 void nsCycleCollector_prepareForGarbageCollection() {
4001 CollectorData* data = sCollectorData.get();
4003 MOZ_ASSERT(data);
4005 if (!data->mCollector) {
4006 return;
4009 data->mCollector->PrepareForGarbageCollection();
4012 void nsCycleCollector_finishAnyCurrentCollection() {
4013 CollectorData* data = sCollectorData.get();
4015 MOZ_ASSERT(data);
4017 if (!data->mCollector) {
4018 return;
4021 data->mCollector->FinishAnyCurrentCollection(CCReason::API);
4024 void nsCycleCollector_shutdown(bool aDoCollect) {
4025 CollectorData* data = sCollectorData.get();
4027 if (data) {
4028 MOZ_ASSERT(data->mCollector);
4029 AUTO_PROFILER_LABEL("nsCycleCollector_shutdown", OTHER);
4032 RefPtr<nsCycleCollector> collector = data->mCollector;
4033 collector->Shutdown(aDoCollect);
4034 data->mCollector = nullptr;
4037 if (!data->mContext) {
4038 delete data;
4039 sCollectorData.set(nullptr);