2 +----------------------------------------------------------------------+
4 +----------------------------------------------------------------------+
5 | Copyright (c) 2010-present Facebook, Inc. (http://www.facebook.com) |
6 +----------------------------------------------------------------------+
7 | This source file is subject to version 3.01 of the PHP license, |
8 | that is bundled with this package in the file LICENSE, and is |
9 | available through the world-wide-web at the following url: |
10 | http://www.php.net/license/3_01.txt |
11 | If you did not receive a copy of the PHP license and are unable to |
12 | obtain it through the world-wide-web, please send a note to |
13 | license@php.net so we can mail you a copy immediately. |
14 +----------------------------------------------------------------------+
16 #ifndef incl_HPHP_RUNTIME_RDS_H_
17 #define incl_HPHP_RUNTIME_RDS_H_
23 #include <unordered_map>
24 #include <boost/variant.hpp>
25 #include <folly/Range.h>
26 #include <folly/Optional.h>
28 #include "hphp/runtime/base/types.h"
30 #include "hphp/util/type-scan.h"
39 struct ArrayKindProfile
;
40 struct ArrayOffsetProfile
;
44 struct RefcountProfile
;
47 struct ReleaseVVProfile
;
52 //////////////////////////////////////////////////////////////////////
54 namespace HPHP
{ namespace rds
{
56 //////////////////////////////////////////////////////////////////////
59 * The RDS (Request Data Segment) is a region of memory quickly accessible to
60 * each hhvm thread that is running a PHP request.
62 * Essentially this is a per-thread memory region, along with an internal
63 * dynamic link table to give the segment the same layout for each thread as
64 * new data is allocated.
66 * The RDS starts with a small header that is statically layed out, followed by
67 * the main "normal" segment, which is (logically) reset at the beginning of
68 * every request. The next section, called "local", contains unshared but still
69 * persistent data---this is data that is local to a thread but retains its
70 * value across requests. The final section contains shared "persistent" data,
71 * which is data that retains the same values across requests.
73 * The shared persistent segment is RDS Layout:
74 * implemented by mapping the same physical
75 * pages to different virtual addresses, so +-------------+ <-- tl_base
76 * are all accessible from the | Header |
77 * per-thread RDS base. The normal +-------------+
78 * region is perhaps analogous to .bss, | |
79 * while the persistent region is | Normal |
80 * analogous to .rodata, and the local region | region |
81 * is similar to .data. | | growing higher
83 * When we're running in C++, the base of RDS | \ \ \ \ \ \ |
84 * is available via a thread local exported +-------------+ ^^^
85 * from this module (tl_base). When running | Local | growing lower
86 * in JIT-compiled code, a machine register | region |
87 * is reserved to always point at the base of +-------------+
90 * +-------------+ addresses
92 * Every element in the "normal" segment has an associated generation number,
93 * and the segment as a whole has a "current" generation number. A particular
94 * element is considered initialized if its generation number matches the
95 * segment's current generation number. If it does not, the element should be
96 * considered to contain garbage and should be initialized as needed by the
97 * element type. Once done, it must be manually marked as initialized.
99 * Allocation/linking API:
101 * You can allocate data from RDS in two primary ways, either by
102 * binding a Link, or anonymously. The distinction is whether the
103 * allocated space is associated with some unique key that allows it
104 * to be re-found for any new attempts to allocate that symbol.
106 * Anonymous allocations are created with rds::alloc. Non-anonymous
107 * allocations can be created in two ways:
109 * rds::bind(Symbol) uses an RDS-internal link table to find if
110 * there is an existing handle for the given symbol.
112 * rds::Link<T>::bind allows the caller to make use of the
113 * uniqueness of other runtime structure (e.g. the Class
114 * structure) to avoid having a special key and needing to do
115 * lookups in the internal RDS link table. The "key" for the
116 * allocation is the rds::Link<> object itself.
118 * Finally, you can allocate anonymous single bits at a time with
123 //////////////////////////////////////////////////////////////////////
126 * Lifetime-related hooks, exported to be called at the appropriate
127 * times. If shouldRegister is false the resultant rds will be excluded from
128 * the global rds list.
132 void threadInit(bool shouldRegister
= true);
133 void threadExit(bool shouldUnregister
= true);
136 * Flushing RDS means to madvise the memory away. Should only be done
137 * while a request is not in flight on this thread.
139 * This is done to conserve memory if a particular thread is unlikely
140 * to need to serve another PHP request for a while.
145 * Return the number of bytes that have been allocated from either
146 * persistent, local, or non-persistent RDS.
149 size_t usedLocalBytes();
150 size_t usedPersistentBytes();
152 folly::Range
<const char*> normalSection();
153 folly::Range
<const char*> localSection();
154 folly::Range
<const char*> persistentSection();
156 // Invoke F on each initialized allocation in the normal section. F is invoked
157 // with a void* pointer to the data, the size of the data, and the stored
159 template <typename F
> void forEachNormalAlloc(F
);
160 template <typename F
> void forEachLocalAlloc(F
);
163 * The thread-local pointer to the base of RDS.
165 extern __thread
void* tl_base
;
167 //////////////////////////////////////////////////////////////////////
170 * RDS symbols are centrally registered here.
172 * All StringData*'s below must be static strings.
176 * Symbol for function static locals. These are RefData's allocated
179 struct StaticLocal
{ FuncId funcId
;
180 LowStringPtr name
; };
183 * Class constant values are TypedValue's stored in RDS.
185 struct ClsConstant
{ LowStringPtr clsName
;
186 LowStringPtr cnsName
; };
189 * StaticMethod{F,}Cache allocations. These are used to cache static
190 * method dispatch targets in a given class context. The `name' field
191 * here is a string that encodes the target class, property, and
194 struct StaticMethod
{ LowStringPtr name
; };
195 struct StaticMethodF
{ LowStringPtr name
; };
198 * Profiling translations may store various kinds of junk under
199 * symbols that are keyed on translation id. These generally should
200 * go in Mode::Local or Mode::Persistent, depending on the use case.
203 struct Profile
{ TransID transId
;
205 LowStringPtr name
; };
208 * Static class properties in Mode::Local
211 struct SPropCache
{ LowPtr
<const Class
> cls
;
214 using Symbol
= boost::variant
< StaticLocal
218 , Profile
<jit::ArrayKindProfile
>
219 , Profile
<jit::ArrayOffsetProfile
>
220 , Profile
<jit::ClsCnsProfile
>
221 , Profile
<jit::DecRefProfile
>
222 , Profile
<jit::MethProfile
>
223 , Profile
<jit::RefcountProfile
>
224 , Profile
<jit::ReleaseVVProfile
>
225 , Profile
<jit::SwitchProfile
>
226 , Profile
<jit::TypeProfile
>
230 //////////////////////////////////////////////////////////////////////
232 enum class Mode
{ Normal
, Local
, Persistent
};
235 * Handles into Request Data Segment. These are offsets from rds::tl_base.
237 using Handle
= uint32_t;
238 constexpr Handle kUninitHandle
= 0;
239 constexpr Handle kInvalidHandleMask
= 0x80000000;
240 constexpr Handle kBeingBound
= 0xffffffff;
241 constexpr Handle kBeingBoundWithWaiters
= 0xfffffffe;
244 * Normal segment element generation numbers.
246 * The segment's current generation number will never be kInvalidGenNumber;
247 * thus, it is safe to use to mark an element as being uninitialized
250 using GenNumber
= uint8_t;
251 constexpr GenNumber kInvalidGenNumber
= 0;
254 * Tag passed to RDS handle initialization routines to indicate that the handle
255 * is known to be normal.
257 enum class NormalTag
{};
260 * rds::Link<T> is a thin, typed wrapper around an rds::Handle.
262 * Note that nothing prevents using non-POD types with this. But nothing here
263 * is going to run the constructor. (In the non-persistent region, the space
264 * for T will contain garbage at the start of each request.)
266 * Links are atomic types. All apis may be called concurrently by
267 * multiple threads, and the alloc() api guarantees only a single
268 * caller will actually allocate new space in RDS.
270 template<class T
, bool normal_only
= false>
272 explicit Link(Handle handle
);
276 Link
& operator=(const Link
& r
);
279 * Ensure this Link is bound to an RDS allocation. If it is not, allocate it
280 * using this Link itself as the symbol.
282 * This function internally synchronizes to avoid double-allocating. It is
283 * legal to call it repeatedly with a link that may already be bound. The
284 * `mode' parameter and `Align' parameters are ignored if the link is already
285 * bound, and only affects the call that allocates RDS memory.
289 template<size_t Align
= alignof(T
)> void bind(Mode mode
= Mode::Normal
);
292 * Ensure this Link is bound to an RDS allocation.
293 * - if its already bound, do nothing;
294 * - if another thread is already calling fun to bind it, wait until its
296 * - otherwise call fun to obtain a handle.
298 * The intent is to ensure that only one thread allocates the
299 * handle, and that in the case of persistent handles, fun has a
300 * chance to fill in the value before the Link is published (so that
301 * other threads only ever see an unbound handle, or a bound handle
302 * with a valid value.
306 template<typename F
> void bind(F fun
);
309 * Dereference a Link and access its RDS memory for the current thread.
313 T
& operator*() const;
314 T
* operator->() const;
318 * Whether this Link is bound to RDS memory or not (i.e., whether its
319 * internal handle is valid).
324 * Access to the underlying rds::Handle.
326 Handle
handle() const;
329 * Access to the underlying rds::Handle; returns kUninitHandle if
332 Handle
maybeHandle() const;
335 * Return the generation number of this element.
337 * Pre: bound() && isNormal()
339 GenNumber
genNumber() const;
342 * Return the handle for this element's generation number.
344 * Pre: bound() && isNormal()
346 Handle
genNumberHandle() const;
349 * Check whether this element is initialized.
351 * This is only an interesting designation for normal handles, where we need
352 * to check the generation number. Persistent handles are expected not to be
353 * "published" until they are in some sort of initialized state (which might
354 * simply be some nullish sentinel value).
359 bool isInit(NormalTag
) const;
362 * Manually mark this element as initialized or uninitialized.
364 * Pre: bound() && isNormal()
366 void markInit() const;
367 void markUninit() const;
370 * Initialize this element to `val'.
372 * Anything previously stored in the element is considered to be garbage, so
373 * it is not destructed. initWith() can thus be used to unconditionally
374 * initialize something that might already be inited, but only if it's
375 * trivially destructible.
379 void initWith(const T
& val
) const;
380 void initWith(T
&& val
) const;
383 * Check which segment this element resides in.
387 bool isNormal() const;
388 bool isLocal() const;
389 bool isPersistent() const;
392 * For access from the JIT only.
394 static constexpr size_t handleOff() {
395 return offsetof(Link
, m_handle
);
399 Handle
raw() const { return m_handle
.load(std::memory_order_relaxed
); }
400 std::atomic
<Handle
> m_handle
;
404 * Return a bound link to memory from RDS, using the given Symbol.
406 * Mode indicates whether the memory should be placed in the persistent region
407 * or not, Align indicates the alignment requirements, and extraSize allows for
408 * allocating additional space beyond sizeof(T), for variable-length structures
409 * (not allowed for normal mode). All three arguments are ignored if there is
410 * already an allocation for the Symbol---they only affect the first caller for
413 * N indicates that the binding for `key' will always be in the "normal" RDS
414 * region; it is allowed to be true only if `key' is only ever bound with
417 template<class T
, bool N
= false, size_t Align
= alignof(T
)>
418 Link
<T
,N
> bind(Symbol key
, Mode mode
= Mode::Normal
, size_t extraSize
= 0);
421 * Remove a bound link from RDS metadata. The actual space in RDS is
424 void unbind(Symbol
, Handle
);
427 * Try to bind to a symbol in RDS, returning an unbound link if the
428 * Symbol isn't already allocated in RDS. Mode and alignment are not
429 * specified---if the symbol is already bound, these are already
430 * fixed. The `T' must be the same `T' that the symbol is mapped to,
431 * if it's already mapped.
434 Link
<T
> attach(Symbol key
);
437 * Allocate anonymous memory from RDS.
439 * The memory is not keyed on any Symbol, so the handle in the returned Link
442 template<class T
, size_t Align
= alignof(T
), bool N
= false>
443 Link
<T
,N
> alloc(Mode mode
= Mode::Normal
);
446 * Allocate a single anonymous bit from non-persistent RDS. The bit
447 * can be manipulated with testAndSetBit().
449 * Note: the returned integer is *not* an rds::Handle.
452 bool testAndSetBit(size_t bit
);
454 folly::Optional
<Symbol
> reverseLink(Handle handle
);
456 //////////////////////////////////////////////////////////////////////
459 * Retrieve the current generation number for the normal segment.
461 GenNumber
currentGenNumber();
464 * Retrieve the handle for the current generation number for the normal segment.
466 Handle
currentGenNumberHandle();
468 //////////////////////////////////////////////////////////////////////
471 * Dereference an un-typed rds::Handle, optionally specifying a
472 * specific RDS base to use.
474 template<class T
> T
& handleToRef(Handle h
);
475 template<class T
> T
& handleToRef(void* base
, Handle h
);
478 * Whether `handle' looks valid---i.e., whether it lies within the RDS bounds.
480 bool isValidHandle(Handle handle
);
483 * Whether `handle' is from the normal RDS region.
485 * Pre: isValidHandle(handle)
487 bool isNormalHandle(Handle handle
);
490 * Whether `handle' is from the local RDS region.
492 * Pre: isValidHandle(handle)
494 bool isLocalHandle(Handle handle
);
497 * Whether `handle' is from the persistent RDS region.
499 * Pre: isValidHandle(handle)
501 bool isPersistentHandle(Handle handle
);
504 * The generation number associated with `handle'.
506 * Pre: isNormalHandle(handle)
508 GenNumber
genNumberOf(Handle handle
);
511 * The handle for the generation number associated with `handle'.
513 * Pre: isNormalHandle(handle)
515 Handle
genNumberHandleFrom(Handle handle
);
518 * Whether the handle has been bound.
520 bool isHandleBound(Handle handle
);
523 * Whether the element associated with `handle' is initialized.
525 bool isHandleInit(Handle handle
);
526 bool isHandleInit(Handle handle
, NormalTag
);
529 * Mark the element associated with `handle' as being initialized.
531 * Pre: isNormalHandle(handle)
533 void initHandle(Handle handle
);
536 * Marks the element associated with the supplied handle as being
537 * uninitialized. This happens automatically after every request, but can be
538 * done manually with this.
540 * Pre: isNormalHandle(handle)
542 void uninitHandle(Handle handle
);
545 * Used to record information about the rds handle h in the
546 * perf-data-pid.map (if enabled).
547 * The type indicates the type of entry (eg StaticLocal), and the
548 * msg identifies this particular entry (eg function-name:local-name)
550 void recordRds(Handle h
, size_t size
,
551 const std::string
& type
, const std::string
& msg
);
552 void recordRds(Handle h
, size_t size
, const Symbol
& sym
);
554 void visitSymbols(std::function
<void(const Symbol
&,Handle
,uint32_t)> fun
);
557 * Return a list of all the tl_bases for any threads that are using RDS
559 std::vector
<void*> allTLBases();
562 * Values for dynamically defined constants are stored as key value
563 * pairs in an array, accessible here.
565 Array
& s_constants();
567 //////////////////////////////////////////////////////////////////////
571 #include "hphp/runtime/base/rds-inl.h"