1 This file describes some special Python build types enabled via
2 compile-time preprocessor defines.
4 It is best to define these options in the EXTRA_CFLAGS environment variable;
5 ``EXTRA_CFLAGS="-DPy_REF_DEBUG" ./configure``.
7 ---------------------------------------------------------------------------
8 Py_REF_DEBUG introduced in 1.4
9 named REF_DEBUG before 1.4
11 Turn on aggregate reference counting. This arranges that extern
12 _Py_RefTotal hold a count of all references, the sum of ob_refcnt across
13 all objects. In a debug-mode build, this is where the "8288" comes from
21 Note that if this count increases when you're not storing away new objects,
22 there's probably a leak. Remember, though, that in interactive mode the
23 special name "_" holds a reference to the last result displayed!
25 Py_REF_DEBUG also checks after every decref to verify that the refcount
26 hasn't gone negative, and causes an immediate fatal error if it has.
30 sys.gettotalrefcount()
31 Return current total of all refcounts.
32 Available under Py_REF_DEBUG in Python 2.3.
33 Before 2.3, Py_TRACE_REFS was required to enable this function.
34 ---------------------------------------------------------------------------
35 Py_TRACE_REFS introduced in 1.4
36 named TRACE_REFS before 1.4
38 Turn on heavy reference debugging. This is major surgery. Every PyObject
39 grows two more pointers, to maintain a doubly-linked list of all live
40 heap-allocated objects. Most builtin type objects are not in this list,
41 as they're statically allocated. Starting in Python 2.3, if COUNT_ALLOCS
42 (see below) is also defined, a static type object T does appear in this
43 list if at least one object of type T has been created.
45 Note that because the fundamental PyObject layout changes, Python modules
46 compiled with Py_TRACE_REFS are incompatible with modules compiled without
49 Py_TRACE_REFS implies Py_REF_DEBUG.
53 sys.getobjects(max[, type])
54 Return list of the (no more than) max most-recently allocated objects,
55 most recently allocated first in the list, least-recently allocated
56 last in the list. max=0 means no limit on list length.
57 If an optional type object is passed, the list is also restricted to
59 The return list itself, and some temp objects created just to call
60 sys.getobjects(), are excluded from the return list. Note that the
61 list returned is just another object, though, so may appear in the
62 return list the next time you call getobjects(); note that every
63 object in the list is kept alive too, simply by virtue of being in
67 If this envar exists, Py_Finalize() arranges to print a list of
68 all still-live heap objects. This is printed twice, in different
69 formats, before and after Py_Finalize has cleaned up everything it
70 can clean up. The first output block produces the repr() of each
71 object so is more informative; however, a lot of stuff destined to
72 die is still alive then. The second output block is much harder
73 to work with (repr() can't be invoked anymore -- the interpreter
74 has been torn down too far), but doesn't list any objects that will
75 die. The tool script combinerefs.py can be run over this to combine
76 the info from both output blocks. The second output block, and
77 combinerefs.py, were new in Python 2.3b1.
78 ---------------------------------------------------------------------------
79 PYMALLOC_DEBUG introduced in 2.3
81 When pymalloc is enabled (WITH_PYMALLOC is defined), calls to the PyObject_
82 memory routines are handled by Python's own small-object allocator, while
83 calls to the PyMem_ memory routines are directed to the system malloc/
84 realloc/free. If PYMALLOC_DEBUG is also defined, calls to both PyObject_
85 and PyMem_ memory routines are directed to a special debugging mode of
86 Python's small-object allocator.
88 This mode fills dynamically allocated memory blocks with special,
89 recognizable bit patterns, and adds debugging info on each end of
90 dynamically allocated memory blocks. The special bit patterns are:
92 #define CLEANBYTE 0xCB /* clean (newly allocated) memory */
93 #define DEADBYTE 0xDB /* dead (newly freed) memory */
94 #define FORBIDDENBYTE 0xFB /* fordidden -- untouchable bytes */
96 Strings of these bytes are unlikely to be valid addresses, floats, or 7-bit
99 8 bytes are added at each end of each block of N bytes requested. The
100 memory layout is like so, where p represents the address returned by a
101 malloc-like or realloc-like function (p[i:j] means the slice of bytes
102 from *(p+i) inclusive up to *(p+j) exclusive; note that the treatment
103 of negative indices differs from a Python slice):
106 Number of bytes originally asked for. 4-byte unsigned integer,
107 big-endian (easier to read in a memory dump).
109 Copies of FORBIDDENBYTE. Used to catch under- writes and reads.
111 The requested memory, filled with copies of CLEANBYTE, used to catch
112 reference to uninitialized memory.
113 When a realloc-like function is called requesting a larger memory
114 block, the new excess bytes are also filled with CLEANBYTE.
115 When a free-like function is called, these are overwritten with
116 DEADBYTE, to catch reference to freed memory. When a realloc-
117 like function is called requesting a smaller memory block, the excess
118 old bytes are also filled with DEADBYTE.
120 Copies of FORBIDDENBYTE. Used to catch over- writes and reads.
122 A serial number, incremented by 1 on each call to a malloc-like or
123 realloc-like function.
124 4-byte unsigned integer, big-endian.
125 If "bad memory" is detected later, the serial number gives an
126 excellent way to set a breakpoint on the next run, to capture the
127 instant at which this block was passed out. The static function
128 bumpserialno() in obmalloc.c is the only place the serial number
129 is incremented, and exists so you can set such a breakpoint easily.
131 A realloc-like or free-like function first checks that the FORBIDDENBYTEs
132 at each end are intact. If they've been altered, diagnostic output is
133 written to stderr, and the program is aborted via Py_FatalError(). The
134 other main failure mode is provoking a memory error when a program
135 reads up one of the special bit patterns and tries to use it as an address.
136 If you get in a debugger then and look at the object, you're likely
137 to see that it's entirely filled with 0xDB (meaning freed memory is
138 getting used) or 0xCB (meaning uninitialized memory is getting used).
140 Note that PYMALLOC_DEBUG requires WITH_PYMALLOC.
144 envar PYTHONMALLOCSTATS
145 If this envar exists, a report of pymalloc summary statistics is
146 printed to stderr whenever a new arena is allocated, and also
148 ---------------------------------------------------------------------------
149 Py_DEBUG introduced in 1.5
150 named DEBUG before 1.5
152 This is what is generally meant by "a debug build" of Python.
154 Py_DEBUG implies LLTRACE, Py_REF_DEBUG, Py_TRACE_REFS, and
155 PYMALLOC_DEBUG (if WITH_PYMALLOC is enabled). In addition, C
156 assert()s are enabled (via the C way: by not defining NDEBUG), and
157 some routines do additional sanity checks inside "#ifdef Py_DEBUG"
159 ---------------------------------------------------------------------------
160 COUNT_ALLOCS introduced in 0.9.9
161 partly broken in 2.2 and 2.2.1
163 Each type object grows three new members:
165 /* Number of times an object of this type was allocated. */
168 /* Number of times an object of this type was deallocated. */
171 /* Highwater mark: the maximum value of tp_allocs - tp_frees so
172 * far; or, IOW, the largest number of objects of this type alive at
177 Allocation and deallocation code keeps these counts up to date.
178 Py_Finalize() displays a summary of the info returned by sys.getcounts()
179 (see below), along with assorted other special allocation counts (like
180 the number of tuple allocations satisfied by a tuple free-list, the number
181 of 1-character strings allocated, etc).
183 Before Python 2.2, type objects were immortal, and the COUNT_ALLOCS
184 implementation relies on that. As of Python 2.2, heap-allocated type/
185 class objects can go away. COUNT_ALLOCS can blow up in 2.2 and 2.2.1
186 because of this; this was fixed in 2.2.2. Use of COUNT_ALLOCS makes
187 all heap-allocated type objects immortal, except for those for which no
188 object of that type is ever allocated.
190 Starting with Python 2.3, If Py_TRACE_REFS is also defined, COUNT_ALLOCS
191 arranges to ensure that the type object for each allocated object
192 appears in the doubly-linked list of all objects maintained by
198 Return a list of 4-tuples, one entry for each type object for which
199 at least one object of that type was allocated. Each tuple is of
202 (tp_name, tp_allocs, tp_frees, tp_maxalloc)
204 Each distinct type object gets a distinct entry in this list, even
205 if two or more type objects have the same tp_name (in which case
206 there's no way to distinguish them by looking at this list). The
207 list is ordered by time of first object allocation: the type object
208 for which the first allocation of an object of that type occurred
209 most recently is at the front of the list.
210 ---------------------------------------------------------------------------
211 LLTRACE introduced well before 1.0
213 Compile in support for Low Level TRACE-ing of the main interpreter loop.
215 When this preprocessor symbol is defined, before PyEval_EvalFrame
216 (eval_frame in 2.3 and 2.2, eval_code2 before that) executes a frame's code
217 it checks the frame's global namespace for a variable "__lltrace__". If
218 such a variable is found, mounds of information about what the interpreter
219 is doing are sprayed to stdout, such as every opcode and opcode argument
220 and values pushed onto and popped off the value stack.
222 Not useful very often, but very useful when needed.
224 ---------------------------------------------------------------------------
225 CALL_PROFILE introduced for Python 2.3
227 Count the number of function calls executed.
229 When this symbol is defined, the ceval mainloop and helper functions
230 count the number of function calls made. It keeps detailed statistics
231 about what kind of object was called and whether the call hit any of
232 the special fast paths in the code.
234 ---------------------------------------------------------------------------
235 WITH_TSC introduced for Python 2.4
237 Super-lowlevel profiling of the interpreter. When enabled, the sys
238 module grows a new function:
241 If true, tell the Python interpreter to dump VM measurements to
242 stderr. If false, turn off dump. The measurements are based on the
243 processor's time-stamp counter.
245 This build option requires a small amount of platform specific code.
246 Currently this code is present for linux/x86 and any PowerPC platform
247 that uses GCC (i.e. OS X and linux/ppc).
249 On the PowerPC the rate at which the time base register is incremented
250 is not defined by the architecture specification, so you'll need to
251 find the manual for your specific processor. For the 750CX, 750CXe
252 and 750FX (all sold as the G3) we find:
254 The time base counter is clocked at a frequency that is
255 one-fourth that of the bus clock.
257 This build is enabled by the --with-tsc flag to configure.