1This file describes some special Python build types enabled via compile-time 2preprocessor defines. 3 4IMPORTANT: if you want to build a debug-enabled Python, it is recommended that 5you use ``./configure --with-pydebug``, rather than the options listed here. 6 7However, if you wish to define some of these options individually, it is best 8to define them in the EXTRA_CFLAGS make variable; 9``make EXTRA_CFLAGS="-DPy_REF_DEBUG"``. 10 11 12Py_REF_DEBUG 13------------ 14 15Turn on aggregate reference counting. This arranges that extern _Py_RefTotal 16hold a count of all references, the sum of ob_refcnt across all objects. In a 17debug-mode build, this is where the "8288" comes from in 18 19 >>> 23 20 23 21 [8288 refs] 22 >>> 23 24Note that if this count increases when you're not storing away new objects, 25there's probably a leak. Remember, though, that in interactive mode the special 26name "_" holds a reference to the last result displayed! 27 28Py_REF_DEBUG also checks after every decref to verify that the refcount hasn't 29gone negative, and causes an immediate fatal error if it has. 30 31Special gimmicks: 32 33sys.gettotalrefcount() 34 Return current total of all refcounts. 35 36 37Py_TRACE_REFS 38------------- 39 40Turn on heavy reference debugging. This is major surgery. Every PyObject grows 41two more pointers, to maintain a doubly-linked list of all live heap-allocated 42objects. Most built-in type objects are not in this list, as they're statically 43allocated. Starting in Python 2.3, if COUNT_ALLOCS (see below) is also defined, 44a static type object T does appear in this list if at least one object of type T 45has been created. 46 47Note that because the fundamental PyObject layout changes, Python modules 48compiled with Py_TRACE_REFS are incompatible with modules compiled without it. 49 50Py_TRACE_REFS implies Py_REF_DEBUG. 51 52Special gimmicks: 53 54sys.getobjects(max[, type]) 55 Return list of the (no more than) max most-recently allocated objects, most 56 recently allocated first in the list, least-recently allocated last in the 57 list. max=0 means no limit on list length. If an optional type object is 58 passed, the list is also restricted to objects of that type. The return 59 list itself, and some temp objects created just to call sys.getobjects(), 60 are excluded from the return list. Note that the list returned is just 61 another object, though, so may appear in the return list the next time you 62 call getobjects(); note that every object in the list is kept alive too, 63 simply by virtue of being in the list. 64 65envvar PYTHONDUMPREFS 66 If this envvar exists, Py_Finalize() arranges to print a list of all 67 still-live heap objects. This is printed twice, in different formats, 68 before and after Py_Finalize has cleaned up everything it can clean up. The 69 first output block produces the repr() of each object so is more 70 informative; however, a lot of stuff destined to die is still alive then. 71 The second output block is much harder to work with (repr() can't be invoked 72 anymore -- the interpreter has been torn down too far), but doesn't list any 73 objects that will die. The tool script combinerefs.py can be run over this 74 to combine the info from both output blocks. The second output block, and 75 combinerefs.py, were new in Python 2.3b1. 76 77 78PYMALLOC_DEBUG 79-------------- 80 81When pymalloc is enabled (WITH_PYMALLOC is defined), calls to the PyObject_ 82memory routines are handled by Python's own small-object allocator, while calls 83to the PyMem_ memory routines are directed to the system malloc/ realloc/free. 84If PYMALLOC_DEBUG is also defined, calls to both PyObject_ and PyMem_ memory 85routines are directed to a special debugging mode of Python's small-object 86allocator. 87 88This mode fills dynamically allocated memory blocks with special, recognizable 89bit patterns, and adds debugging info on each end of dynamically allocated 90memory blocks. The special bit patterns are: 91 92#define CLEANBYTE 0xCB /* clean (newly allocated) memory */ 93#define DEADBYTE 0xDB /* dead (newly freed) memory */ 94#define FORBIDDENBYTE 0xFB /* forbidden -- untouchable bytes */ 95 96Strings of these bytes are unlikely to be valid addresses, floats, or 7-bit 97ASCII strings. 98 99Let S = sizeof(size_t). 2*S bytes are added at each end of each block of N bytes 100requested. The memory layout is like so, where p represents the address 101returned by a malloc-like or realloc-like function (p[i:j] means the slice of 102bytes from *(p+i) inclusive up to *(p+j) exclusive; note that the treatment of 103negative indices differs from a Python slice): 104 105p[-2*S:-S] 106 Number of bytes originally asked for. This is a size_t, big-endian (easier 107 to read in a memory dump). 108p[-S:0] 109 Copies of FORBIDDENBYTE. Used to catch under- writes and reads. 110p[0:N] 111 The requested memory, filled with copies of CLEANBYTE, used to catch 112 reference to uninitialized memory. When a realloc-like function is called 113 requesting a larger memory block, the new excess bytes are also filled with 114 CLEANBYTE. When a free-like function is called, these are overwritten with 115 DEADBYTE, to catch reference to freed memory. When a realloc- like function 116 is called requesting a smaller memory block, the excess old bytes are also 117 filled with DEADBYTE. 118p[N:N+S] 119 Copies of FORBIDDENBYTE. Used to catch over- writes and reads. 120p[N+S:N+2*S] 121 A serial number, incremented by 1 on each call to a malloc-like or 122 realloc-like function. Big-endian size_t. If "bad memory" is detected 123 later, the serial number gives an excellent way to set a breakpoint on the 124 next run, to capture the instant at which this block was passed out. The 125 static function bumpserialno() in obmalloc.c is the only place the serial 126 number is incremented, and exists so you can set such a breakpoint easily. 127 128A realloc-like or free-like function first checks that the FORBIDDENBYTEs at 129each end are intact. If they've been altered, diagnostic output is written to 130stderr, and the program is aborted via Py_FatalError(). The other main failure 131mode is provoking a memory error when a program reads up one of the special bit 132patterns and tries to use it as an address. If you get in a debugger then and 133look at the object, you're likely to see that it's entirely filled with 0xDB 134(meaning freed memory is getting used) or 0xCB (meaning uninitialized memory is 135getting used). 136 137Note that PYMALLOC_DEBUG requires WITH_PYMALLOC. 138 139Special gimmicks: 140 141envvar PYTHONMALLOCSTATS 142 If this envvar exists, a report of pymalloc summary statistics is printed to 143 stderr whenever a new arena is allocated, and also by Py_Finalize(). 144 145Changed in 2.5: The number of extra bytes allocated is 4*sizeof(size_t). 146Before it was 16 on all boxes, reflecting that Python couldn't make use of 147allocations >= 2**32 bytes even on 64-bit boxes before 2.5. 148 149 150Py_DEBUG 151-------- 152 153This is what is generally meant by "a debug build" of Python. 154 155Py_DEBUG implies LLTRACE, Py_REF_DEBUG, Py_TRACE_REFS, and PYMALLOC_DEBUG (if 156WITH_PYMALLOC is enabled). In addition, C assert()s are enabled (via the C way: 157by not defining NDEBUG), and some routines do additional sanity checks inside 158"#ifdef Py_DEBUG" blocks. 159 160 161COUNT_ALLOCS 162------------ 163 164Each type object grows three new members: 165 166 /* Number of times an object of this type was allocated. */ 167 int tp_allocs; 168 169 /* Number of times an object of this type was deallocated. */ 170 int tp_frees; 171 172 /* Highwater mark: the maximum value of tp_allocs - tp_frees so 173 * far; or, IOW, the largest number of objects of this type alive at 174 * the same time. 175 */ 176 int tp_maxalloc; 177 178Allocation and deallocation code keeps these counts up to date. Py_Finalize() 179displays a summary of the info returned by sys.getcounts() (see below), along 180with assorted other special allocation counts (like the number of tuple 181allocations satisfied by a tuple free-list, the number of 1-character strings 182allocated, etc). 183 184Before Python 2.2, type objects were immortal, and the COUNT_ALLOCS 185implementation relies on that. As of Python 2.2, heap-allocated type/ class 186objects can go away. COUNT_ALLOCS can blow up in 2.2 and 2.2.1 because of this; 187this was fixed in 2.2.2. Use of COUNT_ALLOCS makes all heap-allocated type 188objects immortal, except for those for which no object of that type is ever 189allocated. 190 191Starting with Python 2.3, If Py_TRACE_REFS is also defined, COUNT_ALLOCS 192arranges to ensure that the type object for each allocated object appears in the 193doubly-linked list of all objects maintained by Py_TRACE_REFS. 194 195Special gimmicks: 196 197sys.getcounts() 198 Return a list of 4-tuples, one entry for each type object for which at least 199 one object of that type was allocated. Each tuple is of the form: 200 201 (tp_name, tp_allocs, tp_frees, tp_maxalloc) 202 203 Each distinct type object gets a distinct entry in this list, even if two or 204 more type objects have the same tp_name (in which case there's no way to 205 distinguish them by looking at this list). The list is ordered by time of 206 first object allocation: the type object for which the first allocation of 207 an object of that type occurred most recently is at the front of the list. 208 209 210LLTRACE 211------- 212 213Compile in support for Low Level TRACE-ing of the main interpreter loop. 214 215When this preprocessor symbol is defined, before PyEval_EvalFrame executes a 216frame's code it checks the frame's global namespace for a variable 217"__lltrace__". If such a variable is found, mounds of information about what 218the interpreter is doing are sprayed to stdout, such as every opcode and opcode 219argument and values pushed onto and popped off the value stack. 220 221Not useful very often, but very useful when needed. 222 223 224CALL_PROFILE 225------------ 226 227Count the number of function calls executed. 228 229When this symbol is defined, the ceval mainloop and helper functions count the 230number of function calls made. It keeps detailed statistics about what kind of 231object was called and whether the call hit any of the special fast paths in the 232code. 233 234 235WITH_TSC 236-------- 237 238Super-lowlevel profiling of the interpreter. When enabled, the sys module grows 239a new function: 240 241settscdump(bool) 242 If true, tell the Python interpreter to dump VM measurements to stderr. If 243 false, turn off dump. The measurements are based on the processor's 244 time-stamp counter. 245 246This build option requires a small amount of platform specific code. Currently 247this code is present for linux/x86 and any PowerPC platform that uses GCC 248(i.e. OS X and linux/ppc). 249 250On the PowerPC the rate at which the time base register is incremented is not 251defined by the architecture specification, so you'll need to find the manual for 252your specific processor. For the 750CX, 750CXe and 750FX (all sold as the G3) 253we find: 254 255 The time base counter is clocked at a frequency that is one-fourth that of 256 the bus clock. 257 258This build is enabled by the --with-tsc flag to configure. 259