1.. highlight:: c 2 3.. _supporting-cycle-detection: 4 5Supporting Cyclic Garbage Collection 6==================================== 7 8Python's support for detecting and collecting garbage which involves circular 9references requires support from object types which are "containers" for other 10objects which may also be containers. Types which do not store references to 11other objects, or which only store references to atomic types (such as numbers 12or strings), do not need to provide any explicit support for garbage 13collection. 14 15To create a container type, the :c:member:`~PyTypeObject.tp_flags` field of the type object must 16include the :c:macro:`Py_TPFLAGS_HAVE_GC` and provide an implementation of the 17:c:member:`~PyTypeObject.tp_traverse` handler. If instances of the type are mutable, a 18:c:member:`~PyTypeObject.tp_clear` implementation must also be provided. 19 20 21:c:macro:`Py_TPFLAGS_HAVE_GC` 22 Objects with a type with this flag set must conform with the rules 23 documented here. For convenience these objects will be referred to as 24 container objects. 25 26Constructors for container types must conform to two rules: 27 28#. The memory for the object must be allocated using :c:macro:`PyObject_GC_New` 29 or :c:macro:`PyObject_GC_NewVar`. 30 31#. Once all the fields which may contain references to other containers are 32 initialized, it must call :c:func:`PyObject_GC_Track`. 33 34Similarly, the deallocator for the object must conform to a similar pair of 35rules: 36 37#. Before fields which refer to other containers are invalidated, 38 :c:func:`PyObject_GC_UnTrack` must be called. 39 40#. The object's memory must be deallocated using :c:func:`PyObject_GC_Del`. 41 42 .. warning:: 43 If a type adds the Py_TPFLAGS_HAVE_GC, then it *must* implement at least 44 a :c:member:`~PyTypeObject.tp_traverse` handler or explicitly use one 45 from its subclass or subclasses. 46 47 When calling :c:func:`PyType_Ready` or some of the APIs that indirectly 48 call it like :c:func:`PyType_FromSpecWithBases` or 49 :c:func:`PyType_FromSpec` the interpreter will automatically populate the 50 :c:member:`~PyTypeObject.tp_flags`, :c:member:`~PyTypeObject.tp_traverse` 51 and :c:member:`~PyTypeObject.tp_clear` fields if the type inherits from a 52 class that implements the garbage collector protocol and the child class 53 does *not* include the :c:macro:`Py_TPFLAGS_HAVE_GC` flag. 54 55.. c:macro:: PyObject_GC_New(TYPE, typeobj) 56 57 Analogous to :c:macro:`PyObject_New` but for container objects with the 58 :c:macro:`Py_TPFLAGS_HAVE_GC` flag set. 59 60.. c:macro:: PyObject_GC_NewVar(TYPE, typeobj, size) 61 62 Analogous to :c:macro:`PyObject_NewVar` but for container objects with the 63 :c:macro:`Py_TPFLAGS_HAVE_GC` flag set. 64 65.. c:function:: PyObject* PyUnstable_Object_GC_NewWithExtraData(PyTypeObject *type, size_t extra_size) 66 67 Analogous to :c:macro:`PyObject_GC_New` but allocates *extra_size* 68 bytes at the end of the object (at offset 69 :c:member:`~PyTypeObject.tp_basicsize`). 70 The allocated memory is initialized to zeros, 71 except for the :c:type:`Python object header <PyObject>`. 72 73 The extra data will be deallocated with the object, but otherwise it is 74 not managed by Python. 75 76 .. warning:: 77 The function is marked as unstable because the final mechanism 78 for reserving extra data after an instance is not yet decided. 79 For allocating a variable number of fields, prefer using 80 :c:type:`PyVarObject` and :c:member:`~PyTypeObject.tp_itemsize` 81 instead. 82 83 .. versionadded:: 3.12 84 85 86.. c:macro:: PyObject_GC_Resize(TYPE, op, newsize) 87 88 Resize an object allocated by :c:macro:`PyObject_NewVar`. 89 Returns the resized object of type ``TYPE*`` (refers to any C type) 90 or ``NULL`` on failure. 91 92 *op* must be of type :c:expr:`PyVarObject *` 93 and must not be tracked by the collector yet. 94 *newsize* must be of type :c:type:`Py_ssize_t`. 95 96 97.. c:function:: void PyObject_GC_Track(PyObject *op) 98 99 Adds the object *op* to the set of container objects tracked by the 100 collector. The collector can run at unexpected times so objects must be 101 valid while being tracked. This should be called once all the fields 102 followed by the :c:member:`~PyTypeObject.tp_traverse` handler become valid, usually near the 103 end of the constructor. 104 105 106.. c:function:: int PyObject_IS_GC(PyObject *obj) 107 108 Returns non-zero if the object implements the garbage collector protocol, 109 otherwise returns 0. 110 111 The object cannot be tracked by the garbage collector if this function returns 0. 112 113 114.. c:function:: int PyObject_GC_IsTracked(PyObject *op) 115 116 Returns 1 if the object type of *op* implements the GC protocol and *op* is being 117 currently tracked by the garbage collector and 0 otherwise. 118 119 This is analogous to the Python function :func:`gc.is_tracked`. 120 121 .. versionadded:: 3.9 122 123 124.. c:function:: int PyObject_GC_IsFinalized(PyObject *op) 125 126 Returns 1 if the object type of *op* implements the GC protocol and *op* has been 127 already finalized by the garbage collector and 0 otherwise. 128 129 This is analogous to the Python function :func:`gc.is_finalized`. 130 131 .. versionadded:: 3.9 132 133 134.. c:function:: void PyObject_GC_Del(void *op) 135 136 Releases memory allocated to an object using :c:macro:`PyObject_GC_New` or 137 :c:macro:`PyObject_GC_NewVar`. 138 139 140.. c:function:: void PyObject_GC_UnTrack(void *op) 141 142 Remove the object *op* from the set of container objects tracked by the 143 collector. Note that :c:func:`PyObject_GC_Track` can be called again on 144 this object to add it back to the set of tracked objects. The deallocator 145 (:c:member:`~PyTypeObject.tp_dealloc` handler) should call this for the object before any of 146 the fields used by the :c:member:`~PyTypeObject.tp_traverse` handler become invalid. 147 148 149.. versionchanged:: 3.8 150 151 The :c:func:`!_PyObject_GC_TRACK` and :c:func:`!_PyObject_GC_UNTRACK` macros 152 have been removed from the public C API. 153 154The :c:member:`~PyTypeObject.tp_traverse` handler accepts a function parameter of this type: 155 156 157.. c:type:: int (*visitproc)(PyObject *object, void *arg) 158 159 Type of the visitor function passed to the :c:member:`~PyTypeObject.tp_traverse` handler. 160 The function should be called with an object to traverse as *object* and 161 the third parameter to the :c:member:`~PyTypeObject.tp_traverse` handler as *arg*. The 162 Python core uses several visitor functions to implement cyclic garbage 163 detection; it's not expected that users will need to write their own 164 visitor functions. 165 166The :c:member:`~PyTypeObject.tp_traverse` handler must have the following type: 167 168 169.. c:type:: int (*traverseproc)(PyObject *self, visitproc visit, void *arg) 170 171 Traversal function for a container object. Implementations must call the 172 *visit* function for each object directly contained by *self*, with the 173 parameters to *visit* being the contained object and the *arg* value passed 174 to the handler. The *visit* function must not be called with a ``NULL`` 175 object argument. If *visit* returns a non-zero value that value should be 176 returned immediately. 177 178To simplify writing :c:member:`~PyTypeObject.tp_traverse` handlers, a :c:func:`Py_VISIT` macro is 179provided. In order to use this macro, the :c:member:`~PyTypeObject.tp_traverse` implementation 180must name its arguments exactly *visit* and *arg*: 181 182 183.. c:function:: void Py_VISIT(PyObject *o) 184 185 If *o* is not ``NULL``, call the *visit* callback, with arguments *o* 186 and *arg*. If *visit* returns a non-zero value, then return it. 187 Using this macro, :c:member:`~PyTypeObject.tp_traverse` handlers 188 look like:: 189 190 static int 191 my_traverse(Noddy *self, visitproc visit, void *arg) 192 { 193 Py_VISIT(self->foo); 194 Py_VISIT(self->bar); 195 return 0; 196 } 197 198The :c:member:`~PyTypeObject.tp_clear` handler must be of the :c:type:`inquiry` type, or ``NULL`` 199if the object is immutable. 200 201 202.. c:type:: int (*inquiry)(PyObject *self) 203 204 Drop references that may have created reference cycles. Immutable objects 205 do not have to define this method since they can never directly create 206 reference cycles. Note that the object must still be valid after calling 207 this method (don't just call :c:func:`Py_DECREF` on a reference). The 208 collector will call this method if it detects that this object is involved 209 in a reference cycle. 210 211 212Controlling the Garbage Collector State 213--------------------------------------- 214 215The C-API provides the following functions for controlling 216garbage collection runs. 217 218.. c:function:: Py_ssize_t PyGC_Collect(void) 219 220 Perform a full garbage collection, if the garbage collector is enabled. 221 (Note that :func:`gc.collect` runs it unconditionally.) 222 223 Returns the number of collected + unreachable objects which cannot 224 be collected. 225 If the garbage collector is disabled or already collecting, 226 returns ``0`` immediately. 227 Errors during garbage collection are passed to :data:`sys.unraisablehook`. 228 This function does not raise exceptions. 229 230 231.. c:function:: int PyGC_Enable(void) 232 233 Enable the garbage collector: similar to :func:`gc.enable`. 234 Returns the previous state, 0 for disabled and 1 for enabled. 235 236 .. versionadded:: 3.10 237 238 239.. c:function:: int PyGC_Disable(void) 240 241 Disable the garbage collector: similar to :func:`gc.disable`. 242 Returns the previous state, 0 for disabled and 1 for enabled. 243 244 .. versionadded:: 3.10 245 246 247.. c:function:: int PyGC_IsEnabled(void) 248 249 Query the state of the garbage collector: similar to :func:`gc.isenabled`. 250 Returns the current state, 0 for disabled and 1 for enabled. 251 252 .. versionadded:: 3.10 253 254 255Querying Garbage Collector State 256-------------------------------- 257 258The C-API provides the following interface for querying information about 259the garbage collector. 260 261.. c:function:: void PyUnstable_GC_VisitObjects(gcvisitobjects_t callback, void *arg) 262 263 Run supplied *callback* on all live GC-capable objects. *arg* is passed through to 264 all invocations of *callback*. 265 266 .. warning:: 267 If new objects are (de)allocated by the callback it is undefined if they 268 will be visited. 269 270 Garbage collection is disabled during operation. Explicitly running a collection 271 in the callback may lead to undefined behaviour e.g. visiting the same objects 272 multiple times or not at all. 273 274 .. versionadded:: 3.12 275 276.. c:type:: int (*gcvisitobjects_t)(PyObject *object, void *arg) 277 278 Type of the visitor function to be passed to :c:func:`PyUnstable_GC_VisitObjects`. 279 *arg* is the same as the *arg* passed to ``PyUnstable_GC_VisitObjects``. 280 Return ``0`` to continue iteration, return ``1`` to stop iteration. Other return 281 values are reserved for now so behavior on returning anything else is undefined. 282 283 .. versionadded:: 3.12 284 285 286