1:mod:`weakref` --- Weak references 2================================== 3 4.. module:: weakref 5 :synopsis: Support for weak references and weak dictionaries. 6.. moduleauthor:: Fred L. Drake, Jr. <fdrake@acm.org> 7.. moduleauthor:: Neil Schemenauer <nas@arctrix.com> 8.. moduleauthor:: Martin von Löwis <martin@loewis.home.cs.tu-berlin.de> 9.. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org> 10 11 12.. versionadded:: 2.1 13 14**Source code:** :source:`Lib/weakref.py` 15 16-------------- 17 18The :mod:`weakref` module allows the Python programmer to create :dfn:`weak 19references` to objects. 20 21.. When making changes to the examples in this file, be sure to update 22 Lib/test/test_weakref.py::libreftest too! 23 24In the following, the term :dfn:`referent` means the object which is referred to 25by a weak reference. 26 27A weak reference to an object is not enough to keep the object alive: when the 28only remaining references to a referent are weak references, 29:term:`garbage collection` is free to destroy the referent and reuse its memory 30for something else. A primary use for weak references is to implement caches or 31mappings holding large objects, where it's desired that a large object not be 32kept alive solely because it appears in a cache or mapping. 33 34For example, if you have a number of large binary image objects, you may wish to 35associate a name with each. If you used a Python dictionary to map names to 36images, or images to names, the image objects would remain alive just because 37they appeared as values or keys in the dictionaries. The 38:class:`WeakKeyDictionary` and :class:`WeakValueDictionary` classes supplied by 39the :mod:`weakref` module are an alternative, using weak references to construct 40mappings that don't keep objects alive solely because they appear in the mapping 41objects. If, for example, an image object is a value in a 42:class:`WeakValueDictionary`, then when the last remaining references to that 43image object are the weak references held by weak mappings, garbage collection 44can reclaim the object, and its corresponding entries in weak mappings are 45simply deleted. 46 47:class:`WeakKeyDictionary` and :class:`WeakValueDictionary` use weak references 48in their implementation, setting up callback functions on the weak references 49that notify the weak dictionaries when a key or value has been reclaimed by 50garbage collection. Most programs should find that using one of these weak 51dictionary types is all they need -- it's not usually necessary to create your 52own weak references directly. The low-level machinery used by the weak 53dictionary implementations is exposed by the :mod:`weakref` module for the 54benefit of advanced uses. 55 56Not all objects can be weakly referenced; those objects which can include class 57instances, functions written in Python (but not in C), methods (both bound and 58unbound), sets, frozensets, file objects, :term:`generator`\s, type objects, 59:class:`DBcursor` objects from the :mod:`bsddb` module, sockets, arrays, deques, 60regular expression pattern objects, and code objects. 61 62.. versionchanged:: 2.4 63 Added support for files, sockets, arrays, and patterns. 64 65.. versionchanged:: 2.7 66 Added support for thread.lock, threading.Lock, and code objects. 67 68Several built-in types such as :class:`list` and :class:`dict` do not directly 69support weak references but can add support through subclassing:: 70 71 class Dict(dict): 72 pass 73 74 obj = Dict(red=1, green=2, blue=3) # this object is weak referenceable 75 76.. impl-detail:: 77 78 Other built-in types such as :class:`tuple` and :class:`long` do not support 79 weak references even when subclassed. 80 81Extension types can easily be made to support weak references; see 82:ref:`weakref-support`. 83 84 85.. class:: ref(object[, callback]) 86 87 Return a weak reference to *object*. The original object can be retrieved by 88 calling the reference object if the referent is still alive; if the referent is 89 no longer alive, calling the reference object will cause :const:`None` to be 90 returned. If *callback* is provided and not :const:`None`, and the returned 91 weakref object is still alive, the callback will be called when the object is 92 about to be finalized; the weak reference object will be passed as the only 93 parameter to the callback; the referent will no longer be available. 94 95 It is allowable for many weak references to be constructed for the same object. 96 Callbacks registered for each weak reference will be called from the most 97 recently registered callback to the oldest registered callback. 98 99 Exceptions raised by the callback will be noted on the standard error output, 100 but cannot be propagated; they are handled in exactly the same way as exceptions 101 raised from an object's :meth:`__del__` method. 102 103 Weak references are :term:`hashable` if the *object* is hashable. They will maintain 104 their hash value even after the *object* was deleted. If :func:`hash` is called 105 the first time only after the *object* was deleted, the call will raise 106 :exc:`TypeError`. 107 108 Weak references support tests for equality, but not ordering. If the referents 109 are still alive, two references have the same equality relationship as their 110 referents (regardless of the *callback*). If either referent has been deleted, 111 the references are equal only if the reference objects are the same object. 112 113 .. versionchanged:: 2.4 114 This is now a subclassable type rather than a factory function; it derives from 115 :class:`object`. 116 117 118.. function:: proxy(object[, callback]) 119 120 Return a proxy to *object* which uses a weak reference. This supports use of 121 the proxy in most contexts instead of requiring the explicit dereferencing used 122 with weak reference objects. The returned object will have a type of either 123 ``ProxyType`` or ``CallableProxyType``, depending on whether *object* is 124 callable. Proxy objects are not :term:`hashable` regardless of the referent; this 125 avoids a number of problems related to their fundamentally mutable nature, and 126 prevent their use as dictionary keys. *callback* is the same as the parameter 127 of the same name to the :func:`ref` function. 128 129 130.. function:: getweakrefcount(object) 131 132 Return the number of weak references and proxies which refer to *object*. 133 134 135.. function:: getweakrefs(object) 136 137 Return a list of all weak reference and proxy objects which refer to *object*. 138 139 140.. class:: WeakKeyDictionary([dict]) 141 142 Mapping class that references keys weakly. Entries in the dictionary will be 143 discarded when there is no longer a strong reference to the key. This can be 144 used to associate additional data with an object owned by other parts of an 145 application without adding attributes to those objects. This can be especially 146 useful with objects that override attribute accesses. 147 148 .. note:: 149 150 Caution: Because a :class:`WeakKeyDictionary` is built on top of a Python 151 dictionary, it must not change size when iterating over it. This can be 152 difficult to ensure for a :class:`WeakKeyDictionary` because actions 153 performed by the program during iteration may cause items in the 154 dictionary to vanish "by magic" (as a side effect of garbage collection). 155 156:class:`WeakKeyDictionary` objects have the following additional methods. These 157expose the internal references directly. The references are not guaranteed to 158be "live" at the time they are used, so the result of calling the references 159needs to be checked before being used. This can be used to avoid creating 160references that will cause the garbage collector to keep the keys around longer 161than needed. 162 163 164.. method:: WeakKeyDictionary.iterkeyrefs() 165 166 Return an iterable of the weak references to the keys. 167 168 .. versionadded:: 2.5 169 170 171.. method:: WeakKeyDictionary.keyrefs() 172 173 Return a list of weak references to the keys. 174 175 .. versionadded:: 2.5 176 177 178.. class:: WeakValueDictionary([dict]) 179 180 Mapping class that references values weakly. Entries in the dictionary will be 181 discarded when no strong reference to the value exists any more. 182 183 .. note:: 184 185 Caution: Because a :class:`WeakValueDictionary` is built on top of a Python 186 dictionary, it must not change size when iterating over it. This can be 187 difficult to ensure for a :class:`WeakValueDictionary` because actions performed 188 by the program during iteration may cause items in the dictionary to vanish "by 189 magic" (as a side effect of garbage collection). 190 191:class:`WeakValueDictionary` objects have the following additional methods. 192These method have the same issues as the :meth:`iterkeyrefs` and :meth:`keyrefs` 193methods of :class:`WeakKeyDictionary` objects. 194 195 196.. method:: WeakValueDictionary.itervaluerefs() 197 198 Return an iterable of the weak references to the values. 199 200 .. versionadded:: 2.5 201 202 203.. method:: WeakValueDictionary.valuerefs() 204 205 Return a list of weak references to the values. 206 207 .. versionadded:: 2.5 208 209 210.. class:: WeakSet([elements]) 211 212 Set class that keeps weak references to its elements. An element will be 213 discarded when no strong reference to it exists any more. 214 215 .. versionadded:: 2.7 216 217 218.. data:: ReferenceType 219 220 The type object for weak references objects. 221 222 223.. data:: ProxyType 224 225 The type object for proxies of objects which are not callable. 226 227 228.. data:: CallableProxyType 229 230 The type object for proxies of callable objects. 231 232 233.. data:: ProxyTypes 234 235 Sequence containing all the type objects for proxies. This can make it simpler 236 to test if an object is a proxy without being dependent on naming both proxy 237 types. 238 239 240.. exception:: ReferenceError 241 242 Exception raised when a proxy object is used but the underlying object has been 243 collected. This is the same as the standard :exc:`ReferenceError` exception. 244 245 246.. seealso:: 247 248 :pep:`205` - Weak References 249 The proposal and rationale for this feature, including links to earlier 250 implementations and information about similar features in other languages. 251 252 253.. _weakref-objects: 254 255Weak Reference Objects 256---------------------- 257 258Weak reference objects have no attributes or methods, but do allow the referent 259to be obtained, if it still exists, by calling it: 260 261 >>> import weakref 262 >>> class Object: 263 ... pass 264 ... 265 >>> o = Object() 266 >>> r = weakref.ref(o) 267 >>> o2 = r() 268 >>> o is o2 269 True 270 271If the referent no longer exists, calling the reference object returns 272:const:`None`: 273 274 >>> del o, o2 275 >>> print r() 276 None 277 278Testing that a weak reference object is still live should be done using the 279expression ``ref() is not None``. Normally, application code that needs to use 280a reference object should follow this pattern:: 281 282 # r is a weak reference object 283 o = r() 284 if o is None: 285 # referent has been garbage collected 286 print "Object has been deallocated; can't frobnicate." 287 else: 288 print "Object is still live!" 289 o.do_something_useful() 290 291Using a separate test for "liveness" creates race conditions in threaded 292applications; another thread can cause a weak reference to become invalidated 293before the weak reference is called; the idiom shown above is safe in threaded 294applications as well as single-threaded applications. 295 296Specialized versions of :class:`ref` objects can be created through subclassing. 297This is used in the implementation of the :class:`WeakValueDictionary` to reduce 298the memory overhead for each entry in the mapping. This may be most useful to 299associate additional information with a reference, but could also be used to 300insert additional processing on calls to retrieve the referent. 301 302This example shows how a subclass of :class:`ref` can be used to store 303additional information about an object and affect the value that's returned when 304the referent is accessed:: 305 306 import weakref 307 308 class ExtendedRef(weakref.ref): 309 def __init__(self, ob, callback=None, **annotations): 310 super(ExtendedRef, self).__init__(ob, callback) 311 self.__counter = 0 312 for k, v in annotations.iteritems(): 313 setattr(self, k, v) 314 315 def __call__(self): 316 """Return a pair containing the referent and the number of 317 times the reference has been called. 318 """ 319 ob = super(ExtendedRef, self).__call__() 320 if ob is not None: 321 self.__counter += 1 322 ob = (ob, self.__counter) 323 return ob 324 325 326.. _weakref-example: 327 328Example 329------- 330 331This simple example shows how an application can use object IDs to retrieve 332objects that it has seen before. The IDs of the objects can then be used in 333other data structures without forcing the objects to remain alive, but the 334objects can still be retrieved by ID if they do. 335 336.. Example contributed by Tim Peters. 337 338:: 339 340 import weakref 341 342 _id2obj_dict = weakref.WeakValueDictionary() 343 344 def remember(obj): 345 oid = id(obj) 346 _id2obj_dict[oid] = obj 347 return oid 348 349 def id2obj(oid): 350 return _id2obj_dict[oid] 351 352