1:mod:`shelve` --- Python object persistence 2=========================================== 3 4.. module:: shelve 5 :synopsis: Python object persistence. 6 7**Source code:** :source:`Lib/shelve.py` 8 9.. index:: module: pickle 10 11-------------- 12 13A "shelf" is a persistent, dictionary-like object. The difference with "dbm" 14databases is that the values (not the keys!) in a shelf can be essentially 15arbitrary Python objects --- anything that the :mod:`pickle` module can handle. 16This includes most class instances, recursive data types, and objects containing 17lots of shared sub-objects. The keys are ordinary strings. 18 19 20.. function:: open(filename, flag='c', protocol=None, writeback=False) 21 22 Open a persistent dictionary. The filename specified is the base filename for 23 the underlying database. As a side-effect, an extension may be added to the 24 filename and more than one file may be created. By default, the underlying 25 database file is opened for reading and writing. The optional *flag* parameter 26 has the same interpretation as the *flag* parameter of :func:`dbm.open`. 27 28 By default, pickles created with :data:`pickle.DEFAULT_PROTOCOL` are used 29 to serialize values. The version of the pickle protocol can be specified 30 with the *protocol* parameter. 31 32 Because of Python semantics, a shelf cannot know when a mutable 33 persistent-dictionary entry is modified. By default modified objects are 34 written *only* when assigned to the shelf (see :ref:`shelve-example`). If the 35 optional *writeback* parameter is set to ``True``, all entries accessed are also 36 cached in memory, and written back on :meth:`~Shelf.sync` and 37 :meth:`~Shelf.close`; this can make it handier to mutate mutable entries in 38 the persistent dictionary, but, if many entries are accessed, it can consume 39 vast amounts of memory for the cache, and it can make the close operation 40 very slow since all accessed entries are written back (there is no way to 41 determine which accessed entries are mutable, nor which ones were actually 42 mutated). 43 44 .. versionchanged:: 3.10 45 :data:`pickle.DEFAULT_PROTOCOL` is now used as the default pickle 46 protocol. 47 48 .. note:: 49 50 Do not rely on the shelf being closed automatically; always call 51 :meth:`~Shelf.close` explicitly when you don't need it any more, or 52 use :func:`shelve.open` as a context manager:: 53 54 with shelve.open('spam') as db: 55 db['eggs'] = 'eggs' 56 57.. _shelve-security: 58 59.. warning:: 60 61 Because the :mod:`shelve` module is backed by :mod:`pickle`, it is insecure 62 to load a shelf from an untrusted source. Like with pickle, loading a shelf 63 can execute arbitrary code. 64 65Shelf objects support most of methods and operations supported by dictionaries 66(except copying, constructors and operators ``|`` and ``|=``). This eases the 67transition from dictionary based scripts to those requiring persistent storage. 68 69Two additional methods are supported: 70 71.. method:: Shelf.sync() 72 73 Write back all entries in the cache if the shelf was opened with *writeback* 74 set to :const:`True`. Also empty the cache and synchronize the persistent 75 dictionary on disk, if feasible. This is called automatically when the shelf 76 is closed with :meth:`close`. 77 78.. method:: Shelf.close() 79 80 Synchronize and close the persistent *dict* object. Operations on a closed 81 shelf will fail with a :exc:`ValueError`. 82 83 84.. seealso:: 85 86 `Persistent dictionary recipe <https://code.activestate.com/recipes/576642/>`_ 87 with widely supported storage formats and having the speed of native 88 dictionaries. 89 90 91Restrictions 92------------ 93 94 .. index:: 95 module: dbm.ndbm 96 module: dbm.gnu 97 98* The choice of which database package will be used (such as :mod:`dbm.ndbm` or 99 :mod:`dbm.gnu`) depends on which interface is available. Therefore it is not 100 safe to open the database directly using :mod:`dbm`. The database is also 101 (unfortunately) subject to the limitations of :mod:`dbm`, if it is used --- 102 this means that (the pickled representation of) the objects stored in the 103 database should be fairly small, and in rare cases key collisions may cause 104 the database to refuse updates. 105 106* The :mod:`shelve` module does not support *concurrent* read/write access to 107 shelved objects. (Multiple simultaneous read accesses are safe.) When a 108 program has a shelf open for writing, no other program should have it open for 109 reading or writing. Unix file locking can be used to solve this, but this 110 differs across Unix versions and requires knowledge about the database 111 implementation used. 112 113 114.. class:: Shelf(dict, protocol=None, writeback=False, keyencoding='utf-8') 115 116 A subclass of :class:`collections.abc.MutableMapping` which stores pickled 117 values in the *dict* object. 118 119 By default, pickles created with :data:`pickle.DEFAULT_PROTOCOL` are used 120 to serialize values. The version of the pickle protocol can be specified 121 with the *protocol* parameter. See the :mod:`pickle` documentation for a 122 discussion of the pickle protocols. 123 124 If the *writeback* parameter is ``True``, the object will hold a cache of all 125 entries accessed and write them back to the *dict* at sync and close times. 126 This allows natural operations on mutable entries, but can consume much more 127 memory and make sync and close take a long time. 128 129 The *keyencoding* parameter is the encoding used to encode keys before they 130 are used with the underlying dict. 131 132 A :class:`Shelf` object can also be used as a context manager, in which 133 case it will be automatically closed when the :keyword:`with` block ends. 134 135 .. versionchanged:: 3.2 136 Added the *keyencoding* parameter; previously, keys were always encoded in 137 UTF-8. 138 139 .. versionchanged:: 3.4 140 Added context manager support. 141 142 .. versionchanged:: 3.10 143 :data:`pickle.DEFAULT_PROTOCOL` is now used as the default pickle 144 protocol. 145 146 147.. class:: BsdDbShelf(dict, protocol=None, writeback=False, keyencoding='utf-8') 148 149 A subclass of :class:`Shelf` which exposes :meth:`first`, :meth:`!next`, 150 :meth:`previous`, :meth:`last` and :meth:`set_location` which are available 151 in the third-party :mod:`bsddb` module from `pybsddb 152 <https://www.jcea.es/programacion/pybsddb.htm>`_ but not in other database 153 modules. The *dict* object passed to the constructor must support those 154 methods. This is generally accomplished by calling one of 155 :func:`bsddb.hashopen`, :func:`bsddb.btopen` or :func:`bsddb.rnopen`. The 156 optional *protocol*, *writeback*, and *keyencoding* parameters have the same 157 interpretation as for the :class:`Shelf` class. 158 159 160.. class:: DbfilenameShelf(filename, flag='c', protocol=None, writeback=False) 161 162 A subclass of :class:`Shelf` which accepts a *filename* instead of a dict-like 163 object. The underlying file will be opened using :func:`dbm.open`. By 164 default, the file will be created and opened for both read and write. The 165 optional *flag* parameter has the same interpretation as for the :func:`.open` 166 function. The optional *protocol* and *writeback* parameters have the same 167 interpretation as for the :class:`Shelf` class. 168 169 170.. _shelve-example: 171 172Example 173------- 174 175To summarize the interface (``key`` is a string, ``data`` is an arbitrary 176object):: 177 178 import shelve 179 180 d = shelve.open(filename) # open -- file may get suffix added by low-level 181 # library 182 183 d[key] = data # store data at key (overwrites old data if 184 # using an existing key) 185 data = d[key] # retrieve a COPY of data at key (raise KeyError 186 # if no such key) 187 del d[key] # delete data stored at key (raises KeyError 188 # if no such key) 189 190 flag = key in d # true if the key exists 191 klist = list(d.keys()) # a list of all existing keys (slow!) 192 193 # as d was opened WITHOUT writeback=True, beware: 194 d['xx'] = [0, 1, 2] # this works as expected, but... 195 d['xx'].append(3) # *this doesn't!* -- d['xx'] is STILL [0, 1, 2]! 196 197 # having opened d without writeback=True, you need to code carefully: 198 temp = d['xx'] # extracts the copy 199 temp.append(5) # mutates the copy 200 d['xx'] = temp # stores the copy right back, to persist it 201 202 # or, d=shelve.open(filename,writeback=True) would let you just code 203 # d['xx'].append(5) and have it work as expected, BUT it would also 204 # consume more memory and make the d.close() operation slower. 205 206 d.close() # close it 207 208 209.. seealso:: 210 211 Module :mod:`dbm` 212 Generic interface to ``dbm``-style databases. 213 214 Module :mod:`pickle` 215 Object serialization used by :mod:`shelve`. 216 217