1:mod:`mmap` --- Memory-mapped file support 2========================================== 3 4.. module:: mmap 5 :synopsis: Interface to memory-mapped files for Unix and Windows. 6 7-------------- 8 9Memory-mapped file objects behave like both :class:`bytearray` and like 10:term:`file objects <file object>`. You can use mmap objects in most places 11where :class:`bytearray` are expected; for example, you can use the :mod:`re` 12module to search through a memory-mapped file. You can also change a single 13byte by doing ``obj[index] = 97``, or change a subsequence by assigning to a 14slice: ``obj[i1:i2] = b'...'``. You can also read and write data starting at 15the current file position, and :meth:`seek` through the file to different positions. 16 17A memory-mapped file is created by the :class:`~mmap.mmap` constructor, which is 18different on Unix and on Windows. In either case you must provide a file 19descriptor for a file opened for update. If you wish to map an existing Python 20file object, use its :meth:`fileno` method to obtain the correct value for the 21*fileno* parameter. Otherwise, you can open the file using the 22:func:`os.open` function, which returns a file descriptor directly (the file 23still needs to be closed when done). 24 25.. note:: 26 If you want to create a memory-mapping for a writable, buffered file, you 27 should :func:`~io.IOBase.flush` the file first. This is necessary to ensure 28 that local modifications to the buffers are actually available to the 29 mapping. 30 31For both the Unix and Windows versions of the constructor, *access* may be 32specified as an optional keyword parameter. *access* accepts one of four 33values: :const:`ACCESS_READ`, :const:`ACCESS_WRITE`, or :const:`ACCESS_COPY` to 34specify read-only, write-through or copy-on-write memory respectively, or 35:const:`ACCESS_DEFAULT` to defer to *prot*. *access* can be used on both Unix 36and Windows. If *access* is not specified, Windows mmap returns a 37write-through mapping. The initial memory values for all three access types 38are taken from the specified file. Assignment to an :const:`ACCESS_READ` 39memory map raises a :exc:`TypeError` exception. Assignment to an 40:const:`ACCESS_WRITE` memory map affects both memory and the underlying file. 41Assignment to an :const:`ACCESS_COPY` memory map affects memory but does not 42update the underlying file. 43 44.. versionchanged:: 3.7 45 Added :const:`ACCESS_DEFAULT` constant. 46 47To map anonymous memory, -1 should be passed as the fileno along with the length. 48 49.. class:: mmap(fileno, length, tagname=None, access=ACCESS_DEFAULT[, offset]) 50 51 **(Windows version)** Maps *length* bytes from the file specified by the 52 file handle *fileno*, and creates a mmap object. If *length* is larger 53 than the current size of the file, the file is extended to contain *length* 54 bytes. If *length* is ``0``, the maximum length of the map is the current 55 size of the file, except that if the file is empty Windows raises an 56 exception (you cannot create an empty mapping on Windows). 57 58 *tagname*, if specified and not ``None``, is a string giving a tag name for 59 the mapping. Windows allows you to have many different mappings against 60 the same file. If you specify the name of an existing tag, that tag is 61 opened, otherwise a new tag of this name is created. If this parameter is 62 omitted or ``None``, the mapping is created without a name. Avoiding the 63 use of the tag parameter will assist in keeping your code portable between 64 Unix and Windows. 65 66 *offset* may be specified as a non-negative integer offset. mmap references 67 will be relative to the offset from the beginning of the file. *offset* 68 defaults to 0. *offset* must be a multiple of the :const:`ALLOCATIONGRANULARITY`. 69 70 71.. class:: mmap(fileno, length, flags=MAP_SHARED, prot=PROT_WRITE|PROT_READ, access=ACCESS_DEFAULT[, offset]) 72 :noindex: 73 74 **(Unix version)** Maps *length* bytes from the file specified by the file 75 descriptor *fileno*, and returns a mmap object. If *length* is ``0``, the 76 maximum length of the map will be the current size of the file when 77 :class:`~mmap.mmap` is called. 78 79 *flags* specifies the nature of the mapping. :const:`MAP_PRIVATE` creates a 80 private copy-on-write mapping, so changes to the contents of the mmap 81 object will be private to this process, and :const:`MAP_SHARED` creates a 82 mapping that's shared with all other processes mapping the same areas of 83 the file. The default value is :const:`MAP_SHARED`. 84 85 *prot*, if specified, gives the desired memory protection; the two most 86 useful values are :const:`PROT_READ` and :const:`PROT_WRITE`, to specify 87 that the pages may be read or written. *prot* defaults to 88 :const:`PROT_READ \| PROT_WRITE`. 89 90 *access* may be specified in lieu of *flags* and *prot* as an optional 91 keyword parameter. It is an error to specify both *flags*, *prot* and 92 *access*. See the description of *access* above for information on how to 93 use this parameter. 94 95 *offset* may be specified as a non-negative integer offset. mmap references 96 will be relative to the offset from the beginning of the file. *offset* 97 defaults to 0. *offset* must be a multiple of :const:`ALLOCATIONGRANULARITY` 98 which is equal to :const:`PAGESIZE` on Unix systems. 99 100 To ensure validity of the created memory mapping the file specified 101 by the descriptor *fileno* is internally automatically synchronized 102 with physical backing store on Mac OS X and OpenVMS. 103 104 This example shows a simple way of using :class:`~mmap.mmap`:: 105 106 import mmap 107 108 # write a simple example file 109 with open("hello.txt", "wb") as f: 110 f.write(b"Hello Python!\n") 111 112 with open("hello.txt", "r+b") as f: 113 # memory-map the file, size 0 means whole file 114 mm = mmap.mmap(f.fileno(), 0) 115 # read content via standard file methods 116 print(mm.readline()) # prints b"Hello Python!\n" 117 # read content via slice notation 118 print(mm[:5]) # prints b"Hello" 119 # update content using slice notation; 120 # note that new content must have same size 121 mm[6:] = b" world!\n" 122 # ... and read again using standard file methods 123 mm.seek(0) 124 print(mm.readline()) # prints b"Hello world!\n" 125 # close the map 126 mm.close() 127 128 129 :class:`~mmap.mmap` can also be used as a context manager in a :keyword:`with` 130 statement:: 131 132 import mmap 133 134 with mmap.mmap(-1, 13) as mm: 135 mm.write(b"Hello world!") 136 137 .. versionadded:: 3.2 138 Context manager support. 139 140 141 The next example demonstrates how to create an anonymous map and exchange 142 data between the parent and child processes:: 143 144 import mmap 145 import os 146 147 mm = mmap.mmap(-1, 13) 148 mm.write(b"Hello world!") 149 150 pid = os.fork() 151 152 if pid == 0: # In a child process 153 mm.seek(0) 154 print(mm.readline()) 155 156 mm.close() 157 158 159 Memory-mapped file objects support the following methods: 160 161 .. method:: close() 162 163 Closes the mmap. Subsequent calls to other methods of the object will 164 result in a ValueError exception being raised. This will not close 165 the open file. 166 167 168 .. attribute:: closed 169 170 ``True`` if the file is closed. 171 172 .. versionadded:: 3.2 173 174 175 .. method:: find(sub[, start[, end]]) 176 177 Returns the lowest index in the object where the subsequence *sub* is 178 found, such that *sub* is contained in the range [*start*, *end*]. 179 Optional arguments *start* and *end* are interpreted as in slice notation. 180 Returns ``-1`` on failure. 181 182 .. versionchanged:: 3.5 183 Writable :term:`bytes-like object` is now accepted. 184 185 186 .. method:: flush([offset[, size]]) 187 188 Flushes changes made to the in-memory copy of a file back to disk. Without 189 use of this call there is no guarantee that changes are written back before 190 the object is destroyed. If *offset* and *size* are specified, only 191 changes to the given range of bytes will be flushed to disk; otherwise, the 192 whole extent of the mapping is flushed. *offset* must be a multiple of the 193 :const:`PAGESIZE` or :const:`ALLOCATIONGRANULARITY`. 194 195 **(Windows version)** A nonzero value returned indicates success; zero 196 indicates failure. 197 198 **(Unix version)** A zero value is returned to indicate success. An 199 exception is raised when the call failed. 200 201 202 .. method:: move(dest, src, count) 203 204 Copy the *count* bytes starting at offset *src* to the destination index 205 *dest*. If the mmap was created with :const:`ACCESS_READ`, then calls to 206 move will raise a :exc:`TypeError` exception. 207 208 209 .. method:: read([n]) 210 211 Return a :class:`bytes` containing up to *n* bytes starting from the 212 current file position. If the argument is omitted, ``None`` or negative, 213 return all bytes from the current file position to the end of the 214 mapping. The file position is updated to point after the bytes that were 215 returned. 216 217 .. versionchanged:: 3.3 218 Argument can be omitted or ``None``. 219 220 .. method:: read_byte() 221 222 Returns a byte at the current file position as an integer, and advances 223 the file position by 1. 224 225 226 .. method:: readline() 227 228 Returns a single line, starting at the current file position and up to the 229 next newline. 230 231 232 .. method:: resize(newsize) 233 234 Resizes the map and the underlying file, if any. If the mmap was created 235 with :const:`ACCESS_READ` or :const:`ACCESS_COPY`, resizing the map will 236 raise a :exc:`TypeError` exception. 237 238 239 .. method:: rfind(sub[, start[, end]]) 240 241 Returns the highest index in the object where the subsequence *sub* is 242 found, such that *sub* is contained in the range [*start*, *end*]. 243 Optional arguments *start* and *end* are interpreted as in slice notation. 244 Returns ``-1`` on failure. 245 246 .. versionchanged:: 3.5 247 Writable :term:`bytes-like object` is now accepted. 248 249 250 .. method:: seek(pos[, whence]) 251 252 Set the file's current position. *whence* argument is optional and 253 defaults to ``os.SEEK_SET`` or ``0`` (absolute file positioning); other 254 values are ``os.SEEK_CUR`` or ``1`` (seek relative to the current 255 position) and ``os.SEEK_END`` or ``2`` (seek relative to the file's end). 256 257 258 .. method:: size() 259 260 Return the length of the file, which can be larger than the size of the 261 memory-mapped area. 262 263 264 .. method:: tell() 265 266 Returns the current position of the file pointer. 267 268 269 .. method:: write(bytes) 270 271 Write the bytes in *bytes* into memory at the current position of the 272 file pointer and return the number of bytes written (never less than 273 ``len(bytes)``, since if the write fails, a :exc:`ValueError` will be 274 raised). The file position is updated to point after the bytes that 275 were written. If the mmap was created with :const:`ACCESS_READ`, then 276 writing to it will raise a :exc:`TypeError` exception. 277 278 .. versionchanged:: 3.5 279 Writable :term:`bytes-like object` is now accepted. 280 281 .. versionchanged:: 3.6 282 The number of bytes written is now returned. 283 284 285 .. method:: write_byte(byte) 286 287 Write the integer *byte* into memory at the current 288 position of the file pointer; the file position is advanced by ``1``. If 289 the mmap was created with :const:`ACCESS_READ`, then writing to it will 290 raise a :exc:`TypeError` exception. 291