1:mod:`io` --- Core tools for working with streams 2================================================= 3 4.. module:: io 5 :synopsis: Core tools for working with streams. 6.. moduleauthor:: Guido van Rossum <guido@python.org> 7.. moduleauthor:: Mike Verdone <mike.verdone@gmail.com> 8.. moduleauthor:: Mark Russell <mark.russell@zen.co.uk> 9.. moduleauthor:: Antoine Pitrou <solipsis@pitrou.net> 10.. moduleauthor:: Amaury Forgeot d'Arc <amauryfa@gmail.com> 11.. moduleauthor:: Benjamin Peterson <benjamin@python.org> 12.. sectionauthor:: Benjamin Peterson <benjamin@python.org> 13 14.. versionadded:: 2.6 15 16The :mod:`io` module provides the Python interfaces to stream handling. 17Under Python 2.x, this is proposed as an alternative to the built-in 18:class:`file` object, but in Python 3.x it is the default interface to 19access files and streams. 20 21.. note:: 22 23 Since this module has been designed primarily for Python 3.x, you have to 24 be aware that all uses of "bytes" in this document refer to the 25 :class:`str` type (of which :class:`bytes` is an alias), and all uses 26 of "text" refer to the :class:`unicode` type. Furthermore, those two 27 types are not interchangeable in the :mod:`io` APIs. 28 29At the top of the I/O hierarchy is the abstract base class :class:`IOBase`. It 30defines the basic interface to a stream. Note, however, that there is no 31separation between reading and writing to streams; implementations are allowed 32to raise an :exc:`IOError` if they do not support a given operation. 33 34Extending :class:`IOBase` is :class:`RawIOBase` which deals simply with the 35reading and writing of raw bytes to a stream. :class:`FileIO` subclasses 36:class:`RawIOBase` to provide an interface to files in the machine's 37file system. 38 39:class:`BufferedIOBase` deals with buffering on a raw byte stream 40(:class:`RawIOBase`). Its subclasses, :class:`BufferedWriter`, 41:class:`BufferedReader`, and :class:`BufferedRWPair` buffer streams that are 42readable, writable, and both readable and writable. 43:class:`BufferedRandom` provides a buffered interface to random access 44streams. :class:`BytesIO` is a simple stream of in-memory bytes. 45 46Another :class:`IOBase` subclass, :class:`TextIOBase`, deals with 47streams whose bytes represent text, and handles encoding and decoding 48from and to :class:`unicode` strings. :class:`TextIOWrapper`, which extends 49it, is a buffered text interface to a buffered raw stream 50(:class:`BufferedIOBase`). Finally, :class:`~io.StringIO` is an in-memory 51stream for unicode text. 52 53Argument names are not part of the specification, and only the arguments of 54:func:`.open` are intended to be used as keyword arguments. 55 56 57Module Interface 58---------------- 59 60.. data:: DEFAULT_BUFFER_SIZE 61 62 An int containing the default buffer size used by the module's buffered I/O 63 classes. :func:`.open` uses the file's blksize (as obtained by 64 :func:`os.stat`) if possible. 65 66.. function:: open(file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True) 67 68 Open *file* and return a corresponding stream. If the file cannot be opened, 69 an :exc:`IOError` is raised. 70 71 *file* is either a string giving the pathname (absolute or 72 relative to the current working directory) of the file to be opened or 73 an integer file descriptor of the file to be wrapped. (If a file descriptor 74 is given, it is closed when the returned I/O object is closed, unless 75 *closefd* is set to ``False``.) 76 77 *mode* is an optional string that specifies the mode in which the file is 78 opened. It defaults to ``'r'`` which means open for reading in text mode. 79 Other common values are ``'w'`` for writing (truncating the file if it 80 already exists), and ``'a'`` for appending (which on *some* Unix systems, 81 means that *all* writes append to the end of the file regardless of the 82 current seek position). In text mode, if *encoding* is not specified the 83 encoding used is platform dependent. (For reading and writing raw bytes use 84 binary mode and leave *encoding* unspecified.) The available modes are: 85 86 ========= =============================================================== 87 Character Meaning 88 --------- --------------------------------------------------------------- 89 ``'r'`` open for reading (default) 90 ``'w'`` open for writing, truncating the file first 91 ``'a'`` open for writing, appending to the end of the file if it exists 92 ``'b'`` binary mode 93 ``'t'`` text mode (default) 94 ``'+'`` open a disk file for updating (reading and writing) 95 ``'U'`` universal newlines mode (for backwards compatibility; should 96 not be used in new code) 97 ========= =============================================================== 98 99 The default mode is ``'rt'`` (open for reading text). For binary random 100 access, the mode ``'w+b'`` opens and truncates the file to 0 bytes, while 101 ``'r+b'`` opens the file without truncation. 102 103 Python distinguishes between files opened in binary and text modes, even when 104 the underlying operating system doesn't. Files opened in binary mode 105 (including ``'b'`` in the *mode* argument) return contents as :class:`bytes` 106 objects without any decoding. In text mode (the default, or when ``'t'`` is 107 included in the *mode* argument), the contents of the file are returned as 108 :class:`unicode` strings, the bytes having been first decoded using a 109 platform-dependent encoding or using the specified *encoding* if given. 110 111 *buffering* is an optional integer used to set the buffering policy. 112 Pass 0 to switch buffering off (only allowed in binary mode), 1 to select 113 line buffering (only usable in text mode), and an integer > 1 to indicate 114 the size of a fixed-size chunk buffer. When no *buffering* argument is 115 given, the default buffering policy works as follows: 116 117 * Binary files are buffered in fixed-size chunks; the size of the buffer 118 is chosen using a heuristic trying to determine the underlying device's 119 "block size" and falling back on :attr:`DEFAULT_BUFFER_SIZE`. 120 On many systems, the buffer will typically be 4096 or 8192 bytes long. 121 122 * "Interactive" text files (files for which :meth:`isatty` returns True) 123 use line buffering. Other text files use the policy described above 124 for binary files. 125 126 *encoding* is the name of the encoding used to decode or encode the file. 127 This should only be used in text mode. The default encoding is platform 128 dependent (whatever :func:`locale.getpreferredencoding` returns), but any 129 encoding supported by Python can be used. See the :mod:`codecs` module for 130 the list of supported encodings. 131 132 *errors* is an optional string that specifies how encoding and decoding 133 errors are to be handled—this cannot be used in binary mode. Pass 134 ``'strict'`` to raise a :exc:`ValueError` exception if there is an encoding 135 error (the default of ``None`` has the same effect), or pass ``'ignore'`` to 136 ignore errors. (Note that ignoring encoding errors can lead to data loss.) 137 ``'replace'`` causes a replacement marker (such as ``'?'``) to be inserted 138 where there is malformed data. When writing, ``'xmlcharrefreplace'`` 139 (replace with the appropriate XML character reference) or 140 ``'backslashreplace'`` (replace with backslashed escape sequences) can be 141 used. Any other error handling name that has been registered with 142 :func:`codecs.register_error` is also valid. 143 144 .. index:: 145 single: universal newlines; open() (in module io) 146 147 *newline* controls how :term:`universal newlines` works (it only applies to 148 text mode). It can be ``None``, ``''``, ``'\n'``, ``'\r'``, and ``'\r\n'``. 149 It works as follows: 150 151 * On input, if *newline* is ``None``, universal newlines mode is enabled. 152 Lines in the input can end in ``'\n'``, ``'\r'``, or ``'\r\n'``, and these 153 are translated into ``'\n'`` before being returned to the caller. If it is 154 ``''``, universal newlines mode is enabled, but line endings are returned to 155 the caller untranslated. If it has any of the other legal values, input 156 lines are only terminated by the given string, and the line ending is 157 returned to the caller untranslated. 158 159 * On output, if *newline* is ``None``, any ``'\n'`` characters written are 160 translated to the system default line separator, :data:`os.linesep`. If 161 *newline* is ``''``, no translation takes place. If *newline* is any of 162 the other legal values, any ``'\n'`` characters written are translated to 163 the given string. 164 165 If *closefd* is ``False`` and a file descriptor rather than a filename was 166 given, the underlying file descriptor will be kept open when the file is 167 closed. If a filename is given *closefd* has no effect and must be ``True`` 168 (the default). 169 170 The type of file object returned by the :func:`.open` function depends on the 171 mode. When :func:`.open` is used to open a file in a text mode (``'w'``, 172 ``'r'``, ``'wt'``, ``'rt'``, etc.), it returns a subclass of 173 :class:`TextIOBase` (specifically :class:`TextIOWrapper`). When used to open 174 a file in a binary mode with buffering, the returned class is a subclass of 175 :class:`BufferedIOBase`. The exact class varies: in read binary mode, it 176 returns a :class:`BufferedReader`; in write binary and append binary modes, 177 it returns a :class:`BufferedWriter`, and in read/write mode, it returns a 178 :class:`BufferedRandom`. When buffering is disabled, the raw stream, a 179 subclass of :class:`RawIOBase`, :class:`FileIO`, is returned. 180 181 It is also possible to use an :class:`unicode` or :class:`bytes` string 182 as a file for both reading and writing. For :class:`unicode` strings 183 :class:`~io.StringIO` can be used like a file opened in text mode, 184 and for :class:`bytes` a :class:`BytesIO` can be used like a 185 file opened in a binary mode. 186 187 188.. exception:: BlockingIOError 189 190 Error raised when blocking would occur on a non-blocking stream. It inherits 191 :exc:`IOError`. 192 193 In addition to those of :exc:`IOError`, :exc:`BlockingIOError` has one 194 attribute: 195 196 .. attribute:: characters_written 197 198 An integer containing the number of characters written to the stream 199 before it blocked. 200 201 202.. exception:: UnsupportedOperation 203 204 An exception inheriting :exc:`IOError` and :exc:`ValueError` that is raised 205 when an unsupported operation is called on a stream. 206 207 208I/O Base Classes 209---------------- 210 211.. class:: IOBase 212 213 The abstract base class for all I/O classes, acting on streams of bytes. 214 There is no public constructor. 215 216 This class provides empty abstract implementations for many methods 217 that derived classes can override selectively; the default 218 implementations represent a file that cannot be read, written or 219 seeked. 220 221 Even though :class:`IOBase` does not declare :meth:`read`, :meth:`readinto`, 222 or :meth:`write` because their signatures will vary, implementations and 223 clients should consider those methods part of the interface. Also, 224 implementations may raise an :exc:`IOError` when operations they do not 225 support are called. 226 227 The basic type used for binary data read from or written to a file is 228 :class:`bytes` (also known as :class:`str`). Method arguments may 229 also be :class:`bytearray` or :class:`memoryview` of arrays of bytes. 230 In some cases, such as :meth:`~RawIOBase.readinto`, a writable object 231 such as :class:`bytearray` is required. 232 Text I/O classes work with :class:`unicode` data. 233 234 .. versionchanged:: 2.7 235 Implementations should support :class:`memoryview` arguments. 236 237 Note that calling any method (even inquiries) on a closed stream is 238 undefined. Implementations may raise :exc:`IOError` in this case. 239 240 IOBase (and its subclasses) support the iterator protocol, meaning that an 241 :class:`IOBase` object can be iterated over yielding the lines in a stream. 242 Lines are defined slightly differently depending on whether the stream is 243 a binary stream (yielding :class:`bytes`), or a text stream (yielding 244 :class:`unicode` strings). See :meth:`~IOBase.readline` below. 245 246 IOBase is also a context manager and therefore supports the 247 :keyword:`with` statement. In this example, *file* is closed after the 248 :keyword:`with` statement's suite is finished---even if an exception occurs:: 249 250 with io.open('spam.txt', 'w') as file: 251 file.write(u'Spam and eggs!') 252 253 :class:`IOBase` provides these data attributes and methods: 254 255 .. method:: close() 256 257 Flush and close this stream. This method has no effect if the file is 258 already closed. Once the file is closed, any operation on the file 259 (e.g. reading or writing) will raise a :exc:`ValueError`. 260 261 As a convenience, it is allowed to call this method more than once; 262 only the first call, however, will have an effect. 263 264 .. attribute:: closed 265 266 True if the stream is closed. 267 268 .. method:: fileno() 269 270 Return the underlying file descriptor (an integer) of the stream if it 271 exists. An :exc:`IOError` is raised if the IO object does not use a file 272 descriptor. 273 274 .. method:: flush() 275 276 Flush the write buffers of the stream if applicable. This does nothing 277 for read-only and non-blocking streams. 278 279 .. method:: isatty() 280 281 Return ``True`` if the stream is interactive (i.e., connected to 282 a terminal/tty device). 283 284 .. method:: readable() 285 286 Return ``True`` if the stream can be read from. If ``False``, :meth:`read` 287 will raise :exc:`IOError`. 288 289 .. method:: readline(limit=-1) 290 291 Read and return one line from the stream. If *limit* is specified, at 292 most *limit* bytes will be read. 293 294 The line terminator is always ``b'\n'`` for binary files; for text files, 295 the *newline* argument to :func:`.open` can be used to select the line 296 terminator(s) recognized. 297 298 .. method:: readlines(hint=-1) 299 300 Read and return a list of lines from the stream. *hint* can be specified 301 to control the number of lines read: no more lines will be read if the 302 total size (in bytes/characters) of all lines so far exceeds *hint*. 303 304 Note that it's already possible to iterate on file objects using ``for 305 line in file: ...`` without calling ``file.readlines()``. 306 307 .. method:: seek(offset[, whence]) 308 309 Change the stream position to the given byte *offset*. *offset* is 310 interpreted relative to the position indicated by *whence*. The default 311 value for *whence* is :data:`SEEK_SET`. Values for *whence* are: 312 313 * :data:`SEEK_SET` or ``0`` -- start of the stream (the default); 314 *offset* should be zero or positive 315 * :data:`SEEK_CUR` or ``1`` -- current stream position; *offset* may 316 be negative 317 * :data:`SEEK_END` or ``2`` -- end of the stream; *offset* is usually 318 negative 319 320 Return the new absolute position. 321 322 .. versionadded:: 2.7 323 The ``SEEK_*`` constants 324 325 .. method:: seekable() 326 327 Return ``True`` if the stream supports random access. If ``False``, 328 :meth:`seek`, :meth:`tell` and :meth:`truncate` will raise :exc:`IOError`. 329 330 .. method:: tell() 331 332 Return the current stream position. 333 334 .. method:: truncate(size=None) 335 336 Resize the stream to the given *size* in bytes (or the current position 337 if *size* is not specified). The current stream position isn't changed. 338 This resizing can extend or reduce the current file size. In case of 339 extension, the contents of the new file area depend on the platform 340 (on most systems, additional bytes are zero-filled, on Windows they're 341 undetermined). The new file size is returned. 342 343 .. method:: writable() 344 345 Return ``True`` if the stream supports writing. If ``False``, 346 :meth:`write` and :meth:`truncate` will raise :exc:`IOError`. 347 348 .. method:: writelines(lines) 349 350 Write a list of lines to the stream. Line separators are not added, so it 351 is usual for each of the lines provided to have a line separator at the 352 end. 353 354 .. method:: __del__() 355 356 Prepare for object destruction. :class:`IOBase` provides a default 357 implementation of this method that calls the instance's 358 :meth:`~IOBase.close` method. 359 360 361.. class:: RawIOBase 362 363 Base class for raw binary I/O. It inherits :class:`IOBase`. There is no 364 public constructor. 365 366 Raw binary I/O typically provides low-level access to an underlying OS 367 device or API, and does not try to encapsulate it in high-level primitives 368 (this is left to Buffered I/O and Text I/O, described later in this page). 369 370 In addition to the attributes and methods from :class:`IOBase`, 371 RawIOBase provides the following methods: 372 373 .. method:: read(n=-1) 374 375 Read up to *n* bytes from the object and return them. As a convenience, 376 if *n* is unspecified or -1, :meth:`readall` is called. Otherwise, 377 only one system call is ever made. Fewer than *n* bytes may be 378 returned if the operating system call returns fewer than *n* bytes. 379 380 If 0 bytes are returned, and *n* was not 0, this indicates end of file. 381 If the object is in non-blocking mode and no bytes are available, 382 ``None`` is returned. 383 384 .. method:: readall() 385 386 Read and return all the bytes from the stream until EOF, using multiple 387 calls to the stream if necessary. 388 389 .. method:: readinto(b) 390 391 Read up to len(b) bytes into *b*, and return the number 392 of bytes read. The object *b* should be a pre-allocated, writable 393 array of bytes, either :class:`bytearray` or :class:`memoryview`. 394 If the object is in non-blocking mode and no 395 bytes are available, ``None`` is returned. 396 397 .. method:: write(b) 398 399 Write *b* to the underlying raw stream, and return the 400 number of bytes written. The object *b* should be an array 401 of bytes, either :class:`bytes`, :class:`bytearray`, or 402 :class:`memoryview`. The return value can be less than 403 ``len(b)``, depending on specifics of the underlying raw stream, and 404 especially if it is in non-blocking mode. ``None`` is returned if the 405 raw stream is set not to block and no single byte could be readily 406 written to it. The caller may release or mutate *b* after 407 this method returns, so the implementation should only access *b* 408 during the method call. 409 410 411.. class:: BufferedIOBase 412 413 Base class for binary streams that support some kind of buffering. 414 It inherits :class:`IOBase`. There is no public constructor. 415 416 The main difference with :class:`RawIOBase` is that methods :meth:`read`, 417 :meth:`readinto` and :meth:`write` will try (respectively) to read as much 418 input as requested or to consume all given output, at the expense of 419 making perhaps more than one system call. 420 421 In addition, those methods can raise :exc:`BlockingIOError` if the 422 underlying raw stream is in non-blocking mode and cannot take or give 423 enough data; unlike their :class:`RawIOBase` counterparts, they will 424 never return ``None``. 425 426 Besides, the :meth:`read` method does not have a default 427 implementation that defers to :meth:`readinto`. 428 429 A typical :class:`BufferedIOBase` implementation should not inherit from a 430 :class:`RawIOBase` implementation, but wrap one, like 431 :class:`BufferedWriter` and :class:`BufferedReader` do. 432 433 :class:`BufferedIOBase` provides or overrides these methods and attribute in 434 addition to those from :class:`IOBase`: 435 436 .. attribute:: raw 437 438 The underlying raw stream (a :class:`RawIOBase` instance) that 439 :class:`BufferedIOBase` deals with. This is not part of the 440 :class:`BufferedIOBase` API and may not exist on some implementations. 441 442 .. method:: detach() 443 444 Separate the underlying raw stream from the buffer and return it. 445 446 After the raw stream has been detached, the buffer is in an unusable 447 state. 448 449 Some buffers, like :class:`BytesIO`, do not have the concept of a single 450 raw stream to return from this method. They raise 451 :exc:`UnsupportedOperation`. 452 453 .. versionadded:: 2.7 454 455 .. method:: read(n=-1) 456 457 Read and return up to *n* bytes. If the argument is omitted, ``None``, or 458 negative, data is read and returned until EOF is reached. An empty bytes 459 object is returned if the stream is already at EOF. 460 461 If the argument is positive, and the underlying raw stream is not 462 interactive, multiple raw reads may be issued to satisfy the byte count 463 (unless EOF is reached first). But for interactive raw streams, at most 464 one raw read will be issued, and a short result does not imply that EOF is 465 imminent. 466 467 A :exc:`BlockingIOError` is raised if the underlying raw stream is in 468 non blocking-mode, and has no data available at the moment. 469 470 .. method:: read1(n=-1) 471 472 Read and return up to *n* bytes, with at most one call to the underlying 473 raw stream's :meth:`~RawIOBase.read` method. This can be useful if you 474 are implementing your own buffering on top of a :class:`BufferedIOBase` 475 object. 476 477 .. method:: readinto(b) 478 479 Read up to len(b) bytes into *b*, and return the number of bytes read. 480 The object *b* should be a pre-allocated, writable array of bytes, 481 either :class:`bytearray` or :class:`memoryview`. 482 483 Like :meth:`read`, multiple reads may be issued to the underlying raw 484 stream, unless the latter is 'interactive'. 485 486 A :exc:`BlockingIOError` is raised if the underlying raw stream is in 487 non blocking-mode, and has no data available at the moment. 488 489 .. method:: write(b) 490 491 Write *b*, and return the number of bytes written 492 (always equal to ``len(b)``, since if the write fails 493 an :exc:`IOError` will be raised). The object *b* should be 494 an array of bytes, either :class:`bytes`, :class:`bytearray`, 495 or :class:`memoryview`. Depending on the actual 496 implementation, these bytes may be readily written to the underlying 497 stream, or held in a buffer for performance and latency reasons. 498 499 When in non-blocking mode, a :exc:`BlockingIOError` is raised if the 500 data needed to be written to the raw stream but it couldn't accept 501 all the data without blocking. 502 503 The caller may release or mutate *b* after this method returns, 504 so the implementation should only access *b* during the method call. 505 506 507Raw File I/O 508------------ 509 510.. class:: FileIO(name, mode='r', closefd=True) 511 512 :class:`FileIO` represents an OS-level file containing bytes data. 513 It implements the :class:`RawIOBase` interface (and therefore the 514 :class:`IOBase` interface, too). 515 516 The *name* can be one of two things: 517 518 * a string representing the path to the file which will be opened; 519 * an integer representing the number of an existing OS-level file descriptor 520 to which the resulting :class:`FileIO` object will give access. 521 522 The *mode* can be ``'r'``, ``'w'`` or ``'a'`` for reading (default), writing, 523 or appending. The file will be created if it doesn't exist when opened for 524 writing or appending; it will be truncated when opened for writing. Add a 525 ``'+'`` to the mode to allow simultaneous reading and writing. 526 527 The :meth:`read` (when called with a positive argument), :meth:`readinto` 528 and :meth:`write` methods on this class will only make one system call. 529 530 In addition to the attributes and methods from :class:`IOBase` and 531 :class:`RawIOBase`, :class:`FileIO` provides the following data 532 attributes and methods: 533 534 .. attribute:: mode 535 536 The mode as given in the constructor. 537 538 .. attribute:: name 539 540 The file name. This is the file descriptor of the file when no name is 541 given in the constructor. 542 543 544Buffered Streams 545---------------- 546 547Buffered I/O streams provide a higher-level interface to an I/O device 548than raw I/O does. 549 550.. class:: BytesIO([initial_bytes]) 551 552 A stream implementation using an in-memory bytes buffer. It inherits 553 :class:`BufferedIOBase`. 554 555 The optional argument *initial_bytes* is a :class:`bytes` object that 556 contains initial data. 557 558 :class:`BytesIO` provides or overrides these methods in addition to those 559 from :class:`BufferedIOBase` and :class:`IOBase`: 560 561 .. method:: getvalue() 562 563 Return ``bytes`` containing the entire contents of the buffer. 564 565 .. method:: read1() 566 567 In :class:`BytesIO`, this is the same as :meth:`read`. 568 569 570.. class:: BufferedReader(raw, buffer_size=DEFAULT_BUFFER_SIZE) 571 572 A buffer providing higher-level access to a readable, sequential 573 :class:`RawIOBase` object. It inherits :class:`BufferedIOBase`. 574 When reading data from this object, a larger amount of data may be 575 requested from the underlying raw stream, and kept in an internal buffer. 576 The buffered data can then be returned directly on subsequent reads. 577 578 The constructor creates a :class:`BufferedReader` for the given readable 579 *raw* stream and *buffer_size*. If *buffer_size* is omitted, 580 :data:`DEFAULT_BUFFER_SIZE` is used. 581 582 :class:`BufferedReader` provides or overrides these methods in addition to 583 those from :class:`BufferedIOBase` and :class:`IOBase`: 584 585 .. method:: peek([n]) 586 587 Return bytes from the stream without advancing the position. At most one 588 single read on the raw stream is done to satisfy the call. The number of 589 bytes returned may be less or more than requested. 590 591 .. method:: read([n]) 592 593 Read and return *n* bytes, or if *n* is not given or negative, until EOF 594 or if the read call would block in non-blocking mode. 595 596 .. method:: read1(n) 597 598 Read and return up to *n* bytes with only one call on the raw stream. If 599 at least one byte is buffered, only buffered bytes are returned. 600 Otherwise, one raw stream read call is made. 601 602 603.. class:: BufferedWriter(raw, buffer_size=DEFAULT_BUFFER_SIZE) 604 605 A buffer providing higher-level access to a writeable, sequential 606 :class:`RawIOBase` object. It inherits :class:`BufferedIOBase`. 607 When writing to this object, data is normally held into an internal 608 buffer. The buffer will be written out to the underlying :class:`RawIOBase` 609 object under various conditions, including: 610 611 * when the buffer gets too small for all pending data; 612 * when :meth:`flush()` is called; 613 * when a :meth:`seek()` is requested (for :class:`BufferedRandom` objects); 614 * when the :class:`BufferedWriter` object is closed or destroyed. 615 616 The constructor creates a :class:`BufferedWriter` for the given writeable 617 *raw* stream. If the *buffer_size* is not given, it defaults to 618 :data:`DEFAULT_BUFFER_SIZE`. 619 620 A third argument, *max_buffer_size*, is supported, but unused and deprecated. 621 622 :class:`BufferedWriter` provides or overrides these methods in addition to 623 those from :class:`BufferedIOBase` and :class:`IOBase`: 624 625 .. method:: flush() 626 627 Force bytes held in the buffer into the raw stream. A 628 :exc:`BlockingIOError` should be raised if the raw stream blocks. 629 630 .. method:: write(b) 631 632 Write *b*, and return the number of bytes written. 633 The object *b* should be an array of bytes, either 634 :class:`bytes`, :class:`bytearray`, or :class:`memoryview`. 635 When in non-blocking mode, a :exc:`BlockingIOError` is raised 636 if the buffer needs to be written out but the raw stream blocks. 637 638 639.. class:: BufferedRandom(raw, buffer_size=DEFAULT_BUFFER_SIZE) 640 641 A buffered interface to random access streams. It inherits 642 :class:`BufferedReader` and :class:`BufferedWriter`, and further supports 643 :meth:`seek` and :meth:`tell` functionality. 644 645 The constructor creates a reader and writer for a seekable raw stream, given 646 in the first argument. If the *buffer_size* is omitted it defaults to 647 :data:`DEFAULT_BUFFER_SIZE`. 648 649 A third argument, *max_buffer_size*, is supported, but unused and deprecated. 650 651 :class:`BufferedRandom` is capable of anything :class:`BufferedReader` or 652 :class:`BufferedWriter` can do. 653 654 655.. class:: BufferedRWPair(reader, writer, buffer_size=DEFAULT_BUFFER_SIZE) 656 657 A buffered I/O object combining two unidirectional :class:`RawIOBase` 658 objects -- one readable, the other writeable -- into a single bidirectional 659 endpoint. It inherits :class:`BufferedIOBase`. 660 661 *reader* and *writer* are :class:`RawIOBase` objects that are readable and 662 writeable respectively. If the *buffer_size* is omitted it defaults to 663 :data:`DEFAULT_BUFFER_SIZE`. 664 665 A fourth argument, *max_buffer_size*, is supported, but unused and 666 deprecated. 667 668 :class:`BufferedRWPair` implements all of :class:`BufferedIOBase`\'s methods 669 except for :meth:`~BufferedIOBase.detach`, which raises 670 :exc:`UnsupportedOperation`. 671 672 .. warning:: 673 674 :class:`BufferedRWPair` does not attempt to synchronize accesses to 675 its underlying raw streams. You should not pass it the same object 676 as reader and writer; use :class:`BufferedRandom` instead. 677 678 679Text I/O 680-------- 681 682.. class:: TextIOBase 683 684 Base class for text streams. This class provides a unicode character 685 and line based interface to stream I/O. There is no :meth:`readinto` 686 method because Python's :class:`unicode` strings are immutable. 687 It inherits :class:`IOBase`. There is no public constructor. 688 689 :class:`TextIOBase` provides or overrides these data attributes and 690 methods in addition to those from :class:`IOBase`: 691 692 .. attribute:: encoding 693 694 The name of the encoding used to decode the stream's bytes into 695 strings, and to encode strings into bytes. 696 697 .. attribute:: errors 698 699 The error setting of the decoder or encoder. 700 701 .. attribute:: newlines 702 703 A string, a tuple of strings, or ``None``, indicating the newlines 704 translated so far. Depending on the implementation and the initial 705 constructor flags, this may not be available. 706 707 .. attribute:: buffer 708 709 The underlying binary buffer (a :class:`BufferedIOBase` instance) that 710 :class:`TextIOBase` deals with. This is not part of the 711 :class:`TextIOBase` API and may not exist on some implementations. 712 713 .. method:: detach() 714 715 Separate the underlying binary buffer from the :class:`TextIOBase` and 716 return it. 717 718 After the underlying buffer has been detached, the :class:`TextIOBase` is 719 in an unusable state. 720 721 Some :class:`TextIOBase` implementations, like :class:`~io.StringIO`, may not 722 have the concept of an underlying buffer and calling this method will 723 raise :exc:`UnsupportedOperation`. 724 725 .. versionadded:: 2.7 726 727 .. method:: read(n=-1) 728 729 Read and return at most *n* characters from the stream as a single 730 :class:`unicode`. If *n* is negative or ``None``, reads until EOF. 731 732 .. method:: readline(limit=-1) 733 734 Read until newline or EOF and return a single ``unicode``. If the 735 stream is already at EOF, an empty string is returned. 736 737 If *limit* is specified, at most *limit* characters will be read. 738 739 .. method:: seek(offset[, whence]) 740 741 Change the stream position to the given *offset*. Behaviour depends on 742 the *whence* parameter. The default value for *whence* is 743 :data:`SEEK_SET`. 744 745 * :data:`SEEK_SET` or ``0``: seek from the start of the stream 746 (the default); *offset* must either be a number returned by 747 :meth:`TextIOBase.tell`, or zero. Any other *offset* value 748 produces undefined behaviour. 749 * :data:`SEEK_CUR` or ``1``: "seek" to the current position; 750 *offset* must be zero, which is a no-operation (all other values 751 are unsupported). 752 * :data:`SEEK_END` or ``2``: seek to the end of the stream; 753 *offset* must be zero (all other values are unsupported). 754 755 Return the new absolute position as an opaque number. 756 757 .. versionadded:: 2.7 758 The ``SEEK_*`` constants. 759 760 .. method:: tell() 761 762 Return the current stream position as an opaque number. The number 763 does not usually represent a number of bytes in the underlying 764 binary storage. 765 766 .. method:: write(s) 767 768 Write the :class:`unicode` string *s* to the stream and return the 769 number of characters written. 770 771 772.. class:: TextIOWrapper(buffer, encoding=None, errors=None, newline=None, line_buffering=False) 773 774 A buffered text stream over a :class:`BufferedIOBase` binary stream. 775 It inherits :class:`TextIOBase`. 776 777 *encoding* gives the name of the encoding that the stream will be decoded or 778 encoded with. It defaults to :func:`locale.getpreferredencoding`. 779 780 *errors* is an optional string that specifies how encoding and decoding 781 errors are to be handled. Pass ``'strict'`` to raise a :exc:`ValueError` 782 exception if there is an encoding error (the default of ``None`` has the same 783 effect), or pass ``'ignore'`` to ignore errors. (Note that ignoring encoding 784 errors can lead to data loss.) ``'replace'`` causes a replacement marker 785 (such as ``'?'``) to be inserted where there is malformed data. When 786 writing, ``'xmlcharrefreplace'`` (replace with the appropriate XML character 787 reference) or ``'backslashreplace'`` (replace with backslashed escape 788 sequences) can be used. Any other error handling name that has been 789 registered with :func:`codecs.register_error` is also valid. 790 791 .. index:: 792 single: universal newlines; io.TextIOWrapper class 793 794 *newline* controls how line endings are handled. It can be ``None``, 795 ``''``, ``'\n'``, ``'\r'``, and ``'\r\n'``. It works as follows: 796 797 * On input, if *newline* is ``None``, :term:`universal newlines` mode is 798 enabled. Lines in the input can end in ``'\n'``, ``'\r'``, or ``'\r\n'``, 799 and these are translated into ``'\n'`` before being returned to the 800 caller. If it is ``''``, universal newlines mode is enabled, but line 801 endings are returned to the caller untranslated. If it has any of the 802 other legal values, input lines are only terminated by the given string, 803 and the line ending is returned to the caller untranslated. 804 805 * On output, if *newline* is ``None``, any ``'\n'`` characters written are 806 translated to the system default line separator, :data:`os.linesep`. If 807 *newline* is ``''``, no translation takes place. If *newline* is any of 808 the other legal values, any ``'\n'`` characters written are translated to 809 the given string. 810 811 If *line_buffering* is ``True``, :meth:`flush` is implied when a call to 812 write contains a newline character or a carriage return. 813 814 :class:`TextIOWrapper` provides one attribute in addition to those of 815 :class:`TextIOBase` and its parents: 816 817 .. attribute:: line_buffering 818 819 Whether line buffering is enabled. 820 821 822.. class:: StringIO(initial_value=u'', newline=u'\\n') 823 824 An in-memory stream for unicode text. It inherits :class:`TextIOWrapper`. 825 826 The initial value of the buffer can be set by providing *initial_value*. 827 If newline translation is enabled, newlines will be encoded as if by 828 :meth:`~TextIOBase.write`. The stream is positioned at the start of 829 the buffer. 830 831 The *newline* argument works like that of :class:`TextIOWrapper`. 832 The default is to consider only ``\n`` characters as ends of lines and 833 to do no newline translation. If *newline* is set to ``None``, 834 newlines are written as ``\n`` on all platforms, but universal 835 newline decoding is still performed when reading. 836 837 :class:`~io.StringIO` provides this method in addition to those from 838 :class:`TextIOWrapper` and its parents: 839 840 .. method:: getvalue() 841 842 Return a ``unicode`` containing the entire contents of the buffer at any 843 time before the :class:`~io.StringIO` object's :meth:`close` method is 844 called. Newlines are decoded as if by :meth:`~TextIOBase.read`, 845 although the stream position is not changed. 846 847 Example usage:: 848 849 import io 850 851 output = io.StringIO() 852 output.write(u'First line.\n') 853 output.write(u'Second line.\n') 854 855 # Retrieve file contents -- this will be 856 # u'First line.\nSecond line.\n' 857 contents = output.getvalue() 858 859 # Close object and discard memory buffer -- 860 # .getvalue() will now raise an exception. 861 output.close() 862 863 864.. index:: 865 single: universal newlines; io.IncrementalNewlineDecoder class 866 867.. class:: IncrementalNewlineDecoder 868 869 A helper codec that decodes newlines for :term:`universal newlines` mode. 870 It inherits :class:`codecs.IncrementalDecoder`. 871 872 873Advanced topics 874--------------- 875 876Here we will discuss several advanced topics pertaining to the concrete 877I/O implementations described above. 878 879Performance 880^^^^^^^^^^^ 881 882Binary I/O 883"""""""""" 884 885By reading and writing only large chunks of data even when the user asks 886for a single byte, buffered I/O is designed to hide any inefficiency in 887calling and executing the operating system's unbuffered I/O routines. The 888gain will vary very much depending on the OS and the kind of I/O which is 889performed (for example, on some contemporary OSes such as Linux, unbuffered 890disk I/O can be as fast as buffered I/O). The bottom line, however, is 891that buffered I/O will offer you predictable performance regardless of the 892platform and the backing device. Therefore, it is most always preferable to 893use buffered I/O rather than unbuffered I/O. 894 895Text I/O 896"""""""" 897 898Text I/O over a binary storage (such as a file) is significantly slower than 899binary I/O over the same storage, because it implies conversions from 900unicode to binary data using a character codec. This can become noticeable 901if you handle huge amounts of text data (for example very large log files). 902Also, :meth:`TextIOWrapper.tell` and :meth:`TextIOWrapper.seek` are both 903quite slow due to the reconstruction algorithm used. 904 905:class:`~io.StringIO`, however, is a native in-memory unicode container and will 906exhibit similar speed to :class:`BytesIO`. 907 908Multi-threading 909^^^^^^^^^^^^^^^ 910 911:class:`FileIO` objects are thread-safe to the extent that the operating 912system calls (such as ``read(2)`` under Unix) they are wrapping are thread-safe 913too. 914 915Binary buffered objects (instances of :class:`BufferedReader`, 916:class:`BufferedWriter`, :class:`BufferedRandom` and :class:`BufferedRWPair`) 917protect their internal structures using a lock; it is therefore safe to call 918them from multiple threads at once. 919 920:class:`TextIOWrapper` objects are not thread-safe. 921 922Reentrancy 923^^^^^^^^^^ 924 925Binary buffered objects (instances of :class:`BufferedReader`, 926:class:`BufferedWriter`, :class:`BufferedRandom` and :class:`BufferedRWPair`) 927are not reentrant. While reentrant calls will not happen in normal situations, 928they can arise if you are doing I/O in a :mod:`signal` handler. If it is 929attempted to enter a buffered object again while already being accessed 930*from the same thread*, then a :exc:`RuntimeError` is raised. 931 932The above implicitly extends to text files, since the :func:`open()` 933function will wrap a buffered object inside a :class:`TextIOWrapper`. This 934includes standard streams and therefore affects the built-in function 935:func:`print()` as well. 936 937