1.. _tut-io: 2 3**************** 4Input and Output 5**************** 6 7There are several ways to present the output of a program; data can be printed 8in a human-readable form, or written to a file for future use. This chapter will 9discuss some of the possibilities. 10 11 12.. _tut-formatting: 13 14Fancier Output Formatting 15========================= 16 17So far we've encountered two ways of writing values: *expression statements* and 18the :func:`print` function. (A third way is using the :meth:`~io.TextIOBase.write` method 19of file objects; the standard output file can be referenced as ``sys.stdout``. 20See the Library Reference for more information on this.) 21 22Often you'll want more control over the formatting of your output than simply 23printing space-separated values. There are several ways to format output. 24 25* To use :ref:`formatted string literals <tut-f-strings>`, begin a string 26 with ``f`` or ``F`` before the opening quotation mark or triple quotation mark. 27 Inside this string, you can write a Python expression between ``{`` and ``}`` 28 characters that can refer to variables or literal values. 29 30 :: 31 32 >>> year = 2016 33 >>> event = 'Referendum' 34 >>> f'Results of the {year} {event}' 35 'Results of the 2016 Referendum' 36 37* The :meth:`str.format` method of strings requires more manual 38 effort. You'll still use ``{`` and ``}`` to mark where a variable 39 will be substituted and can provide detailed formatting directives, 40 but you'll also need to provide the information to be formatted. In the following code 41 block there are two examples of how to format variables: 42 43 44 :: 45 46 >>> yes_votes = 42_572_654 47 >>> total_votes = 85_705_149 48 >>> percentage = yes_votes / total_votes 49 >>> '{:-9} YES votes {:2.2%}'.format(yes_votes, percentage) 50 ' 42572654 YES votes 49.67%' 51 52 Notice how the ``yes_votes`` are padded with spaces and a negative sign only for negative numbers. 53 The example also prints ``percentage`` multiplied by 100, with 2 decimal 54 places and followed by a percent sign (see :ref:`formatspec` for details). 55 56 57* Finally, you can do all the string handling yourself by using string slicing and 58 concatenation operations to create any layout you can imagine. The 59 string type has some methods that perform useful operations for padding 60 strings to a given column width. 61 62When you don't need fancy output but just want a quick display of some 63variables for debugging purposes, you can convert any value to a string with 64the :func:`repr` or :func:`str` functions. 65 66The :func:`str` function is meant to return representations of values which are 67fairly human-readable, while :func:`repr` is meant to generate representations 68which can be read by the interpreter (or will force a :exc:`SyntaxError` if 69there is no equivalent syntax). For objects which don't have a particular 70representation for human consumption, :func:`str` will return the same value as 71:func:`repr`. Many values, such as numbers or structures like lists and 72dictionaries, have the same representation using either function. Strings, in 73particular, have two distinct representations. 74 75Some examples:: 76 77 >>> s = 'Hello, world.' 78 >>> str(s) 79 'Hello, world.' 80 >>> repr(s) 81 "'Hello, world.'" 82 >>> str(1/7) 83 '0.14285714285714285' 84 >>> x = 10 * 3.25 85 >>> y = 200 * 200 86 >>> s = 'The value of x is ' + repr(x) + ', and y is ' + repr(y) + '...' 87 >>> print(s) 88 The value of x is 32.5, and y is 40000... 89 >>> # The repr() of a string adds string quotes and backslashes: 90 >>> hello = 'hello, world\n' 91 >>> hellos = repr(hello) 92 >>> print(hellos) 93 'hello, world\n' 94 >>> # The argument to repr() may be any Python object: 95 >>> repr((x, y, ('spam', 'eggs'))) 96 "(32.5, 40000, ('spam', 'eggs'))" 97 98The :mod:`string` module contains a :class:`~string.Template` class that offers 99yet another way to substitute values into strings, using placeholders like 100``$x`` and replacing them with values from a dictionary, but offers much less 101control of the formatting. 102 103.. index:: 104 single: formatted string literal 105 single: interpolated string literal 106 single: string; formatted literal 107 single: string; interpolated literal 108 single: f-string 109 single: fstring 110 111.. _tut-f-strings: 112 113Formatted String Literals 114------------------------- 115 116:ref:`Formatted string literals <f-strings>` (also called f-strings for 117short) let you include the value of Python expressions inside a string by 118prefixing the string with ``f`` or ``F`` and writing expressions as 119``{expression}``. 120 121An optional format specifier can follow the expression. This allows greater 122control over how the value is formatted. The following example rounds pi to 123three places after the decimal:: 124 125 >>> import math 126 >>> print(f'The value of pi is approximately {math.pi:.3f}.') 127 The value of pi is approximately 3.142. 128 129Passing an integer after the ``':'`` will cause that field to be a minimum 130number of characters wide. This is useful for making columns line up. :: 131 132 >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 7678} 133 >>> for name, phone in table.items(): 134 ... print(f'{name:10} ==> {phone:10d}') 135 ... 136 Sjoerd ==> 4127 137 Jack ==> 4098 138 Dcab ==> 7678 139 140Other modifiers can be used to convert the value before it is formatted. 141``'!a'`` applies :func:`ascii`, ``'!s'`` applies :func:`str`, and ``'!r'`` 142applies :func:`repr`:: 143 144 >>> animals = 'eels' 145 >>> print(f'My hovercraft is full of {animals}.') 146 My hovercraft is full of eels. 147 >>> print(f'My hovercraft is full of {animals!r}.') 148 My hovercraft is full of 'eels'. 149 150The ``=`` specifier can be used to expand an expression to the text of the 151expression, an equal sign, then the representation of the evaluated expression: 152 153 >>> bugs = 'roaches' 154 >>> count = 13 155 >>> area = 'living room' 156 >>> print(f'Debugging {bugs=} {count=} {area=}') 157 Debugging bugs='roaches' count=13 area='living room' 158 159See :ref:`self-documenting expressions <bpo-36817-whatsnew>` for more information 160on the ``=`` specifier. For a reference on these format specifications, see 161the reference guide for the :ref:`formatspec`. 162 163.. _tut-string-format: 164 165The String format() Method 166-------------------------- 167 168Basic usage of the :meth:`str.format` method looks like this:: 169 170 >>> print('We are the {} who say "{}!"'.format('knights', 'Ni')) 171 We are the knights who say "Ni!" 172 173The brackets and characters within them (called format fields) are replaced with 174the objects passed into the :meth:`str.format` method. A number in the 175brackets can be used to refer to the position of the object passed into the 176:meth:`str.format` method. :: 177 178 >>> print('{0} and {1}'.format('spam', 'eggs')) 179 spam and eggs 180 >>> print('{1} and {0}'.format('spam', 'eggs')) 181 eggs and spam 182 183If keyword arguments are used in the :meth:`str.format` method, their values 184are referred to by using the name of the argument. :: 185 186 >>> print('This {food} is {adjective}.'.format( 187 ... food='spam', adjective='absolutely horrible')) 188 This spam is absolutely horrible. 189 190Positional and keyword arguments can be arbitrarily combined:: 191 192 >>> print('The story of {0}, {1}, and {other}.'.format('Bill', 'Manfred', 193 ... other='Georg')) 194 The story of Bill, Manfred, and Georg. 195 196If you have a really long format string that you don't want to split up, it 197would be nice if you could reference the variables to be formatted by name 198instead of by position. This can be done by simply passing the dict and using 199square brackets ``'[]'`` to access the keys. :: 200 201 >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 8637678} 202 >>> print('Jack: {0[Jack]:d}; Sjoerd: {0[Sjoerd]:d}; ' 203 ... 'Dcab: {0[Dcab]:d}'.format(table)) 204 Jack: 4098; Sjoerd: 4127; Dcab: 8637678 205 206This could also be done by passing the ``table`` dictionary as keyword arguments with the ``**`` 207notation. :: 208 209 >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 8637678} 210 >>> print('Jack: {Jack:d}; Sjoerd: {Sjoerd:d}; Dcab: {Dcab:d}'.format(**table)) 211 Jack: 4098; Sjoerd: 4127; Dcab: 8637678 212 213This is particularly useful in combination with the built-in function 214:func:`vars`, which returns a dictionary containing all local variables:: 215 216 >>> table = {k: str(v) for k, v in vars().items()} 217 >>> message = " ".join([f'{k}: ' + '{' + k +'};' for k in table.keys()]) 218 >>> print(message.format(**table)) 219 __name__: __main__; __doc__: None; __package__: None; __loader__: ... 220 221As an example, the following lines produce a tidily aligned 222set of columns giving integers and their squares and cubes:: 223 224 >>> for x in range(1, 11): 225 ... print('{0:2d} {1:3d} {2:4d}'.format(x, x*x, x*x*x)) 226 ... 227 1 1 1 228 2 4 8 229 3 9 27 230 4 16 64 231 5 25 125 232 6 36 216 233 7 49 343 234 8 64 512 235 9 81 729 236 10 100 1000 237 238For a complete overview of string formatting with :meth:`str.format`, see 239:ref:`formatstrings`. 240 241 242Manual String Formatting 243------------------------ 244 245Here's the same table of squares and cubes, formatted manually:: 246 247 >>> for x in range(1, 11): 248 ... print(repr(x).rjust(2), repr(x*x).rjust(3), end=' ') 249 ... # Note use of 'end' on previous line 250 ... print(repr(x*x*x).rjust(4)) 251 ... 252 1 1 1 253 2 4 8 254 3 9 27 255 4 16 64 256 5 25 125 257 6 36 216 258 7 49 343 259 8 64 512 260 9 81 729 261 10 100 1000 262 263(Note that the one space between each column was added by the 264way :func:`print` works: it always adds spaces between its arguments.) 265 266The :meth:`str.rjust` method of string objects right-justifies a string in a 267field of a given width by padding it with spaces on the left. There are 268similar methods :meth:`str.ljust` and :meth:`str.center`. These methods do 269not write anything, they just return a new string. If the input string is too 270long, they don't truncate it, but return it unchanged; this will mess up your 271column lay-out but that's usually better than the alternative, which would be 272lying about a value. (If you really want truncation you can always add a 273slice operation, as in ``x.ljust(n)[:n]``.) 274 275There is another method, :meth:`str.zfill`, which pads a numeric string on the 276left with zeros. It understands about plus and minus signs:: 277 278 >>> '12'.zfill(5) 279 '00012' 280 >>> '-3.14'.zfill(7) 281 '-003.14' 282 >>> '3.14159265359'.zfill(5) 283 '3.14159265359' 284 285 286Old string formatting 287--------------------- 288 289The % operator (modulo) can also be used for string formatting. 290Given ``format % values`` (where *format* is a string), 291``%`` conversion specifications in *format* are replaced with 292zero or more elements of *values*. 293This operation is commonly known as string 294interpolation. For example:: 295 296 >>> import math 297 >>> print('The value of pi is approximately %5.3f.' % math.pi) 298 The value of pi is approximately 3.142. 299 300More information can be found in the :ref:`old-string-formatting` section. 301 302 303.. _tut-files: 304 305Reading and Writing Files 306========================= 307 308.. index:: 309 pair: built-in function; open 310 pair: object; file 311 312:func:`open` returns a :term:`file object`, and is most commonly used with 313two positional arguments and one keyword argument: 314``open(filename, mode, encoding=None)`` 315 316:: 317 318 >>> f = open('workfile', 'w', encoding="utf-8") 319 320.. XXX str(f) is <io.TextIOWrapper object at 0x82e8dc4> 321 322 >>> print(f) 323 <open file 'workfile', mode 'w' at 80a0960> 324 325The first argument is a string containing the filename. The second argument is 326another string containing a few characters describing the way in which the file 327will be used. *mode* can be ``'r'`` when the file will only be read, ``'w'`` 328for only writing (an existing file with the same name will be erased), and 329``'a'`` opens the file for appending; any data written to the file is 330automatically added to the end. ``'r+'`` opens the file for both reading and 331writing. The *mode* argument is optional; ``'r'`` will be assumed if it's 332omitted. 333 334Normally, files are opened in :dfn:`text mode`, that means, you read and write 335strings from and to the file, which are encoded in a specific *encoding*. 336If *encoding* is not specified, the default is platform dependent 337(see :func:`open`). 338Because UTF-8 is the modern de-facto standard, ``encoding="utf-8"`` is 339recommended unless you know that you need to use a different encoding. 340Appending a ``'b'`` to the mode opens the file in :dfn:`binary mode`. 341Binary mode data is read and written as :class:`bytes` objects. 342You can not specify *encoding* when opening file in binary mode. 343 344In text mode, the default when reading is to convert platform-specific line 345endings (``\n`` on Unix, ``\r\n`` on Windows) to just ``\n``. When writing in 346text mode, the default is to convert occurrences of ``\n`` back to 347platform-specific line endings. This behind-the-scenes modification 348to file data is fine for text files, but will corrupt binary data like that in 349:file:`JPEG` or :file:`EXE` files. Be very careful to use binary mode when 350reading and writing such files. 351 352It is good practice to use the :keyword:`with` keyword when dealing 353with file objects. The advantage is that the file is properly closed 354after its suite finishes, even if an exception is raised at some 355point. Using :keyword:`!with` is also much shorter than writing 356equivalent :keyword:`try`\ -\ :keyword:`finally` blocks:: 357 358 >>> with open('workfile', encoding="utf-8") as f: 359 ... read_data = f.read() 360 361 >>> # We can check that the file has been automatically closed. 362 >>> f.closed 363 True 364 365If you're not using the :keyword:`with` keyword, then you should call 366``f.close()`` to close the file and immediately free up any system 367resources used by it. 368 369.. warning:: 370 Calling ``f.write()`` without using the :keyword:`!with` keyword or calling 371 ``f.close()`` **might** result in the arguments 372 of ``f.write()`` not being completely written to the disk, even if the 373 program exits successfully. 374 375.. 376 See also https://bugs.python.org/issue17852 377 378After a file object is closed, either by a :keyword:`with` statement 379or by calling ``f.close()``, attempts to use the file object will 380automatically fail. :: 381 382 >>> f.close() 383 >>> f.read() 384 Traceback (most recent call last): 385 File "<stdin>", line 1, in <module> 386 ValueError: I/O operation on closed file. 387 388 389.. _tut-filemethods: 390 391Methods of File Objects 392----------------------- 393 394The rest of the examples in this section will assume that a file object called 395``f`` has already been created. 396 397To read a file's contents, call ``f.read(size)``, which reads some quantity of 398data and returns it as a string (in text mode) or bytes object (in binary mode). 399*size* is an optional numeric argument. When *size* is omitted or negative, the 400entire contents of the file will be read and returned; it's your problem if the 401file is twice as large as your machine's memory. Otherwise, at most *size* 402characters (in text mode) or *size* bytes (in binary mode) are read and returned. 403If the end of the file has been reached, ``f.read()`` will return an empty 404string (``''``). :: 405 406 >>> f.read() 407 'This is the entire file.\n' 408 >>> f.read() 409 '' 410 411``f.readline()`` reads a single line from the file; a newline character (``\n``) 412is left at the end of the string, and is only omitted on the last line of the 413file if the file doesn't end in a newline. This makes the return value 414unambiguous; if ``f.readline()`` returns an empty string, the end of the file 415has been reached, while a blank line is represented by ``'\n'``, a string 416containing only a single newline. :: 417 418 >>> f.readline() 419 'This is the first line of the file.\n' 420 >>> f.readline() 421 'Second line of the file\n' 422 >>> f.readline() 423 '' 424 425For reading lines from a file, you can loop over the file object. This is memory 426efficient, fast, and leads to simple code:: 427 428 >>> for line in f: 429 ... print(line, end='') 430 ... 431 This is the first line of the file. 432 Second line of the file 433 434If you want to read all the lines of a file in a list you can also use 435``list(f)`` or ``f.readlines()``. 436 437``f.write(string)`` writes the contents of *string* to the file, returning 438the number of characters written. :: 439 440 >>> f.write('This is a test\n') 441 15 442 443Other types of objects need to be converted -- either to a string (in text mode) 444or a bytes object (in binary mode) -- before writing them:: 445 446 >>> value = ('the answer', 42) 447 >>> s = str(value) # convert the tuple to string 448 >>> f.write(s) 449 18 450 451``f.tell()`` returns an integer giving the file object's current position in the file 452represented as number of bytes from the beginning of the file when in binary mode and 453an opaque number when in text mode. 454 455To change the file object's position, use ``f.seek(offset, whence)``. The position is computed 456from adding *offset* to a reference point; the reference point is selected by 457the *whence* argument. A *whence* value of 0 measures from the beginning 458of the file, 1 uses the current file position, and 2 uses the end of the file as 459the reference point. *whence* can be omitted and defaults to 0, using the 460beginning of the file as the reference point. :: 461 462 >>> f = open('workfile', 'rb+') 463 >>> f.write(b'0123456789abcdef') 464 16 465 >>> f.seek(5) # Go to the 6th byte in the file 466 5 467 >>> f.read(1) 468 b'5' 469 >>> f.seek(-3, 2) # Go to the 3rd byte before the end 470 13 471 >>> f.read(1) 472 b'd' 473 474In text files (those opened without a ``b`` in the mode string), only seeks 475relative to the beginning of the file are allowed (the exception being seeking 476to the very file end with ``seek(0, 2)``) and the only valid *offset* values are 477those returned from the ``f.tell()``, or zero. Any other *offset* value produces 478undefined behaviour. 479 480File objects have some additional methods, such as :meth:`~io.IOBase.isatty` and 481:meth:`~io.IOBase.truncate` which are less frequently used; consult the Library 482Reference for a complete guide to file objects. 483 484 485.. _tut-json: 486 487Saving structured data with :mod:`json` 488--------------------------------------- 489 490.. index:: pair: module; json 491 492Strings can easily be written to and read from a file. Numbers take a bit more 493effort, since the :meth:`~io.TextIOBase.read` method only returns strings, which will have to 494be passed to a function like :func:`int`, which takes a string like ``'123'`` 495and returns its numeric value 123. When you want to save more complex data 496types like nested lists and dictionaries, parsing and serializing by hand 497becomes complicated. 498 499Rather than having users constantly writing and debugging code to save 500complicated data types to files, Python allows you to use the popular data 501interchange format called `JSON (JavaScript Object Notation) 502<https://json.org>`_. The standard module called :mod:`json` can take Python 503data hierarchies, and convert them to string representations; this process is 504called :dfn:`serializing`. Reconstructing the data from the string representation 505is called :dfn:`deserializing`. Between serializing and deserializing, the 506string representing the object may have been stored in a file or data, or 507sent over a network connection to some distant machine. 508 509.. note:: 510 The JSON format is commonly used by modern applications to allow for data 511 exchange. Many programmers are already familiar with it, which makes 512 it a good choice for interoperability. 513 514If you have an object ``x``, you can view its JSON string representation with a 515simple line of code:: 516 517 >>> import json 518 >>> x = [1, 'simple', 'list'] 519 >>> json.dumps(x) 520 '[1, "simple", "list"]' 521 522Another variant of the :func:`~json.dumps` function, called :func:`~json.dump`, 523simply serializes the object to a :term:`text file`. So if ``f`` is a 524:term:`text file` object opened for writing, we can do this:: 525 526 json.dump(x, f) 527 528To decode the object again, if ``f`` is a :term:`binary file` or 529:term:`text file` object which has been opened for reading:: 530 531 x = json.load(f) 532 533.. note:: 534 JSON files must be encoded in UTF-8. Use ``encoding="utf-8"`` when opening 535 JSON file as a :term:`text file` for both of reading and writing. 536 537This simple serialization technique can handle lists and dictionaries, but 538serializing arbitrary class instances in JSON requires a bit of extra effort. 539The reference for the :mod:`json` module contains an explanation of this. 540 541.. seealso:: 542 543 :mod:`pickle` - the pickle module 544 545 Contrary to :ref:`JSON <tut-json>`, *pickle* is a protocol which allows 546 the serialization of arbitrarily complex Python objects. As such, it is 547 specific to Python and cannot be used to communicate with applications 548 written in other languages. It is also insecure by default: 549 deserializing pickle data coming from an untrusted source can execute 550 arbitrary code, if the data was crafted by a skilled attacker. 551