1.. _tut-io: 2 3**************** 4Input and Output 5**************** 6 7There are several ways to present the output of a program; data can be printed 8in a human-readable form, or written to a file for future use. This chapter will 9discuss some of the possibilities. 10 11 12.. _tut-formatting: 13 14Fancier Output Formatting 15========================= 16 17So far we've encountered two ways of writing values: *expression statements* and 18the :func:`print` function. (A third way is using the :meth:`write` method 19of file objects; the standard output file can be referenced as ``sys.stdout``. 20See the Library Reference for more information on this.) 21 22Often you'll want more control over the formatting of your output than simply 23printing space-separated values. There are several ways to format output. 24 25* To use :ref:`formatted string literals <tut-f-strings>`, begin a string 26 with ``f`` or ``F`` before the opening quotation mark or triple quotation mark. 27 Inside this string, you can write a Python expression between ``{`` and ``}`` 28 characters that can refer to variables or literal values. 29 30 :: 31 32 >>> year = 2016 33 >>> event = 'Referendum' 34 >>> f'Results of the {year} {event}' 35 'Results of the 2016 Referendum' 36 37* The :meth:`str.format` method of strings requires more manual 38 effort. You'll still use ``{`` and ``}`` to mark where a variable 39 will be substituted and can provide detailed formatting directives, 40 but you'll also need to provide the information to be formatted. 41 42 :: 43 44 >>> yes_votes = 42_572_654 45 >>> no_votes = 43_132_495 46 >>> percentage = yes_votes / (yes_votes + no_votes) 47 >>> '{:-9} YES votes {:2.2%}'.format(yes_votes, percentage) 48 ' 42572654 YES votes 49.67%' 49 50* Finally, you can do all the string handling yourself by using string slicing and 51 concatenation operations to create any layout you can imagine. The 52 string type has some methods that perform useful operations for padding 53 strings to a given column width. 54 55When you don't need fancy output but just want a quick display of some 56variables for debugging purposes, you can convert any value to a string with 57the :func:`repr` or :func:`str` functions. 58 59The :func:`str` function is meant to return representations of values which are 60fairly human-readable, while :func:`repr` is meant to generate representations 61which can be read by the interpreter (or will force a :exc:`SyntaxError` if 62there is no equivalent syntax). For objects which don't have a particular 63representation for human consumption, :func:`str` will return the same value as 64:func:`repr`. Many values, such as numbers or structures like lists and 65dictionaries, have the same representation using either function. Strings, in 66particular, have two distinct representations. 67 68Some examples:: 69 70 >>> s = 'Hello, world.' 71 >>> str(s) 72 'Hello, world.' 73 >>> repr(s) 74 "'Hello, world.'" 75 >>> str(1/7) 76 '0.14285714285714285' 77 >>> x = 10 * 3.25 78 >>> y = 200 * 200 79 >>> s = 'The value of x is ' + repr(x) + ', and y is ' + repr(y) + '...' 80 >>> print(s) 81 The value of x is 32.5, and y is 40000... 82 >>> # The repr() of a string adds string quotes and backslashes: 83 ... hello = 'hello, world\n' 84 >>> hellos = repr(hello) 85 >>> print(hellos) 86 'hello, world\n' 87 >>> # The argument to repr() may be any Python object: 88 ... repr((x, y, ('spam', 'eggs'))) 89 "(32.5, 40000, ('spam', 'eggs'))" 90 91The :mod:`string` module contains a :class:`~string.Template` class that offers 92yet another way to substitute values into strings, using placeholders like 93``$x`` and replacing them with values from a dictionary, but offers much less 94control of the formatting. 95 96 97.. _tut-f-strings: 98 99Formatted String Literals 100------------------------- 101 102:ref:`Formatted string literals <f-strings>` (also called f-strings for 103short) let you include the value of Python expressions inside a string by 104prefixing the string with ``f`` or ``F`` and writing expressions as 105``{expression}``. 106 107An optional format specifier can follow the expression. This allows greater 108control over how the value is formatted. The following example rounds pi to 109three places after the decimal:: 110 111 >>> import math 112 >>> print(f'The value of pi is approximately {math.pi:.3f}.') 113 The value of pi is approximately 3.142. 114 115Passing an integer after the ``':'`` will cause that field to be a minimum 116number of characters wide. This is useful for making columns line up. :: 117 118 >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 7678} 119 >>> for name, phone in table.items(): 120 ... print(f'{name:10} ==> {phone:10d}') 121 ... 122 Sjoerd ==> 4127 123 Jack ==> 4098 124 Dcab ==> 7678 125 126Other modifiers can be used to convert the value before it is formatted. 127``'!a'`` applies :func:`ascii`, ``'!s'`` applies :func:`str`, and ``'!r'`` 128applies :func:`repr`:: 129 130 >>> animals = 'eels' 131 >>> print(f'My hovercraft is full of {animals}.') 132 My hovercraft is full of eels. 133 >>> print(f'My hovercraft is full of {animals!r}.') 134 My hovercraft is full of 'eels'. 135 136For a reference on these format specifications, see 137the reference guide for the :ref:`formatspec`. 138 139.. _tut-string-format: 140 141The String format() Method 142-------------------------- 143 144Basic usage of the :meth:`str.format` method looks like this:: 145 146 >>> print('We are the {} who say "{}!"'.format('knights', 'Ni')) 147 We are the knights who say "Ni!" 148 149The brackets and characters within them (called format fields) are replaced with 150the objects passed into the :meth:`str.format` method. A number in the 151brackets can be used to refer to the position of the object passed into the 152:meth:`str.format` method. :: 153 154 >>> print('{0} and {1}'.format('spam', 'eggs')) 155 spam and eggs 156 >>> print('{1} and {0}'.format('spam', 'eggs')) 157 eggs and spam 158 159If keyword arguments are used in the :meth:`str.format` method, their values 160are referred to by using the name of the argument. :: 161 162 >>> print('This {food} is {adjective}.'.format( 163 ... food='spam', adjective='absolutely horrible')) 164 This spam is absolutely horrible. 165 166Positional and keyword arguments can be arbitrarily combined:: 167 168 >>> print('The story of {0}, {1}, and {other}.'.format('Bill', 'Manfred', 169 other='Georg')) 170 The story of Bill, Manfred, and Georg. 171 172If you have a really long format string that you don't want to split up, it 173would be nice if you could reference the variables to be formatted by name 174instead of by position. This can be done by simply passing the dict and using 175square brackets ``'[]'`` to access the keys. :: 176 177 >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 8637678} 178 >>> print('Jack: {0[Jack]:d}; Sjoerd: {0[Sjoerd]:d}; ' 179 ... 'Dcab: {0[Dcab]:d}'.format(table)) 180 Jack: 4098; Sjoerd: 4127; Dcab: 8637678 181 182This could also be done by passing the table as keyword arguments with the '**' 183notation. :: 184 185 >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 8637678} 186 >>> print('Jack: {Jack:d}; Sjoerd: {Sjoerd:d}; Dcab: {Dcab:d}'.format(**table)) 187 Jack: 4098; Sjoerd: 4127; Dcab: 8637678 188 189This is particularly useful in combination with the built-in function 190:func:`vars`, which returns a dictionary containing all local variables. 191 192As an example, the following lines produce a tidily-aligned 193set of columns giving integers and their squares and cubes:: 194 195 >>> for x in range(1, 11): 196 ... print('{0:2d} {1:3d} {2:4d}'.format(x, x*x, x*x*x)) 197 ... 198 1 1 1 199 2 4 8 200 3 9 27 201 4 16 64 202 5 25 125 203 6 36 216 204 7 49 343 205 8 64 512 206 9 81 729 207 10 100 1000 208 209For a complete overview of string formatting with :meth:`str.format`, see 210:ref:`formatstrings`. 211 212 213Manual String Formatting 214------------------------ 215 216Here's the same table of squares and cubes, formatted manually:: 217 218 >>> for x in range(1, 11): 219 ... print(repr(x).rjust(2), repr(x*x).rjust(3), end=' ') 220 ... # Note use of 'end' on previous line 221 ... print(repr(x*x*x).rjust(4)) 222 ... 223 1 1 1 224 2 4 8 225 3 9 27 226 4 16 64 227 5 25 125 228 6 36 216 229 7 49 343 230 8 64 512 231 9 81 729 232 10 100 1000 233 234(Note that the one space between each column was added by the 235way :func:`print` works: it always adds spaces between its arguments.) 236 237The :meth:`str.rjust` method of string objects right-justifies a string in a 238field of a given width by padding it with spaces on the left. There are 239similar methods :meth:`str.ljust` and :meth:`str.center`. These methods do 240not write anything, they just return a new string. If the input string is too 241long, they don't truncate it, but return it unchanged; this will mess up your 242column lay-out but that's usually better than the alternative, which would be 243lying about a value. (If you really want truncation you can always add a 244slice operation, as in ``x.ljust(n)[:n]``.) 245 246There is another method, :meth:`str.zfill`, which pads a numeric string on the 247left with zeros. It understands about plus and minus signs:: 248 249 >>> '12'.zfill(5) 250 '00012' 251 >>> '-3.14'.zfill(7) 252 '-003.14' 253 >>> '3.14159265359'.zfill(5) 254 '3.14159265359' 255 256 257Old string formatting 258--------------------- 259 260The % operator (modulo) can also be used for string formatting. Given ``'string' 261% values``, instances of ``%`` in ``string`` are replaced with zero or more 262elements of ``values``. This operation is commonly known as string 263interpolation. For example:: 264 265 >>> import math 266 >>> print('The value of pi is approximately %5.3f.' % math.pi) 267 The value of pi is approximately 3.142. 268 269More information can be found in the :ref:`old-string-formatting` section. 270 271 272.. _tut-files: 273 274Reading and Writing Files 275========================= 276 277.. index:: 278 builtin: open 279 object: file 280 281:func:`open` returns a :term:`file object`, and is most commonly used with 282two arguments: ``open(filename, mode)``. 283 284:: 285 286 >>> f = open('workfile', 'w') 287 288.. XXX str(f) is <io.TextIOWrapper object at 0x82e8dc4> 289 290 >>> print(f) 291 <open file 'workfile', mode 'w' at 80a0960> 292 293The first argument is a string containing the filename. The second argument is 294another string containing a few characters describing the way in which the file 295will be used. *mode* can be ``'r'`` when the file will only be read, ``'w'`` 296for only writing (an existing file with the same name will be erased), and 297``'a'`` opens the file for appending; any data written to the file is 298automatically added to the end. ``'r+'`` opens the file for both reading and 299writing. The *mode* argument is optional; ``'r'`` will be assumed if it's 300omitted. 301 302Normally, files are opened in :dfn:`text mode`, that means, you read and write 303strings from and to the file, which are encoded in a specific encoding. If 304encoding is not specified, the default is platform dependent (see 305:func:`open`). ``'b'`` appended to the mode opens the file in 306:dfn:`binary mode`: now the data is read and written in the form of bytes 307objects. This mode should be used for all files that don't contain text. 308 309In text mode, the default when reading is to convert platform-specific line 310endings (``\n`` on Unix, ``\r\n`` on Windows) to just ``\n``. When writing in 311text mode, the default is to convert occurrences of ``\n`` back to 312platform-specific line endings. This behind-the-scenes modification 313to file data is fine for text files, but will corrupt binary data like that in 314:file:`JPEG` or :file:`EXE` files. Be very careful to use binary mode when 315reading and writing such files. 316 317It is good practice to use the :keyword:`with` keyword when dealing 318with file objects. The advantage is that the file is properly closed 319after its suite finishes, even if an exception is raised at some 320point. Using :keyword:`!with` is also much shorter than writing 321equivalent :keyword:`try`\ -\ :keyword:`finally` blocks:: 322 323 >>> with open('workfile') as f: 324 ... read_data = f.read() 325 326 >>> # We can check that the file has been automatically closed. 327 >>> f.closed 328 True 329 330If you're not using the :keyword:`with` keyword, then you should call 331``f.close()`` to close the file and immediately free up any system 332resources used by it. 333 334.. warning:: 335 Calling ``f.write()`` without using the :keyword:`!with` keyword or calling 336 ``f.close()`` **might** result in the arguments 337 of ``f.write()`` not being completely written to the disk, even if the 338 program exits successfully. 339 340.. 341 See also https://bugs.python.org/issue17852 342 343After a file object is closed, either by a :keyword:`with` statement 344or by calling ``f.close()``, attempts to use the file object will 345automatically fail. :: 346 347 >>> f.close() 348 >>> f.read() 349 Traceback (most recent call last): 350 File "<stdin>", line 1, in <module> 351 ValueError: I/O operation on closed file. 352 353 354.. _tut-filemethods: 355 356Methods of File Objects 357----------------------- 358 359The rest of the examples in this section will assume that a file object called 360``f`` has already been created. 361 362To read a file's contents, call ``f.read(size)``, which reads some quantity of 363data and returns it as a string (in text mode) or bytes object (in binary mode). 364*size* is an optional numeric argument. When *size* is omitted or negative, the 365entire contents of the file will be read and returned; it's your problem if the 366file is twice as large as your machine's memory. Otherwise, at most *size* 367characters (in text mode) or *size* bytes (in binary mode) are read and returned. 368If the end of the file has been reached, ``f.read()`` will return an empty 369string (``''``). :: 370 371 >>> f.read() 372 'This is the entire file.\n' 373 >>> f.read() 374 '' 375 376``f.readline()`` reads a single line from the file; a newline character (``\n``) 377is left at the end of the string, and is only omitted on the last line of the 378file if the file doesn't end in a newline. This makes the return value 379unambiguous; if ``f.readline()`` returns an empty string, the end of the file 380has been reached, while a blank line is represented by ``'\n'``, a string 381containing only a single newline. :: 382 383 >>> f.readline() 384 'This is the first line of the file.\n' 385 >>> f.readline() 386 'Second line of the file\n' 387 >>> f.readline() 388 '' 389 390For reading lines from a file, you can loop over the file object. This is memory 391efficient, fast, and leads to simple code:: 392 393 >>> for line in f: 394 ... print(line, end='') 395 ... 396 This is the first line of the file. 397 Second line of the file 398 399If you want to read all the lines of a file in a list you can also use 400``list(f)`` or ``f.readlines()``. 401 402``f.write(string)`` writes the contents of *string* to the file, returning 403the number of characters written. :: 404 405 >>> f.write('This is a test\n') 406 15 407 408Other types of objects need to be converted -- either to a string (in text mode) 409or a bytes object (in binary mode) -- before writing them:: 410 411 >>> value = ('the answer', 42) 412 >>> s = str(value) # convert the tuple to string 413 >>> f.write(s) 414 18 415 416``f.tell()`` returns an integer giving the file object's current position in the file 417represented as number of bytes from the beginning of the file when in binary mode and 418an opaque number when in text mode. 419 420To change the file object's position, use ``f.seek(offset, whence)``. The position is computed 421from adding *offset* to a reference point; the reference point is selected by 422the *whence* argument. A *whence* value of 0 measures from the beginning 423of the file, 1 uses the current file position, and 2 uses the end of the file as 424the reference point. *whence* can be omitted and defaults to 0, using the 425beginning of the file as the reference point. :: 426 427 >>> f = open('workfile', 'rb+') 428 >>> f.write(b'0123456789abcdef') 429 16 430 >>> f.seek(5) # Go to the 6th byte in the file 431 5 432 >>> f.read(1) 433 b'5' 434 >>> f.seek(-3, 2) # Go to the 3rd byte before the end 435 13 436 >>> f.read(1) 437 b'd' 438 439In text files (those opened without a ``b`` in the mode string), only seeks 440relative to the beginning of the file are allowed (the exception being seeking 441to the very file end with ``seek(0, 2)``) and the only valid *offset* values are 442those returned from the ``f.tell()``, or zero. Any other *offset* value produces 443undefined behaviour. 444 445File objects have some additional methods, such as :meth:`~file.isatty` and 446:meth:`~file.truncate` which are less frequently used; consult the Library 447Reference for a complete guide to file objects. 448 449 450.. _tut-json: 451 452Saving structured data with :mod:`json` 453--------------------------------------- 454 455.. index:: module: json 456 457Strings can easily be written to and read from a file. Numbers take a bit more 458effort, since the :meth:`read` method only returns strings, which will have to 459be passed to a function like :func:`int`, which takes a string like ``'123'`` 460and returns its numeric value 123. When you want to save more complex data 461types like nested lists and dictionaries, parsing and serializing by hand 462becomes complicated. 463 464Rather than having users constantly writing and debugging code to save 465complicated data types to files, Python allows you to use the popular data 466interchange format called `JSON (JavaScript Object Notation) 467<http://json.org>`_. The standard module called :mod:`json` can take Python 468data hierarchies, and convert them to string representations; this process is 469called :dfn:`serializing`. Reconstructing the data from the string representation 470is called :dfn:`deserializing`. Between serializing and deserializing, the 471string representing the object may have been stored in a file or data, or 472sent over a network connection to some distant machine. 473 474.. note:: 475 The JSON format is commonly used by modern applications to allow for data 476 exchange. Many programmers are already familiar with it, which makes 477 it a good choice for interoperability. 478 479If you have an object ``x``, you can view its JSON string representation with a 480simple line of code:: 481 482 >>> import json 483 >>> json.dumps([1, 'simple', 'list']) 484 '[1, "simple", "list"]' 485 486Another variant of the :func:`~json.dumps` function, called :func:`~json.dump`, 487simply serializes the object to a :term:`text file`. So if ``f`` is a 488:term:`text file` object opened for writing, we can do this:: 489 490 json.dump(x, f) 491 492To decode the object again, if ``f`` is a :term:`text file` object which has 493been opened for reading:: 494 495 x = json.load(f) 496 497This simple serialization technique can handle lists and dictionaries, but 498serializing arbitrary class instances in JSON requires a bit of extra effort. 499The reference for the :mod:`json` module contains an explanation of this. 500 501.. seealso:: 502 503 :mod:`pickle` - the pickle module 504 505 Contrary to :ref:`JSON <tut-json>`, *pickle* is a protocol which allows 506 the serialization of arbitrarily complex Python objects. As such, it is 507 specific to Python and cannot be used to communicate with applications 508 written in other languages. It is also insecure by default: 509 deserializing pickle data coming from an untrusted source can execute 510 arbitrary code, if the data was crafted by a skilled attacker. 511