1.. _tut-io: 2 3**************** 4Input and Output 5**************** 6 7There are several ways to present the output of a program; data can be printed 8in a human-readable form, or written to a file for future use. This chapter will 9discuss some of the possibilities. 10 11 12.. _tut-formatting: 13 14Fancier Output Formatting 15========================= 16 17So far we've encountered two ways of writing values: *expression statements* and 18the :keyword:`print` statement. (A third way is using the :meth:`write` method 19of file objects; the standard output file can be referenced as ``sys.stdout``. 20See the Library Reference for more information on this.) 21 22Often you'll want more control over the formatting of your output than simply 23printing space-separated values. There are two ways to format your output; the 24first way is to do all the string handling yourself; using string slicing and 25concatenation operations you can create any layout you can imagine. The 26string types have some methods that perform useful operations for padding 27strings to a given column width; these will be discussed shortly. The second 28way is to use the :meth:`str.format` method. 29 30The :mod:`string` module contains a :class:`~string.Template` class which offers 31yet another way to substitute values into strings. 32 33One question remains, of course: how do you convert values to strings? Luckily, 34Python has ways to convert any value to a string: pass it to the :func:`repr` 35or :func:`str` functions. 36 37The :func:`str` function is meant to return representations of values which are 38fairly human-readable, while :func:`repr` is meant to generate representations 39which can be read by the interpreter (or will force a :exc:`SyntaxError` if 40there is no equivalent syntax). For objects which don't have a particular 41representation for human consumption, :func:`str` will return the same value as 42:func:`repr`. Many values, such as numbers or structures like lists and 43dictionaries, have the same representation using either function. Strings and 44floating point numbers, in particular, have two distinct representations. 45 46Some examples:: 47 48 >>> s = 'Hello, world.' 49 >>> str(s) 50 'Hello, world.' 51 >>> repr(s) 52 "'Hello, world.'" 53 >>> str(1.0/7.0) 54 '0.142857142857' 55 >>> repr(1.0/7.0) 56 '0.14285714285714285' 57 >>> x = 10 * 3.25 58 >>> y = 200 * 200 59 >>> s = 'The value of x is ' + repr(x) + ', and y is ' + repr(y) + '...' 60 >>> print s 61 The value of x is 32.5, and y is 40000... 62 >>> # The repr() of a string adds string quotes and backslashes: 63 ... hello = 'hello, world\n' 64 >>> hellos = repr(hello) 65 >>> print hellos 66 'hello, world\n' 67 >>> # The argument to repr() may be any Python object: 68 ... repr((x, y, ('spam', 'eggs'))) 69 "(32.5, 40000, ('spam', 'eggs'))" 70 71Here are two ways to write a table of squares and cubes:: 72 73 >>> for x in range(1, 11): 74 ... print repr(x).rjust(2), repr(x*x).rjust(3), 75 ... # Note trailing comma on previous line 76 ... print repr(x*x*x).rjust(4) 77 ... 78 1 1 1 79 2 4 8 80 3 9 27 81 4 16 64 82 5 25 125 83 6 36 216 84 7 49 343 85 8 64 512 86 9 81 729 87 10 100 1000 88 89 >>> for x in range(1,11): 90 ... print '{0:2d} {1:3d} {2:4d}'.format(x, x*x, x*x*x) 91 ... 92 1 1 1 93 2 4 8 94 3 9 27 95 4 16 64 96 5 25 125 97 6 36 216 98 7 49 343 99 8 64 512 100 9 81 729 101 10 100 1000 102 103(Note that in the first example, one space between each column was added by the 104way :keyword:`print` works: it always adds spaces between its arguments.) 105 106This example demonstrates the :meth:`str.rjust` method of string 107objects, which right-justifies a string in a field of a given width by padding 108it with spaces on the left. There are similar methods :meth:`str.ljust` and 109:meth:`str.center`. These methods do not write anything, they just return a 110new string. If the input string is too long, they don't truncate it, but 111return it unchanged; this will mess up your column lay-out but that's usually 112better than the alternative, which would be lying about a value. (If you 113really want truncation you can always add a slice operation, as in 114``x.ljust(n)[:n]``.) 115 116There is another method, :meth:`str.zfill`, which pads a numeric string on the 117left with zeros. It understands about plus and minus signs:: 118 119 >>> '12'.zfill(5) 120 '00012' 121 >>> '-3.14'.zfill(7) 122 '-003.14' 123 >>> '3.14159265359'.zfill(5) 124 '3.14159265359' 125 126Basic usage of the :meth:`str.format` method looks like this:: 127 128 >>> print 'We are the {} who say "{}!"'.format('knights', 'Ni') 129 We are the knights who say "Ni!" 130 131The brackets and characters within them (called format fields) are replaced with 132the objects passed into the :meth:`str.format` method. A number in the 133brackets refers to the position of the object passed into the 134:meth:`str.format` method. :: 135 136 >>> print '{0} and {1}'.format('spam', 'eggs') 137 spam and eggs 138 >>> print '{1} and {0}'.format('spam', 'eggs') 139 eggs and spam 140 141If keyword arguments are used in the :meth:`str.format` method, their values 142are referred to by using the name of the argument. :: 143 144 >>> print 'This {food} is {adjective}.'.format( 145 ... food='spam', adjective='absolutely horrible') 146 This spam is absolutely horrible. 147 148Positional and keyword arguments can be arbitrarily combined:: 149 150 >>> print 'The story of {0}, {1}, and {other}.'.format('Bill', 'Manfred', 151 ... other='Georg') 152 The story of Bill, Manfred, and Georg. 153 154``'!s'`` (apply :func:`str`) and ``'!r'`` (apply :func:`repr`) can be used to 155convert the value before it is formatted. :: 156 157 >>> import math 158 >>> print 'The value of PI is approximately {}.'.format(math.pi) 159 The value of PI is approximately 3.14159265359. 160 >>> print 'The value of PI is approximately {!r}.'.format(math.pi) 161 The value of PI is approximately 3.141592653589793. 162 163An optional ``':'`` and format specifier can follow the field name. This allows 164greater control over how the value is formatted. The following example 165rounds Pi to three places after the decimal. 166 167 >>> import math 168 >>> print 'The value of PI is approximately {0:.3f}.'.format(math.pi) 169 The value of PI is approximately 3.142. 170 171Passing an integer after the ``':'`` will cause that field to be a minimum 172number of characters wide. This is useful for making tables pretty. :: 173 174 >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 7678} 175 >>> for name, phone in table.items(): 176 ... print '{0:10} ==> {1:10d}'.format(name, phone) 177 ... 178 Jack ==> 4098 179 Dcab ==> 7678 180 Sjoerd ==> 4127 181 182If you have a really long format string that you don't want to split up, it 183would be nice if you could reference the variables to be formatted by name 184instead of by position. This can be done by simply passing the dict and using 185square brackets ``'[]'`` to access the keys :: 186 187 >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 8637678} 188 >>> print ('Jack: {0[Jack]:d}; Sjoerd: {0[Sjoerd]:d}; ' 189 ... 'Dcab: {0[Dcab]:d}'.format(table)) 190 Jack: 4098; Sjoerd: 4127; Dcab: 8637678 191 192This could also be done by passing the table as keyword arguments with the '**' 193notation. :: 194 195 >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 8637678} 196 >>> print 'Jack: {Jack:d}; Sjoerd: {Sjoerd:d}; Dcab: {Dcab:d}'.format(**table) 197 Jack: 4098; Sjoerd: 4127; Dcab: 8637678 198 199This is particularly useful in combination with the built-in function 200:func:`vars`, which returns a dictionary containing all local variables. 201 202For a complete overview of string formatting with :meth:`str.format`, see 203:ref:`formatstrings`. 204 205 206Old string formatting 207--------------------- 208 209The ``%`` operator can also be used for string formatting. It interprets the 210left argument much like a :c:func:`sprintf`\ -style format string to be applied 211to the right argument, and returns the string resulting from this formatting 212operation. For example:: 213 214 >>> import math 215 >>> print 'The value of PI is approximately %5.3f.' % math.pi 216 The value of PI is approximately 3.142. 217 218More information can be found in the :ref:`string-formatting` section. 219 220 221.. _tut-files: 222 223Reading and Writing Files 224========================= 225 226.. index:: 227 builtin: open 228 object: file 229 230:func:`open` returns a file object, and is most commonly used with two 231arguments: ``open(filename, mode)``. 232 233:: 234 235 >>> f = open('workfile', 'w') 236 >>> print f 237 <open file 'workfile', mode 'w' at 80a0960> 238 239The first argument is a string containing the filename. The second argument is 240another string containing a few characters describing the way in which the file 241will be used. *mode* can be ``'r'`` when the file will only be read, ``'w'`` 242for only writing (an existing file with the same name will be erased), and 243``'a'`` opens the file for appending; any data written to the file is 244automatically added to the end. ``'r+'`` opens the file for both reading and 245writing. The *mode* argument is optional; ``'r'`` will be assumed if it's 246omitted. 247 248On Windows, ``'b'`` appended to the mode opens the file in binary mode, so there 249are also modes like ``'rb'``, ``'wb'``, and ``'r+b'``. Python on Windows makes 250a distinction between text and binary files; the end-of-line characters in text 251files are automatically altered slightly when data is read or written. This 252behind-the-scenes modification to file data is fine for ASCII text files, but 253it'll corrupt binary data like that in :file:`JPEG` or :file:`EXE` files. Be 254very careful to use binary mode when reading and writing such files. On Unix, 255it doesn't hurt to append a ``'b'`` to the mode, so you can use it 256platform-independently for all binary files. 257 258 259.. _tut-filemethods: 260 261Methods of File Objects 262----------------------- 263 264The rest of the examples in this section will assume that a file object called 265``f`` has already been created. 266 267To read a file's contents, call ``f.read(size)``, which reads some quantity of 268data and returns it as a string. *size* is an optional numeric argument. When 269*size* is omitted or negative, the entire contents of the file will be read and 270returned; it's your problem if the file is twice as large as your machine's 271memory. Otherwise, at most *size* bytes are read and returned. If the end of 272the file has been reached, ``f.read()`` will return an empty string (``""``). 273:: 274 275 >>> f.read() 276 'This is the entire file.\n' 277 >>> f.read() 278 '' 279 280``f.readline()`` reads a single line from the file; a newline character (``\n``) 281is left at the end of the string, and is only omitted on the last line of the 282file if the file doesn't end in a newline. This makes the return value 283unambiguous; if ``f.readline()`` returns an empty string, the end of the file 284has been reached, while a blank line is represented by ``'\n'``, a string 285containing only a single newline. :: 286 287 >>> f.readline() 288 'This is the first line of the file.\n' 289 >>> f.readline() 290 'Second line of the file\n' 291 >>> f.readline() 292 '' 293 294For reading lines from a file, you can loop over the file object. This is memory 295efficient, fast, and leads to simple code:: 296 297 >>> for line in f: 298 print line, 299 300 This is the first line of the file. 301 Second line of the file 302 303If you want to read all the lines of a file in a list you can also use 304``list(f)`` or ``f.readlines()``. 305 306``f.write(string)`` writes the contents of *string* to the file, returning 307``None``. :: 308 309 >>> f.write('This is a test\n') 310 311To write something other than a string, it needs to be converted to a string 312first:: 313 314 >>> value = ('the answer', 42) 315 >>> s = str(value) 316 >>> f.write(s) 317 318``f.tell()`` returns an integer giving the file object's current position in the 319file, measured in bytes from the beginning of the file. To change the file 320object's position, use ``f.seek(offset, from_what)``. The position is computed 321from adding *offset* to a reference point; the reference point is selected by 322the *from_what* argument. A *from_what* value of 0 measures from the beginning 323of the file, 1 uses the current file position, and 2 uses the end of the file as 324the reference point. *from_what* can be omitted and defaults to 0, using the 325beginning of the file as the reference point. :: 326 327 >>> f = open('workfile', 'r+') 328 >>> f.write('0123456789abcdef') 329 >>> f.seek(5) # Go to the 6th byte in the file 330 >>> f.read(1) 331 '5' 332 >>> f.seek(-3, 2) # Go to the 3rd byte before the end 333 >>> f.read(1) 334 'd' 335 336When you're done with a file, call ``f.close()`` to close it and free up any 337system resources taken up by the open file. After calling ``f.close()``, 338attempts to use the file object will automatically fail. :: 339 340 >>> f.close() 341 >>> f.read() 342 Traceback (most recent call last): 343 File "<stdin>", line 1, in ? 344 ValueError: I/O operation on closed file 345 346It is good practice to use the :keyword:`with` keyword when dealing with file 347objects. This has the advantage that the file is properly closed after its 348suite finishes, even if an exception is raised on the way. It is also much 349shorter than writing equivalent :keyword:`try`\ -\ :keyword:`finally` blocks:: 350 351 >>> with open('workfile', 'r') as f: 352 ... read_data = f.read() 353 >>> f.closed 354 True 355 356File objects have some additional methods, such as :meth:`~file.isatty` and 357:meth:`~file.truncate` which are less frequently used; consult the Library 358Reference for a complete guide to file objects. 359 360 361.. _tut-json: 362 363Saving structured data with :mod:`json` 364--------------------------------------- 365 366.. index:: module: json 367 368Strings can easily be written to and read from a file. Numbers take a bit more 369effort, since the :meth:`read` method only returns strings, which will have to 370be passed to a function like :func:`int`, which takes a string like ``'123'`` 371and returns its numeric value 123. When you want to save more complex data 372types like nested lists and dictionaries, parsing and serializing by hand 373becomes complicated. 374 375Rather than having users constantly writing and debugging code to save 376complicated data types to files, Python allows you to use the popular data 377interchange format called `JSON (JavaScript Object Notation) 378<http://json.org>`_. The standard module called :mod:`json` can take Python 379data hierarchies, and convert them to string representations; this process is 380called :dfn:`serializing`. Reconstructing the data from the string representation 381is called :dfn:`deserializing`. Between serializing and deserializing, the 382string representing the object may have been stored in a file or data, or 383sent over a network connection to some distant machine. 384 385.. note:: 386 The JSON format is commonly used by modern applications to allow for data 387 exchange. Many programmers are already familiar with it, which makes 388 it a good choice for interoperability. 389 390If you have an object ``x``, you can view its JSON string representation with a 391simple line of code:: 392 393 >>> json.dumps([1, 'simple', 'list']) 394 '[1, "simple", "list"]' 395 396Another variant of the :func:`~json.dumps` function, called :func:`~json.dump`, 397simply serializes the object to a file. So if ``f`` is a :term:`file object` 398opened for writing, we can do this:: 399 400 json.dump(x, f) 401 402To decode the object again, if ``f`` is a :term:`file object` which has 403been opened for reading:: 404 405 x = json.load(f) 406 407This simple serialization technique can handle lists and dictionaries, but 408serializing arbitrary class instances in JSON requires a bit of extra effort. 409The reference for the :mod:`json` module contains an explanation of this. 410 411.. seealso:: 412 413 :mod:`pickle` - the pickle module 414 415 Contrary to :ref:`JSON <tut-json>`, *pickle* is a protocol which allows 416 the serialization of arbitrarily complex Python objects. As such, it is 417 specific to Python and cannot be used to communicate with applications 418 written in other languages. It is also insecure by default: 419 deserializing pickle data coming from an untrusted source can execute 420 arbitrary code, if the data was crafted by a skilled attacker. 421 422