1.. _2to3-reference: 2 32to3 - Automated Python 2 to 3 code translation 4=============================================== 5 6.. sectionauthor:: Benjamin Peterson <benjamin@python.org> 7 82to3 is a Python program that reads Python 2.x source code and applies a series 9of *fixers* to transform it into valid Python 3.x code. The standard library 10contains a rich set of fixers that will handle almost all code. 2to3 supporting 11library :mod:`lib2to3` is, however, a flexible and generic library, so it is 12possible to write your own fixers for 2to3. :mod:`lib2to3` could also be 13adapted to custom applications in which Python code needs to be edited 14automatically. 15 16 17.. _2to3-using: 18 19Using 2to3 20---------- 21 222to3 will usually be installed with the Python interpreter as a script. It is 23also located in the :file:`Tools/scripts` directory of the Python root. 24 252to3's basic arguments are a list of files or directories to transform. The 26directories are recursively traversed for Python sources. 27 28Here is a sample Python 2.x source file, :file:`example.py`:: 29 30 def greet(name): 31 print "Hello, {0}!".format(name) 32 print "What's your name?" 33 name = raw_input() 34 greet(name) 35 36It can be converted to Python 3.x code via 2to3 on the command line: 37 38.. code-block:: shell-session 39 40 $ 2to3 example.py 41 42A diff against the original source file is printed. 2to3 can also write the 43needed modifications right back to the source file. (A backup of the original 44file is made unless :option:`!-n` is also given.) Writing the changes back is 45enabled with the :option:`!-w` flag: 46 47.. code-block:: shell-session 48 49 $ 2to3 -w example.py 50 51After transformation, :file:`example.py` looks like this:: 52 53 def greet(name): 54 print("Hello, {0}!".format(name)) 55 print("What's your name?") 56 name = input() 57 greet(name) 58 59Comments and exact indentation are preserved throughout the translation process. 60 61By default, 2to3 runs a set of :ref:`predefined fixers <2to3-fixers>`. The 62:option:`!-l` flag lists all available fixers. An explicit set of fixers to run 63can be given with :option:`!-f`. Likewise the :option:`!-x` explicitly disables a 64fixer. The following example runs only the ``imports`` and ``has_key`` fixers: 65 66.. code-block:: shell-session 67 68 $ 2to3 -f imports -f has_key example.py 69 70This command runs every fixer except the ``apply`` fixer: 71 72.. code-block:: shell-session 73 74 $ 2to3 -x apply example.py 75 76Some fixers are *explicit*, meaning they aren't run by default and must be 77listed on the command line to be run. Here, in addition to the default fixers, 78the ``idioms`` fixer is run: 79 80.. code-block:: shell-session 81 82 $ 2to3 -f all -f idioms example.py 83 84Notice how passing ``all`` enables all default fixers. 85 86Sometimes 2to3 will find a place in your source code that needs to be changed, 87but 2to3 cannot fix automatically. In this case, 2to3 will print a warning 88beneath the diff for a file. You should address the warning in order to have 89compliant 3.x code. 90 912to3 can also refactor doctests. To enable this mode, use the :option:`!-d` 92flag. Note that *only* doctests will be refactored. This also doesn't require 93the module to be valid Python. For example, doctest like examples in a reST 94document could also be refactored with this option. 95 96The :option:`!-v` option enables output of more information on the translation 97process. 98 99Since some print statements can be parsed as function calls or statements, 2to3 100cannot always read files containing the print function. When 2to3 detects the 101presence of the ``from __future__ import print_function`` compiler directive, it 102modifies its internal grammar to interpret :func:`print` as a function. This 103change can also be enabled manually with the :option:`!-p` flag. Use 104:option:`!-p` to run fixers on code that already has had its print statements 105converted. 106 107The :option:`!-o` or :option:`!--output-dir` option allows specification of an 108alternate directory for processed output files to be written to. The 109:option:`!-n` flag is required when using this as backup files do not make sense 110when not overwriting the input files. 111 112.. versionadded:: 2.7.3 113 The :option:`!-o` option was added. 114 115The :option:`!-W` or :option:`!--write-unchanged-files` flag tells 2to3 to always 116write output files even if no changes were required to the file. This is most 117useful with :option:`!-o` so that an entire Python source tree is copied with 118translation from one directory to another. 119This option implies the :option:`!-w` flag as it would not make sense otherwise. 120 121.. versionadded:: 2.7.3 122 The :option:`!-W` flag was added. 123 124The :option:`!--add-suffix` option specifies a string to append to all output 125filenames. The :option:`!-n` flag is required when specifying this as backups 126are not necessary when writing to different filenames. Example: 127 128.. code-block:: shell-session 129 130 $ 2to3 -n -W --add-suffix=3 example.py 131 132Will cause a converted file named ``example.py3`` to be written. 133 134.. versionadded:: 2.7.3 135 The :option:`!--add-suffix` option was added. 136 137To translate an entire project from one directory tree to another use: 138 139.. code-block:: shell-session 140 141 $ 2to3 --output-dir=python3-version/mycode -W -n python2-version/mycode 142 143 144.. _2to3-fixers: 145 146Fixers 147------ 148 149Each step of transforming code is encapsulated in a fixer. The command ``2to3 150-l`` lists them. As :ref:`documented above <2to3-using>`, each can be turned on 151and off individually. They are described here in more detail. 152 153 154.. 2to3fixer:: apply 155 156 Removes usage of :func:`apply`. For example ``apply(function, *args, 157 **kwargs)`` is converted to ``function(*args, **kwargs)``. 158 159.. 2to3fixer:: asserts 160 161 Replaces deprecated :mod:`unittest` method names with the correct ones. 162 163 ================================ ========================================== 164 From To 165 ================================ ========================================== 166 ``failUnlessEqual(a, b)`` :meth:`assertEqual(a, b) 167 <unittest.TestCase.assertEqual>` 168 ``assertEquals(a, b)`` :meth:`assertEqual(a, b) 169 <unittest.TestCase.assertEqual>` 170 ``failIfEqual(a, b)`` :meth:`assertNotEqual(a, b) 171 <unittest.TestCase.assertNotEqual>` 172 ``assertNotEquals(a, b)`` :meth:`assertNotEqual(a, b) 173 <unittest.TestCase.assertNotEqual>` 174 ``failUnless(a)`` :meth:`assertTrue(a) 175 <unittest.TestCase.assertTrue>` 176 ``assert_(a)`` :meth:`assertTrue(a) 177 <unittest.TestCase.assertTrue>` 178 ``failIf(a)`` :meth:`assertFalse(a) 179 <unittest.TestCase.assertFalse>` 180 ``failUnlessRaises(exc, cal)`` :meth:`assertRaises(exc, cal) 181 <unittest.TestCase.assertRaises>` 182 ``failUnlessAlmostEqual(a, b)`` :meth:`assertAlmostEqual(a, b) 183 <unittest.TestCase.assertAlmostEqual>` 184 ``assertAlmostEquals(a, b)`` :meth:`assertAlmostEqual(a, b) 185 <unittest.TestCase.assertAlmostEqual>` 186 ``failIfAlmostEqual(a, b)`` :meth:`assertNotAlmostEqual(a, b) 187 <unittest.TestCase.assertNotAlmostEqual>` 188 ``assertNotAlmostEquals(a, b)`` :meth:`assertNotAlmostEqual(a, b) 189 <unittest.TestCase.assertNotAlmostEqual>` 190 ================================ ========================================== 191 192.. 2to3fixer:: basestring 193 194 Converts :class:`basestring` to :class:`str`. 195 196.. 2to3fixer:: buffer 197 198 Converts :class:`buffer` to :class:`memoryview`. This fixer is optional 199 because the :class:`memoryview` API is similar but not exactly the same as 200 that of :class:`buffer`. 201 202.. 2to3fixer:: dict 203 204 Fixes dictionary iteration methods. :meth:`dict.iteritems` is converted to 205 :meth:`dict.items`, :meth:`dict.iterkeys` to :meth:`dict.keys`, and 206 :meth:`dict.itervalues` to :meth:`dict.values`. Similarly, 207 :meth:`dict.viewitems`, :meth:`dict.viewkeys` and :meth:`dict.viewvalues` are 208 converted respectively to :meth:`dict.items`, :meth:`dict.keys` and 209 :meth:`dict.values`. It also wraps existing usages of :meth:`dict.items`, 210 :meth:`dict.keys`, and :meth:`dict.values` in a call to :class:`list`. 211 212.. 2to3fixer:: except 213 214 Converts ``except X, T`` to ``except X as T``. 215 216.. 2to3fixer:: exec 217 218 Converts the :keyword:`exec` statement to the :func:`exec` function. 219 220.. 2to3fixer:: execfile 221 222 Removes usage of :func:`execfile`. The argument to :func:`execfile` is 223 wrapped in calls to :func:`open`, :func:`compile`, and :func:`exec`. 224 225.. 2to3fixer:: exitfunc 226 227 Changes assignment of :attr:`sys.exitfunc` to use of the :mod:`atexit` 228 module. 229 230.. 2to3fixer:: filter 231 232 Wraps :func:`filter` usage in a :class:`list` call. 233 234.. 2to3fixer:: funcattrs 235 236 Fixes function attributes that have been renamed. For example, 237 ``my_function.func_closure`` is converted to ``my_function.__closure__``. 238 239.. 2to3fixer:: future 240 241 Removes ``from __future__ import new_feature`` statements. 242 243.. 2to3fixer:: getcwdu 244 245 Renames :func:`os.getcwdu` to :func:`os.getcwd`. 246 247.. 2to3fixer:: has_key 248 249 Changes ``dict.has_key(key)`` to ``key in dict``. 250 251.. 2to3fixer:: idioms 252 253 This optional fixer performs several transformations that make Python code 254 more idiomatic. Type comparisons like ``type(x) is SomeClass`` and 255 ``type(x) == SomeClass`` are converted to ``isinstance(x, SomeClass)``. 256 ``while 1`` becomes ``while True``. This fixer also tries to make use of 257 :func:`sorted` in appropriate places. For example, this block :: 258 259 L = list(some_iterable) 260 L.sort() 261 262 is changed to :: 263 264 L = sorted(some_iterable) 265 266.. 2to3fixer:: import 267 268 Detects sibling imports and converts them to relative imports. 269 270.. 2to3fixer:: imports 271 272 Handles module renames in the standard library. 273 274.. 2to3fixer:: imports2 275 276 Handles other modules renames in the standard library. It is separate from 277 the :2to3fixer:`imports` fixer only because of technical limitations. 278 279.. 2to3fixer:: input 280 281 Converts ``input(prompt)`` to ``eval(input(prompt))``. 282 283.. 2to3fixer:: intern 284 285 Converts :func:`intern` to :func:`sys.intern`. 286 287.. 2to3fixer:: isinstance 288 289 Fixes duplicate types in the second argument of :func:`isinstance`. For 290 example, ``isinstance(x, (int, int))`` is converted to ``isinstance(x, 291 (int))``. 292 293.. 2to3fixer:: itertools_imports 294 295 Removes imports of :func:`itertools.ifilter`, :func:`itertools.izip`, and 296 :func:`itertools.imap`. Imports of :func:`itertools.ifilterfalse` are also 297 changed to :func:`itertools.filterfalse`. 298 299.. 2to3fixer:: itertools 300 301 Changes usage of :func:`itertools.ifilter`, :func:`itertools.izip`, and 302 :func:`itertools.imap` to their built-in equivalents. 303 :func:`itertools.ifilterfalse` is changed to :func:`itertools.filterfalse`. 304 305.. 2to3fixer:: long 306 307 Renames :class:`long` to :class:`int`. 308 309.. 2to3fixer:: map 310 311 Wraps :func:`map` in a :class:`list` call. It also changes ``map(None, x)`` 312 to ``list(x)``. Using ``from future_builtins import map`` disables this 313 fixer. 314 315.. 2to3fixer:: metaclass 316 317 Converts the old metaclass syntax (``__metaclass__ = Meta`` in the class 318 body) to the new (``class X(metaclass=Meta)``). 319 320.. 2to3fixer:: methodattrs 321 322 Fixes old method attribute names. For example, ``meth.im_func`` is converted 323 to ``meth.__func__``. 324 325.. 2to3fixer:: ne 326 327 Converts the old not-equal syntax, ``<>``, to ``!=``. 328 329.. 2to3fixer:: next 330 331 Converts the use of iterator's :meth:`~iterator.next` methods to the 332 :func:`next` function. It also renames :meth:`~iterator.next` methods to 333 :meth:`~iterator.__next__`. 334 335.. 2to3fixer:: nonzero 336 337 Renames :meth:`__nonzero__` to :meth:`~object.__bool__`. 338 339.. 2to3fixer:: numliterals 340 341 Converts octal literals into the new syntax. 342 343.. 2to3fixer:: paren 344 345 Add extra parenthesis where they are required in list comprehensions. For 346 example, ``[x for x in 1, 2]`` becomes ``[x for x in (1, 2)]``. 347 348.. 2to3fixer:: print 349 350 Converts the :keyword:`print` statement to the :func:`print` function. 351 352.. 2to3fixer:: raise 353 354 Converts ``raise E, V`` to ``raise E(V)``, and ``raise E, V, T`` to ``raise 355 E(V).with_traceback(T)``. If ``E`` is a tuple, the translation will be 356 incorrect because substituting tuples for exceptions has been removed in Python 3. 357 358.. 2to3fixer:: raw_input 359 360 Converts :func:`raw_input` to :func:`input`. 361 362.. 2to3fixer:: reduce 363 364 Handles the move of :func:`reduce` to :func:`functools.reduce`. 365 366.. 2to3fixer:: renames 367 368 Changes :data:`sys.maxint` to :data:`sys.maxsize`. 369 370.. 2to3fixer:: repr 371 372 Replaces backtick repr with the :func:`repr` function. 373 374.. 2to3fixer:: set_literal 375 376 Replaces use of the :class:`set` constructor with set literals. This fixer 377 is optional. 378 379.. 2to3fixer:: standarderror 380 381 Renames :exc:`StandardError` to :exc:`Exception`. 382 383.. 2to3fixer:: sys_exc 384 385 Changes the deprecated :data:`sys.exc_value`, :data:`sys.exc_type`, 386 :data:`sys.exc_traceback` to use :func:`sys.exc_info`. 387 388.. 2to3fixer:: throw 389 390 Fixes the API change in generator's :meth:`throw` method. 391 392.. 2to3fixer:: tuple_params 393 394 Removes implicit tuple parameter unpacking. This fixer inserts temporary 395 variables. 396 397.. 2to3fixer:: types 398 399 Fixes code broken from the removal of some members in the :mod:`types` 400 module. 401 402.. 2to3fixer:: unicode 403 404 Renames :class:`unicode` to :class:`str`. 405 406.. 2to3fixer:: urllib 407 408 Handles the rename of :mod:`urllib` and :mod:`urllib2` to the :mod:`urllib` 409 package. 410 411.. 2to3fixer:: ws_comma 412 413 Removes excess whitespace from comma separated items. This fixer is 414 optional. 415 416.. 2to3fixer:: xrange 417 418 Renames :func:`xrange` to :func:`range` and wraps existing :func:`range` 419 calls with :class:`list`. 420 421.. 2to3fixer:: xreadlines 422 423 Changes ``for x in file.xreadlines()`` to ``for x in file``. 424 425.. 2to3fixer:: zip 426 427 Wraps :func:`zip` usage in a :class:`list` call. This is disabled when 428 ``from future_builtins import zip`` appears. 429 430 431:mod:`lib2to3` - 2to3's library 432------------------------------- 433 434.. module:: lib2to3 435 :synopsis: the 2to3 library 436.. moduleauthor:: Guido van Rossum 437.. moduleauthor:: Collin Winter 438.. moduleauthor:: Benjamin Peterson <benjamin@python.org> 439 440 441.. note:: 442 443 The :mod:`lib2to3` API should be considered unstable and may change 444 drastically in the future. 445 446.. XXX What is the public interface anyway? 447