1**************************** 2 What's New in Python 2.7 3**************************** 4 5:Author: A.M. Kuchling (amk at amk.ca) 6 7.. hyperlink all the methods & functions. 8 9.. T_STRING_INPLACE not described in main docs 10 11.. $Id$ 12 Rules for maintenance: 13 14 * Anyone can add text to this document. Do not spend very much time 15 on the wording of your changes, because your text will probably 16 get rewritten to some degree. 17 18 * The maintainer will go through Misc/NEWS periodically and add 19 changes; it's therefore more important to add your changes to 20 Misc/NEWS than to this file. 21 22 * This is not a complete list of every single change; completeness 23 is the purpose of Misc/NEWS. Some changes I consider too small 24 or esoteric to include. If such a change is added to the text, 25 I'll just remove it. (This is another reason you shouldn't spend 26 too much time on writing your addition.) 27 28 * If you want to draw your new text to the attention of the 29 maintainer, add 'XXX' to the beginning of the paragraph or 30 section. 31 32 * It's OK to just add a fragmentary note about a change. For 33 example: "XXX Describe the transmogrify() function added to the 34 socket module." The maintainer will research the change and 35 write the necessary text. 36 37 * You can comment out your additions if you like, but it's not 38 necessary (especially when a final release is some months away). 39 40 * Credit the author of a patch or bugfix. Just the name is 41 sufficient; the e-mail address isn't necessary. 42 43 * It's helpful to add the bug/patch number in a parenthetical comment. 44 45 XXX Describe the transmogrify() function added to the socket 46 module. 47 (Contributed by P.Y. Developer; :issue:`12345`.) 48 49 This saves the maintainer some effort going through the SVN logs 50 when researching a change. 51 52This article explains the new features in Python 2.7. Python 2.7 was released 53on July 3, 2010. 54 55Numeric handling has been improved in many ways, for both 56floating-point numbers and for the :class:`~decimal.Decimal` class. 57There are some useful additions to the standard library, such as a 58greatly enhanced :mod:`unittest` module, the :mod:`argparse` module 59for parsing command-line options, convenient :class:`~collections.OrderedDict` 60and :class:`~collections.Counter` classes in the :mod:`collections` module, 61and many other improvements. 62 63Python 2.7 is planned to be the last of the 2.x releases, so we worked 64on making it a good release for the long term. To help with porting 65to Python 3, several new features from the Python 3.x series have been 66included in 2.7. 67 68This article doesn't attempt to provide a complete specification of 69the new features, but instead provides a convenient overview. For 70full details, you should refer to the documentation for Python 2.7 at 71https://docs.python.org. If you want to understand the rationale for 72the design and implementation, refer to the PEP for a particular new 73feature or the issue on https://bugs.python.org in which a change was 74discussed. Whenever possible, "What's New in Python" links to the 75bug/patch item for each change. 76 77.. _whatsnew27-python31: 78 79The Future for Python 2.x 80========================= 81 82Python 2.7 is the last major release in the 2.x series, as the Python 83maintainers have shifted the focus of their new feature development efforts 84to the Python 3.x series. This means that while Python 2 continues to 85receive bug fixes, and to be updated to build correctly on new hardware and 86versions of supported operated systems, there will be no new full feature 87releases for the language or standard library. 88 89However, while there is a large common subset between Python 2.7 and Python 903, and many of the changes involved in migrating to that common subset, or 91directly to Python 3, can be safely automated, some other changes (notably 92those associated with Unicode handling) may require careful consideration, 93and preferably robust automated regression test suites, to migrate 94effectively. 95 96This means that Python 2.7 will remain in place for a long time, providing a 97stable and supported base platform for production systems that have not yet 98been ported to Python 3. The full expected lifecycle of the Python 2.7 99series is detailed in :pep:`373`. 100 101Some key consequences of the long-term significance of 2.7 are: 102 103* As noted above, the 2.7 release has a much longer period of maintenance 104 when compared to earlier 2.x versions. Python 2.7 is currently expected to 105 remain supported by the core development team (receiving security updates 106 and other bug fixes) until at least 2020 (10 years after its initial 107 release, compared to the more typical support period of 18--24 months). 108 109* As the Python 2.7 standard library ages, making effective use of the 110 Python Package Index (either directly or via a redistributor) becomes 111 more important for Python 2 users. In addition to a wide variety of third 112 party packages for various tasks, the available packages include backports 113 of new modules and features from the Python 3 standard library that are 114 compatible with Python 2, as well as various tools and libraries that can 115 make it easier to migrate to Python 3. The `Python Packaging User Guide 116 <https://packaging.python.org>`__ provides guidance on downloading and 117 installing software from the Python Package Index. 118 119* While the preferred approach to enhancing Python 2 is now the publication 120 of new packages on the Python Package Index, this approach doesn't 121 necessarily work in all cases, especially those related to network 122 security. In exceptional cases that cannot be handled adequately by 123 publishing new or updated packages on PyPI, the Python Enhancement 124 Proposal process may be used to make the case for adding new features 125 directly to the Python 2 standard library. Any such additions, and the 126 maintenance releases where they were added, will be noted in the 127 :ref:`py27-maintenance-enhancements` section below. 128 129For projects wishing to migrate from Python 2 to Python 3, or for library 130and framework developers wishing to support users on both Python 2 and 131Python 3, there are a variety of tools and guides available to help decide 132on a suitable approach and manage some of the technical details involved. 133The recommended starting point is the :ref:`pyporting-howto` HOWTO guide. 134 135 136Changes to the Handling of Deprecation Warnings 137=============================================== 138 139For Python 2.7, a policy decision was made to silence warnings only of 140interest to developers by default. :exc:`DeprecationWarning` and its 141descendants are now ignored unless otherwise requested, preventing 142users from seeing warnings triggered by an application. This change 143was also made in the branch that became Python 3.2. (Discussed 144on stdlib-sig and carried out in :issue:`7319`.) 145 146In previous releases, :exc:`DeprecationWarning` messages were 147enabled by default, providing Python developers with a clear 148indication of where their code may break in a future major version 149of Python. 150 151However, there are increasingly many users of Python-based 152applications who are not directly involved in the development of 153those applications. :exc:`DeprecationWarning` messages are 154irrelevant to such users, making them worry about an application 155that's actually working correctly and burdening application developers 156with responding to these concerns. 157 158You can re-enable display of :exc:`DeprecationWarning` messages by 159running Python with the :option:`-Wdefault <-W>` (short form: 160:option:`-Wd <-W>`) switch, or by setting the :envvar:`PYTHONWARNINGS` 161environment variable to ``"default"`` (or ``"d"``) before running 162Python. Python code can also re-enable them 163by calling ``warnings.simplefilter('default')``. 164 165The ``unittest`` module also automatically reenables deprecation warnings 166when running tests. 167 168 169Python 3.1 Features 170======================= 171 172Much as Python 2.6 incorporated features from Python 3.0, 173version 2.7 incorporates some of the new features 174in Python 3.1. The 2.x series continues to provide tools 175for migrating to the 3.x series. 176 177A partial list of 3.1 features that were backported to 2.7: 178 179* The syntax for set literals (``{1,2,3}`` is a mutable set). 180* Dictionary and set comprehensions (``{i: i*2 for i in range(3)}``). 181* Multiple context managers in a single :keyword:`with` statement. 182* A new version of the :mod:`io` library, rewritten in C for performance. 183* The ordered-dictionary type described in :ref:`pep-0372`. 184* The new ``","`` format specifier described in :ref:`pep-0378`. 185* The :class:`memoryview` object. 186* A small subset of the :mod:`importlib` module, 187 `described below <#importlib-section>`__. 188* The :func:`repr` of a float ``x`` is shorter in many cases: it's now 189 based on the shortest decimal string that's guaranteed to round back 190 to ``x``. As in previous versions of Python, it's guaranteed that 191 ``float(repr(x))`` recovers ``x``. 192* Float-to-string and string-to-float conversions are correctly rounded. 193 The :func:`round` function is also now correctly rounded. 194* The :c:type:`PyCapsule` type, used to provide a C API for extension modules. 195* The :c:func:`PyLong_AsLongAndOverflow` C API function. 196 197Other new Python3-mode warnings include: 198 199* :func:`operator.isCallable` and :func:`operator.sequenceIncludes`, 200 which are not supported in 3.x, now trigger warnings. 201* The :option:`-3` switch now automatically 202 enables the :option:`-Qwarn <-Q>` switch that causes warnings 203 about using classic division with integers and long integers. 204 205 206 207.. ======================================================================== 208.. Large, PEP-level features and changes should be described here. 209.. ======================================================================== 210 211.. _pep-0372: 212 213PEP 372: Adding an Ordered Dictionary to collections 214==================================================== 215 216Regular Python dictionaries iterate over key/value pairs in arbitrary order. 217Over the years, a number of authors have written alternative implementations 218that remember the order that the keys were originally inserted. Based on 219the experiences from those implementations, 2.7 introduces a new 220:class:`~collections.OrderedDict` class in the :mod:`collections` module. 221 222The :class:`~collections.OrderedDict` API provides the same interface as regular 223dictionaries but iterates over keys and values in a guaranteed order 224depending on when a key was first inserted:: 225 226 >>> from collections import OrderedDict 227 >>> d = OrderedDict([('first', 1), 228 ... ('second', 2), 229 ... ('third', 3)]) 230 >>> d.items() 231 [('first', 1), ('second', 2), ('third', 3)] 232 233If a new entry overwrites an existing entry, the original insertion 234position is left unchanged:: 235 236 >>> d['second'] = 4 237 >>> d.items() 238 [('first', 1), ('second', 4), ('third', 3)] 239 240Deleting an entry and reinserting it will move it to the end:: 241 242 >>> del d['second'] 243 >>> d['second'] = 5 244 >>> d.items() 245 [('first', 1), ('third', 3), ('second', 5)] 246 247The :meth:`~collections.OrderedDict.popitem` method has an optional *last* 248argument that defaults to ``True``. If *last* is true, the most recently 249added key is returned and removed; if it's false, the 250oldest key is selected:: 251 252 >>> od = OrderedDict([(x,0) for x in range(20)]) 253 >>> od.popitem() 254 (19, 0) 255 >>> od.popitem() 256 (18, 0) 257 >>> od.popitem(last=False) 258 (0, 0) 259 >>> od.popitem(last=False) 260 (1, 0) 261 262Comparing two ordered dictionaries checks both the keys and values, 263and requires that the insertion order was the same:: 264 265 >>> od1 = OrderedDict([('first', 1), 266 ... ('second', 2), 267 ... ('third', 3)]) 268 >>> od2 = OrderedDict([('third', 3), 269 ... ('first', 1), 270 ... ('second', 2)]) 271 >>> od1 == od2 272 False 273 >>> # Move 'third' key to the end 274 >>> del od2['third']; od2['third'] = 3 275 >>> od1 == od2 276 True 277 278Comparing an :class:`~collections.OrderedDict` with a regular dictionary 279ignores the insertion order and just compares the keys and values. 280 281How does the :class:`~collections.OrderedDict` work? It maintains a 282doubly-linked list of keys, appending new keys to the list as they're inserted. 283A secondary dictionary maps keys to their corresponding list node, so 284deletion doesn't have to traverse the entire linked list and therefore 285remains O(1). 286 287The standard library now supports use of ordered dictionaries in several 288modules. 289 290* The :mod:`ConfigParser` module uses them by default, meaning that 291 configuration files can now be read, modified, and then written back 292 in their original order. 293 294* The :meth:`~collections.somenamedtuple._asdict()` method for 295 :func:`collections.namedtuple` now returns an ordered dictionary with the 296 values appearing in the same order as the underlying tuple indices. 297 298* The :mod:`json` module's :class:`~json.JSONDecoder` class 299 constructor was extended with an *object_pairs_hook* parameter to 300 allow :class:`OrderedDict` instances to be built by the decoder. 301 Support was also added for third-party tools like 302 `PyYAML <http://pyyaml.org/>`_. 303 304.. seealso:: 305 306 :pep:`372` - Adding an ordered dictionary to collections 307 PEP written by Armin Ronacher and Raymond Hettinger; 308 implemented by Raymond Hettinger. 309 310.. _pep-0378: 311 312PEP 378: Format Specifier for Thousands Separator 313================================================= 314 315To make program output more readable, it can be useful to add 316separators to large numbers, rendering them as 31718,446,744,073,709,551,616 instead of 18446744073709551616. 318 319The fully general solution for doing this is the :mod:`locale` module, 320which can use different separators ("," in North America, "." in 321Europe) and different grouping sizes, but :mod:`locale` is complicated 322to use and unsuitable for multi-threaded applications where different 323threads are producing output for different locales. 324 325Therefore, a simple comma-grouping mechanism has been added to the 326mini-language used by the :meth:`str.format` method. When 327formatting a floating-point number, simply include a comma between the 328width and the precision:: 329 330 >>> '{:20,.2f}'.format(18446744073709551616.0) 331 '18,446,744,073,709,551,616.00' 332 333When formatting an integer, include the comma after the width: 334 335 >>> '{:20,d}'.format(18446744073709551616) 336 '18,446,744,073,709,551,616' 337 338This mechanism is not adaptable at all; commas are always used as the 339separator and the grouping is always into three-digit groups. The 340comma-formatting mechanism isn't as general as the :mod:`locale` 341module, but it's easier to use. 342 343.. seealso:: 344 345 :pep:`378` - Format Specifier for Thousands Separator 346 PEP written by Raymond Hettinger; implemented by Eric Smith. 347 348PEP 389: The argparse Module for Parsing Command Lines 349====================================================== 350 351The :mod:`argparse` module for parsing command-line arguments was 352added as a more powerful replacement for the 353:mod:`optparse` module. 354 355This means Python now supports three different modules for parsing 356command-line arguments: :mod:`getopt`, :mod:`optparse`, and 357:mod:`argparse`. The :mod:`getopt` module closely resembles the C 358library's :c:func:`getopt` function, so it remains useful if you're writing a 359Python prototype that will eventually be rewritten in C. 360:mod:`optparse` becomes redundant, but there are no plans to remove it 361because there are many scripts still using it, and there's no 362automated way to update these scripts. (Making the :mod:`argparse` 363API consistent with :mod:`optparse`'s interface was discussed but 364rejected as too messy and difficult.) 365 366In short, if you're writing a new script and don't need to worry 367about compatibility with earlier versions of Python, use 368:mod:`argparse` instead of :mod:`optparse`. 369 370Here's an example:: 371 372 import argparse 373 374 parser = argparse.ArgumentParser(description='Command-line example.') 375 376 # Add optional switches 377 parser.add_argument('-v', action='store_true', dest='is_verbose', 378 help='produce verbose output') 379 parser.add_argument('-o', action='store', dest='output', 380 metavar='FILE', 381 help='direct output to FILE instead of stdout') 382 parser.add_argument('-C', action='store', type=int, dest='context', 383 metavar='NUM', default=0, 384 help='display NUM lines of added context') 385 386 # Allow any number of additional arguments. 387 parser.add_argument(nargs='*', action='store', dest='inputs', 388 help='input filenames (default is stdin)') 389 390 args = parser.parse_args() 391 print args.__dict__ 392 393Unless you override it, :option:`!-h` and :option:`!--help` switches 394are automatically added, and produce neatly formatted output:: 395 396 -> ./python.exe argparse-example.py --help 397 usage: argparse-example.py [-h] [-v] [-o FILE] [-C NUM] [inputs [inputs ...]] 398 399 Command-line example. 400 401 positional arguments: 402 inputs input filenames (default is stdin) 403 404 optional arguments: 405 -h, --help show this help message and exit 406 -v produce verbose output 407 -o FILE direct output to FILE instead of stdout 408 -C NUM display NUM lines of added context 409 410As with :mod:`optparse`, the command-line switches and arguments 411are returned as an object with attributes named by the *dest* parameters:: 412 413 -> ./python.exe argparse-example.py -v 414 {'output': None, 415 'is_verbose': True, 416 'context': 0, 417 'inputs': []} 418 419 -> ./python.exe argparse-example.py -v -o /tmp/output -C 4 file1 file2 420 {'output': '/tmp/output', 421 'is_verbose': True, 422 'context': 4, 423 'inputs': ['file1', 'file2']} 424 425:mod:`argparse` has much fancier validation than :mod:`optparse`; you 426can specify an exact number of arguments as an integer, 0 or more 427arguments by passing ``'*'``, 1 or more by passing ``'+'``, or an 428optional argument with ``'?'``. A top-level parser can contain 429sub-parsers to define subcommands that have different sets of 430switches, as in ``svn commit``, ``svn checkout``, etc. You can 431specify an argument's type as :class:`~argparse.FileType`, which will 432automatically open files for you and understands that ``'-'`` means 433standard input or output. 434 435.. seealso:: 436 437 :mod:`argparse` documentation 438 The documentation page of the argparse module. 439 440 :ref:`argparse-from-optparse` 441 Part of the Python documentation, describing how to convert 442 code that uses :mod:`optparse`. 443 444 :pep:`389` - argparse - New Command Line Parsing Module 445 PEP written and implemented by Steven Bethard. 446 447PEP 391: Dictionary-Based Configuration For Logging 448==================================================== 449 450The :mod:`logging` module is very flexible; applications can define 451a tree of logging subsystems, and each logger in this tree can filter 452out certain messages, format them differently, and direct messages to 453a varying number of handlers. 454 455All this flexibility can require a lot of configuration. You can 456write Python statements to create objects and set their properties, 457but a complex set-up requires verbose but boring code. 458:mod:`logging` also supports a :func:`~logging.fileConfig` 459function that parses a file, but the file format doesn't support 460configuring filters, and it's messier to generate programmatically. 461 462Python 2.7 adds a :func:`~logging.dictConfig` function that 463uses a dictionary to configure logging. There are many ways to 464produce a dictionary from different sources: construct one with code; 465parse a file containing JSON; or use a YAML parsing library if one is 466installed. For more information see :ref:`logging-config-api`. 467 468The following example configures two loggers, the root logger and a 469logger named "network". Messages sent to the root logger will be 470sent to the system log using the syslog protocol, and messages 471to the "network" logger will be written to a :file:`network.log` file 472that will be rotated once the log reaches 1MB. 473 474:: 475 476 import logging 477 import logging.config 478 479 configdict = { 480 'version': 1, # Configuration schema in use; must be 1 for now 481 'formatters': { 482 'standard': { 483 'format': ('%(asctime)s %(name)-15s ' 484 '%(levelname)-8s %(message)s')}}, 485 486 'handlers': {'netlog': {'backupCount': 10, 487 'class': 'logging.handlers.RotatingFileHandler', 488 'filename': '/logs/network.log', 489 'formatter': 'standard', 490 'level': 'INFO', 491 'maxBytes': 1000000}, 492 'syslog': {'class': 'logging.handlers.SysLogHandler', 493 'formatter': 'standard', 494 'level': 'ERROR'}}, 495 496 # Specify all the subordinate loggers 497 'loggers': { 498 'network': { 499 'handlers': ['netlog'] 500 } 501 }, 502 # Specify properties of the root logger 503 'root': { 504 'handlers': ['syslog'] 505 }, 506 } 507 508 # Set up configuration 509 logging.config.dictConfig(configdict) 510 511 # As an example, log two error messages 512 logger = logging.getLogger('/') 513 logger.error('Database not found') 514 515 netlogger = logging.getLogger('network') 516 netlogger.error('Connection failed') 517 518Three smaller enhancements to the :mod:`logging` module, all 519implemented by Vinay Sajip, are: 520 521.. rev79293 522 523* The :class:`~logging.handlers.SysLogHandler` class now supports 524 syslogging over TCP. The constructor has a *socktype* parameter 525 giving the type of socket to use, either :const:`socket.SOCK_DGRAM` 526 for UDP or :const:`socket.SOCK_STREAM` for TCP. The default 527 protocol remains UDP. 528 529* :class:`~logging.Logger` instances gained a :meth:`~logging.Logger.getChild` 530 method that retrieves a descendant logger using a relative path. 531 For example, once you retrieve a logger by doing ``log = getLogger('app')``, 532 calling ``log.getChild('network.listen')`` is equivalent to 533 ``getLogger('app.network.listen')``. 534 535* The :class:`~logging.LoggerAdapter` class gained an 536 :meth:`~logging.LoggerAdapter.isEnabledFor` method that takes a 537 *level* and returns whether the underlying logger would 538 process a message of that level of importance. 539 540.. XXX: Logger objects don't have a class declaration so the link don't work 541 542.. seealso:: 543 544 :pep:`391` - Dictionary-Based Configuration For Logging 545 PEP written and implemented by Vinay Sajip. 546 547PEP 3106: Dictionary Views 548==================================================== 549 550The dictionary methods :meth:`~dict.keys`, :meth:`~dict.values`, and 551:meth:`~dict.items` are different in Python 3.x. They return an object 552called a :dfn:`view` instead of a fully materialized list. 553 554It's not possible to change the return values of :meth:`~dict.keys`, 555:meth:`~dict.values`, and :meth:`~dict.items` in Python 2.7 because 556too much code would break. Instead the 3.x versions were added 557under the new names :meth:`~dict.viewkeys`, :meth:`~dict.viewvalues`, 558and :meth:`~dict.viewitems`. 559 560:: 561 562 >>> d = dict((i*10, chr(65+i)) for i in range(26)) 563 >>> d 564 {0: 'A', 130: 'N', 10: 'B', 140: 'O', 20: ..., 250: 'Z'} 565 >>> d.viewkeys() 566 dict_keys([0, 130, 10, 140, 20, 150, 30, ..., 250]) 567 568Views can be iterated over, but the key and item views also behave 569like sets. The ``&`` operator performs intersection, and ``|`` 570performs a union:: 571 572 >>> d1 = dict((i*10, chr(65+i)) for i in range(26)) 573 >>> d2 = dict((i**.5, i) for i in range(1000)) 574 >>> d1.viewkeys() & d2.viewkeys() 575 set([0.0, 10.0, 20.0, 30.0]) 576 >>> d1.viewkeys() | range(0, 30) 577 set([0, 1, 130, 3, 4, 5, 6, ..., 120, 250]) 578 579The view keeps track of the dictionary and its contents change as the 580dictionary is modified:: 581 582 >>> vk = d.viewkeys() 583 >>> vk 584 dict_keys([0, 130, 10, ..., 250]) 585 >>> d[260] = '&' 586 >>> vk 587 dict_keys([0, 130, 260, 10, ..., 250]) 588 589However, note that you can't add or remove keys while you're iterating 590over the view:: 591 592 >>> for k in vk: 593 ... d[k*2] = k 594 ... 595 Traceback (most recent call last): 596 File "<stdin>", line 1, in <module> 597 RuntimeError: dictionary changed size during iteration 598 599You can use the view methods in Python 2.x code, and the 2to3 600converter will change them to the standard :meth:`~dict.keys`, 601:meth:`~dict.values`, and :meth:`~dict.items` methods. 602 603.. seealso:: 604 605 :pep:`3106` - Revamping dict.keys(), .values() and .items() 606 PEP written by Guido van Rossum. 607 Backported to 2.7 by Alexandre Vassalotti; :issue:`1967`. 608 609 610PEP 3137: The memoryview Object 611==================================================== 612 613The :class:`memoryview` object provides a view of another object's 614memory content that matches the :class:`bytes` type's interface. 615 616 >>> import string 617 >>> m = memoryview(string.letters) 618 >>> m 619 <memory at 0x37f850> 620 >>> len(m) # Returns length of underlying object 621 52 622 >>> m[0], m[25], m[26] # Indexing returns one byte 623 ('a', 'z', 'A') 624 >>> m2 = m[0:26] # Slicing returns another memoryview 625 >>> m2 626 <memory at 0x37f080> 627 628The content of the view can be converted to a string of bytes or 629a list of integers: 630 631 >>> m2.tobytes() 632 'abcdefghijklmnopqrstuvwxyz' 633 >>> m2.tolist() 634 [97, 98, 99, 100, 101, 102, 103, ... 121, 122] 635 >>> 636 637:class:`memoryview` objects allow modifying the underlying object if 638it's a mutable object. 639 640 >>> m2[0] = 75 641 Traceback (most recent call last): 642 File "<stdin>", line 1, in <module> 643 TypeError: cannot modify read-only memory 644 >>> b = bytearray(string.letters) # Creating a mutable object 645 >>> b 646 bytearray(b'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ') 647 >>> mb = memoryview(b) 648 >>> mb[0] = '*' # Assign to view, changing the bytearray. 649 >>> b[0:5] # The bytearray has been changed. 650 bytearray(b'*bcde') 651 >>> 652 653.. seealso:: 654 655 :pep:`3137` - Immutable Bytes and Mutable Buffer 656 PEP written by Guido van Rossum. 657 Implemented by Travis Oliphant, Antoine Pitrou and others. 658 Backported to 2.7 by Antoine Pitrou; :issue:`2396`. 659 660 661 662Other Language Changes 663====================== 664 665Some smaller changes made to the core Python language are: 666 667* The syntax for set literals has been backported from Python 3.x. 668 Curly brackets are used to surround the contents of the resulting 669 mutable set; set literals are 670 distinguished from dictionaries by not containing colons and values. 671 ``{}`` continues to represent an empty dictionary; use 672 ``set()`` for an empty set. 673 674 >>> {1, 2, 3, 4, 5} 675 set([1, 2, 3, 4, 5]) 676 >>> set() # empty set 677 set([]) 678 >>> {} # empty dict 679 {} 680 681 Backported by Alexandre Vassalotti; :issue:`2335`. 682 683* Dictionary and set comprehensions are another feature backported from 684 3.x, generalizing list/generator comprehensions to use 685 the literal syntax for sets and dictionaries. 686 687 >>> {x: x*x for x in range(6)} 688 {0: 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 25} 689 >>> {('a'*x) for x in range(6)} 690 set(['', 'a', 'aa', 'aaa', 'aaaa', 'aaaaa']) 691 692 Backported by Alexandre Vassalotti; :issue:`2333`. 693 694* The :keyword:`with` statement can now use multiple context managers 695 in one statement. Context managers are processed from left to right 696 and each one is treated as beginning a new :keyword:`with` statement. 697 This means that:: 698 699 with A() as a, B() as b: 700 ... suite of statements ... 701 702 is equivalent to:: 703 704 with A() as a: 705 with B() as b: 706 ... suite of statements ... 707 708 The :func:`contextlib.nested` function provides a very similar 709 function, so it's no longer necessary and has been deprecated. 710 711 (Proposed in https://codereview.appspot.com/53094; implemented by 712 Georg Brandl.) 713 714* Conversions between floating-point numbers and strings are 715 now correctly rounded on most platforms. These conversions occur 716 in many different places: :func:`str` on 717 floats and complex numbers; the :class:`float` and :class:`complex` 718 constructors; 719 numeric formatting; serializing and 720 deserializing floats and complex numbers using the 721 :mod:`marshal`, :mod:`pickle` 722 and :mod:`json` modules; 723 parsing of float and imaginary literals in Python code; 724 and :class:`~decimal.Decimal`-to-float conversion. 725 726 Related to this, the :func:`repr` of a floating-point number *x* 727 now returns a result based on the shortest decimal string that's 728 guaranteed to round back to *x* under correct rounding (with 729 round-half-to-even rounding mode). Previously it gave a string 730 based on rounding x to 17 decimal digits. 731 732 .. maybe add an example? 733 734 The rounding library responsible for this improvement works on 735 Windows and on Unix platforms using the gcc, icc, or suncc 736 compilers. There may be a small number of platforms where correct 737 operation of this code cannot be guaranteed, so the code is not 738 used on such systems. You can find out which code is being used 739 by checking :data:`sys.float_repr_style`, which will be ``short`` 740 if the new code is in use and ``legacy`` if it isn't. 741 742 Implemented by Eric Smith and Mark Dickinson, using David Gay's 743 :file:`dtoa.c` library; :issue:`7117`. 744 745* Conversions from long integers and regular integers to floating 746 point now round differently, returning the floating-point number 747 closest to the number. This doesn't matter for small integers that 748 can be converted exactly, but for large numbers that will 749 unavoidably lose precision, Python 2.7 now approximates more 750 closely. For example, Python 2.6 computed the following:: 751 752 >>> n = 295147905179352891391 753 >>> float(n) 754 2.9514790517935283e+20 755 >>> n - long(float(n)) 756 65535L 757 758 Python 2.7's floating-point result is larger, but much closer to the 759 true value:: 760 761 >>> n = 295147905179352891391 762 >>> float(n) 763 2.9514790517935289e+20 764 >>> n - long(float(n)) 765 -1L 766 767 (Implemented by Mark Dickinson; :issue:`3166`.) 768 769 Integer division is also more accurate in its rounding behaviours. (Also 770 implemented by Mark Dickinson; :issue:`1811`.) 771 772* Implicit coercion for complex numbers has been removed; the interpreter 773 will no longer ever attempt to call a :meth:`__coerce__` method on complex 774 objects. (Removed by Meador Inge and Mark Dickinson; :issue:`5211`.) 775 776* The :meth:`str.format` method now supports automatic numbering of the replacement 777 fields. This makes using :meth:`str.format` more closely resemble using 778 ``%s`` formatting:: 779 780 >>> '{}:{}:{}'.format(2009, 04, 'Sunday') 781 '2009:4:Sunday' 782 >>> '{}:{}:{day}'.format(2009, 4, day='Sunday') 783 '2009:4:Sunday' 784 785 The auto-numbering takes the fields from left to right, so the first ``{...}`` 786 specifier will use the first argument to :meth:`str.format`, the next 787 specifier will use the next argument, and so on. You can't mix auto-numbering 788 and explicit numbering -- either number all of your specifier fields or none 789 of them -- but you can mix auto-numbering and named fields, as in the second 790 example above. (Contributed by Eric Smith; :issue:`5237`.) 791 792 Complex numbers now correctly support usage with :func:`format`, 793 and default to being right-aligned. 794 Specifying a precision or comma-separation applies to both the real 795 and imaginary parts of the number, but a specified field width and 796 alignment is applied to the whole of the resulting ``1.5+3j`` 797 output. (Contributed by Eric Smith; :issue:`1588` and :issue:`7988`.) 798 799 The 'F' format code now always formats its output using uppercase characters, 800 so it will now produce 'INF' and 'NAN'. 801 (Contributed by Eric Smith; :issue:`3382`.) 802 803 A low-level change: the :meth:`object.__format__` method now triggers 804 a :exc:`PendingDeprecationWarning` if it's passed a format string, 805 because the :meth:`__format__` method for :class:`object` converts 806 the object to a string representation and formats that. Previously 807 the method silently applied the format string to the string 808 representation, but that could hide mistakes in Python code. If 809 you're supplying formatting information such as an alignment or 810 precision, presumably you're expecting the formatting to be applied 811 in some object-specific way. (Fixed by Eric Smith; :issue:`7994`.) 812 813* The :func:`int` and :func:`long` types gained a ``bit_length`` 814 method that returns the number of bits necessary to represent 815 its argument in binary:: 816 817 >>> n = 37 818 >>> bin(n) 819 '0b100101' 820 >>> n.bit_length() 821 6 822 >>> n = 2**123-1 823 >>> n.bit_length() 824 123 825 >>> (n+1).bit_length() 826 124 827 828 (Contributed by Fredrik Johansson and Victor Stinner; :issue:`3439`.) 829 830* The :keyword:`import` statement will no longer try an absolute import 831 if a relative import (e.g. ``from .os import sep``) fails. This 832 fixes a bug, but could possibly break certain :keyword:`import` 833 statements that were only working by accident. (Fixed by Meador Inge; 834 :issue:`7902`.) 835 836* It's now possible for a subclass of the built-in :class:`unicode` type 837 to override the :meth:`__unicode__` method. (Implemented by 838 Victor Stinner; :issue:`1583863`.) 839 840* The :class:`bytearray` type's :meth:`~bytearray.translate` method now accepts 841 ``None`` as its first argument. (Fixed by Georg Brandl; 842 :issue:`4759`.) 843 844 .. XXX bytearray doesn't seem to be documented 845 846* When using ``@classmethod`` and ``@staticmethod`` to wrap 847 methods as class or static methods, the wrapper object now 848 exposes the wrapped function as their :attr:`__func__` attribute. 849 (Contributed by Amaury Forgeot d'Arc, after a suggestion by 850 George Sakkis; :issue:`5982`.) 851 852* When a restricted set of attributes were set using ``__slots__``, 853 deleting an unset attribute would not raise :exc:`AttributeError` 854 as you would expect. Fixed by Benjamin Peterson; :issue:`7604`.) 855 856* Two new encodings are now supported: "cp720", used primarily for 857 Arabic text; and "cp858", a variant of CP 850 that adds the euro 858 symbol. (CP720 contributed by Alexander Belchenko and Amaury 859 Forgeot d'Arc in :issue:`1616979`; CP858 contributed by Tim Hatch in 860 :issue:`8016`.) 861 862* The :class:`file` object will now set the :attr:`filename` attribute 863 on the :exc:`IOError` exception when trying to open a directory 864 on POSIX platforms (noted by Jan Kaliszewski; :issue:`4764`), and 865 now explicitly checks for and forbids writing to read-only file objects 866 instead of trusting the C library to catch and report the error 867 (fixed by Stefan Krah; :issue:`5677`). 868 869* The Python tokenizer now translates line endings itself, so the 870 :func:`compile` built-in function now accepts code using any 871 line-ending convention. Additionally, it no longer requires that the 872 code end in a newline. 873 874* Extra parentheses in function definitions are illegal in Python 3.x, 875 meaning that you get a syntax error from ``def f((x)): pass``. In 876 Python3-warning mode, Python 2.7 will now warn about this odd usage. 877 (Noted by James Lingard; :issue:`7362`.) 878 879* It's now possible to create weak references to old-style class 880 objects. New-style classes were always weak-referenceable. (Fixed 881 by Antoine Pitrou; :issue:`8268`.) 882 883* When a module object is garbage-collected, the module's dictionary is 884 now only cleared if no one else is holding a reference to the 885 dictionary (:issue:`7140`). 886 887.. ====================================================================== 888 889.. _new-27-interpreter: 890 891Interpreter Changes 892------------------------------- 893 894A new environment variable, :envvar:`PYTHONWARNINGS`, 895allows controlling warnings. It should be set to a string 896containing warning settings, equivalent to those 897used with the :option:`-W` switch, separated by commas. 898(Contributed by Brian Curtin; :issue:`7301`.) 899 900For example, the following setting will print warnings every time 901they occur, but turn warnings from the :mod:`Cookie` module into an 902error. (The exact syntax for setting an environment variable varies 903across operating systems and shells.) 904 905:: 906 907 export PYTHONWARNINGS=all,error:::Cookie:0 908 909.. ====================================================================== 910 911 912Optimizations 913------------- 914 915Several performance enhancements have been added: 916 917* A new opcode was added to perform the initial setup for 918 :keyword:`with` statements, looking up the :meth:`__enter__` and 919 :meth:`__exit__` methods. (Contributed by Benjamin Peterson.) 920 921* The garbage collector now performs better for one common usage 922 pattern: when many objects are being allocated without deallocating 923 any of them. This would previously take quadratic 924 time for garbage collection, but now the number of full garbage collections 925 is reduced as the number of objects on the heap grows. 926 The new logic only performs a full garbage collection pass when 927 the middle generation has been collected 10 times and when the 928 number of survivor objects from the middle generation exceeds 10% of 929 the number of objects in the oldest generation. (Suggested by Martin 930 von Löwis and implemented by Antoine Pitrou; :issue:`4074`.) 931 932* The garbage collector tries to avoid tracking simple containers 933 which can't be part of a cycle. In Python 2.7, this is now true for 934 tuples and dicts containing atomic types (such as ints, strings, 935 etc.). Transitively, a dict containing tuples of atomic types won't 936 be tracked either. This helps reduce the cost of each 937 garbage collection by decreasing the number of objects to be 938 considered and traversed by the collector. 939 (Contributed by Antoine Pitrou; :issue:`4688`.) 940 941* Long integers are now stored internally either in base 2**15 or in base 942 2**30, the base being determined at build time. Previously, they 943 were always stored in base 2**15. Using base 2**30 gives 944 significant performance improvements on 64-bit machines, but 945 benchmark results on 32-bit machines have been mixed. Therefore, 946 the default is to use base 2**30 on 64-bit machines and base 2**15 947 on 32-bit machines; on Unix, there's a new configure option 948 :option:`!--enable-big-digits` that can be used to override this default. 949 950 Apart from the performance improvements this change should be 951 invisible to end users, with one exception: for testing and 952 debugging purposes there's a new structseq :data:`sys.long_info` that 953 provides information about the internal format, giving the number of 954 bits per digit and the size in bytes of the C type used to store 955 each digit:: 956 957 >>> import sys 958 >>> sys.long_info 959 sys.long_info(bits_per_digit=30, sizeof_digit=4) 960 961 (Contributed by Mark Dickinson; :issue:`4258`.) 962 963 Another set of changes made long objects a few bytes smaller: 2 bytes 964 smaller on 32-bit systems and 6 bytes on 64-bit. 965 (Contributed by Mark Dickinson; :issue:`5260`.) 966 967* The division algorithm for long integers has been made faster 968 by tightening the inner loop, doing shifts instead of multiplications, 969 and fixing an unnecessary extra iteration. 970 Various benchmarks show speedups of between 50% and 150% for long 971 integer divisions and modulo operations. 972 (Contributed by Mark Dickinson; :issue:`5512`.) 973 Bitwise operations are also significantly faster (initial patch by 974 Gregory Smith; :issue:`1087418`). 975 976* The implementation of ``%`` checks for the left-side operand being 977 a Python string and special-cases it; this results in a 1--3% 978 performance increase for applications that frequently use ``%`` 979 with strings, such as templating libraries. 980 (Implemented by Collin Winter; :issue:`5176`.) 981 982* List comprehensions with an ``if`` condition are compiled into 983 faster bytecode. (Patch by Antoine Pitrou, back-ported to 2.7 984 by Jeffrey Yasskin; :issue:`4715`.) 985 986* Converting an integer or long integer to a decimal string was made 987 faster by special-casing base 10 instead of using a generalized 988 conversion function that supports arbitrary bases. 989 (Patch by Gawain Bolton; :issue:`6713`.) 990 991* The :meth:`split`, :meth:`replace`, :meth:`rindex`, 992 :meth:`rpartition`, and :meth:`rsplit` methods of string-like types 993 (strings, Unicode strings, and :class:`bytearray` objects) now use a 994 fast reverse-search algorithm instead of a character-by-character 995 scan. This is sometimes faster by a factor of 10. (Added by 996 Florent Xicluna; :issue:`7462` and :issue:`7622`.) 997 998* The :mod:`pickle` and :mod:`cPickle` modules now automatically 999 intern the strings used for attribute names, reducing memory usage 1000 of the objects resulting from unpickling. (Contributed by Jake 1001 McGuire; :issue:`5084`.) 1002 1003* The :mod:`cPickle` module now special-cases dictionaries, 1004 nearly halving the time required to pickle them. 1005 (Contributed by Collin Winter; :issue:`5670`.) 1006 1007.. ====================================================================== 1008 1009New and Improved Modules 1010======================== 1011 1012As in every release, Python's standard library received a number of 1013enhancements and bug fixes. Here's a partial list of the most notable 1014changes, sorted alphabetically by module name. Consult the 1015:file:`Misc/NEWS` file in the source tree for a more complete list of 1016changes, or look through the Subversion logs for all the details. 1017 1018* The :mod:`bdb` module's base debugging class :class:`~bdb.Bdb` 1019 gained a feature for skipping modules. The constructor 1020 now takes an iterable containing glob-style patterns such as 1021 ``django.*``; the debugger will not step into stack frames 1022 from a module that matches one of these patterns. 1023 (Contributed by Maru Newby after a suggestion by 1024 Senthil Kumaran; :issue:`5142`.) 1025 1026* The :mod:`binascii` module now supports the buffer API, so it can be 1027 used with :class:`memoryview` instances and other similar buffer objects. 1028 (Backported from 3.x by Florent Xicluna; :issue:`7703`.) 1029 1030* Updated module: the :mod:`bsddb` module has been updated from 4.7.2devel9 1031 to version 4.8.4 of 1032 `the pybsddb package <https://www.jcea.es/programacion/pybsddb.htm>`__. 1033 The new version features better Python 3.x compatibility, various bug fixes, 1034 and adds several new BerkeleyDB flags and methods. 1035 (Updated by Jesús Cea Avión; :issue:`8156`. The pybsddb 1036 changelog can be read at http://hg.jcea.es/pybsddb/file/tip/ChangeLog.) 1037 1038* The :mod:`bz2` module's :class:`~bz2.BZ2File` now supports the context 1039 management protocol, so you can write ``with bz2.BZ2File(...) as f:``. 1040 (Contributed by Hagen Fürstenau; :issue:`3860`.) 1041 1042* New class: the :class:`~collections.Counter` class in the :mod:`collections` 1043 module is useful for tallying data. :class:`~collections.Counter` instances 1044 behave mostly like dictionaries but return zero for missing keys instead of 1045 raising a :exc:`KeyError`: 1046 1047 .. doctest:: 1048 :options: +NORMALIZE_WHITESPACE 1049 1050 >>> from collections import Counter 1051 >>> c = Counter() 1052 >>> for letter in 'here is a sample of english text': 1053 ... c[letter] += 1 1054 ... 1055 >>> c 1056 Counter({' ': 6, 'e': 5, 's': 3, 'a': 2, 'i': 2, 'h': 2, 1057 'l': 2, 't': 2, 'g': 1, 'f': 1, 'm': 1, 'o': 1, 'n': 1, 1058 'p': 1, 'r': 1, 'x': 1}) 1059 >>> c['e'] 1060 5 1061 >>> c['z'] 1062 0 1063 1064 There are three additional :class:`~collections.Counter` methods. 1065 :meth:`~collections.Counter.most_common` returns the N most common 1066 elements and their counts. :meth:`~collections.Counter.elements` 1067 returns an iterator over the contained elements, repeating each 1068 element as many times as its count. 1069 :meth:`~collections.Counter.subtract` takes an iterable and 1070 subtracts one for each element instead of adding; if the argument is 1071 a dictionary or another :class:`Counter`, the counts are 1072 subtracted. :: 1073 1074 >>> c.most_common(5) 1075 [(' ', 6), ('e', 5), ('s', 3), ('a', 2), ('i', 2)] 1076 >>> c.elements() -> 1077 'a', 'a', ' ', ' ', ' ', ' ', ' ', ' ', 1078 'e', 'e', 'e', 'e', 'e', 'g', 'f', 'i', 'i', 1079 'h', 'h', 'm', 'l', 'l', 'o', 'n', 'p', 's', 1080 's', 's', 'r', 't', 't', 'x' 1081 >>> c['e'] 1082 5 1083 >>> c.subtract('very heavy on the letter e') 1084 >>> c['e'] # Count is now lower 1085 -1 1086 1087 Contributed by Raymond Hettinger; :issue:`1696199`. 1088 1089 .. revision 79660 1090 1091 New class: :class:`~collections.OrderedDict` is described in the earlier 1092 section :ref:`pep-0372`. 1093 1094 New method: The :class:`~collections.deque` data type now has a 1095 :meth:`~collections.deque.count` method that returns the number of 1096 contained elements equal to the supplied argument *x*, and a 1097 :meth:`~collections.deque.reverse` method that reverses the elements 1098 of the deque in-place. :class:`~collections.deque` also exposes its maximum 1099 length as the read-only :attr:`~collections.deque.maxlen` attribute. 1100 (Both features added by Raymond Hettinger.) 1101 1102 The :class:`~collections.namedtuple` class now has an optional *rename* parameter. 1103 If *rename* is true, field names that are invalid because they've 1104 been repeated or aren't legal Python identifiers will be 1105 renamed to legal names that are derived from the field's 1106 position within the list of fields: 1107 1108 >>> from collections import namedtuple 1109 >>> T = namedtuple('T', ['field1', '$illegal', 'for', 'field2'], rename=True) 1110 >>> T._fields 1111 ('field1', '_1', '_2', 'field2') 1112 1113 (Added by Raymond Hettinger; :issue:`1818`.) 1114 1115 Finally, the :class:`~collections.Mapping` abstract base class now 1116 returns :const:`NotImplemented` if a mapping is compared to 1117 another type that isn't a :class:`Mapping`. 1118 (Fixed by Daniel Stutzbach; :issue:`8729`.) 1119 1120* Constructors for the parsing classes in the :mod:`ConfigParser` module now 1121 take an *allow_no_value* parameter, defaulting to false; if true, 1122 options without values will be allowed. For example:: 1123 1124 >>> import ConfigParser, StringIO 1125 >>> sample_config = """ 1126 ... [mysqld] 1127 ... user = mysql 1128 ... pid-file = /var/run/mysqld/mysqld.pid 1129 ... skip-bdb 1130 ... """ 1131 >>> config = ConfigParser.RawConfigParser(allow_no_value=True) 1132 >>> config.readfp(StringIO.StringIO(sample_config)) 1133 >>> config.get('mysqld', 'user') 1134 'mysql' 1135 >>> print config.get('mysqld', 'skip-bdb') 1136 None 1137 >>> print config.get('mysqld', 'unknown') 1138 Traceback (most recent call last): 1139 ... 1140 NoOptionError: No option 'unknown' in section: 'mysqld' 1141 1142 (Contributed by Mats Kindahl; :issue:`7005`.) 1143 1144* Deprecated function: :func:`contextlib.nested`, which allows 1145 handling more than one context manager with a single :keyword:`with` 1146 statement, has been deprecated, because the :keyword:`with` statement 1147 now supports multiple context managers. 1148 1149* The :mod:`cookielib` module now ignores cookies that have an invalid 1150 version field, one that doesn't contain an integer value. (Fixed by 1151 John J. Lee; :issue:`3924`.) 1152 1153* The :mod:`copy` module's :func:`~copy.deepcopy` function will now 1154 correctly copy bound instance methods. (Implemented by 1155 Robert Collins; :issue:`1515`.) 1156 1157* The :mod:`ctypes` module now always converts ``None`` to a C NULL 1158 pointer for arguments declared as pointers. (Changed by Thomas 1159 Heller; :issue:`4606`.) The underlying `libffi library 1160 <https://sourceware.org/libffi/>`__ has been updated to version 1161 3.0.9, containing various fixes for different platforms. (Updated 1162 by Matthias Klose; :issue:`8142`.) 1163 1164* New method: the :mod:`datetime` module's :class:`~datetime.timedelta` class 1165 gained a :meth:`~datetime.timedelta.total_seconds` method that returns the 1166 number of seconds in the duration. (Contributed by Brian Quinlan; :issue:`5788`.) 1167 1168* New method: the :class:`~decimal.Decimal` class gained a 1169 :meth:`~decimal.Decimal.from_float` class method that performs an exact 1170 conversion of a floating-point number to a :class:`~decimal.Decimal`. 1171 This exact conversion strives for the 1172 closest decimal approximation to the floating-point representation's value; 1173 the resulting decimal value will therefore still include the inaccuracy, 1174 if any. 1175 For example, ``Decimal.from_float(0.1)`` returns 1176 ``Decimal('0.1000000000000000055511151231257827021181583404541015625')``. 1177 (Implemented by Raymond Hettinger; :issue:`4796`.) 1178 1179 Comparing instances of :class:`~decimal.Decimal` with floating-point 1180 numbers now produces sensible results based on the numeric values 1181 of the operands. Previously such comparisons would fall back to 1182 Python's default rules for comparing objects, which produced arbitrary 1183 results based on their type. Note that you still cannot combine 1184 :class:`Decimal` and floating-point in other operations such as addition, 1185 since you should be explicitly choosing how to convert between float and 1186 :class:`~decimal.Decimal`. (Fixed by Mark Dickinson; :issue:`2531`.) 1187 1188 The constructor for :class:`~decimal.Decimal` now accepts 1189 floating-point numbers (added by Raymond Hettinger; :issue:`8257`) 1190 and non-European Unicode characters such as Arabic-Indic digits 1191 (contributed by Mark Dickinson; :issue:`6595`). 1192 1193 Most of the methods of the :class:`~decimal.Context` class now accept integers 1194 as well as :class:`~decimal.Decimal` instances; the only exceptions are the 1195 :meth:`~decimal.Context.canonical` and :meth:`~decimal.Context.is_canonical` 1196 methods. (Patch by Juan José Conti; :issue:`7633`.) 1197 1198 When using :class:`~decimal.Decimal` instances with a string's 1199 :meth:`~str.format` method, the default alignment was previously 1200 left-alignment. This has been changed to right-alignment, which is 1201 more sensible for numeric types. (Changed by Mark Dickinson; :issue:`6857`.) 1202 1203 Comparisons involving a signaling NaN value (or ``sNAN``) now signal 1204 :const:`InvalidOperation` instead of silently returning a true or 1205 false value depending on the comparison operator. Quiet NaN values 1206 (or ``NaN``) are now hashable. (Fixed by Mark Dickinson; 1207 :issue:`7279`.) 1208 1209* The :mod:`difflib` module now produces output that is more 1210 compatible with modern :command:`diff`/:command:`patch` tools 1211 through one small change, using a tab character instead of spaces as 1212 a separator in the header giving the filename. (Fixed by Anatoly 1213 Techtonik; :issue:`7585`.) 1214 1215* The Distutils ``sdist`` command now always regenerates the 1216 :file:`MANIFEST` file, since even if the :file:`MANIFEST.in` or 1217 :file:`setup.py` files haven't been modified, the user might have 1218 created some new files that should be included. 1219 (Fixed by Tarek Ziadé; :issue:`8688`.) 1220 1221* The :mod:`doctest` module's :const:`IGNORE_EXCEPTION_DETAIL` flag 1222 will now ignore the name of the module containing the exception 1223 being tested. (Patch by Lennart Regebro; :issue:`7490`.) 1224 1225* The :mod:`email` module's :class:`~email.message.Message` class will 1226 now accept a Unicode-valued payload, automatically converting the 1227 payload to the encoding specified by :attr:`output_charset`. 1228 (Added by R. David Murray; :issue:`1368247`.) 1229 1230* The :class:`~fractions.Fraction` class now accepts a single float or 1231 :class:`~decimal.Decimal` instance, or two rational numbers, as 1232 arguments to its constructor. (Implemented by Mark Dickinson; 1233 rationals added in :issue:`5812`, and float/decimal in 1234 :issue:`8294`.) 1235 1236 Ordering comparisons (``<``, ``<=``, ``>``, ``>=``) between 1237 fractions and complex numbers now raise a :exc:`TypeError`. 1238 This fixes an oversight, making the :class:`~fractions.Fraction` 1239 match the other numeric types. 1240 1241 .. revision 79455 1242 1243* New class: :class:`~ftplib.FTP_TLS` in 1244 the :mod:`ftplib` module provides secure FTP 1245 connections using TLS encapsulation of authentication as well as 1246 subsequent control and data transfers. 1247 (Contributed by Giampaolo Rodola; :issue:`2054`.) 1248 1249 The :meth:`~ftplib.FTP.storbinary` method for binary uploads can now restart 1250 uploads thanks to an added *rest* parameter (patch by Pablo Mouzo; 1251 :issue:`6845`.) 1252 1253* New class decorator: :func:`~functools.total_ordering` in the :mod:`functools` 1254 module takes a class that defines an :meth:`__eq__` method and one of 1255 :meth:`__lt__`, :meth:`__le__`, :meth:`__gt__`, or :meth:`__ge__`, 1256 and generates the missing comparison methods. Since the 1257 :meth:`__cmp__` method is being deprecated in Python 3.x, 1258 this decorator makes it easier to define ordered classes. 1259 (Added by Raymond Hettinger; :issue:`5479`.) 1260 1261 New function: :func:`~functools.cmp_to_key` will take an old-style comparison 1262 function that expects two arguments and return a new callable that 1263 can be used as the *key* parameter to functions such as 1264 :func:`sorted`, :func:`min` and :func:`max`, etc. The primary 1265 intended use is to help with making code compatible with Python 3.x. 1266 (Added by Raymond Hettinger.) 1267 1268* New function: the :mod:`gc` module's :func:`~gc.is_tracked` returns 1269 true if a given instance is tracked by the garbage collector, false 1270 otherwise. (Contributed by Antoine Pitrou; :issue:`4688`.) 1271 1272* The :mod:`gzip` module's :class:`~gzip.GzipFile` now supports the context 1273 management protocol, so you can write ``with gzip.GzipFile(...) as f:`` 1274 (contributed by Hagen Fürstenau; :issue:`3860`), and it now implements 1275 the :class:`io.BufferedIOBase` ABC, so you can wrap it with 1276 :class:`io.BufferedReader` for faster processing 1277 (contributed by Nir Aides; :issue:`7471`). 1278 It's also now possible to override the modification time 1279 recorded in a gzipped file by providing an optional timestamp to 1280 the constructor. (Contributed by Jacques Frechet; :issue:`4272`.) 1281 1282 Files in gzip format can be padded with trailing zero bytes; the 1283 :mod:`gzip` module will now consume these trailing bytes. (Fixed by 1284 Tadek Pietraszek and Brian Curtin; :issue:`2846`.) 1285 1286* New attribute: the :mod:`hashlib` module now has an :attr:`~hashlib.hashlib.algorithms` 1287 attribute containing a tuple naming the supported algorithms. 1288 In Python 2.7, ``hashlib.algorithms`` contains 1289 ``('md5', 'sha1', 'sha224', 'sha256', 'sha384', 'sha512')``. 1290 (Contributed by Carl Chenet; :issue:`7418`.) 1291 1292* The default :class:`~httplib.HTTPResponse` class used by the :mod:`httplib` module now 1293 supports buffering, resulting in much faster reading of HTTP responses. 1294 (Contributed by Kristján Valur Jónsson; :issue:`4879`.) 1295 1296 The :class:`~httplib.HTTPConnection` and :class:`~httplib.HTTPSConnection` classes 1297 now support a *source_address* parameter, a ``(host, port)`` 2-tuple 1298 giving the source address that will be used for the connection. 1299 (Contributed by Eldon Ziegler; :issue:`3972`.) 1300 1301* The :mod:`ihooks` module now supports relative imports. Note that 1302 :mod:`ihooks` is an older module for customizing imports, 1303 superseded by the :mod:`imputil` module added in Python 2.0. 1304 (Relative import support added by Neil Schemenauer.) 1305 1306 .. revision 75423 1307 1308* The :mod:`imaplib` module now supports IPv6 addresses. 1309 (Contributed by Derek Morr; :issue:`1655`.) 1310 1311* New function: the :mod:`inspect` module's :func:`~inspect.getcallargs` 1312 takes a callable and its positional and keyword arguments, 1313 and figures out which of the callable's parameters will receive each argument, 1314 returning a dictionary mapping argument names to their values. For example:: 1315 1316 >>> from inspect import getcallargs 1317 >>> def f(a, b=1, *pos, **named): 1318 ... pass 1319 >>> getcallargs(f, 1, 2, 3) 1320 {'a': 1, 'b': 2, 'pos': (3,), 'named': {}} 1321 >>> getcallargs(f, a=2, x=4) 1322 {'a': 2, 'b': 1, 'pos': (), 'named': {'x': 4}} 1323 >>> getcallargs(f) 1324 Traceback (most recent call last): 1325 ... 1326 TypeError: f() takes at least 1 argument (0 given) 1327 1328 Contributed by George Sakkis; :issue:`3135`. 1329 1330* Updated module: The :mod:`io` library has been upgraded to the version shipped with 1331 Python 3.1. For 3.1, the I/O library was entirely rewritten in C 1332 and is 2 to 20 times faster depending on the task being performed. The 1333 original Python version was renamed to the :mod:`_pyio` module. 1334 1335 One minor resulting change: the :class:`io.TextIOBase` class now 1336 has an :attr:`errors` attribute giving the error setting 1337 used for encoding and decoding errors (one of ``'strict'``, ``'replace'``, 1338 ``'ignore'``). 1339 1340 The :class:`io.FileIO` class now raises an :exc:`OSError` when passed 1341 an invalid file descriptor. (Implemented by Benjamin Peterson; 1342 :issue:`4991`.) The :meth:`~io.IOBase.truncate` method now preserves the 1343 file position; previously it would change the file position to the 1344 end of the new file. (Fixed by Pascal Chambon; :issue:`6939`.) 1345 1346* New function: ``itertools.compress(data, selectors)`` takes two 1347 iterators. Elements of *data* are returned if the corresponding 1348 value in *selectors* is true:: 1349 1350 itertools.compress('ABCDEF', [1,0,1,0,1,1]) => 1351 A, C, E, F 1352 1353 .. maybe here is better to use >>> list(itertools.compress(...)) instead 1354 1355 New function: ``itertools.combinations_with_replacement(iter, r)`` 1356 returns all the possible *r*-length combinations of elements from the 1357 iterable *iter*. Unlike :func:`~itertools.combinations`, individual elements 1358 can be repeated in the generated combinations:: 1359 1360 itertools.combinations_with_replacement('abc', 2) => 1361 ('a', 'a'), ('a', 'b'), ('a', 'c'), 1362 ('b', 'b'), ('b', 'c'), ('c', 'c') 1363 1364 Note that elements are treated as unique depending on their position 1365 in the input, not their actual values. 1366 1367 The :func:`itertools.count` function now has a *step* argument that 1368 allows incrementing by values other than 1. :func:`~itertools.count` also 1369 now allows keyword arguments, and using non-integer values such as 1370 floats or :class:`~decimal.Decimal` instances. (Implemented by Raymond 1371 Hettinger; :issue:`5032`.) 1372 1373 :func:`itertools.combinations` and :func:`itertools.product` 1374 previously raised :exc:`ValueError` for values of *r* larger than 1375 the input iterable. This was deemed a specification error, so they 1376 now return an empty iterator. (Fixed by Raymond Hettinger; :issue:`4816`.) 1377 1378* Updated module: The :mod:`json` module was upgraded to version 2.0.9 of the 1379 simplejson package, which includes a C extension that makes 1380 encoding and decoding faster. 1381 (Contributed by Bob Ippolito; :issue:`4136`.) 1382 1383 To support the new :class:`collections.OrderedDict` type, :func:`json.load` 1384 now has an optional *object_pairs_hook* parameter that will be called 1385 with any object literal that decodes to a list of pairs. 1386 (Contributed by Raymond Hettinger; :issue:`5381`.) 1387 1388* The :mod:`mailbox` module's :class:`~mailbox.Maildir` class now records the 1389 timestamp on the directories it reads, and only re-reads them if the 1390 modification time has subsequently changed. This improves 1391 performance by avoiding unneeded directory scans. (Fixed by 1392 A.M. Kuchling and Antoine Pitrou; :issue:`1607951`, :issue:`6896`.) 1393 1394* New functions: the :mod:`math` module gained 1395 :func:`~math.erf` and :func:`~math.erfc` for the error function and the complementary error function, 1396 :func:`~math.expm1` which computes ``e**x - 1`` with more precision than 1397 using :func:`~math.exp` and subtracting 1, 1398 :func:`~math.gamma` for the Gamma function, and 1399 :func:`~math.lgamma` for the natural log of the Gamma function. 1400 (Contributed by Mark Dickinson and nirinA raseliarison; :issue:`3366`.) 1401 1402* The :mod:`multiprocessing` module's :class:`Manager*` classes 1403 can now be passed a callable that will be called whenever 1404 a subprocess is started, along with a set of arguments that will be 1405 passed to the callable. 1406 (Contributed by lekma; :issue:`5585`.) 1407 1408 The :class:`~multiprocessing.Pool` class, which controls a pool of worker processes, 1409 now has an optional *maxtasksperchild* parameter. Worker processes 1410 will perform the specified number of tasks and then exit, causing the 1411 :class:`~multiprocessing.Pool` to start a new worker. This is useful if tasks may leak 1412 memory or other resources, or if some tasks will cause the worker to 1413 become very large. 1414 (Contributed by Charles Cazabon; :issue:`6963`.) 1415 1416* The :mod:`nntplib` module now supports IPv6 addresses. 1417 (Contributed by Derek Morr; :issue:`1664`.) 1418 1419* New functions: the :mod:`os` module wraps the following POSIX system 1420 calls: :func:`~os.getresgid` and :func:`~os.getresuid`, which return the 1421 real, effective, and saved GIDs and UIDs; 1422 :func:`~os.setresgid` and :func:`~os.setresuid`, which set 1423 real, effective, and saved GIDs and UIDs to new values; 1424 :func:`~os.initgroups`, which initialize the group access list 1425 for the current process. (GID/UID functions 1426 contributed by Travis H.; :issue:`6508`. Support for initgroups added 1427 by Jean-Paul Calderone; :issue:`7333`.) 1428 1429 The :func:`os.fork` function now re-initializes the import lock in 1430 the child process; this fixes problems on Solaris when :func:`~os.fork` 1431 is called from a thread. (Fixed by Zsolt Cserna; :issue:`7242`.) 1432 1433* In the :mod:`os.path` module, the :func:`~os.path.normpath` and 1434 :func:`~os.path.abspath` functions now preserve Unicode; if their input path 1435 is a Unicode string, the return value is also a Unicode string. 1436 (:meth:`~os.path.normpath` fixed by Matt Giuca in :issue:`5827`; 1437 :meth:`~os.path.abspath` fixed by Ezio Melotti in :issue:`3426`.) 1438 1439* The :mod:`pydoc` module now has help for the various symbols that Python 1440 uses. You can now do ``help('<<')`` or ``help('@')``, for example. 1441 (Contributed by David Laban; :issue:`4739`.) 1442 1443* The :mod:`re` module's :func:`~re.split`, :func:`~re.sub`, and :func:`~re.subn` 1444 now accept an optional *flags* argument, for consistency with the 1445 other functions in the module. (Added by Gregory P. Smith.) 1446 1447* New function: :func:`~runpy.run_path` in the :mod:`runpy` module 1448 will execute the code at a provided *path* argument. *path* can be 1449 the path of a Python source file (:file:`example.py`), a compiled 1450 bytecode file (:file:`example.pyc`), a directory 1451 (:file:`./package/`), or a zip archive (:file:`example.zip`). If a 1452 directory or zip path is provided, it will be added to the front of 1453 ``sys.path`` and the module :mod:`__main__` will be imported. It's 1454 expected that the directory or zip contains a :file:`__main__.py`; 1455 if it doesn't, some other :file:`__main__.py` might be imported from 1456 a location later in ``sys.path``. This makes more of the machinery 1457 of :mod:`runpy` available to scripts that want to mimic the way 1458 Python's command line processes an explicit path name. 1459 (Added by Nick Coghlan; :issue:`6816`.) 1460 1461* New function: in the :mod:`shutil` module, :func:`~shutil.make_archive` 1462 takes a filename, archive type (zip or tar-format), and a directory 1463 path, and creates an archive containing the directory's contents. 1464 (Added by Tarek Ziadé.) 1465 1466 :mod:`shutil`'s :func:`~shutil.copyfile` and :func:`~shutil.copytree` 1467 functions now raise a :exc:`~shutil.SpecialFileError` exception when 1468 asked to copy a named pipe. Previously the code would treat 1469 named pipes like a regular file by opening them for reading, and 1470 this would block indefinitely. (Fixed by Antoine Pitrou; :issue:`3002`.) 1471 1472* The :mod:`signal` module no longer re-installs the signal handler 1473 unless this is truly necessary, which fixes a bug that could make it 1474 impossible to catch the EINTR signal robustly. (Fixed by 1475 Charles-Francois Natali; :issue:`8354`.) 1476 1477* New functions: in the :mod:`site` module, three new functions 1478 return various site- and user-specific paths. 1479 :func:`~site.getsitepackages` returns a list containing all 1480 global site-packages directories, 1481 :func:`~site.getusersitepackages` returns the path of the user's 1482 site-packages directory, and 1483 :func:`~site.getuserbase` returns the value of the :envvar:`USER_BASE` 1484 environment variable, giving the path to a directory that can be used 1485 to store data. 1486 (Contributed by Tarek Ziadé; :issue:`6693`.) 1487 1488 The :mod:`site` module now reports exceptions occurring 1489 when the :mod:`sitecustomize` module is imported, and will no longer 1490 catch and swallow the :exc:`KeyboardInterrupt` exception. (Fixed by 1491 Victor Stinner; :issue:`3137`.) 1492 1493* The :func:`~socket.create_connection` function 1494 gained a *source_address* parameter, a ``(host, port)`` 2-tuple 1495 giving the source address that will be used for the connection. 1496 (Contributed by Eldon Ziegler; :issue:`3972`.) 1497 1498 The :meth:`~socket.socket.recv_into` and :meth:`~socket.socket.recvfrom_into` 1499 methods will now write into objects that support the buffer API, most usefully 1500 the :class:`bytearray` and :class:`memoryview` objects. (Implemented by 1501 Antoine Pitrou; :issue:`8104`.) 1502 1503* The :mod:`SocketServer` module's :class:`~SocketServer.TCPServer` class now 1504 supports socket timeouts and disabling the Nagle algorithm. 1505 The :attr:`~SocketServer.TCPServer.disable_nagle_algorithm` class attribute 1506 defaults to ``False``; if overridden to be true, 1507 new request connections will have the TCP_NODELAY option set to 1508 prevent buffering many small sends into a single TCP packet. 1509 The :attr:`~SocketServer.BaseServer.timeout` class attribute can hold 1510 a timeout in seconds that will be applied to the request socket; if 1511 no request is received within that time, :meth:`~SocketServer.BaseServer.handle_timeout` 1512 will be called and :meth:`~SocketServer.BaseServer.handle_request` will return. 1513 (Contributed by Kristján Valur Jónsson; :issue:`6192` and :issue:`6267`.) 1514 1515* Updated module: the :mod:`sqlite3` module has been updated to 1516 version 2.6.0 of the `pysqlite package <https://github.com/ghaering/pysqlite>`__. Version 2.6.0 includes a number of bugfixes, and adds 1517 the ability to load SQLite extensions from shared libraries. 1518 Call the ``enable_load_extension(True)`` method to enable extensions, 1519 and then call :meth:`~sqlite3.Connection.load_extension` to load a particular shared library. 1520 (Updated by Gerhard Häring.) 1521 1522* The :mod:`ssl` module's :class:`~ssl.SSLSocket` objects now support the 1523 buffer API, which fixed a test suite failure (fix by Antoine Pitrou; 1524 :issue:`7133`) and automatically set 1525 OpenSSL's :c:macro:`SSL_MODE_AUTO_RETRY`, which will prevent an error 1526 code being returned from :meth:`recv` operations that trigger an SSL 1527 renegotiation (fix by Antoine Pitrou; :issue:`8222`). 1528 1529 The :func:`ssl.wrap_socket` constructor function now takes a 1530 *ciphers* argument that's a string listing the encryption algorithms 1531 to be allowed; the format of the string is described 1532 `in the OpenSSL documentation 1533 <https://www.openssl.org/docs/apps/ciphers.html#CIPHER-LIST-FORMAT>`__. 1534 (Added by Antoine Pitrou; :issue:`8322`.) 1535 1536 Another change makes the extension load all of OpenSSL's ciphers and 1537 digest algorithms so that they're all available. Some SSL 1538 certificates couldn't be verified, reporting an "unknown algorithm" 1539 error. (Reported by Beda Kosata, and fixed by Antoine Pitrou; 1540 :issue:`8484`.) 1541 1542 The version of OpenSSL being used is now available as the module 1543 attributes :data:`ssl.OPENSSL_VERSION` (a string), 1544 :data:`ssl.OPENSSL_VERSION_INFO` (a 5-tuple), and 1545 :data:`ssl.OPENSSL_VERSION_NUMBER` (an integer). (Added by Antoine 1546 Pitrou; :issue:`8321`.) 1547 1548* The :mod:`struct` module will no longer silently ignore overflow 1549 errors when a value is too large for a particular integer format 1550 code (one of ``bBhHiIlLqQ``); it now always raises a 1551 :exc:`struct.error` exception. (Changed by Mark Dickinson; 1552 :issue:`1523`.) The :func:`~struct.pack` function will also 1553 attempt to use :meth:`__index__` to convert and pack non-integers 1554 before trying the :meth:`__int__` method or reporting an error. 1555 (Changed by Mark Dickinson; :issue:`8300`.) 1556 1557* New function: the :mod:`subprocess` module's 1558 :func:`~subprocess.check_output` runs a command with a specified set of arguments 1559 and returns the command's output as a string when the command runs without 1560 error, or raises a :exc:`~subprocess.CalledProcessError` exception otherwise. 1561 1562 :: 1563 1564 >>> subprocess.check_output(['df', '-h', '.']) 1565 'Filesystem Size Used Avail Capacity Mounted on\n 1566 /dev/disk0s2 52G 49G 3.0G 94% /\n' 1567 1568 >>> subprocess.check_output(['df', '-h', '/bogus']) 1569 ... 1570 subprocess.CalledProcessError: Command '['df', '-h', '/bogus']' returned non-zero exit status 1 1571 1572 (Contributed by Gregory P. Smith.) 1573 1574 The :mod:`subprocess` module will now retry its internal system calls 1575 on receiving an :const:`EINTR` signal. (Reported by several people; final 1576 patch by Gregory P. Smith in :issue:`1068268`.) 1577 1578* New function: :func:`~symtable.Symbol.is_declared_global` in the :mod:`symtable` module 1579 returns true for variables that are explicitly declared to be global, 1580 false for ones that are implicitly global. 1581 (Contributed by Jeremy Hylton.) 1582 1583* The :mod:`syslog` module will now use the value of ``sys.argv[0]`` as the 1584 identifier instead of the previous default value of ``'python'``. 1585 (Changed by Sean Reifschneider; :issue:`8451`.) 1586 1587* The ``sys.version_info`` value is now a named tuple, with attributes 1588 named :attr:`major`, :attr:`minor`, :attr:`micro`, 1589 :attr:`releaselevel`, and :attr:`serial`. (Contributed by Ross 1590 Light; :issue:`4285`.) 1591 1592 :func:`sys.getwindowsversion` also returns a named tuple, 1593 with attributes named :attr:`major`, :attr:`minor`, :attr:`build`, 1594 :attr:`platform`, :attr:`service_pack`, :attr:`service_pack_major`, 1595 :attr:`service_pack_minor`, :attr:`suite_mask`, and 1596 :attr:`product_type`. (Contributed by Brian Curtin; :issue:`7766`.) 1597 1598* The :mod:`tarfile` module's default error handling has changed, to 1599 no longer suppress fatal errors. The default error level was previously 0, 1600 which meant that errors would only result in a message being written to the 1601 debug log, but because the debug log is not activated by default, 1602 these errors go unnoticed. The default error level is now 1, 1603 which raises an exception if there's an error. 1604 (Changed by Lars Gustäbel; :issue:`7357`.) 1605 1606 :mod:`tarfile` now supports filtering the :class:`~tarfile.TarInfo` 1607 objects being added to a tar file. When you call :meth:`~tarfile.TarFile.add`, 1608 you may supply an optional *filter* argument 1609 that's a callable. The *filter* callable will be passed the 1610 :class:`~tarfile.TarInfo` for every file being added, and can modify and return it. 1611 If the callable returns ``None``, the file will be excluded from the 1612 resulting archive. This is more powerful than the existing 1613 *exclude* argument, which has therefore been deprecated. 1614 (Added by Lars Gustäbel; :issue:`6856`.) 1615 The :class:`~tarfile.TarFile` class also now supports the context management protocol. 1616 (Added by Lars Gustäbel; :issue:`7232`.) 1617 1618* The :meth:`~threading.Event.wait` method of the :class:`threading.Event` class 1619 now returns the internal flag on exit. This means the method will usually 1620 return true because :meth:`~threading.Event.wait` is supposed to block until the 1621 internal flag becomes true. The return value will only be false if 1622 a timeout was provided and the operation timed out. 1623 (Contributed by Tim Lesher; :issue:`1674032`.) 1624 1625* The Unicode database provided by the :mod:`unicodedata` module is 1626 now used internally to determine which characters are numeric, 1627 whitespace, or represent line breaks. The database also 1628 includes information from the :file:`Unihan.txt` data file (patch 1629 by Anders Chrigström and Amaury Forgeot d'Arc; :issue:`1571184`) 1630 and has been updated to version 5.2.0 (updated by 1631 Florent Xicluna; :issue:`8024`). 1632 1633* The :mod:`urlparse` module's :func:`~urlparse.urlsplit` now handles 1634 unknown URL schemes in a fashion compliant with :rfc:`3986`: if the 1635 URL is of the form ``"<something>://..."``, the text before the 1636 ``://`` is treated as the scheme, even if it's a made-up scheme that 1637 the module doesn't know about. This change may break code that 1638 worked around the old behaviour. For example, Python 2.6.4 or 2.5 1639 will return the following: 1640 1641 >>> import urlparse 1642 >>> urlparse.urlsplit('invented://host/filename?query') 1643 ('invented', '', '//host/filename?query', '', '') 1644 1645 Python 2.7 (and Python 2.6.5) will return: 1646 1647 >>> import urlparse 1648 >>> urlparse.urlsplit('invented://host/filename?query') 1649 ('invented', 'host', '/filename?query', '', '') 1650 1651 (Python 2.7 actually produces slightly different output, since it 1652 returns a named tuple instead of a standard tuple.) 1653 1654 The :mod:`urlparse` module also supports IPv6 literal addresses as defined by 1655 :rfc:`2732` (contributed by Senthil Kumaran; :issue:`2987`). :: 1656 1657 >>> urlparse.urlparse('http://[1080::8:800:200C:417A]/foo') 1658 ParseResult(scheme='http', netloc='[1080::8:800:200C:417A]', 1659 path='/foo', params='', query='', fragment='') 1660 1661* New class: the :class:`~weakref.WeakSet` class in the :mod:`weakref` 1662 module is a set that only holds weak references to its elements; elements 1663 will be removed once there are no references pointing to them. 1664 (Originally implemented in Python 3.x by Raymond Hettinger, and backported 1665 to 2.7 by Michael Foord.) 1666 1667* The ElementTree library, :mod:`xml.etree`, no longer escapes 1668 ampersands and angle brackets when outputting an XML processing 1669 instruction (which looks like ``<?xml-stylesheet href="#style1"?>``) 1670 or comment (which looks like ``<!-- comment -->``). 1671 (Patch by Neil Muller; :issue:`2746`.) 1672 1673* The XML-RPC client and server, provided by the :mod:`xmlrpclib` and 1674 :mod:`SimpleXMLRPCServer` modules, have improved performance by 1675 supporting HTTP/1.1 keep-alive and by optionally using gzip encoding 1676 to compress the XML being exchanged. The gzip compression is 1677 controlled by the :attr:`encode_threshold` attribute of 1678 :class:`SimpleXMLRPCRequestHandler`, which contains a size in bytes; 1679 responses larger than this will be compressed. 1680 (Contributed by Kristján Valur Jónsson; :issue:`6267`.) 1681 1682* The :mod:`zipfile` module's :class:`~zipfile.ZipFile` now supports the context 1683 management protocol, so you can write ``with zipfile.ZipFile(...) as f:``. 1684 (Contributed by Brian Curtin; :issue:`5511`.) 1685 1686 :mod:`zipfile` now also supports archiving empty directories and 1687 extracts them correctly. (Fixed by Kuba Wieczorek; :issue:`4710`.) 1688 Reading files out of an archive is faster, and interleaving 1689 :meth:`~zipfile.ZipFile.read` and :meth:`~zipfile.ZipFile.readline` now works correctly. 1690 (Contributed by Nir Aides; :issue:`7610`.) 1691 1692 The :func:`~zipfile.is_zipfile` function now 1693 accepts a file object, in addition to the path names accepted in earlier 1694 versions. (Contributed by Gabriel Genellina; :issue:`4756`.) 1695 1696 The :meth:`~zipfile.ZipFile.writestr` method now has an optional *compress_type* parameter 1697 that lets you override the default compression method specified in the 1698 :class:`~zipfile.ZipFile` constructor. (Contributed by Ronald Oussoren; 1699 :issue:`6003`.) 1700 1701 1702.. ====================================================================== 1703.. whole new modules get described in subsections here 1704 1705 1706.. _importlib-section: 1707 1708New module: importlib 1709------------------------------ 1710 1711Python 3.1 includes the :mod:`importlib` package, a re-implementation 1712of the logic underlying Python's :keyword:`import` statement. 1713:mod:`importlib` is useful for implementors of Python interpreters and 1714to users who wish to write new importers that can participate in the 1715import process. Python 2.7 doesn't contain the complete 1716:mod:`importlib` package, but instead has a tiny subset that contains 1717a single function, :func:`~importlib.import_module`. 1718 1719``import_module(name, package=None)`` imports a module. *name* is 1720a string containing the module or package's name. It's possible to do 1721relative imports by providing a string that begins with a ``.`` 1722character, such as ``..utils.errors``. For relative imports, the 1723*package* argument must be provided and is the name of the package that 1724will be used as the anchor for 1725the relative import. :func:`~importlib.import_module` both inserts the imported 1726module into ``sys.modules`` and returns the module object. 1727 1728Here are some examples:: 1729 1730 >>> from importlib import import_module 1731 >>> anydbm = import_module('anydbm') # Standard absolute import 1732 >>> anydbm 1733 <module 'anydbm' from '/p/python/Lib/anydbm.py'> 1734 >>> # Relative import 1735 >>> file_util = import_module('..file_util', 'distutils.command') 1736 >>> file_util 1737 <module 'distutils.file_util' from '/python/Lib/distutils/file_util.pyc'> 1738 1739:mod:`importlib` was implemented by Brett Cannon and introduced in 1740Python 3.1. 1741 1742 1743New module: sysconfig 1744--------------------------------- 1745 1746The :mod:`sysconfig` module has been pulled out of the Distutils 1747package, becoming a new top-level module in its own right. 1748:mod:`sysconfig` provides functions for getting information about 1749Python's build process: compiler switches, installation paths, the 1750platform name, and whether Python is running from its source 1751directory. 1752 1753Some of the functions in the module are: 1754 1755* :func:`~sysconfig.get_config_var` returns variables from Python's 1756 Makefile and the :file:`pyconfig.h` file. 1757* :func:`~sysconfig.get_config_vars` returns a dictionary containing 1758 all of the configuration variables. 1759* :func:`~sysconfig.get_path` returns the configured path for 1760 a particular type of module: the standard library, 1761 site-specific modules, platform-specific modules, etc. 1762* :func:`~sysconfig.is_python_build` returns true if you're running a 1763 binary from a Python source tree, and false otherwise. 1764 1765Consult the :mod:`sysconfig` documentation for more details and for 1766a complete list of functions. 1767 1768The Distutils package and :mod:`sysconfig` are now maintained by Tarek 1769Ziadé, who has also started a Distutils2 package (source repository at 1770https://hg.python.org/distutils2/) for developing a next-generation 1771version of Distutils. 1772 1773 1774ttk: Themed Widgets for Tk 1775-------------------------- 1776 1777Tcl/Tk 8.5 includes a set of themed widgets that re-implement basic Tk 1778widgets but have a more customizable appearance and can therefore more 1779closely resemble the native platform's widgets. This widget 1780set was originally called Tile, but was renamed to Ttk (for "themed Tk") 1781on being added to Tcl/Tck release 8.5. 1782 1783To learn more, read the :mod:`ttk` module documentation. You may also 1784wish to read the Tcl/Tk manual page describing the 1785Ttk theme engine, available at 1786https://www.tcl.tk/man/tcl8.5/TkCmd/ttk_intro.htm. Some 1787screenshots of the Python/Ttk code in use are at 1788http://code.google.com/p/python-ttk/wiki/Screenshots. 1789 1790The :mod:`ttk` module was written by Guilherme Polo and added in 1791:issue:`2983`. An alternate version called ``Tile.py``, written by 1792Martin Franklin and maintained by Kevin Walzer, was proposed for 1793inclusion in :issue:`2618`, but the authors argued that Guilherme 1794Polo's work was more comprehensive. 1795 1796 1797.. _unittest-section: 1798 1799Updated module: unittest 1800--------------------------------- 1801 1802The :mod:`unittest` module was greatly enhanced; many 1803new features were added. Most of these features were implemented 1804by Michael Foord, unless otherwise noted. The enhanced version of 1805the module is downloadable separately for use with Python versions 2.4 to 2.6, 1806packaged as the :mod:`unittest2` package, from 1807https://pypi.python.org/pypi/unittest2. 1808 1809When used from the command line, the module can automatically discover 1810tests. It's not as fancy as `py.test <http://pytest.org>`__ or 1811`nose <http://code.google.com/p/python-nose/>`__, but provides a simple way 1812to run tests kept within a set of package directories. For example, 1813the following command will search the :file:`test/` subdirectory for 1814any importable test files named ``test*.py``:: 1815 1816 python -m unittest discover -s test 1817 1818Consult the :mod:`unittest` module documentation for more details. 1819(Developed in :issue:`6001`.) 1820 1821The :func:`~unittest.main` function supports some other new options: 1822 1823* :option:`-b <unittest -b>` or :option:`!--buffer` will buffer the standard output 1824 and standard error streams during each test. If the test passes, 1825 any resulting output will be discarded; on failure, the buffered 1826 output will be displayed. 1827 1828* :option:`-c <unittest -c>` or :option:`!--catch` will cause the control-C interrupt 1829 to be handled more gracefully. Instead of interrupting the test 1830 process immediately, the currently running test will be completed 1831 and then the partial results up to the interruption will be reported. 1832 If you're impatient, a second press of control-C will cause an immediate 1833 interruption. 1834 1835 This control-C handler tries to avoid causing problems when the code 1836 being tested or the tests being run have defined a signal handler of 1837 their own, by noticing that a signal handler was already set and 1838 calling it. If this doesn't work for you, there's a 1839 :func:`~unittest.removeHandler` decorator that can be used to mark tests that 1840 should have the control-C handling disabled. 1841 1842* :option:`-f <unittest -f>` or :option:`!--failfast` makes 1843 test execution stop immediately when a test fails instead of 1844 continuing to execute further tests. (Suggested by Cliff Dyer and 1845 implemented by Michael Foord; :issue:`8074`.) 1846 1847The progress messages now show 'x' for expected failures 1848and 'u' for unexpected successes when run in verbose mode. 1849(Contributed by Benjamin Peterson.) 1850 1851Test cases can raise the :exc:`~unittest.SkipTest` exception to skip a 1852test (:issue:`1034053`). 1853 1854The error messages for :meth:`~unittest.TestCase.assertEqual`, 1855:meth:`~unittest.TestCase.assertTrue`, and :meth:`~unittest.TestCase.assertFalse` 1856failures now provide more information. If you set the 1857:attr:`~unittest.TestCase.longMessage` attribute of your :class:`~unittest.TestCase` classes to 1858true, both the standard error message and any additional message you 1859provide will be printed for failures. (Added by Michael Foord; :issue:`5663`.) 1860 1861The :meth:`~unittest.TestCase.assertRaises` method now 1862returns a context handler when called without providing a callable 1863object to run. For example, you can write this:: 1864 1865 with self.assertRaises(KeyError): 1866 {}['foo'] 1867 1868(Implemented by Antoine Pitrou; :issue:`4444`.) 1869 1870.. rev 78774 1871 1872Module- and class-level setup and teardown fixtures are now supported. 1873Modules can contain :func:`~unittest.setUpModule` and :func:`~unittest.tearDownModule` 1874functions. Classes can have :meth:`~unittest.TestCase.setUpClass` and 1875:meth:`~unittest.TestCase.tearDownClass` methods that must be defined as class methods 1876(using ``@classmethod`` or equivalent). These functions and 1877methods are invoked when the test runner switches to a test case in a 1878different module or class. 1879 1880The methods :meth:`~unittest.TestCase.addCleanup` and 1881:meth:`~unittest.TestCase.doCleanups` were added. 1882:meth:`~unittest.TestCase.addCleanup` lets you add cleanup functions that 1883will be called unconditionally (after :meth:`~unittest.TestCase.setUp` if 1884:meth:`~unittest.TestCase.setUp` fails, otherwise after :meth:`~unittest.TestCase.tearDown`). This allows 1885for much simpler resource allocation and deallocation during tests 1886(:issue:`5679`). 1887 1888A number of new methods were added that provide more specialized 1889tests. Many of these methods were written by Google engineers 1890for use in their test suites; Gregory P. Smith, Michael Foord, and 1891GvR worked on merging them into Python's version of :mod:`unittest`. 1892 1893* :meth:`~unittest.TestCase.assertIsNone` and :meth:`~unittest.TestCase.assertIsNotNone` take one 1894 expression and verify that the result is or is not ``None``. 1895 1896* :meth:`~unittest.TestCase.assertIs` and :meth:`~unittest.TestCase.assertIsNot` 1897 take two values and check whether the two values evaluate to the same object or not. 1898 (Added by Michael Foord; :issue:`2578`.) 1899 1900* :meth:`~unittest.TestCase.assertIsInstance` and 1901 :meth:`~unittest.TestCase.assertNotIsInstance` check whether 1902 the resulting object is an instance of a particular class, or of 1903 one of a tuple of classes. (Added by Georg Brandl; :issue:`7031`.) 1904 1905* :meth:`~unittest.TestCase.assertGreater`, :meth:`~unittest.TestCase.assertGreaterEqual`, 1906 :meth:`~unittest.TestCase.assertLess`, and :meth:`~unittest.TestCase.assertLessEqual` compare 1907 two quantities. 1908 1909* :meth:`~unittest.TestCase.assertMultiLineEqual` compares two strings, and if they're 1910 not equal, displays a helpful comparison that highlights the 1911 differences in the two strings. This comparison is now used by 1912 default when Unicode strings are compared with :meth:`~unittest.TestCase.assertEqual`. 1913 1914* :meth:`~unittest.TestCase.assertRegexpMatches` and 1915 :meth:`~unittest.TestCase.assertNotRegexpMatches` checks whether the 1916 first argument is a string matching or not matching the regular 1917 expression provided as the second argument (:issue:`8038`). 1918 1919* :meth:`~unittest.TestCase.assertRaisesRegexp` checks whether a particular exception 1920 is raised, and then also checks that the string representation of 1921 the exception matches the provided regular expression. 1922 1923* :meth:`~unittest.TestCase.assertIn` and :meth:`~unittest.TestCase.assertNotIn` 1924 tests whether *first* is or is not in *second*. 1925 1926* :meth:`~unittest.TestCase.assertItemsEqual` tests whether two provided sequences 1927 contain the same elements. 1928 1929* :meth:`~unittest.TestCase.assertSetEqual` compares whether two sets are equal, and 1930 only reports the differences between the sets in case of error. 1931 1932* Similarly, :meth:`~unittest.TestCase.assertListEqual` and :meth:`~unittest.TestCase.assertTupleEqual` 1933 compare the specified types and explain any differences without necessarily 1934 printing their full values; these methods are now used by default 1935 when comparing lists and tuples using :meth:`~unittest.TestCase.assertEqual`. 1936 More generally, :meth:`~unittest.TestCase.assertSequenceEqual` compares two sequences 1937 and can optionally check whether both sequences are of a 1938 particular type. 1939 1940* :meth:`~unittest.TestCase.assertDictEqual` compares two dictionaries and reports the 1941 differences; it's now used by default when you compare two dictionaries 1942 using :meth:`~unittest.TestCase.assertEqual`. :meth:`~unittest.TestCase.assertDictContainsSubset` checks whether 1943 all of the key/value pairs in *first* are found in *second*. 1944 1945* :meth:`~unittest.TestCase.assertAlmostEqual` and :meth:`~unittest.TestCase.assertNotAlmostEqual` test 1946 whether *first* and *second* are approximately equal. This method 1947 can either round their difference to an optionally-specified number 1948 of *places* (the default is 7) and compare it to zero, or require 1949 the difference to be smaller than a supplied *delta* value. 1950 1951* :meth:`~unittest.TestLoader.loadTestsFromName` properly honors the 1952 :attr:`~unittest.TestLoader.suiteClass` attribute of 1953 the :class:`~unittest.TestLoader`. (Fixed by Mark Roddy; :issue:`6866`.) 1954 1955* A new hook lets you extend the :meth:`~unittest.TestCase.assertEqual` method to handle 1956 new data types. The :meth:`~unittest.TestCase.addTypeEqualityFunc` method takes a type 1957 object and a function. The function will be used when both of the 1958 objects being compared are of the specified type. This function 1959 should compare the two objects and raise an exception if they don't 1960 match; it's a good idea for the function to provide additional 1961 information about why the two objects aren't matching, much as the new 1962 sequence comparison methods do. 1963 1964:func:`unittest.main` now takes an optional ``exit`` argument. If 1965false, :func:`~unittest.main` doesn't call :func:`sys.exit`, allowing 1966:func:`~unittest.main` to be used from the interactive interpreter. 1967(Contributed by J. Pablo Fernández; :issue:`3379`.) 1968 1969:class:`~unittest.TestResult` has new :meth:`~unittest.TestResult.startTestRun` and 1970:meth:`~unittest.TestResult.stopTestRun` methods that are called immediately before 1971and after a test run. (Contributed by Robert Collins; :issue:`5728`.) 1972 1973With all these changes, the :file:`unittest.py` was becoming awkwardly 1974large, so the module was turned into a package and the code split into 1975several files (by Benjamin Peterson). This doesn't affect how the 1976module is imported or used. 1977 1978.. seealso:: 1979 1980 http://www.voidspace.org.uk/python/articles/unittest2.shtml 1981 Describes the new features, how to use them, and the 1982 rationale for various design decisions. (By Michael Foord.) 1983 1984.. _elementtree-section: 1985 1986Updated module: ElementTree 1.3 1987--------------------------------- 1988 1989The version of the ElementTree library included with Python was updated to 1990version 1.3. Some of the new features are: 1991 1992* The various parsing functions now take a *parser* keyword argument 1993 giving an :class:`~xml.etree.ElementTree.XMLParser` instance that will 1994 be used. This makes it possible to override the file's internal encoding:: 1995 1996 p = ET.XMLParser(encoding='utf-8') 1997 t = ET.XML("""<root/>""", parser=p) 1998 1999 Errors in parsing XML now raise a :exc:`ParseError` exception, whose 2000 instances have a :attr:`position` attribute 2001 containing a (*line*, *column*) tuple giving the location of the problem. 2002 2003* ElementTree's code for converting trees to a string has been 2004 significantly reworked, making it roughly twice as fast in many 2005 cases. The :meth:`ElementTree.write() <xml.etree.ElementTree.ElementTree.write>` 2006 and :meth:`Element.write` methods now have a *method* parameter that can be 2007 "xml" (the default), "html", or "text". HTML mode will output empty 2008 elements as ``<empty></empty>`` instead of ``<empty/>``, and text 2009 mode will skip over elements and only output the text chunks. If 2010 you set the :attr:`tag` attribute of an element to ``None`` but 2011 leave its children in place, the element will be omitted when the 2012 tree is written out, so you don't need to do more extensive rearrangement 2013 to remove a single element. 2014 2015 Namespace handling has also been improved. All ``xmlns:<whatever>`` 2016 declarations are now output on the root element, not scattered throughout 2017 the resulting XML. You can set the default namespace for a tree 2018 by setting the :attr:`default_namespace` attribute and can 2019 register new prefixes with :meth:`~xml.etree.ElementTree.register_namespace`. In XML mode, 2020 you can use the true/false *xml_declaration* parameter to suppress the 2021 XML declaration. 2022 2023* New :class:`~xml.etree.ElementTree.Element` method: 2024 :meth:`~xml.etree.ElementTree.Element.extend` appends the items from a 2025 sequence to the element's children. Elements themselves behave like 2026 sequences, so it's easy to move children from one element to 2027 another:: 2028 2029 from xml.etree import ElementTree as ET 2030 2031 t = ET.XML("""<list> 2032 <item>1</item> <item>2</item> <item>3</item> 2033 </list>""") 2034 new = ET.XML('<root/>') 2035 new.extend(t) 2036 2037 # Outputs <root><item>1</item>...</root> 2038 print ET.tostring(new) 2039 2040* New :class:`Element` method: 2041 :meth:`~xml.etree.ElementTree.Element.iter` yields the children of the 2042 element as a generator. It's also possible to write ``for child in 2043 elem:`` to loop over an element's children. The existing method 2044 :meth:`getiterator` is now deprecated, as is :meth:`getchildren` 2045 which constructs and returns a list of children. 2046 2047* New :class:`Element` method: 2048 :meth:`~xml.etree.ElementTree.Element.itertext` yields all chunks of 2049 text that are descendants of the element. For example:: 2050 2051 t = ET.XML("""<list> 2052 <item>1</item> <item>2</item> <item>3</item> 2053 </list>""") 2054 2055 # Outputs ['\n ', '1', ' ', '2', ' ', '3', '\n'] 2056 print list(t.itertext()) 2057 2058* Deprecated: using an element as a Boolean (i.e., ``if elem:``) would 2059 return true if the element had any children, or false if there were 2060 no children. This behaviour is confusing -- ``None`` is false, but 2061 so is a childless element? -- so it will now trigger a 2062 :exc:`FutureWarning`. In your code, you should be explicit: write 2063 ``len(elem) != 0`` if you're interested in the number of children, 2064 or ``elem is not None``. 2065 2066Fredrik Lundh develops ElementTree and produced the 1.3 version; 2067you can read his article describing 1.3 at 2068http://effbot.org/zone/elementtree-13-intro.htm. 2069Florent Xicluna updated the version included with 2070Python, after discussions on python-dev and in :issue:`6472`.) 2071 2072.. ====================================================================== 2073 2074 2075Build and C API Changes 2076======================= 2077 2078Changes to Python's build process and to the C API include: 2079 2080* The latest release of the GNU Debugger, GDB 7, can be `scripted 2081 using Python 2082 <https://sourceware.org/gdb/current/onlinedocs/gdb/Python.html>`__. 2083 When you begin debugging an executable program P, GDB will look for 2084 a file named ``P-gdb.py`` and automatically read it. Dave Malcolm 2085 contributed a :file:`python-gdb.py` that adds a number of 2086 commands useful when debugging Python itself. For example, 2087 ``py-up`` and ``py-down`` go up or down one Python stack frame, 2088 which usually corresponds to several C stack frames. ``py-print`` 2089 prints the value of a Python variable, and ``py-bt`` prints the 2090 Python stack trace. (Added as a result of :issue:`8032`.) 2091 2092* If you use the :file:`.gdbinit` file provided with Python, 2093 the "pyo" macro in the 2.7 version now works correctly when the thread being 2094 debugged doesn't hold the GIL; the macro now acquires it before printing. 2095 (Contributed by Victor Stinner; :issue:`3632`.) 2096 2097* :c:func:`Py_AddPendingCall` is now thread-safe, letting any 2098 worker thread submit notifications to the main Python thread. This 2099 is particularly useful for asynchronous IO operations. 2100 (Contributed by Kristján Valur Jónsson; :issue:`4293`.) 2101 2102* New function: :c:func:`PyCode_NewEmpty` creates an empty code object; 2103 only the filename, function name, and first line number are required. 2104 This is useful for extension modules that are attempting to 2105 construct a more useful traceback stack. Previously such 2106 extensions needed to call :c:func:`PyCode_New`, which had many 2107 more arguments. (Added by Jeffrey Yasskin.) 2108 2109* New function: :c:func:`PyErr_NewExceptionWithDoc` creates a new 2110 exception class, just as the existing :c:func:`PyErr_NewException` does, 2111 but takes an extra ``char *`` argument containing the docstring for the 2112 new exception class. (Added by 'lekma' on the Python bug tracker; 2113 :issue:`7033`.) 2114 2115* New function: :c:func:`PyFrame_GetLineNumber` takes a frame object 2116 and returns the line number that the frame is currently executing. 2117 Previously code would need to get the index of the bytecode 2118 instruction currently executing, and then look up the line number 2119 corresponding to that address. (Added by Jeffrey Yasskin.) 2120 2121* New functions: :c:func:`PyLong_AsLongAndOverflow` and 2122 :c:func:`PyLong_AsLongLongAndOverflow` approximates a Python long 2123 integer as a C :c:type:`long` or :c:type:`long long`. 2124 If the number is too large to fit into 2125 the output type, an *overflow* flag is set and returned to the caller. 2126 (Contributed by Case Van Horsen; :issue:`7528` and :issue:`7767`.) 2127 2128* New function: stemming from the rewrite of string-to-float conversion, 2129 a new :c:func:`PyOS_string_to_double` function was added. The old 2130 :c:func:`PyOS_ascii_strtod` and :c:func:`PyOS_ascii_atof` functions 2131 are now deprecated. 2132 2133* New function: :c:func:`PySys_SetArgvEx` sets the value of 2134 ``sys.argv`` and can optionally update ``sys.path`` to include the 2135 directory containing the script named by ``sys.argv[0]`` depending 2136 on the value of an *updatepath* parameter. 2137 2138 This function was added to close a security hole for applications 2139 that embed Python. The old function, :c:func:`PySys_SetArgv`, would 2140 always update ``sys.path``, and sometimes it would add the current 2141 directory. This meant that, if you ran an application embedding 2142 Python in a directory controlled by someone else, attackers could 2143 put a Trojan-horse module in the directory (say, a file named 2144 :file:`os.py`) that your application would then import and run. 2145 2146 If you maintain a C/C++ application that embeds Python, check 2147 whether you're calling :c:func:`PySys_SetArgv` and carefully consider 2148 whether the application should be using :c:func:`PySys_SetArgvEx` 2149 with *updatepath* set to false. 2150 2151 Security issue reported as `CVE-2008-5983 2152 <https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-5983>`_; 2153 discussed in :issue:`5753`, and fixed by Antoine Pitrou. 2154 2155* New macros: the Python header files now define the following macros: 2156 :c:macro:`Py_ISALNUM`, 2157 :c:macro:`Py_ISALPHA`, 2158 :c:macro:`Py_ISDIGIT`, 2159 :c:macro:`Py_ISLOWER`, 2160 :c:macro:`Py_ISSPACE`, 2161 :c:macro:`Py_ISUPPER`, 2162 :c:macro:`Py_ISXDIGIT`, 2163 :c:macro:`Py_TOLOWER`, and :c:macro:`Py_TOUPPER`. 2164 All of these functions are analogous to the C 2165 standard macros for classifying characters, but ignore the current 2166 locale setting, because in 2167 several places Python needs to analyze characters in a 2168 locale-independent way. (Added by Eric Smith; 2169 :issue:`5793`.) 2170 2171 .. XXX these macros don't seem to be described in the c-api docs. 2172 2173* Removed function: :c:macro:`PyEval_CallObject` is now only available 2174 as a macro. A function version was being kept around to preserve 2175 ABI linking compatibility, but that was in 1997; it can certainly be 2176 deleted by now. (Removed by Antoine Pitrou; :issue:`8276`.) 2177 2178* New format codes: the :c:func:`PyFormat_FromString`, 2179 :c:func:`PyFormat_FromStringV`, and :c:func:`PyErr_Format` functions now 2180 accept ``%lld`` and ``%llu`` format codes for displaying 2181 C's :c:type:`long long` types. 2182 (Contributed by Mark Dickinson; :issue:`7228`.) 2183 2184* The complicated interaction between threads and process forking has 2185 been changed. Previously, the child process created by 2186 :func:`os.fork` might fail because the child is created with only a 2187 single thread running, the thread performing the :func:`os.fork`. 2188 If other threads were holding a lock, such as Python's import lock, 2189 when the fork was performed, the lock would still be marked as 2190 "held" in the new process. But in the child process nothing would 2191 ever release the lock, since the other threads weren't replicated, 2192 and the child process would no longer be able to perform imports. 2193 2194 Python 2.7 acquires the import lock before performing an 2195 :func:`os.fork`, and will also clean up any locks created using the 2196 :mod:`threading` module. C extension modules that have internal 2197 locks, or that call :c:func:`fork()` themselves, will not benefit 2198 from this clean-up. 2199 2200 (Fixed by Thomas Wouters; :issue:`1590864`.) 2201 2202* The :c:func:`Py_Finalize` function now calls the internal 2203 :func:`threading._shutdown` function; this prevents some exceptions from 2204 being raised when an interpreter shuts down. 2205 (Patch by Adam Olsen; :issue:`1722344`.) 2206 2207* When using the :c:type:`PyMemberDef` structure to define attributes 2208 of a type, Python will no longer let you try to delete or set a 2209 :const:`T_STRING_INPLACE` attribute. 2210 2211 .. rev 79644 2212 2213* Global symbols defined by the :mod:`ctypes` module are now prefixed 2214 with ``Py``, or with ``_ctypes``. (Implemented by Thomas 2215 Heller; :issue:`3102`.) 2216 2217* New configure option: the :option:`!--with-system-expat` switch allows 2218 building the :mod:`pyexpat` module to use the system Expat library. 2219 (Contributed by Arfrever Frehtes Taifersar Arahesis; :issue:`7609`.) 2220 2221* New configure option: the 2222 :option:`!--with-valgrind` option will now disable the pymalloc 2223 allocator, which is difficult for the Valgrind memory-error detector 2224 to analyze correctly. 2225 Valgrind will therefore be better at detecting memory leaks and 2226 overruns. (Contributed by James Henstridge; :issue:`2422`.) 2227 2228* New configure option: you can now supply an empty string to 2229 :option:`!--with-dbmliborder=` in order to disable all of the various 2230 DBM modules. (Added by Arfrever Frehtes Taifersar Arahesis; 2231 :issue:`6491`.) 2232 2233* The :program:`configure` script now checks for floating-point rounding bugs 2234 on certain 32-bit Intel chips and defines a :c:macro:`X87_DOUBLE_ROUNDING` 2235 preprocessor definition. No code currently uses this definition, 2236 but it's available if anyone wishes to use it. 2237 (Added by Mark Dickinson; :issue:`2937`.) 2238 2239 :program:`configure` also now sets a :envvar:`LDCXXSHARED` Makefile 2240 variable for supporting C++ linking. (Contributed by Arfrever 2241 Frehtes Taifersar Arahesis; :issue:`1222585`.) 2242 2243* The build process now creates the necessary files for pkg-config 2244 support. (Contributed by Clinton Roy; :issue:`3585`.) 2245 2246* The build process now supports Subversion 1.7. (Contributed by 2247 Arfrever Frehtes Taifersar Arahesis; :issue:`6094`.) 2248 2249 2250.. _whatsnew27-capsules: 2251 2252Capsules 2253------------------- 2254 2255Python 3.1 adds a new C datatype, :c:type:`PyCapsule`, for providing a 2256C API to an extension module. A capsule is essentially the holder of 2257a C ``void *`` pointer, and is made available as a module attribute; for 2258example, the :mod:`socket` module's API is exposed as ``socket.CAPI``, 2259and :mod:`unicodedata` exposes ``ucnhash_CAPI``. Other extensions 2260can import the module, access its dictionary to get the capsule 2261object, and then get the ``void *`` pointer, which will usually point 2262to an array of pointers to the module's various API functions. 2263 2264There is an existing data type already used for this, 2265:c:type:`PyCObject`, but it doesn't provide type safety. Evil code 2266written in pure Python could cause a segmentation fault by taking a 2267:c:type:`PyCObject` from module A and somehow substituting it for the 2268:c:type:`PyCObject` in module B. Capsules know their own name, 2269and getting the pointer requires providing the name: 2270 2271.. code-block:: c 2272 2273 void *vtable; 2274 2275 if (!PyCapsule_IsValid(capsule, "mymodule.CAPI") { 2276 PyErr_SetString(PyExc_ValueError, "argument type invalid"); 2277 return NULL; 2278 } 2279 2280 vtable = PyCapsule_GetPointer(capsule, "mymodule.CAPI"); 2281 2282You are assured that ``vtable`` points to whatever you're expecting. 2283If a different capsule was passed in, :c:func:`PyCapsule_IsValid` would 2284detect the mismatched name and return false. Refer to 2285:ref:`using-capsules` for more information on using these objects. 2286 2287Python 2.7 now uses capsules internally to provide various 2288extension-module APIs, but the :c:func:`PyCObject_AsVoidPtr` was 2289modified to handle capsules, preserving compile-time compatibility 2290with the :c:type:`CObject` interface. Use of 2291:c:func:`PyCObject_AsVoidPtr` will signal a 2292:exc:`PendingDeprecationWarning`, which is silent by default. 2293 2294Implemented in Python 3.1 and backported to 2.7 by Larry Hastings; 2295discussed in :issue:`5630`. 2296 2297 2298.. ====================================================================== 2299 2300Port-Specific Changes: Windows 2301----------------------------------- 2302 2303* The :mod:`msvcrt` module now contains some constants from 2304 the :file:`crtassem.h` header file: 2305 :data:`CRT_ASSEMBLY_VERSION`, 2306 :data:`VC_ASSEMBLY_PUBLICKEYTOKEN`, 2307 and :data:`LIBRARIES_ASSEMBLY_NAME_PREFIX`. 2308 (Contributed by David Cournapeau; :issue:`4365`.) 2309 2310* The :mod:`_winreg` module for accessing the registry now implements 2311 the :func:`~_winreg.CreateKeyEx` and :func:`~_winreg.DeleteKeyEx` 2312 functions, extended versions of previously-supported functions that 2313 take several extra arguments. The :func:`~_winreg.DisableReflectionKey`, 2314 :func:`~_winreg.EnableReflectionKey`, and :func:`~_winreg.QueryReflectionKey` 2315 were also tested and documented. 2316 (Implemented by Brian Curtin: :issue:`7347`.) 2317 2318* The new :c:func:`_beginthreadex` API is used to start threads, and 2319 the native thread-local storage functions are now used. 2320 (Contributed by Kristján Valur Jónsson; :issue:`3582`.) 2321 2322* The :func:`os.kill` function now works on Windows. The signal value 2323 can be the constants :const:`CTRL_C_EVENT`, 2324 :const:`CTRL_BREAK_EVENT`, or any integer. The first two constants 2325 will send :kbd:`Control-C` and :kbd:`Control-Break` keystroke events to 2326 subprocesses; any other value will use the :c:func:`TerminateProcess` 2327 API. (Contributed by Miki Tebeka; :issue:`1220212`.) 2328 2329* The :func:`os.listdir` function now correctly fails 2330 for an empty path. (Fixed by Hirokazu Yamamoto; :issue:`5913`.) 2331 2332* The :mod:`mimelib` module will now read the MIME database from 2333 the Windows registry when initializing. 2334 (Patch by Gabriel Genellina; :issue:`4969`.) 2335 2336.. ====================================================================== 2337 2338Port-Specific Changes: Mac OS X 2339----------------------------------- 2340 2341* The path ``/Library/Python/2.7/site-packages`` is now appended to 2342 ``sys.path``, in order to share added packages between the system 2343 installation and a user-installed copy of the same version. 2344 (Changed by Ronald Oussoren; :issue:`4865`.) 2345 2346 .. versionchanged:: 2.7.13 2347 2348 As of 2.7.13, this change was removed. 2349 ``/Library/Python/2.7/site-packages``, the site-packages directory 2350 used by the Apple-supplied system Python 2.7 is no longer appended to 2351 ``sys.path`` for user-installed Pythons such as from the python.org 2352 installers. As of macOS 10.12, Apple changed how the system 2353 site-packages directory is configured, which could cause installation 2354 of pip components, like setuptools, to fail. Packages installed for 2355 the system Python will no longer be shared with user-installed 2356 Pythons. (:issue:`28440`) 2357 2358Port-Specific Changes: FreeBSD 2359----------------------------------- 2360 2361* FreeBSD 7.1's :const:`SO_SETFIB` constant, used with 2362 :func:`~socket.getsockopt`/:func:`~socket.setsockopt` to select an 2363 alternate routing table, is now available in the :mod:`socket` 2364 module. (Added by Kyle VanderBeek; :issue:`8235`.) 2365 2366Other Changes and Fixes 2367======================= 2368 2369* Two benchmark scripts, :file:`iobench` and :file:`ccbench`, were 2370 added to the :file:`Tools` directory. :file:`iobench` measures the 2371 speed of the built-in file I/O objects returned by :func:`open` 2372 while performing various operations, and :file:`ccbench` is a 2373 concurrency benchmark that tries to measure computing throughput, 2374 thread switching latency, and IO processing bandwidth when 2375 performing several tasks using a varying number of threads. 2376 2377* The :file:`Tools/i18n/msgfmt.py` script now understands plural 2378 forms in :file:`.po` files. (Fixed by Martin von Löwis; 2379 :issue:`5464`.) 2380 2381* When importing a module from a :file:`.pyc` or :file:`.pyo` file 2382 with an existing :file:`.py` counterpart, the :attr:`co_filename` 2383 attributes of the resulting code objects are overwritten when the 2384 original filename is obsolete. This can happen if the file has been 2385 renamed, moved, or is accessed through different paths. (Patch by 2386 Ziga Seilnacht and Jean-Paul Calderone; :issue:`1180193`.) 2387 2388* The :file:`regrtest.py` script now takes a :option:`!--randseed=` 2389 switch that takes an integer that will be used as the random seed 2390 for the :option:`!-r` option that executes tests in random order. 2391 The :option:`!-r` option also reports the seed that was used 2392 (Added by Collin Winter.) 2393 2394* Another :file:`regrtest.py` switch is :option:`!-j`, which 2395 takes an integer specifying how many tests run in parallel. This 2396 allows reducing the total runtime on multi-core machines. 2397 This option is compatible with several other options, including the 2398 :option:`!-R` switch which is known to produce long runtimes. 2399 (Added by Antoine Pitrou, :issue:`6152`.) This can also be used 2400 with a new :option:`!-F` switch that runs selected tests in a loop 2401 until they fail. (Added by Antoine Pitrou; :issue:`7312`.) 2402 2403* When executed as a script, the :file:`py_compile.py` module now 2404 accepts ``'-'`` as an argument, which will read standard input for 2405 the list of filenames to be compiled. (Contributed by Piotr 2406 Ożarowski; :issue:`8233`.) 2407 2408.. ====================================================================== 2409 2410Porting to Python 2.7 2411===================== 2412 2413This section lists previously described changes and other bugfixes 2414that may require changes to your code: 2415 2416* The :func:`range` function processes its arguments more 2417 consistently; it will now call :meth:`__int__` on non-float, 2418 non-integer arguments that are supplied to it. (Fixed by Alexander 2419 Belopolsky; :issue:`1533`.) 2420 2421* The string :meth:`format` method changed the default precision used 2422 for floating-point and complex numbers from 6 decimal 2423 places to 12, which matches the precision used by :func:`str`. 2424 (Changed by Eric Smith; :issue:`5920`.) 2425 2426* Because of an optimization for the :keyword:`with` statement, the special 2427 methods :meth:`__enter__` and :meth:`__exit__` must belong to the object's 2428 type, and cannot be directly attached to the object's instance. This 2429 affects new-style classes (derived from :class:`object`) and C extension 2430 types. (:issue:`6101`.) 2431 2432* Due to a bug in Python 2.6, the *exc_value* parameter to 2433 :meth:`__exit__` methods was often the string representation of the 2434 exception, not an instance. This was fixed in 2.7, so *exc_value* 2435 will be an instance as expected. (Fixed by Florent Xicluna; 2436 :issue:`7853`.) 2437 2438* When a restricted set of attributes were set using ``__slots__``, 2439 deleting an unset attribute would not raise :exc:`AttributeError` 2440 as you would expect. Fixed by Benjamin Peterson; :issue:`7604`.) 2441 2442In the standard library: 2443 2444* Operations with :class:`~datetime.datetime` instances that resulted in a year 2445 falling outside the supported range didn't always raise 2446 :exc:`OverflowError`. Such errors are now checked more carefully 2447 and will now raise the exception. (Reported by Mark Leander, patch 2448 by Anand B. Pillai and Alexander Belopolsky; :issue:`7150`.) 2449 2450* When using :class:`~decimal.Decimal` instances with a string's 2451 :meth:`format` method, the default alignment was previously 2452 left-alignment. This has been changed to right-alignment, which might 2453 change the output of your programs. 2454 (Changed by Mark Dickinson; :issue:`6857`.) 2455 2456 Comparisons involving a signaling NaN value (or ``sNAN``) now signal 2457 :const:`~decimal.InvalidOperation` instead of silently returning a true or 2458 false value depending on the comparison operator. Quiet NaN values 2459 (or ``NaN``) are now hashable. (Fixed by Mark Dickinson; 2460 :issue:`7279`.) 2461 2462* The ElementTree library, :mod:`xml.etree`, no longer escapes 2463 ampersands and angle brackets when outputting an XML processing 2464 instruction (which looks like `<?xml-stylesheet href="#style1"?>`) 2465 or comment (which looks like `<!-- comment -->`). 2466 (Patch by Neil Muller; :issue:`2746`.) 2467 2468* The :meth:`~StringIO.StringIO.readline` method of :class:`~StringIO.StringIO` objects now does 2469 nothing when a negative length is requested, as other file-like 2470 objects do. (:issue:`7348`). 2471 2472* The :mod:`syslog` module will now use the value of ``sys.argv[0]`` as the 2473 identifier instead of the previous default value of ``'python'``. 2474 (Changed by Sean Reifschneider; :issue:`8451`.) 2475 2476* The :mod:`tarfile` module's default error handling has changed, to 2477 no longer suppress fatal errors. The default error level was previously 0, 2478 which meant that errors would only result in a message being written to the 2479 debug log, but because the debug log is not activated by default, 2480 these errors go unnoticed. The default error level is now 1, 2481 which raises an exception if there's an error. 2482 (Changed by Lars Gustäbel; :issue:`7357`.) 2483 2484* The :mod:`urlparse` module's :func:`~urlparse.urlsplit` now handles 2485 unknown URL schemes in a fashion compliant with :rfc:`3986`: if the 2486 URL is of the form ``"<something>://..."``, the text before the 2487 ``://`` is treated as the scheme, even if it's a made-up scheme that 2488 the module doesn't know about. This change may break code that 2489 worked around the old behaviour. For example, Python 2.6.4 or 2.5 2490 will return the following: 2491 2492 >>> import urlparse 2493 >>> urlparse.urlsplit('invented://host/filename?query') 2494 ('invented', '', '//host/filename?query', '', '') 2495 2496 Python 2.7 (and Python 2.6.5) will return: 2497 2498 >>> import urlparse 2499 >>> urlparse.urlsplit('invented://host/filename?query') 2500 ('invented', 'host', '/filename?query', '', '') 2501 2502 (Python 2.7 actually produces slightly different output, since it 2503 returns a named tuple instead of a standard tuple.) 2504 2505For C extensions: 2506 2507* C extensions that use integer format codes with the ``PyArg_Parse*`` 2508 family of functions will now raise a :exc:`TypeError` exception 2509 instead of triggering a :exc:`DeprecationWarning` (:issue:`5080`). 2510 2511* Use the new :c:func:`PyOS_string_to_double` function instead of the old 2512 :c:func:`PyOS_ascii_strtod` and :c:func:`PyOS_ascii_atof` functions, 2513 which are now deprecated. 2514 2515For applications that embed Python: 2516 2517* The :c:func:`PySys_SetArgvEx` function was added, letting 2518 applications close a security hole when the existing 2519 :c:func:`PySys_SetArgv` function was used. Check whether you're 2520 calling :c:func:`PySys_SetArgv` and carefully consider whether the 2521 application should be using :c:func:`PySys_SetArgvEx` with 2522 *updatepath* set to false. 2523 2524.. ====================================================================== 2525 2526 2527.. _py27-maintenance-enhancements: 2528 2529New Features Added to Python 2.7 Maintenance Releases 2530===================================================== 2531 2532New features may be added to Python 2.7 maintenance releases when the 2533situation genuinely calls for it. Any such additions must go through 2534the Python Enhancement Proposal process, and make a compelling case for why 2535they can't be adequately addressed by either adding the new feature solely to 2536Python 3, or else by publishing it on the Python Package Index. 2537 2538In addition to the specific proposals listed below, there is a general 2539exemption allowing new ``-3`` warnings to be added in any Python 2.7 2540maintenance release. 2541 2542 2543PEP 434: IDLE Enhancement Exception for All Branches 2544---------------------------------------------------- 2545 2546:pep:`434` describes a general exemption for changes made to the IDLE 2547development environment shipped along with Python. This exemption makes it 2548possible for the IDLE developers to provide a more consistent user 2549experience across all supported versions of Python 2 and 3. 2550 2551For details of any IDLE changes, refer to the NEWS file for the specific 2552release. 2553 2554 2555PEP 466: Network Security Enhancements for Python 2.7 2556----------------------------------------------------- 2557 2558:pep:`466` describes a number of network security enhancement proposals 2559that have been approved for inclusion in Python 2.7 maintenance releases, 2560with the first of those changes appearing in the Python 2.7.7 release. 2561 2562:pep:`466` related features added in Python 2.7.7: 2563 2564* :func:`hmac.compare_digest` was backported from Python 3 to make a timing 2565 attack resistant comparison operation available to Python 2 applications. 2566 (Contributed by Alex Gaynor; :issue:`21306`.) 2567 2568* OpenSSL 1.0.1g was upgraded in the official Windows installers published on 2569 python.org. (Contributed by Zachary Ware; :issue:`21462`.) 2570 2571:pep:`466` related features added in Python 2.7.8: 2572 2573* :func:`hashlib.pbkdf2_hmac` was backported from Python 3 to make a hashing 2574 algorithm suitable for secure password storage broadly available to Python 2575 2 applications. (Contributed by Alex Gaynor; :issue:`21304`.) 2576 2577* OpenSSL 1.0.1h was upgraded for the official Windows installers published on 2578 python.org. (contributed by Zachary Ware in :issue:`21671` for CVE-2014-0224) 2579 2580:pep:`466` related features added in Python 2.7.9: 2581 2582* Most of Python 3.4's :mod:`ssl` module was backported. This means :mod:`ssl` 2583 now supports Server Name Indication, TLS1.x settings, access to the platform 2584 certificate store, the :class:`~ssl.SSLContext` class, and other 2585 features. (Contributed by Alex Gaynor and David Reid; :issue:`21308`.) 2586 2587 Refer to the "Version added: 2.7.9" notes in the module documentation for 2588 specific details. 2589 2590* :func:`os.urandom` was changed to cache a file descriptor to ``/dev/urandom`` 2591 instead of reopening ``/dev/urandom`` on every call. (Contributed by Alex 2592 Gaynor; :issue:`21305`.) 2593 2594* :data:`hashlib.algorithms_guaranteed` and 2595 :data:`hashlib.algorithms_available` were backported from Python 3 to make 2596 it easier for Python 2 applications to select the strongest available hash 2597 algorithm. (Contributed by Alex Gaynor in :issue:`21307`) 2598 2599 2600PEP 477: Backport ensurepip (PEP 453) to Python 2.7 2601--------------------------------------------------- 2602 2603:pep:`477` approves the inclusion of the :pep:`453` ensurepip module and the 2604improved documentation that was enabled by it in the Python 2.7 maintenance 2605releases, appearing first in the Python 2.7.9 release. 2606 2607 2608Bootstrapping pip By Default 2609~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2610 2611The new :mod:`ensurepip` module (defined in :pep:`453`) provides a standard 2612cross-platform mechanism to bootstrap the pip installer into Python 2613installations. The version of ``pip`` included with Python 2.7.9 is ``pip`` 26141.5.6, and future 2.7.x maintenance releases will update the bundled version to 2615the latest version of ``pip`` that is available at the time of creating the 2616release candidate. 2617 2618By default, the commands ``pip``, ``pipX`` and ``pipX.Y`` will be installed on 2619all platforms (where X.Y stands for the version of the Python installation), 2620along with the ``pip`` Python package and its dependencies. 2621 2622For CPython :ref:`source builds on POSIX systems <building-python-on-unix>`, 2623the ``make install`` and ``make altinstall`` commands do not bootstrap ``pip`` 2624by default. This behaviour can be controlled through configure options, and 2625overridden through Makefile options. 2626 2627On Windows and Mac OS X, the CPython installers now default to installing 2628``pip`` along with CPython itself (users may opt out of installing it 2629during the installation process). Window users will need to opt in to the 2630automatic ``PATH`` modifications to have ``pip`` available from the command 2631line by default, otherwise it can still be accessed through the Python 2632launcher for Windows as ``py -m pip``. 2633 2634As `discussed in the PEP`__, platform packagers may choose not to install 2635these commands by default, as long as, when invoked, they provide clear and 2636simple directions on how to install them on that platform (usually using 2637the system package manager). 2638 2639__ https://www.python.org/dev/peps/pep-0477/#disabling-ensurepip-by-downstream-distributors 2640 2641 2642Documentation Changes 2643~~~~~~~~~~~~~~~~~~~~~ 2644 2645As part of this change, the :ref:`installing-index` and 2646:ref:`distributing-index` sections of the documentation have been 2647completely redesigned as short getting started and FAQ documents. Most 2648packaging documentation has now been moved out to the Python Packaging 2649Authority maintained `Python Packaging User Guide 2650<http://packaging.python.org>`__ and the documentation of the individual 2651projects. 2652 2653However, as this migration is currently still incomplete, the legacy 2654versions of those guides remaining available as :ref:`install-index` 2655and :ref:`distutils-index`. 2656 2657.. seealso:: 2658 2659 :pep:`453` -- Explicit bootstrapping of pip in Python installations 2660 PEP written by Donald Stufft and Nick Coghlan, implemented by 2661 Donald Stufft, Nick Coghlan, Martin von Löwis and Ned Deily. 2662 2663PEP 476: Enabling certificate verification by default for stdlib http clients 2664----------------------------------------------------------------------------- 2665 2666:pep:`476` updated :mod:`httplib` and modules which use it, such as 2667:mod:`urllib2` and :mod:`xmlrpclib`, to now verify that the server 2668presents a certificate which is signed by a Certificate Authority in the 2669platform trust store and whose hostname matches the hostname being requested 2670by default, significantly improving security for many applications. This 2671change was made in the Python 2.7.9 release. 2672 2673For applications which require the old previous behavior, they can pass an 2674alternate context:: 2675 2676 import urllib2 2677 import ssl 2678 2679 # This disables all verification 2680 context = ssl._create_unverified_context() 2681 2682 # This allows using a specific certificate for the host, which doesn't need 2683 # to be in the trust store 2684 context = ssl.create_default_context(cafile="/path/to/file.crt") 2685 2686 urllib2.urlopen("https://invalid-cert", context=context) 2687 2688 2689PEP 493: HTTPS verification migration tools for Python 2.7 2690---------------------------------------------------------- 2691 2692:pep:`493` provides additional migration tools to support a more incremental 2693infrastructure upgrade process for environments containing applications and 2694services relying on the historically permissive processing of server 2695certificates when establishing client HTTPS connections. These additions were 2696made in the Python 2.7.12 release. 2697 2698These tools are intended for use in cases where affected applications and 2699services can't be modified to explicitly pass a more permissive SSL context 2700when establishing the connection. 2701 2702For applications and services which can't be modified at all, the new 2703``PYTHONHTTPSVERIFY`` environment variable may be set to ``0`` to revert an 2704entire Python process back to the default permissive behaviour of Python 2.7.8 2705and earlier. 2706 2707For cases where the connection establishment code can't be modified, but the 2708overall application can be, the new :func:`ssl._https_verify_certificates` 2709function can be used to adjust the default behaviour at runtime. 2710 2711.. ====================================================================== 2712 2713.. _acks27: 2714 2715Acknowledgements 2716================ 2717 2718The author would like to thank the following people for offering 2719suggestions, corrections and assistance with various drafts of this 2720article: Nick Coghlan, Philip Jenvey, Ryan Lovett, R. David Murray, 2721Hugh Secker-Walker. 2722