1**************************** 2 What's New In Python 3.7 3**************************** 4 5:Editor: Elvis Pranskevichus <elvis@magic.io> 6 7.. Rules for maintenance: 8 9 * Anyone can add text to this document. Do not spend very much time 10 on the wording of your changes, because your text will probably 11 get rewritten to some degree. 12 13 * The maintainer will go through Misc/NEWS periodically and add 14 changes; it's therefore more important to add your changes to 15 Misc/NEWS than to this file. 16 17 * This is not a complete list of every single change; completeness 18 is the purpose of Misc/NEWS. Some changes I consider too small 19 or esoteric to include. If such a change is added to the text, 20 I'll just remove it. (This is another reason you shouldn't spend 21 too much time on writing your addition.) 22 23 * If you want to draw your new text to the attention of the 24 maintainer, add 'XXX' to the beginning of the paragraph or 25 section. 26 27 * It's OK to just add a fragmentary note about a change. For 28 example: "XXX Describe the transmogrify() function added to the 29 socket module." The maintainer will research the change and 30 write the necessary text. 31 32 * You can comment out your additions if you like, but it's not 33 necessary (especially when a final release is some months away). 34 35 * Credit the author of a patch or bugfix. Just the name is 36 sufficient; the e-mail address isn't necessary. 37 38 * It's helpful to add the bug/patch number as a comment: 39 40 XXX Describe the transmogrify() function added to the socket 41 module. 42 (Contributed by P.Y. Developer in :issue:`12345`.) 43 44 This saves the maintainer the effort of going through the Git log 45 when researching a change. 46 47This article explains the new features in Python 3.7, compared to 3.6. 48Python 3.7 was released on June 27, 2018. 49For full details, see the :ref:`changelog <changelog>`. 50 51 52Summary -- Release Highlights 53============================= 54 55.. This section singles out the most important changes in Python 3.7. 56 Brevity is key. 57 58New syntax features: 59 60* :ref:`PEP 563 <whatsnew37-pep563>`, postponed evaluation of type annotations. 61 62Backwards incompatible syntax changes: 63 64* :keyword:`async` and :keyword:`await` are now reserved keywords. 65 66New library modules: 67 68* :mod:`contextvars`: :ref:`PEP 567 -- Context Variables <whatsnew37-pep567>` 69* :mod:`dataclasses`: :ref:`PEP 557 -- Data Classes <whatsnew37-pep557>` 70* :ref:`whatsnew37_importlib_resources` 71 72New built-in features: 73 74* :ref:`PEP 553 <whatsnew37-pep553>`, the new :func:`breakpoint` function. 75 76Python data model improvements: 77 78* :ref:`PEP 562 <whatsnew37-pep562>`, customization of access to 79 module attributes. 80 81* :ref:`PEP 560 <whatsnew37-pep560>`, core support for typing module and 82 generic types. 83 84* the insertion-order preservation nature of :ref:`dict <typesmapping>` 85 objects `has been declared`_ to be an official 86 part of the Python language spec. 87 88.. _has been declared: https://mail.python.org/pipermail/python-dev/2017-December/151283.html 89 90Significant improvements in the standard library: 91 92* The :mod:`asyncio` module has received new features, significant 93 :ref:`usability and performance improvements <whatsnew37_asyncio>`. 94 95* The :mod:`time` module gained support for 96 :ref:`functions with nanosecond resolution <whatsnew37-pep564>`. 97 98CPython implementation improvements: 99 100* Avoiding the use of ASCII as a default text encoding: 101 102 * :ref:`PEP 538 <whatsnew37-pep538>`, legacy C locale coercion 103 * :ref:`PEP 540 <whatsnew37-pep540>`, forced UTF-8 runtime mode 104* :ref:`PEP 552 <whatsnew37-pep552>`, deterministic .pycs 105* :ref:`the new development runtime mode <whatsnew37-devmode>` 106* :ref:`PEP 565 <whatsnew37-pep565>`, improved :exc:`DeprecationWarning` 107 handling 108 109C API improvements: 110 111* :ref:`PEP 539 <whatsnew37-pep539>`, new C API for thread-local storage 112 113Documentation improvements: 114 115* :ref:`PEP 545 <whatsnew37-pep545>`, Python documentation translations 116* New documentation translations: `Japanese <https://docs.python.org/ja/>`_, 117 `French <https://docs.python.org/fr/>`_, and 118 `Korean <https://docs.python.org/ko/>`_. 119 120This release features notable performance improvements in many areas. 121The :ref:`whatsnew37-perf` section lists them in detail. 122 123For a list of changes that may affect compatibility with previous Python 124releases please refer to the :ref:`porting-to-python-37` section. 125 126 127New Features 128============ 129 130.. _whatsnew37-pep563: 131 132PEP 563: Postponed Evaluation of Annotations 133-------------------------------------------- 134 135The advent of type hints in Python uncovered two glaring usability issues 136with the functionality of annotations added in :pep:`3107` and refined 137further in :pep:`526`: 138 139* annotations could only use names which were already available in the 140 current scope, in other words they didn't support forward references 141 of any kind; and 142 143* annotating source code had adverse effects on startup time of Python 144 programs. 145 146Both of these issues are fixed by postponing the evaluation of 147annotations. Instead of compiling code which executes expressions in 148annotations at their definition time, the compiler stores the annotation 149in a string form equivalent to the AST of the expression in question. 150If needed, annotations can be resolved at runtime using 151:func:`typing.get_type_hints`. In the common case where this is not 152required, the annotations are cheaper to store (since short strings 153are interned by the interpreter) and make startup time faster. 154 155Usability-wise, annotations now support forward references, making the 156following syntax valid:: 157 158 class C: 159 @classmethod 160 def from_string(cls, source: str) -> C: 161 ... 162 163 def validate_b(self, obj: B) -> bool: 164 ... 165 166 class B: 167 ... 168 169Since this change breaks compatibility, the new behavior needs to be enabled 170on a per-module basis in Python 3.7 using a :mod:`__future__` import:: 171 172 from __future__ import annotations 173 174It will become the default in Python 4.0. 175 176.. seealso:: 177 178 :pep:`563` -- Postponed evaluation of annotations 179 PEP written and implemented by Łukasz Langa. 180 181 182.. _whatsnew37-pep538: 183 184PEP 538: Legacy C Locale Coercion 185--------------------------------- 186 187An ongoing challenge within the Python 3 series has been determining a sensible 188default strategy for handling the "7-bit ASCII" text encoding assumption 189currently implied by the use of the default C or POSIX locale on non-Windows 190platforms. 191 192:pep:`538` updates the default interpreter command line interface to 193automatically coerce that locale to an available UTF-8 based locale as 194described in the documentation of the new :envvar:`PYTHONCOERCECLOCALE` 195environment variable. Automatically setting ``LC_CTYPE`` this way means that 196both the core interpreter and locale-aware C extensions (such as 197:mod:`readline`) will assume the use of UTF-8 as the default text encoding, 198rather than ASCII. 199 200The platform support definition in :pep:`11` has also been updated to limit 201full text handling support to suitably configured non-ASCII based locales. 202 203As part of this change, the default error handler for :data:`~sys.stdin` and 204:data:`~sys.stdout` is now ``surrogateescape`` (rather than ``strict``) when 205using any of the defined coercion target locales (currently ``C.UTF-8``, 206``C.utf8``, and ``UTF-8``). The default error handler for :data:`~sys.stderr` 207continues to be ``backslashreplace``, regardless of locale. 208 209Locale coercion is silent by default, but to assist in debugging potentially 210locale related integration problems, explicit warnings (emitted directly on 211:data:`~sys.stderr`) can be requested by setting ``PYTHONCOERCECLOCALE=warn``. 212This setting will also cause the Python runtime to emit a warning if the 213legacy C locale remains active when the core interpreter is initialized. 214 215While :pep:`538`'s locale coercion has the benefit of also affecting extension 216modules (such as GNU ``readline``), as well as child processes (including those 217running non-Python applications and older versions of Python), it has the 218downside of requiring that a suitable target locale be present on the running 219system. To better handle the case where no suitable target locale is available 220(as occurs on RHEL/CentOS 7, for example), Python 3.7 also implements 221:ref:`whatsnew37-pep540`. 222 223.. seealso:: 224 225 :pep:`538` -- Coercing the legacy C locale to a UTF-8 based locale 226 PEP written and implemented by Nick Coghlan. 227 228 229.. _whatsnew37-pep540: 230 231PEP 540: Forced UTF-8 Runtime Mode 232----------------------------------- 233 234The new :option:`-X` ``utf8`` command line option and :envvar:`PYTHONUTF8` 235environment variable can be used to enable the CPython *UTF-8 mode*. 236 237When in UTF-8 mode, CPython ignores the locale settings, and uses the 238UTF-8 encoding by default. The error handlers for :data:`sys.stdin` and 239:data:`sys.stdout` streams are set to ``surrogateescape``. 240 241The forced UTF-8 mode can be used to change the text handling behavior in 242an embedded Python interpreter without changing the locale settings of 243an embedding application. 244 245While :pep:`540`'s UTF-8 mode has the benefit of working regardless of which 246locales are available on the running system, it has the downside of having no 247effect on extension modules (such as GNU ``readline``), child processes running 248non-Python applications, and child processes running older versions of Python. 249To reduce the risk of corrupting text data when communicating with such 250components, Python 3.7 also implements :ref:`whatsnew37-pep540`). 251 252The UTF-8 mode is enabled by default when the locale is ``C`` or ``POSIX``, and 253the :pep:`538` locale coercion feature fails to change it to a UTF-8 based 254alternative (whether that failure is due to ``PYTHONCOERCECLOCALE=0`` being set, 255``LC_ALL`` being set, or the lack of a suitable target locale). 256 257.. seealso:: 258 259 :pep:`540` -- Add a new UTF-8 mode 260 PEP written and implemented by Victor Stinner 261 262 263.. _whatsnew37-pep553: 264 265PEP 553: Built-in ``breakpoint()`` 266---------------------------------- 267 268Python 3.7 includes the new built-in :func:`breakpoint` function as 269an easy and consistent way to enter the Python debugger. 270 271Built-in ``breakpoint()`` calls :func:`sys.breakpointhook`. By default, the 272latter imports :mod:`pdb` and then calls ``pdb.set_trace()``, but by binding 273``sys.breakpointhook()`` to the function of your choosing, ``breakpoint()`` can 274enter any debugger. Additionally, the environment variable 275:envvar:`PYTHONBREAKPOINT` can be set to the callable of your debugger of 276choice. Set ``PYTHONBREAKPOINT=0`` to completely disable built-in 277``breakpoint()``. 278 279.. seealso:: 280 281 :pep:`553` -- Built-in breakpoint() 282 PEP written and implemented by Barry Warsaw 283 284 285.. _whatsnew37-pep539: 286 287PEP 539: New C API for Thread-Local Storage 288------------------------------------------- 289 290While Python provides a C API for thread-local storage support; the existing 291:ref:`Thread Local Storage (TLS) API <thread-local-storage-api>` has used 292:c:type:`int` to represent TLS keys across all platforms. This has not 293generally been a problem for officially-support platforms, but that is neither 294POSIX-compliant, nor portable in any practical sense. 295 296:pep:`539` changes this by providing a new :ref:`Thread Specific Storage (TSS) 297API <thread-specific-storage-api>` to CPython which supersedes use of the 298existing TLS API within the CPython interpreter, while deprecating the existing 299API. The TSS API uses a new type :c:type:`Py_tss_t` instead of :c:type:`int` 300to represent TSS keys--an opaque type the definition of which may depend on 301the underlying TLS implementation. Therefore, this will allow to build CPython 302on platforms where the native TLS key is defined in a way that cannot be safely 303cast to :c:type:`int`. 304 305Note that on platforms where the native TLS key is defined in a way that cannot 306be safely cast to :c:type:`int`, all functions of the existing TLS API will be 307no-op and immediately return failure. This indicates clearly that the old API 308is not supported on platforms where it cannot be used reliably, and that no 309effort will be made to add such support. 310 311.. seealso:: 312 313 :pep:`539` -- A New C-API for Thread-Local Storage in CPython 314 PEP written by Erik M. Bray; implementation by Masayuki Yamamoto. 315 316 317.. _whatsnew37-pep562: 318 319PEP 562: Customization of Access to Module Attributes 320----------------------------------------------------- 321 322Python 3.7 allows defining :meth:`__getattr__` on modules and will call 323it whenever a module attribute is otherwise not found. Defining 324:meth:`__dir__` on modules is now also allowed. 325 326A typical example of where this may be useful is module attribute deprecation 327and lazy loading. 328 329.. seealso:: 330 331 :pep:`562` -- Module ``__getattr__`` and ``__dir__`` 332 PEP written and implemented by Ivan Levkivskyi 333 334 335.. _whatsnew37-pep564: 336 337PEP 564: New Time Functions With Nanosecond Resolution 338------------------------------------------------------ 339 340The resolution of clocks in modern systems can exceed the limited precision 341of a floating point number returned by the :func:`time.time` function 342and its variants. To avoid loss of precision, :pep:`564` adds six new 343"nanosecond" variants of the existing timer functions to the :mod:`time` 344module: 345 346* :func:`time.clock_gettime_ns` 347* :func:`time.clock_settime_ns` 348* :func:`time.monotonic_ns` 349* :func:`time.perf_counter_ns` 350* :func:`time.process_time_ns` 351* :func:`time.time_ns` 352 353The new functions return the number of nanoseconds as an integer value. 354 355`Measurements <https://www.python.org/dev/peps/pep-0564/#annex-clocks-resolution-in-python>`_ 356show that on Linux and Windows the resolution of :func:`time.time_ns` is 357approximately 3 times better than that of :func:`time.time`. 358 359.. seealso:: 360 361 :pep:`564` -- Add new time functions with nanosecond resolution 362 PEP written and implemented by Victor Stinner 363 364 365.. _whatsnew37-pep565: 366 367PEP 565: Show DeprecationWarning in ``__main__`` 368------------------------------------------------ 369 370The default handling of :exc:`DeprecationWarning` has been changed such that 371these warnings are once more shown by default, but only when the code 372triggering them is running directly in the :mod:`__main__` module. As a result, 373developers of single file scripts and those using Python interactively should 374once again start seeing deprecation warnings for the APIs they use, but 375deprecation warnings triggered by imported application, library and framework 376modules will continue to be hidden by default. 377 378As a result of this change, the standard library now allows developers to choose 379between three different deprecation warning behaviours: 380 381* :exc:`FutureWarning`: always displayed by default, recommended for warnings 382 intended to be seen by application end users (e.g. for deprecated application 383 configuration settings). 384* :exc:`DeprecationWarning`: displayed by default only in :mod:`__main__` and when 385 running tests, recommended for warnings intended to be seen by other Python 386 developers where a version upgrade may result in changed behaviour or an 387 error. 388* :exc:`PendingDeprecationWarning`: displayed by default only when running 389 tests, intended for cases where a future version upgrade will change the 390 warning category to :exc:`DeprecationWarning` or :exc:`FutureWarning`. 391 392Previously both :exc:`DeprecationWarning` and :exc:`PendingDeprecationWarning` 393were only visible when running tests, which meant that developers primarily 394writing single file scripts or using Python interactively could be surprised 395by breaking changes in the APIs they used. 396 397.. seealso:: 398 399 :pep:`565` -- Show DeprecationWarning in ``__main__`` 400 PEP written and implemented by Nick Coghlan 401 402 403.. _whatsnew37-pep560: 404 405PEP 560: Core Support for ``typing`` module and Generic Types 406------------------------------------------------------------- 407 408Initially :pep:`484` was designed in such way that it would not introduce *any* 409changes to the core CPython interpreter. Now type hints and the :mod:`typing` 410module are extensively used by the community, so this restriction is removed. 411The PEP introduces two special methods :meth:`__class_getitem__` and 412``__mro_entries__``, these methods are now used by most classes and special 413constructs in :mod:`typing`. As a result, the speed of various operations 414with types increased up to 7 times, the generic types can be used without 415metaclass conflicts, and several long standing bugs in :mod:`typing` module are 416fixed. 417 418.. seealso:: 419 420 :pep:`560` -- Core support for typing module and generic types 421 PEP written and implemented by Ivan Levkivskyi 422 423 424.. _whatsnew37-pep552: 425 426PEP 552: Hash-based .pyc Files 427------------------------------ 428 429Python has traditionally checked the up-to-dateness of bytecode cache files 430(i.e., ``.pyc`` files) by comparing the source metadata (last-modified timestamp 431and size) with source metadata saved in the cache file header when it was 432generated. While effective, this invalidation method has its drawbacks. When 433filesystem timestamps are too coarse, Python can miss source updates, leading to 434user confusion. Additionally, having a timestamp in the cache file is 435problematic for `build reproducibility <https://reproducible-builds.org/>`_ and 436content-based build systems. 437 438:pep:`552` extends the pyc format to allow the hash of the source file to be 439used for invalidation instead of the source timestamp. Such ``.pyc`` files are 440called "hash-based". By default, Python still uses timestamp-based invalidation 441and does not generate hash-based ``.pyc`` files at runtime. Hash-based ``.pyc`` 442files may be generated with :mod:`py_compile` or :mod:`compileall`. 443 444Hash-based ``.pyc`` files come in two variants: checked and unchecked. Python 445validates checked hash-based ``.pyc`` files against the corresponding source 446files at runtime but doesn't do so for unchecked hash-based pycs. Unchecked 447hash-based ``.pyc`` files are a useful performance optimization for environments 448where a system external to Python (e.g., the build system) is responsible for 449keeping ``.pyc`` files up-to-date. 450 451See :ref:`pyc-invalidation` for more information. 452 453.. seealso:: 454 455 :pep:`552` -- Deterministic pycs 456 PEP written and implemented by Benjamin Peterson 457 458 459.. _whatsnew37-pep545: 460 461PEP 545: Python Documentation Translations 462------------------------------------------ 463 464:pep:`545` describes the process of creating and maintaining Python 465documentation translations. 466 467Three new translations have been added: 468 469- Japanese: https://docs.python.org/ja/ 470- French: https://docs.python.org/fr/ 471- Korean: https://docs.python.org/ko/ 472 473.. seealso:: 474 475 :pep:`545` -- Python Documentation Translations 476 PEP written and implemented by Julien Palard, Inada Naoki, and 477 Victor Stinner. 478 479 480.. _whatsnew37-devmode: 481 482Development Runtime Mode: -X dev 483-------------------------------- 484 485The new :option:`-X` ``dev`` command line option or the new 486:envvar:`PYTHONDEVMODE` environment variable can be used to enable 487CPython's *development mode*. When in development mode, CPython performs 488additional runtime checks that are too expensive to be enabled by default. 489See :option:`-X` ``dev`` documentation for the full description of the effects 490of this mode. 491 492 493Other Language Changes 494====================== 495 496* More than 255 arguments can now be passed to a function, and a function can 497 now have more than 255 parameters. (Contributed by Serhiy Storchaka in 498 :issue:`12844` and :issue:`18896`.) 499 500* :meth:`bytes.fromhex` and :meth:`bytearray.fromhex` now ignore all ASCII 501 whitespace, not only spaces. (Contributed by Robert Xiao in :issue:`28927`.) 502 503* :class:`str`, :class:`bytes`, and :class:`bytearray` gained support for 504 the new :meth:`isascii() <str.isascii>` method, which can be used to 505 test if a string or bytes contain only the ASCII characters. 506 (Contributed by INADA Naoki in :issue:`32677`.) 507 508* :exc:`ImportError` now displays module name and module ``__file__`` path when 509 ``from ... import ...`` fails. (Contributed by Matthias Bussonnier in 510 :issue:`29546`.) 511 512* Circular imports involving absolute imports with binding a submodule to 513 a name are now supported. 514 (Contributed by Serhiy Storchaka in :issue:`30024`.) 515 516* ``object.__format__(x, '')`` is now equivalent to ``str(x)`` rather than 517 ``format(str(self), '')``. 518 (Contributed by Serhiy Storchaka in :issue:`28974`.) 519 520* In order to better support dynamic creation of stack traces, 521 :class:`types.TracebackType` can now be instantiated from Python code, and 522 the ``tb_next`` attribute on :ref:`tracebacks <traceback-objects>` is now 523 writable. 524 (Contributed by Nathaniel J. Smith in :issue:`30579`.) 525 526* When using the :option:`-m` switch, ``sys.path[0]`` is now eagerly expanded 527 to the full starting directory path, rather than being left as the empty 528 directory (which allows imports from the *current* working directory at the 529 time when an import occurs) 530 (Contributed by Nick Coghlan in :issue:`33053`.) 531 532* The new :option:`-X` ``importtime`` option or the 533 :envvar:`PYTHONPROFILEIMPORTTIME` environment variable can be used to show 534 the timing of each module import. 535 (Contributed by Victor Stinner in :issue:`31415`.) 536 537 538New Modules 539=========== 540 541.. _whatsnew37-pep567: 542 543contextvars 544----------- 545 546The new :mod:`contextvars` module and a set of 547:ref:`new C APIs <contextvarsobjects>` introduce 548support for *context variables*. Context variables are conceptually 549similar to thread-local variables. Unlike TLS, context variables 550support asynchronous code correctly. 551 552The :mod:`asyncio` and :mod:`decimal` modules have been updated to use 553and support context variables out of the box. Particularly the active 554decimal context is now stored in a context variable, which allows 555decimal operations to work with the correct context in asynchronous code. 556 557.. seealso:: 558 559 :pep:`567` -- Context Variables 560 PEP written and implemented by Yury Selivanov 561 562 563.. _whatsnew37-pep557: 564 565dataclasses 566----------- 567 568The new :func:`~dataclasses.dataclass` decorator provides a way to declare 569*data classes*. A data class describes its attributes using class variable 570annotations. Its constructor and other magic methods, such as 571:meth:`~object.__repr__`, :meth:`~object.__eq__`, and 572:meth:`~object.__hash__` are generated automatically. 573 574Example:: 575 576 @dataclass 577 class Point: 578 x: float 579 y: float 580 z: float = 0.0 581 582 p = Point(1.5, 2.5) 583 print(p) # produces "Point(x=1.5, y=2.5, z=0.0)" 584 585.. seealso:: 586 587 :pep:`557` -- Data Classes 588 PEP written and implemented by Eric V. Smith 589 590 591.. _whatsnew37_importlib_resources: 592 593importlib.resources 594------------------- 595 596The new :mod:`importlib.resources` module provides several new APIs and one 597new ABC for access to, opening, and reading *resources* inside packages. 598Resources are roughly similar to files inside packages, but they needn't 599be actual files on the physical file system. Module loaders can provide a 600:meth:`get_resource_reader()` function which returns 601a :class:`importlib.abc.ResourceReader` instance to support this 602new API. Built-in file path loaders and zip file loaders both support this. 603 604Contributed by Barry Warsaw and Brett Cannon in :issue:`32248`. 605 606.. seealso:: 607 608 `importlib_resources <http://importlib-resources.readthedocs.io/en/latest/>`_ 609 -- a PyPI backport for earlier Python versions. 610 611 612Improved Modules 613================ 614 615 616argparse 617-------- 618 619The new :meth:`ArgumentParser.parse_intermixed_args() 620<argparse.ArgumentParser.parse_intermixed_args>` 621method allows intermixing options and positional arguments. 622(Contributed by paul.j3 in :issue:`14191`.) 623 624 625.. _whatsnew37_asyncio: 626 627asyncio 628------- 629 630The :mod:`asyncio` module has received many new features, usability and 631:ref:`performance improvements <whatsnew37-asyncio-perf>`. Notable changes 632include: 633 634* The new :term:`provisional <provisional api>` :func:`asyncio.run` function can 635 be used to run a coroutine from synchronous code by automatically creating and 636 destroying the event loop. 637 (Contributed by Yury Selivanov in :issue:`32314`.) 638 639* asyncio gained support for :mod:`contextvars`. 640 :meth:`loop.call_soon() <asyncio.loop.call_soon>`, 641 :meth:`loop.call_soon_threadsafe() <asyncio.loop.call_soon_threadsafe>`, 642 :meth:`loop.call_later() <asyncio.loop.call_later>`, 643 :meth:`loop.call_at() <asyncio.loop.call_at>`, and 644 :meth:`Future.add_done_callback() <asyncio.Future.add_done_callback>` 645 have a new optional keyword-only *context* parameter. 646 :class:`Tasks <asyncio.Task>` now track their context automatically. 647 See :pep:`567` for more details. 648 (Contributed by Yury Selivanov in :issue:`32436`.) 649 650* The new :func:`asyncio.create_task` function has been added as a shortcut 651 to ``asyncio.get_event_loop().create_task()``. 652 (Contributed by Andrew Svetlov in :issue:`32311`.) 653 654* The new :meth:`loop.start_tls() <asyncio.loop.start_tls>` 655 method can be used to upgrade an existing connection to TLS. 656 (Contributed by Yury Selivanov in :issue:`23749`.) 657 658* The new :meth:`loop.sock_recv_into() <asyncio.loop.sock_recv_into>` 659 method allows reading data from a socket directly into a provided buffer making 660 it possible to reduce data copies. 661 (Contributed by Antoine Pitrou in :issue:`31819`.) 662 663* The new :func:`asyncio.current_task` function returns the currently running 664 :class:`~asyncio.Task` instance, and the new :func:`asyncio.all_tasks` 665 function returns a set of all existing ``Task`` instances in a given loop. 666 The :meth:`Task.current_task() <asyncio.Task.current_task>` and 667 :meth:`Task.all_tasks() <asyncio.Task.all_tasks>` methods have been deprecated. 668 (Contributed by Andrew Svetlov in :issue:`32250`.) 669 670* The new *provisional* :class:`~asyncio.BufferedProtocol` class allows 671 implementing streaming protocols with manual control over the receive buffer. 672 (Contributed by Yury Selivanov in :issue:`32251`.) 673 674* The new :func:`asyncio.get_running_loop` function returns the currently 675 running loop, and raises a :exc:`RuntimeError` if no loop is running. 676 This is in contrast with :func:`asyncio.get_event_loop`, which will *create* 677 a new event loop if none is running. 678 (Contributed by Yury Selivanov in :issue:`32269`.) 679 680* The new :meth:`StreamWriter.wait_closed() <asyncio.StreamWriter.wait_closed>` 681 coroutine method allows waiting until the stream writer is closed. The new 682 :meth:`StreamWriter.is_closing() <asyncio.StreamWriter.is_closing>` method 683 can be used to determine if the writer is closing. 684 (Contributed by Andrew Svetlov in :issue:`32391`.) 685 686* The new :meth:`loop.sock_sendfile() <asyncio.loop.sock_sendfile>` 687 coroutine method allows sending files using :mod:`os.sendfile` when possible. 688 (Contributed by Andrew Svetlov in :issue:`32410`.) 689 690* The new :meth:`Future.get_loop() <asyncio.Future.get_loop>` and 691 ``Task.get_loop()`` methods return the instance of the loop on which a task or 692 a future were created. 693 :meth:`Server.get_loop() <asyncio.Server.get_loop>` allows doing the same for 694 :class:`asyncio.Server` objects. 695 (Contributed by Yury Selivanov in :issue:`32415` and 696 Srinivas Reddy Thatiparthy in :issue:`32418`.) 697 698* It is now possible to control how instances of :class:`asyncio.Server` begin 699 serving. Previously, the server would start serving immediately when created. 700 The new *start_serving* keyword argument to 701 :meth:`loop.create_server() <asyncio.loop.create_server>` and 702 :meth:`loop.create_unix_server() <asyncio.loop.create_unix_server>`, 703 as well as :meth:`Server.start_serving() <asyncio.Server.start_serving>`, and 704 :meth:`Server.serve_forever() <asyncio.Server.serve_forever>` 705 can be used to decouple server instantiation and serving. The new 706 :meth:`Server.is_serving() <asyncio.Server.is_serving>` method returns ``True`` 707 if the server is serving. :class:`~asyncio.Server` objects are now 708 asynchronous context managers:: 709 710 srv = await loop.create_server(...) 711 712 async with srv: 713 # some code 714 715 # At this point, srv is closed and no longer accepts new connections. 716 717 (Contributed by Yury Selivanov in :issue:`32662`.) 718 719* Callback objects returned by 720 :func:`loop.call_later() <asyncio.loop.call_later>` 721 gained the new :meth:`when() <asyncio.TimerHandle.when>` method which 722 returns an absolute scheduled callback timestamp. 723 (Contributed by Andrew Svetlov in :issue:`32741`.) 724 725* The :meth:`loop.create_datagram_endpoint() \ 726 <asyncio.loop.create_datagram_endpoint>` method 727 gained support for Unix sockets. 728 (Contributed by Quentin Dawans in :issue:`31245`.) 729 730* The :func:`asyncio.open_connection`, :func:`asyncio.start_server` functions, 731 :meth:`loop.create_connection() <asyncio.loop.create_connection>`, 732 :meth:`loop.create_server() <asyncio.loop.create_server>`, 733 :meth:`loop.create_accepted_socket() <asyncio.loop.connect_accepted_socket>` 734 methods and their corresponding UNIX socket variants now accept the 735 *ssl_handshake_timeout* keyword argument. 736 (Contributed by Neil Aspinall in :issue:`29970`.) 737 738* The new :meth:`Handle.cancelled() <asyncio.Handle.cancelled>` method returns 739 ``True`` if the callback was cancelled. 740 (Contributed by Marat Sharafutdinov in :issue:`31943`.) 741 742* The asyncio source has been converted to use the 743 :keyword:`async`/:keyword:`await` syntax. 744 (Contributed by Andrew Svetlov in :issue:`32193`.) 745 746* The new :meth:`ReadTransport.is_reading() <asyncio.ReadTransport.is_reading>` 747 method can be used to determine the reading state of the transport. 748 Additionally, calls to 749 :meth:`ReadTransport.resume_reading() <asyncio.ReadTransport.resume_reading>` 750 and :meth:`ReadTransport.pause_reading() <asyncio.ReadTransport.pause_reading>` 751 are now idempotent. 752 (Contributed by Yury Selivanov in :issue:`32356`.) 753 754* Loop methods which accept socket paths now support passing 755 :term:`path-like objects <path-like object>`. 756 (Contributed by Yury Selivanov in :issue:`32066`.) 757 758* In :mod:`asyncio` TCP sockets on Linux are now created with ``TCP_NODELAY`` 759 flag set by default. 760 (Contributed by Yury Selivanov and Victor Stinner in :issue:`27456`.) 761 762* Exceptions occurring in cancelled tasks are no longer logged. 763 (Contributed by Yury Selivanov in :issue:`30508`.) 764 765* New ``WindowsSelectorEventLoopPolicy`` and 766 ``WindowsProactorEventLoopPolicy`` classes. 767 (Contributed by Yury Selivanov in :issue:`33792`.) 768 769Several ``asyncio`` APIs have been 770:ref:`deprecated <whatsnew37-asyncio-deprecated>`. 771 772 773binascii 774-------- 775 776The :func:`~binascii.b2a_uu` function now accepts an optional *backtick* 777keyword argument. When it's true, zeros are represented by ``'`'`` 778instead of spaces. (Contributed by Xiang Zhang in :issue:`30103`.) 779 780 781calendar 782-------- 783 784The :class:`~calendar.HTMLCalendar` class has new class attributes which ease 785the customization of CSS classes in the produced HTML calendar. 786(Contributed by Oz Tiram in :issue:`30095`.) 787 788 789collections 790----------- 791 792``collections.namedtuple()`` now supports default values. 793(Contributed by Raymond Hettinger in :issue:`32320`.) 794 795 796compileall 797---------- 798 799:func:`compileall.compile_dir` learned the new *invalidation_mode* parameter, 800which can be used to enable 801:ref:`hash-based .pyc invalidation <whatsnew37-pep552>`. The invalidation 802mode can also be specified on the command line using the new 803``--invalidation-mode`` argument. 804(Contributed by Benjamin Peterson in :issue:`31650`.) 805 806 807concurrent.futures 808------------------ 809 810:class:`ProcessPoolExecutor <concurrent.futures.ProcessPoolExecutor>` and 811:class:`ThreadPoolExecutor <concurrent.futures.ThreadPoolExecutor>` now 812support the new *initializer* and *initargs* constructor arguments. 813(Contributed by Antoine Pitrou in :issue:`21423`.) 814 815The :class:`ProcessPoolExecutor <concurrent.futures.ProcessPoolExecutor>` 816can now take the multiprocessing context via the new *mp_context* argument. 817(Contributed by Thomas Moreau in :issue:`31540`.) 818 819 820contextlib 821---------- 822 823The new :func:`~contextlib.nullcontext` is a simpler and faster no-op 824context manager than :class:`~contextlib.ExitStack`. 825(Contributed by Jesse-Bakker in :issue:`10049`.) 826 827The new :func:`~contextlib.asynccontextmanager`, 828:class:`~contextlib.AbstractAsyncContextManager`, and 829:class:`~contextlib.AsyncExitStack` have been added to 830complement their synchronous counterparts. (Contributed 831by Jelle Zijlstra in :issue:`29679` and :issue:`30241`, 832and by Alexander Mohr and Ilya Kulakov in :issue:`29302`.) 833 834 835cProfile 836-------- 837 838The :mod:`cProfile` command line now accepts ``-m module_name`` as an 839alternative to script path. (Contributed by Sanyam Khurana in :issue:`21862`.) 840 841 842crypt 843----- 844 845The :mod:`crypt` module now supports the Blowfish hashing method. 846(Contributed by Serhiy Storchaka in :issue:`31664`.) 847 848The :func:`~crypt.mksalt` function now allows specifying the number of rounds 849for hashing. (Contributed by Serhiy Storchaka in :issue:`31702`.) 850 851 852datetime 853-------- 854 855The new :meth:`datetime.fromisoformat() <datetime.datetime.fromisoformat>` 856method constructs a :class:`~datetime.datetime` object from a string 857in one of the formats output by 858:meth:`datetime.isoformat() <datetime.datetime.isoformat>`. 859(Contributed by Paul Ganssle in :issue:`15873`.) 860 861The :class:`tzinfo <datetime.tzinfo>` class now supports sub-minute offsets. 862(Contributed by Alexander Belopolsky in :issue:`5288`.) 863 864 865dbm 866--- 867 868:mod:`dbm.dumb` now supports reading read-only files and no longer writes the 869index file when it is not changed. 870 871 872decimal 873------- 874 875The :mod:`decimal` module now uses :ref:`context variables <whatsnew37-pep567>` 876to store the decimal context. 877(Contributed by Yury Selivanov in :issue:`32630`.) 878 879 880dis 881--- 882 883The :func:`~dis.dis` function is now able to 884disassemble nested code objects (the code of comprehensions, generator 885expressions and nested functions, and the code used for building nested 886classes). The maximum depth of disassembly recursion is controlled by 887the new *depth* parameter. 888(Contributed by Serhiy Storchaka in :issue:`11822`.) 889 890 891distutils 892--------- 893 894``README.rst`` is now included in the list of distutils standard READMEs and 895therefore included in source distributions. 896(Contributed by Ryan Gonzalez in :issue:`11913`.) 897 898 899enum 900---- 901 902The :class:`Enum <enum.Enum>` learned the new ``_ignore_`` class property, 903which allows listing the names of properties which should not become 904enum members. 905(Contributed by Ethan Furman in :issue:`31801`.) 906 907In Python 3.8, attempting to check for non-Enum objects in :class:`Enum` 908classes will raise a :exc:`TypeError` (e.g. ``1 in Color``); similarly, 909attempting to check for non-Flag objects in a :class:`Flag` member will 910raise :exc:`TypeError` (e.g. ``1 in Perm.RW``); currently, both operations 911return :const:`False` instead and are deprecated. 912(Contributed by Ethan Furman in :issue:`33217`.) 913 914 915functools 916--------- 917 918:func:`functools.singledispatch` now supports registering implementations 919using type annotations. 920(Contributed by Łukasz Langa in :issue:`32227`.) 921 922 923gc 924-- 925 926The new :func:`gc.freeze` function allows freezing all objects tracked 927by the garbage collector and excluding them from future collections. 928This can be used before a POSIX ``fork()`` call to make the GC copy-on-write 929friendly or to speed up collection. The new :func:`gc.unfreeze` functions 930reverses this operation. Additionally, :func:`gc.get_freeze_count` can 931be used to obtain the number of frozen objects. 932(Contributed by Li Zekun in :issue:`31558`.) 933 934 935hmac 936---- 937 938The :mod:`hmac` module now has an optimized one-shot :func:`~hmac.digest` 939function, which is up to three times faster than :func:`~hmac.HMAC`. 940(Contributed by Christian Heimes in :issue:`32433`.) 941 942 943http.client 944----------- 945 946:class:`~http.client.HTTPConnection` and :class:`~http.client.HTTPSConnection` 947now support the new *blocksize* argument for improved upload throughput. 948(Contributed by Nir Soffer in :issue:`31945`.) 949 950 951http.server 952----------- 953 954:class:`~http.server.SimpleHTTPRequestHandler` now supports the HTTP 955``If-Modified-Since`` header. The server returns the 304 response status if 956the target file was not modified after the time specified in the header. 957(Contributed by Pierre Quentel in :issue:`29654`.) 958 959:class:`~http.server.SimpleHTTPRequestHandler` accepts the new *directory* 960argument, in addition to the new ``--directory`` command line argument. 961With this parameter, the server serves the specified directory, by default it 962uses the current working directory. 963(Contributed by Stéphane Wirtel and Julien Palard in :issue:`28707`.) 964 965The new :class:`ThreadingHTTPServer <http.server.ThreadingHTTPServer>` class 966uses threads to handle requests using :class:`~socketserver.ThreadingMixin`. 967It is used when ``http.server`` is run with ``-m``. 968(Contributed by Julien Palard in :issue:`31639`.) 969 970 971idlelib and IDLE 972---------------- 973 974Multiple fixes for autocompletion. (Contributed by Louie Lu in :issue:`15786`.) 975 976Module Browser (on the File menu, formerly called Class Browser), 977now displays nested functions and classes in addition to top-level 978functions and classes. 979(Contributed by Guilherme Polo, Cheryl Sabella, and Terry Jan Reedy 980in :issue:`1612262`.) 981 982The Settings dialog (Options, Configure IDLE) has been partly rewritten 983to improve both appearance and function. 984(Contributed by Cheryl Sabella and Terry Jan Reedy in multiple issues.) 985 986The font sample now includes a selection of non-Latin characters so that 987users can better see the effect of selecting a particular font. 988(Contributed by Terry Jan Reedy in :issue:`13802`.) 989The sample can be edited to include other characters. 990(Contributed by Serhiy Storchaka in :issue:`31860`.) 991 992The IDLE features formerly implemented as extensions have been reimplemented 993as normal features. Their settings have been moved from the Extensions tab 994to other dialog tabs. 995(Contributed by Charles Wohlganger and Terry Jan Reedy in :issue:`27099`.) 996 997Editor code context option revised. Box displays all context lines up to 998maxlines. Clicking on a context line jumps the editor to that line. Context 999colors for custom themes is added to Highlights tab of Settings dialog. 1000(Contributed by Cheryl Sabella and Terry Jan Reedy in :issue:`33642`, 1001:issue:`33768`, and :issue:`33679`.) 1002 1003On Windows, a new API call tells Windows that tk scales for DPI. On Windows 10048.1+ or 10, with DPI compatibility properties of the Python binary 1005unchanged, and a monitor resolution greater than 96 DPI, this should 1006make text and lines sharper. It should otherwise have no effect. 1007(Contributed by Terry Jan Reedy in :issue:`33656`.) 1008 1009New in 3.7.1: 1010 1011Output over N lines (50 by default) is squeezed down to a button. 1012N can be changed in the PyShell section of the General page of the 1013Settings dialog. Fewer, but possibly extra long, lines can be squeezed by 1014right clicking on the output. Squeezed output can be expanded in place 1015by double-clicking the button or into the clipboard or a separate window 1016by right-clicking the button. (Contributed by Tal Einat in :issue:`1529353`.) 1017 1018The changes above have been backported to 3.6 maintenance releases. 1019 1020 1021importlib 1022--------- 1023 1024The :class:`importlib.abc.ResourceReader` ABC was introduced to 1025support the loading of resources from packages. See also 1026:ref:`whatsnew37_importlib_resources`. 1027(Contributed by Barry Warsaw, Brett Cannon in :issue:`32248`.) 1028 1029:func:`importlib.reload` now raises :exc:`ModuleNotFoundError` if the module 1030lacks a spec. 1031(Contributed by Garvit Khatri in :issue:`29851`.) 1032 1033:func:`importlib.find_spec` now raises :exc:`ModuleNotFoundError` instead of 1034:exc:`AttributeError` if the specified parent module is not a package (i.e. 1035lacks a ``__path__`` attribute). 1036(Contributed by Milan Oberkirch in :issue:`30436`.) 1037 1038The new :func:`importlib.source_hash` can be used to compute the hash of 1039the passed source. A :ref:`hash-based .pyc file <whatsnew37-pep552>` 1040embeds the value returned by this function. 1041 1042 1043io 1044-- 1045 1046The new :meth:`TextIOWrapper.reconfigure() <io.TextIOWrapper.reconfigure>` 1047method can be used to reconfigure the text stream with the new settings. 1048(Contributed by Antoine Pitrou in :issue:`30526` and 1049INADA Naoki in :issue:`15216`.) 1050 1051 1052ipaddress 1053--------- 1054 1055The new ``subnet_of()`` and ``supernet_of()`` methods of 1056:class:`ipaddress.IPv6Network` and :class:`ipaddress.IPv4Network` can 1057be used for network containment tests. 1058(Contributed by Michel Albert and Cheryl Sabella in :issue:`20825`.) 1059 1060 1061itertools 1062--------- 1063 1064:func:`itertools.islice` now accepts 1065:meth:`integer-like objects <object.__index__>` as start, stop, 1066and slice arguments. 1067(Contributed by Will Roberts in :issue:`30537`.) 1068 1069 1070locale 1071------ 1072 1073The new *monetary* argument to :func:`locale.format_string` can be used 1074to make the conversion use monetary thousands separators and 1075grouping strings. (Contributed by Garvit in :issue:`10379`.) 1076 1077The :func:`locale.getpreferredencoding` function now always returns ``'UTF-8'`` 1078on Android or when in the :ref:`forced UTF-8 mode <whatsnew37-pep540>`. 1079 1080 1081logging 1082------- 1083 1084:class:`~logging.Logger` instances can now be pickled. 1085(Contributed by Vinay Sajip in :issue:`30520`.) 1086 1087The new :meth:`StreamHandler.setStream() <logging.StreamHandler.setStream>` 1088method can be used to replace the logger stream after handler creation. 1089(Contributed by Vinay Sajip in :issue:`30522`.) 1090 1091It is now possible to specify keyword arguments to handler constructors in 1092configuration passed to :func:`logging.config.fileConfig`. 1093(Contributed by Preston Landers in :issue:`31080`.) 1094 1095 1096math 1097---- 1098 1099The new :func:`math.remainder` function implements the IEEE 754-style remainder 1100operation. (Contributed by Mark Dickinson in :issue:`29962`.) 1101 1102 1103mimetypes 1104--------- 1105 1106The MIME type of .bmp has been changed from ``'image/x-ms-bmp'`` to 1107``'image/bmp'``. 1108(Contributed by Nitish Chandra in :issue:`22589`.) 1109 1110 1111msilib 1112------ 1113 1114The new :meth:`Database.Close() <msilib.Database.Close>` method can be used 1115to close the :abbr:`MSI` database. 1116(Contributed by Berker Peksag in :issue:`20486`.) 1117 1118 1119multiprocessing 1120--------------- 1121 1122The new :meth:`Process.close() <multiprocessing.Process.close>` method 1123explicitly closes the process object and releases all resources associated 1124with it. :exc:`ValueError` is raised if the underlying process is still 1125running. 1126(Contributed by Antoine Pitrou in :issue:`30596`.) 1127 1128The new :meth:`Process.kill() <multiprocessing.Process.kill>` method can 1129be used to terminate the process using the :data:`SIGKILL` signal on Unix. 1130(Contributed by Vitor Pereira in :issue:`30794`.) 1131 1132Non-daemonic threads created by :class:`~multiprocessing.Process` are now 1133joined on process exit. 1134(Contributed by Antoine Pitrou in :issue:`18966`.) 1135 1136 1137os 1138-- 1139 1140:func:`os.fwalk` now accepts the *path* argument as :class:`bytes`. 1141(Contributed by Serhiy Storchaka in :issue:`28682`.) 1142 1143:func:`os.scandir` gained support for :ref:`file descriptors <path_fd>`. 1144(Contributed by Serhiy Storchaka in :issue:`25996`.) 1145 1146The new :func:`~os.register_at_fork` function allows registering Python 1147callbacks to be executed at process fork. 1148(Contributed by Antoine Pitrou in :issue:`16500`.) 1149 1150Added :func:`os.preadv` (combine the functionality of :func:`os.readv` and 1151:func:`os.pread`) and :func:`os.pwritev` functions (combine the functionality 1152of :func:`os.writev` and :func:`os.pwrite`). (Contributed by Pablo Galindo in 1153:issue:`31368`.) 1154 1155The mode argument of :func:`os.makedirs` no longer affects the file 1156permission bits of newly-created intermediate-level directories. 1157(Contributed by Serhiy Storchaka in :issue:`19930`.) 1158 1159:func:`os.dup2` now returns the new file descriptor. Previously, ``None`` 1160was always returned. 1161(Contributed by Benjamin Peterson in :issue:`32441`.) 1162 1163The structure returned by :func:`os.stat` now contains the 1164:attr:`~os.stat_result.st_fstype` attribute on Solaris and its derivatives. 1165(Contributed by Jesús Cea Avión in :issue:`32659`.) 1166 1167 1168pathlib 1169------- 1170 1171The new :meth:`Path.is_mount() <pathlib.Path.is_mount>` method is now available 1172on POSIX systems and can be used to determine whether a path is a mount point. 1173(Contributed by Cooper Ry Lees in :issue:`30897`.) 1174 1175 1176pdb 1177--- 1178 1179:func:`pdb.set_trace` now takes an optional *header* keyword-only 1180argument. If given, it is printed to the console just before debugging 1181begins. (Contributed by Barry Warsaw in :issue:`31389`.) 1182 1183:mod:`pdb` command line now accepts ``-m module_name`` as an alternative to 1184script file. (Contributed by Mario Corchero in :issue:`32206`.) 1185 1186 1187py_compile 1188---------- 1189 1190:func:`py_compile.compile` -- and by extension, :mod:`compileall` -- now 1191respects the :envvar:`SOURCE_DATE_EPOCH` environment variable by 1192unconditionally creating ``.pyc`` files for hash-based validation. 1193This allows for guaranteeing 1194`reproducible builds <https://reproducible-builds.org/>`_ of ``.pyc`` 1195files when they are created eagerly. (Contributed by Bernhard M. Wiedemann 1196in :issue:`29708`.) 1197 1198 1199pydoc 1200----- 1201 1202The pydoc server can now bind to an arbitrary hostname specified by the 1203new ``-n`` command-line argument. 1204(Contributed by Feanil Patel in :issue:`31128`.) 1205 1206 1207queue 1208----- 1209 1210The new :class:`~queue.SimpleQueue` class is an unbounded :abbr:`FIFO` queue. 1211(Contributed by Antoine Pitrou in :issue:`14976`.) 1212 1213 1214re 1215-- 1216 1217The flags :const:`re.ASCII`, :const:`re.LOCALE` and :const:`re.UNICODE` 1218can be set within the scope of a group. 1219(Contributed by Serhiy Storchaka in :issue:`31690`.) 1220 1221:func:`re.split` now supports splitting on a pattern like ``r'\b'``, 1222``'^$'`` or ``(?=-)`` that matches an empty string. 1223(Contributed by Serhiy Storchaka in :issue:`25054`.) 1224 1225Regular expressions compiled with the :const:`re.LOCALE` flag no longer 1226depend on the locale at compile time. Locale settings are applied only 1227when the compiled regular expression is used. 1228(Contributed by Serhiy Storchaka in :issue:`30215`.) 1229 1230:exc:`FutureWarning` is now emitted if a regular expression contains 1231character set constructs that will change semantically in the future, 1232such as nested sets and set operations. 1233(Contributed by Serhiy Storchaka in :issue:`30349`.) 1234 1235Compiled regular expression and match objects can now be copied 1236using :func:`copy.copy` and :func:`copy.deepcopy`. 1237(Contributed by Serhiy Storchaka in :issue:`10076`.) 1238 1239 1240signal 1241------ 1242 1243The new *warn_on_full_buffer* argument to the :func:`signal.set_wakeup_fd` 1244function makes it possible to specify whether Python prints a warning on 1245stderr when the wakeup buffer overflows. 1246(Contributed by Nathaniel J. Smith in :issue:`30050`.) 1247 1248 1249socket 1250------ 1251 1252The new :func:`socket.getblocking() <socket.socket.getblocking>` method 1253returns ``True`` if the socket is in blocking mode and ``False`` otherwise. 1254(Contributed by Yury Selivanov in :issue:`32373`.) 1255 1256The new :func:`socket.close` function closes the passed socket file descriptor. 1257This function should be used instead of :func:`os.close` for better 1258compatibility across platforms. 1259(Contributed by Christian Heimes in :issue:`32454`.) 1260 1261The :mod:`socket` module now exposes the :data:`socket.TCP_CONGESTION` 1262(Linux 2.6.13), :data:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37), and 1263:data:`socket.TCP_NOTSENT_LOWAT` (Linux 3.12) constants. 1264(Contributed by Omar Sandoval in :issue:`26273` and 1265Nathaniel J. Smith in :issue:`29728`.) 1266 1267Support for :data:`socket.AF_VSOCK` sockets has been added to allow 1268communication between virtual machines and their hosts. 1269(Contributed by Cathy Avery in :issue:`27584`.) 1270 1271Sockets now auto-detect family, type and protocol from file descriptor 1272by default. 1273(Contributed by Christian Heimes in :issue:`28134`.) 1274 1275 1276socketserver 1277------------ 1278 1279:meth:`socketserver.ThreadingMixIn.server_close` now waits until all non-daemon 1280threads complete. :meth:`socketserver.ForkingMixIn.server_close` now waits 1281until all child processes complete. 1282 1283Add a new :attr:`socketserver.ForkingMixIn.block_on_close` class attribute to 1284:class:`socketserver.ForkingMixIn` and :class:`socketserver.ThreadingMixIn` 1285classes. Set the class attribute to ``False`` to get the pre-3.7 behaviour. 1286 1287 1288sqlite3 1289------- 1290 1291:class:`sqlite3.Connection` now exposes the :meth:`~sqlite3.Connection.backup` 1292method when the underlying SQLite library is at version 3.6.11 or higher. 1293(Contributed by Lele Gaifax in :issue:`27645`.) 1294 1295The *database* argument of :func:`sqlite3.connect` now accepts any 1296:term:`path-like object`, instead of just a string. 1297(Contributed by Anders Lorentsen in :issue:`31843`.) 1298 1299 1300ssl 1301--- 1302 1303The :mod:`ssl` module now uses OpenSSL's builtin API instead of 1304:func:`~ssl.match_hostname` to check a host name or an IP address. Values 1305are validated during TLS handshake. Any certificate validation error 1306including failing the host name check now raises 1307:exc:`~ssl.SSLCertVerificationError` and aborts the handshake with a proper 1308TLS Alert message. The new exception contains additional information. 1309Host name validation can be customized with 1310:attr:`SSLContext.host_flags <ssl.SSLContext.host_flags>`. 1311(Contributed by Christian Heimes in :issue:`31399`.) 1312 1313.. note:: 1314 The improved host name check requires a *libssl* implementation compatible 1315 with OpenSSL 1.0.2 or 1.1. Consequently, OpenSSL 0.9.8 and 1.0.1 are no 1316 longer supported (see :ref:`37-platform-support-removals` for more details). 1317 The ssl module is mostly compatible with LibreSSL 2.7.2 and newer. 1318 1319The ``ssl`` module no longer sends IP addresses in SNI TLS extension. 1320(Contributed by Christian Heimes in :issue:`32185`.) 1321 1322:func:`~ssl.match_hostname` no longer supports partial wildcards like 1323``www*.example.org``. :attr:`SSLContext.host_flags <ssl.SSLContext.host_flags>` 1324has partial wildcard matching disabled by default. 1325(Contributed by Mandeep Singh in :issue:`23033` and Christian Heimes in 1326:issue:`31399`.) 1327 1328The default cipher suite selection of the ``ssl`` module now uses a blacklist 1329approach rather than a hard-coded whitelist. Python no longer re-enables 1330ciphers that have been blocked by OpenSSL security updates. Default cipher 1331suite selection can be configured at compile time. 1332(Contributed by Christian Heimes in :issue:`31429`.) 1333 1334Validation of server certificates containing internationalized domain names 1335(IDNs) is now supported. As part of this change, the 1336:attr:`SSLSocket.server_hostname <ssl.SSLSocket.server_hostname>` attribute 1337now stores the expected hostname in A-label form (``"xn--pythn-mua.org"``), 1338rather than the U-label form (``"pythön.org"``). (Contributed by 1339Nathaniel J. Smith and Christian Heimes in :issue:`28414`.) 1340 1341The ``ssl`` module has preliminary and experimental support for TLS 1.3 and 1342OpenSSL 1.1.1. At the time of Python 3.7.0 release, OpenSSL 1.1.1 is still 1343under development and TLS 1.3 hasn't been finalized yet. The TLS 1.3 1344handshake and protocol behaves slightly differently than TLS 1.2 and earlier, 1345see :ref:`ssl-tlsv1_3`. 1346(Contributed by Christian Heimes in :issue:`32947`, :issue:`20995`, 1347:issue:`29136`, :issue:`30622` and :issue:`33618`) 1348 1349:class:`~ssl.SSLSocket` and :class:`~ssl.SSLObject` no longer have a public 1350constructor. Direct instantiation was never a documented and supported 1351feature. Instances must be created with :class:`~ssl.SSLContext` methods 1352:meth:`~ssl.SSLContext.wrap_socket` and :meth:`~ssl.SSLContext.wrap_bio`. 1353(Contributed by Christian Heimes in :issue:`32951`) 1354 1355OpenSSL 1.1 APIs for setting the minimum and maximum TLS protocol version are 1356available as :attr:`SSLContext.minimum_version <ssl.SSLContext.minimum_version>` 1357and :attr:`SSLContext.maximum_version <ssl.SSLContext.maximum_version>`. 1358Supported protocols are indicated by several new flags, such as 1359:data:`~ssl.HAS_TLSv1_1`. 1360(Contributed by Christian Heimes in :issue:`32609`.) 1361 1362Added :attr:`SSLContext.post_handshake_auth` to enable and 1363:meth:`ssl.SSLSocket.verify_client_post_handshake` to initiate TLS 1.3 1364post-handshake authentication. 1365(Contributed by Christian Heimes in :issue:`34670`.) 1366 1367string 1368------ 1369 1370:class:`string.Template` now lets you to optionally modify the regular 1371expression pattern for braced placeholders and non-braced placeholders 1372separately. (Contributed by Barry Warsaw in :issue:`1198569`.) 1373 1374 1375subprocess 1376---------- 1377 1378The :func:`subprocess.run` function accepts the new *capture_output* 1379keyword argument. When true, stdout and stderr will be captured. 1380This is equivalent to passing :data:`subprocess.PIPE` as *stdout* and 1381*stderr* arguments. 1382(Contributed by Bo Bayles in :issue:`32102`.) 1383 1384The ``subprocess.run`` function and the :class:`subprocess.Popen` constructor 1385now accept the *text* keyword argument as an alias 1386to *universal_newlines*. 1387(Contributed by Andrew Clegg in :issue:`31756`.) 1388 1389On Windows the default for *close_fds* was changed from ``False`` to 1390``True`` when redirecting the standard handles. It's now possible to set 1391*close_fds* to true when redirecting the standard handles. See 1392:class:`subprocess.Popen`. This means that *close_fds* now defaults to 1393``True`` on all supported platforms. 1394(Contributed by Segev Finer in :issue:`19764`.) 1395 1396The subprocess module is now more graceful when handling 1397:exc:`KeyboardInterrupt` during :func:`subprocess.call`, 1398:func:`subprocess.run`, or in a :class:`~subprocess.Popen` 1399context manager. It now waits a short amount of time for the child 1400to exit, before continuing the handling of the ``KeyboardInterrupt`` 1401exception. 1402(Contributed by Gregory P. Smith in :issue:`25942`.) 1403 1404 1405sys 1406--- 1407 1408The new :func:`sys.breakpointhook` hook function is called by the 1409built-in :func:`breakpoint`. 1410(Contributed by Barry Warsaw in :issue:`31353`.) 1411 1412On Android, the new :func:`sys.getandroidapilevel` returns the build-time 1413Android API version. 1414(Contributed by Victor Stinner in :issue:`28740`.) 1415 1416The new :func:`sys.get_coroutine_origin_tracking_depth` function returns 1417the current coroutine origin tracking depth, as set by 1418the new :func:`sys.set_coroutine_origin_tracking_depth`. :mod:`asyncio` 1419has been converted to use this new API instead of 1420the deprecated :func:`sys.set_coroutine_wrapper`. 1421(Contributed by Nathaniel J. Smith in :issue:`32591`.) 1422 1423 1424time 1425---- 1426 1427:pep:`564` adds six new functions with nanosecond resolution to the 1428:mod:`time` module: 1429 1430* :func:`time.clock_gettime_ns` 1431* :func:`time.clock_settime_ns` 1432* :func:`time.monotonic_ns` 1433* :func:`time.perf_counter_ns` 1434* :func:`time.process_time_ns` 1435* :func:`time.time_ns` 1436 1437New clock identifiers have been added: 1438 1439* :data:`time.CLOCK_BOOTTIME` (Linux): Identical to 1440 :data:`time.CLOCK_MONOTONIC`, except it also includes any time that the 1441 system is suspended. 1442* :data:`time.CLOCK_PROF` (FreeBSD, NetBSD and OpenBSD): High-resolution 1443 per-process CPU timer. 1444* :data:`time.CLOCK_UPTIME` (FreeBSD, OpenBSD): Time whose absolute value is 1445 the time the system has been running and not suspended, providing accurate 1446 uptime measurement. 1447 1448The new :func:`time.thread_time` and :func:`time.thread_time_ns` functions 1449can be used to get per-thread CPU time measurements. 1450(Contributed by Antoine Pitrou in :issue:`32025`.) 1451 1452The new :func:`time.pthread_getcpuclockid` function returns the clock ID 1453of the thread-specific CPU-time clock. 1454 1455 1456tkinter 1457------- 1458 1459The new :class:`tkinter.ttk.Spinbox` class is now available. 1460(Contributed by Alan Moore in :issue:`32585`.) 1461 1462 1463tracemalloc 1464----------- 1465 1466:class:`tracemalloc.Traceback` behaves more like regular tracebacks, 1467sorting the frames from oldest to most recent. 1468:meth:`Traceback.format() <tracemalloc.Traceback.format>` 1469now accepts negative *limit*, truncating the result to the 1470``abs(limit)`` oldest frames. To get the old behaviour, use 1471the new *most_recent_first* argument to ``Traceback.format()``. 1472(Contributed by Jesse Bakker in :issue:`32121`.) 1473 1474 1475types 1476----- 1477 1478The new :class:`~types.WrapperDescriptorType`, 1479:class:`~types.MethodWrapperType`, :class:`~types.MethodDescriptorType`, 1480and :class:`~types.ClassMethodDescriptorType` classes are now available. 1481(Contributed by Manuel Krebber and Guido van Rossum in :issue:`29377`, 1482and Serhiy Storchaka in :issue:`32265`.) 1483 1484The new :func:`types.resolve_bases` function resolves MRO entries 1485dynamically as specified by :pep:`560`. 1486(Contributed by Ivan Levkivskyi in :issue:`32717`.) 1487 1488 1489unicodedata 1490----------- 1491 1492The internal :mod:`unicodedata` database has been upgraded to use `Unicode 11 1493<http://www.unicode.org/versions/Unicode11.0.0/>`_. (Contributed by Benjamin 1494Peterson.) 1495 1496 1497unittest 1498-------- 1499 1500The new ``-k`` command-line option allows filtering tests by a name 1501substring or a Unix shell-like pattern. 1502For example, ``python -m unittest -k foo`` runs 1503``foo_tests.SomeTest.test_something``, ``bar_tests.SomeTest.test_foo``, 1504but not ``bar_tests.FooTest.test_something``. 1505(Contributed by Jonas Haag in :issue:`32071`.) 1506 1507 1508unittest.mock 1509------------- 1510 1511The :const:`~unittest.mock.sentinel` attributes now preserve their identity 1512when they are :mod:`copied <copy>` or :mod:`pickled <pickle>`. (Contributed by 1513Serhiy Storchaka in :issue:`20804`.) 1514 1515The new :func:`~unittest.mock.seal` function allows sealing 1516:class:`~unittest.mock.Mock` instances, which will disallow further creation 1517of attribute mocks. The seal is applied recursively to all attributes that 1518are themselves mocks. 1519(Contributed by Mario Corchero in :issue:`30541`.) 1520 1521 1522urllib.parse 1523------------ 1524 1525:func:`urllib.parse.quote` has been updated from :rfc:`2396` to :rfc:`3986`, 1526adding ``~`` to the set of characters that are never quoted by default. 1527(Contributed by Christian Theune and Ratnadeep Debnath in :issue:`16285`.) 1528 1529 1530uu 1531-- 1532 1533The :func:`uu.encode` function now accepts an optional *backtick* 1534keyword argument. When it's true, zeros are represented by ``'`'`` 1535instead of spaces. (Contributed by Xiang Zhang in :issue:`30103`.) 1536 1537 1538uuid 1539---- 1540 1541The new :attr:`UUID.is_safe <uuid.UUID.is_safe>` attribute relays information 1542from the platform about whether generated UUIDs are generated with a 1543multiprocessing-safe method. 1544(Contributed by Barry Warsaw in :issue:`22807`.) 1545 1546:func:`uuid.getnode` now prefers universally administered 1547MAC addresses over locally administered MAC addresses. 1548This makes a better guarantee for global uniqueness of UUIDs returned 1549from :func:`uuid.uuid1`. If only locally administered MAC addresses are 1550available, the first such one found is returned. 1551(Contributed by Barry Warsaw in :issue:`32107`.) 1552 1553 1554warnings 1555-------- 1556 1557The initialization of the default warnings filters has changed as follows: 1558 1559* warnings enabled via command line options (including those for :option:`-b` 1560 and the new CPython-specific :option:`-X` ``dev`` option) are always passed 1561 to the warnings machinery via the :data:`sys.warnoptions` attribute. 1562 1563* warnings filters enabled via the command line or the environment now have the 1564 following order of precedence: 1565 1566 * the ``BytesWarning`` filter for :option:`-b` (or ``-bb``) 1567 * any filters specified with the :option:`-W` option 1568 * any filters specified with the :envvar:`PYTHONWARNINGS` environment 1569 variable 1570 * any other CPython specific filters (e.g. the ``default`` filter added 1571 for the new ``-X dev`` mode) 1572 * any implicit filters defined directly by the warnings machinery 1573 1574* in CPython debug builds, all warnings are now displayed by default (the 1575 implicit filter list is empty) 1576 1577(Contributed by Nick Coghlan and Victor Stinner in :issue:`20361`, 1578:issue:`32043`, and :issue:`32230`.) 1579 1580Deprecation warnings are once again shown by default in single-file scripts and 1581at the interactive prompt. See :ref:`whatsnew37-pep565` for details. 1582(Contributed by Nick Coghlan in :issue:`31975`.) 1583 1584 1585xml 1586--- 1587 1588As mitigation against DTD and external entity retrieval, the 1589:mod:`xml.dom.minidom` and :mod:`xml.sax` modules no longer process 1590external entities by default. 1591(Contributed by Christian Heimes in :issue:`17239`.) 1592 1593 1594xml.etree 1595--------- 1596 1597:ref:`ElementPath <elementtree-xpath>` predicates in the :meth:`find` 1598methods can now compare text of the current node with ``[. = "text"]``, 1599not only text in children. Predicates also allow adding spaces for 1600better readability. (Contributed by Stefan Behnel in :issue:`31648`.) 1601 1602 1603xmlrpc.server 1604------------- 1605 1606:meth:`SimpleXMLRPCDispatcher.register_function <xmlrpc.server.SimpleXMLRPCDispatcher>` 1607can now be used as a decorator. (Contributed by Xiang Zhang in 1608:issue:`7769`.) 1609 1610 1611zipapp 1612------ 1613 1614Function :func:`~zipapp.create_archive` now accepts an optional *filter* 1615argument to allow the user to select which files should be included in the 1616archive. (Contributed by Irmen de Jong in :issue:`31072`.) 1617 1618Function :func:`~zipapp.create_archive` now accepts an optional *compressed* 1619argument to generate a compressed archive. A command line option 1620``--compress`` has also been added to support compression. 1621(Contributed by Zhiming Wang in :issue:`31638`.) 1622 1623 1624zipfile 1625------- 1626 1627:class:`~zipfile.ZipFile` now accepts the new *compresslevel* parameter to 1628control the compression level. 1629(Contributed by Bo Bayles in :issue:`21417`.) 1630 1631Subdirectories in archives created by ``ZipFile`` are now stored in 1632alphabetical order. 1633(Contributed by Bernhard M. Wiedemann in :issue:`30693`.) 1634 1635 1636C API Changes 1637============= 1638 1639A new API for thread-local storage has been implemented. See 1640:ref:`whatsnew37-pep539` for an overview and 1641:ref:`thread-specific-storage-api` for a complete reference. 1642(Contributed by Masayuki Yamamoto in :issue:`25658`.) 1643 1644The new :ref:`context variables <whatsnew37-pep567>` functionality 1645exposes a number of :ref:`new C APIs <contextvarsobjects>`. 1646 1647The new :c:func:`PyImport_GetModule` function returns the previously 1648imported module with the given name. 1649(Contributed by Eric Snow in :issue:`28411`.) 1650 1651The new :c:macro:`Py_RETURN_RICHCOMPARE` macro eases writing rich 1652comparison functions. 1653(Contributed by Petr Victorin in :issue:`23699`.) 1654 1655The new :c:macro:`Py_UNREACHABLE` macro can be used to mark unreachable 1656code paths. 1657(Contributed by Barry Warsaw in :issue:`31338`.) 1658 1659The :mod:`tracemalloc` now exposes a C API through the new 1660:c:func:`PyTraceMalloc_Track` and :c:func:`PyTraceMalloc_Untrack` 1661functions. 1662(Contributed by Victor Stinner in :issue:`30054`.) 1663 1664The new :c:func:`import__find__load__start` and 1665:c:func:`import__find__load__done` static markers can be used to trace 1666module imports. 1667(Contributed by Christian Heimes in :issue:`31574`.) 1668 1669The fields :c:member:`name` and :c:member:`doc` of structures 1670:c:type:`PyMemberDef`, :c:type:`PyGetSetDef`, 1671:c:type:`PyStructSequence_Field`, :c:type:`PyStructSequence_Desc`, 1672and :c:type:`wrapperbase` are now of type ``const char *`` rather of 1673``char *``. (Contributed by Serhiy Storchaka in :issue:`28761`.) 1674 1675The result of :c:func:`PyUnicode_AsUTF8AndSize` and :c:func:`PyUnicode_AsUTF8` 1676is now of type ``const char *`` rather of ``char *``. (Contributed by Serhiy 1677Storchaka in :issue:`28769`.) 1678 1679The result of :c:func:`PyMapping_Keys`, :c:func:`PyMapping_Values` and 1680:c:func:`PyMapping_Items` is now always a list, rather than a list or a 1681tuple. (Contributed by Oren Milman in :issue:`28280`.) 1682 1683Added functions :c:func:`PySlice_Unpack` and :c:func:`PySlice_AdjustIndices`. 1684(Contributed by Serhiy Storchaka in :issue:`27867`.) 1685 1686:c:func:`PyOS_AfterFork` is deprecated in favour of the new functions 1687:c:func:`PyOS_BeforeFork`, :c:func:`PyOS_AfterFork_Parent` and 1688:c:func:`PyOS_AfterFork_Child`. (Contributed by Antoine Pitrou in 1689:issue:`16500`.) 1690 1691The ``PyExc_RecursionErrorInst`` singleton that was part of the public API 1692has been removed as its members being never cleared may cause a segfault 1693during finalization of the interpreter. Contributed by Xavier de Gaye in 1694:issue:`22898` and :issue:`30697`. 1695 1696Added C API support for timezones with timezone constructors 1697:c:func:`PyTimeZone_FromOffset` and :c:func:`PyTimeZone_FromOffsetAndName`, 1698and access to the UTC singleton with :c:data:`PyDateTime_TimeZone_UTC`. 1699Contributed by Paul Ganssle in :issue:`10381`. 1700 1701The type of results of :c:func:`PyThread_start_new_thread` and 1702:c:func:`PyThread_get_thread_ident`, and the *id* parameter of 1703:c:func:`PyThreadState_SetAsyncExc` changed from :c:type:`long` to 1704:c:type:`unsigned long`. 1705(Contributed by Serhiy Storchaka in :issue:`6532`.) 1706 1707:c:func:`PyUnicode_AsWideCharString` now raises a :exc:`ValueError` if the 1708second argument is *NULL* and the :c:type:`wchar_t*` string contains null 1709characters. (Contributed by Serhiy Storchaka in :issue:`30708`.) 1710 1711Changes to the startup sequence and the management of dynamic memory 1712allocators mean that the long documented requirement to call 1713:c:func:`Py_Initialize` before calling most C API functions is now 1714relied on more heavily, and failing to abide by it may lead to segfaults in 1715embedding applications. See the :ref:`porting-to-python-37` section in this 1716document and the :ref:`pre-init-safe` section in the C API documentation 1717for more details. 1718 1719The new :c:func:`PyInterpreterState_GetID` returns the unique ID for a 1720given interpreter. 1721(Contributed by Eric Snow in :issue:`29102`.) 1722 1723:c:func:`Py_DecodeLocale`, :c:func:`Py_EncodeLocale` now use the UTF-8 1724encoding when the :ref:`UTF-8 mode <whatsnew37-pep540>` is enabled. 1725(Contributed by Victor Stinner in :issue:`29240`.) 1726 1727:c:func:`PyUnicode_DecodeLocaleAndSize` and :c:func:`PyUnicode_EncodeLocale` 1728now use the current locale encoding for ``surrogateescape`` error handler. 1729(Contributed by Victor Stinner in :issue:`29240`.) 1730 1731The *start* and *end* parameters of :c:func:`PyUnicode_FindChar` are 1732now adjusted to behave like string slices. 1733(Contributed by Xiang Zhang in :issue:`28822`.) 1734 1735 1736Build Changes 1737============= 1738 1739Support for building ``--without-threads`` has been removed. The 1740:mod:`threading` module is now always available. 1741(Contributed by Antoine Pitrou in :issue:`31370`.). 1742 1743A full copy of libffi is no longer bundled for use when building the 1744:mod:`_ctypes <ctypes>` module on non-OSX UNIX platforms. An installed copy 1745of libffi is now required when building ``_ctypes`` on such platforms. 1746(Contributed by Zachary Ware in :issue:`27979`.) 1747 1748The Windows build process no longer depends on Subversion to pull in external 1749sources, a Python script is used to download zipfiles from GitHub instead. 1750If Python 3.6 is not found on the system (via ``py -3.6``), NuGet is used to 1751download a copy of 32-bit Python for this purpose. (Contributed by Zachary 1752Ware in :issue:`30450`.) 1753 1754The :mod:`ssl` module requires OpenSSL 1.0.2 or 1.1 compatible libssl. 1755OpenSSL 1.0.1 has reached end of lifetime on 2016-12-31 and is no longer 1756supported. LibreSSL is temporarily not supported as well. LibreSSL releases 1757up to version 2.6.4 are missing required OpenSSL 1.0.2 APIs. 1758 1759 1760.. _whatsnew37-perf: 1761 1762Optimizations 1763============= 1764 1765The overhead of calling many methods of various standard library classes 1766implemented in C has been significantly reduced by porting more code 1767to use the ``METH_FASTCALL`` convention. 1768(Contributed by Victor Stinner in :issue:`29300`, :issue:`29507`, 1769:issue:`29452`, and :issue:`29286`.) 1770 1771Various optimizations have reduced Python startup time by 10% on Linux and 1772up to 30% on macOS. 1773(Contributed by Victor Stinner, INADA Naoki in :issue:`29585`, and 1774Ivan Levkivskyi in :issue:`31333`.) 1775 1776Method calls are now up to 20% faster due to the bytecode changes which 1777avoid creating bound method instances. 1778(Contributed by Yury Selivanov and INADA Naoki in :issue:`26110`.) 1779 1780.. _whatsnew37-asyncio-perf: 1781 1782The :mod:`asyncio` module received a number of notable optimizations for 1783commonly used functions: 1784 1785* The :func:`asyncio.get_event_loop` function has been reimplemented in C to 1786 make it up to 15 times faster. 1787 (Contributed by Yury Selivanov in :issue:`32296`.) 1788 1789* :class:`asyncio.Future` callback management has been optimized. 1790 (Contributed by Yury Selivanov in :issue:`32348`.) 1791 1792* :func:`asyncio.gather` is now up to 15% faster. 1793 (Contributed by Yury Selivanov in :issue:`32355`.) 1794 1795* :func:`asyncio.sleep` is now up to 2 times faster when the *delay* 1796 argument is zero or negative. 1797 (Contributed by Andrew Svetlov in :issue:`32351`.) 1798 1799* The performance overhead of asyncio debug mode has been reduced. 1800 (Contributed by Antoine Pitrou in :issue:`31970`.) 1801 1802As a result of :ref:`PEP 560 work <whatsnew37-pep560>`, the import time 1803of :mod:`typing` has been reduced by a factor of 7, and many typing operations 1804are now faster. 1805(Contributed by Ivan Levkivskyi in :issue:`32226`.) 1806 1807:func:`sorted` and :meth:`list.sort` have been optimized for common cases 1808to be up to 40-75% faster. 1809(Contributed by Elliot Gorokhovsky in :issue:`28685`.) 1810 1811:meth:`dict.copy` is now up to 5.5 times faster. 1812(Contributed by Yury Selivanov in :issue:`31179`.) 1813 1814:func:`hasattr` and :func:`getattr` are now about 4 times faster when 1815*name* is not found and *obj* does not override :meth:`object.__getattr__` 1816or :meth:`object.__getattribute__`. 1817(Contributed by INADA Naoki in :issue:`32544`.) 1818 1819Searching for certain Unicode characters (like Ukrainian capital "Є") 1820in a string was up to 25 times slower than searching for other characters. 1821It is now only 3 times slower in the worst case. 1822(Contributed by Serhiy Storchaka in :issue:`24821`.) 1823 1824The :func:`collections.namedtuple` factory has been reimplemented to 1825make the creation of named tuples 4 to 6 times faster. 1826(Contributed by Jelle Zijlstra with further improvements by INADA Naoki, 1827Serhiy Storchaka, and Raymond Hettinger in :issue:`28638`.) 1828 1829:meth:`date.fromordinal` and :meth:`date.fromtimestamp` are now up to 183030% faster in the common case. 1831(Contributed by Paul Ganssle in :issue:`32403`.) 1832 1833The :func:`os.fwalk` function is now up to 2 times faster thanks to 1834the use of :func:`os.scandir`. 1835(Contributed by Serhiy Storchaka in :issue:`25996`.) 1836 1837The speed of the :func:`shutil.rmtree` function has been improved by 183820--40% thanks to the use of the :func:`os.scandir` function. 1839(Contributed by Serhiy Storchaka in :issue:`28564`.) 1840 1841Optimized case-insensitive matching and searching of :mod:`regular 1842expressions <re>`. Searching some patterns can now be up to 20 times faster. 1843(Contributed by Serhiy Storchaka in :issue:`30285`.) 1844 1845:func:`re.compile` now converts ``flags`` parameter to int object if 1846it is ``RegexFlag``. It is now as fast as Python 3.5, and faster than 1847Python 3.6 by about 10% depending on the pattern. 1848(Contributed by INADA Naoki in :issue:`31671`.) 1849 1850The :meth:`~selectors.BaseSelector.modify` methods of classes 1851:class:`selectors.EpollSelector`, :class:`selectors.PollSelector` 1852and :class:`selectors.DevpollSelector` may be around 10% faster under 1853heavy loads. (Contributed by Giampaolo Rodola' in :issue:`30014`) 1854 1855Constant folding has been moved from the peephole optimizer to the new AST 1856optimizer, which is able perform optimizations more consistently. 1857(Contributed by Eugene Toder and INADA Naoki in :issue:`29469` and 1858:issue:`11549`.) 1859 1860Most functions and methods in :mod:`abc` have been rewritten in C. 1861This makes creation of abstract base classes, and calling :func:`isinstance` 1862and :func:`issubclass` on them 1.5x faster. This also reduces Python 1863start-up time by up to 10%. (Contributed by Ivan Levkivskyi and INADA Naoki 1864in :issue:`31333`) 1865 1866Significant speed improvements to alternate constructors for 1867:class:`datetime.date` and :class:`datetime.datetime` by using fast-path 1868constructors when not constructing subclasses. (Contributed by Paul Ganssle 1869in :issue:`32403`) 1870 1871The speed of comparison of :class:`array.array` instances has been 1872improved considerably in certain cases. It is now from 10x to 70x faster 1873when comparing arrays holding values of the same integer type. 1874(Contributed by Adrian Wielgosik in :issue:`24700`.) 1875 1876The :func:`math.erf` and :func:`math.erfc` functions now use the (faster) 1877C library implementation on most platforms. 1878(Contributed by Serhiy Storchaka in :issue:`26121`.) 1879 1880 1881Other CPython Implementation Changes 1882==================================== 1883 1884* Trace hooks may now opt out of receiving the ``line`` and opt into receiving 1885 the ``opcode`` events from the interpreter by setting the corresponding new 1886 ``f_trace_lines`` and ``f_trace_opcodes`` attributes on the 1887 frame being traced. (Contributed by Nick Coghlan in :issue:`31344`.) 1888 1889* Fixed some consistency problems with namespace package module attributes. 1890 Namespace module objects now have an ``__file__`` that is set to ``None`` 1891 (previously unset), and their ``__spec__.origin`` is also set to ``None`` 1892 (previously the string ``"namespace"``). See :issue:`32305`. Also, the 1893 namespace module object's ``__spec__.loader`` is set to the same value as 1894 ``__loader__`` (previously, the former was set to ``None``). See 1895 :issue:`32303`. 1896 1897* The :func:`locals` dictionary now displays in the lexical order that 1898 variables were defined. Previously, the order was undefined. 1899 (Contributed by Raymond Hettinger in :issue:`32690`.) 1900 1901* The :mod:`distutils` ``upload`` command no longer tries to change CR 1902 end-of-line characters to CRLF. This fixes a corruption issue with sdists 1903 that ended with a byte equivalent to CR. 1904 (Contributed by Bo Bayles in :issue:`32304`.) 1905 1906 1907Deprecated Python Behavior 1908========================== 1909 1910Yield expressions (both ``yield`` and ``yield from`` clauses) are now deprecated 1911in comprehensions and generator expressions (aside from the iterable expression 1912in the leftmost :keyword:`!for` clause). This ensures that comprehensions 1913always immediately return a container of the appropriate type (rather than 1914potentially returning a :term:`generator iterator` object), while generator 1915expressions won't attempt to interleave their implicit output with the output 1916from any explicit yield expressions. In Python 3.7, such expressions emit 1917:exc:`DeprecationWarning` when compiled, in Python 3.8 this will be a 1918:exc:`SyntaxError`. 1919(Contributed by Serhiy Storchaka in :issue:`10544`.) 1920 1921Returning a subclass of :class:`complex` from :meth:`object.__complex__` is 1922deprecated and will be an error in future Python versions. This makes 1923``__complex__()`` consistent with :meth:`object.__int__` and 1924:meth:`object.__float__`. 1925(Contributed by Serhiy Storchaka in :issue:`28894`.) 1926 1927 1928 1929Deprecated Python modules, functions and methods 1930================================================ 1931 1932aifc 1933---- 1934 1935:func:`aifc.openfp` has been deprecated and will be removed in Python 3.9. 1936Use :func:`aifc.open` instead. 1937(Contributed by Brian Curtin in :issue:`31985`.) 1938 1939 1940.. _whatsnew37-asyncio-deprecated: 1941 1942asyncio 1943------- 1944 1945Support for directly ``await``-ing instances of :class:`asyncio.Lock` and 1946other asyncio synchronization primitives has been deprecated. An 1947asynchronous context manager must be used in order to acquire and release 1948the synchronization resource. 1949(Contributed by Andrew Svetlov in :issue:`32253`.) 1950 1951The :meth:`asyncio.Task.current_task` and :meth:`asyncio.Task.all_tasks` 1952methods have been deprecated. 1953(Contributed by Andrew Svetlov in :issue:`32250`.) 1954 1955 1956collections 1957----------- 1958 1959In Python 3.8, the abstract base classes in :mod:`collections.abc` will no 1960longer be exposed in the regular :mod:`collections` module. This will help 1961create a clearer distinction between the concrete classes and the abstract 1962base classes. 1963(Contributed by Serhiy Storchaka in :issue:`25988`.) 1964 1965 1966dbm 1967--- 1968 1969:mod:`dbm.dumb` now supports reading read-only files and no longer writes the 1970index file when it is not changed. A deprecation warning is now emitted 1971if the index file is missing and recreated in the ``'r'`` and ``'w'`` 1972modes (this will be an error in future Python releases). 1973(Contributed by Serhiy Storchaka in :issue:`28847`.) 1974 1975 1976enum 1977---- 1978 1979In Python 3.8, attempting to check for non-Enum objects in :class:`Enum` 1980classes will raise a :exc:`TypeError` (e.g. ``1 in Color``); similarly, 1981attempting to check for non-Flag objects in a :class:`Flag` member will 1982raise :exc:`TypeError` (e.g. ``1 in Perm.RW``); currently, both operations 1983return :const:`False` instead. 1984(Contributed by Ethan Furman in :issue:`33217`.) 1985 1986 1987gettext 1988------- 1989 1990Using non-integer value for selecting a plural form in :mod:`gettext` is 1991now deprecated. It never correctly worked. (Contributed by Serhiy Storchaka 1992in :issue:`28692`.) 1993 1994 1995importlib 1996--------- 1997 1998Methods 1999:meth:`MetaPathFinder.find_module() <importlib.abc.MetaPathFinder.find_module>` 2000(replaced by 2001:meth:`MetaPathFinder.find_spec() <importlib.abc.MetaPathFinder.find_spec>`) 2002and 2003:meth:`PathEntryFinder.find_loader() <importlib.abc.PathEntryFinder.find_loader>` 2004(replaced by 2005:meth:`PathEntryFinder.find_spec() <importlib.abc.PathEntryFinder.find_spec>`) 2006both deprecated in Python 3.4 now emit :exc:`DeprecationWarning`. 2007(Contributed by Matthias Bussonnier in :issue:`29576`) 2008 2009The :class:`importlib.abc.ResourceLoader` ABC has been deprecated in 2010favour of :class:`importlib.abc.ResourceReader`. 2011 2012 2013locale 2014------ 2015 2016:func:`locale.format` has been deprecated, use :meth:`locale.format_string` 2017instead. (Contributed by Garvit in :issue:`10379`.) 2018 2019 2020macpath 2021------- 2022 2023The :mod:`macpath` is now deprecated and will be removed in Python 3.8. 2024(Contributed by Chi Hsuan Yen in :issue:`9850`.) 2025 2026 2027threading 2028--------- 2029 2030:mod:`dummy_threading` and :mod:`_dummy_thread` have been deprecated. It is 2031no longer possible to build Python with threading disabled. 2032Use :mod:`threading` instead. 2033(Contributed by Antoine Pitrou in :issue:`31370`.) 2034 2035 2036socket 2037------ 2038 2039The silent argument value truncation in :func:`socket.htons` and 2040:func:`socket.ntohs` has been deprecated. In future versions of Python, 2041if the passed argument is larger than 16 bits, an exception will be raised. 2042(Contributed by Oren Milman in :issue:`28332`.) 2043 2044 2045ssl 2046--- 2047 2048:func:`ssl.wrap_socket` is deprecated. Use 2049:meth:`ssl.SSLContext.wrap_socket` instead. 2050(Contributed by Christian Heimes in :issue:`28124`.) 2051 2052 2053sunau 2054----- 2055 2056:func:`sunau.openfp` has been deprecated and will be removed in Python 3.9. 2057Use :func:`sunau.open` instead. 2058(Contributed by Brian Curtin in :issue:`31985`.) 2059 2060 2061sys 2062--- 2063 2064Deprecated :func:`sys.set_coroutine_wrapper` and 2065:func:`sys.get_coroutine_wrapper`. 2066 2067The undocumented ``sys.callstats()`` function has been deprecated and 2068will be removed in a future Python version. 2069(Contributed by Victor Stinner in :issue:`28799`.) 2070 2071 2072wave 2073---- 2074 2075:func:`wave.openfp` has been deprecated and will be removed in Python 3.9. 2076Use :func:`wave.open` instead. 2077(Contributed by Brian Curtin in :issue:`31985`.) 2078 2079 2080Deprecated functions and types of the C API 2081=========================================== 2082 2083Function :c:func:`PySlice_GetIndicesEx` is deprecated and replaced with 2084a macro if ``Py_LIMITED_API`` is not set or set to a value in the range 2085between ``0x03050400`` and ``0x03060000`` (not inclusive), or is ``0x03060100`` 2086or higher. (Contributed by Serhiy Storchaka in :issue:`27867`.) 2087 2088:c:func:`PyOS_AfterFork` has been deprecated. Use :c:func:`PyOS_BeforeFork`, 2089:c:func:`PyOS_AfterFork_Parent` or :c:func:`PyOS_AfterFork_Child()` instead. 2090(Contributed by Antoine Pitrou in :issue:`16500`.) 2091 2092 2093.. _37-platform-support-removals: 2094 2095Platform Support Removals 2096========================= 2097 2098* FreeBSD 9 and older are no longer officially supported. 2099* For full Unicode support, including within extension modules, \*nix platforms 2100 are now expected to provide at least one of ``C.UTF-8`` (full locale), 2101 ``C.utf8`` (full locale) or ``UTF-8`` (``LC_CTYPE``-only locale) as an 2102 alternative to the legacy ``ASCII``-based ``C`` locale. 2103* OpenSSL 0.9.8 and 1.0.1 are no longer supported, which means building CPython 2104 3.7 with SSL/TLS support on older platforms still using these versions 2105 requires custom build options that link to a more recent version of OpenSSL. 2106 2107 Notably, this issue affects the Debian 8 (aka "jessie") and Ubuntu 14.04 2108 (aka "Trusty") LTS Linux distributions, as they still use OpenSSL 1.0.1 by 2109 default. 2110 2111 Debian 9 ("stretch") and Ubuntu 16.04 ("xenial"), as well as recent releases 2112 of other LTS Linux releases (e.g. RHEL/CentOS 7.5, SLES 12-SP3), use OpenSSL 2113 1.0.2 or later, and remain supported in the default build configuration. 2114 2115 CPython's own :source:`CI configuration file <.travis.yml>` provides an 2116 example of using the SSL 2117 :source:`compatibility testing infrastructure <Tools/ssl/multissltests.py>` in 2118 CPython's test suite to build and link against OpenSSL 1.1.0 rather than an 2119 outdated system provided OpenSSL. 2120 2121 2122API and Feature Removals 2123======================== 2124 2125The following features and APIs have been removed from Python 3.7: 2126 2127* The ``os.stat_float_times()`` function has been removed. It was introduced in 2128 Python 2.3 for backward compatibility with Python 2.2, and was deprecated 2129 since Python 3.1. 2130 2131* Unknown escapes consisting of ``'\'`` and an ASCII letter in replacement 2132 templates for :func:`re.sub` were deprecated in Python 3.5, and will now 2133 cause an error. 2134 2135* Removed support of the *exclude* argument in :meth:`tarfile.TarFile.add`. 2136 It was deprecated in Python 2.7 and 3.2. Use the *filter* argument instead. 2137 2138* The ``splitunc()`` function in the :mod:`ntpath` module was deprecated in 2139 Python 3.1, and has now been removed. Use the :func:`~os.path.splitdrive` 2140 function instead. 2141 2142* :func:`collections.namedtuple` no longer supports the *verbose* parameter 2143 or ``_source`` attribute which showed the generated source code for the 2144 named tuple class. This was part of an optimization designed to speed-up 2145 class creation. (Contributed by Jelle Zijlstra with further improvements 2146 by INADA Naoki, Serhiy Storchaka, and Raymond Hettinger in :issue:`28638`.) 2147 2148* Functions :func:`bool`, :func:`float`, :func:`list` and :func:`tuple` no 2149 longer take keyword arguments. The first argument of :func:`int` can now 2150 be passed only as positional argument. 2151 2152* Removed previously deprecated in Python 2.4 classes ``Plist``, ``Dict`` and 2153 ``_InternalDict`` in the :mod:`plistlib` module. Dict values in the result 2154 of functions :func:`~plistlib.readPlist` and 2155 :func:`~plistlib.readPlistFromBytes` are now normal dicts. You no longer 2156 can use attribute access to access items of these dictionaries. 2157 2158* The ``asyncio.windows_utils.socketpair()`` function has been 2159 removed. Use the :func:`socket.socketpair` function instead, 2160 it is available on all platforms since Python 3.5. 2161 ``asyncio.windows_utils.socketpair`` was just an alias to 2162 ``socket.socketpair`` on Python 3.5 and newer. 2163 2164* :mod:`asyncio` no longer exports the :mod:`selectors` and 2165 :mod:`_overlapped` modules as ``asyncio.selectors`` and 2166 ``asyncio._overlapped``. Replace ``from asyncio import selectors`` with 2167 ``import selectors``. 2168 2169* Direct instantiation of :class:`ssl.SSLSocket` and :class:`ssl.SSLObject` 2170 objects is now prohibited. The constructors were never documented, tested, 2171 or designed as public constructors. Users were supposed to use 2172 :func:`ssl.wrap_socket` or :class:`ssl.SSLContext`. 2173 (Contributed by Christian Heimes in :issue:`32951`.) 2174 2175* The unused :mod:`distutils` ``install_misc`` command has been removed. 2176 (Contributed by Eric N. Vander Weele in :issue:`29218`.) 2177 2178 2179Module Removals 2180=============== 2181 2182The ``fpectl`` module has been removed. It was never enabled by 2183default, never worked correctly on x86-64, and it changed the Python 2184ABI in ways that caused unexpected breakage of C extensions. 2185(Contributed by Nathaniel J. Smith in :issue:`29137`.) 2186 2187 2188Windows-only Changes 2189==================== 2190 2191The python launcher, (py.exe), can accept 32 & 64 bit specifiers **without** 2192having to specify a minor version as well. So ``py -3-32`` and ``py -3-64`` 2193become valid as well as ``py -3.7-32``, also the -*m*-64 and -*m.n*-64 forms 2194are now accepted to force 64 bit python even if 32 bit would have otherwise 2195been used. If the specified version is not available py.exe will error exit. 2196(Contributed by Steve Barnes in :issue:`30291`.) 2197 2198The launcher can be run as ``py -0`` to produce a list of the installed pythons, 2199*with default marked with an asterisk*. Running ``py -0p`` will include the paths. 2200If py is run with a version specifier that cannot be matched it will also print 2201the *short form* list of available specifiers. 2202(Contributed by Steve Barnes in :issue:`30362`.) 2203 2204 2205.. _porting-to-python-37: 2206 2207Porting to Python 3.7 2208===================== 2209 2210This section lists previously described changes and other bugfixes 2211that may require changes to your code. 2212 2213 2214Changes in Python Behavior 2215-------------------------- 2216 2217* :keyword:`async` and :keyword:`await` names are now reserved keywords. 2218 Code using these names as identifiers will now raise a :exc:`SyntaxError`. 2219 (Contributed by Jelle Zijlstra in :issue:`30406`.) 2220 2221* :pep:`479` is enabled for all code in Python 3.7, meaning that 2222 :exc:`StopIteration` exceptions raised directly or indirectly in 2223 coroutines and generators are transformed into :exc:`RuntimeError` 2224 exceptions. 2225 (Contributed by Yury Selivanov in :issue:`32670`.) 2226 2227* :meth:`object.__aiter__` methods can no longer be declared as 2228 asynchronous. (Contributed by Yury Selivanov in :issue:`31709`.) 2229 2230* Due to an oversight, earlier Python versions erroneously accepted the 2231 following syntax:: 2232 2233 f(1 for x in [1],) 2234 2235 class C(1 for x in [1]): 2236 pass 2237 2238 Python 3.7 now correctly raises a :exc:`SyntaxError`, as a generator 2239 expression always needs to be directly inside a set of parentheses 2240 and cannot have a comma on either side, and the duplication of the 2241 parentheses can be omitted only on calls. 2242 (Contributed by Serhiy Storchaka in :issue:`32012` and :issue:`32023`.) 2243 2244* When using the :option:`-m` switch, the initial working directory is now added 2245 to :data:`sys.path`, rather than an empty string (which dynamically denoted 2246 the current working directory at the time of each import). Any programs that 2247 are checking for the empty string, or otherwise relying on the previous 2248 behaviour, will need to be updated accordingly (e.g. by also checking for 2249 ``os.getcwd()`` or ``os.path.dirname(__main__.__file__)``, depending on why 2250 the code was checking for the empty string in the first place). 2251 2252 2253Changes in the Python API 2254------------------------- 2255 2256* :meth:`socketserver.ThreadingMixIn.server_close` now waits until all 2257 non-daemon threads complete. Set the new 2258 :attr:`socketserver.ThreadingMixIn.block_on_close` class attribute to 2259 ``False`` to get the pre-3.7 behaviour. 2260 (Contributed by Victor Stinner in :issue:`31233` and :issue:`33540`.) 2261 2262* :meth:`socketserver.ForkingMixIn.server_close` now waits until all 2263 child processes complete. Set the new 2264 :attr:`socketserver.ForkingMixIn.block_on_close` class attribute to ``False`` 2265 to get the pre-3.7 behaviour. 2266 (Contributed by Victor Stinner in :issue:`31151` and :issue:`33540`.) 2267 2268* The :func:`locale.localeconv` function now temporarily sets the ``LC_CTYPE`` 2269 locale to the value of ``LC_NUMERIC`` in some cases. 2270 (Contributed by Victor Stinner in :issue:`31900`.) 2271 2272* :meth:`pkgutil.walk_packages` now raises a :exc:`ValueError` if *path* is 2273 a string. Previously an empty list was returned. 2274 (Contributed by Sanyam Khurana in :issue:`24744`.) 2275 2276* A format string argument for :meth:`string.Formatter.format` 2277 is now :ref:`positional-only <positional-only_parameter>`. 2278 Passing it as a keyword argument was deprecated in Python 3.5. (Contributed 2279 by Serhiy Storchaka in :issue:`29193`.) 2280 2281* Attributes :attr:`~http.cookies.Morsel.key`, 2282 :attr:`~http.cookies.Morsel.value` and 2283 :attr:`~http.cookies.Morsel.coded_value` of class 2284 :class:`http.cookies.Morsel` are now read-only. 2285 Assigning to them was deprecated in Python 3.5. 2286 Use the :meth:`~http.cookies.Morsel.set` method for setting them. 2287 (Contributed by Serhiy Storchaka in :issue:`29192`.) 2288 2289* The *mode* argument of :func:`os.makedirs` no longer affects the file 2290 permission bits of newly-created intermediate-level directories. 2291 To set their file permission bits you can set the umask before invoking 2292 ``makedirs()``. 2293 (Contributed by Serhiy Storchaka in :issue:`19930`.) 2294 2295* The :attr:`struct.Struct.format` type is now :class:`str` instead of 2296 :class:`bytes`. (Contributed by Victor Stinner in :issue:`21071`.) 2297 2298* :func:`~cgi.parse_multipart` now accepts the *encoding* and *errors* 2299 arguments and returns the same results as 2300 :class:`~FieldStorage`: for non-file fields, the value associated to a key 2301 is a list of strings, not bytes. 2302 (Contributed by Pierre Quentel in :issue:`29979`.) 2303 2304* Due to internal changes in :mod:`socket`, calling :func:`socket.fromshare` 2305 on a socket created by :func:`socket.share <socket.socket.share>` in older 2306 Python versions is not supported. 2307 2308* ``repr`` for :exc:`BaseException` has changed to not include the trailing 2309 comma. Most exceptions are affected by this change. 2310 (Contributed by Serhiy Storchaka in :issue:`30399`.) 2311 2312* ``repr`` for :class:`datetime.timedelta` has changed to include the keyword 2313 arguments in the output. (Contributed by Utkarsh Upadhyay in :issue:`30302`.) 2314 2315* Because :func:`shutil.rmtree` is now implemented using the :func:`os.scandir` 2316 function, the user specified handler *onerror* is now called with the first 2317 argument ``os.scandir`` instead of ``os.listdir`` when listing the directory 2318 is failed. 2319 2320* Support for nested sets and set operations in regular expressions as in 2321 `Unicode Technical Standard #18`_ might be added in the future. This would 2322 change the syntax. To facilitate this future change a :exc:`FutureWarning` 2323 will be raised in ambiguous cases for the time being. 2324 That include sets starting with a literal ``'['`` or containing literal 2325 character sequences ``'--'``, ``'&&'``, ``'~~'``, and ``'||'``. To 2326 avoid a warning, escape them with a backslash. 2327 (Contributed by Serhiy Storchaka in :issue:`30349`.) 2328 2329 .. _Unicode Technical Standard #18: https://unicode.org/reports/tr18/ 2330 2331* The result of splitting a string on a :mod:`regular expression <re>` 2332 that could match an empty string has been changed. For example 2333 splitting on ``r'\s*'`` will now split not only on whitespaces as it 2334 did previously, but also on empty strings before all non-whitespace 2335 characters and just before the end of the string. 2336 The previous behavior can be restored by changing the pattern 2337 to ``r'\s+'``. A :exc:`FutureWarning` was emitted for such patterns since 2338 Python 3.5. 2339 2340 For patterns that match both empty and non-empty strings, the result of 2341 searching for all matches may also be changed in other cases. For example 2342 in the string ``'a\n\n'``, the pattern ``r'(?m)^\s*?$'`` will not only 2343 match empty strings at positions 2 and 3, but also the string ``'\n'`` at 2344 positions 2--3. To match only blank lines, the pattern should be rewritten 2345 as ``r'(?m)^[^\S\n]*$'``. 2346 2347 :func:`re.sub()` now replaces empty matches adjacent to a previous 2348 non-empty match. For example ``re.sub('x*', '-', 'abxd')`` returns now 2349 ``'-a-b--d-'`` instead of ``'-a-b-d-'`` (the first minus between 'b' and 2350 'd' replaces 'x', and the second minus replaces an empty string between 2351 'x' and 'd'). 2352 2353 (Contributed by Serhiy Storchaka in :issue:`25054` and :issue:`32308`.) 2354 2355* Change :func:`re.escape` to only escape regex special characters instead 2356 of escaping all characters other than ASCII letters, numbers, and ``'_'``. 2357 (Contributed by Serhiy Storchaka in :issue:`29995`.) 2358 2359* :class:`tracemalloc.Traceback` frames are now sorted from oldest to most 2360 recent to be more consistent with :mod:`traceback`. 2361 (Contributed by Jesse Bakker in :issue:`32121`.) 2362 2363* On OSes that support :const:`socket.SOCK_NONBLOCK` or 2364 :const:`socket.SOCK_CLOEXEC` bit flags, the 2365 :attr:`socket.type <socket.socket.type>` no longer has them applied. 2366 Therefore, checks like ``if sock.type == socket.SOCK_STREAM`` 2367 work as expected on all platforms. 2368 (Contributed by Yury Selivanov in :issue:`32331`.) 2369 2370* On Windows the default for the *close_fds* argument of 2371 :class:`subprocess.Popen` was changed from :const:`False` to :const:`True` 2372 when redirecting the standard handles. If you previously depended on handles 2373 being inherited when using :class:`subprocess.Popen` with standard io 2374 redirection, you will have to pass ``close_fds=False`` to preserve the 2375 previous behaviour, or use 2376 :attr:`STARTUPINFO.lpAttributeList <subprocess.STARTUPINFO.lpAttributeList>`. 2377 2378* :meth:`importlib.machinery.PathFinder.invalidate_caches` -- which implicitly 2379 affects :func:`importlib.invalidate_caches` -- now deletes entries 2380 in :data:`sys.path_importer_cache` which are set to ``None``. 2381 (Contributed by Brett Cannon in :issue:`33169`.) 2382 2383* In :mod:`asyncio`, 2384 :meth:`loop.sock_recv() <asyncio.loop.sock_recv>`, 2385 :meth:`loop.sock_sendall() <asyncio.loop.sock_sendall>`, 2386 :meth:`loop.sock_accept() <asyncio.loop.sock_accept>`, 2387 :meth:`loop.getaddrinfo() <asyncio.loop.getaddrinfo>`, 2388 :meth:`loop.getnameinfo() <asyncio.loop.getnameinfo>` 2389 have been changed to be proper coroutine methods to match their 2390 documentation. Previously, these methods returned :class:`asyncio.Future` 2391 instances. 2392 (Contributed by Yury Selivanov in :issue:`32327`.) 2393 2394* :attr:`asyncio.Server.sockets` now returns a copy of the internal list 2395 of server sockets, instead of returning it directly. 2396 (Contributed by Yury Selivanov in :issue:`32662`.) 2397 2398* :attr:`Struct.format <struct.Struct.format>` is now a :class:`str` instance 2399 instead of a :class:`bytes` instance. 2400 (Contributed by Victor Stinner in :issue:`21071`.) 2401 2402* :meth:`ast.literal_eval()` is now stricter. Addition and subtraction of 2403 arbitrary numbers are no longer allowed. 2404 (Contributed by Serhiy Storchaka in :issue:`31778`.) 2405 2406* :meth:`Calendar.itermonthdates <calendar.Calendar.itermonthdates>` 2407 will now consistently raise an exception when a date falls outside of the 2408 ``0001-01-01`` through ``9999-12-31`` range. To support applications that 2409 cannot tolerate such exceptions, the new 2410 :meth:`Calendar.itermonthdays3 <calendar.Calendar.itermonthdays3>` and 2411 :meth:`Calendar.itermonthdays4 <calendar.Calendar.itermonthdays4>` can be used. 2412 The new methods return tuples and are not restricted by the range supported by 2413 :class:`datetime.date`. 2414 (Contributed by Alexander Belopolsky in :issue:`28292`.) 2415 2416* :class:`collections.ChainMap` now preserves the order of the underlying 2417 mappings. (Contributed by Raymond Hettinger in :issue:`32792`.) 2418 2419* The ``submit()`` method of :class:`concurrent.futures.ThreadPoolExecutor` 2420 and :class:`concurrent.futures.ProcessPoolExecutor` now raises 2421 a :exc:`RuntimeError` if called during interpreter shutdown. 2422 (Contributed by Mark Nemec in :issue:`33097`.) 2423 2424* The :class:`configparser.ConfigParser` constructor now uses ``read_dict()`` 2425 to process the default values, making its behavior consistent with the 2426 rest of the parser. Non-string keys and values in the defaults 2427 dictionary are now being implicitly converted to strings. 2428 (Contributed by James Tocknell in :issue:`23835`.) 2429 2430* Several undocumented internal imports were removed. 2431 One example is that ``os.errno`` is no longer available; use ``import errno`` 2432 directly instead. 2433 Note that such undocumented internal imports may be removed any time without 2434 notice, even in micro version releases. 2435 2436 2437Changes in the C API 2438-------------------- 2439 2440The function :c:func:`PySlice_GetIndicesEx` is considered unsafe for 2441resizable sequences. If the slice indices are not instances of :class:`int`, 2442but objects that implement the :meth:`!__index__` method, the sequence can be 2443resized after passing its length to :c:func:`!PySlice_GetIndicesEx`. This 2444can lead to returning indices out of the length of the sequence. For 2445avoiding possible problems use new functions :c:func:`PySlice_Unpack` and 2446:c:func:`PySlice_AdjustIndices`. 2447(Contributed by Serhiy Storchaka in :issue:`27867`.) 2448 2449 2450CPython bytecode changes 2451------------------------ 2452 2453There are two new opcodes: :opcode:`LOAD_METHOD` and :opcode:`CALL_METHOD`. 2454(Contributed by Yury Selivanov and INADA Naoki in :issue:`26110`.) 2455 2456The :opcode:`STORE_ANNOTATION` opcode has been removed. 2457(Contributed by Mark Shannon in :issue:`32550`.) 2458 2459 2460Windows-only Changes 2461-------------------- 2462 2463The file used to override :data:`sys.path` is now called 2464``<python-executable>._pth`` instead of ``'sys.path'``. 2465See :ref:`finding_modules` for more information. 2466(Contributed by Steve Dower in :issue:`28137`.) 2467 2468 2469Other CPython implementation changes 2470------------------------------------ 2471 2472In preparation for potential future changes to the public CPython runtime 2473initialization API (see :pep:`432` for an initial, but somewhat outdated, 2474draft), CPython's internal startup 2475and configuration management logic has been significantly refactored. While 2476these updates are intended to be entirely transparent to both embedding 2477applications and users of the regular CPython CLI, they're being mentioned 2478here as the refactoring changes the internal order of various operations 2479during interpreter startup, and hence may uncover previously latent defects, 2480either in embedding applications, or in CPython itself. 2481(Initially contributed by Nick Coghlan and Eric Snow as part of 2482:issue:`22257`, and further updated by Nick, Eric, and Victor Stinner in a 2483number of other issues). Some known details affected: 2484 2485* :c:func:`PySys_AddWarnOptionUnicode` is not currently usable by embedding 2486 applications due to the requirement to create a Unicode object prior to 2487 calling `Py_Initialize`. Use :c:func:`PySys_AddWarnOption` instead. 2488 2489* warnings filters added by an embedding application with 2490 :c:func:`PySys_AddWarnOption` should now more consistently take precedence 2491 over the default filters set by the interpreter 2492 2493Due to changes in the way the default warnings filters are configured, 2494setting :c:data:`Py_BytesWarningFlag` to a value greater than one is no longer 2495sufficient to both emit :exc:`BytesWarning` messages and have them converted 2496to exceptions. Instead, the flag must be set (to cause the warnings to be 2497emitted in the first place), and an explicit ``error::BytesWarning`` 2498warnings filter added to convert them to exceptions. 2499 2500Due to a change in the way docstrings are handled by the compiler, the 2501implicit ``return None`` in a function body consisting solely of a docstring 2502is now marked as occurring on the same line as the docstring, not on the 2503function's header line. 2504 2505The current exception state has been moved from the frame object to the co-routine. 2506This simplified the interpreter and fixed a couple of obscure bugs caused by 2507having swap exception state when entering or exiting a generator. 2508(Contributed by Mark Shannon in :issue:`25612`.) 2509 2510Notable changes in Python 3.7.1 2511=============================== 2512 2513Starting in 3.7.1, :c:func:`Py_Initialize` now consistently reads and respects 2514all of the same environment settings as :c:func:`Py_Main` (in earlier Python 2515versions, it respected an ill-defined subset of those environment variables, 2516while in Python 3.7.0 it didn't read any of them due to :issue:`34247`). If 2517this behavior is unwanted, set :c:data:`Py_IgnoreEnvironmentFlag` to 1 before 2518calling :c:func:`Py_Initialize`. 2519 2520In 3.7.1 the C API for Context Variables 2521:ref:`was updated <contextvarsobjects_pointertype_change>` to use 2522:c:type:`PyObject` pointers. See also :issue:`34762`. 2523 2524:mod:`xml.dom.minidom` and :mod:`xml.sax` modules no longer process 2525external entities by default. See also :issue:`17239`. 2526 2527In 3.7.1 the :mod:`tokenize` module now implicitly emits a ``NEWLINE`` token 2528when provided with input that does not have a trailing new line. This behavior 2529now matches what the C tokenizer does internally. 2530(Contributed by Ammar Askar in :issue:`33899`.) 2531 2532Notable changes in Python 3.7.2 2533=============================== 2534 2535In 3.7.2, :mod:`venv` on Windows no longer copies the original binaries, but 2536creates redirector scripts named ``python.exe`` and ``pythonw.exe`` instead. 2537This resolves a long standing issue where all virtual environments would have 2538to be upgraded or recreated with each Python update. However, note that this 2539release will still require recreation of virtual environments in order to get 2540the new scripts. 2541