1**************************** 2 What's New In Python 3.7 3**************************** 4 5:Editor: Elvis Pranskevichus <elvis@magic.io> 6 7.. Rules for maintenance: 8 9 * Anyone can add text to this document. Do not spend very much time 10 on the wording of your changes, because your text will probably 11 get rewritten to some degree. 12 13 * The maintainer will go through Misc/NEWS periodically and add 14 changes; it's therefore more important to add your changes to 15 Misc/NEWS than to this file. 16 17 * This is not a complete list of every single change; completeness 18 is the purpose of Misc/NEWS. Some changes I consider too small 19 or esoteric to include. If such a change is added to the text, 20 I'll just remove it. (This is another reason you shouldn't spend 21 too much time on writing your addition.) 22 23 * If you want to draw your new text to the attention of the 24 maintainer, add 'XXX' to the beginning of the paragraph or 25 section. 26 27 * It's OK to just add a fragmentary note about a change. For 28 example: "XXX Describe the transmogrify() function added to the 29 socket module." The maintainer will research the change and 30 write the necessary text. 31 32 * You can comment out your additions if you like, but it's not 33 necessary (especially when a final release is some months away). 34 35 * Credit the author of a patch or bugfix. Just the name is 36 sufficient; the e-mail address isn't necessary. 37 38 * It's helpful to add the bug/patch number as a comment: 39 40 XXX Describe the transmogrify() function added to the socket 41 module. 42 (Contributed by P.Y. Developer in :issue:`12345`.) 43 44 This saves the maintainer the effort of going through the Git log 45 when researching a change. 46 47This article explains the new features in Python 3.7, compared to 3.6. 48Python 3.7 was released on June 27, 2018. 49For full details, see the :ref:`changelog <changelog>`. 50 51 52Summary -- Release Highlights 53============================= 54 55.. This section singles out the most important changes in Python 3.7. 56 Brevity is key. 57 58New syntax features: 59 60* :ref:`PEP 563 <whatsnew37-pep563>`, postponed evaluation of type annotations. 61 62Backwards incompatible syntax changes: 63 64* :keyword:`async` and :keyword:`await` are now reserved keywords. 65 66New library modules: 67 68* :mod:`contextvars`: :ref:`PEP 567 -- Context Variables <whatsnew37-pep567>` 69* :mod:`dataclasses`: :ref:`PEP 557 -- Data Classes <whatsnew37-pep557>` 70* :ref:`whatsnew37_importlib_resources` 71 72New built-in features: 73 74* :ref:`PEP 553 <whatsnew37-pep553>`, the new :func:`breakpoint` function. 75 76Python data model improvements: 77 78* :ref:`PEP 562 <whatsnew37-pep562>`, customization of access to 79 module attributes. 80 81* :ref:`PEP 560 <whatsnew37-pep560>`, core support for typing module and 82 generic types. 83 84* the insertion-order preservation nature of :ref:`dict <typesmapping>` 85 objects `has been declared`_ to be an official 86 part of the Python language spec. 87 88.. _has been declared: https://mail.python.org/pipermail/python-dev/2017-December/151283.html 89 90Significant improvements in the standard library: 91 92* The :mod:`asyncio` module has received new features, significant 93 :ref:`usability and performance improvements <whatsnew37_asyncio>`. 94 95* The :mod:`time` module gained support for 96 :ref:`functions with nanosecond resolution <whatsnew37-pep564>`. 97 98CPython implementation improvements: 99 100* Avoiding the use of ASCII as a default text encoding: 101 102 * :ref:`PEP 538 <whatsnew37-pep538>`, legacy C locale coercion 103 * :ref:`PEP 540 <whatsnew37-pep540>`, forced UTF-8 runtime mode 104* :ref:`PEP 552 <whatsnew37-pep552>`, deterministic .pycs 105* :ref:`New Python Development Mode <whatsnew37-devmode>` 106* :ref:`PEP 565 <whatsnew37-pep565>`, improved :exc:`DeprecationWarning` 107 handling 108 109C API improvements: 110 111* :ref:`PEP 539 <whatsnew37-pep539>`, new C API for thread-local storage 112 113Documentation improvements: 114 115* :ref:`PEP 545 <whatsnew37-pep545>`, Python documentation translations 116* New documentation translations: `Japanese <https://docs.python.org/ja/>`_, 117 `French <https://docs.python.org/fr/>`_, and 118 `Korean <https://docs.python.org/ko/>`_. 119 120This release features notable performance improvements in many areas. 121The :ref:`whatsnew37-perf` section lists them in detail. 122 123For a list of changes that may affect compatibility with previous Python 124releases please refer to the :ref:`porting-to-python-37` section. 125 126 127New Features 128============ 129 130.. _whatsnew37-pep563: 131 132PEP 563: Postponed Evaluation of Annotations 133-------------------------------------------- 134 135The advent of type hints in Python uncovered two glaring usability issues 136with the functionality of annotations added in :pep:`3107` and refined 137further in :pep:`526`: 138 139* annotations could only use names which were already available in the 140 current scope, in other words they didn't support forward references 141 of any kind; and 142 143* annotating source code had adverse effects on startup time of Python 144 programs. 145 146Both of these issues are fixed by postponing the evaluation of 147annotations. Instead of compiling code which executes expressions in 148annotations at their definition time, the compiler stores the annotation 149in a string form equivalent to the AST of the expression in question. 150If needed, annotations can be resolved at runtime using 151:func:`typing.get_type_hints`. In the common case where this is not 152required, the annotations are cheaper to store (since short strings 153are interned by the interpreter) and make startup time faster. 154 155Usability-wise, annotations now support forward references, making the 156following syntax valid:: 157 158 class C: 159 @classmethod 160 def from_string(cls, source: str) -> C: 161 ... 162 163 def validate_b(self, obj: B) -> bool: 164 ... 165 166 class B: 167 ... 168 169Since this change breaks compatibility, the new behavior needs to be enabled 170on a per-module basis in Python 3.7 using a :mod:`__future__` import:: 171 172 from __future__ import annotations 173 174It will become the default in Python 3.10. 175 176.. seealso:: 177 178 :pep:`563` -- Postponed evaluation of annotations 179 PEP written and implemented by Łukasz Langa. 180 181 182.. _whatsnew37-pep538: 183 184PEP 538: Legacy C Locale Coercion 185--------------------------------- 186 187An ongoing challenge within the Python 3 series has been determining a sensible 188default strategy for handling the "7-bit ASCII" text encoding assumption 189currently implied by the use of the default C or POSIX locale on non-Windows 190platforms. 191 192:pep:`538` updates the default interpreter command line interface to 193automatically coerce that locale to an available UTF-8 based locale as 194described in the documentation of the new :envvar:`PYTHONCOERCECLOCALE` 195environment variable. Automatically setting ``LC_CTYPE`` this way means that 196both the core interpreter and locale-aware C extensions (such as 197:mod:`readline`) will assume the use of UTF-8 as the default text encoding, 198rather than ASCII. 199 200The platform support definition in :pep:`11` has also been updated to limit 201full text handling support to suitably configured non-ASCII based locales. 202 203As part of this change, the default error handler for :data:`~sys.stdin` and 204:data:`~sys.stdout` is now ``surrogateescape`` (rather than ``strict``) when 205using any of the defined coercion target locales (currently ``C.UTF-8``, 206``C.utf8``, and ``UTF-8``). The default error handler for :data:`~sys.stderr` 207continues to be ``backslashreplace``, regardless of locale. 208 209Locale coercion is silent by default, but to assist in debugging potentially 210locale related integration problems, explicit warnings (emitted directly on 211:data:`~sys.stderr`) can be requested by setting ``PYTHONCOERCECLOCALE=warn``. 212This setting will also cause the Python runtime to emit a warning if the 213legacy C locale remains active when the core interpreter is initialized. 214 215While :pep:`538`'s locale coercion has the benefit of also affecting extension 216modules (such as GNU ``readline``), as well as child processes (including those 217running non-Python applications and older versions of Python), it has the 218downside of requiring that a suitable target locale be present on the running 219system. To better handle the case where no suitable target locale is available 220(as occurs on RHEL/CentOS 7, for example), Python 3.7 also implements 221:ref:`whatsnew37-pep540`. 222 223.. seealso:: 224 225 :pep:`538` -- Coercing the legacy C locale to a UTF-8 based locale 226 PEP written and implemented by Nick Coghlan. 227 228 229.. _whatsnew37-pep540: 230 231PEP 540: Forced UTF-8 Runtime Mode 232----------------------------------- 233 234The new :option:`-X` ``utf8`` command line option and :envvar:`PYTHONUTF8` 235environment variable can be used to enable the CPython *UTF-8 mode*. 236 237When in UTF-8 mode, CPython ignores the locale settings, and uses the 238UTF-8 encoding by default. The error handlers for :data:`sys.stdin` and 239:data:`sys.stdout` streams are set to ``surrogateescape``. 240 241The forced UTF-8 mode can be used to change the text handling behavior in 242an embedded Python interpreter without changing the locale settings of 243an embedding application. 244 245While :pep:`540`'s UTF-8 mode has the benefit of working regardless of which 246locales are available on the running system, it has the downside of having no 247effect on extension modules (such as GNU ``readline``), child processes running 248non-Python applications, and child processes running older versions of Python. 249To reduce the risk of corrupting text data when communicating with such 250components, Python 3.7 also implements :ref:`whatsnew37-pep540`). 251 252The UTF-8 mode is enabled by default when the locale is ``C`` or ``POSIX``, and 253the :pep:`538` locale coercion feature fails to change it to a UTF-8 based 254alternative (whether that failure is due to ``PYTHONCOERCECLOCALE=0`` being set, 255``LC_ALL`` being set, or the lack of a suitable target locale). 256 257.. seealso:: 258 259 :pep:`540` -- Add a new UTF-8 mode 260 PEP written and implemented by Victor Stinner 261 262 263.. _whatsnew37-pep553: 264 265PEP 553: Built-in ``breakpoint()`` 266---------------------------------- 267 268Python 3.7 includes the new built-in :func:`breakpoint` function as 269an easy and consistent way to enter the Python debugger. 270 271Built-in ``breakpoint()`` calls :func:`sys.breakpointhook`. By default, the 272latter imports :mod:`pdb` and then calls ``pdb.set_trace()``, but by binding 273``sys.breakpointhook()`` to the function of your choosing, ``breakpoint()`` can 274enter any debugger. Additionally, the environment variable 275:envvar:`PYTHONBREAKPOINT` can be set to the callable of your debugger of 276choice. Set ``PYTHONBREAKPOINT=0`` to completely disable built-in 277``breakpoint()``. 278 279.. seealso:: 280 281 :pep:`553` -- Built-in breakpoint() 282 PEP written and implemented by Barry Warsaw 283 284 285.. _whatsnew37-pep539: 286 287PEP 539: New C API for Thread-Local Storage 288------------------------------------------- 289 290While Python provides a C API for thread-local storage support; the existing 291:ref:`Thread Local Storage (TLS) API <thread-local-storage-api>` has used 292:c:type:`int` to represent TLS keys across all platforms. This has not 293generally been a problem for officially-support platforms, but that is neither 294POSIX-compliant, nor portable in any practical sense. 295 296:pep:`539` changes this by providing a new :ref:`Thread Specific Storage (TSS) 297API <thread-specific-storage-api>` to CPython which supersedes use of the 298existing TLS API within the CPython interpreter, while deprecating the existing 299API. The TSS API uses a new type :c:type:`Py_tss_t` instead of :c:type:`int` 300to represent TSS keys--an opaque type the definition of which may depend on 301the underlying TLS implementation. Therefore, this will allow to build CPython 302on platforms where the native TLS key is defined in a way that cannot be safely 303cast to :c:type:`int`. 304 305Note that on platforms where the native TLS key is defined in a way that cannot 306be safely cast to :c:type:`int`, all functions of the existing TLS API will be 307no-op and immediately return failure. This indicates clearly that the old API 308is not supported on platforms where it cannot be used reliably, and that no 309effort will be made to add such support. 310 311.. seealso:: 312 313 :pep:`539` -- A New C-API for Thread-Local Storage in CPython 314 PEP written by Erik M. Bray; implementation by Masayuki Yamamoto. 315 316 317.. _whatsnew37-pep562: 318 319PEP 562: Customization of Access to Module Attributes 320----------------------------------------------------- 321 322Python 3.7 allows defining :meth:`__getattr__` on modules and will call 323it whenever a module attribute is otherwise not found. Defining 324:meth:`__dir__` on modules is now also allowed. 325 326A typical example of where this may be useful is module attribute deprecation 327and lazy loading. 328 329.. seealso:: 330 331 :pep:`562` -- Module ``__getattr__`` and ``__dir__`` 332 PEP written and implemented by Ivan Levkivskyi 333 334 335.. _whatsnew37-pep564: 336 337PEP 564: New Time Functions With Nanosecond Resolution 338------------------------------------------------------ 339 340The resolution of clocks in modern systems can exceed the limited precision 341of a floating point number returned by the :func:`time.time` function 342and its variants. To avoid loss of precision, :pep:`564` adds six new 343"nanosecond" variants of the existing timer functions to the :mod:`time` 344module: 345 346* :func:`time.clock_gettime_ns` 347* :func:`time.clock_settime_ns` 348* :func:`time.monotonic_ns` 349* :func:`time.perf_counter_ns` 350* :func:`time.process_time_ns` 351* :func:`time.time_ns` 352 353The new functions return the number of nanoseconds as an integer value. 354 355`Measurements <https://www.python.org/dev/peps/pep-0564/#annex-clocks-resolution-in-python>`_ 356show that on Linux and Windows the resolution of :func:`time.time_ns` is 357approximately 3 times better than that of :func:`time.time`. 358 359.. seealso:: 360 361 :pep:`564` -- Add new time functions with nanosecond resolution 362 PEP written and implemented by Victor Stinner 363 364 365.. _whatsnew37-pep565: 366 367PEP 565: Show DeprecationWarning in ``__main__`` 368------------------------------------------------ 369 370The default handling of :exc:`DeprecationWarning` has been changed such that 371these warnings are once more shown by default, but only when the code 372triggering them is running directly in the :mod:`__main__` module. As a result, 373developers of single file scripts and those using Python interactively should 374once again start seeing deprecation warnings for the APIs they use, but 375deprecation warnings triggered by imported application, library and framework 376modules will continue to be hidden by default. 377 378As a result of this change, the standard library now allows developers to choose 379between three different deprecation warning behaviours: 380 381* :exc:`FutureWarning`: always displayed by default, recommended for warnings 382 intended to be seen by application end users (e.g. for deprecated application 383 configuration settings). 384* :exc:`DeprecationWarning`: displayed by default only in :mod:`__main__` and when 385 running tests, recommended for warnings intended to be seen by other Python 386 developers where a version upgrade may result in changed behaviour or an 387 error. 388* :exc:`PendingDeprecationWarning`: displayed by default only when running 389 tests, intended for cases where a future version upgrade will change the 390 warning category to :exc:`DeprecationWarning` or :exc:`FutureWarning`. 391 392Previously both :exc:`DeprecationWarning` and :exc:`PendingDeprecationWarning` 393were only visible when running tests, which meant that developers primarily 394writing single file scripts or using Python interactively could be surprised 395by breaking changes in the APIs they used. 396 397.. seealso:: 398 399 :pep:`565` -- Show DeprecationWarning in ``__main__`` 400 PEP written and implemented by Nick Coghlan 401 402 403.. _whatsnew37-pep560: 404 405PEP 560: Core Support for ``typing`` module and Generic Types 406------------------------------------------------------------- 407 408Initially :pep:`484` was designed in such way that it would not introduce *any* 409changes to the core CPython interpreter. Now type hints and the :mod:`typing` 410module are extensively used by the community, so this restriction is removed. 411The PEP introduces two special methods :meth:`__class_getitem__` and 412``__mro_entries__``, these methods are now used by most classes and special 413constructs in :mod:`typing`. As a result, the speed of various operations 414with types increased up to 7 times, the generic types can be used without 415metaclass conflicts, and several long standing bugs in :mod:`typing` module are 416fixed. 417 418.. seealso:: 419 420 :pep:`560` -- Core support for typing module and generic types 421 PEP written and implemented by Ivan Levkivskyi 422 423 424.. _whatsnew37-pep552: 425 426PEP 552: Hash-based .pyc Files 427------------------------------ 428 429Python has traditionally checked the up-to-dateness of bytecode cache files 430(i.e., ``.pyc`` files) by comparing the source metadata (last-modified timestamp 431and size) with source metadata saved in the cache file header when it was 432generated. While effective, this invalidation method has its drawbacks. When 433filesystem timestamps are too coarse, Python can miss source updates, leading to 434user confusion. Additionally, having a timestamp in the cache file is 435problematic for `build reproducibility <https://reproducible-builds.org/>`_ and 436content-based build systems. 437 438:pep:`552` extends the pyc format to allow the hash of the source file to be 439used for invalidation instead of the source timestamp. Such ``.pyc`` files are 440called "hash-based". By default, Python still uses timestamp-based invalidation 441and does not generate hash-based ``.pyc`` files at runtime. Hash-based ``.pyc`` 442files may be generated with :mod:`py_compile` or :mod:`compileall`. 443 444Hash-based ``.pyc`` files come in two variants: checked and unchecked. Python 445validates checked hash-based ``.pyc`` files against the corresponding source 446files at runtime but doesn't do so for unchecked hash-based pycs. Unchecked 447hash-based ``.pyc`` files are a useful performance optimization for environments 448where a system external to Python (e.g., the build system) is responsible for 449keeping ``.pyc`` files up-to-date. 450 451See :ref:`pyc-invalidation` for more information. 452 453.. seealso:: 454 455 :pep:`552` -- Deterministic pycs 456 PEP written and implemented by Benjamin Peterson 457 458 459.. _whatsnew37-pep545: 460 461PEP 545: Python Documentation Translations 462------------------------------------------ 463 464:pep:`545` describes the process of creating and maintaining Python 465documentation translations. 466 467Three new translations have been added: 468 469- Japanese: https://docs.python.org/ja/ 470- French: https://docs.python.org/fr/ 471- Korean: https://docs.python.org/ko/ 472 473.. seealso:: 474 475 :pep:`545` -- Python Documentation Translations 476 PEP written and implemented by Julien Palard, Inada Naoki, and 477 Victor Stinner. 478 479 480.. _whatsnew37-devmode: 481 482Python Development Mode (-X dev) 483-------------------------------- 484 485The new :option:`-X` ``dev`` command line option or the new 486:envvar:`PYTHONDEVMODE` environment variable can be used to enable 487:ref:`Python Development Mode <devmode>`. When in development mode, Python performs 488additional runtime checks that are too expensive to be enabled by default. 489See :ref:`Python Development Mode <devmode>` documentation for the full 490description. 491 492 493Other Language Changes 494====================== 495 496* An :keyword:`await` expression and comprehensions containing an 497 :keyword:`async for` clause were illegal in the expressions in 498 :ref:`formatted string literals <f-strings>` due to a problem with the 499 implementation. In Python 3.7 this restriction was lifted. 500 501* More than 255 arguments can now be passed to a function, and a function can 502 now have more than 255 parameters. (Contributed by Serhiy Storchaka in 503 :issue:`12844` and :issue:`18896`.) 504 505* :meth:`bytes.fromhex` and :meth:`bytearray.fromhex` now ignore all ASCII 506 whitespace, not only spaces. (Contributed by Robert Xiao in :issue:`28927`.) 507 508* :class:`str`, :class:`bytes`, and :class:`bytearray` gained support for 509 the new :meth:`isascii() <str.isascii>` method, which can be used to 510 test if a string or bytes contain only the ASCII characters. 511 (Contributed by INADA Naoki in :issue:`32677`.) 512 513* :exc:`ImportError` now displays module name and module ``__file__`` path when 514 ``from ... import ...`` fails. (Contributed by Matthias Bussonnier in 515 :issue:`29546`.) 516 517* Circular imports involving absolute imports with binding a submodule to 518 a name are now supported. 519 (Contributed by Serhiy Storchaka in :issue:`30024`.) 520 521* ``object.__format__(x, '')`` is now equivalent to ``str(x)`` rather than 522 ``format(str(self), '')``. 523 (Contributed by Serhiy Storchaka in :issue:`28974`.) 524 525* In order to better support dynamic creation of stack traces, 526 :class:`types.TracebackType` can now be instantiated from Python code, and 527 the ``tb_next`` attribute on :ref:`tracebacks <traceback-objects>` is now 528 writable. 529 (Contributed by Nathaniel J. Smith in :issue:`30579`.) 530 531* When using the :option:`-m` switch, ``sys.path[0]`` is now eagerly expanded 532 to the full starting directory path, rather than being left as the empty 533 directory (which allows imports from the *current* working directory at the 534 time when an import occurs) 535 (Contributed by Nick Coghlan in :issue:`33053`.) 536 537* The new :option:`-X` ``importtime`` option or the 538 :envvar:`PYTHONPROFILEIMPORTTIME` environment variable can be used to show 539 the timing of each module import. 540 (Contributed by Victor Stinner in :issue:`31415`.) 541 542 543New Modules 544=========== 545 546.. _whatsnew37-pep567: 547 548contextvars 549----------- 550 551The new :mod:`contextvars` module and a set of 552:ref:`new C APIs <contextvarsobjects>` introduce 553support for *context variables*. Context variables are conceptually 554similar to thread-local variables. Unlike TLS, context variables 555support asynchronous code correctly. 556 557The :mod:`asyncio` and :mod:`decimal` modules have been updated to use 558and support context variables out of the box. Particularly the active 559decimal context is now stored in a context variable, which allows 560decimal operations to work with the correct context in asynchronous code. 561 562.. seealso:: 563 564 :pep:`567` -- Context Variables 565 PEP written and implemented by Yury Selivanov 566 567 568.. _whatsnew37-pep557: 569 570dataclasses 571----------- 572 573The new :func:`~dataclasses.dataclass` decorator provides a way to declare 574*data classes*. A data class describes its attributes using class variable 575annotations. Its constructor and other magic methods, such as 576:meth:`~object.__repr__`, :meth:`~object.__eq__`, and 577:meth:`~object.__hash__` are generated automatically. 578 579Example:: 580 581 @dataclass 582 class Point: 583 x: float 584 y: float 585 z: float = 0.0 586 587 p = Point(1.5, 2.5) 588 print(p) # produces "Point(x=1.5, y=2.5, z=0.0)" 589 590.. seealso:: 591 592 :pep:`557` -- Data Classes 593 PEP written and implemented by Eric V. Smith 594 595 596.. _whatsnew37_importlib_resources: 597 598importlib.resources 599------------------- 600 601The new :mod:`importlib.resources` module provides several new APIs and one 602new ABC for access to, opening, and reading *resources* inside packages. 603Resources are roughly similar to files inside packages, but they needn't 604be actual files on the physical file system. Module loaders can provide a 605:meth:`get_resource_reader()` function which returns 606a :class:`importlib.abc.ResourceReader` instance to support this 607new API. Built-in file path loaders and zip file loaders both support this. 608 609Contributed by Barry Warsaw and Brett Cannon in :issue:`32248`. 610 611.. seealso:: 612 613 `importlib_resources <http://importlib-resources.readthedocs.io/en/latest/>`_ 614 -- a PyPI backport for earlier Python versions. 615 616 617Improved Modules 618================ 619 620 621argparse 622-------- 623 624The new :meth:`ArgumentParser.parse_intermixed_args() 625<argparse.ArgumentParser.parse_intermixed_args>` 626method allows intermixing options and positional arguments. 627(Contributed by paul.j3 in :issue:`14191`.) 628 629 630.. _whatsnew37_asyncio: 631 632asyncio 633------- 634 635The :mod:`asyncio` module has received many new features, usability and 636:ref:`performance improvements <whatsnew37-asyncio-perf>`. Notable changes 637include: 638 639* The new :term:`provisional <provisional API>` :func:`asyncio.run` function can 640 be used to run a coroutine from synchronous code by automatically creating and 641 destroying the event loop. 642 (Contributed by Yury Selivanov in :issue:`32314`.) 643 644* asyncio gained support for :mod:`contextvars`. 645 :meth:`loop.call_soon() <asyncio.loop.call_soon>`, 646 :meth:`loop.call_soon_threadsafe() <asyncio.loop.call_soon_threadsafe>`, 647 :meth:`loop.call_later() <asyncio.loop.call_later>`, 648 :meth:`loop.call_at() <asyncio.loop.call_at>`, and 649 :meth:`Future.add_done_callback() <asyncio.Future.add_done_callback>` 650 have a new optional keyword-only *context* parameter. 651 :class:`Tasks <asyncio.Task>` now track their context automatically. 652 See :pep:`567` for more details. 653 (Contributed by Yury Selivanov in :issue:`32436`.) 654 655* The new :func:`asyncio.create_task` function has been added as a shortcut 656 to ``asyncio.get_event_loop().create_task()``. 657 (Contributed by Andrew Svetlov in :issue:`32311`.) 658 659* The new :meth:`loop.start_tls() <asyncio.loop.start_tls>` 660 method can be used to upgrade an existing connection to TLS. 661 (Contributed by Yury Selivanov in :issue:`23749`.) 662 663* The new :meth:`loop.sock_recv_into() <asyncio.loop.sock_recv_into>` 664 method allows reading data from a socket directly into a provided buffer making 665 it possible to reduce data copies. 666 (Contributed by Antoine Pitrou in :issue:`31819`.) 667 668* The new :func:`asyncio.current_task` function returns the currently running 669 :class:`~asyncio.Task` instance, and the new :func:`asyncio.all_tasks` 670 function returns a set of all existing ``Task`` instances in a given loop. 671 The :meth:`Task.current_task() <asyncio.Task.current_task>` and 672 :meth:`Task.all_tasks() <asyncio.Task.all_tasks>` methods have been deprecated. 673 (Contributed by Andrew Svetlov in :issue:`32250`.) 674 675* The new *provisional* :class:`~asyncio.BufferedProtocol` class allows 676 implementing streaming protocols with manual control over the receive buffer. 677 (Contributed by Yury Selivanov in :issue:`32251`.) 678 679* The new :func:`asyncio.get_running_loop` function returns the currently 680 running loop, and raises a :exc:`RuntimeError` if no loop is running. 681 This is in contrast with :func:`asyncio.get_event_loop`, which will *create* 682 a new event loop if none is running. 683 (Contributed by Yury Selivanov in :issue:`32269`.) 684 685* The new :meth:`StreamWriter.wait_closed() <asyncio.StreamWriter.wait_closed>` 686 coroutine method allows waiting until the stream writer is closed. The new 687 :meth:`StreamWriter.is_closing() <asyncio.StreamWriter.is_closing>` method 688 can be used to determine if the writer is closing. 689 (Contributed by Andrew Svetlov in :issue:`32391`.) 690 691* The new :meth:`loop.sock_sendfile() <asyncio.loop.sock_sendfile>` 692 coroutine method allows sending files using :mod:`os.sendfile` when possible. 693 (Contributed by Andrew Svetlov in :issue:`32410`.) 694 695* The new :meth:`Future.get_loop() <asyncio.Future.get_loop>` and 696 ``Task.get_loop()`` methods return the instance of the loop on which a task or 697 a future were created. 698 :meth:`Server.get_loop() <asyncio.Server.get_loop>` allows doing the same for 699 :class:`asyncio.Server` objects. 700 (Contributed by Yury Selivanov in :issue:`32415` and 701 Srinivas Reddy Thatiparthy in :issue:`32418`.) 702 703* It is now possible to control how instances of :class:`asyncio.Server` begin 704 serving. Previously, the server would start serving immediately when created. 705 The new *start_serving* keyword argument to 706 :meth:`loop.create_server() <asyncio.loop.create_server>` and 707 :meth:`loop.create_unix_server() <asyncio.loop.create_unix_server>`, 708 as well as :meth:`Server.start_serving() <asyncio.Server.start_serving>`, and 709 :meth:`Server.serve_forever() <asyncio.Server.serve_forever>` 710 can be used to decouple server instantiation and serving. The new 711 :meth:`Server.is_serving() <asyncio.Server.is_serving>` method returns ``True`` 712 if the server is serving. :class:`~asyncio.Server` objects are now 713 asynchronous context managers:: 714 715 srv = await loop.create_server(...) 716 717 async with srv: 718 # some code 719 720 # At this point, srv is closed and no longer accepts new connections. 721 722 (Contributed by Yury Selivanov in :issue:`32662`.) 723 724* Callback objects returned by 725 :func:`loop.call_later() <asyncio.loop.call_later>` 726 gained the new :meth:`when() <asyncio.TimerHandle.when>` method which 727 returns an absolute scheduled callback timestamp. 728 (Contributed by Andrew Svetlov in :issue:`32741`.) 729 730* The :meth:`loop.create_datagram_endpoint() \ 731 <asyncio.loop.create_datagram_endpoint>` method 732 gained support for Unix sockets. 733 (Contributed by Quentin Dawans in :issue:`31245`.) 734 735* The :func:`asyncio.open_connection`, :func:`asyncio.start_server` functions, 736 :meth:`loop.create_connection() <asyncio.loop.create_connection>`, 737 :meth:`loop.create_server() <asyncio.loop.create_server>`, 738 :meth:`loop.create_accepted_socket() <asyncio.loop.connect_accepted_socket>` 739 methods and their corresponding UNIX socket variants now accept the 740 *ssl_handshake_timeout* keyword argument. 741 (Contributed by Neil Aspinall in :issue:`29970`.) 742 743* The new :meth:`Handle.cancelled() <asyncio.Handle.cancelled>` method returns 744 ``True`` if the callback was cancelled. 745 (Contributed by Marat Sharafutdinov in :issue:`31943`.) 746 747* The asyncio source has been converted to use the 748 :keyword:`async`/:keyword:`await` syntax. 749 (Contributed by Andrew Svetlov in :issue:`32193`.) 750 751* The new :meth:`ReadTransport.is_reading() <asyncio.ReadTransport.is_reading>` 752 method can be used to determine the reading state of the transport. 753 Additionally, calls to 754 :meth:`ReadTransport.resume_reading() <asyncio.ReadTransport.resume_reading>` 755 and :meth:`ReadTransport.pause_reading() <asyncio.ReadTransport.pause_reading>` 756 are now idempotent. 757 (Contributed by Yury Selivanov in :issue:`32356`.) 758 759* Loop methods which accept socket paths now support passing 760 :term:`path-like objects <path-like object>`. 761 (Contributed by Yury Selivanov in :issue:`32066`.) 762 763* In :mod:`asyncio` TCP sockets on Linux are now created with ``TCP_NODELAY`` 764 flag set by default. 765 (Contributed by Yury Selivanov and Victor Stinner in :issue:`27456`.) 766 767* Exceptions occurring in cancelled tasks are no longer logged. 768 (Contributed by Yury Selivanov in :issue:`30508`.) 769 770* New ``WindowsSelectorEventLoopPolicy`` and 771 ``WindowsProactorEventLoopPolicy`` classes. 772 (Contributed by Yury Selivanov in :issue:`33792`.) 773 774Several ``asyncio`` APIs have been 775:ref:`deprecated <whatsnew37-asyncio-deprecated>`. 776 777 778binascii 779-------- 780 781The :func:`~binascii.b2a_uu` function now accepts an optional *backtick* 782keyword argument. When it's true, zeros are represented by ``'`'`` 783instead of spaces. (Contributed by Xiang Zhang in :issue:`30103`.) 784 785 786calendar 787-------- 788 789The :class:`~calendar.HTMLCalendar` class has new class attributes which ease 790the customization of CSS classes in the produced HTML calendar. 791(Contributed by Oz Tiram in :issue:`30095`.) 792 793 794collections 795----------- 796 797``collections.namedtuple()`` now supports default values. 798(Contributed by Raymond Hettinger in :issue:`32320`.) 799 800 801compileall 802---------- 803 804:func:`compileall.compile_dir` learned the new *invalidation_mode* parameter, 805which can be used to enable 806:ref:`hash-based .pyc invalidation <whatsnew37-pep552>`. The invalidation 807mode can also be specified on the command line using the new 808``--invalidation-mode`` argument. 809(Contributed by Benjamin Peterson in :issue:`31650`.) 810 811 812concurrent.futures 813------------------ 814 815:class:`ProcessPoolExecutor <concurrent.futures.ProcessPoolExecutor>` and 816:class:`ThreadPoolExecutor <concurrent.futures.ThreadPoolExecutor>` now 817support the new *initializer* and *initargs* constructor arguments. 818(Contributed by Antoine Pitrou in :issue:`21423`.) 819 820The :class:`ProcessPoolExecutor <concurrent.futures.ProcessPoolExecutor>` 821can now take the multiprocessing context via the new *mp_context* argument. 822(Contributed by Thomas Moreau in :issue:`31540`.) 823 824 825contextlib 826---------- 827 828The new :func:`~contextlib.nullcontext` is a simpler and faster no-op 829context manager than :class:`~contextlib.ExitStack`. 830(Contributed by Jesse-Bakker in :issue:`10049`.) 831 832The new :func:`~contextlib.asynccontextmanager`, 833:class:`~contextlib.AbstractAsyncContextManager`, and 834:class:`~contextlib.AsyncExitStack` have been added to 835complement their synchronous counterparts. (Contributed 836by Jelle Zijlstra in :issue:`29679` and :issue:`30241`, 837and by Alexander Mohr and Ilya Kulakov in :issue:`29302`.) 838 839 840cProfile 841-------- 842 843The :mod:`cProfile` command line now accepts ``-m module_name`` as an 844alternative to script path. (Contributed by Sanyam Khurana in :issue:`21862`.) 845 846 847crypt 848----- 849 850The :mod:`crypt` module now supports the Blowfish hashing method. 851(Contributed by Serhiy Storchaka in :issue:`31664`.) 852 853The :func:`~crypt.mksalt` function now allows specifying the number of rounds 854for hashing. (Contributed by Serhiy Storchaka in :issue:`31702`.) 855 856 857datetime 858-------- 859 860The new :meth:`datetime.fromisoformat() <datetime.datetime.fromisoformat>` 861method constructs a :class:`~datetime.datetime` object from a string 862in one of the formats output by 863:meth:`datetime.isoformat() <datetime.datetime.isoformat>`. 864(Contributed by Paul Ganssle in :issue:`15873`.) 865 866The :class:`tzinfo <datetime.tzinfo>` class now supports sub-minute offsets. 867(Contributed by Alexander Belopolsky in :issue:`5288`.) 868 869 870dbm 871--- 872 873:mod:`dbm.dumb` now supports reading read-only files and no longer writes the 874index file when it is not changed. 875 876 877decimal 878------- 879 880The :mod:`decimal` module now uses :ref:`context variables <whatsnew37-pep567>` 881to store the decimal context. 882(Contributed by Yury Selivanov in :issue:`32630`.) 883 884 885dis 886--- 887 888The :func:`~dis.dis` function is now able to 889disassemble nested code objects (the code of comprehensions, generator 890expressions and nested functions, and the code used for building nested 891classes). The maximum depth of disassembly recursion is controlled by 892the new *depth* parameter. 893(Contributed by Serhiy Storchaka in :issue:`11822`.) 894 895 896distutils 897--------- 898 899``README.rst`` is now included in the list of distutils standard READMEs and 900therefore included in source distributions. 901(Contributed by Ryan Gonzalez in :issue:`11913`.) 902 903 904enum 905---- 906 907The :class:`Enum <enum.Enum>` learned the new ``_ignore_`` class property, 908which allows listing the names of properties which should not become 909enum members. 910(Contributed by Ethan Furman in :issue:`31801`.) 911 912In Python 3.8, attempting to check for non-Enum objects in :class:`Enum` 913classes will raise a :exc:`TypeError` (e.g. ``1 in Color``); similarly, 914attempting to check for non-Flag objects in a :class:`Flag` member will 915raise :exc:`TypeError` (e.g. ``1 in Perm.RW``); currently, both operations 916return :const:`False` instead and are deprecated. 917(Contributed by Ethan Furman in :issue:`33217`.) 918 919 920functools 921--------- 922 923:func:`functools.singledispatch` now supports registering implementations 924using type annotations. 925(Contributed by Łukasz Langa in :issue:`32227`.) 926 927 928gc 929-- 930 931The new :func:`gc.freeze` function allows freezing all objects tracked 932by the garbage collector and excluding them from future collections. 933This can be used before a POSIX ``fork()`` call to make the GC copy-on-write 934friendly or to speed up collection. The new :func:`gc.unfreeze` functions 935reverses this operation. Additionally, :func:`gc.get_freeze_count` can 936be used to obtain the number of frozen objects. 937(Contributed by Li Zekun in :issue:`31558`.) 938 939 940hmac 941---- 942 943The :mod:`hmac` module now has an optimized one-shot :func:`~hmac.digest` 944function, which is up to three times faster than :func:`~hmac.HMAC`. 945(Contributed by Christian Heimes in :issue:`32433`.) 946 947 948http.client 949----------- 950 951:class:`~http.client.HTTPConnection` and :class:`~http.client.HTTPSConnection` 952now support the new *blocksize* argument for improved upload throughput. 953(Contributed by Nir Soffer in :issue:`31945`.) 954 955 956http.server 957----------- 958 959:class:`~http.server.SimpleHTTPRequestHandler` now supports the HTTP 960``If-Modified-Since`` header. The server returns the 304 response status if 961the target file was not modified after the time specified in the header. 962(Contributed by Pierre Quentel in :issue:`29654`.) 963 964:class:`~http.server.SimpleHTTPRequestHandler` accepts the new *directory* 965argument, in addition to the new ``--directory`` command line argument. 966With this parameter, the server serves the specified directory, by default it 967uses the current working directory. 968(Contributed by Stéphane Wirtel and Julien Palard in :issue:`28707`.) 969 970The new :class:`ThreadingHTTPServer <http.server.ThreadingHTTPServer>` class 971uses threads to handle requests using :class:`~socketserver.ThreadingMixin`. 972It is used when ``http.server`` is run with ``-m``. 973(Contributed by Julien Palard in :issue:`31639`.) 974 975 976idlelib and IDLE 977---------------- 978 979Multiple fixes for autocompletion. (Contributed by Louie Lu in :issue:`15786`.) 980 981Module Browser (on the File menu, formerly called Class Browser), 982now displays nested functions and classes in addition to top-level 983functions and classes. 984(Contributed by Guilherme Polo, Cheryl Sabella, and Terry Jan Reedy 985in :issue:`1612262`.) 986 987The Settings dialog (Options, Configure IDLE) has been partly rewritten 988to improve both appearance and function. 989(Contributed by Cheryl Sabella and Terry Jan Reedy in multiple issues.) 990 991The font sample now includes a selection of non-Latin characters so that 992users can better see the effect of selecting a particular font. 993(Contributed by Terry Jan Reedy in :issue:`13802`.) 994The sample can be edited to include other characters. 995(Contributed by Serhiy Storchaka in :issue:`31860`.) 996 997The IDLE features formerly implemented as extensions have been reimplemented 998as normal features. Their settings have been moved from the Extensions tab 999to other dialog tabs. 1000(Contributed by Charles Wohlganger and Terry Jan Reedy in :issue:`27099`.) 1001 1002Editor code context option revised. Box displays all context lines up to 1003maxlines. Clicking on a context line jumps the editor to that line. Context 1004colors for custom themes is added to Highlights tab of Settings dialog. 1005(Contributed by Cheryl Sabella and Terry Jan Reedy in :issue:`33642`, 1006:issue:`33768`, and :issue:`33679`.) 1007 1008On Windows, a new API call tells Windows that tk scales for DPI. On Windows 10098.1+ or 10, with DPI compatibility properties of the Python binary 1010unchanged, and a monitor resolution greater than 96 DPI, this should 1011make text and lines sharper. It should otherwise have no effect. 1012(Contributed by Terry Jan Reedy in :issue:`33656`.) 1013 1014New in 3.7.1: 1015 1016Output over N lines (50 by default) is squeezed down to a button. 1017N can be changed in the PyShell section of the General page of the 1018Settings dialog. Fewer, but possibly extra long, lines can be squeezed by 1019right clicking on the output. Squeezed output can be expanded in place 1020by double-clicking the button or into the clipboard or a separate window 1021by right-clicking the button. (Contributed by Tal Einat in :issue:`1529353`.) 1022 1023The changes above have been backported to 3.6 maintenance releases. 1024 1025NEW in 3.7.4: 1026 1027Add "Run Customized" to the Run menu to run a module with customized 1028settings. Any command line arguments entered are added to sys.argv. 1029They re-appear in the box for the next customized run. One can also 1030suppress the normal Shell main module restart. (Contributed by Cheryl 1031Sabella, Terry Jan Reedy, and others in :issue:`5680` and :issue:`37627`.) 1032 1033New in 3.7.5: 1034 1035Add optional line numbers for IDLE editor windows. Windows 1036open without line numbers unless set otherwise in the General 1037tab of the configuration dialog. Line numbers for an existing 1038window are shown and hidden in the Options menu. 1039(Contributed by Tal Einat and Saimadhav Heblikar in :issue:`17535`.) 1040 1041 1042importlib 1043--------- 1044 1045The :class:`importlib.abc.ResourceReader` ABC was introduced to 1046support the loading of resources from packages. See also 1047:ref:`whatsnew37_importlib_resources`. 1048(Contributed by Barry Warsaw, Brett Cannon in :issue:`32248`.) 1049 1050:func:`importlib.reload` now raises :exc:`ModuleNotFoundError` if the module 1051lacks a spec. 1052(Contributed by Garvit Khatri in :issue:`29851`.) 1053 1054:func:`importlib.find_spec` now raises :exc:`ModuleNotFoundError` instead of 1055:exc:`AttributeError` if the specified parent module is not a package (i.e. 1056lacks a ``__path__`` attribute). 1057(Contributed by Milan Oberkirch in :issue:`30436`.) 1058 1059The new :func:`importlib.source_hash` can be used to compute the hash of 1060the passed source. A :ref:`hash-based .pyc file <whatsnew37-pep552>` 1061embeds the value returned by this function. 1062 1063 1064io 1065-- 1066 1067The new :meth:`TextIOWrapper.reconfigure() <io.TextIOWrapper.reconfigure>` 1068method can be used to reconfigure the text stream with the new settings. 1069(Contributed by Antoine Pitrou in :issue:`30526` and 1070INADA Naoki in :issue:`15216`.) 1071 1072 1073ipaddress 1074--------- 1075 1076The new ``subnet_of()`` and ``supernet_of()`` methods of 1077:class:`ipaddress.IPv6Network` and :class:`ipaddress.IPv4Network` can 1078be used for network containment tests. 1079(Contributed by Michel Albert and Cheryl Sabella in :issue:`20825`.) 1080 1081 1082itertools 1083--------- 1084 1085:func:`itertools.islice` now accepts 1086:meth:`integer-like objects <object.__index__>` as start, stop, 1087and slice arguments. 1088(Contributed by Will Roberts in :issue:`30537`.) 1089 1090 1091locale 1092------ 1093 1094The new *monetary* argument to :func:`locale.format_string` can be used 1095to make the conversion use monetary thousands separators and 1096grouping strings. (Contributed by Garvit in :issue:`10379`.) 1097 1098The :func:`locale.getpreferredencoding` function now always returns ``'UTF-8'`` 1099on Android or when in the :ref:`forced UTF-8 mode <whatsnew37-pep540>`. 1100 1101 1102logging 1103------- 1104 1105:class:`~logging.Logger` instances can now be pickled. 1106(Contributed by Vinay Sajip in :issue:`30520`.) 1107 1108The new :meth:`StreamHandler.setStream() <logging.StreamHandler.setStream>` 1109method can be used to replace the logger stream after handler creation. 1110(Contributed by Vinay Sajip in :issue:`30522`.) 1111 1112It is now possible to specify keyword arguments to handler constructors in 1113configuration passed to :func:`logging.config.fileConfig`. 1114(Contributed by Preston Landers in :issue:`31080`.) 1115 1116 1117math 1118---- 1119 1120The new :func:`math.remainder` function implements the IEEE 754-style remainder 1121operation. (Contributed by Mark Dickinson in :issue:`29962`.) 1122 1123 1124mimetypes 1125--------- 1126 1127The MIME type of .bmp has been changed from ``'image/x-ms-bmp'`` to 1128``'image/bmp'``. 1129(Contributed by Nitish Chandra in :issue:`22589`.) 1130 1131 1132msilib 1133------ 1134 1135The new :meth:`Database.Close() <msilib.Database.Close>` method can be used 1136to close the :abbr:`MSI` database. 1137(Contributed by Berker Peksag in :issue:`20486`.) 1138 1139 1140multiprocessing 1141--------------- 1142 1143The new :meth:`Process.close() <multiprocessing.Process.close>` method 1144explicitly closes the process object and releases all resources associated 1145with it. :exc:`ValueError` is raised if the underlying process is still 1146running. 1147(Contributed by Antoine Pitrou in :issue:`30596`.) 1148 1149The new :meth:`Process.kill() <multiprocessing.Process.kill>` method can 1150be used to terminate the process using the :data:`SIGKILL` signal on Unix. 1151(Contributed by Vitor Pereira in :issue:`30794`.) 1152 1153Non-daemonic threads created by :class:`~multiprocessing.Process` are now 1154joined on process exit. 1155(Contributed by Antoine Pitrou in :issue:`18966`.) 1156 1157 1158os 1159-- 1160 1161:func:`os.fwalk` now accepts the *path* argument as :class:`bytes`. 1162(Contributed by Serhiy Storchaka in :issue:`28682`.) 1163 1164:func:`os.scandir` gained support for :ref:`file descriptors <path_fd>`. 1165(Contributed by Serhiy Storchaka in :issue:`25996`.) 1166 1167The new :func:`~os.register_at_fork` function allows registering Python 1168callbacks to be executed at process fork. 1169(Contributed by Antoine Pitrou in :issue:`16500`.) 1170 1171Added :func:`os.preadv` (combine the functionality of :func:`os.readv` and 1172:func:`os.pread`) and :func:`os.pwritev` functions (combine the functionality 1173of :func:`os.writev` and :func:`os.pwrite`). (Contributed by Pablo Galindo in 1174:issue:`31368`.) 1175 1176The mode argument of :func:`os.makedirs` no longer affects the file 1177permission bits of newly-created intermediate-level directories. 1178(Contributed by Serhiy Storchaka in :issue:`19930`.) 1179 1180:func:`os.dup2` now returns the new file descriptor. Previously, ``None`` 1181was always returned. 1182(Contributed by Benjamin Peterson in :issue:`32441`.) 1183 1184The structure returned by :func:`os.stat` now contains the 1185:attr:`~os.stat_result.st_fstype` attribute on Solaris and its derivatives. 1186(Contributed by Jesús Cea Avión in :issue:`32659`.) 1187 1188 1189pathlib 1190------- 1191 1192The new :meth:`Path.is_mount() <pathlib.Path.is_mount>` method is now available 1193on POSIX systems and can be used to determine whether a path is a mount point. 1194(Contributed by Cooper Ry Lees in :issue:`30897`.) 1195 1196 1197pdb 1198--- 1199 1200:func:`pdb.set_trace` now takes an optional *header* keyword-only 1201argument. If given, it is printed to the console just before debugging 1202begins. (Contributed by Barry Warsaw in :issue:`31389`.) 1203 1204:mod:`pdb` command line now accepts ``-m module_name`` as an alternative to 1205script file. (Contributed by Mario Corchero in :issue:`32206`.) 1206 1207 1208py_compile 1209---------- 1210 1211:func:`py_compile.compile` -- and by extension, :mod:`compileall` -- now 1212respects the :envvar:`SOURCE_DATE_EPOCH` environment variable by 1213unconditionally creating ``.pyc`` files for hash-based validation. 1214This allows for guaranteeing 1215`reproducible builds <https://reproducible-builds.org/>`_ of ``.pyc`` 1216files when they are created eagerly. (Contributed by Bernhard M. Wiedemann 1217in :issue:`29708`.) 1218 1219 1220pydoc 1221----- 1222 1223The pydoc server can now bind to an arbitrary hostname specified by the 1224new ``-n`` command-line argument. 1225(Contributed by Feanil Patel in :issue:`31128`.) 1226 1227 1228queue 1229----- 1230 1231The new :class:`~queue.SimpleQueue` class is an unbounded :abbr:`FIFO` queue. 1232(Contributed by Antoine Pitrou in :issue:`14976`.) 1233 1234 1235re 1236-- 1237 1238The flags :const:`re.ASCII`, :const:`re.LOCALE` and :const:`re.UNICODE` 1239can be set within the scope of a group. 1240(Contributed by Serhiy Storchaka in :issue:`31690`.) 1241 1242:func:`re.split` now supports splitting on a pattern like ``r'\b'``, 1243``'^$'`` or ``(?=-)`` that matches an empty string. 1244(Contributed by Serhiy Storchaka in :issue:`25054`.) 1245 1246Regular expressions compiled with the :const:`re.LOCALE` flag no longer 1247depend on the locale at compile time. Locale settings are applied only 1248when the compiled regular expression is used. 1249(Contributed by Serhiy Storchaka in :issue:`30215`.) 1250 1251:exc:`FutureWarning` is now emitted if a regular expression contains 1252character set constructs that will change semantically in the future, 1253such as nested sets and set operations. 1254(Contributed by Serhiy Storchaka in :issue:`30349`.) 1255 1256Compiled regular expression and match objects can now be copied 1257using :func:`copy.copy` and :func:`copy.deepcopy`. 1258(Contributed by Serhiy Storchaka in :issue:`10076`.) 1259 1260 1261signal 1262------ 1263 1264The new *warn_on_full_buffer* argument to the :func:`signal.set_wakeup_fd` 1265function makes it possible to specify whether Python prints a warning on 1266stderr when the wakeup buffer overflows. 1267(Contributed by Nathaniel J. Smith in :issue:`30050`.) 1268 1269 1270socket 1271------ 1272 1273The new :func:`socket.getblocking() <socket.socket.getblocking>` method 1274returns ``True`` if the socket is in blocking mode and ``False`` otherwise. 1275(Contributed by Yury Selivanov in :issue:`32373`.) 1276 1277The new :func:`socket.close` function closes the passed socket file descriptor. 1278This function should be used instead of :func:`os.close` for better 1279compatibility across platforms. 1280(Contributed by Christian Heimes in :issue:`32454`.) 1281 1282The :mod:`socket` module now exposes the :data:`socket.TCP_CONGESTION` 1283(Linux 2.6.13), :data:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37), and 1284:data:`socket.TCP_NOTSENT_LOWAT` (Linux 3.12) constants. 1285(Contributed by Omar Sandoval in :issue:`26273` and 1286Nathaniel J. Smith in :issue:`29728`.) 1287 1288Support for :data:`socket.AF_VSOCK` sockets has been added to allow 1289communication between virtual machines and their hosts. 1290(Contributed by Cathy Avery in :issue:`27584`.) 1291 1292Sockets now auto-detect family, type and protocol from file descriptor 1293by default. 1294(Contributed by Christian Heimes in :issue:`28134`.) 1295 1296 1297socketserver 1298------------ 1299 1300:meth:`socketserver.ThreadingMixIn.server_close` now waits until all non-daemon 1301threads complete. :meth:`socketserver.ForkingMixIn.server_close` now waits 1302until all child processes complete. 1303 1304Add a new :attr:`socketserver.ForkingMixIn.block_on_close` class attribute to 1305:class:`socketserver.ForkingMixIn` and :class:`socketserver.ThreadingMixIn` 1306classes. Set the class attribute to ``False`` to get the pre-3.7 behaviour. 1307 1308 1309sqlite3 1310------- 1311 1312:class:`sqlite3.Connection` now exposes the :meth:`~sqlite3.Connection.backup` 1313method when the underlying SQLite library is at version 3.6.11 or higher. 1314(Contributed by Lele Gaifax in :issue:`27645`.) 1315 1316The *database* argument of :func:`sqlite3.connect` now accepts any 1317:term:`path-like object`, instead of just a string. 1318(Contributed by Anders Lorentsen in :issue:`31843`.) 1319 1320 1321ssl 1322--- 1323 1324The :mod:`ssl` module now uses OpenSSL's builtin API instead of 1325:func:`~ssl.match_hostname` to check a host name or an IP address. Values 1326are validated during TLS handshake. Any certificate validation error 1327including failing the host name check now raises 1328:exc:`~ssl.SSLCertVerificationError` and aborts the handshake with a proper 1329TLS Alert message. The new exception contains additional information. 1330Host name validation can be customized with 1331:attr:`SSLContext.hostname_checks_common_name <ssl.SSLContext.hostname_checks_common_name>`. 1332(Contributed by Christian Heimes in :issue:`31399`.) 1333 1334.. note:: 1335 The improved host name check requires a *libssl* implementation compatible 1336 with OpenSSL 1.0.2 or 1.1. Consequently, OpenSSL 0.9.8 and 1.0.1 are no 1337 longer supported (see :ref:`37-platform-support-removals` for more details). 1338 The ssl module is mostly compatible with LibreSSL 2.7.2 and newer. 1339 1340The ``ssl`` module no longer sends IP addresses in SNI TLS extension. 1341(Contributed by Christian Heimes in :issue:`32185`.) 1342 1343:func:`~ssl.match_hostname` no longer supports partial wildcards like 1344``www*.example.org``. 1345(Contributed by Mandeep Singh in :issue:`23033` and Christian Heimes in 1346:issue:`31399`.) 1347 1348The default cipher suite selection of the ``ssl`` module now uses a blacklist 1349approach rather than a hard-coded whitelist. Python no longer re-enables 1350ciphers that have been blocked by OpenSSL security updates. Default cipher 1351suite selection can be configured at compile time. 1352(Contributed by Christian Heimes in :issue:`31429`.) 1353 1354Validation of server certificates containing internationalized domain names 1355(IDNs) is now supported. As part of this change, the 1356:attr:`SSLSocket.server_hostname <ssl.SSLSocket.server_hostname>` attribute 1357now stores the expected hostname in A-label form (``"xn--pythn-mua.org"``), 1358rather than the U-label form (``"pythön.org"``). (Contributed by 1359Nathaniel J. Smith and Christian Heimes in :issue:`28414`.) 1360 1361The ``ssl`` module has preliminary and experimental support for TLS 1.3 and 1362OpenSSL 1.1.1. At the time of Python 3.7.0 release, OpenSSL 1.1.1 is still 1363under development and TLS 1.3 hasn't been finalized yet. The TLS 1.3 1364handshake and protocol behaves slightly differently than TLS 1.2 and earlier, 1365see :ref:`ssl-tlsv1_3`. 1366(Contributed by Christian Heimes in :issue:`32947`, :issue:`20995`, 1367:issue:`29136`, :issue:`30622` and :issue:`33618`) 1368 1369:class:`~ssl.SSLSocket` and :class:`~ssl.SSLObject` no longer have a public 1370constructor. Direct instantiation was never a documented and supported 1371feature. Instances must be created with :class:`~ssl.SSLContext` methods 1372:meth:`~ssl.SSLContext.wrap_socket` and :meth:`~ssl.SSLContext.wrap_bio`. 1373(Contributed by Christian Heimes in :issue:`32951`) 1374 1375OpenSSL 1.1 APIs for setting the minimum and maximum TLS protocol version are 1376available as :attr:`SSLContext.minimum_version <ssl.SSLContext.minimum_version>` 1377and :attr:`SSLContext.maximum_version <ssl.SSLContext.maximum_version>`. 1378Supported protocols are indicated by several new flags, such as 1379:data:`~ssl.HAS_TLSv1_1`. 1380(Contributed by Christian Heimes in :issue:`32609`.) 1381 1382 1383string 1384------ 1385 1386:class:`string.Template` now lets you to optionally modify the regular 1387expression pattern for braced placeholders and non-braced placeholders 1388separately. (Contributed by Barry Warsaw in :issue:`1198569`.) 1389 1390 1391subprocess 1392---------- 1393 1394The :func:`subprocess.run` function accepts the new *capture_output* 1395keyword argument. When true, stdout and stderr will be captured. 1396This is equivalent to passing :data:`subprocess.PIPE` as *stdout* and 1397*stderr* arguments. 1398(Contributed by Bo Bayles in :issue:`32102`.) 1399 1400The ``subprocess.run`` function and the :class:`subprocess.Popen` constructor 1401now accept the *text* keyword argument as an alias 1402to *universal_newlines*. 1403(Contributed by Andrew Clegg in :issue:`31756`.) 1404 1405On Windows the default for *close_fds* was changed from ``False`` to 1406``True`` when redirecting the standard handles. It's now possible to set 1407*close_fds* to true when redirecting the standard handles. See 1408:class:`subprocess.Popen`. This means that *close_fds* now defaults to 1409``True`` on all supported platforms. 1410(Contributed by Segev Finer in :issue:`19764`.) 1411 1412The subprocess module is now more graceful when handling 1413:exc:`KeyboardInterrupt` during :func:`subprocess.call`, 1414:func:`subprocess.run`, or in a :class:`~subprocess.Popen` 1415context manager. It now waits a short amount of time for the child 1416to exit, before continuing the handling of the ``KeyboardInterrupt`` 1417exception. 1418(Contributed by Gregory P. Smith in :issue:`25942`.) 1419 1420 1421sys 1422--- 1423 1424The new :func:`sys.breakpointhook` hook function is called by the 1425built-in :func:`breakpoint`. 1426(Contributed by Barry Warsaw in :issue:`31353`.) 1427 1428On Android, the new :func:`sys.getandroidapilevel` returns the build-time 1429Android API version. 1430(Contributed by Victor Stinner in :issue:`28740`.) 1431 1432The new :func:`sys.get_coroutine_origin_tracking_depth` function returns 1433the current coroutine origin tracking depth, as set by 1434the new :func:`sys.set_coroutine_origin_tracking_depth`. :mod:`asyncio` 1435has been converted to use this new API instead of 1436the deprecated :func:`sys.set_coroutine_wrapper`. 1437(Contributed by Nathaniel J. Smith in :issue:`32591`.) 1438 1439 1440time 1441---- 1442 1443:pep:`564` adds six new functions with nanosecond resolution to the 1444:mod:`time` module: 1445 1446* :func:`time.clock_gettime_ns` 1447* :func:`time.clock_settime_ns` 1448* :func:`time.monotonic_ns` 1449* :func:`time.perf_counter_ns` 1450* :func:`time.process_time_ns` 1451* :func:`time.time_ns` 1452 1453New clock identifiers have been added: 1454 1455* :data:`time.CLOCK_BOOTTIME` (Linux): Identical to 1456 :data:`time.CLOCK_MONOTONIC`, except it also includes any time that the 1457 system is suspended. 1458* :data:`time.CLOCK_PROF` (FreeBSD, NetBSD and OpenBSD): High-resolution 1459 per-process CPU timer. 1460* :data:`time.CLOCK_UPTIME` (FreeBSD, OpenBSD): Time whose absolute value is 1461 the time the system has been running and not suspended, providing accurate 1462 uptime measurement. 1463 1464The new :func:`time.thread_time` and :func:`time.thread_time_ns` functions 1465can be used to get per-thread CPU time measurements. 1466(Contributed by Antoine Pitrou in :issue:`32025`.) 1467 1468The new :func:`time.pthread_getcpuclockid` function returns the clock ID 1469of the thread-specific CPU-time clock. 1470 1471 1472tkinter 1473------- 1474 1475The new :class:`tkinter.ttk.Spinbox` class is now available. 1476(Contributed by Alan Moore in :issue:`32585`.) 1477 1478 1479tracemalloc 1480----------- 1481 1482:class:`tracemalloc.Traceback` behaves more like regular tracebacks, 1483sorting the frames from oldest to most recent. 1484:meth:`Traceback.format() <tracemalloc.Traceback.format>` 1485now accepts negative *limit*, truncating the result to the 1486``abs(limit)`` oldest frames. To get the old behaviour, use 1487the new *most_recent_first* argument to ``Traceback.format()``. 1488(Contributed by Jesse Bakker in :issue:`32121`.) 1489 1490 1491types 1492----- 1493 1494The new :class:`~types.WrapperDescriptorType`, 1495:class:`~types.MethodWrapperType`, :class:`~types.MethodDescriptorType`, 1496and :class:`~types.ClassMethodDescriptorType` classes are now available. 1497(Contributed by Manuel Krebber and Guido van Rossum in :issue:`29377`, 1498and Serhiy Storchaka in :issue:`32265`.) 1499 1500The new :func:`types.resolve_bases` function resolves MRO entries 1501dynamically as specified by :pep:`560`. 1502(Contributed by Ivan Levkivskyi in :issue:`32717`.) 1503 1504 1505unicodedata 1506----------- 1507 1508The internal :mod:`unicodedata` database has been upgraded to use `Unicode 11 1509<http://www.unicode.org/versions/Unicode11.0.0/>`_. (Contributed by Benjamin 1510Peterson.) 1511 1512 1513unittest 1514-------- 1515 1516The new ``-k`` command-line option allows filtering tests by a name 1517substring or a Unix shell-like pattern. 1518For example, ``python -m unittest -k foo`` runs 1519``foo_tests.SomeTest.test_something``, ``bar_tests.SomeTest.test_foo``, 1520but not ``bar_tests.FooTest.test_something``. 1521(Contributed by Jonas Haag in :issue:`32071`.) 1522 1523 1524unittest.mock 1525------------- 1526 1527The :const:`~unittest.mock.sentinel` attributes now preserve their identity 1528when they are :mod:`copied <copy>` or :mod:`pickled <pickle>`. (Contributed by 1529Serhiy Storchaka in :issue:`20804`.) 1530 1531The new :func:`~unittest.mock.seal` function allows sealing 1532:class:`~unittest.mock.Mock` instances, which will disallow further creation 1533of attribute mocks. The seal is applied recursively to all attributes that 1534are themselves mocks. 1535(Contributed by Mario Corchero in :issue:`30541`.) 1536 1537 1538urllib.parse 1539------------ 1540 1541:func:`urllib.parse.quote` has been updated from :rfc:`2396` to :rfc:`3986`, 1542adding ``~`` to the set of characters that are never quoted by default. 1543(Contributed by Christian Theune and Ratnadeep Debnath in :issue:`16285`.) 1544 1545 1546uu 1547-- 1548 1549The :func:`uu.encode` function now accepts an optional *backtick* 1550keyword argument. When it's true, zeros are represented by ``'`'`` 1551instead of spaces. (Contributed by Xiang Zhang in :issue:`30103`.) 1552 1553 1554uuid 1555---- 1556 1557The new :attr:`UUID.is_safe <uuid.UUID.is_safe>` attribute relays information 1558from the platform about whether generated UUIDs are generated with a 1559multiprocessing-safe method. 1560(Contributed by Barry Warsaw in :issue:`22807`.) 1561 1562:func:`uuid.getnode` now prefers universally administered 1563MAC addresses over locally administered MAC addresses. 1564This makes a better guarantee for global uniqueness of UUIDs returned 1565from :func:`uuid.uuid1`. If only locally administered MAC addresses are 1566available, the first such one found is returned. 1567(Contributed by Barry Warsaw in :issue:`32107`.) 1568 1569 1570warnings 1571-------- 1572 1573The initialization of the default warnings filters has changed as follows: 1574 1575* warnings enabled via command line options (including those for :option:`-b` 1576 and the new CPython-specific :option:`-X` ``dev`` option) are always passed 1577 to the warnings machinery via the :data:`sys.warnoptions` attribute. 1578 1579* warnings filters enabled via the command line or the environment now have the 1580 following order of precedence: 1581 1582 * the ``BytesWarning`` filter for :option:`-b` (or ``-bb``) 1583 * any filters specified with the :option:`-W` option 1584 * any filters specified with the :envvar:`PYTHONWARNINGS` environment 1585 variable 1586 * any other CPython specific filters (e.g. the ``default`` filter added 1587 for the new ``-X dev`` mode) 1588 * any implicit filters defined directly by the warnings machinery 1589 1590* in CPython debug builds, all warnings are now displayed by default (the 1591 implicit filter list is empty) 1592 1593(Contributed by Nick Coghlan and Victor Stinner in :issue:`20361`, 1594:issue:`32043`, and :issue:`32230`.) 1595 1596Deprecation warnings are once again shown by default in single-file scripts and 1597at the interactive prompt. See :ref:`whatsnew37-pep565` for details. 1598(Contributed by Nick Coghlan in :issue:`31975`.) 1599 1600 1601xml.etree 1602--------- 1603 1604:ref:`ElementPath <elementtree-xpath>` predicates in the :meth:`find` 1605methods can now compare text of the current node with ``[. = "text"]``, 1606not only text in children. Predicates also allow adding spaces for 1607better readability. (Contributed by Stefan Behnel in :issue:`31648`.) 1608 1609 1610xmlrpc.server 1611------------- 1612 1613:meth:`SimpleXMLRPCDispatcher.register_function <xmlrpc.server.SimpleXMLRPCDispatcher>` 1614can now be used as a decorator. (Contributed by Xiang Zhang in 1615:issue:`7769`.) 1616 1617 1618zipapp 1619------ 1620 1621Function :func:`~zipapp.create_archive` now accepts an optional *filter* 1622argument to allow the user to select which files should be included in the 1623archive. (Contributed by Irmen de Jong in :issue:`31072`.) 1624 1625Function :func:`~zipapp.create_archive` now accepts an optional *compressed* 1626argument to generate a compressed archive. A command line option 1627``--compress`` has also been added to support compression. 1628(Contributed by Zhiming Wang in :issue:`31638`.) 1629 1630 1631zipfile 1632------- 1633 1634:class:`~zipfile.ZipFile` now accepts the new *compresslevel* parameter to 1635control the compression level. 1636(Contributed by Bo Bayles in :issue:`21417`.) 1637 1638Subdirectories in archives created by ``ZipFile`` are now stored in 1639alphabetical order. 1640(Contributed by Bernhard M. Wiedemann in :issue:`30693`.) 1641 1642 1643C API Changes 1644============= 1645 1646A new API for thread-local storage has been implemented. See 1647:ref:`whatsnew37-pep539` for an overview and 1648:ref:`thread-specific-storage-api` for a complete reference. 1649(Contributed by Masayuki Yamamoto in :issue:`25658`.) 1650 1651The new :ref:`context variables <whatsnew37-pep567>` functionality 1652exposes a number of :ref:`new C APIs <contextvarsobjects>`. 1653 1654The new :c:func:`PyImport_GetModule` function returns the previously 1655imported module with the given name. 1656(Contributed by Eric Snow in :issue:`28411`.) 1657 1658The new :c:macro:`Py_RETURN_RICHCOMPARE` macro eases writing rich 1659comparison functions. 1660(Contributed by Petr Victorin in :issue:`23699`.) 1661 1662The new :c:macro:`Py_UNREACHABLE` macro can be used to mark unreachable 1663code paths. 1664(Contributed by Barry Warsaw in :issue:`31338`.) 1665 1666The :mod:`tracemalloc` now exposes a C API through the new 1667:c:func:`PyTraceMalloc_Track` and :c:func:`PyTraceMalloc_Untrack` 1668functions. 1669(Contributed by Victor Stinner in :issue:`30054`.) 1670 1671The new :c:func:`import__find__load__start` and 1672:c:func:`import__find__load__done` static markers can be used to trace 1673module imports. 1674(Contributed by Christian Heimes in :issue:`31574`.) 1675 1676The fields :c:member:`name` and :c:member:`doc` of structures 1677:c:type:`PyMemberDef`, :c:type:`PyGetSetDef`, 1678:c:type:`PyStructSequence_Field`, :c:type:`PyStructSequence_Desc`, 1679and :c:type:`wrapperbase` are now of type ``const char *`` rather of 1680``char *``. (Contributed by Serhiy Storchaka in :issue:`28761`.) 1681 1682The result of :c:func:`PyUnicode_AsUTF8AndSize` and :c:func:`PyUnicode_AsUTF8` 1683is now of type ``const char *`` rather of ``char *``. (Contributed by Serhiy 1684Storchaka in :issue:`28769`.) 1685 1686The result of :c:func:`PyMapping_Keys`, :c:func:`PyMapping_Values` and 1687:c:func:`PyMapping_Items` is now always a list, rather than a list or a 1688tuple. (Contributed by Oren Milman in :issue:`28280`.) 1689 1690Added functions :c:func:`PySlice_Unpack` and :c:func:`PySlice_AdjustIndices`. 1691(Contributed by Serhiy Storchaka in :issue:`27867`.) 1692 1693:c:func:`PyOS_AfterFork` is deprecated in favour of the new functions 1694:c:func:`PyOS_BeforeFork`, :c:func:`PyOS_AfterFork_Parent` and 1695:c:func:`PyOS_AfterFork_Child`. (Contributed by Antoine Pitrou in 1696:issue:`16500`.) 1697 1698The ``PyExc_RecursionErrorInst`` singleton that was part of the public API 1699has been removed as its members being never cleared may cause a segfault 1700during finalization of the interpreter. Contributed by Xavier de Gaye in 1701:issue:`22898` and :issue:`30697`. 1702 1703Added C API support for timezones with timezone constructors 1704:c:func:`PyTimeZone_FromOffset` and :c:func:`PyTimeZone_FromOffsetAndName`, 1705and access to the UTC singleton with :c:data:`PyDateTime_TimeZone_UTC`. 1706Contributed by Paul Ganssle in :issue:`10381`. 1707 1708The type of results of :c:func:`PyThread_start_new_thread` and 1709:c:func:`PyThread_get_thread_ident`, and the *id* parameter of 1710:c:func:`PyThreadState_SetAsyncExc` changed from :c:type:`long` to 1711:c:type:`unsigned long`. 1712(Contributed by Serhiy Storchaka in :issue:`6532`.) 1713 1714:c:func:`PyUnicode_AsWideCharString` now raises a :exc:`ValueError` if the 1715second argument is ``NULL`` and the :c:type:`wchar_t*` string contains null 1716characters. (Contributed by Serhiy Storchaka in :issue:`30708`.) 1717 1718Changes to the startup sequence and the management of dynamic memory 1719allocators mean that the long documented requirement to call 1720:c:func:`Py_Initialize` before calling most C API functions is now 1721relied on more heavily, and failing to abide by it may lead to segfaults in 1722embedding applications. See the :ref:`porting-to-python-37` section in this 1723document and the :ref:`pre-init-safe` section in the C API documentation 1724for more details. 1725 1726The new :c:func:`PyInterpreterState_GetID` returns the unique ID for a 1727given interpreter. 1728(Contributed by Eric Snow in :issue:`29102`.) 1729 1730:c:func:`Py_DecodeLocale`, :c:func:`Py_EncodeLocale` now use the UTF-8 1731encoding when the :ref:`UTF-8 mode <whatsnew37-pep540>` is enabled. 1732(Contributed by Victor Stinner in :issue:`29240`.) 1733 1734:c:func:`PyUnicode_DecodeLocaleAndSize` and :c:func:`PyUnicode_EncodeLocale` 1735now use the current locale encoding for ``surrogateescape`` error handler. 1736(Contributed by Victor Stinner in :issue:`29240`.) 1737 1738The *start* and *end* parameters of :c:func:`PyUnicode_FindChar` are 1739now adjusted to behave like string slices. 1740(Contributed by Xiang Zhang in :issue:`28822`.) 1741 1742 1743Build Changes 1744============= 1745 1746Support for building ``--without-threads`` has been removed. The 1747:mod:`threading` module is now always available. 1748(Contributed by Antoine Pitrou in :issue:`31370`.). 1749 1750A full copy of libffi is no longer bundled for use when building the 1751:mod:`_ctypes <ctypes>` module on non-OSX UNIX platforms. An installed copy 1752of libffi is now required when building ``_ctypes`` on such platforms. 1753(Contributed by Zachary Ware in :issue:`27979`.) 1754 1755The Windows build process no longer depends on Subversion to pull in external 1756sources, a Python script is used to download zipfiles from GitHub instead. 1757If Python 3.6 is not found on the system (via ``py -3.6``), NuGet is used to 1758download a copy of 32-bit Python for this purpose. (Contributed by Zachary 1759Ware in :issue:`30450`.) 1760 1761The :mod:`ssl` module requires OpenSSL 1.0.2 or 1.1 compatible libssl. 1762OpenSSL 1.0.1 has reached end of lifetime on 2016-12-31 and is no longer 1763supported. LibreSSL is temporarily not supported as well. LibreSSL releases 1764up to version 2.6.4 are missing required OpenSSL 1.0.2 APIs. 1765 1766 1767.. _whatsnew37-perf: 1768 1769Optimizations 1770============= 1771 1772The overhead of calling many methods of various standard library classes 1773implemented in C has been significantly reduced by porting more code 1774to use the ``METH_FASTCALL`` convention. 1775(Contributed by Victor Stinner in :issue:`29300`, :issue:`29507`, 1776:issue:`29452`, and :issue:`29286`.) 1777 1778Various optimizations have reduced Python startup time by 10% on Linux and 1779up to 30% on macOS. 1780(Contributed by Victor Stinner, INADA Naoki in :issue:`29585`, and 1781Ivan Levkivskyi in :issue:`31333`.) 1782 1783Method calls are now up to 20% faster due to the bytecode changes which 1784avoid creating bound method instances. 1785(Contributed by Yury Selivanov and INADA Naoki in :issue:`26110`.) 1786 1787.. _whatsnew37-asyncio-perf: 1788 1789The :mod:`asyncio` module received a number of notable optimizations for 1790commonly used functions: 1791 1792* The :func:`asyncio.get_event_loop` function has been reimplemented in C to 1793 make it up to 15 times faster. 1794 (Contributed by Yury Selivanov in :issue:`32296`.) 1795 1796* :class:`asyncio.Future` callback management has been optimized. 1797 (Contributed by Yury Selivanov in :issue:`32348`.) 1798 1799* :func:`asyncio.gather` is now up to 15% faster. 1800 (Contributed by Yury Selivanov in :issue:`32355`.) 1801 1802* :func:`asyncio.sleep` is now up to 2 times faster when the *delay* 1803 argument is zero or negative. 1804 (Contributed by Andrew Svetlov in :issue:`32351`.) 1805 1806* The performance overhead of asyncio debug mode has been reduced. 1807 (Contributed by Antoine Pitrou in :issue:`31970`.) 1808 1809As a result of :ref:`PEP 560 work <whatsnew37-pep560>`, the import time 1810of :mod:`typing` has been reduced by a factor of 7, and many typing operations 1811are now faster. 1812(Contributed by Ivan Levkivskyi in :issue:`32226`.) 1813 1814:func:`sorted` and :meth:`list.sort` have been optimized for common cases 1815to be up to 40-75% faster. 1816(Contributed by Elliot Gorokhovsky in :issue:`28685`.) 1817 1818:meth:`dict.copy` is now up to 5.5 times faster. 1819(Contributed by Yury Selivanov in :issue:`31179`.) 1820 1821:func:`hasattr` and :func:`getattr` are now about 4 times faster when 1822*name* is not found and *obj* does not override :meth:`object.__getattr__` 1823or :meth:`object.__getattribute__`. 1824(Contributed by INADA Naoki in :issue:`32544`.) 1825 1826Searching for certain Unicode characters (like Ukrainian capital "Є") 1827in a string was up to 25 times slower than searching for other characters. 1828It is now only 3 times slower in the worst case. 1829(Contributed by Serhiy Storchaka in :issue:`24821`.) 1830 1831The :func:`collections.namedtuple` factory has been reimplemented to 1832make the creation of named tuples 4 to 6 times faster. 1833(Contributed by Jelle Zijlstra with further improvements by INADA Naoki, 1834Serhiy Storchaka, and Raymond Hettinger in :issue:`28638`.) 1835 1836:meth:`date.fromordinal` and :meth:`date.fromtimestamp` are now up to 183730% faster in the common case. 1838(Contributed by Paul Ganssle in :issue:`32403`.) 1839 1840The :func:`os.fwalk` function is now up to 2 times faster thanks to 1841the use of :func:`os.scandir`. 1842(Contributed by Serhiy Storchaka in :issue:`25996`.) 1843 1844The speed of the :func:`shutil.rmtree` function has been improved by 184520--40% thanks to the use of the :func:`os.scandir` function. 1846(Contributed by Serhiy Storchaka in :issue:`28564`.) 1847 1848Optimized case-insensitive matching and searching of :mod:`regular 1849expressions <re>`. Searching some patterns can now be up to 20 times faster. 1850(Contributed by Serhiy Storchaka in :issue:`30285`.) 1851 1852:func:`re.compile` now converts ``flags`` parameter to int object if 1853it is ``RegexFlag``. It is now as fast as Python 3.5, and faster than 1854Python 3.6 by about 10% depending on the pattern. 1855(Contributed by INADA Naoki in :issue:`31671`.) 1856 1857The :meth:`~selectors.BaseSelector.modify` methods of classes 1858:class:`selectors.EpollSelector`, :class:`selectors.PollSelector` 1859and :class:`selectors.DevpollSelector` may be around 10% faster under 1860heavy loads. (Contributed by Giampaolo Rodola' in :issue:`30014`) 1861 1862Constant folding has been moved from the peephole optimizer to the new AST 1863optimizer, which is able perform optimizations more consistently. 1864(Contributed by Eugene Toder and INADA Naoki in :issue:`29469` and 1865:issue:`11549`.) 1866 1867Most functions and methods in :mod:`abc` have been rewritten in C. 1868This makes creation of abstract base classes, and calling :func:`isinstance` 1869and :func:`issubclass` on them 1.5x faster. This also reduces Python 1870start-up time by up to 10%. (Contributed by Ivan Levkivskyi and INADA Naoki 1871in :issue:`31333`) 1872 1873Significant speed improvements to alternate constructors for 1874:class:`datetime.date` and :class:`datetime.datetime` by using fast-path 1875constructors when not constructing subclasses. (Contributed by Paul Ganssle 1876in :issue:`32403`) 1877 1878The speed of comparison of :class:`array.array` instances has been 1879improved considerably in certain cases. It is now from 10x to 70x faster 1880when comparing arrays holding values of the same integer type. 1881(Contributed by Adrian Wielgosik in :issue:`24700`.) 1882 1883The :func:`math.erf` and :func:`math.erfc` functions now use the (faster) 1884C library implementation on most platforms. 1885(Contributed by Serhiy Storchaka in :issue:`26121`.) 1886 1887 1888Other CPython Implementation Changes 1889==================================== 1890 1891* Trace hooks may now opt out of receiving the ``line`` and opt into receiving 1892 the ``opcode`` events from the interpreter by setting the corresponding new 1893 ``f_trace_lines`` and ``f_trace_opcodes`` attributes on the 1894 frame being traced. (Contributed by Nick Coghlan in :issue:`31344`.) 1895 1896* Fixed some consistency problems with namespace package module attributes. 1897 Namespace module objects now have an ``__file__`` that is set to ``None`` 1898 (previously unset), and their ``__spec__.origin`` is also set to ``None`` 1899 (previously the string ``"namespace"``). See :issue:`32305`. Also, the 1900 namespace module object's ``__spec__.loader`` is set to the same value as 1901 ``__loader__`` (previously, the former was set to ``None``). See 1902 :issue:`32303`. 1903 1904* The :func:`locals` dictionary now displays in the lexical order that 1905 variables were defined. Previously, the order was undefined. 1906 (Contributed by Raymond Hettinger in :issue:`32690`.) 1907 1908* The :mod:`distutils` ``upload`` command no longer tries to change CR 1909 end-of-line characters to CRLF. This fixes a corruption issue with sdists 1910 that ended with a byte equivalent to CR. 1911 (Contributed by Bo Bayles in :issue:`32304`.) 1912 1913 1914Deprecated Python Behavior 1915========================== 1916 1917Yield expressions (both ``yield`` and ``yield from`` clauses) are now deprecated 1918in comprehensions and generator expressions (aside from the iterable expression 1919in the leftmost :keyword:`!for` clause). This ensures that comprehensions 1920always immediately return a container of the appropriate type (rather than 1921potentially returning a :term:`generator iterator` object), while generator 1922expressions won't attempt to interleave their implicit output with the output 1923from any explicit yield expressions. In Python 3.7, such expressions emit 1924:exc:`DeprecationWarning` when compiled, in Python 3.8 this will be a 1925:exc:`SyntaxError`. 1926(Contributed by Serhiy Storchaka in :issue:`10544`.) 1927 1928Returning a subclass of :class:`complex` from :meth:`object.__complex__` is 1929deprecated and will be an error in future Python versions. This makes 1930``__complex__()`` consistent with :meth:`object.__int__` and 1931:meth:`object.__float__`. 1932(Contributed by Serhiy Storchaka in :issue:`28894`.) 1933 1934 1935 1936Deprecated Python modules, functions and methods 1937================================================ 1938 1939aifc 1940---- 1941 1942:func:`aifc.openfp` has been deprecated and will be removed in Python 3.9. 1943Use :func:`aifc.open` instead. 1944(Contributed by Brian Curtin in :issue:`31985`.) 1945 1946 1947.. _whatsnew37-asyncio-deprecated: 1948 1949asyncio 1950------- 1951 1952Support for directly ``await``-ing instances of :class:`asyncio.Lock` and 1953other asyncio synchronization primitives has been deprecated. An 1954asynchronous context manager must be used in order to acquire and release 1955the synchronization resource. 1956(Contributed by Andrew Svetlov in :issue:`32253`.) 1957 1958The :meth:`asyncio.Task.current_task` and :meth:`asyncio.Task.all_tasks` 1959methods have been deprecated. 1960(Contributed by Andrew Svetlov in :issue:`32250`.) 1961 1962 1963collections 1964----------- 1965 1966In Python 3.8, the abstract base classes in :mod:`collections.abc` will no 1967longer be exposed in the regular :mod:`collections` module. This will help 1968create a clearer distinction between the concrete classes and the abstract 1969base classes. 1970(Contributed by Serhiy Storchaka in :issue:`25988`.) 1971 1972 1973dbm 1974--- 1975 1976:mod:`dbm.dumb` now supports reading read-only files and no longer writes the 1977index file when it is not changed. A deprecation warning is now emitted 1978if the index file is missing and recreated in the ``'r'`` and ``'w'`` 1979modes (this will be an error in future Python releases). 1980(Contributed by Serhiy Storchaka in :issue:`28847`.) 1981 1982 1983enum 1984---- 1985 1986In Python 3.8, attempting to check for non-Enum objects in :class:`Enum` 1987classes will raise a :exc:`TypeError` (e.g. ``1 in Color``); similarly, 1988attempting to check for non-Flag objects in a :class:`Flag` member will 1989raise :exc:`TypeError` (e.g. ``1 in Perm.RW``); currently, both operations 1990return :const:`False` instead. 1991(Contributed by Ethan Furman in :issue:`33217`.) 1992 1993 1994gettext 1995------- 1996 1997Using non-integer value for selecting a plural form in :mod:`gettext` is 1998now deprecated. It never correctly worked. (Contributed by Serhiy Storchaka 1999in :issue:`28692`.) 2000 2001 2002importlib 2003--------- 2004 2005Methods 2006:meth:`MetaPathFinder.find_module() <importlib.abc.MetaPathFinder.find_module>` 2007(replaced by 2008:meth:`MetaPathFinder.find_spec() <importlib.abc.MetaPathFinder.find_spec>`) 2009and 2010:meth:`PathEntryFinder.find_loader() <importlib.abc.PathEntryFinder.find_loader>` 2011(replaced by 2012:meth:`PathEntryFinder.find_spec() <importlib.abc.PathEntryFinder.find_spec>`) 2013both deprecated in Python 3.4 now emit :exc:`DeprecationWarning`. 2014(Contributed by Matthias Bussonnier in :issue:`29576`) 2015 2016The :class:`importlib.abc.ResourceLoader` ABC has been deprecated in 2017favour of :class:`importlib.abc.ResourceReader`. 2018 2019 2020locale 2021------ 2022 2023:func:`locale.format` has been deprecated, use :meth:`locale.format_string` 2024instead. (Contributed by Garvit in :issue:`10379`.) 2025 2026 2027macpath 2028------- 2029 2030The :mod:`macpath` is now deprecated and will be removed in Python 3.8. 2031(Contributed by Chi Hsuan Yen in :issue:`9850`.) 2032 2033 2034threading 2035--------- 2036 2037:mod:`dummy_threading` and :mod:`_dummy_thread` have been deprecated. It is 2038no longer possible to build Python with threading disabled. 2039Use :mod:`threading` instead. 2040(Contributed by Antoine Pitrou in :issue:`31370`.) 2041 2042 2043socket 2044------ 2045 2046The silent argument value truncation in :func:`socket.htons` and 2047:func:`socket.ntohs` has been deprecated. In future versions of Python, 2048if the passed argument is larger than 16 bits, an exception will be raised. 2049(Contributed by Oren Milman in :issue:`28332`.) 2050 2051 2052ssl 2053--- 2054 2055:func:`ssl.wrap_socket` is deprecated. Use 2056:meth:`ssl.SSLContext.wrap_socket` instead. 2057(Contributed by Christian Heimes in :issue:`28124`.) 2058 2059 2060sunau 2061----- 2062 2063:func:`sunau.openfp` has been deprecated and will be removed in Python 3.9. 2064Use :func:`sunau.open` instead. 2065(Contributed by Brian Curtin in :issue:`31985`.) 2066 2067 2068sys 2069--- 2070 2071Deprecated :func:`sys.set_coroutine_wrapper` and 2072:func:`sys.get_coroutine_wrapper`. 2073 2074The undocumented ``sys.callstats()`` function has been deprecated and 2075will be removed in a future Python version. 2076(Contributed by Victor Stinner in :issue:`28799`.) 2077 2078 2079wave 2080---- 2081 2082:func:`wave.openfp` has been deprecated and will be removed in Python 3.9. 2083Use :func:`wave.open` instead. 2084(Contributed by Brian Curtin in :issue:`31985`.) 2085 2086 2087Deprecated functions and types of the C API 2088=========================================== 2089 2090Function :c:func:`PySlice_GetIndicesEx` is deprecated and replaced with 2091a macro if ``Py_LIMITED_API`` is not set or set to a value in the range 2092between ``0x03050400`` and ``0x03060000`` (not inclusive), or is ``0x03060100`` 2093or higher. (Contributed by Serhiy Storchaka in :issue:`27867`.) 2094 2095:c:func:`PyOS_AfterFork` has been deprecated. Use :c:func:`PyOS_BeforeFork`, 2096:c:func:`PyOS_AfterFork_Parent` or :c:func:`PyOS_AfterFork_Child()` instead. 2097(Contributed by Antoine Pitrou in :issue:`16500`.) 2098 2099 2100.. _37-platform-support-removals: 2101 2102Platform Support Removals 2103========================= 2104 2105* FreeBSD 9 and older are no longer officially supported. 2106* For full Unicode support, including within extension modules, \*nix platforms 2107 are now expected to provide at least one of ``C.UTF-8`` (full locale), 2108 ``C.utf8`` (full locale) or ``UTF-8`` (``LC_CTYPE``-only locale) as an 2109 alternative to the legacy ``ASCII``-based ``C`` locale. 2110* OpenSSL 0.9.8 and 1.0.1 are no longer supported, which means building CPython 2111 3.7 with SSL/TLS support on older platforms still using these versions 2112 requires custom build options that link to a more recent version of OpenSSL. 2113 2114 Notably, this issue affects the Debian 8 (aka "jessie") and Ubuntu 14.04 2115 (aka "Trusty") LTS Linux distributions, as they still use OpenSSL 1.0.1 by 2116 default. 2117 2118 Debian 9 ("stretch") and Ubuntu 16.04 ("xenial"), as well as recent releases 2119 of other LTS Linux releases (e.g. RHEL/CentOS 7.5, SLES 12-SP3), use OpenSSL 2120 1.0.2 or later, and remain supported in the default build configuration. 2121 2122 CPython's own :source:`CI configuration file <.travis.yml>` provides an 2123 example of using the SSL 2124 :source:`compatibility testing infrastructure <Tools/ssl/multissltests.py>` in 2125 CPython's test suite to build and link against OpenSSL 1.1.0 rather than an 2126 outdated system provided OpenSSL. 2127 2128 2129API and Feature Removals 2130======================== 2131 2132The following features and APIs have been removed from Python 3.7: 2133 2134* The ``os.stat_float_times()`` function has been removed. It was introduced in 2135 Python 2.3 for backward compatibility with Python 2.2, and was deprecated 2136 since Python 3.1. 2137 2138* Unknown escapes consisting of ``'\'`` and an ASCII letter in replacement 2139 templates for :func:`re.sub` were deprecated in Python 3.5, and will now 2140 cause an error. 2141 2142* Removed support of the *exclude* argument in :meth:`tarfile.TarFile.add`. 2143 It was deprecated in Python 2.7 and 3.2. Use the *filter* argument instead. 2144 2145* The ``splitunc()`` function in the :mod:`ntpath` module was deprecated in 2146 Python 3.1, and has now been removed. Use the :func:`~os.path.splitdrive` 2147 function instead. 2148 2149* :func:`collections.namedtuple` no longer supports the *verbose* parameter 2150 or ``_source`` attribute which showed the generated source code for the 2151 named tuple class. This was part of an optimization designed to speed-up 2152 class creation. (Contributed by Jelle Zijlstra with further improvements 2153 by INADA Naoki, Serhiy Storchaka, and Raymond Hettinger in :issue:`28638`.) 2154 2155* Functions :func:`bool`, :func:`float`, :func:`list` and :func:`tuple` no 2156 longer take keyword arguments. The first argument of :func:`int` can now 2157 be passed only as positional argument. 2158 2159* Removed previously deprecated in Python 2.4 classes ``Plist``, ``Dict`` and 2160 ``_InternalDict`` in the :mod:`plistlib` module. Dict values in the result 2161 of functions :func:`~plistlib.readPlist` and 2162 :func:`~plistlib.readPlistFromBytes` are now normal dicts. You no longer 2163 can use attribute access to access items of these dictionaries. 2164 2165* The ``asyncio.windows_utils.socketpair()`` function has been 2166 removed. Use the :func:`socket.socketpair` function instead, 2167 it is available on all platforms since Python 3.5. 2168 ``asyncio.windows_utils.socketpair`` was just an alias to 2169 ``socket.socketpair`` on Python 3.5 and newer. 2170 2171* :mod:`asyncio` no longer exports the :mod:`selectors` and 2172 :mod:`_overlapped` modules as ``asyncio.selectors`` and 2173 ``asyncio._overlapped``. Replace ``from asyncio import selectors`` with 2174 ``import selectors``. 2175 2176* Direct instantiation of :class:`ssl.SSLSocket` and :class:`ssl.SSLObject` 2177 objects is now prohibited. The constructors were never documented, tested, 2178 or designed as public constructors. Users were supposed to use 2179 :func:`ssl.wrap_socket` or :class:`ssl.SSLContext`. 2180 (Contributed by Christian Heimes in :issue:`32951`.) 2181 2182* The unused :mod:`distutils` ``install_misc`` command has been removed. 2183 (Contributed by Eric N. Vander Weele in :issue:`29218`.) 2184 2185 2186Module Removals 2187=============== 2188 2189The ``fpectl`` module has been removed. It was never enabled by 2190default, never worked correctly on x86-64, and it changed the Python 2191ABI in ways that caused unexpected breakage of C extensions. 2192(Contributed by Nathaniel J. Smith in :issue:`29137`.) 2193 2194 2195Windows-only Changes 2196==================== 2197 2198The python launcher, (py.exe), can accept 32 & 64 bit specifiers **without** 2199having to specify a minor version as well. So ``py -3-32`` and ``py -3-64`` 2200become valid as well as ``py -3.7-32``, also the -*m*-64 and -*m.n*-64 forms 2201are now accepted to force 64 bit python even if 32 bit would have otherwise 2202been used. If the specified version is not available py.exe will error exit. 2203(Contributed by Steve Barnes in :issue:`30291`.) 2204 2205The launcher can be run as ``py -0`` to produce a list of the installed pythons, 2206*with default marked with an asterisk*. Running ``py -0p`` will include the paths. 2207If py is run with a version specifier that cannot be matched it will also print 2208the *short form* list of available specifiers. 2209(Contributed by Steve Barnes in :issue:`30362`.) 2210 2211 2212.. _porting-to-python-37: 2213 2214Porting to Python 3.7 2215===================== 2216 2217This section lists previously described changes and other bugfixes 2218that may require changes to your code. 2219 2220 2221Changes in Python Behavior 2222-------------------------- 2223 2224* :keyword:`async` and :keyword:`await` names are now reserved keywords. 2225 Code using these names as identifiers will now raise a :exc:`SyntaxError`. 2226 (Contributed by Jelle Zijlstra in :issue:`30406`.) 2227 2228* :pep:`479` is enabled for all code in Python 3.7, meaning that 2229 :exc:`StopIteration` exceptions raised directly or indirectly in 2230 coroutines and generators are transformed into :exc:`RuntimeError` 2231 exceptions. 2232 (Contributed by Yury Selivanov in :issue:`32670`.) 2233 2234* :meth:`object.__aiter__` methods can no longer be declared as 2235 asynchronous. (Contributed by Yury Selivanov in :issue:`31709`.) 2236 2237* Due to an oversight, earlier Python versions erroneously accepted the 2238 following syntax:: 2239 2240 f(1 for x in [1],) 2241 2242 class C(1 for x in [1]): 2243 pass 2244 2245 Python 3.7 now correctly raises a :exc:`SyntaxError`, as a generator 2246 expression always needs to be directly inside a set of parentheses 2247 and cannot have a comma on either side, and the duplication of the 2248 parentheses can be omitted only on calls. 2249 (Contributed by Serhiy Storchaka in :issue:`32012` and :issue:`32023`.) 2250 2251* When using the :option:`-m` switch, the initial working directory is now added 2252 to :data:`sys.path`, rather than an empty string (which dynamically denoted 2253 the current working directory at the time of each import). Any programs that 2254 are checking for the empty string, or otherwise relying on the previous 2255 behaviour, will need to be updated accordingly (e.g. by also checking for 2256 ``os.getcwd()`` or ``os.path.dirname(__main__.__file__)``, depending on why 2257 the code was checking for the empty string in the first place). 2258 2259 2260Changes in the Python API 2261------------------------- 2262 2263* :meth:`socketserver.ThreadingMixIn.server_close` now waits until all 2264 non-daemon threads complete. Set the new 2265 :attr:`socketserver.ThreadingMixIn.block_on_close` class attribute to 2266 ``False`` to get the pre-3.7 behaviour. 2267 (Contributed by Victor Stinner in :issue:`31233` and :issue:`33540`.) 2268 2269* :meth:`socketserver.ForkingMixIn.server_close` now waits until all 2270 child processes complete. Set the new 2271 :attr:`socketserver.ForkingMixIn.block_on_close` class attribute to ``False`` 2272 to get the pre-3.7 behaviour. 2273 (Contributed by Victor Stinner in :issue:`31151` and :issue:`33540`.) 2274 2275* The :func:`locale.localeconv` function now temporarily sets the ``LC_CTYPE`` 2276 locale to the value of ``LC_NUMERIC`` in some cases. 2277 (Contributed by Victor Stinner in :issue:`31900`.) 2278 2279* :meth:`pkgutil.walk_packages` now raises a :exc:`ValueError` if *path* is 2280 a string. Previously an empty list was returned. 2281 (Contributed by Sanyam Khurana in :issue:`24744`.) 2282 2283* A format string argument for :meth:`string.Formatter.format` 2284 is now :ref:`positional-only <positional-only_parameter>`. 2285 Passing it as a keyword argument was deprecated in Python 3.5. (Contributed 2286 by Serhiy Storchaka in :issue:`29193`.) 2287 2288* Attributes :attr:`~http.cookies.Morsel.key`, 2289 :attr:`~http.cookies.Morsel.value` and 2290 :attr:`~http.cookies.Morsel.coded_value` of class 2291 :class:`http.cookies.Morsel` are now read-only. 2292 Assigning to them was deprecated in Python 3.5. 2293 Use the :meth:`~http.cookies.Morsel.set` method for setting them. 2294 (Contributed by Serhiy Storchaka in :issue:`29192`.) 2295 2296* The *mode* argument of :func:`os.makedirs` no longer affects the file 2297 permission bits of newly-created intermediate-level directories. 2298 To set their file permission bits you can set the umask before invoking 2299 ``makedirs()``. 2300 (Contributed by Serhiy Storchaka in :issue:`19930`.) 2301 2302* The :attr:`struct.Struct.format` type is now :class:`str` instead of 2303 :class:`bytes`. (Contributed by Victor Stinner in :issue:`21071`.) 2304 2305* :func:`~cgi.parse_multipart` now accepts the *encoding* and *errors* 2306 arguments and returns the same results as 2307 :class:`~FieldStorage`: for non-file fields, the value associated to a key 2308 is a list of strings, not bytes. 2309 (Contributed by Pierre Quentel in :issue:`29979`.) 2310 2311* Due to internal changes in :mod:`socket`, calling :func:`socket.fromshare` 2312 on a socket created by :func:`socket.share <socket.socket.share>` in older 2313 Python versions is not supported. 2314 2315* ``repr`` for :exc:`BaseException` has changed to not include the trailing 2316 comma. Most exceptions are affected by this change. 2317 (Contributed by Serhiy Storchaka in :issue:`30399`.) 2318 2319* ``repr`` for :class:`datetime.timedelta` has changed to include the keyword 2320 arguments in the output. (Contributed by Utkarsh Upadhyay in :issue:`30302`.) 2321 2322* Because :func:`shutil.rmtree` is now implemented using the :func:`os.scandir` 2323 function, the user specified handler *onerror* is now called with the first 2324 argument ``os.scandir`` instead of ``os.listdir`` when listing the directory 2325 is failed. 2326 2327* Support for nested sets and set operations in regular expressions as in 2328 `Unicode Technical Standard #18`_ might be added in the future. This would 2329 change the syntax. To facilitate this future change a :exc:`FutureWarning` 2330 will be raised in ambiguous cases for the time being. 2331 That include sets starting with a literal ``'['`` or containing literal 2332 character sequences ``'--'``, ``'&&'``, ``'~~'``, and ``'||'``. To 2333 avoid a warning, escape them with a backslash. 2334 (Contributed by Serhiy Storchaka in :issue:`30349`.) 2335 2336 .. _Unicode Technical Standard #18: https://unicode.org/reports/tr18/ 2337 2338* The result of splitting a string on a :mod:`regular expression <re>` 2339 that could match an empty string has been changed. For example 2340 splitting on ``r'\s*'`` will now split not only on whitespaces as it 2341 did previously, but also on empty strings before all non-whitespace 2342 characters and just before the end of the string. 2343 The previous behavior can be restored by changing the pattern 2344 to ``r'\s+'``. A :exc:`FutureWarning` was emitted for such patterns since 2345 Python 3.5. 2346 2347 For patterns that match both empty and non-empty strings, the result of 2348 searching for all matches may also be changed in other cases. For example 2349 in the string ``'a\n\n'``, the pattern ``r'(?m)^\s*?$'`` will not only 2350 match empty strings at positions 2 and 3, but also the string ``'\n'`` at 2351 positions 2--3. To match only blank lines, the pattern should be rewritten 2352 as ``r'(?m)^[^\S\n]*$'``. 2353 2354 :func:`re.sub()` now replaces empty matches adjacent to a previous 2355 non-empty match. For example ``re.sub('x*', '-', 'abxd')`` returns now 2356 ``'-a-b--d-'`` instead of ``'-a-b-d-'`` (the first minus between 'b' and 2357 'd' replaces 'x', and the second minus replaces an empty string between 2358 'x' and 'd'). 2359 2360 (Contributed by Serhiy Storchaka in :issue:`25054` and :issue:`32308`.) 2361 2362* Change :func:`re.escape` to only escape regex special characters instead 2363 of escaping all characters other than ASCII letters, numbers, and ``'_'``. 2364 (Contributed by Serhiy Storchaka in :issue:`29995`.) 2365 2366* :class:`tracemalloc.Traceback` frames are now sorted from oldest to most 2367 recent to be more consistent with :mod:`traceback`. 2368 (Contributed by Jesse Bakker in :issue:`32121`.) 2369 2370* On OSes that support :const:`socket.SOCK_NONBLOCK` or 2371 :const:`socket.SOCK_CLOEXEC` bit flags, the 2372 :attr:`socket.type <socket.socket.type>` no longer has them applied. 2373 Therefore, checks like ``if sock.type == socket.SOCK_STREAM`` 2374 work as expected on all platforms. 2375 (Contributed by Yury Selivanov in :issue:`32331`.) 2376 2377* On Windows the default for the *close_fds* argument of 2378 :class:`subprocess.Popen` was changed from :const:`False` to :const:`True` 2379 when redirecting the standard handles. If you previously depended on handles 2380 being inherited when using :class:`subprocess.Popen` with standard io 2381 redirection, you will have to pass ``close_fds=False`` to preserve the 2382 previous behaviour, or use 2383 :attr:`STARTUPINFO.lpAttributeList <subprocess.STARTUPINFO.lpAttributeList>`. 2384 2385* :meth:`importlib.machinery.PathFinder.invalidate_caches` -- which implicitly 2386 affects :func:`importlib.invalidate_caches` -- now deletes entries 2387 in :data:`sys.path_importer_cache` which are set to ``None``. 2388 (Contributed by Brett Cannon in :issue:`33169`.) 2389 2390* In :mod:`asyncio`, 2391 :meth:`loop.sock_recv() <asyncio.loop.sock_recv>`, 2392 :meth:`loop.sock_sendall() <asyncio.loop.sock_sendall>`, 2393 :meth:`loop.sock_accept() <asyncio.loop.sock_accept>`, 2394 :meth:`loop.getaddrinfo() <asyncio.loop.getaddrinfo>`, 2395 :meth:`loop.getnameinfo() <asyncio.loop.getnameinfo>` 2396 have been changed to be proper coroutine methods to match their 2397 documentation. Previously, these methods returned :class:`asyncio.Future` 2398 instances. 2399 (Contributed by Yury Selivanov in :issue:`32327`.) 2400 2401* :attr:`asyncio.Server.sockets` now returns a copy of the internal list 2402 of server sockets, instead of returning it directly. 2403 (Contributed by Yury Selivanov in :issue:`32662`.) 2404 2405* :attr:`Struct.format <struct.Struct.format>` is now a :class:`str` instance 2406 instead of a :class:`bytes` instance. 2407 (Contributed by Victor Stinner in :issue:`21071`.) 2408 2409* :mod:`argparse` subparsers can now be made mandatory by passing ``required=True`` 2410 to :meth:`ArgumentParser.add_subparsers() <argparse.ArgumentParser.add_subparsers>`. 2411 (Contributed by Anthony Sottile in :issue:`26510`.) 2412 2413* :meth:`ast.literal_eval()` is now stricter. Addition and subtraction of 2414 arbitrary numbers are no longer allowed. 2415 (Contributed by Serhiy Storchaka in :issue:`31778`.) 2416 2417* :meth:`Calendar.itermonthdates <calendar.Calendar.itermonthdates>` 2418 will now consistently raise an exception when a date falls outside of the 2419 ``0001-01-01`` through ``9999-12-31`` range. To support applications that 2420 cannot tolerate such exceptions, the new 2421 :meth:`Calendar.itermonthdays3 <calendar.Calendar.itermonthdays3>` and 2422 :meth:`Calendar.itermonthdays4 <calendar.Calendar.itermonthdays4>` can be used. 2423 The new methods return tuples and are not restricted by the range supported by 2424 :class:`datetime.date`. 2425 (Contributed by Alexander Belopolsky in :issue:`28292`.) 2426 2427* :class:`collections.ChainMap` now preserves the order of the underlying 2428 mappings. (Contributed by Raymond Hettinger in :issue:`32792`.) 2429 2430* The ``submit()`` method of :class:`concurrent.futures.ThreadPoolExecutor` 2431 and :class:`concurrent.futures.ProcessPoolExecutor` now raises 2432 a :exc:`RuntimeError` if called during interpreter shutdown. 2433 (Contributed by Mark Nemec in :issue:`33097`.) 2434 2435* The :class:`configparser.ConfigParser` constructor now uses ``read_dict()`` 2436 to process the default values, making its behavior consistent with the 2437 rest of the parser. Non-string keys and values in the defaults 2438 dictionary are now being implicitly converted to strings. 2439 (Contributed by James Tocknell in :issue:`23835`.) 2440 2441* Several undocumented internal imports were removed. 2442 One example is that ``os.errno`` is no longer available; use ``import errno`` 2443 directly instead. 2444 Note that such undocumented internal imports may be removed any time without 2445 notice, even in micro version releases. 2446 2447 2448Changes in the C API 2449-------------------- 2450 2451The function :c:func:`PySlice_GetIndicesEx` is considered unsafe for 2452resizable sequences. If the slice indices are not instances of :class:`int`, 2453but objects that implement the :meth:`!__index__` method, the sequence can be 2454resized after passing its length to :c:func:`!PySlice_GetIndicesEx`. This 2455can lead to returning indices out of the length of the sequence. For 2456avoiding possible problems use new functions :c:func:`PySlice_Unpack` and 2457:c:func:`PySlice_AdjustIndices`. 2458(Contributed by Serhiy Storchaka in :issue:`27867`.) 2459 2460 2461CPython bytecode changes 2462------------------------ 2463 2464There are two new opcodes: :opcode:`LOAD_METHOD` and :opcode:`CALL_METHOD`. 2465(Contributed by Yury Selivanov and INADA Naoki in :issue:`26110`.) 2466 2467The :opcode:`STORE_ANNOTATION` opcode has been removed. 2468(Contributed by Mark Shannon in :issue:`32550`.) 2469 2470 2471Windows-only Changes 2472-------------------- 2473 2474The file used to override :data:`sys.path` is now called 2475``<python-executable>._pth`` instead of ``'sys.path'``. 2476See :ref:`finding_modules` for more information. 2477(Contributed by Steve Dower in :issue:`28137`.) 2478 2479 2480Other CPython implementation changes 2481------------------------------------ 2482 2483In preparation for potential future changes to the public CPython runtime 2484initialization API (see :pep:`432` for an initial, but somewhat outdated, 2485draft), CPython's internal startup 2486and configuration management logic has been significantly refactored. While 2487these updates are intended to be entirely transparent to both embedding 2488applications and users of the regular CPython CLI, they're being mentioned 2489here as the refactoring changes the internal order of various operations 2490during interpreter startup, and hence may uncover previously latent defects, 2491either in embedding applications, or in CPython itself. 2492(Initially contributed by Nick Coghlan and Eric Snow as part of 2493:issue:`22257`, and further updated by Nick, Eric, and Victor Stinner in a 2494number of other issues). Some known details affected: 2495 2496* :c:func:`PySys_AddWarnOptionUnicode` is not currently usable by embedding 2497 applications due to the requirement to create a Unicode object prior to 2498 calling `Py_Initialize`. Use :c:func:`PySys_AddWarnOption` instead. 2499 2500* warnings filters added by an embedding application with 2501 :c:func:`PySys_AddWarnOption` should now more consistently take precedence 2502 over the default filters set by the interpreter 2503 2504Due to changes in the way the default warnings filters are configured, 2505setting :c:data:`Py_BytesWarningFlag` to a value greater than one is no longer 2506sufficient to both emit :exc:`BytesWarning` messages and have them converted 2507to exceptions. Instead, the flag must be set (to cause the warnings to be 2508emitted in the first place), and an explicit ``error::BytesWarning`` 2509warnings filter added to convert them to exceptions. 2510 2511Due to a change in the way docstrings are handled by the compiler, the 2512implicit ``return None`` in a function body consisting solely of a docstring 2513is now marked as occurring on the same line as the docstring, not on the 2514function's header line. 2515 2516The current exception state has been moved from the frame object to the co-routine. 2517This simplified the interpreter and fixed a couple of obscure bugs caused by 2518having swap exception state when entering or exiting a generator. 2519(Contributed by Mark Shannon in :issue:`25612`.) 2520 2521Notable changes in Python 3.7.1 2522=============================== 2523 2524Starting in 3.7.1, :c:func:`Py_Initialize` now consistently reads and respects 2525all of the same environment settings as :c:func:`Py_Main` (in earlier Python 2526versions, it respected an ill-defined subset of those environment variables, 2527while in Python 3.7.0 it didn't read any of them due to :issue:`34247`). If 2528this behavior is unwanted, set :c:data:`Py_IgnoreEnvironmentFlag` to 1 before 2529calling :c:func:`Py_Initialize`. 2530 2531In 3.7.1 the C API for Context Variables 2532:ref:`was updated <contextvarsobjects_pointertype_change>` to use 2533:c:type:`PyObject` pointers. See also :issue:`34762`. 2534 2535In 3.7.1 the :mod:`tokenize` module now implicitly emits a ``NEWLINE`` token 2536when provided with input that does not have a trailing new line. This behavior 2537now matches what the C tokenizer does internally. 2538(Contributed by Ammar Askar in :issue:`33899`.) 2539 2540Notable changes in Python 3.7.2 2541=============================== 2542 2543In 3.7.2, :mod:`venv` on Windows no longer copies the original binaries, but 2544creates redirector scripts named ``python.exe`` and ``pythonw.exe`` instead. 2545This resolves a long standing issue where all virtual environments would have 2546to be upgraded or recreated with each Python update. However, note that this 2547release will still require recreation of virtual environments in order to get 2548the new scripts. 2549 2550Notable changes in Python 3.7.6 2551=============================== 2552 2553Due to significant security concerns, the *reuse_address* parameter of 2554:meth:`asyncio.loop.create_datagram_endpoint` is no longer supported. This is 2555because of the behavior of the socket option ``SO_REUSEADDR`` in UDP. For more 2556details, see the documentation for ``loop.create_datagram_endpoint()``. 2557(Contributed by Kyle Stanley, Antoine Pitrou, and Yury Selivanov in 2558:issue:`37228`.) 2559