1**************************** 2 What's New In Python 3.7 3**************************** 4 5:Editor: Elvis Pranskevichus <elvis@magic.io> 6 7.. Rules for maintenance: 8 9 * Anyone can add text to this document. Do not spend very much time 10 on the wording of your changes, because your text will probably 11 get rewritten to some degree. 12 13 * The maintainer will go through Misc/NEWS periodically and add 14 changes; it's therefore more important to add your changes to 15 Misc/NEWS than to this file. 16 17 * This is not a complete list of every single change; completeness 18 is the purpose of Misc/NEWS. Some changes I consider too small 19 or esoteric to include. If such a change is added to the text, 20 I'll just remove it. (This is another reason you shouldn't spend 21 too much time on writing your addition.) 22 23 * If you want to draw your new text to the attention of the 24 maintainer, add 'XXX' to the beginning of the paragraph or 25 section. 26 27 * It's OK to just add a fragmentary note about a change. For 28 example: "XXX Describe the transmogrify() function added to the 29 socket module." The maintainer will research the change and 30 write the necessary text. 31 32 * You can comment out your additions if you like, but it's not 33 necessary (especially when a final release is some months away). 34 35 * Credit the author of a patch or bugfix. Just the name is 36 sufficient; the e-mail address isn't necessary. 37 38 * It's helpful to add the bug/patch number as a comment: 39 40 XXX Describe the transmogrify() function added to the socket 41 module. 42 (Contributed by P.Y. Developer in :issue:`12345`.) 43 44 This saves the maintainer the effort of going through the Git log 45 when researching a change. 46 47This article explains the new features in Python 3.7, compared to 3.6. 48Python 3.7 was released on June 27, 2018. 49For full details, see the :ref:`changelog <changelog>`. 50 51 52Summary -- Release Highlights 53============================= 54 55.. This section singles out the most important changes in Python 3.7. 56 Brevity is key. 57 58New syntax features: 59 60* :ref:`PEP 563 <whatsnew37-pep563>`, postponed evaluation of type annotations. 61 62Backwards incompatible syntax changes: 63 64* :keyword:`async` and :keyword:`await` are now reserved keywords. 65 66New library modules: 67 68* :mod:`contextvars`: :ref:`PEP 567 -- Context Variables <whatsnew37-pep567>` 69* :mod:`dataclasses`: :ref:`PEP 557 -- Data Classes <whatsnew37-pep557>` 70* :ref:`whatsnew37_importlib_resources` 71 72New built-in features: 73 74* :ref:`PEP 553 <whatsnew37-pep553>`, the new :func:`breakpoint` function. 75 76Python data model improvements: 77 78* :ref:`PEP 562 <whatsnew37-pep562>`, customization of access to 79 module attributes. 80 81* :ref:`PEP 560 <whatsnew37-pep560>`, core support for typing module and 82 generic types. 83 84* the insertion-order preservation nature of :ref:`dict <typesmapping>` 85 objects `has been declared`_ to be an official 86 part of the Python language spec. 87 88.. _has been declared: https://mail.python.org/pipermail/python-dev/2017-December/151283.html 89 90Significant improvements in the standard library: 91 92* The :mod:`asyncio` module has received new features, significant 93 :ref:`usability and performance improvements <whatsnew37_asyncio>`. 94 95* The :mod:`time` module gained support for 96 :ref:`functions with nanosecond resolution <whatsnew37-pep564>`. 97 98CPython implementation improvements: 99 100* Avoiding the use of ASCII as a default text encoding: 101 102 * :ref:`PEP 538 <whatsnew37-pep538>`, legacy C locale coercion 103 * :ref:`PEP 540 <whatsnew37-pep540>`, forced UTF-8 runtime mode 104* :ref:`PEP 552 <whatsnew37-pep552>`, deterministic .pycs 105* :ref:`New Python Development Mode <whatsnew37-devmode>` 106* :ref:`PEP 565 <whatsnew37-pep565>`, improved :exc:`DeprecationWarning` 107 handling 108 109C API improvements: 110 111* :ref:`PEP 539 <whatsnew37-pep539>`, new C API for thread-local storage 112 113Documentation improvements: 114 115* :ref:`PEP 545 <whatsnew37-pep545>`, Python documentation translations 116* New documentation translations: `Japanese <https://docs.python.org/ja/>`_, 117 `French <https://docs.python.org/fr/>`_, and 118 `Korean <https://docs.python.org/ko/>`_. 119 120This release features notable performance improvements in many areas. 121The :ref:`whatsnew37-perf` section lists them in detail. 122 123For a list of changes that may affect compatibility with previous Python 124releases please refer to the :ref:`porting-to-python-37` section. 125 126 127New Features 128============ 129 130.. _whatsnew37-pep563: 131 132PEP 563: Postponed Evaluation of Annotations 133-------------------------------------------- 134 135The advent of type hints in Python uncovered two glaring usability issues 136with the functionality of annotations added in :pep:`3107` and refined 137further in :pep:`526`: 138 139* annotations could only use names which were already available in the 140 current scope, in other words they didn't support forward references 141 of any kind; and 142 143* annotating source code had adverse effects on startup time of Python 144 programs. 145 146Both of these issues are fixed by postponing the evaluation of 147annotations. Instead of compiling code which executes expressions in 148annotations at their definition time, the compiler stores the annotation 149in a string form equivalent to the AST of the expression in question. 150If needed, annotations can be resolved at runtime using 151:func:`typing.get_type_hints`. In the common case where this is not 152required, the annotations are cheaper to store (since short strings 153are interned by the interpreter) and make startup time faster. 154 155Usability-wise, annotations now support forward references, making the 156following syntax valid:: 157 158 class C: 159 @classmethod 160 def from_string(cls, source: str) -> C: 161 ... 162 163 def validate_b(self, obj: B) -> bool: 164 ... 165 166 class B: 167 ... 168 169Since this change breaks compatibility, the new behavior needs to be enabled 170on a per-module basis in Python 3.7 using a :mod:`__future__` import:: 171 172 from __future__ import annotations 173 174It will become the default in Python 3.10. 175 176.. seealso:: 177 178 :pep:`563` -- Postponed evaluation of annotations 179 PEP written and implemented by Łukasz Langa. 180 181 182.. _whatsnew37-pep538: 183 184PEP 538: Legacy C Locale Coercion 185--------------------------------- 186 187An ongoing challenge within the Python 3 series has been determining a sensible 188default strategy for handling the "7-bit ASCII" text encoding assumption 189currently implied by the use of the default C or POSIX locale on non-Windows 190platforms. 191 192:pep:`538` updates the default interpreter command line interface to 193automatically coerce that locale to an available UTF-8 based locale as 194described in the documentation of the new :envvar:`PYTHONCOERCECLOCALE` 195environment variable. Automatically setting ``LC_CTYPE`` this way means that 196both the core interpreter and locale-aware C extensions (such as 197:mod:`readline`) will assume the use of UTF-8 as the default text encoding, 198rather than ASCII. 199 200The platform support definition in :pep:`11` has also been updated to limit 201full text handling support to suitably configured non-ASCII based locales. 202 203As part of this change, the default error handler for :data:`~sys.stdin` and 204:data:`~sys.stdout` is now ``surrogateescape`` (rather than ``strict``) when 205using any of the defined coercion target locales (currently ``C.UTF-8``, 206``C.utf8``, and ``UTF-8``). The default error handler for :data:`~sys.stderr` 207continues to be ``backslashreplace``, regardless of locale. 208 209Locale coercion is silent by default, but to assist in debugging potentially 210locale related integration problems, explicit warnings (emitted directly on 211:data:`~sys.stderr`) can be requested by setting ``PYTHONCOERCECLOCALE=warn``. 212This setting will also cause the Python runtime to emit a warning if the 213legacy C locale remains active when the core interpreter is initialized. 214 215While :pep:`538`'s locale coercion has the benefit of also affecting extension 216modules (such as GNU ``readline``), as well as child processes (including those 217running non-Python applications and older versions of Python), it has the 218downside of requiring that a suitable target locale be present on the running 219system. To better handle the case where no suitable target locale is available 220(as occurs on RHEL/CentOS 7, for example), Python 3.7 also implements 221:ref:`whatsnew37-pep540`. 222 223.. seealso:: 224 225 :pep:`538` -- Coercing the legacy C locale to a UTF-8 based locale 226 PEP written and implemented by Nick Coghlan. 227 228 229.. _whatsnew37-pep540: 230 231PEP 540: Forced UTF-8 Runtime Mode 232----------------------------------- 233 234The new :option:`-X` ``utf8`` command line option and :envvar:`PYTHONUTF8` 235environment variable can be used to enable the :ref:`Python UTF-8 Mode 236<utf8-mode>`. 237 238When in UTF-8 mode, CPython ignores the locale settings, and uses the 239UTF-8 encoding by default. The error handlers for :data:`sys.stdin` and 240:data:`sys.stdout` streams are set to ``surrogateescape``. 241 242The forced UTF-8 mode can be used to change the text handling behavior in 243an embedded Python interpreter without changing the locale settings of 244an embedding application. 245 246While :pep:`540`'s UTF-8 mode has the benefit of working regardless of which 247locales are available on the running system, it has the downside of having no 248effect on extension modules (such as GNU ``readline``), child processes running 249non-Python applications, and child processes running older versions of Python. 250To reduce the risk of corrupting text data when communicating with such 251components, Python 3.7 also implements :ref:`whatsnew37-pep540`). 252 253The UTF-8 mode is enabled by default when the locale is ``C`` or ``POSIX``, and 254the :pep:`538` locale coercion feature fails to change it to a UTF-8 based 255alternative (whether that failure is due to ``PYTHONCOERCECLOCALE=0`` being set, 256``LC_ALL`` being set, or the lack of a suitable target locale). 257 258.. seealso:: 259 260 :pep:`540` -- Add a new UTF-8 mode 261 PEP written and implemented by Victor Stinner 262 263 264.. _whatsnew37-pep553: 265 266PEP 553: Built-in ``breakpoint()`` 267---------------------------------- 268 269Python 3.7 includes the new built-in :func:`breakpoint` function as 270an easy and consistent way to enter the Python debugger. 271 272Built-in ``breakpoint()`` calls :func:`sys.breakpointhook`. By default, the 273latter imports :mod:`pdb` and then calls ``pdb.set_trace()``, but by binding 274``sys.breakpointhook()`` to the function of your choosing, ``breakpoint()`` can 275enter any debugger. Additionally, the environment variable 276:envvar:`PYTHONBREAKPOINT` can be set to the callable of your debugger of 277choice. Set ``PYTHONBREAKPOINT=0`` to completely disable built-in 278``breakpoint()``. 279 280.. seealso:: 281 282 :pep:`553` -- Built-in breakpoint() 283 PEP written and implemented by Barry Warsaw 284 285 286.. _whatsnew37-pep539: 287 288PEP 539: New C API for Thread-Local Storage 289------------------------------------------- 290 291While Python provides a C API for thread-local storage support; the existing 292:ref:`Thread Local Storage (TLS) API <thread-local-storage-api>` has used 293:c:type:`int` to represent TLS keys across all platforms. This has not 294generally been a problem for officially-support platforms, but that is neither 295POSIX-compliant, nor portable in any practical sense. 296 297:pep:`539` changes this by providing a new :ref:`Thread Specific Storage (TSS) 298API <thread-specific-storage-api>` to CPython which supersedes use of the 299existing TLS API within the CPython interpreter, while deprecating the existing 300API. The TSS API uses a new type :c:type:`Py_tss_t` instead of :c:type:`int` 301to represent TSS keys--an opaque type the definition of which may depend on 302the underlying TLS implementation. Therefore, this will allow to build CPython 303on platforms where the native TLS key is defined in a way that cannot be safely 304cast to :c:type:`int`. 305 306Note that on platforms where the native TLS key is defined in a way that cannot 307be safely cast to :c:type:`int`, all functions of the existing TLS API will be 308no-op and immediately return failure. This indicates clearly that the old API 309is not supported on platforms where it cannot be used reliably, and that no 310effort will be made to add such support. 311 312.. seealso:: 313 314 :pep:`539` -- A New C-API for Thread-Local Storage in CPython 315 PEP written by Erik M. Bray; implementation by Masayuki Yamamoto. 316 317 318.. _whatsnew37-pep562: 319 320PEP 562: Customization of Access to Module Attributes 321----------------------------------------------------- 322 323Python 3.7 allows defining :meth:`__getattr__` on modules and will call 324it whenever a module attribute is otherwise not found. Defining 325:meth:`__dir__` on modules is now also allowed. 326 327A typical example of where this may be useful is module attribute deprecation 328and lazy loading. 329 330.. seealso:: 331 332 :pep:`562` -- Module ``__getattr__`` and ``__dir__`` 333 PEP written and implemented by Ivan Levkivskyi 334 335 336.. _whatsnew37-pep564: 337 338PEP 564: New Time Functions With Nanosecond Resolution 339------------------------------------------------------ 340 341The resolution of clocks in modern systems can exceed the limited precision 342of a floating point number returned by the :func:`time.time` function 343and its variants. To avoid loss of precision, :pep:`564` adds six new 344"nanosecond" variants of the existing timer functions to the :mod:`time` 345module: 346 347* :func:`time.clock_gettime_ns` 348* :func:`time.clock_settime_ns` 349* :func:`time.monotonic_ns` 350* :func:`time.perf_counter_ns` 351* :func:`time.process_time_ns` 352* :func:`time.time_ns` 353 354The new functions return the number of nanoseconds as an integer value. 355 356`Measurements <https://www.python.org/dev/peps/pep-0564/#annex-clocks-resolution-in-python>`_ 357show that on Linux and Windows the resolution of :func:`time.time_ns` is 358approximately 3 times better than that of :func:`time.time`. 359 360.. seealso:: 361 362 :pep:`564` -- Add new time functions with nanosecond resolution 363 PEP written and implemented by Victor Stinner 364 365 366.. _whatsnew37-pep565: 367 368PEP 565: Show DeprecationWarning in ``__main__`` 369------------------------------------------------ 370 371The default handling of :exc:`DeprecationWarning` has been changed such that 372these warnings are once more shown by default, but only when the code 373triggering them is running directly in the :mod:`__main__` module. As a result, 374developers of single file scripts and those using Python interactively should 375once again start seeing deprecation warnings for the APIs they use, but 376deprecation warnings triggered by imported application, library and framework 377modules will continue to be hidden by default. 378 379As a result of this change, the standard library now allows developers to choose 380between three different deprecation warning behaviours: 381 382* :exc:`FutureWarning`: always displayed by default, recommended for warnings 383 intended to be seen by application end users (e.g. for deprecated application 384 configuration settings). 385* :exc:`DeprecationWarning`: displayed by default only in :mod:`__main__` and when 386 running tests, recommended for warnings intended to be seen by other Python 387 developers where a version upgrade may result in changed behaviour or an 388 error. 389* :exc:`PendingDeprecationWarning`: displayed by default only when running 390 tests, intended for cases where a future version upgrade will change the 391 warning category to :exc:`DeprecationWarning` or :exc:`FutureWarning`. 392 393Previously both :exc:`DeprecationWarning` and :exc:`PendingDeprecationWarning` 394were only visible when running tests, which meant that developers primarily 395writing single file scripts or using Python interactively could be surprised 396by breaking changes in the APIs they used. 397 398.. seealso:: 399 400 :pep:`565` -- Show DeprecationWarning in ``__main__`` 401 PEP written and implemented by Nick Coghlan 402 403 404.. _whatsnew37-pep560: 405 406PEP 560: Core Support for ``typing`` module and Generic Types 407------------------------------------------------------------- 408 409Initially :pep:`484` was designed in such way that it would not introduce *any* 410changes to the core CPython interpreter. Now type hints and the :mod:`typing` 411module are extensively used by the community, so this restriction is removed. 412The PEP introduces two special methods :meth:`__class_getitem__` and 413``__mro_entries__``, these methods are now used by most classes and special 414constructs in :mod:`typing`. As a result, the speed of various operations 415with types increased up to 7 times, the generic types can be used without 416metaclass conflicts, and several long standing bugs in :mod:`typing` module are 417fixed. 418 419.. seealso:: 420 421 :pep:`560` -- Core support for typing module and generic types 422 PEP written and implemented by Ivan Levkivskyi 423 424 425.. _whatsnew37-pep552: 426 427PEP 552: Hash-based .pyc Files 428------------------------------ 429 430Python has traditionally checked the up-to-dateness of bytecode cache files 431(i.e., ``.pyc`` files) by comparing the source metadata (last-modified timestamp 432and size) with source metadata saved in the cache file header when it was 433generated. While effective, this invalidation method has its drawbacks. When 434filesystem timestamps are too coarse, Python can miss source updates, leading to 435user confusion. Additionally, having a timestamp in the cache file is 436problematic for `build reproducibility <https://reproducible-builds.org/>`_ and 437content-based build systems. 438 439:pep:`552` extends the pyc format to allow the hash of the source file to be 440used for invalidation instead of the source timestamp. Such ``.pyc`` files are 441called "hash-based". By default, Python still uses timestamp-based invalidation 442and does not generate hash-based ``.pyc`` files at runtime. Hash-based ``.pyc`` 443files may be generated with :mod:`py_compile` or :mod:`compileall`. 444 445Hash-based ``.pyc`` files come in two variants: checked and unchecked. Python 446validates checked hash-based ``.pyc`` files against the corresponding source 447files at runtime but doesn't do so for unchecked hash-based pycs. Unchecked 448hash-based ``.pyc`` files are a useful performance optimization for environments 449where a system external to Python (e.g., the build system) is responsible for 450keeping ``.pyc`` files up-to-date. 451 452See :ref:`pyc-invalidation` for more information. 453 454.. seealso:: 455 456 :pep:`552` -- Deterministic pycs 457 PEP written and implemented by Benjamin Peterson 458 459 460.. _whatsnew37-pep545: 461 462PEP 545: Python Documentation Translations 463------------------------------------------ 464 465:pep:`545` describes the process of creating and maintaining Python 466documentation translations. 467 468Three new translations have been added: 469 470- Japanese: https://docs.python.org/ja/ 471- French: https://docs.python.org/fr/ 472- Korean: https://docs.python.org/ko/ 473 474.. seealso:: 475 476 :pep:`545` -- Python Documentation Translations 477 PEP written and implemented by Julien Palard, Inada Naoki, and 478 Victor Stinner. 479 480 481.. _whatsnew37-devmode: 482 483Python Development Mode (-X dev) 484-------------------------------- 485 486The new :option:`-X` ``dev`` command line option or the new 487:envvar:`PYTHONDEVMODE` environment variable can be used to enable 488:ref:`Python Development Mode <devmode>`. When in development mode, Python performs 489additional runtime checks that are too expensive to be enabled by default. 490See :ref:`Python Development Mode <devmode>` documentation for the full 491description. 492 493 494Other Language Changes 495====================== 496 497* An :keyword:`await` expression and comprehensions containing an 498 :keyword:`async for` clause were illegal in the expressions in 499 :ref:`formatted string literals <f-strings>` due to a problem with the 500 implementation. In Python 3.7 this restriction was lifted. 501 502* More than 255 arguments can now be passed to a function, and a function can 503 now have more than 255 parameters. (Contributed by Serhiy Storchaka in 504 :issue:`12844` and :issue:`18896`.) 505 506* :meth:`bytes.fromhex` and :meth:`bytearray.fromhex` now ignore all ASCII 507 whitespace, not only spaces. (Contributed by Robert Xiao in :issue:`28927`.) 508 509* :class:`str`, :class:`bytes`, and :class:`bytearray` gained support for 510 the new :meth:`isascii() <str.isascii>` method, which can be used to 511 test if a string or bytes contain only the ASCII characters. 512 (Contributed by INADA Naoki in :issue:`32677`.) 513 514* :exc:`ImportError` now displays module name and module ``__file__`` path when 515 ``from ... import ...`` fails. (Contributed by Matthias Bussonnier in 516 :issue:`29546`.) 517 518* Circular imports involving absolute imports with binding a submodule to 519 a name are now supported. 520 (Contributed by Serhiy Storchaka in :issue:`30024`.) 521 522* ``object.__format__(x, '')`` is now equivalent to ``str(x)`` rather than 523 ``format(str(self), '')``. 524 (Contributed by Serhiy Storchaka in :issue:`28974`.) 525 526* In order to better support dynamic creation of stack traces, 527 :class:`types.TracebackType` can now be instantiated from Python code, and 528 the ``tb_next`` attribute on :ref:`tracebacks <traceback-objects>` is now 529 writable. 530 (Contributed by Nathaniel J. Smith in :issue:`30579`.) 531 532* When using the :option:`-m` switch, ``sys.path[0]`` is now eagerly expanded 533 to the full starting directory path, rather than being left as the empty 534 directory (which allows imports from the *current* working directory at the 535 time when an import occurs) 536 (Contributed by Nick Coghlan in :issue:`33053`.) 537 538* The new :option:`-X` ``importtime`` option or the 539 :envvar:`PYTHONPROFILEIMPORTTIME` environment variable can be used to show 540 the timing of each module import. 541 (Contributed by Victor Stinner in :issue:`31415`.) 542 543 544New Modules 545=========== 546 547.. _whatsnew37-pep567: 548 549contextvars 550----------- 551 552The new :mod:`contextvars` module and a set of 553:ref:`new C APIs <contextvarsobjects>` introduce 554support for *context variables*. Context variables are conceptually 555similar to thread-local variables. Unlike TLS, context variables 556support asynchronous code correctly. 557 558The :mod:`asyncio` and :mod:`decimal` modules have been updated to use 559and support context variables out of the box. Particularly the active 560decimal context is now stored in a context variable, which allows 561decimal operations to work with the correct context in asynchronous code. 562 563.. seealso:: 564 565 :pep:`567` -- Context Variables 566 PEP written and implemented by Yury Selivanov 567 568 569.. _whatsnew37-pep557: 570 571dataclasses 572----------- 573 574The new :func:`~dataclasses.dataclass` decorator provides a way to declare 575*data classes*. A data class describes its attributes using class variable 576annotations. Its constructor and other magic methods, such as 577:meth:`~object.__repr__`, :meth:`~object.__eq__`, and 578:meth:`~object.__hash__` are generated automatically. 579 580Example:: 581 582 @dataclass 583 class Point: 584 x: float 585 y: float 586 z: float = 0.0 587 588 p = Point(1.5, 2.5) 589 print(p) # produces "Point(x=1.5, y=2.5, z=0.0)" 590 591.. seealso:: 592 593 :pep:`557` -- Data Classes 594 PEP written and implemented by Eric V. Smith 595 596 597.. _whatsnew37_importlib_resources: 598 599importlib.resources 600------------------- 601 602The new :mod:`importlib.resources` module provides several new APIs and one 603new ABC for access to, opening, and reading *resources* inside packages. 604Resources are roughly similar to files inside packages, but they needn't 605be actual files on the physical file system. Module loaders can provide a 606:meth:`get_resource_reader()` function which returns 607a :class:`importlib.abc.ResourceReader` instance to support this 608new API. Built-in file path loaders and zip file loaders both support this. 609 610Contributed by Barry Warsaw and Brett Cannon in :issue:`32248`. 611 612.. seealso:: 613 614 `importlib_resources <http://importlib-resources.readthedocs.io/en/latest/>`_ 615 -- a PyPI backport for earlier Python versions. 616 617 618Improved Modules 619================ 620 621 622argparse 623-------- 624 625The new :meth:`ArgumentParser.parse_intermixed_args() 626<argparse.ArgumentParser.parse_intermixed_args>` 627method allows intermixing options and positional arguments. 628(Contributed by paul.j3 in :issue:`14191`.) 629 630 631.. _whatsnew37_asyncio: 632 633asyncio 634------- 635 636The :mod:`asyncio` module has received many new features, usability and 637:ref:`performance improvements <whatsnew37-asyncio-perf>`. Notable changes 638include: 639 640* The new :term:`provisional <provisional API>` :func:`asyncio.run` function can 641 be used to run a coroutine from synchronous code by automatically creating and 642 destroying the event loop. 643 (Contributed by Yury Selivanov in :issue:`32314`.) 644 645* asyncio gained support for :mod:`contextvars`. 646 :meth:`loop.call_soon() <asyncio.loop.call_soon>`, 647 :meth:`loop.call_soon_threadsafe() <asyncio.loop.call_soon_threadsafe>`, 648 :meth:`loop.call_later() <asyncio.loop.call_later>`, 649 :meth:`loop.call_at() <asyncio.loop.call_at>`, and 650 :meth:`Future.add_done_callback() <asyncio.Future.add_done_callback>` 651 have a new optional keyword-only *context* parameter. 652 :class:`Tasks <asyncio.Task>` now track their context automatically. 653 See :pep:`567` for more details. 654 (Contributed by Yury Selivanov in :issue:`32436`.) 655 656* The new :func:`asyncio.create_task` function has been added as a shortcut 657 to ``asyncio.get_event_loop().create_task()``. 658 (Contributed by Andrew Svetlov in :issue:`32311`.) 659 660* The new :meth:`loop.start_tls() <asyncio.loop.start_tls>` 661 method can be used to upgrade an existing connection to TLS. 662 (Contributed by Yury Selivanov in :issue:`23749`.) 663 664* The new :meth:`loop.sock_recv_into() <asyncio.loop.sock_recv_into>` 665 method allows reading data from a socket directly into a provided buffer making 666 it possible to reduce data copies. 667 (Contributed by Antoine Pitrou in :issue:`31819`.) 668 669* The new :func:`asyncio.current_task` function returns the currently running 670 :class:`~asyncio.Task` instance, and the new :func:`asyncio.all_tasks` 671 function returns a set of all existing ``Task`` instances in a given loop. 672 The :meth:`Task.current_task() <asyncio.Task.current_task>` and 673 :meth:`Task.all_tasks() <asyncio.Task.all_tasks>` methods have been deprecated. 674 (Contributed by Andrew Svetlov in :issue:`32250`.) 675 676* The new *provisional* :class:`~asyncio.BufferedProtocol` class allows 677 implementing streaming protocols with manual control over the receive buffer. 678 (Contributed by Yury Selivanov in :issue:`32251`.) 679 680* The new :func:`asyncio.get_running_loop` function returns the currently 681 running loop, and raises a :exc:`RuntimeError` if no loop is running. 682 This is in contrast with :func:`asyncio.get_event_loop`, which will *create* 683 a new event loop if none is running. 684 (Contributed by Yury Selivanov in :issue:`32269`.) 685 686* The new :meth:`StreamWriter.wait_closed() <asyncio.StreamWriter.wait_closed>` 687 coroutine method allows waiting until the stream writer is closed. The new 688 :meth:`StreamWriter.is_closing() <asyncio.StreamWriter.is_closing>` method 689 can be used to determine if the writer is closing. 690 (Contributed by Andrew Svetlov in :issue:`32391`.) 691 692* The new :meth:`loop.sock_sendfile() <asyncio.loop.sock_sendfile>` 693 coroutine method allows sending files using :mod:`os.sendfile` when possible. 694 (Contributed by Andrew Svetlov in :issue:`32410`.) 695 696* The new :meth:`Future.get_loop() <asyncio.Future.get_loop>` and 697 ``Task.get_loop()`` methods return the instance of the loop on which a task or 698 a future were created. 699 :meth:`Server.get_loop() <asyncio.Server.get_loop>` allows doing the same for 700 :class:`asyncio.Server` objects. 701 (Contributed by Yury Selivanov in :issue:`32415` and 702 Srinivas Reddy Thatiparthy in :issue:`32418`.) 703 704* It is now possible to control how instances of :class:`asyncio.Server` begin 705 serving. Previously, the server would start serving immediately when created. 706 The new *start_serving* keyword argument to 707 :meth:`loop.create_server() <asyncio.loop.create_server>` and 708 :meth:`loop.create_unix_server() <asyncio.loop.create_unix_server>`, 709 as well as :meth:`Server.start_serving() <asyncio.Server.start_serving>`, and 710 :meth:`Server.serve_forever() <asyncio.Server.serve_forever>` 711 can be used to decouple server instantiation and serving. The new 712 :meth:`Server.is_serving() <asyncio.Server.is_serving>` method returns ``True`` 713 if the server is serving. :class:`~asyncio.Server` objects are now 714 asynchronous context managers:: 715 716 srv = await loop.create_server(...) 717 718 async with srv: 719 # some code 720 721 # At this point, srv is closed and no longer accepts new connections. 722 723 (Contributed by Yury Selivanov in :issue:`32662`.) 724 725* Callback objects returned by 726 :func:`loop.call_later() <asyncio.loop.call_later>` 727 gained the new :meth:`when() <asyncio.TimerHandle.when>` method which 728 returns an absolute scheduled callback timestamp. 729 (Contributed by Andrew Svetlov in :issue:`32741`.) 730 731* The :meth:`loop.create_datagram_endpoint() \ 732 <asyncio.loop.create_datagram_endpoint>` method 733 gained support for Unix sockets. 734 (Contributed by Quentin Dawans in :issue:`31245`.) 735 736* The :func:`asyncio.open_connection`, :func:`asyncio.start_server` functions, 737 :meth:`loop.create_connection() <asyncio.loop.create_connection>`, 738 :meth:`loop.create_server() <asyncio.loop.create_server>`, 739 :meth:`loop.create_accepted_socket() <asyncio.loop.connect_accepted_socket>` 740 methods and their corresponding UNIX socket variants now accept the 741 *ssl_handshake_timeout* keyword argument. 742 (Contributed by Neil Aspinall in :issue:`29970`.) 743 744* The new :meth:`Handle.cancelled() <asyncio.Handle.cancelled>` method returns 745 ``True`` if the callback was cancelled. 746 (Contributed by Marat Sharafutdinov in :issue:`31943`.) 747 748* The asyncio source has been converted to use the 749 :keyword:`async`/:keyword:`await` syntax. 750 (Contributed by Andrew Svetlov in :issue:`32193`.) 751 752* The new :meth:`ReadTransport.is_reading() <asyncio.ReadTransport.is_reading>` 753 method can be used to determine the reading state of the transport. 754 Additionally, calls to 755 :meth:`ReadTransport.resume_reading() <asyncio.ReadTransport.resume_reading>` 756 and :meth:`ReadTransport.pause_reading() <asyncio.ReadTransport.pause_reading>` 757 are now idempotent. 758 (Contributed by Yury Selivanov in :issue:`32356`.) 759 760* Loop methods which accept socket paths now support passing 761 :term:`path-like objects <path-like object>`. 762 (Contributed by Yury Selivanov in :issue:`32066`.) 763 764* In :mod:`asyncio` TCP sockets on Linux are now created with ``TCP_NODELAY`` 765 flag set by default. 766 (Contributed by Yury Selivanov and Victor Stinner in :issue:`27456`.) 767 768* Exceptions occurring in cancelled tasks are no longer logged. 769 (Contributed by Yury Selivanov in :issue:`30508`.) 770 771* New ``WindowsSelectorEventLoopPolicy`` and 772 ``WindowsProactorEventLoopPolicy`` classes. 773 (Contributed by Yury Selivanov in :issue:`33792`.) 774 775Several ``asyncio`` APIs have been 776:ref:`deprecated <whatsnew37-asyncio-deprecated>`. 777 778 779binascii 780-------- 781 782The :func:`~binascii.b2a_uu` function now accepts an optional *backtick* 783keyword argument. When it's true, zeros are represented by ``'`'`` 784instead of spaces. (Contributed by Xiang Zhang in :issue:`30103`.) 785 786 787calendar 788-------- 789 790The :class:`~calendar.HTMLCalendar` class has new class attributes which ease 791the customization of CSS classes in the produced HTML calendar. 792(Contributed by Oz Tiram in :issue:`30095`.) 793 794 795collections 796----------- 797 798``collections.namedtuple()`` now supports default values. 799(Contributed by Raymond Hettinger in :issue:`32320`.) 800 801 802compileall 803---------- 804 805:func:`compileall.compile_dir` learned the new *invalidation_mode* parameter, 806which can be used to enable 807:ref:`hash-based .pyc invalidation <whatsnew37-pep552>`. The invalidation 808mode can also be specified on the command line using the new 809``--invalidation-mode`` argument. 810(Contributed by Benjamin Peterson in :issue:`31650`.) 811 812 813concurrent.futures 814------------------ 815 816:class:`ProcessPoolExecutor <concurrent.futures.ProcessPoolExecutor>` and 817:class:`ThreadPoolExecutor <concurrent.futures.ThreadPoolExecutor>` now 818support the new *initializer* and *initargs* constructor arguments. 819(Contributed by Antoine Pitrou in :issue:`21423`.) 820 821The :class:`ProcessPoolExecutor <concurrent.futures.ProcessPoolExecutor>` 822can now take the multiprocessing context via the new *mp_context* argument. 823(Contributed by Thomas Moreau in :issue:`31540`.) 824 825 826contextlib 827---------- 828 829The new :func:`~contextlib.nullcontext` is a simpler and faster no-op 830context manager than :class:`~contextlib.ExitStack`. 831(Contributed by Jesse-Bakker in :issue:`10049`.) 832 833The new :func:`~contextlib.asynccontextmanager`, 834:class:`~contextlib.AbstractAsyncContextManager`, and 835:class:`~contextlib.AsyncExitStack` have been added to 836complement their synchronous counterparts. (Contributed 837by Jelle Zijlstra in :issue:`29679` and :issue:`30241`, 838and by Alexander Mohr and Ilya Kulakov in :issue:`29302`.) 839 840 841cProfile 842-------- 843 844The :mod:`cProfile` command line now accepts ``-m module_name`` as an 845alternative to script path. (Contributed by Sanyam Khurana in :issue:`21862`.) 846 847 848crypt 849----- 850 851The :mod:`crypt` module now supports the Blowfish hashing method. 852(Contributed by Serhiy Storchaka in :issue:`31664`.) 853 854The :func:`~crypt.mksalt` function now allows specifying the number of rounds 855for hashing. (Contributed by Serhiy Storchaka in :issue:`31702`.) 856 857 858datetime 859-------- 860 861The new :meth:`datetime.fromisoformat() <datetime.datetime.fromisoformat>` 862method constructs a :class:`~datetime.datetime` object from a string 863in one of the formats output by 864:meth:`datetime.isoformat() <datetime.datetime.isoformat>`. 865(Contributed by Paul Ganssle in :issue:`15873`.) 866 867The :class:`tzinfo <datetime.tzinfo>` class now supports sub-minute offsets. 868(Contributed by Alexander Belopolsky in :issue:`5288`.) 869 870 871dbm 872--- 873 874:mod:`dbm.dumb` now supports reading read-only files and no longer writes the 875index file when it is not changed. 876 877 878decimal 879------- 880 881The :mod:`decimal` module now uses :ref:`context variables <whatsnew37-pep567>` 882to store the decimal context. 883(Contributed by Yury Selivanov in :issue:`32630`.) 884 885 886dis 887--- 888 889The :func:`~dis.dis` function is now able to 890disassemble nested code objects (the code of comprehensions, generator 891expressions and nested functions, and the code used for building nested 892classes). The maximum depth of disassembly recursion is controlled by 893the new *depth* parameter. 894(Contributed by Serhiy Storchaka in :issue:`11822`.) 895 896 897distutils 898--------- 899 900``README.rst`` is now included in the list of distutils standard READMEs and 901therefore included in source distributions. 902(Contributed by Ryan Gonzalez in :issue:`11913`.) 903 904 905enum 906---- 907 908The :class:`Enum <enum.Enum>` learned the new ``_ignore_`` class property, 909which allows listing the names of properties which should not become 910enum members. 911(Contributed by Ethan Furman in :issue:`31801`.) 912 913In Python 3.8, attempting to check for non-Enum objects in :class:`Enum` 914classes will raise a :exc:`TypeError` (e.g. ``1 in Color``); similarly, 915attempting to check for non-Flag objects in a :class:`Flag` member will 916raise :exc:`TypeError` (e.g. ``1 in Perm.RW``); currently, both operations 917return :const:`False` instead and are deprecated. 918(Contributed by Ethan Furman in :issue:`33217`.) 919 920 921functools 922--------- 923 924:func:`functools.singledispatch` now supports registering implementations 925using type annotations. 926(Contributed by Łukasz Langa in :issue:`32227`.) 927 928 929gc 930-- 931 932The new :func:`gc.freeze` function allows freezing all objects tracked 933by the garbage collector and excluding them from future collections. 934This can be used before a POSIX ``fork()`` call to make the GC copy-on-write 935friendly or to speed up collection. The new :func:`gc.unfreeze` functions 936reverses this operation. Additionally, :func:`gc.get_freeze_count` can 937be used to obtain the number of frozen objects. 938(Contributed by Li Zekun in :issue:`31558`.) 939 940 941hmac 942---- 943 944The :mod:`hmac` module now has an optimized one-shot :func:`~hmac.digest` 945function, which is up to three times faster than :func:`~hmac.HMAC`. 946(Contributed by Christian Heimes in :issue:`32433`.) 947 948 949http.client 950----------- 951 952:class:`~http.client.HTTPConnection` and :class:`~http.client.HTTPSConnection` 953now support the new *blocksize* argument for improved upload throughput. 954(Contributed by Nir Soffer in :issue:`31945`.) 955 956 957http.server 958----------- 959 960:class:`~http.server.SimpleHTTPRequestHandler` now supports the HTTP 961``If-Modified-Since`` header. The server returns the 304 response status if 962the target file was not modified after the time specified in the header. 963(Contributed by Pierre Quentel in :issue:`29654`.) 964 965:class:`~http.server.SimpleHTTPRequestHandler` accepts the new *directory* 966argument, in addition to the new ``--directory`` command line argument. 967With this parameter, the server serves the specified directory, by default it 968uses the current working directory. 969(Contributed by Stéphane Wirtel and Julien Palard in :issue:`28707`.) 970 971The new :class:`ThreadingHTTPServer <http.server.ThreadingHTTPServer>` class 972uses threads to handle requests using :class:`~socketserver.ThreadingMixin`. 973It is used when ``http.server`` is run with ``-m``. 974(Contributed by Julien Palard in :issue:`31639`.) 975 976 977idlelib and IDLE 978---------------- 979 980Multiple fixes for autocompletion. (Contributed by Louie Lu in :issue:`15786`.) 981 982Module Browser (on the File menu, formerly called Class Browser), 983now displays nested functions and classes in addition to top-level 984functions and classes. 985(Contributed by Guilherme Polo, Cheryl Sabella, and Terry Jan Reedy 986in :issue:`1612262`.) 987 988The Settings dialog (Options, Configure IDLE) has been partly rewritten 989to improve both appearance and function. 990(Contributed by Cheryl Sabella and Terry Jan Reedy in multiple issues.) 991 992The font sample now includes a selection of non-Latin characters so that 993users can better see the effect of selecting a particular font. 994(Contributed by Terry Jan Reedy in :issue:`13802`.) 995The sample can be edited to include other characters. 996(Contributed by Serhiy Storchaka in :issue:`31860`.) 997 998The IDLE features formerly implemented as extensions have been reimplemented 999as normal features. Their settings have been moved from the Extensions tab 1000to other dialog tabs. 1001(Contributed by Charles Wohlganger and Terry Jan Reedy in :issue:`27099`.) 1002 1003Editor code context option revised. Box displays all context lines up to 1004maxlines. Clicking on a context line jumps the editor to that line. Context 1005colors for custom themes is added to Highlights tab of Settings dialog. 1006(Contributed by Cheryl Sabella and Terry Jan Reedy in :issue:`33642`, 1007:issue:`33768`, and :issue:`33679`.) 1008 1009On Windows, a new API call tells Windows that tk scales for DPI. On Windows 10108.1+ or 10, with DPI compatibility properties of the Python binary 1011unchanged, and a monitor resolution greater than 96 DPI, this should 1012make text and lines sharper. It should otherwise have no effect. 1013(Contributed by Terry Jan Reedy in :issue:`33656`.) 1014 1015New in 3.7.1: 1016 1017Output over N lines (50 by default) is squeezed down to a button. 1018N can be changed in the PyShell section of the General page of the 1019Settings dialog. Fewer, but possibly extra long, lines can be squeezed by 1020right clicking on the output. Squeezed output can be expanded in place 1021by double-clicking the button or into the clipboard or a separate window 1022by right-clicking the button. (Contributed by Tal Einat in :issue:`1529353`.) 1023 1024The changes above have been backported to 3.6 maintenance releases. 1025 1026NEW in 3.7.4: 1027 1028Add "Run Customized" to the Run menu to run a module with customized 1029settings. Any command line arguments entered are added to sys.argv. 1030They re-appear in the box for the next customized run. One can also 1031suppress the normal Shell main module restart. (Contributed by Cheryl 1032Sabella, Terry Jan Reedy, and others in :issue:`5680` and :issue:`37627`.) 1033 1034New in 3.7.5: 1035 1036Add optional line numbers for IDLE editor windows. Windows 1037open without line numbers unless set otherwise in the General 1038tab of the configuration dialog. Line numbers for an existing 1039window are shown and hidden in the Options menu. 1040(Contributed by Tal Einat and Saimadhav Heblikar in :issue:`17535`.) 1041 1042 1043importlib 1044--------- 1045 1046The :class:`importlib.abc.ResourceReader` ABC was introduced to 1047support the loading of resources from packages. See also 1048:ref:`whatsnew37_importlib_resources`. 1049(Contributed by Barry Warsaw, Brett Cannon in :issue:`32248`.) 1050 1051:func:`importlib.reload` now raises :exc:`ModuleNotFoundError` if the module 1052lacks a spec. 1053(Contributed by Garvit Khatri in :issue:`29851`.) 1054 1055:func:`importlib.find_spec` now raises :exc:`ModuleNotFoundError` instead of 1056:exc:`AttributeError` if the specified parent module is not a package (i.e. 1057lacks a ``__path__`` attribute). 1058(Contributed by Milan Oberkirch in :issue:`30436`.) 1059 1060The new :func:`importlib.source_hash` can be used to compute the hash of 1061the passed source. A :ref:`hash-based .pyc file <whatsnew37-pep552>` 1062embeds the value returned by this function. 1063 1064 1065io 1066-- 1067 1068The new :meth:`TextIOWrapper.reconfigure() <io.TextIOWrapper.reconfigure>` 1069method can be used to reconfigure the text stream with the new settings. 1070(Contributed by Antoine Pitrou in :issue:`30526` and 1071INADA Naoki in :issue:`15216`.) 1072 1073 1074ipaddress 1075--------- 1076 1077The new ``subnet_of()`` and ``supernet_of()`` methods of 1078:class:`ipaddress.IPv6Network` and :class:`ipaddress.IPv4Network` can 1079be used for network containment tests. 1080(Contributed by Michel Albert and Cheryl Sabella in :issue:`20825`.) 1081 1082 1083itertools 1084--------- 1085 1086:func:`itertools.islice` now accepts 1087:meth:`integer-like objects <object.__index__>` as start, stop, 1088and slice arguments. 1089(Contributed by Will Roberts in :issue:`30537`.) 1090 1091 1092locale 1093------ 1094 1095The new *monetary* argument to :func:`locale.format_string` can be used 1096to make the conversion use monetary thousands separators and 1097grouping strings. (Contributed by Garvit in :issue:`10379`.) 1098 1099The :func:`locale.getpreferredencoding` function now always returns ``'UTF-8'`` 1100on Android or when in the :ref:`forced UTF-8 mode <whatsnew37-pep540>`. 1101 1102 1103logging 1104------- 1105 1106:class:`~logging.Logger` instances can now be pickled. 1107(Contributed by Vinay Sajip in :issue:`30520`.) 1108 1109The new :meth:`StreamHandler.setStream() <logging.StreamHandler.setStream>` 1110method can be used to replace the logger stream after handler creation. 1111(Contributed by Vinay Sajip in :issue:`30522`.) 1112 1113It is now possible to specify keyword arguments to handler constructors in 1114configuration passed to :func:`logging.config.fileConfig`. 1115(Contributed by Preston Landers in :issue:`31080`.) 1116 1117 1118math 1119---- 1120 1121The new :func:`math.remainder` function implements the IEEE 754-style remainder 1122operation. (Contributed by Mark Dickinson in :issue:`29962`.) 1123 1124 1125mimetypes 1126--------- 1127 1128The MIME type of .bmp has been changed from ``'image/x-ms-bmp'`` to 1129``'image/bmp'``. 1130(Contributed by Nitish Chandra in :issue:`22589`.) 1131 1132 1133msilib 1134------ 1135 1136The new :meth:`Database.Close() <msilib.Database.Close>` method can be used 1137to close the :abbr:`MSI` database. 1138(Contributed by Berker Peksag in :issue:`20486`.) 1139 1140 1141multiprocessing 1142--------------- 1143 1144The new :meth:`Process.close() <multiprocessing.Process.close>` method 1145explicitly closes the process object and releases all resources associated 1146with it. :exc:`ValueError` is raised if the underlying process is still 1147running. 1148(Contributed by Antoine Pitrou in :issue:`30596`.) 1149 1150The new :meth:`Process.kill() <multiprocessing.Process.kill>` method can 1151be used to terminate the process using the :data:`SIGKILL` signal on Unix. 1152(Contributed by Vitor Pereira in :issue:`30794`.) 1153 1154Non-daemonic threads created by :class:`~multiprocessing.Process` are now 1155joined on process exit. 1156(Contributed by Antoine Pitrou in :issue:`18966`.) 1157 1158 1159os 1160-- 1161 1162:func:`os.fwalk` now accepts the *path* argument as :class:`bytes`. 1163(Contributed by Serhiy Storchaka in :issue:`28682`.) 1164 1165:func:`os.scandir` gained support for :ref:`file descriptors <path_fd>`. 1166(Contributed by Serhiy Storchaka in :issue:`25996`.) 1167 1168The new :func:`~os.register_at_fork` function allows registering Python 1169callbacks to be executed at process fork. 1170(Contributed by Antoine Pitrou in :issue:`16500`.) 1171 1172Added :func:`os.preadv` (combine the functionality of :func:`os.readv` and 1173:func:`os.pread`) and :func:`os.pwritev` functions (combine the functionality 1174of :func:`os.writev` and :func:`os.pwrite`). (Contributed by Pablo Galindo in 1175:issue:`31368`.) 1176 1177The mode argument of :func:`os.makedirs` no longer affects the file 1178permission bits of newly-created intermediate-level directories. 1179(Contributed by Serhiy Storchaka in :issue:`19930`.) 1180 1181:func:`os.dup2` now returns the new file descriptor. Previously, ``None`` 1182was always returned. 1183(Contributed by Benjamin Peterson in :issue:`32441`.) 1184 1185The structure returned by :func:`os.stat` now contains the 1186:attr:`~os.stat_result.st_fstype` attribute on Solaris and its derivatives. 1187(Contributed by Jesús Cea Avión in :issue:`32659`.) 1188 1189 1190pathlib 1191------- 1192 1193The new :meth:`Path.is_mount() <pathlib.Path.is_mount>` method is now available 1194on POSIX systems and can be used to determine whether a path is a mount point. 1195(Contributed by Cooper Ry Lees in :issue:`30897`.) 1196 1197 1198pdb 1199--- 1200 1201:func:`pdb.set_trace` now takes an optional *header* keyword-only 1202argument. If given, it is printed to the console just before debugging 1203begins. (Contributed by Barry Warsaw in :issue:`31389`.) 1204 1205:mod:`pdb` command line now accepts ``-m module_name`` as an alternative to 1206script file. (Contributed by Mario Corchero in :issue:`32206`.) 1207 1208 1209py_compile 1210---------- 1211 1212:func:`py_compile.compile` -- and by extension, :mod:`compileall` -- now 1213respects the :envvar:`SOURCE_DATE_EPOCH` environment variable by 1214unconditionally creating ``.pyc`` files for hash-based validation. 1215This allows for guaranteeing 1216`reproducible builds <https://reproducible-builds.org/>`_ of ``.pyc`` 1217files when they are created eagerly. (Contributed by Bernhard M. Wiedemann 1218in :issue:`29708`.) 1219 1220 1221pydoc 1222----- 1223 1224The pydoc server can now bind to an arbitrary hostname specified by the 1225new ``-n`` command-line argument. 1226(Contributed by Feanil Patel in :issue:`31128`.) 1227 1228 1229queue 1230----- 1231 1232The new :class:`~queue.SimpleQueue` class is an unbounded :abbr:`FIFO` queue. 1233(Contributed by Antoine Pitrou in :issue:`14976`.) 1234 1235 1236re 1237-- 1238 1239The flags :const:`re.ASCII`, :const:`re.LOCALE` and :const:`re.UNICODE` 1240can be set within the scope of a group. 1241(Contributed by Serhiy Storchaka in :issue:`31690`.) 1242 1243:func:`re.split` now supports splitting on a pattern like ``r'\b'``, 1244``'^$'`` or ``(?=-)`` that matches an empty string. 1245(Contributed by Serhiy Storchaka in :issue:`25054`.) 1246 1247Regular expressions compiled with the :const:`re.LOCALE` flag no longer 1248depend on the locale at compile time. Locale settings are applied only 1249when the compiled regular expression is used. 1250(Contributed by Serhiy Storchaka in :issue:`30215`.) 1251 1252:exc:`FutureWarning` is now emitted if a regular expression contains 1253character set constructs that will change semantically in the future, 1254such as nested sets and set operations. 1255(Contributed by Serhiy Storchaka in :issue:`30349`.) 1256 1257Compiled regular expression and match objects can now be copied 1258using :func:`copy.copy` and :func:`copy.deepcopy`. 1259(Contributed by Serhiy Storchaka in :issue:`10076`.) 1260 1261 1262signal 1263------ 1264 1265The new *warn_on_full_buffer* argument to the :func:`signal.set_wakeup_fd` 1266function makes it possible to specify whether Python prints a warning on 1267stderr when the wakeup buffer overflows. 1268(Contributed by Nathaniel J. Smith in :issue:`30050`.) 1269 1270 1271socket 1272------ 1273 1274The new :func:`socket.getblocking() <socket.socket.getblocking>` method 1275returns ``True`` if the socket is in blocking mode and ``False`` otherwise. 1276(Contributed by Yury Selivanov in :issue:`32373`.) 1277 1278The new :func:`socket.close` function closes the passed socket file descriptor. 1279This function should be used instead of :func:`os.close` for better 1280compatibility across platforms. 1281(Contributed by Christian Heimes in :issue:`32454`.) 1282 1283The :mod:`socket` module now exposes the :data:`socket.TCP_CONGESTION` 1284(Linux 2.6.13), :data:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37), and 1285:data:`socket.TCP_NOTSENT_LOWAT` (Linux 3.12) constants. 1286(Contributed by Omar Sandoval in :issue:`26273` and 1287Nathaniel J. Smith in :issue:`29728`.) 1288 1289Support for :data:`socket.AF_VSOCK` sockets has been added to allow 1290communication between virtual machines and their hosts. 1291(Contributed by Cathy Avery in :issue:`27584`.) 1292 1293Sockets now auto-detect family, type and protocol from file descriptor 1294by default. 1295(Contributed by Christian Heimes in :issue:`28134`.) 1296 1297 1298socketserver 1299------------ 1300 1301:meth:`socketserver.ThreadingMixIn.server_close` now waits until all non-daemon 1302threads complete. :meth:`socketserver.ForkingMixIn.server_close` now waits 1303until all child processes complete. 1304 1305Add a new :attr:`socketserver.ForkingMixIn.block_on_close` class attribute to 1306:class:`socketserver.ForkingMixIn` and :class:`socketserver.ThreadingMixIn` 1307classes. Set the class attribute to ``False`` to get the pre-3.7 behaviour. 1308 1309 1310sqlite3 1311------- 1312 1313:class:`sqlite3.Connection` now exposes the :meth:`~sqlite3.Connection.backup` 1314method when the underlying SQLite library is at version 3.6.11 or higher. 1315(Contributed by Lele Gaifax in :issue:`27645`.) 1316 1317The *database* argument of :func:`sqlite3.connect` now accepts any 1318:term:`path-like object`, instead of just a string. 1319(Contributed by Anders Lorentsen in :issue:`31843`.) 1320 1321 1322ssl 1323--- 1324 1325The :mod:`ssl` module now uses OpenSSL's builtin API instead of 1326:func:`~ssl.match_hostname` to check a host name or an IP address. Values 1327are validated during TLS handshake. Any certificate validation error 1328including failing the host name check now raises 1329:exc:`~ssl.SSLCertVerificationError` and aborts the handshake with a proper 1330TLS Alert message. The new exception contains additional information. 1331Host name validation can be customized with 1332:attr:`SSLContext.hostname_checks_common_name <ssl.SSLContext.hostname_checks_common_name>`. 1333(Contributed by Christian Heimes in :issue:`31399`.) 1334 1335.. note:: 1336 The improved host name check requires a *libssl* implementation compatible 1337 with OpenSSL 1.0.2 or 1.1. Consequently, OpenSSL 0.9.8 and 1.0.1 are no 1338 longer supported (see :ref:`37-platform-support-removals` for more details). 1339 The ssl module is mostly compatible with LibreSSL 2.7.2 and newer. 1340 1341The ``ssl`` module no longer sends IP addresses in SNI TLS extension. 1342(Contributed by Christian Heimes in :issue:`32185`.) 1343 1344:func:`~ssl.match_hostname` no longer supports partial wildcards like 1345``www*.example.org``. 1346(Contributed by Mandeep Singh in :issue:`23033` and Christian Heimes in 1347:issue:`31399`.) 1348 1349The default cipher suite selection of the ``ssl`` module now uses a blacklist 1350approach rather than a hard-coded whitelist. Python no longer re-enables 1351ciphers that have been blocked by OpenSSL security updates. Default cipher 1352suite selection can be configured at compile time. 1353(Contributed by Christian Heimes in :issue:`31429`.) 1354 1355Validation of server certificates containing internationalized domain names 1356(IDNs) is now supported. As part of this change, the 1357:attr:`SSLSocket.server_hostname <ssl.SSLSocket.server_hostname>` attribute 1358now stores the expected hostname in A-label form (``"xn--pythn-mua.org"``), 1359rather than the U-label form (``"pythön.org"``). (Contributed by 1360Nathaniel J. Smith and Christian Heimes in :issue:`28414`.) 1361 1362The ``ssl`` module has preliminary and experimental support for TLS 1.3 and 1363OpenSSL 1.1.1. At the time of Python 3.7.0 release, OpenSSL 1.1.1 is still 1364under development and TLS 1.3 hasn't been finalized yet. The TLS 1.3 1365handshake and protocol behaves slightly differently than TLS 1.2 and earlier, 1366see :ref:`ssl-tlsv1_3`. 1367(Contributed by Christian Heimes in :issue:`32947`, :issue:`20995`, 1368:issue:`29136`, :issue:`30622` and :issue:`33618`) 1369 1370:class:`~ssl.SSLSocket` and :class:`~ssl.SSLObject` no longer have a public 1371constructor. Direct instantiation was never a documented and supported 1372feature. Instances must be created with :class:`~ssl.SSLContext` methods 1373:meth:`~ssl.SSLContext.wrap_socket` and :meth:`~ssl.SSLContext.wrap_bio`. 1374(Contributed by Christian Heimes in :issue:`32951`) 1375 1376OpenSSL 1.1 APIs for setting the minimum and maximum TLS protocol version are 1377available as :attr:`SSLContext.minimum_version <ssl.SSLContext.minimum_version>` 1378and :attr:`SSLContext.maximum_version <ssl.SSLContext.maximum_version>`. 1379Supported protocols are indicated by several new flags, such as 1380:data:`~ssl.HAS_TLSv1_1`. 1381(Contributed by Christian Heimes in :issue:`32609`.) 1382 1383 1384string 1385------ 1386 1387:class:`string.Template` now lets you to optionally modify the regular 1388expression pattern for braced placeholders and non-braced placeholders 1389separately. (Contributed by Barry Warsaw in :issue:`1198569`.) 1390 1391 1392subprocess 1393---------- 1394 1395The :func:`subprocess.run` function accepts the new *capture_output* 1396keyword argument. When true, stdout and stderr will be captured. 1397This is equivalent to passing :data:`subprocess.PIPE` as *stdout* and 1398*stderr* arguments. 1399(Contributed by Bo Bayles in :issue:`32102`.) 1400 1401The ``subprocess.run`` function and the :class:`subprocess.Popen` constructor 1402now accept the *text* keyword argument as an alias 1403to *universal_newlines*. 1404(Contributed by Andrew Clegg in :issue:`31756`.) 1405 1406On Windows the default for *close_fds* was changed from ``False`` to 1407``True`` when redirecting the standard handles. It's now possible to set 1408*close_fds* to true when redirecting the standard handles. See 1409:class:`subprocess.Popen`. This means that *close_fds* now defaults to 1410``True`` on all supported platforms. 1411(Contributed by Segev Finer in :issue:`19764`.) 1412 1413The subprocess module is now more graceful when handling 1414:exc:`KeyboardInterrupt` during :func:`subprocess.call`, 1415:func:`subprocess.run`, or in a :class:`~subprocess.Popen` 1416context manager. It now waits a short amount of time for the child 1417to exit, before continuing the handling of the ``KeyboardInterrupt`` 1418exception. 1419(Contributed by Gregory P. Smith in :issue:`25942`.) 1420 1421 1422sys 1423--- 1424 1425The new :func:`sys.breakpointhook` hook function is called by the 1426built-in :func:`breakpoint`. 1427(Contributed by Barry Warsaw in :issue:`31353`.) 1428 1429On Android, the new :func:`sys.getandroidapilevel` returns the build-time 1430Android API version. 1431(Contributed by Victor Stinner in :issue:`28740`.) 1432 1433The new :func:`sys.get_coroutine_origin_tracking_depth` function returns 1434the current coroutine origin tracking depth, as set by 1435the new :func:`sys.set_coroutine_origin_tracking_depth`. :mod:`asyncio` 1436has been converted to use this new API instead of 1437the deprecated :func:`sys.set_coroutine_wrapper`. 1438(Contributed by Nathaniel J. Smith in :issue:`32591`.) 1439 1440 1441time 1442---- 1443 1444:pep:`564` adds six new functions with nanosecond resolution to the 1445:mod:`time` module: 1446 1447* :func:`time.clock_gettime_ns` 1448* :func:`time.clock_settime_ns` 1449* :func:`time.monotonic_ns` 1450* :func:`time.perf_counter_ns` 1451* :func:`time.process_time_ns` 1452* :func:`time.time_ns` 1453 1454New clock identifiers have been added: 1455 1456* :data:`time.CLOCK_BOOTTIME` (Linux): Identical to 1457 :data:`time.CLOCK_MONOTONIC`, except it also includes any time that the 1458 system is suspended. 1459* :data:`time.CLOCK_PROF` (FreeBSD, NetBSD and OpenBSD): High-resolution 1460 per-process CPU timer. 1461* :data:`time.CLOCK_UPTIME` (FreeBSD, OpenBSD): Time whose absolute value is 1462 the time the system has been running and not suspended, providing accurate 1463 uptime measurement. 1464 1465The new :func:`time.thread_time` and :func:`time.thread_time_ns` functions 1466can be used to get per-thread CPU time measurements. 1467(Contributed by Antoine Pitrou in :issue:`32025`.) 1468 1469The new :func:`time.pthread_getcpuclockid` function returns the clock ID 1470of the thread-specific CPU-time clock. 1471 1472 1473tkinter 1474------- 1475 1476The new :class:`tkinter.ttk.Spinbox` class is now available. 1477(Contributed by Alan Moore in :issue:`32585`.) 1478 1479 1480tracemalloc 1481----------- 1482 1483:class:`tracemalloc.Traceback` behaves more like regular tracebacks, 1484sorting the frames from oldest to most recent. 1485:meth:`Traceback.format() <tracemalloc.Traceback.format>` 1486now accepts negative *limit*, truncating the result to the 1487``abs(limit)`` oldest frames. To get the old behaviour, use 1488the new *most_recent_first* argument to ``Traceback.format()``. 1489(Contributed by Jesse Bakker in :issue:`32121`.) 1490 1491 1492types 1493----- 1494 1495The new :class:`~types.WrapperDescriptorType`, 1496:class:`~types.MethodWrapperType`, :class:`~types.MethodDescriptorType`, 1497and :class:`~types.ClassMethodDescriptorType` classes are now available. 1498(Contributed by Manuel Krebber and Guido van Rossum in :issue:`29377`, 1499and Serhiy Storchaka in :issue:`32265`.) 1500 1501The new :func:`types.resolve_bases` function resolves MRO entries 1502dynamically as specified by :pep:`560`. 1503(Contributed by Ivan Levkivskyi in :issue:`32717`.) 1504 1505 1506unicodedata 1507----------- 1508 1509The internal :mod:`unicodedata` database has been upgraded to use `Unicode 11 1510<http://www.unicode.org/versions/Unicode11.0.0/>`_. (Contributed by Benjamin 1511Peterson.) 1512 1513 1514unittest 1515-------- 1516 1517The new ``-k`` command-line option allows filtering tests by a name 1518substring or a Unix shell-like pattern. 1519For example, ``python -m unittest -k foo`` runs 1520``foo_tests.SomeTest.test_something``, ``bar_tests.SomeTest.test_foo``, 1521but not ``bar_tests.FooTest.test_something``. 1522(Contributed by Jonas Haag in :issue:`32071`.) 1523 1524 1525unittest.mock 1526------------- 1527 1528The :const:`~unittest.mock.sentinel` attributes now preserve their identity 1529when they are :mod:`copied <copy>` or :mod:`pickled <pickle>`. (Contributed by 1530Serhiy Storchaka in :issue:`20804`.) 1531 1532The new :func:`~unittest.mock.seal` function allows sealing 1533:class:`~unittest.mock.Mock` instances, which will disallow further creation 1534of attribute mocks. The seal is applied recursively to all attributes that 1535are themselves mocks. 1536(Contributed by Mario Corchero in :issue:`30541`.) 1537 1538 1539urllib.parse 1540------------ 1541 1542:func:`urllib.parse.quote` has been updated from :rfc:`2396` to :rfc:`3986`, 1543adding ``~`` to the set of characters that are never quoted by default. 1544(Contributed by Christian Theune and Ratnadeep Debnath in :issue:`16285`.) 1545 1546 1547uu 1548-- 1549 1550The :func:`uu.encode` function now accepts an optional *backtick* 1551keyword argument. When it's true, zeros are represented by ``'`'`` 1552instead of spaces. (Contributed by Xiang Zhang in :issue:`30103`.) 1553 1554 1555uuid 1556---- 1557 1558The new :attr:`UUID.is_safe <uuid.UUID.is_safe>` attribute relays information 1559from the platform about whether generated UUIDs are generated with a 1560multiprocessing-safe method. 1561(Contributed by Barry Warsaw in :issue:`22807`.) 1562 1563:func:`uuid.getnode` now prefers universally administered 1564MAC addresses over locally administered MAC addresses. 1565This makes a better guarantee for global uniqueness of UUIDs returned 1566from :func:`uuid.uuid1`. If only locally administered MAC addresses are 1567available, the first such one found is returned. 1568(Contributed by Barry Warsaw in :issue:`32107`.) 1569 1570 1571warnings 1572-------- 1573 1574The initialization of the default warnings filters has changed as follows: 1575 1576* warnings enabled via command line options (including those for :option:`-b` 1577 and the new CPython-specific :option:`-X` ``dev`` option) are always passed 1578 to the warnings machinery via the :data:`sys.warnoptions` attribute. 1579 1580* warnings filters enabled via the command line or the environment now have the 1581 following order of precedence: 1582 1583 * the ``BytesWarning`` filter for :option:`-b` (or ``-bb``) 1584 * any filters specified with the :option:`-W` option 1585 * any filters specified with the :envvar:`PYTHONWARNINGS` environment 1586 variable 1587 * any other CPython specific filters (e.g. the ``default`` filter added 1588 for the new ``-X dev`` mode) 1589 * any implicit filters defined directly by the warnings machinery 1590 1591* in :ref:`CPython debug builds <debug-build>`, all warnings are now displayed 1592 by default (the implicit filter list is empty) 1593 1594(Contributed by Nick Coghlan and Victor Stinner in :issue:`20361`, 1595:issue:`32043`, and :issue:`32230`.) 1596 1597Deprecation warnings are once again shown by default in single-file scripts and 1598at the interactive prompt. See :ref:`whatsnew37-pep565` for details. 1599(Contributed by Nick Coghlan in :issue:`31975`.) 1600 1601 1602xml.etree 1603--------- 1604 1605:ref:`ElementPath <elementtree-xpath>` predicates in the :meth:`find` 1606methods can now compare text of the current node with ``[. = "text"]``, 1607not only text in children. Predicates also allow adding spaces for 1608better readability. (Contributed by Stefan Behnel in :issue:`31648`.) 1609 1610 1611xmlrpc.server 1612------------- 1613 1614:meth:`SimpleXMLRPCDispatcher.register_function <xmlrpc.server.SimpleXMLRPCDispatcher>` 1615can now be used as a decorator. (Contributed by Xiang Zhang in 1616:issue:`7769`.) 1617 1618 1619zipapp 1620------ 1621 1622Function :func:`~zipapp.create_archive` now accepts an optional *filter* 1623argument to allow the user to select which files should be included in the 1624archive. (Contributed by Irmen de Jong in :issue:`31072`.) 1625 1626Function :func:`~zipapp.create_archive` now accepts an optional *compressed* 1627argument to generate a compressed archive. A command line option 1628``--compress`` has also been added to support compression. 1629(Contributed by Zhiming Wang in :issue:`31638`.) 1630 1631 1632zipfile 1633------- 1634 1635:class:`~zipfile.ZipFile` now accepts the new *compresslevel* parameter to 1636control the compression level. 1637(Contributed by Bo Bayles in :issue:`21417`.) 1638 1639Subdirectories in archives created by ``ZipFile`` are now stored in 1640alphabetical order. 1641(Contributed by Bernhard M. Wiedemann in :issue:`30693`.) 1642 1643 1644C API Changes 1645============= 1646 1647A new API for thread-local storage has been implemented. See 1648:ref:`whatsnew37-pep539` for an overview and 1649:ref:`thread-specific-storage-api` for a complete reference. 1650(Contributed by Masayuki Yamamoto in :issue:`25658`.) 1651 1652The new :ref:`context variables <whatsnew37-pep567>` functionality 1653exposes a number of :ref:`new C APIs <contextvarsobjects>`. 1654 1655The new :c:func:`PyImport_GetModule` function returns the previously 1656imported module with the given name. 1657(Contributed by Eric Snow in :issue:`28411`.) 1658 1659The new :c:macro:`Py_RETURN_RICHCOMPARE` macro eases writing rich 1660comparison functions. 1661(Contributed by Petr Victorin in :issue:`23699`.) 1662 1663The new :c:macro:`Py_UNREACHABLE` macro can be used to mark unreachable 1664code paths. 1665(Contributed by Barry Warsaw in :issue:`31338`.) 1666 1667The :mod:`tracemalloc` now exposes a C API through the new 1668:c:func:`PyTraceMalloc_Track` and :c:func:`PyTraceMalloc_Untrack` 1669functions. 1670(Contributed by Victor Stinner in :issue:`30054`.) 1671 1672The new :c:func:`import__find__load__start` and 1673:c:func:`import__find__load__done` static markers can be used to trace 1674module imports. 1675(Contributed by Christian Heimes in :issue:`31574`.) 1676 1677The fields :c:member:`name` and :c:member:`doc` of structures 1678:c:type:`PyMemberDef`, :c:type:`PyGetSetDef`, 1679:c:type:`PyStructSequence_Field`, :c:type:`PyStructSequence_Desc`, 1680and :c:type:`wrapperbase` are now of type ``const char *`` rather of 1681``char *``. (Contributed by Serhiy Storchaka in :issue:`28761`.) 1682 1683The result of :c:func:`PyUnicode_AsUTF8AndSize` and :c:func:`PyUnicode_AsUTF8` 1684is now of type ``const char *`` rather of ``char *``. (Contributed by Serhiy 1685Storchaka in :issue:`28769`.) 1686 1687The result of :c:func:`PyMapping_Keys`, :c:func:`PyMapping_Values` and 1688:c:func:`PyMapping_Items` is now always a list, rather than a list or a 1689tuple. (Contributed by Oren Milman in :issue:`28280`.) 1690 1691Added functions :c:func:`PySlice_Unpack` and :c:func:`PySlice_AdjustIndices`. 1692(Contributed by Serhiy Storchaka in :issue:`27867`.) 1693 1694:c:func:`PyOS_AfterFork` is deprecated in favour of the new functions 1695:c:func:`PyOS_BeforeFork`, :c:func:`PyOS_AfterFork_Parent` and 1696:c:func:`PyOS_AfterFork_Child`. (Contributed by Antoine Pitrou in 1697:issue:`16500`.) 1698 1699The ``PyExc_RecursionErrorInst`` singleton that was part of the public API 1700has been removed as its members being never cleared may cause a segfault 1701during finalization of the interpreter. Contributed by Xavier de Gaye in 1702:issue:`22898` and :issue:`30697`. 1703 1704Added C API support for timezones with timezone constructors 1705:c:func:`PyTimeZone_FromOffset` and :c:func:`PyTimeZone_FromOffsetAndName`, 1706and access to the UTC singleton with :c:data:`PyDateTime_TimeZone_UTC`. 1707Contributed by Paul Ganssle in :issue:`10381`. 1708 1709The type of results of :c:func:`PyThread_start_new_thread` and 1710:c:func:`PyThread_get_thread_ident`, and the *id* parameter of 1711:c:func:`PyThreadState_SetAsyncExc` changed from :c:type:`long` to 1712:c:type:`unsigned long`. 1713(Contributed by Serhiy Storchaka in :issue:`6532`.) 1714 1715:c:func:`PyUnicode_AsWideCharString` now raises a :exc:`ValueError` if the 1716second argument is ``NULL`` and the :c:type:`wchar_t*` string contains null 1717characters. (Contributed by Serhiy Storchaka in :issue:`30708`.) 1718 1719Changes to the startup sequence and the management of dynamic memory 1720allocators mean that the long documented requirement to call 1721:c:func:`Py_Initialize` before calling most C API functions is now 1722relied on more heavily, and failing to abide by it may lead to segfaults in 1723embedding applications. See the :ref:`porting-to-python-37` section in this 1724document and the :ref:`pre-init-safe` section in the C API documentation 1725for more details. 1726 1727The new :c:func:`PyInterpreterState_GetID` returns the unique ID for a 1728given interpreter. 1729(Contributed by Eric Snow in :issue:`29102`.) 1730 1731:c:func:`Py_DecodeLocale`, :c:func:`Py_EncodeLocale` now use the UTF-8 1732encoding when the :ref:`UTF-8 mode <whatsnew37-pep540>` is enabled. 1733(Contributed by Victor Stinner in :issue:`29240`.) 1734 1735:c:func:`PyUnicode_DecodeLocaleAndSize` and :c:func:`PyUnicode_EncodeLocale` 1736now use the current locale encoding for ``surrogateescape`` error handler. 1737(Contributed by Victor Stinner in :issue:`29240`.) 1738 1739The *start* and *end* parameters of :c:func:`PyUnicode_FindChar` are 1740now adjusted to behave like string slices. 1741(Contributed by Xiang Zhang in :issue:`28822`.) 1742 1743 1744Build Changes 1745============= 1746 1747Support for building ``--without-threads`` has been removed. The 1748:mod:`threading` module is now always available. 1749(Contributed by Antoine Pitrou in :issue:`31370`.). 1750 1751A full copy of libffi is no longer bundled for use when building the 1752:mod:`_ctypes <ctypes>` module on non-OSX UNIX platforms. An installed copy 1753of libffi is now required when building ``_ctypes`` on such platforms. 1754(Contributed by Zachary Ware in :issue:`27979`.) 1755 1756The Windows build process no longer depends on Subversion to pull in external 1757sources, a Python script is used to download zipfiles from GitHub instead. 1758If Python 3.6 is not found on the system (via ``py -3.6``), NuGet is used to 1759download a copy of 32-bit Python for this purpose. (Contributed by Zachary 1760Ware in :issue:`30450`.) 1761 1762The :mod:`ssl` module requires OpenSSL 1.0.2 or 1.1 compatible libssl. 1763OpenSSL 1.0.1 has reached end of lifetime on 2016-12-31 and is no longer 1764supported. LibreSSL is temporarily not supported as well. LibreSSL releases 1765up to version 2.6.4 are missing required OpenSSL 1.0.2 APIs. 1766 1767 1768.. _whatsnew37-perf: 1769 1770Optimizations 1771============= 1772 1773The overhead of calling many methods of various standard library classes 1774implemented in C has been significantly reduced by porting more code 1775to use the ``METH_FASTCALL`` convention. 1776(Contributed by Victor Stinner in :issue:`29300`, :issue:`29507`, 1777:issue:`29452`, and :issue:`29286`.) 1778 1779Various optimizations have reduced Python startup time by 10% on Linux and 1780up to 30% on macOS. 1781(Contributed by Victor Stinner, INADA Naoki in :issue:`29585`, and 1782Ivan Levkivskyi in :issue:`31333`.) 1783 1784Method calls are now up to 20% faster due to the bytecode changes which 1785avoid creating bound method instances. 1786(Contributed by Yury Selivanov and INADA Naoki in :issue:`26110`.) 1787 1788.. _whatsnew37-asyncio-perf: 1789 1790The :mod:`asyncio` module received a number of notable optimizations for 1791commonly used functions: 1792 1793* The :func:`asyncio.get_event_loop` function has been reimplemented in C to 1794 make it up to 15 times faster. 1795 (Contributed by Yury Selivanov in :issue:`32296`.) 1796 1797* :class:`asyncio.Future` callback management has been optimized. 1798 (Contributed by Yury Selivanov in :issue:`32348`.) 1799 1800* :func:`asyncio.gather` is now up to 15% faster. 1801 (Contributed by Yury Selivanov in :issue:`32355`.) 1802 1803* :func:`asyncio.sleep` is now up to 2 times faster when the *delay* 1804 argument is zero or negative. 1805 (Contributed by Andrew Svetlov in :issue:`32351`.) 1806 1807* The performance overhead of asyncio debug mode has been reduced. 1808 (Contributed by Antoine Pitrou in :issue:`31970`.) 1809 1810As a result of :ref:`PEP 560 work <whatsnew37-pep560>`, the import time 1811of :mod:`typing` has been reduced by a factor of 7, and many typing operations 1812are now faster. 1813(Contributed by Ivan Levkivskyi in :issue:`32226`.) 1814 1815:func:`sorted` and :meth:`list.sort` have been optimized for common cases 1816to be up to 40-75% faster. 1817(Contributed by Elliot Gorokhovsky in :issue:`28685`.) 1818 1819:meth:`dict.copy` is now up to 5.5 times faster. 1820(Contributed by Yury Selivanov in :issue:`31179`.) 1821 1822:func:`hasattr` and :func:`getattr` are now about 4 times faster when 1823*name* is not found and *obj* does not override :meth:`object.__getattr__` 1824or :meth:`object.__getattribute__`. 1825(Contributed by INADA Naoki in :issue:`32544`.) 1826 1827Searching for certain Unicode characters (like Ukrainian capital "Є") 1828in a string was up to 25 times slower than searching for other characters. 1829It is now only 3 times slower in the worst case. 1830(Contributed by Serhiy Storchaka in :issue:`24821`.) 1831 1832The :func:`collections.namedtuple` factory has been reimplemented to 1833make the creation of named tuples 4 to 6 times faster. 1834(Contributed by Jelle Zijlstra with further improvements by INADA Naoki, 1835Serhiy Storchaka, and Raymond Hettinger in :issue:`28638`.) 1836 1837:meth:`date.fromordinal` and :meth:`date.fromtimestamp` are now up to 183830% faster in the common case. 1839(Contributed by Paul Ganssle in :issue:`32403`.) 1840 1841The :func:`os.fwalk` function is now up to 2 times faster thanks to 1842the use of :func:`os.scandir`. 1843(Contributed by Serhiy Storchaka in :issue:`25996`.) 1844 1845The speed of the :func:`shutil.rmtree` function has been improved by 184620--40% thanks to the use of the :func:`os.scandir` function. 1847(Contributed by Serhiy Storchaka in :issue:`28564`.) 1848 1849Optimized case-insensitive matching and searching of :mod:`regular 1850expressions <re>`. Searching some patterns can now be up to 20 times faster. 1851(Contributed by Serhiy Storchaka in :issue:`30285`.) 1852 1853:func:`re.compile` now converts ``flags`` parameter to int object if 1854it is ``RegexFlag``. It is now as fast as Python 3.5, and faster than 1855Python 3.6 by about 10% depending on the pattern. 1856(Contributed by INADA Naoki in :issue:`31671`.) 1857 1858The :meth:`~selectors.BaseSelector.modify` methods of classes 1859:class:`selectors.EpollSelector`, :class:`selectors.PollSelector` 1860and :class:`selectors.DevpollSelector` may be around 10% faster under 1861heavy loads. (Contributed by Giampaolo Rodola' in :issue:`30014`) 1862 1863Constant folding has been moved from the peephole optimizer to the new AST 1864optimizer, which is able perform optimizations more consistently. 1865(Contributed by Eugene Toder and INADA Naoki in :issue:`29469` and 1866:issue:`11549`.) 1867 1868Most functions and methods in :mod:`abc` have been rewritten in C. 1869This makes creation of abstract base classes, and calling :func:`isinstance` 1870and :func:`issubclass` on them 1.5x faster. This also reduces Python 1871start-up time by up to 10%. (Contributed by Ivan Levkivskyi and INADA Naoki 1872in :issue:`31333`) 1873 1874Significant speed improvements to alternate constructors for 1875:class:`datetime.date` and :class:`datetime.datetime` by using fast-path 1876constructors when not constructing subclasses. (Contributed by Paul Ganssle 1877in :issue:`32403`) 1878 1879The speed of comparison of :class:`array.array` instances has been 1880improved considerably in certain cases. It is now from 10x to 70x faster 1881when comparing arrays holding values of the same integer type. 1882(Contributed by Adrian Wielgosik in :issue:`24700`.) 1883 1884The :func:`math.erf` and :func:`math.erfc` functions now use the (faster) 1885C library implementation on most platforms. 1886(Contributed by Serhiy Storchaka in :issue:`26121`.) 1887 1888 1889Other CPython Implementation Changes 1890==================================== 1891 1892* Trace hooks may now opt out of receiving the ``line`` and opt into receiving 1893 the ``opcode`` events from the interpreter by setting the corresponding new 1894 ``f_trace_lines`` and ``f_trace_opcodes`` attributes on the 1895 frame being traced. (Contributed by Nick Coghlan in :issue:`31344`.) 1896 1897* Fixed some consistency problems with namespace package module attributes. 1898 Namespace module objects now have an ``__file__`` that is set to ``None`` 1899 (previously unset), and their ``__spec__.origin`` is also set to ``None`` 1900 (previously the string ``"namespace"``). See :issue:`32305`. Also, the 1901 namespace module object's ``__spec__.loader`` is set to the same value as 1902 ``__loader__`` (previously, the former was set to ``None``). See 1903 :issue:`32303`. 1904 1905* The :func:`locals` dictionary now displays in the lexical order that 1906 variables were defined. Previously, the order was undefined. 1907 (Contributed by Raymond Hettinger in :issue:`32690`.) 1908 1909* The :mod:`distutils` ``upload`` command no longer tries to change CR 1910 end-of-line characters to CRLF. This fixes a corruption issue with sdists 1911 that ended with a byte equivalent to CR. 1912 (Contributed by Bo Bayles in :issue:`32304`.) 1913 1914 1915Deprecated Python Behavior 1916========================== 1917 1918Yield expressions (both ``yield`` and ``yield from`` clauses) are now deprecated 1919in comprehensions and generator expressions (aside from the iterable expression 1920in the leftmost :keyword:`!for` clause). This ensures that comprehensions 1921always immediately return a container of the appropriate type (rather than 1922potentially returning a :term:`generator iterator` object), while generator 1923expressions won't attempt to interleave their implicit output with the output 1924from any explicit yield expressions. In Python 3.7, such expressions emit 1925:exc:`DeprecationWarning` when compiled, in Python 3.8 this will be a 1926:exc:`SyntaxError`. 1927(Contributed by Serhiy Storchaka in :issue:`10544`.) 1928 1929Returning a subclass of :class:`complex` from :meth:`object.__complex__` is 1930deprecated and will be an error in future Python versions. This makes 1931``__complex__()`` consistent with :meth:`object.__int__` and 1932:meth:`object.__float__`. 1933(Contributed by Serhiy Storchaka in :issue:`28894`.) 1934 1935 1936 1937Deprecated Python modules, functions and methods 1938================================================ 1939 1940aifc 1941---- 1942 1943:func:`aifc.openfp` has been deprecated and will be removed in Python 3.9. 1944Use :func:`aifc.open` instead. 1945(Contributed by Brian Curtin in :issue:`31985`.) 1946 1947 1948.. _whatsnew37-asyncio-deprecated: 1949 1950asyncio 1951------- 1952 1953Support for directly ``await``-ing instances of :class:`asyncio.Lock` and 1954other asyncio synchronization primitives has been deprecated. An 1955asynchronous context manager must be used in order to acquire and release 1956the synchronization resource. 1957(Contributed by Andrew Svetlov in :issue:`32253`.) 1958 1959The :meth:`asyncio.Task.current_task` and :meth:`asyncio.Task.all_tasks` 1960methods have been deprecated. 1961(Contributed by Andrew Svetlov in :issue:`32250`.) 1962 1963 1964collections 1965----------- 1966 1967In Python 3.8, the abstract base classes in :mod:`collections.abc` will no 1968longer be exposed in the regular :mod:`collections` module. This will help 1969create a clearer distinction between the concrete classes and the abstract 1970base classes. 1971(Contributed by Serhiy Storchaka in :issue:`25988`.) 1972 1973 1974dbm 1975--- 1976 1977:mod:`dbm.dumb` now supports reading read-only files and no longer writes the 1978index file when it is not changed. A deprecation warning is now emitted 1979if the index file is missing and recreated in the ``'r'`` and ``'w'`` 1980modes (this will be an error in future Python releases). 1981(Contributed by Serhiy Storchaka in :issue:`28847`.) 1982 1983 1984enum 1985---- 1986 1987In Python 3.8, attempting to check for non-Enum objects in :class:`Enum` 1988classes will raise a :exc:`TypeError` (e.g. ``1 in Color``); similarly, 1989attempting to check for non-Flag objects in a :class:`Flag` member will 1990raise :exc:`TypeError` (e.g. ``1 in Perm.RW``); currently, both operations 1991return :const:`False` instead. 1992(Contributed by Ethan Furman in :issue:`33217`.) 1993 1994 1995gettext 1996------- 1997 1998Using non-integer value for selecting a plural form in :mod:`gettext` is 1999now deprecated. It never correctly worked. (Contributed by Serhiy Storchaka 2000in :issue:`28692`.) 2001 2002 2003importlib 2004--------- 2005 2006Methods 2007:meth:`MetaPathFinder.find_module() <importlib.abc.MetaPathFinder.find_module>` 2008(replaced by 2009:meth:`MetaPathFinder.find_spec() <importlib.abc.MetaPathFinder.find_spec>`) 2010and 2011:meth:`PathEntryFinder.find_loader() <importlib.abc.PathEntryFinder.find_loader>` 2012(replaced by 2013:meth:`PathEntryFinder.find_spec() <importlib.abc.PathEntryFinder.find_spec>`) 2014both deprecated in Python 3.4 now emit :exc:`DeprecationWarning`. 2015(Contributed by Matthias Bussonnier in :issue:`29576`) 2016 2017The :class:`importlib.abc.ResourceLoader` ABC has been deprecated in 2018favour of :class:`importlib.abc.ResourceReader`. 2019 2020 2021locale 2022------ 2023 2024:func:`locale.format` has been deprecated, use :meth:`locale.format_string` 2025instead. (Contributed by Garvit in :issue:`10379`.) 2026 2027 2028macpath 2029------- 2030 2031The :mod:`macpath` is now deprecated and will be removed in Python 3.8. 2032(Contributed by Chi Hsuan Yen in :issue:`9850`.) 2033 2034 2035threading 2036--------- 2037 2038:mod:`dummy_threading` and :mod:`_dummy_thread` have been deprecated. It is 2039no longer possible to build Python with threading disabled. 2040Use :mod:`threading` instead. 2041(Contributed by Antoine Pitrou in :issue:`31370`.) 2042 2043 2044socket 2045------ 2046 2047The silent argument value truncation in :func:`socket.htons` and 2048:func:`socket.ntohs` has been deprecated. In future versions of Python, 2049if the passed argument is larger than 16 bits, an exception will be raised. 2050(Contributed by Oren Milman in :issue:`28332`.) 2051 2052 2053ssl 2054--- 2055 2056:func:`ssl.wrap_socket` is deprecated. Use 2057:meth:`ssl.SSLContext.wrap_socket` instead. 2058(Contributed by Christian Heimes in :issue:`28124`.) 2059 2060 2061sunau 2062----- 2063 2064:func:`sunau.openfp` has been deprecated and will be removed in Python 3.9. 2065Use :func:`sunau.open` instead. 2066(Contributed by Brian Curtin in :issue:`31985`.) 2067 2068 2069sys 2070--- 2071 2072Deprecated :func:`sys.set_coroutine_wrapper` and 2073:func:`sys.get_coroutine_wrapper`. 2074 2075The undocumented ``sys.callstats()`` function has been deprecated and 2076will be removed in a future Python version. 2077(Contributed by Victor Stinner in :issue:`28799`.) 2078 2079 2080wave 2081---- 2082 2083:func:`wave.openfp` has been deprecated and will be removed in Python 3.9. 2084Use :func:`wave.open` instead. 2085(Contributed by Brian Curtin in :issue:`31985`.) 2086 2087 2088Deprecated functions and types of the C API 2089=========================================== 2090 2091Function :c:func:`PySlice_GetIndicesEx` is deprecated and replaced with 2092a macro if ``Py_LIMITED_API`` is not set or set to a value in the range 2093between ``0x03050400`` and ``0x03060000`` (not inclusive), or is ``0x03060100`` 2094or higher. (Contributed by Serhiy Storchaka in :issue:`27867`.) 2095 2096:c:func:`PyOS_AfterFork` has been deprecated. Use :c:func:`PyOS_BeforeFork`, 2097:c:func:`PyOS_AfterFork_Parent` or :c:func:`PyOS_AfterFork_Child()` instead. 2098(Contributed by Antoine Pitrou in :issue:`16500`.) 2099 2100 2101.. _37-platform-support-removals: 2102 2103Platform Support Removals 2104========================= 2105 2106* FreeBSD 9 and older are no longer officially supported. 2107* For full Unicode support, including within extension modules, \*nix platforms 2108 are now expected to provide at least one of ``C.UTF-8`` (full locale), 2109 ``C.utf8`` (full locale) or ``UTF-8`` (``LC_CTYPE``-only locale) as an 2110 alternative to the legacy ``ASCII``-based ``C`` locale. 2111* OpenSSL 0.9.8 and 1.0.1 are no longer supported, which means building CPython 2112 3.7 with SSL/TLS support on older platforms still using these versions 2113 requires custom build options that link to a more recent version of OpenSSL. 2114 2115 Notably, this issue affects the Debian 8 (aka "jessie") and Ubuntu 14.04 2116 (aka "Trusty") LTS Linux distributions, as they still use OpenSSL 1.0.1 by 2117 default. 2118 2119 Debian 9 ("stretch") and Ubuntu 16.04 ("xenial"), as well as recent releases 2120 of other LTS Linux releases (e.g. RHEL/CentOS 7.5, SLES 12-SP3), use OpenSSL 2121 1.0.2 or later, and remain supported in the default build configuration. 2122 2123 CPython's own :source:`CI configuration file <.travis.yml>` provides an 2124 example of using the SSL 2125 :source:`compatibility testing infrastructure <Tools/ssl/multissltests.py>` in 2126 CPython's test suite to build and link against OpenSSL 1.1.0 rather than an 2127 outdated system provided OpenSSL. 2128 2129 2130API and Feature Removals 2131======================== 2132 2133The following features and APIs have been removed from Python 3.7: 2134 2135* The ``os.stat_float_times()`` function has been removed. It was introduced in 2136 Python 2.3 for backward compatibility with Python 2.2, and was deprecated 2137 since Python 3.1. 2138 2139* Unknown escapes consisting of ``'\'`` and an ASCII letter in replacement 2140 templates for :func:`re.sub` were deprecated in Python 3.5, and will now 2141 cause an error. 2142 2143* Removed support of the *exclude* argument in :meth:`tarfile.TarFile.add`. 2144 It was deprecated in Python 2.7 and 3.2. Use the *filter* argument instead. 2145 2146* The ``splitunc()`` function in the :mod:`ntpath` module was deprecated in 2147 Python 3.1, and has now been removed. Use the :func:`~os.path.splitdrive` 2148 function instead. 2149 2150* :func:`collections.namedtuple` no longer supports the *verbose* parameter 2151 or ``_source`` attribute which showed the generated source code for the 2152 named tuple class. This was part of an optimization designed to speed-up 2153 class creation. (Contributed by Jelle Zijlstra with further improvements 2154 by INADA Naoki, Serhiy Storchaka, and Raymond Hettinger in :issue:`28638`.) 2155 2156* Functions :func:`bool`, :func:`float`, :func:`list` and :func:`tuple` no 2157 longer take keyword arguments. The first argument of :func:`int` can now 2158 be passed only as positional argument. 2159 2160* Removed previously deprecated in Python 2.4 classes ``Plist``, ``Dict`` and 2161 ``_InternalDict`` in the :mod:`plistlib` module. Dict values in the result 2162 of functions :func:`~plistlib.readPlist` and 2163 :func:`~plistlib.readPlistFromBytes` are now normal dicts. You no longer 2164 can use attribute access to access items of these dictionaries. 2165 2166* The ``asyncio.windows_utils.socketpair()`` function has been 2167 removed. Use the :func:`socket.socketpair` function instead, 2168 it is available on all platforms since Python 3.5. 2169 ``asyncio.windows_utils.socketpair`` was just an alias to 2170 ``socket.socketpair`` on Python 3.5 and newer. 2171 2172* :mod:`asyncio` no longer exports the :mod:`selectors` and 2173 :mod:`_overlapped` modules as ``asyncio.selectors`` and 2174 ``asyncio._overlapped``. Replace ``from asyncio import selectors`` with 2175 ``import selectors``. 2176 2177* Direct instantiation of :class:`ssl.SSLSocket` and :class:`ssl.SSLObject` 2178 objects is now prohibited. The constructors were never documented, tested, 2179 or designed as public constructors. Users were supposed to use 2180 :func:`ssl.wrap_socket` or :class:`ssl.SSLContext`. 2181 (Contributed by Christian Heimes in :issue:`32951`.) 2182 2183* The unused :mod:`distutils` ``install_misc`` command has been removed. 2184 (Contributed by Eric N. Vander Weele in :issue:`29218`.) 2185 2186 2187Module Removals 2188=============== 2189 2190The ``fpectl`` module has been removed. It was never enabled by 2191default, never worked correctly on x86-64, and it changed the Python 2192ABI in ways that caused unexpected breakage of C extensions. 2193(Contributed by Nathaniel J. Smith in :issue:`29137`.) 2194 2195 2196Windows-only Changes 2197==================== 2198 2199The python launcher, (py.exe), can accept 32 & 64 bit specifiers **without** 2200having to specify a minor version as well. So ``py -3-32`` and ``py -3-64`` 2201become valid as well as ``py -3.7-32``, also the -*m*-64 and -*m.n*-64 forms 2202are now accepted to force 64 bit python even if 32 bit would have otherwise 2203been used. If the specified version is not available py.exe will error exit. 2204(Contributed by Steve Barnes in :issue:`30291`.) 2205 2206The launcher can be run as ``py -0`` to produce a list of the installed pythons, 2207*with default marked with an asterisk*. Running ``py -0p`` will include the paths. 2208If py is run with a version specifier that cannot be matched it will also print 2209the *short form* list of available specifiers. 2210(Contributed by Steve Barnes in :issue:`30362`.) 2211 2212 2213.. _porting-to-python-37: 2214 2215Porting to Python 3.7 2216===================== 2217 2218This section lists previously described changes and other bugfixes 2219that may require changes to your code. 2220 2221 2222Changes in Python Behavior 2223-------------------------- 2224 2225* :keyword:`async` and :keyword:`await` names are now reserved keywords. 2226 Code using these names as identifiers will now raise a :exc:`SyntaxError`. 2227 (Contributed by Jelle Zijlstra in :issue:`30406`.) 2228 2229* :pep:`479` is enabled for all code in Python 3.7, meaning that 2230 :exc:`StopIteration` exceptions raised directly or indirectly in 2231 coroutines and generators are transformed into :exc:`RuntimeError` 2232 exceptions. 2233 (Contributed by Yury Selivanov in :issue:`32670`.) 2234 2235* :meth:`object.__aiter__` methods can no longer be declared as 2236 asynchronous. (Contributed by Yury Selivanov in :issue:`31709`.) 2237 2238* Due to an oversight, earlier Python versions erroneously accepted the 2239 following syntax:: 2240 2241 f(1 for x in [1],) 2242 2243 class C(1 for x in [1]): 2244 pass 2245 2246 Python 3.7 now correctly raises a :exc:`SyntaxError`, as a generator 2247 expression always needs to be directly inside a set of parentheses 2248 and cannot have a comma on either side, and the duplication of the 2249 parentheses can be omitted only on calls. 2250 (Contributed by Serhiy Storchaka in :issue:`32012` and :issue:`32023`.) 2251 2252* When using the :option:`-m` switch, the initial working directory is now added 2253 to :data:`sys.path`, rather than an empty string (which dynamically denoted 2254 the current working directory at the time of each import). Any programs that 2255 are checking for the empty string, or otherwise relying on the previous 2256 behaviour, will need to be updated accordingly (e.g. by also checking for 2257 ``os.getcwd()`` or ``os.path.dirname(__main__.__file__)``, depending on why 2258 the code was checking for the empty string in the first place). 2259 2260 2261Changes in the Python API 2262------------------------- 2263 2264* :meth:`socketserver.ThreadingMixIn.server_close` now waits until all 2265 non-daemon threads complete. Set the new 2266 :attr:`socketserver.ThreadingMixIn.block_on_close` class attribute to 2267 ``False`` to get the pre-3.7 behaviour. 2268 (Contributed by Victor Stinner in :issue:`31233` and :issue:`33540`.) 2269 2270* :meth:`socketserver.ForkingMixIn.server_close` now waits until all 2271 child processes complete. Set the new 2272 :attr:`socketserver.ForkingMixIn.block_on_close` class attribute to ``False`` 2273 to get the pre-3.7 behaviour. 2274 (Contributed by Victor Stinner in :issue:`31151` and :issue:`33540`.) 2275 2276* The :func:`locale.localeconv` function now temporarily sets the ``LC_CTYPE`` 2277 locale to the value of ``LC_NUMERIC`` in some cases. 2278 (Contributed by Victor Stinner in :issue:`31900`.) 2279 2280* :meth:`pkgutil.walk_packages` now raises a :exc:`ValueError` if *path* is 2281 a string. Previously an empty list was returned. 2282 (Contributed by Sanyam Khurana in :issue:`24744`.) 2283 2284* A format string argument for :meth:`string.Formatter.format` 2285 is now :ref:`positional-only <positional-only_parameter>`. 2286 Passing it as a keyword argument was deprecated in Python 3.5. (Contributed 2287 by Serhiy Storchaka in :issue:`29193`.) 2288 2289* Attributes :attr:`~http.cookies.Morsel.key`, 2290 :attr:`~http.cookies.Morsel.value` and 2291 :attr:`~http.cookies.Morsel.coded_value` of class 2292 :class:`http.cookies.Morsel` are now read-only. 2293 Assigning to them was deprecated in Python 3.5. 2294 Use the :meth:`~http.cookies.Morsel.set` method for setting them. 2295 (Contributed by Serhiy Storchaka in :issue:`29192`.) 2296 2297* The *mode* argument of :func:`os.makedirs` no longer affects the file 2298 permission bits of newly-created intermediate-level directories. 2299 To set their file permission bits you can set the umask before invoking 2300 ``makedirs()``. 2301 (Contributed by Serhiy Storchaka in :issue:`19930`.) 2302 2303* The :attr:`struct.Struct.format` type is now :class:`str` instead of 2304 :class:`bytes`. (Contributed by Victor Stinner in :issue:`21071`.) 2305 2306* :func:`~cgi.parse_multipart` now accepts the *encoding* and *errors* 2307 arguments and returns the same results as 2308 :class:`~FieldStorage`: for non-file fields, the value associated to a key 2309 is a list of strings, not bytes. 2310 (Contributed by Pierre Quentel in :issue:`29979`.) 2311 2312* Due to internal changes in :mod:`socket`, calling :func:`socket.fromshare` 2313 on a socket created by :func:`socket.share <socket.socket.share>` in older 2314 Python versions is not supported. 2315 2316* ``repr`` for :exc:`BaseException` has changed to not include the trailing 2317 comma. Most exceptions are affected by this change. 2318 (Contributed by Serhiy Storchaka in :issue:`30399`.) 2319 2320* ``repr`` for :class:`datetime.timedelta` has changed to include the keyword 2321 arguments in the output. (Contributed by Utkarsh Upadhyay in :issue:`30302`.) 2322 2323* Because :func:`shutil.rmtree` is now implemented using the :func:`os.scandir` 2324 function, the user specified handler *onerror* is now called with the first 2325 argument ``os.scandir`` instead of ``os.listdir`` when listing the directory 2326 is failed. 2327 2328* Support for nested sets and set operations in regular expressions as in 2329 `Unicode Technical Standard #18`_ might be added in the future. This would 2330 change the syntax. To facilitate this future change a :exc:`FutureWarning` 2331 will be raised in ambiguous cases for the time being. 2332 That include sets starting with a literal ``'['`` or containing literal 2333 character sequences ``'--'``, ``'&&'``, ``'~~'``, and ``'||'``. To 2334 avoid a warning, escape them with a backslash. 2335 (Contributed by Serhiy Storchaka in :issue:`30349`.) 2336 2337 .. _Unicode Technical Standard #18: https://unicode.org/reports/tr18/ 2338 2339* The result of splitting a string on a :mod:`regular expression <re>` 2340 that could match an empty string has been changed. For example 2341 splitting on ``r'\s*'`` will now split not only on whitespaces as it 2342 did previously, but also on empty strings before all non-whitespace 2343 characters and just before the end of the string. 2344 The previous behavior can be restored by changing the pattern 2345 to ``r'\s+'``. A :exc:`FutureWarning` was emitted for such patterns since 2346 Python 3.5. 2347 2348 For patterns that match both empty and non-empty strings, the result of 2349 searching for all matches may also be changed in other cases. For example 2350 in the string ``'a\n\n'``, the pattern ``r'(?m)^\s*?$'`` will not only 2351 match empty strings at positions 2 and 3, but also the string ``'\n'`` at 2352 positions 2--3. To match only blank lines, the pattern should be rewritten 2353 as ``r'(?m)^[^\S\n]*$'``. 2354 2355 :func:`re.sub()` now replaces empty matches adjacent to a previous 2356 non-empty match. For example ``re.sub('x*', '-', 'abxd')`` returns now 2357 ``'-a-b--d-'`` instead of ``'-a-b-d-'`` (the first minus between 'b' and 2358 'd' replaces 'x', and the second minus replaces an empty string between 2359 'x' and 'd'). 2360 2361 (Contributed by Serhiy Storchaka in :issue:`25054` and :issue:`32308`.) 2362 2363* Change :func:`re.escape` to only escape regex special characters instead 2364 of escaping all characters other than ASCII letters, numbers, and ``'_'``. 2365 (Contributed by Serhiy Storchaka in :issue:`29995`.) 2366 2367* :class:`tracemalloc.Traceback` frames are now sorted from oldest to most 2368 recent to be more consistent with :mod:`traceback`. 2369 (Contributed by Jesse Bakker in :issue:`32121`.) 2370 2371* On OSes that support :const:`socket.SOCK_NONBLOCK` or 2372 :const:`socket.SOCK_CLOEXEC` bit flags, the 2373 :attr:`socket.type <socket.socket.type>` no longer has them applied. 2374 Therefore, checks like ``if sock.type == socket.SOCK_STREAM`` 2375 work as expected on all platforms. 2376 (Contributed by Yury Selivanov in :issue:`32331`.) 2377 2378* On Windows the default for the *close_fds* argument of 2379 :class:`subprocess.Popen` was changed from :const:`False` to :const:`True` 2380 when redirecting the standard handles. If you previously depended on handles 2381 being inherited when using :class:`subprocess.Popen` with standard io 2382 redirection, you will have to pass ``close_fds=False`` to preserve the 2383 previous behaviour, or use 2384 :attr:`STARTUPINFO.lpAttributeList <subprocess.STARTUPINFO.lpAttributeList>`. 2385 2386* :meth:`importlib.machinery.PathFinder.invalidate_caches` -- which implicitly 2387 affects :func:`importlib.invalidate_caches` -- now deletes entries 2388 in :data:`sys.path_importer_cache` which are set to ``None``. 2389 (Contributed by Brett Cannon in :issue:`33169`.) 2390 2391* In :mod:`asyncio`, 2392 :meth:`loop.sock_recv() <asyncio.loop.sock_recv>`, 2393 :meth:`loop.sock_sendall() <asyncio.loop.sock_sendall>`, 2394 :meth:`loop.sock_accept() <asyncio.loop.sock_accept>`, 2395 :meth:`loop.getaddrinfo() <asyncio.loop.getaddrinfo>`, 2396 :meth:`loop.getnameinfo() <asyncio.loop.getnameinfo>` 2397 have been changed to be proper coroutine methods to match their 2398 documentation. Previously, these methods returned :class:`asyncio.Future` 2399 instances. 2400 (Contributed by Yury Selivanov in :issue:`32327`.) 2401 2402* :attr:`asyncio.Server.sockets` now returns a copy of the internal list 2403 of server sockets, instead of returning it directly. 2404 (Contributed by Yury Selivanov in :issue:`32662`.) 2405 2406* :attr:`Struct.format <struct.Struct.format>` is now a :class:`str` instance 2407 instead of a :class:`bytes` instance. 2408 (Contributed by Victor Stinner in :issue:`21071`.) 2409 2410* :mod:`argparse` subparsers can now be made mandatory by passing ``required=True`` 2411 to :meth:`ArgumentParser.add_subparsers() <argparse.ArgumentParser.add_subparsers>`. 2412 (Contributed by Anthony Sottile in :issue:`26510`.) 2413 2414* :meth:`ast.literal_eval()` is now stricter. Addition and subtraction of 2415 arbitrary numbers are no longer allowed. 2416 (Contributed by Serhiy Storchaka in :issue:`31778`.) 2417 2418* :meth:`Calendar.itermonthdates <calendar.Calendar.itermonthdates>` 2419 will now consistently raise an exception when a date falls outside of the 2420 ``0001-01-01`` through ``9999-12-31`` range. To support applications that 2421 cannot tolerate such exceptions, the new 2422 :meth:`Calendar.itermonthdays3 <calendar.Calendar.itermonthdays3>` and 2423 :meth:`Calendar.itermonthdays4 <calendar.Calendar.itermonthdays4>` can be used. 2424 The new methods return tuples and are not restricted by the range supported by 2425 :class:`datetime.date`. 2426 (Contributed by Alexander Belopolsky in :issue:`28292`.) 2427 2428* :class:`collections.ChainMap` now preserves the order of the underlying 2429 mappings. (Contributed by Raymond Hettinger in :issue:`32792`.) 2430 2431* The ``submit()`` method of :class:`concurrent.futures.ThreadPoolExecutor` 2432 and :class:`concurrent.futures.ProcessPoolExecutor` now raises 2433 a :exc:`RuntimeError` if called during interpreter shutdown. 2434 (Contributed by Mark Nemec in :issue:`33097`.) 2435 2436* The :class:`configparser.ConfigParser` constructor now uses ``read_dict()`` 2437 to process the default values, making its behavior consistent with the 2438 rest of the parser. Non-string keys and values in the defaults 2439 dictionary are now being implicitly converted to strings. 2440 (Contributed by James Tocknell in :issue:`23835`.) 2441 2442* Several undocumented internal imports were removed. 2443 One example is that ``os.errno`` is no longer available; use ``import errno`` 2444 directly instead. 2445 Note that such undocumented internal imports may be removed any time without 2446 notice, even in micro version releases. 2447 2448 2449Changes in the C API 2450-------------------- 2451 2452The function :c:func:`PySlice_GetIndicesEx` is considered unsafe for 2453resizable sequences. If the slice indices are not instances of :class:`int`, 2454but objects that implement the :meth:`!__index__` method, the sequence can be 2455resized after passing its length to :c:func:`!PySlice_GetIndicesEx`. This 2456can lead to returning indices out of the length of the sequence. For 2457avoiding possible problems use new functions :c:func:`PySlice_Unpack` and 2458:c:func:`PySlice_AdjustIndices`. 2459(Contributed by Serhiy Storchaka in :issue:`27867`.) 2460 2461 2462CPython bytecode changes 2463------------------------ 2464 2465There are two new opcodes: :opcode:`LOAD_METHOD` and :opcode:`CALL_METHOD`. 2466(Contributed by Yury Selivanov and INADA Naoki in :issue:`26110`.) 2467 2468The :opcode:`STORE_ANNOTATION` opcode has been removed. 2469(Contributed by Mark Shannon in :issue:`32550`.) 2470 2471 2472Windows-only Changes 2473-------------------- 2474 2475The file used to override :data:`sys.path` is now called 2476``<python-executable>._pth`` instead of ``'sys.path'``. 2477See :ref:`finding_modules` for more information. 2478(Contributed by Steve Dower in :issue:`28137`.) 2479 2480 2481Other CPython implementation changes 2482------------------------------------ 2483 2484In preparation for potential future changes to the public CPython runtime 2485initialization API (see :pep:`432` for an initial, but somewhat outdated, 2486draft), CPython's internal startup 2487and configuration management logic has been significantly refactored. While 2488these updates are intended to be entirely transparent to both embedding 2489applications and users of the regular CPython CLI, they're being mentioned 2490here as the refactoring changes the internal order of various operations 2491during interpreter startup, and hence may uncover previously latent defects, 2492either in embedding applications, or in CPython itself. 2493(Initially contributed by Nick Coghlan and Eric Snow as part of 2494:issue:`22257`, and further updated by Nick, Eric, and Victor Stinner in a 2495number of other issues). Some known details affected: 2496 2497* :c:func:`PySys_AddWarnOptionUnicode` is not currently usable by embedding 2498 applications due to the requirement to create a Unicode object prior to 2499 calling `Py_Initialize`. Use :c:func:`PySys_AddWarnOption` instead. 2500 2501* warnings filters added by an embedding application with 2502 :c:func:`PySys_AddWarnOption` should now more consistently take precedence 2503 over the default filters set by the interpreter 2504 2505Due to changes in the way the default warnings filters are configured, 2506setting :c:data:`Py_BytesWarningFlag` to a value greater than one is no longer 2507sufficient to both emit :exc:`BytesWarning` messages and have them converted 2508to exceptions. Instead, the flag must be set (to cause the warnings to be 2509emitted in the first place), and an explicit ``error::BytesWarning`` 2510warnings filter added to convert them to exceptions. 2511 2512Due to a change in the way docstrings are handled by the compiler, the 2513implicit ``return None`` in a function body consisting solely of a docstring 2514is now marked as occurring on the same line as the docstring, not on the 2515function's header line. 2516 2517The current exception state has been moved from the frame object to the co-routine. 2518This simplified the interpreter and fixed a couple of obscure bugs caused by 2519having swap exception state when entering or exiting a generator. 2520(Contributed by Mark Shannon in :issue:`25612`.) 2521 2522Notable changes in Python 3.7.1 2523=============================== 2524 2525Starting in 3.7.1, :c:func:`Py_Initialize` now consistently reads and respects 2526all of the same environment settings as :c:func:`Py_Main` (in earlier Python 2527versions, it respected an ill-defined subset of those environment variables, 2528while in Python 3.7.0 it didn't read any of them due to :issue:`34247`). If 2529this behavior is unwanted, set :c:data:`Py_IgnoreEnvironmentFlag` to 1 before 2530calling :c:func:`Py_Initialize`. 2531 2532In 3.7.1 the C API for Context Variables 2533:ref:`was updated <contextvarsobjects_pointertype_change>` to use 2534:c:type:`PyObject` pointers. See also :issue:`34762`. 2535 2536In 3.7.1 the :mod:`tokenize` module now implicitly emits a ``NEWLINE`` token 2537when provided with input that does not have a trailing new line. This behavior 2538now matches what the C tokenizer does internally. 2539(Contributed by Ammar Askar in :issue:`33899`.) 2540 2541Notable changes in Python 3.7.2 2542=============================== 2543 2544In 3.7.2, :mod:`venv` on Windows no longer copies the original binaries, but 2545creates redirector scripts named ``python.exe`` and ``pythonw.exe`` instead. 2546This resolves a long standing issue where all virtual environments would have 2547to be upgraded or recreated with each Python update. However, note that this 2548release will still require recreation of virtual environments in order to get 2549the new scripts. 2550 2551Notable changes in Python 3.7.6 2552=============================== 2553 2554Due to significant security concerns, the *reuse_address* parameter of 2555:meth:`asyncio.loop.create_datagram_endpoint` is no longer supported. This is 2556because of the behavior of the socket option ``SO_REUSEADDR`` in UDP. For more 2557details, see the documentation for ``loop.create_datagram_endpoint()``. 2558(Contributed by Kyle Stanley, Antoine Pitrou, and Yury Selivanov in 2559:issue:`37228`.) 2560 2561Notable changes in Python 3.7.10 2562================================ 2563 2564Earlier Python versions allowed using both ``;`` and ``&`` as 2565query parameter separators in :func:`urllib.parse.parse_qs` and 2566:func:`urllib.parse.parse_qsl`. Due to security concerns, and to conform with 2567newer W3C recommendations, this has been changed to allow only a single 2568separator key, with ``&`` as the default. This change also affects 2569:func:`cgi.parse` and :func:`cgi.parse_multipart` as they use the affected 2570functions internally. For more details, please see their respective 2571documentation. 2572(Contributed by Adam Goldschmidt, Senthil Kumaran and Ken Jin in :issue:`42967`.) 2573