1**************************** 2 What's New In Python 3.8 3**************************** 4 5.. Rules for maintenance: 6 7 * Anyone can add text to this document. Do not spend very much time 8 on the wording of your changes, because your text will probably 9 get rewritten to some degree. 10 11 * The maintainer will go through Misc/NEWS periodically and add 12 changes; it's therefore more important to add your changes to 13 Misc/NEWS than to this file. 14 15 * This is not a complete list of every single change; completeness 16 is the purpose of Misc/NEWS. Some changes I consider too small 17 or esoteric to include. If such a change is added to the text, 18 I'll just remove it. (This is another reason you shouldn't spend 19 too much time on writing your addition.) 20 21 * If you want to draw your new text to the attention of the 22 maintainer, add 'XXX' to the beginning of the paragraph or 23 section. 24 25 * It's OK to just add a fragmentary note about a change. For 26 example: "XXX Describe the transmogrify() function added to the 27 socket module." The maintainer will research the change and 28 write the necessary text. 29 30 * You can comment out your additions if you like, but it's not 31 necessary (especially when a final release is some months away). 32 33 * Credit the author of a patch or bugfix. Just the name is 34 sufficient; the e-mail address isn't necessary. 35 36 * It's helpful to add the bug/patch number as a comment: 37 38 XXX Describe the transmogrify() function added to the socket 39 module. 40 (Contributed by P.Y. Developer in :issue:`12345`.) 41 42 This saves the maintainer the effort of going through the Git log 43 when researching a change. 44 45:Editor: Raymond Hettinger 46 47This article explains the new features in Python 3.8, compared to 3.7. 48Python 3.8 was released on October 14, 2019. 49For full details, see the :ref:`changelog <changelog>`. 50 51.. testsetup:: 52 53 from datetime import date 54 from math import cos, radians 55 from unicodedata import normalize 56 import re 57 import math 58 59 60Summary -- Release highlights 61============================= 62 63.. This section singles out the most important changes in Python 3.8. 64 Brevity is key. 65 66 67.. PEP-sized items next. 68 69 70 71New Features 72============ 73 74Assignment expressions 75---------------------- 76 77There is new syntax ``:=`` that assigns values to variables as part of a larger 78expression. It is affectionately known as "the walrus operator" due to 79its resemblance to `the eyes and tusks of a walrus 80<https://en.wikipedia.org/wiki/Walrus#/media/File:Pacific_Walrus_-_Bull_(8247646168).jpg>`_. 81 82In this example, the assignment expression helps avoid calling 83:func:`len` twice:: 84 85 if (n := len(a)) > 10: 86 print(f"List is too long ({n} elements, expected <= 10)") 87 88A similar benefit arises during regular expression matching where 89match objects are needed twice, once to test whether a match 90occurred and another to extract a subgroup:: 91 92 discount = 0.0 93 if (mo := re.search(r'(\d+)% discount', advertisement)): 94 discount = float(mo.group(1)) / 100.0 95 96The operator is also useful with while-loops that compute 97a value to test loop termination and then need that same 98value again in the body of the loop:: 99 100 # Loop over fixed length blocks 101 while (block := f.read(256)) != '': 102 process(block) 103 104Another motivating use case arises in list comprehensions where 105a value computed in a filtering condition is also needed in 106the expression body:: 107 108 [clean_name.title() for name in names 109 if (clean_name := normalize('NFC', name)) in allowed_names] 110 111Try to limit use of the walrus operator to clean cases that reduce 112complexity and improve readability. 113 114See :pep:`572` for a full description. 115 116(Contributed by Emily Morehouse in :issue:`35224`.) 117 118 119Positional-only parameters 120-------------------------- 121 122There is a new function parameter syntax ``/`` to indicate that some 123function parameters must be specified positionally and cannot be used as 124keyword arguments. This is the same notation shown by ``help()`` for C 125functions annotated with Larry Hastings' 126`Argument Clinic <https://devguide.python.org/development-tools/clinic/>`__ tool. 127 128In the following example, parameters *a* and *b* are positional-only, 129while *c* or *d* can be positional or keyword, and *e* or *f* are 130required to be keywords:: 131 132 def f(a, b, /, c, d, *, e, f): 133 print(a, b, c, d, e, f) 134 135The following is a valid call:: 136 137 f(10, 20, 30, d=40, e=50, f=60) 138 139However, these are invalid calls:: 140 141 f(10, b=20, c=30, d=40, e=50, f=60) # b cannot be a keyword argument 142 f(10, 20, 30, 40, 50, f=60) # e must be a keyword argument 143 144One use case for this notation is that it allows pure Python functions 145to fully emulate behaviors of existing C coded functions. For example, 146the built-in :func:`divmod` function does not accept keyword arguments:: 147 148 def divmod(a, b, /): 149 "Emulate the built in divmod() function" 150 return (a // b, a % b) 151 152Another use case is to preclude keyword arguments when the parameter 153name is not helpful. For example, the builtin :func:`len` function has 154the signature ``len(obj, /)``. This precludes awkward calls such as:: 155 156 len(obj='hello') # The "obj" keyword argument impairs readability 157 158A further benefit of marking a parameter as positional-only is that it 159allows the parameter name to be changed in the future without risk of 160breaking client code. For example, in the :mod:`statistics` module, the 161parameter name *dist* may be changed in the future. This was made 162possible with the following function specification:: 163 164 def quantiles(dist, /, *, n=4, method='exclusive') 165 ... 166 167Since the parameters to the left of ``/`` are not exposed as possible 168keywords, the parameters names remain available for use in ``**kwargs``:: 169 170 >>> def f(a, b, /, **kwargs): 171 ... print(a, b, kwargs) 172 ... 173 >>> f(10, 20, a=1, b=2, c=3) # a and b are used in two ways 174 10 20 {'a': 1, 'b': 2, 'c': 3} 175 176This greatly simplifies the implementation of functions and methods 177that need to accept arbitrary keyword arguments. For example, here 178is an excerpt from code in the :mod:`collections` module:: 179 180 class Counter(dict): 181 182 def __init__(self, iterable=None, /, **kwds): 183 # Note "iterable" is a possible keyword argument 184 185See :pep:`570` for a full description. 186 187(Contributed by Pablo Galindo in :issue:`36540`.) 188 189.. TODO: Pablo will sprint on docs at PyCon US 2019. 190 191 192Parallel filesystem cache for compiled bytecode files 193----------------------------------------------------- 194 195The new :envvar:`PYTHONPYCACHEPREFIX` setting (also available as 196:option:`-X` ``pycache_prefix``) configures the implicit bytecode 197cache to use a separate parallel filesystem tree, rather than 198the default ``__pycache__`` subdirectories within each source 199directory. 200 201The location of the cache is reported in :data:`sys.pycache_prefix` 202(:const:`None` indicates the default location in ``__pycache__`` 203subdirectories). 204 205(Contributed by Carl Meyer in :issue:`33499`.) 206 207 208Debug build uses the same ABI as release build 209----------------------------------------------- 210 211Python now uses the same ABI whether it's built in release or debug mode. On 212Unix, when Python is built in debug mode, it is now possible to load C 213extensions built in release mode and C extensions built using the stable ABI. 214 215Release builds and :ref:`debug builds <debug-build>` are now ABI compatible: defining the 216``Py_DEBUG`` macro no longer implies the ``Py_TRACE_REFS`` macro, which 217introduces the only ABI incompatibility. The ``Py_TRACE_REFS`` macro, which 218adds the :func:`sys.getobjects` function and the :envvar:`PYTHONDUMPREFS` 219environment variable, can be set using the new :option:`./configure 220--with-trace-refs <--with-trace-refs>` build option. 221(Contributed by Victor Stinner in :issue:`36465`.) 222 223On Unix, C extensions are no longer linked to libpython except on Android 224and Cygwin. 225It is now possible 226for a statically linked Python to load a C extension built using a shared 227library Python. 228(Contributed by Victor Stinner in :issue:`21536`.) 229 230On Unix, when Python is built in debug mode, import now also looks for C 231extensions compiled in release mode and for C extensions compiled with the 232stable ABI. 233(Contributed by Victor Stinner in :issue:`36722`.) 234 235To embed Python into an application, a new ``--embed`` option must be passed to 236``python3-config --libs --embed`` to get ``-lpython3.8`` (link the application 237to libpython). To support both 3.8 and older, try ``python3-config --libs 238--embed`` first and fallback to ``python3-config --libs`` (without ``--embed``) 239if the previous command fails. 240 241Add a pkg-config ``python-3.8-embed`` module to embed Python into an 242application: ``pkg-config python-3.8-embed --libs`` includes ``-lpython3.8``. 243To support both 3.8 and older, try ``pkg-config python-X.Y-embed --libs`` first 244and fallback to ``pkg-config python-X.Y --libs`` (without ``--embed``) if the 245previous command fails (replace ``X.Y`` with the Python version). 246 247On the other hand, ``pkg-config python3.8 --libs`` no longer contains 248``-lpython3.8``. C extensions must not be linked to libpython (except on 249Android and Cygwin, whose cases are handled by the script); 250this change is backward incompatible on purpose. 251(Contributed by Victor Stinner in :issue:`36721`.) 252 253.. _bpo-36817-whatsnew: 254 255f-strings support ``=`` for self-documenting expressions and debugging 256---------------------------------------------------------------------- 257 258Added an ``=`` specifier to :term:`f-string`\s. An f-string such as 259``f'{expr=}'`` will expand to the text of the expression, an equal sign, 260then the representation of the evaluated expression. For example: 261 262 >>> user = 'eric_idle' 263 >>> member_since = date(1975, 7, 31) 264 >>> f'{user=} {member_since=}' 265 "user='eric_idle' member_since=datetime.date(1975, 7, 31)" 266 267The usual :ref:`f-string format specifiers <f-strings>` allow more 268control over how the result of the expression is displayed:: 269 270 >>> delta = date.today() - member_since 271 >>> f'{user=!s} {delta.days=:,d}' 272 'user=eric_idle delta.days=16,075' 273 274The ``=`` specifier will display the whole expression so that 275calculations can be shown:: 276 277 >>> print(f'{theta=} {cos(radians(theta))=:.3f}') 278 theta=30 cos(radians(theta))=0.866 279 280(Contributed by Eric V. Smith and Larry Hastings in :issue:`36817`.) 281 282 283PEP 578: Python Runtime Audit Hooks 284----------------------------------- 285 286The PEP adds an Audit Hook and Verified Open Hook. Both are available from 287Python and native code, allowing applications and frameworks written in pure 288Python code to take advantage of extra notifications, while also allowing 289embedders or system administrators to deploy builds of Python where auditing is 290always enabled. 291 292See :pep:`578` for full details. 293 294 295PEP 587: Python Initialization Configuration 296-------------------------------------------- 297 298The :pep:`587` adds a new C API to configure the Python Initialization 299providing finer control on the whole configuration and better error reporting. 300 301New structures: 302 303* :c:type:`PyConfig` 304* :c:type:`PyPreConfig` 305* :c:type:`PyStatus` 306* :c:type:`PyWideStringList` 307 308New functions: 309 310* :c:func:`PyConfig_Clear` 311* :c:func:`PyConfig_InitIsolatedConfig` 312* :c:func:`PyConfig_InitPythonConfig` 313* :c:func:`PyConfig_Read` 314* :c:func:`PyConfig_SetArgv` 315* :c:func:`PyConfig_SetBytesArgv` 316* :c:func:`PyConfig_SetBytesString` 317* :c:func:`PyConfig_SetString` 318* :c:func:`PyPreConfig_InitIsolatedConfig` 319* :c:func:`PyPreConfig_InitPythonConfig` 320* :c:func:`PyStatus_Error` 321* :c:func:`PyStatus_Exception` 322* :c:func:`PyStatus_Exit` 323* :c:func:`PyStatus_IsError` 324* :c:func:`PyStatus_IsExit` 325* :c:func:`PyStatus_NoMemory` 326* :c:func:`PyStatus_Ok` 327* :c:func:`PyWideStringList_Append` 328* :c:func:`PyWideStringList_Insert` 329* :c:func:`Py_BytesMain` 330* :c:func:`Py_ExitStatusException` 331* :c:func:`Py_InitializeFromConfig` 332* :c:func:`Py_PreInitialize` 333* :c:func:`Py_PreInitializeFromArgs` 334* :c:func:`Py_PreInitializeFromBytesArgs` 335* :c:func:`Py_RunMain` 336 337This PEP also adds ``_PyRuntimeState.preconfig`` (:c:type:`PyPreConfig` type) 338and ``PyInterpreterState.config`` (:c:type:`PyConfig` type) fields to these 339internal structures. ``PyInterpreterState.config`` becomes the new 340reference configuration, replacing global configuration variables and 341other private variables. 342 343See :ref:`Python Initialization Configuration <init-config>` for the 344documentation. 345 346See :pep:`587` for a full description. 347 348(Contributed by Victor Stinner in :issue:`36763`.) 349 350 351PEP 590: Vectorcall: a fast calling protocol for CPython 352-------------------------------------------------------- 353 354:ref:`vectorcall` is added to the Python/C API. 355It is meant to formalize existing optimizations which were already done 356for various classes. 357Any :ref:`static type <static-types>` implementing a callable can use this 358protocol. 359 360This is currently provisional. 361The aim is to make it fully public in Python 3.9. 362 363See :pep:`590` for a full description. 364 365(Contributed by Jeroen Demeyer, Mark Shannon and Petr Viktorin in :issue:`36974`.) 366 367 368Pickle protocol 5 with out-of-band data buffers 369----------------------------------------------- 370 371When :mod:`pickle` is used to transfer large data between Python processes 372in order to take advantage of multi-core or multi-machine processing, 373it is important to optimize the transfer by reducing memory copies, and 374possibly by applying custom techniques such as data-dependent compression. 375 376The :mod:`pickle` protocol 5 introduces support for out-of-band buffers 377where :pep:`3118`-compatible data can be transmitted separately from the 378main pickle stream, at the discretion of the communication layer. 379 380See :pep:`574` for a full description. 381 382(Contributed by Antoine Pitrou in :issue:`36785`.) 383 384 385Other Language Changes 386====================== 387 388* A :keyword:`continue` statement was illegal in the :keyword:`finally` clause 389 due to a problem with the implementation. In Python 3.8 this restriction 390 was lifted. 391 (Contributed by Serhiy Storchaka in :issue:`32489`.) 392 393* The :class:`bool`, :class:`int`, and :class:`fractions.Fraction` types 394 now have an :meth:`~int.as_integer_ratio` method like that found in 395 :class:`float` and :class:`decimal.Decimal`. This minor API extension 396 makes it possible to write ``numerator, denominator = 397 x.as_integer_ratio()`` and have it work across multiple numeric types. 398 (Contributed by Lisa Roach in :issue:`33073` and Raymond Hettinger in 399 :issue:`37819`.) 400 401* Constructors of :class:`int`, :class:`float` and :class:`complex` will now 402 use the :meth:`~object.__index__` special method, if available and the 403 corresponding method :meth:`~object.__int__`, :meth:`~object.__float__` 404 or :meth:`~object.__complex__` is not available. 405 (Contributed by Serhiy Storchaka in :issue:`20092`.) 406 407* Added support of :samp:`\\N\\{{name}\\}` escapes in :mod:`regular expressions <re>`:: 408 409 >>> notice = 'Copyright © 2019' 410 >>> copyright_year_pattern = re.compile(r'\N{copyright sign}\s*(\d{4})') 411 >>> int(copyright_year_pattern.search(notice).group(1)) 412 2019 413 414 (Contributed by Jonathan Eunice and Serhiy Storchaka in :issue:`30688`.) 415 416* Dict and dictviews are now iterable in reversed insertion order using 417 :func:`reversed`. (Contributed by Rémi Lapeyre in :issue:`33462`.) 418 419* The syntax allowed for keyword names in function calls was further 420 restricted. In particular, ``f((keyword)=arg)`` is no longer allowed. It was 421 never intended to permit more than a bare name on the left-hand side of a 422 keyword argument assignment term. 423 (Contributed by Benjamin Peterson in :issue:`34641`.) 424 425* Generalized iterable unpacking in :keyword:`yield` and 426 :keyword:`return` statements no longer requires enclosing parentheses. 427 This brings the *yield* and *return* syntax into better agreement with 428 normal assignment syntax:: 429 430 >>> def parse(family): 431 lastname, *members = family.split() 432 return lastname.upper(), *members 433 434 >>> parse('simpsons homer marge bart lisa maggie') 435 ('SIMPSONS', 'homer', 'marge', 'bart', 'lisa', 'maggie') 436 437 (Contributed by David Cuthbert and Jordan Chapman in :issue:`32117`.) 438 439* When a comma is missed in code such as ``[(10, 20) (30, 40)]``, the 440 compiler displays a :exc:`SyntaxWarning` with a helpful suggestion. 441 This improves on just having a :exc:`TypeError` indicating that the 442 first tuple was not callable. (Contributed by Serhiy Storchaka in 443 :issue:`15248`.) 444 445* Arithmetic operations between subclasses of :class:`datetime.date` or 446 :class:`datetime.datetime` and :class:`datetime.timedelta` objects now return 447 an instance of the subclass, rather than the base class. This also affects 448 the return type of operations whose implementation (directly or indirectly) 449 uses :class:`datetime.timedelta` arithmetic, such as 450 :meth:`~datetime.datetime.astimezone`. 451 (Contributed by Paul Ganssle in :issue:`32417`.) 452 453* When the Python interpreter is interrupted by Ctrl-C (SIGINT) and the 454 resulting :exc:`KeyboardInterrupt` exception is not caught, the Python process 455 now exits via a SIGINT signal or with the correct exit code such that the 456 calling process can detect that it died due to a Ctrl-C. Shells on POSIX 457 and Windows use this to properly terminate scripts in interactive sessions. 458 (Contributed by Google via Gregory P. Smith in :issue:`1054041`.) 459 460* Some advanced styles of programming require updating the 461 :class:`types.CodeType` object for an existing function. Since code 462 objects are immutable, a new code object needs to be created, one 463 that is modeled on the existing code object. With 19 parameters, 464 this was somewhat tedious. Now, the new ``replace()`` method makes 465 it possible to create a clone with a few altered parameters. 466 467 Here's an example that alters the :func:`statistics.mean` function to 468 prevent the *data* parameter from being used as a keyword argument:: 469 470 >>> from statistics import mean 471 >>> mean(data=[10, 20, 90]) 472 40 473 >>> mean.__code__ = mean.__code__.replace(co_posonlyargcount=1) 474 >>> mean(data=[10, 20, 90]) 475 Traceback (most recent call last): 476 ... 477 TypeError: mean() got some positional-only arguments passed as keyword arguments: 'data' 478 479 (Contributed by Victor Stinner in :issue:`37032`.) 480 481* For integers, the three-argument form of the :func:`pow` function now 482 permits the exponent to be negative in the case where the base is 483 relatively prime to the modulus. It then computes a modular inverse to 484 the base when the exponent is ``-1``, and a suitable power of that 485 inverse for other negative exponents. For example, to compute the 486 `modular multiplicative inverse 487 <https://en.wikipedia.org/wiki/Modular_multiplicative_inverse>`_ of 38 488 modulo 137, write:: 489 490 >>> pow(38, -1, 137) 491 119 492 >>> 119 * 38 % 137 493 1 494 495 Modular inverses arise in the solution of `linear Diophantine 496 equations <https://en.wikipedia.org/wiki/Diophantine_equation>`_. 497 For example, to find integer solutions for ``4258�� + 147�� = 369``, 498 first rewrite as ``4258�� ≡ 369 (mod 147)`` then solve: 499 500 >>> x = 369 * pow(4258, -1, 147) % 147 501 >>> y = (4258 * x - 369) // -147 502 >>> 4258 * x + 147 * y 503 369 504 505 (Contributed by Mark Dickinson in :issue:`36027`.) 506 507* Dict comprehensions have been synced-up with dict literals so that the 508 key is computed first and the value second:: 509 510 >>> # Dict comprehension 511 >>> cast = {input('role? '): input('actor? ') for i in range(2)} 512 role? King Arthur 513 actor? Chapman 514 role? Black Knight 515 actor? Cleese 516 517 >>> # Dict literal 518 >>> cast = {input('role? '): input('actor? ')} 519 role? Sir Robin 520 actor? Eric Idle 521 522 The guaranteed execution order is helpful with assignment expressions 523 because variables assigned in the key expression will be available in 524 the value expression:: 525 526 >>> names = ['Martin von Löwis', 'Łukasz Langa', 'Walter Dörwald'] 527 >>> {(n := normalize('NFC', name)).casefold() : n for name in names} 528 {'martin von löwis': 'Martin von Löwis', 529 'łukasz langa': 'Łukasz Langa', 530 'walter dörwald': 'Walter Dörwald'} 531 532 (Contributed by Jörn Heissler in :issue:`35224`.) 533 534* The :meth:`object.__reduce__` method can now return a tuple from two to 535 six elements long. Formerly, five was the limit. The new, optional sixth 536 element is a callable with a ``(obj, state)`` signature. This allows the 537 direct control over the state-updating behavior of a specific object. If 538 not *None*, this callable will have priority over the object's 539 :meth:`~__setstate__` method. 540 (Contributed by Pierre Glaser and Olivier Grisel in :issue:`35900`.) 541 542New Modules 543=========== 544 545* The new :mod:`importlib.metadata` module provides (provisional) support for 546 reading metadata from third-party packages. For example, it can extract an 547 installed package's version number, list of entry points, and more:: 548 549 >>> # Note following example requires that the popular "requests" 550 >>> # package has been installed. 551 >>> 552 >>> from importlib.metadata import version, requires, files 553 >>> version('requests') 554 '2.22.0' 555 >>> list(requires('requests')) 556 ['chardet (<3.1.0,>=3.0.2)'] 557 >>> list(files('requests'))[:5] 558 [PackagePath('requests-2.22.0.dist-info/INSTALLER'), 559 PackagePath('requests-2.22.0.dist-info/LICENSE'), 560 PackagePath('requests-2.22.0.dist-info/METADATA'), 561 PackagePath('requests-2.22.0.dist-info/RECORD'), 562 PackagePath('requests-2.22.0.dist-info/WHEEL')] 563 564 (Contributed by Barry Warsaw and Jason R. Coombs in :issue:`34632`.) 565 566 567Improved Modules 568================ 569 570ast 571--- 572 573AST nodes now have ``end_lineno`` and ``end_col_offset`` attributes, 574which give the precise location of the end of the node. (This only 575applies to nodes that have ``lineno`` and ``col_offset`` attributes.) 576 577New function :func:`ast.get_source_segment` returns the source code 578for a specific AST node. 579 580(Contributed by Ivan Levkivskyi in :issue:`33416`.) 581 582The :func:`ast.parse` function has some new flags: 583 584* ``type_comments=True`` causes it to return the text of :pep:`484` and 585 :pep:`526` type comments associated with certain AST nodes; 586 587* ``mode='func_type'`` can be used to parse :pep:`484` "signature type 588 comments" (returned for function definition AST nodes); 589 590* ``feature_version=(3, N)`` allows specifying an earlier Python 3 591 version. For example, ``feature_version=(3, 4)`` will treat 592 :keyword:`async` and :keyword:`await` as non-reserved words. 593 594(Contributed by Guido van Rossum in :issue:`35766`.) 595 596 597asyncio 598------- 599 600:func:`asyncio.run` has graduated from the provisional to stable API. This 601function can be used to execute a :term:`coroutine` and return the result while 602automatically managing the event loop. For example:: 603 604 import asyncio 605 606 async def main(): 607 await asyncio.sleep(0) 608 return 42 609 610 asyncio.run(main()) 611 612This is *roughly* equivalent to:: 613 614 import asyncio 615 616 async def main(): 617 await asyncio.sleep(0) 618 return 42 619 620 loop = asyncio.new_event_loop() 621 asyncio.set_event_loop(loop) 622 try: 623 loop.run_until_complete(main()) 624 finally: 625 asyncio.set_event_loop(None) 626 loop.close() 627 628 629The actual implementation is significantly more complex. Thus, 630:func:`asyncio.run` should be the preferred way of running asyncio programs. 631 632(Contributed by Yury Selivanov in :issue:`32314`.) 633 634Running ``python -m asyncio`` launches a natively async REPL. This allows rapid 635experimentation with code that has a top-level :keyword:`await`. There is no 636longer a need to directly call ``asyncio.run()`` which would spawn a new event 637loop on every invocation: 638 639.. code-block:: none 640 641 $ python -m asyncio 642 asyncio REPL 3.8.0 643 Use "await" directly instead of "asyncio.run()". 644 Type "help", "copyright", "credits" or "license" for more information. 645 >>> import asyncio 646 >>> await asyncio.sleep(10, result='hello') 647 hello 648 649(Contributed by Yury Selivanov in :issue:`37028`.) 650 651The exception :class:`asyncio.CancelledError` now inherits from 652:class:`BaseException` rather than :class:`Exception` and no longer inherits 653from :class:`concurrent.futures.CancelledError`. 654(Contributed by Yury Selivanov in :issue:`32528`.) 655 656On Windows, the default event loop is now :class:`~asyncio.ProactorEventLoop`. 657(Contributed by Victor Stinner in :issue:`34687`.) 658 659:class:`~asyncio.ProactorEventLoop` now also supports UDP. 660(Contributed by Adam Meily and Andrew Svetlov in :issue:`29883`.) 661 662:class:`~asyncio.ProactorEventLoop` can now be interrupted by 663:exc:`KeyboardInterrupt` ("CTRL+C"). 664(Contributed by Vladimir Matveev in :issue:`23057`.) 665 666Added :meth:`asyncio.Task.get_coro` for getting the wrapped coroutine 667within an :class:`asyncio.Task`. 668(Contributed by Alex Grönholm in :issue:`36999`.) 669 670Asyncio tasks can now be named, either by passing the ``name`` keyword 671argument to :func:`asyncio.create_task` or 672the :meth:`~asyncio.loop.create_task` event loop method, or by 673calling the :meth:`~asyncio.Task.set_name` method on the task object. The 674task name is visible in the ``repr()`` output of :class:`asyncio.Task` and 675can also be retrieved using the :meth:`~asyncio.Task.get_name` method. 676(Contributed by Alex Grönholm in :issue:`34270`.) 677 678Added support for 679`Happy Eyeballs <https://en.wikipedia.org/wiki/Happy_Eyeballs>`_ to 680:func:`asyncio.loop.create_connection`. To specify the behavior, two new 681parameters have been added: *happy_eyeballs_delay* and *interleave*. The Happy 682Eyeballs algorithm improves responsiveness in applications that support IPv4 683and IPv6 by attempting to simultaneously connect using both. 684(Contributed by twisteroid ambassador in :issue:`33530`.) 685 686 687builtins 688-------- 689 690The :func:`compile` built-in has been improved to accept the 691``ast.PyCF_ALLOW_TOP_LEVEL_AWAIT`` flag. With this new flag passed, 692:func:`compile` will allow top-level ``await``, ``async for`` and ``async with`` 693constructs that are usually considered invalid syntax. Asynchronous code object 694marked with the ``CO_COROUTINE`` flag may then be returned. 695(Contributed by Matthias Bussonnier in :issue:`34616`) 696 697 698collections 699----------- 700 701The :meth:`~collections.somenamedtuple._asdict` method for 702:func:`collections.namedtuple` now returns a :class:`dict` instead of a 703:class:`collections.OrderedDict`. This works because regular dicts have 704guaranteed ordering since Python 3.7. If the extra features of 705:class:`OrderedDict` are required, the suggested remediation is to cast the 706result to the desired type: ``OrderedDict(nt._asdict())``. 707(Contributed by Raymond Hettinger in :issue:`35864`.) 708 709 710cProfile 711-------- 712 713The :class:`cProfile.Profile <profile.Profile>` class can now be used as a context manager. 714Profile a block of code by running:: 715 716 import cProfile 717 718 with cProfile.Profile() as profiler: 719 # code to be profiled 720 ... 721 722(Contributed by Scott Sanderson in :issue:`29235`.) 723 724 725csv 726--- 727 728The :class:`csv.DictReader` now returns instances of :class:`dict` instead of 729a :class:`collections.OrderedDict`. The tool is now faster and uses less 730memory while still preserving the field order. 731(Contributed by Michael Selik in :issue:`34003`.) 732 733 734curses 735------- 736 737Added a new variable holding structured version information for the 738underlying ncurses library: :data:`~curses.ncurses_version`. 739(Contributed by Serhiy Storchaka in :issue:`31680`.) 740 741 742ctypes 743------ 744 745On Windows, :class:`~ctypes.CDLL` and subclasses now accept a *winmode* parameter 746to specify flags for the underlying ``LoadLibraryEx`` call. The default flags are 747set to only load DLL dependencies from trusted locations, including the path 748where the DLL is stored (if a full or partial path is used to load the initial 749DLL) and paths added by :func:`~os.add_dll_directory`. 750(Contributed by Steve Dower in :issue:`36085`.) 751 752 753datetime 754-------- 755 756Added new alternate constructors :meth:`datetime.date.fromisocalendar` and 757:meth:`datetime.datetime.fromisocalendar`, which construct :class:`~datetime.date` and 758:class:`~datetime.datetime` objects respectively from ISO year, week number, and weekday; 759these are the inverse of each class's ``isocalendar`` method. 760(Contributed by Paul Ganssle in :issue:`36004`.) 761 762 763functools 764--------- 765 766:func:`functools.lru_cache` can now be used as a straight decorator rather 767than as a function returning a decorator. So both of these are now supported:: 768 769 @lru_cache 770 def f(x): 771 ... 772 773 @lru_cache(maxsize=256) 774 def f(x): 775 ... 776 777(Contributed by Raymond Hettinger in :issue:`36772`.) 778 779Added a new :func:`functools.cached_property` decorator, for computed properties 780cached for the life of the instance. :: 781 782 import functools 783 import statistics 784 785 class Dataset: 786 def __init__(self, sequence_of_numbers): 787 self.data = sequence_of_numbers 788 789 @functools.cached_property 790 def variance(self): 791 return statistics.variance(self.data) 792 793(Contributed by Carl Meyer in :issue:`21145`) 794 795 796Added a new :func:`functools.singledispatchmethod` decorator that converts 797methods into :term:`generic functions <generic function>` using 798:term:`single dispatch`:: 799 800 from functools import singledispatchmethod 801 from contextlib import suppress 802 803 class TaskManager: 804 805 def __init__(self, tasks): 806 self.tasks = list(tasks) 807 808 @singledispatchmethod 809 def discard(self, value): 810 with suppress(ValueError): 811 self.tasks.remove(value) 812 813 @discard.register(list) 814 def _(self, tasks): 815 targets = set(tasks) 816 self.tasks = [x for x in self.tasks if x not in targets] 817 818(Contributed by Ethan Smith in :issue:`32380`) 819 820gc 821-- 822 823:func:`~gc.get_objects` can now receive an optional *generation* parameter 824indicating a generation to get objects from. 825(Contributed by Pablo Galindo in :issue:`36016`.) 826 827 828gettext 829------- 830 831Added :func:`~gettext.pgettext` and its variants. 832(Contributed by Franz Glasner, Éric Araujo, and Cheryl Sabella in :issue:`2504`.) 833 834 835gzip 836---- 837 838Added the *mtime* parameter to :func:`gzip.compress` for reproducible output. 839(Contributed by Guo Ci Teo in :issue:`34898`.) 840 841A :exc:`~gzip.BadGzipFile` exception is now raised instead of :exc:`OSError` 842for certain types of invalid or corrupt gzip files. 843(Contributed by Filip Gruszczyński, Michele Orrù, and Zackery Spytz in 844:issue:`6584`.) 845 846 847IDLE and idlelib 848---------------- 849 850Output over N lines (50 by default) is squeezed down to a button. 851N can be changed in the PyShell section of the General page of the 852Settings dialog. Fewer, but possibly extra long, lines can be squeezed by 853right clicking on the output. Squeezed output can be expanded in place 854by double-clicking the button or into the clipboard or a separate window 855by right-clicking the button. (Contributed by Tal Einat in :issue:`1529353`.) 856 857Add "Run Customized" to the Run menu to run a module with customized 858settings. Any command line arguments entered are added to sys.argv. 859They also re-appear in the box for the next customized run. One can also 860suppress the normal Shell main module restart. (Contributed by Cheryl 861Sabella, Terry Jan Reedy, and others in :issue:`5680` and :issue:`37627`.) 862 863Added optional line numbers for IDLE editor windows. Windows 864open without line numbers unless set otherwise in the General 865tab of the configuration dialog. Line numbers for an existing 866window are shown and hidden in the Options menu. 867(Contributed by Tal Einat and Saimadhav Heblikar in :issue:`17535`.) 868 869OS native encoding is now used for converting between Python strings and Tcl 870objects. This allows IDLE to work with emoji and other non-BMP characters. 871These characters can be displayed or copied and pasted to or from the 872clipboard. Converting strings from Tcl to Python and back now never fails. 873(Many people worked on this for eight years but the problem was finally 874solved by Serhiy Storchaka in :issue:`13153`.) 875 876New in 3.8.1: 877 878Add option to toggle cursor blink off. (Contributed by Zackery Spytz 879in :issue:`4603`.) 880 881Escape key now closes IDLE completion windows. (Contributed by Johnny 882Najera in :issue:`38944`.) 883 884The changes above have been backported to 3.7 maintenance releases. 885 886Add keywords to module name completion list. (Contributed by Terry J. 887Reedy in :issue:`37765`.) 888 889inspect 890------- 891 892The :func:`inspect.getdoc` function can now find docstrings for ``__slots__`` 893if that attribute is a :class:`dict` where the values are docstrings. 894This provides documentation options similar to what we already have 895for :func:`property`, :func:`classmethod`, and :func:`staticmethod`:: 896 897 class AudioClip: 898 __slots__ = {'bit_rate': 'expressed in kilohertz to one decimal place', 899 'duration': 'in seconds, rounded up to an integer'} 900 def __init__(self, bit_rate, duration): 901 self.bit_rate = round(bit_rate / 1000.0, 1) 902 self.duration = ceil(duration) 903 904(Contributed by Raymond Hettinger in :issue:`36326`.) 905 906 907io 908-- 909 910In development mode (:option:`-X` ``env``) and in :ref:`debug build <debug-build>`, the 911:class:`io.IOBase` finalizer now logs the exception if the ``close()`` method 912fails. The exception is ignored silently by default in release build. 913(Contributed by Victor Stinner in :issue:`18748`.) 914 915 916itertools 917--------- 918 919The :func:`itertools.accumulate` function added an option *initial* keyword 920argument to specify an initial value:: 921 922 >>> from itertools import accumulate 923 >>> list(accumulate([10, 5, 30, 15], initial=1000)) 924 [1000, 1010, 1015, 1045, 1060] 925 926(Contributed by Lisa Roach in :issue:`34659`.) 927 928 929json.tool 930--------- 931 932Add option ``--json-lines`` to parse every input line as a separate JSON object. 933(Contributed by Weipeng Hong in :issue:`31553`.) 934 935 936logging 937------- 938 939Added a *force* keyword argument to :func:`logging.basicConfig`. 940When set to true, any existing handlers attached 941to the root logger are removed and closed before carrying out the 942configuration specified by the other arguments. 943 944This solves a long-standing problem. Once a logger or *basicConfig()* had 945been called, subsequent calls to *basicConfig()* were silently ignored. 946This made it difficult to update, experiment with, or teach the various 947logging configuration options using the interactive prompt or a Jupyter 948notebook. 949 950(Suggested by Raymond Hettinger, implemented by Donghee Na, and 951reviewed by Vinay Sajip in :issue:`33897`.) 952 953 954math 955---- 956 957Added new function :func:`math.dist` for computing Euclidean distance 958between two points. (Contributed by Raymond Hettinger in :issue:`33089`.) 959 960Expanded the :func:`math.hypot` function to handle multiple dimensions. 961Formerly, it only supported the 2-D case. 962(Contributed by Raymond Hettinger in :issue:`33089`.) 963 964Added new function, :func:`math.prod`, as analogous function to :func:`sum` 965that returns the product of a 'start' value (default: 1) times an iterable of 966numbers:: 967 968 >>> prior = 0.8 969 >>> likelihoods = [0.625, 0.84, 0.30] 970 >>> math.prod(likelihoods, start=prior) 971 0.126 972 973(Contributed by Pablo Galindo in :issue:`35606`.) 974 975Added two new combinatoric functions :func:`math.perm` and :func:`math.comb`:: 976 977 >>> math.perm(10, 3) # Permutations of 10 things taken 3 at a time 978 720 979 >>> math.comb(10, 3) # Combinations of 10 things taken 3 at a time 980 120 981 982(Contributed by Yash Aggarwal, Keller Fuchs, Serhiy Storchaka, and Raymond 983Hettinger in :issue:`37128`, :issue:`37178`, and :issue:`35431`.) 984 985Added a new function :func:`math.isqrt` for computing accurate integer square 986roots without conversion to floating point. The new function supports 987arbitrarily large integers. It is faster than ``floor(sqrt(n))`` but slower 988than :func:`math.sqrt`:: 989 990 >>> r = 650320427 991 >>> s = r ** 2 992 >>> isqrt(s - 1) # correct 993 650320426 994 >>> floor(sqrt(s - 1)) # incorrect 995 650320427 996 997(Contributed by Mark Dickinson in :issue:`36887`.) 998 999The function :func:`math.factorial` no longer accepts arguments that are not 1000int-like. (Contributed by Pablo Galindo in :issue:`33083`.) 1001 1002 1003mmap 1004---- 1005 1006The :class:`mmap.mmap` class now has an :meth:`~mmap.mmap.madvise` method to 1007access the ``madvise()`` system call. 1008(Contributed by Zackery Spytz in :issue:`32941`.) 1009 1010 1011multiprocessing 1012--------------- 1013 1014Added new :mod:`multiprocessing.shared_memory` module. 1015(Contributed by Davin Potts in :issue:`35813`.) 1016 1017On macOS, the *spawn* start method is now used by default. 1018(Contributed by Victor Stinner in :issue:`33725`.) 1019 1020 1021os 1022-- 1023 1024Added new function :func:`~os.add_dll_directory` on Windows for providing 1025additional search paths for native dependencies when importing extension 1026modules or loading DLLs using :mod:`ctypes`. 1027(Contributed by Steve Dower in :issue:`36085`.) 1028 1029A new :func:`os.memfd_create` function was added to wrap the 1030``memfd_create()`` syscall. 1031(Contributed by Zackery Spytz and Christian Heimes in :issue:`26836`.) 1032 1033On Windows, much of the manual logic for handling reparse points (including 1034symlinks and directory junctions) has been delegated to the operating system. 1035Specifically, :func:`os.stat` will now traverse anything supported by the 1036operating system, while :func:`os.lstat` will only open reparse points that 1037identify as "name surrogates" while others are opened as for :func:`os.stat`. 1038In all cases, :attr:`stat_result.st_mode` will only have ``S_IFLNK`` set for 1039symbolic links and not other kinds of reparse points. To identify other kinds 1040of reparse point, check the new :attr:`stat_result.st_reparse_tag` attribute. 1041 1042On Windows, :func:`os.readlink` is now able to read directory junctions. Note 1043that :func:`~os.path.islink` will return ``False`` for directory junctions, 1044and so code that checks ``islink`` first will continue to treat junctions as 1045directories, while code that handles errors from :func:`os.readlink` may now 1046treat junctions as links. 1047 1048(Contributed by Steve Dower in :issue:`37834`.) 1049 1050 1051os.path 1052------- 1053 1054:mod:`os.path` functions that return a boolean result like 1055:func:`~os.path.exists`, :func:`~os.path.lexists`, :func:`~os.path.isdir`, 1056:func:`~os.path.isfile`, :func:`~os.path.islink`, and :func:`~os.path.ismount` 1057now return ``False`` instead of raising :exc:`ValueError` or its subclasses 1058:exc:`UnicodeEncodeError` and :exc:`UnicodeDecodeError` for paths that contain 1059characters or bytes unrepresentable at the OS level. 1060(Contributed by Serhiy Storchaka in :issue:`33721`.) 1061 1062:func:`~os.path.expanduser` on Windows now prefers the :envvar:`USERPROFILE` 1063environment variable and does not use :envvar:`HOME`, which is not normally set 1064for regular user accounts. 1065(Contributed by Anthony Sottile in :issue:`36264`.) 1066 1067:func:`~os.path.isdir` on Windows no longer returns ``True`` for a link to a 1068non-existent directory. 1069 1070:func:`~os.path.realpath` on Windows now resolves reparse points, including 1071symlinks and directory junctions. 1072 1073(Contributed by Steve Dower in :issue:`37834`.) 1074 1075 1076pathlib 1077------- 1078 1079:mod:`pathlib.Path` methods that return a boolean result like 1080:meth:`~pathlib.Path.exists`, :meth:`~pathlib.Path.is_dir`, 1081:meth:`~pathlib.Path.is_file`, :meth:`~pathlib.Path.is_mount`, 1082:meth:`~pathlib.Path.is_symlink`, :meth:`~pathlib.Path.is_block_device`, 1083:meth:`~pathlib.Path.is_char_device`, :meth:`~pathlib.Path.is_fifo`, 1084:meth:`~pathlib.Path.is_socket` now return ``False`` instead of raising 1085:exc:`ValueError` or its subclass :exc:`UnicodeEncodeError` for paths that 1086contain characters unrepresentable at the OS level. 1087(Contributed by Serhiy Storchaka in :issue:`33721`.) 1088 1089Added :meth:`!pathlib.Path.link_to` which creates a hard link pointing 1090to a path. 1091(Contributed by Joannah Nanjekye in :issue:`26978`) 1092Note that ``link_to`` was deprecated in 3.10 and removed in 3.12 in 1093favor of a ``hardlink_to`` method added in 3.10 which matches the 1094semantics of the existing ``symlink_to`` method. 1095 1096 1097pickle 1098------ 1099 1100:mod:`pickle` extensions subclassing the C-optimized :class:`~pickle.Pickler` 1101can now override the pickling logic of functions and classes by defining the 1102special :meth:`~pickle.Pickler.reducer_override` method. 1103(Contributed by Pierre Glaser and Olivier Grisel in :issue:`35900`.) 1104 1105 1106plistlib 1107-------- 1108 1109Added new :class:`plistlib.UID` and enabled support for reading and writing 1110NSKeyedArchiver-encoded binary plists. 1111(Contributed by Jon Janzen in :issue:`26707`.) 1112 1113 1114pprint 1115------ 1116 1117The :mod:`pprint` module added a *sort_dicts* parameter to several functions. 1118By default, those functions continue to sort dictionaries before rendering or 1119printing. However, if *sort_dicts* is set to false, the dictionaries retain 1120the order that keys were inserted. This can be useful for comparison to JSON 1121inputs during debugging. 1122 1123In addition, there is a convenience new function, :func:`pprint.pp` that is 1124like :func:`pprint.pprint` but with *sort_dicts* defaulting to ``False``:: 1125 1126 >>> from pprint import pprint, pp 1127 >>> d = dict(source='input.txt', operation='filter', destination='output.txt') 1128 >>> pp(d, width=40) # Original order 1129 {'source': 'input.txt', 1130 'operation': 'filter', 1131 'destination': 'output.txt'} 1132 >>> pprint(d, width=40) # Keys sorted alphabetically 1133 {'destination': 'output.txt', 1134 'operation': 'filter', 1135 'source': 'input.txt'} 1136 1137(Contributed by Rémi Lapeyre in :issue:`30670`.) 1138 1139 1140py_compile 1141---------- 1142 1143:func:`py_compile.compile` now supports silent mode. 1144(Contributed by Joannah Nanjekye in :issue:`22640`.) 1145 1146 1147shlex 1148----- 1149 1150The new :func:`shlex.join` function acts as the inverse of :func:`shlex.split`. 1151(Contributed by Bo Bayles in :issue:`32102`.) 1152 1153 1154shutil 1155------ 1156 1157:func:`shutil.copytree` now accepts a new ``dirs_exist_ok`` keyword argument. 1158(Contributed by Josh Bronson in :issue:`20849`.) 1159 1160:func:`shutil.make_archive` now defaults to the modern pax (POSIX.1-2001) 1161format for new archives to improve portability and standards conformance, 1162inherited from the corresponding change to the :mod:`tarfile` module. 1163(Contributed by C.A.M. Gerlach in :issue:`30661`.) 1164 1165:func:`shutil.rmtree` on Windows now removes directory junctions without 1166recursively removing their contents first. 1167(Contributed by Steve Dower in :issue:`37834`.) 1168 1169 1170socket 1171------ 1172 1173Added :meth:`~socket.create_server` and :meth:`~socket.has_dualstack_ipv6` 1174convenience functions to automate the necessary tasks usually involved when 1175creating a server socket, including accepting both IPv4 and IPv6 connections 1176on the same socket. (Contributed by Giampaolo Rodolà in :issue:`17561`.) 1177 1178The :func:`socket.if_nameindex`, :func:`socket.if_nametoindex`, and 1179:func:`socket.if_indextoname` functions have been implemented on Windows. 1180(Contributed by Zackery Spytz in :issue:`37007`.) 1181 1182 1183ssl 1184--- 1185 1186Added :attr:`~ssl.SSLContext.post_handshake_auth` to enable and 1187:meth:`~ssl.SSLSocket.verify_client_post_handshake` to initiate TLS 1.3 1188post-handshake authentication. 1189(Contributed by Christian Heimes in :issue:`34670`.) 1190 1191 1192statistics 1193---------- 1194 1195Added :func:`statistics.fmean` as a faster, floating-point variant of 1196:func:`statistics.mean`. (Contributed by Raymond Hettinger and 1197Steven D'Aprano in :issue:`35904`.) 1198 1199Added :func:`statistics.geometric_mean` 1200(Contributed by Raymond Hettinger in :issue:`27181`.) 1201 1202Added :func:`statistics.multimode` that returns a list of the most 1203common values. (Contributed by Raymond Hettinger in :issue:`35892`.) 1204 1205Added :func:`statistics.quantiles` that divides data or a distribution 1206in to equiprobable intervals (e.g. quartiles, deciles, or percentiles). 1207(Contributed by Raymond Hettinger in :issue:`36546`.) 1208 1209Added :class:`statistics.NormalDist`, a tool for creating 1210and manipulating normal distributions of a random variable. 1211(Contributed by Raymond Hettinger in :issue:`36018`.) 1212 1213:: 1214 1215 >>> temperature_feb = NormalDist.from_samples([4, 12, -3, 2, 7, 14]) 1216 >>> temperature_feb.mean 1217 6.0 1218 >>> temperature_feb.stdev 1219 6.356099432828281 1220 1221 >>> temperature_feb.cdf(3) # Chance of being under 3 degrees 1222 0.3184678262814532 1223 >>> # Relative chance of being 7 degrees versus 10 degrees 1224 >>> temperature_feb.pdf(7) / temperature_feb.pdf(10) 1225 1.2039930378537762 1226 1227 >>> el_niño = NormalDist(4, 2.5) 1228 >>> temperature_feb += el_niño # Add in a climate effect 1229 >>> temperature_feb 1230 NormalDist(mu=10.0, sigma=6.830080526611674) 1231 1232 >>> temperature_feb * (9/5) + 32 # Convert to Fahrenheit 1233 NormalDist(mu=50.0, sigma=12.294144947901014) 1234 >>> temperature_feb.samples(3) # Generate random samples 1235 [7.672102882379219, 12.000027119750287, 4.647488369766392] 1236 1237 1238sys 1239--- 1240 1241Add new :func:`sys.unraisablehook` function which can be overridden to control 1242how "unraisable exceptions" are handled. It is called when an exception has 1243occurred but there is no way for Python to handle it. For example, when a 1244destructor raises an exception or during garbage collection 1245(:func:`gc.collect`). 1246(Contributed by Victor Stinner in :issue:`36829`.) 1247 1248 1249tarfile 1250------- 1251 1252The :mod:`tarfile` module now defaults to the modern pax (POSIX.1-2001) 1253format for new archives, instead of the previous GNU-specific one. 1254This improves cross-platform portability with a consistent encoding (UTF-8) 1255in a standardized and extensible format, and offers several other benefits. 1256(Contributed by C.A.M. Gerlach in :issue:`36268`.) 1257 1258 1259threading 1260--------- 1261 1262Add a new :func:`threading.excepthook` function which handles uncaught 1263:meth:`threading.Thread.run` exception. It can be overridden to control how 1264uncaught :meth:`threading.Thread.run` exceptions are handled. 1265(Contributed by Victor Stinner in :issue:`1230540`.) 1266 1267Add a new :func:`threading.get_native_id` function and 1268a :data:`~threading.Thread.native_id` 1269attribute to the :class:`threading.Thread` class. These return the native 1270integral Thread ID of the current thread assigned by the kernel. 1271This feature is only available on certain platforms, see 1272:func:`get_native_id <threading.get_native_id>` for more information. 1273(Contributed by Jake Tesler in :issue:`36084`.) 1274 1275 1276tokenize 1277-------- 1278 1279The :mod:`tokenize` module now implicitly emits a ``NEWLINE`` token when 1280provided with input that does not have a trailing new line. This behavior 1281now matches what the C tokenizer does internally. 1282(Contributed by Ammar Askar in :issue:`33899`.) 1283 1284 1285tkinter 1286------- 1287 1288Added methods :meth:`~tkinter.Spinbox.selection_from`, 1289:meth:`~tkinter.Spinbox.selection_present`, 1290:meth:`~tkinter.Spinbox.selection_range` and 1291:meth:`~tkinter.Spinbox.selection_to` 1292in the :class:`tkinter.Spinbox` class. 1293(Contributed by Juliette Monsel in :issue:`34829`.) 1294 1295Added method :meth:`~tkinter.Canvas.moveto` 1296in the :class:`tkinter.Canvas` class. 1297(Contributed by Juliette Monsel in :issue:`23831`.) 1298 1299The :class:`tkinter.PhotoImage` class now has 1300:meth:`~tkinter.PhotoImage.transparency_get` and 1301:meth:`~tkinter.PhotoImage.transparency_set` methods. (Contributed by 1302Zackery Spytz in :issue:`25451`.) 1303 1304 1305time 1306---- 1307 1308Added new clock :const:`~time.CLOCK_UPTIME_RAW` for macOS 10.12. 1309(Contributed by Joannah Nanjekye in :issue:`35702`.) 1310 1311 1312typing 1313------ 1314 1315The :mod:`typing` module incorporates several new features: 1316 1317* A dictionary type with per-key types. See :pep:`589` and 1318 :class:`typing.TypedDict`. 1319 TypedDict uses only string keys. By default, every key is required 1320 to be present. Specify "total=False" to allow keys to be optional:: 1321 1322 class Location(TypedDict, total=False): 1323 lat_long: tuple 1324 grid_square: str 1325 xy_coordinate: tuple 1326 1327* Literal types. See :pep:`586` and :class:`typing.Literal`. 1328 Literal types indicate that a parameter or return value 1329 is constrained to one or more specific literal values:: 1330 1331 def get_status(port: int) -> Literal['connected', 'disconnected']: 1332 ... 1333 1334* "Final" variables, functions, methods and classes. See :pep:`591`, 1335 :class:`typing.Final` and :func:`typing.final`. 1336 The final qualifier instructs a static type checker to restrict 1337 subclassing, overriding, or reassignment:: 1338 1339 pi: Final[float] = 3.1415926536 1340 1341* Protocol definitions. See :pep:`544`, :class:`typing.Protocol` and 1342 :func:`typing.runtime_checkable`. Simple ABCs like 1343 :class:`typing.SupportsInt` are now ``Protocol`` subclasses. 1344 1345* New protocol class :class:`typing.SupportsIndex`. 1346 1347* New functions :func:`typing.get_origin` and :func:`typing.get_args`. 1348 1349 1350unicodedata 1351----------- 1352 1353The :mod:`unicodedata` module has been upgraded to use the `Unicode 12.1.0 1354<https://blog.unicode.org/2019/05/unicode-12-1-en.html>`_ release. 1355 1356New function :func:`~unicodedata.is_normalized` can be used to verify a string 1357is in a specific normal form, often much faster than by actually normalizing 1358the string. (Contributed by Max Belanger, David Euresti, and Greg Price in 1359:issue:`32285` and :issue:`37966`). 1360 1361 1362unittest 1363-------- 1364 1365Added :class:`~unittest.mock.AsyncMock` to support an asynchronous version of 1366:class:`~unittest.mock.Mock`. Appropriate new assert functions for testing 1367have been added as well. 1368(Contributed by Lisa Roach in :issue:`26467`). 1369 1370Added :func:`~unittest.addModuleCleanup` and 1371:meth:`~unittest.TestCase.addClassCleanup` to unittest to support 1372cleanups for :func:`~unittest.setUpModule` and 1373:meth:`~unittest.TestCase.setUpClass`. 1374(Contributed by Lisa Roach in :issue:`24412`.) 1375 1376Several mock assert functions now also print a list of actual calls upon 1377failure. (Contributed by Petter Strandmark in :issue:`35047`.) 1378 1379:mod:`unittest` module gained support for coroutines to be used as test cases 1380with :class:`unittest.IsolatedAsyncioTestCase`. 1381(Contributed by Andrew Svetlov in :issue:`32972`.) 1382 1383Example:: 1384 1385 import unittest 1386 1387 1388 class TestRequest(unittest.IsolatedAsyncioTestCase): 1389 1390 async def asyncSetUp(self): 1391 self.connection = await AsyncConnection() 1392 1393 async def test_get(self): 1394 response = await self.connection.get("https://example.com") 1395 self.assertEqual(response.status_code, 200) 1396 1397 async def asyncTearDown(self): 1398 await self.connection.close() 1399 1400 1401 if __name__ == "__main__": 1402 unittest.main() 1403 1404 1405venv 1406---- 1407 1408:mod:`venv` now includes an ``Activate.ps1`` script on all platforms for 1409activating virtual environments under PowerShell Core 6.1. 1410(Contributed by Brett Cannon in :issue:`32718`.) 1411 1412 1413weakref 1414------- 1415 1416The proxy objects returned by :func:`weakref.proxy` now support the matrix 1417multiplication operators ``@`` and ``@=`` in addition to the other 1418numeric operators. (Contributed by Mark Dickinson in :issue:`36669`.) 1419 1420 1421xml 1422--- 1423 1424As mitigation against DTD and external entity retrieval, the 1425:mod:`xml.dom.minidom` and :mod:`xml.sax` modules no longer process 1426external entities by default. 1427(Contributed by Christian Heimes in :issue:`17239`.) 1428 1429The ``.find*()`` methods in the :mod:`xml.etree.ElementTree` module 1430support wildcard searches like ``{*}tag`` which ignores the namespace 1431and ``{namespace}*`` which returns all tags in the given namespace. 1432(Contributed by Stefan Behnel in :issue:`28238`.) 1433 1434The :mod:`xml.etree.ElementTree` module provides a new function 1435:func:`–xml.etree.ElementTree.canonicalize` that implements C14N 2.0. 1436(Contributed by Stefan Behnel in :issue:`13611`.) 1437 1438The target object of :class:`xml.etree.ElementTree.XMLParser` can 1439receive namespace declaration events through the new callback methods 1440``start_ns()`` and ``end_ns()``. Additionally, the 1441:class:`xml.etree.ElementTree.TreeBuilder` target can be configured 1442to process events about comments and processing instructions to include 1443them in the generated tree. 1444(Contributed by Stefan Behnel in :issue:`36676` and :issue:`36673`.) 1445 1446 1447xmlrpc 1448------ 1449 1450:class:`xmlrpc.client.ServerProxy` now supports an optional *headers* keyword 1451argument for a sequence of HTTP headers to be sent with each request. Among 1452other things, this makes it possible to upgrade from default basic 1453authentication to faster session authentication. 1454(Contributed by Cédric Krier in :issue:`35153`.) 1455 1456 1457Optimizations 1458============= 1459 1460* The :mod:`subprocess` module can now use the :func:`os.posix_spawn` function 1461 in some cases for better performance. Currently, it is only used on macOS 1462 and Linux (using glibc 2.24 or newer) if all these conditions are met: 1463 1464 * *close_fds* is false; 1465 * *preexec_fn*, *pass_fds*, *cwd* and *start_new_session* parameters 1466 are not set; 1467 * the *executable* path contains a directory. 1468 1469 (Contributed by Joannah Nanjekye and Victor Stinner in :issue:`35537`.) 1470 1471* :func:`shutil.copyfile`, :func:`shutil.copy`, :func:`shutil.copy2`, 1472 :func:`shutil.copytree` and :func:`shutil.move` use platform-specific 1473 "fast-copy" syscalls on Linux and macOS in order to copy the file 1474 more efficiently. 1475 "fast-copy" means that the copying operation occurs within the kernel, 1476 avoiding the use of userspace buffers in Python as in 1477 "``outfd.write(infd.read())``". 1478 On Windows :func:`shutil.copyfile` uses a bigger default buffer size (1 MiB 1479 instead of 16 KiB) and a :func:`memoryview`-based variant of 1480 :func:`shutil.copyfileobj` is used. 1481 The speedup for copying a 512 MiB file within the same partition is about 1482 +26% on Linux, +50% on macOS and +40% on Windows. Also, much less CPU cycles 1483 are consumed. 1484 See :ref:`shutil-platform-dependent-efficient-copy-operations` section. 1485 (Contributed by Giampaolo Rodolà in :issue:`33671`.) 1486 1487* :func:`shutil.copytree` uses :func:`os.scandir` function and all copy 1488 functions depending from it use cached :func:`os.stat` values. The speedup 1489 for copying a directory with 8000 files is around +9% on Linux, +20% on 1490 Windows and +30% on a Windows SMB share. Also the number of :func:`os.stat` 1491 syscalls is reduced by 38% making :func:`shutil.copytree` especially faster 1492 on network filesystems. (Contributed by Giampaolo Rodolà in :issue:`33695`.) 1493 1494* The default protocol in the :mod:`pickle` module is now Protocol 4, 1495 first introduced in Python 3.4. It offers better performance and smaller 1496 size compared to Protocol 3 available since Python 3.0. 1497 1498* Removed one :c:type:`Py_ssize_t` member from ``PyGC_Head``. All GC tracked 1499 objects (e.g. tuple, list, dict) size is reduced 4 or 8 bytes. 1500 (Contributed by Inada Naoki in :issue:`33597`.) 1501 1502* :class:`uuid.UUID` now uses ``__slots__`` to reduce its memory footprint. 1503 (Contributed by Wouter Bolsterlee and Tal Einat in :issue:`30977`) 1504 1505* Improved performance of :func:`operator.itemgetter` by 33%. Optimized 1506 argument handling and added a fast path for the common case of a single 1507 non-negative integer index into a tuple (which is the typical use case in 1508 the standard library). (Contributed by Raymond Hettinger in 1509 :issue:`35664`.) 1510 1511* Sped-up field lookups in :func:`collections.namedtuple`. They are now more 1512 than two times faster, making them the fastest form of instance variable 1513 lookup in Python. (Contributed by Raymond Hettinger, Pablo Galindo, and 1514 Joe Jevnik, Serhiy Storchaka in :issue:`32492`.) 1515 1516* The :class:`list` constructor does not overallocate the internal item buffer 1517 if the input iterable has a known length (the input implements ``__len__``). 1518 This makes the created list 12% smaller on average. (Contributed by 1519 Raymond Hettinger and Pablo Galindo in :issue:`33234`.) 1520 1521* Doubled the speed of class variable writes. When a non-dunder attribute 1522 was updated, there was an unnecessary call to update slots. 1523 (Contributed by Stefan Behnel, Pablo Galindo Salgado, Raymond Hettinger, 1524 Neil Schemenauer, and Serhiy Storchaka in :issue:`36012`.) 1525 1526* Reduced an overhead of converting arguments passed to many builtin functions 1527 and methods. This sped up calling some simple builtin functions and 1528 methods up to 20--50%. (Contributed by Serhiy Storchaka in :issue:`23867`, 1529 :issue:`35582` and :issue:`36127`.) 1530 1531* ``LOAD_GLOBAL`` instruction now uses new "per opcode cache" mechanism. 1532 It is about 40% faster now. (Contributed by Yury Selivanov and Inada Naoki in 1533 :issue:`26219`.) 1534 1535 1536Build and C API Changes 1537======================= 1538 1539* Default :data:`sys.abiflags` became an empty string: the ``m`` flag for 1540 pymalloc became useless (builds with and without pymalloc are ABI compatible) 1541 and so has been removed. (Contributed by Victor Stinner in :issue:`36707`.) 1542 1543 Example of changes: 1544 1545 * Only ``python3.8`` program is installed, ``python3.8m`` program is gone. 1546 * Only ``python3.8-config`` script is installed, ``python3.8m-config`` script 1547 is gone. 1548 * The ``m`` flag has been removed from the suffix of dynamic library 1549 filenames: extension modules in the standard library as well as those 1550 produced and installed by third-party packages, like those downloaded from 1551 PyPI. On Linux, for example, the Python 3.7 suffix 1552 ``.cpython-37m-x86_64-linux-gnu.so`` became 1553 ``.cpython-38-x86_64-linux-gnu.so`` in Python 3.8. 1554 1555* The header files have been reorganized to better separate the different kinds 1556 of APIs: 1557 1558 * ``Include/*.h`` should be the portable public stable C API. 1559 * ``Include/cpython/*.h`` should be the unstable C API specific to CPython; 1560 public API, with some private API prefixed by ``_Py`` or ``_PY``. 1561 * ``Include/internal/*.h`` is the private internal C API very specific to 1562 CPython. This API comes with no backward compatibility warranty and should 1563 not be used outside CPython. It is only exposed for very specific needs 1564 like debuggers and profiles which has to access to CPython internals 1565 without calling functions. This API is now installed by ``make install``. 1566 1567 (Contributed by Victor Stinner in :issue:`35134` and :issue:`35081`, 1568 work initiated by Eric Snow in Python 3.7.) 1569 1570* Some macros have been converted to static inline functions: parameter types 1571 and return type are well defined, they don't have issues specific to macros, 1572 variables have a local scopes. Examples: 1573 1574 * :c:func:`Py_INCREF`, :c:func:`Py_DECREF` 1575 * :c:func:`Py_XINCREF`, :c:func:`Py_XDECREF` 1576 * :c:func:`PyObject_INIT`, :c:func:`PyObject_INIT_VAR` 1577 * Private functions: :c:func:`!_PyObject_GC_TRACK`, 1578 :c:func:`!_PyObject_GC_UNTRACK`, :c:func:`!_Py_Dealloc` 1579 1580 (Contributed by Victor Stinner in :issue:`35059`.) 1581 1582* The :c:func:`!PyByteArray_Init` and :c:func:`!PyByteArray_Fini` functions have 1583 been removed. They did nothing since Python 2.7.4 and Python 3.2.0, were 1584 excluded from the limited API (stable ABI), and were not documented. 1585 (Contributed by Victor Stinner in :issue:`35713`.) 1586 1587* The result of :c:func:`PyExceptionClass_Name` is now of type 1588 ``const char *`` rather of ``char *``. 1589 (Contributed by Serhiy Storchaka in :issue:`33818`.) 1590 1591* The duality of ``Modules/Setup.dist`` and ``Modules/Setup`` has been 1592 removed. Previously, when updating the CPython source tree, one had 1593 to manually copy ``Modules/Setup.dist`` (inside the source tree) to 1594 ``Modules/Setup`` (inside the build tree) in order to reflect any changes 1595 upstream. This was of a small benefit to packagers at the expense of 1596 a frequent annoyance to developers following CPython development, as 1597 forgetting to copy the file could produce build failures. 1598 1599 Now the build system always reads from ``Modules/Setup`` inside the source 1600 tree. People who want to customize that file are encouraged to maintain 1601 their changes in a git fork of CPython or as patch files, as they would do 1602 for any other change to the source tree. 1603 1604 (Contributed by Antoine Pitrou in :issue:`32430`.) 1605 1606* Functions that convert Python number to C integer like 1607 :c:func:`PyLong_AsLong` and argument parsing functions like 1608 :c:func:`PyArg_ParseTuple` with integer converting format units like ``'i'`` 1609 will now use the :meth:`~object.__index__` special method instead of 1610 :meth:`~object.__int__`, if available. The deprecation warning will be 1611 emitted for objects with the ``__int__()`` method but without the 1612 ``__index__()`` method (like :class:`~decimal.Decimal` and 1613 :class:`~fractions.Fraction`). :c:func:`PyNumber_Check` will now return 1614 ``1`` for objects implementing ``__index__()``. 1615 :c:func:`PyNumber_Long`, :c:func:`PyNumber_Float` and 1616 :c:func:`PyFloat_AsDouble` also now use the ``__index__()`` method if 1617 available. 1618 (Contributed by Serhiy Storchaka in :issue:`36048` and :issue:`20092`.) 1619 1620* Heap-allocated type objects will now increase their reference count 1621 in :c:func:`PyObject_Init` (and its parallel macro ``PyObject_INIT``) 1622 instead of in :c:func:`PyType_GenericAlloc`. Types that modify instance 1623 allocation or deallocation may need to be adjusted. 1624 (Contributed by Eddie Elizondo in :issue:`35810`.) 1625 1626* The new function :c:func:`!PyCode_NewWithPosOnlyArgs` allows to create 1627 code objects like :c:func:`!PyCode_New`, but with an extra *posonlyargcount* 1628 parameter for indicating the number of positional-only arguments. 1629 (Contributed by Pablo Galindo in :issue:`37221`.) 1630 1631* :c:func:`!Py_SetPath` now sets :data:`sys.executable` to the program full 1632 path (:c:func:`Py_GetProgramFullPath`) rather than to the program name 1633 (:c:func:`Py_GetProgramName`). 1634 (Contributed by Victor Stinner in :issue:`38234`.) 1635 1636 1637Deprecated 1638========== 1639 1640* The distutils ``bdist_wininst`` command is now deprecated, use 1641 ``bdist_wheel`` (wheel packages) instead. 1642 (Contributed by Victor Stinner in :issue:`37481`.) 1643 1644* Deprecated methods ``getchildren()`` and ``getiterator()`` in 1645 the :mod:`~xml.etree.ElementTree` module now emit a 1646 :exc:`DeprecationWarning` instead of :exc:`PendingDeprecationWarning`. 1647 They will be removed in Python 3.9. 1648 (Contributed by Serhiy Storchaka in :issue:`29209`.) 1649 1650* Passing an object that is not an instance of 1651 :class:`concurrent.futures.ThreadPoolExecutor` to 1652 :meth:`loop.set_default_executor() <asyncio.loop.set_default_executor>` is 1653 deprecated and will be prohibited in Python 3.9. 1654 (Contributed by Elvis Pranskevichus in :issue:`34075`.) 1655 1656* The :meth:`~object.__getitem__` methods of :class:`xml.dom.pulldom.DOMEventStream`, 1657 :class:`wsgiref.util.FileWrapper` and :class:`fileinput.FileInput` have been 1658 deprecated. 1659 1660 Implementations of these methods have been ignoring their *index* parameter, 1661 and returning the next item instead. 1662 (Contributed by Berker Peksag in :issue:`9372`.) 1663 1664* The :class:`typing.NamedTuple` class has deprecated the ``_field_types`` 1665 attribute in favor of the ``__annotations__`` attribute which has the same 1666 information. (Contributed by Raymond Hettinger in :issue:`36320`.) 1667 1668* :mod:`ast` classes ``Num``, ``Str``, ``Bytes``, ``NameConstant`` and 1669 ``Ellipsis`` are considered deprecated and will be removed in future Python 1670 versions. :class:`~ast.Constant` should be used instead. 1671 (Contributed by Serhiy Storchaka in :issue:`32892`.) 1672 1673* :class:`ast.NodeVisitor` methods ``visit_Num()``, ``visit_Str()``, 1674 ``visit_Bytes()``, ``visit_NameConstant()`` and ``visit_Ellipsis()`` are 1675 deprecated now and will not be called in future Python versions. 1676 Add the :meth:`~ast.NodeVisitor.visit_Constant` method to handle all 1677 constant nodes. 1678 (Contributed by Serhiy Storchaka in :issue:`36917`.) 1679 1680* The :func:`asyncio.coroutine` :term:`decorator` is deprecated and will be 1681 removed in version 3.10. Instead of ``@asyncio.coroutine``, use 1682 :keyword:`async def` instead. 1683 (Contributed by Andrew Svetlov in :issue:`36921`.) 1684 1685* In :mod:`asyncio`, the explicit passing of a *loop* argument has been 1686 deprecated and will be removed in version 3.10 for the following: 1687 :func:`asyncio.sleep`, :func:`asyncio.gather`, :func:`asyncio.shield`, 1688 :func:`asyncio.wait_for`, :func:`asyncio.wait`, :func:`asyncio.as_completed`, 1689 :class:`asyncio.Task`, :class:`asyncio.Lock`, :class:`asyncio.Event`, 1690 :class:`asyncio.Condition`, :class:`asyncio.Semaphore`, 1691 :class:`asyncio.BoundedSemaphore`, :class:`asyncio.Queue`, 1692 :func:`asyncio.create_subprocess_exec`, and 1693 :func:`asyncio.create_subprocess_shell`. 1694 1695* The explicit passing of coroutine objects to :func:`asyncio.wait` has been 1696 deprecated and will be removed in version 3.11. 1697 (Contributed by Yury Selivanov in :issue:`34790`.) 1698 1699* The following functions and methods are deprecated in the :mod:`gettext` 1700 module: :func:`~gettext.lgettext`, :func:`~gettext.ldgettext`, 1701 :func:`~gettext.lngettext` and :func:`~gettext.ldngettext`. 1702 They return encoded bytes, and it's possible that you will get unexpected 1703 Unicode-related exceptions if there are encoding problems with the 1704 translated strings. It's much better to use alternatives which return 1705 Unicode strings in Python 3. These functions have been broken for a long time. 1706 1707 Function :func:`~gettext.bind_textdomain_codeset`, methods 1708 :meth:`~gettext.NullTranslations.output_charset` and 1709 :meth:`~gettext.NullTranslations.set_output_charset`, and the *codeset* 1710 parameter of functions :func:`~gettext.translation` and 1711 :func:`~gettext.install` are also deprecated, since they are only used for 1712 the ``l*gettext()`` functions. 1713 (Contributed by Serhiy Storchaka in :issue:`33710`.) 1714 1715* The :meth:`~threading.Thread.isAlive` method of :class:`threading.Thread` 1716 has been deprecated. 1717 (Contributed by Donghee Na in :issue:`35283`.) 1718 1719* Many builtin and extension functions that take integer arguments will 1720 now emit a deprecation warning for :class:`~decimal.Decimal`\ s, 1721 :class:`~fractions.Fraction`\ s and any other objects that can be converted 1722 to integers only with a loss (e.g. that have the :meth:`~object.__int__` 1723 method but do not have the :meth:`~object.__index__` method). In future 1724 version they will be errors. 1725 (Contributed by Serhiy Storchaka in :issue:`36048`.) 1726 1727* Deprecated passing the following arguments as keyword arguments: 1728 1729 - *func* in :func:`functools.partialmethod`, :func:`weakref.finalize`, 1730 :meth:`profile.Profile.runcall`, :meth:`cProfile.Profile.runcall`, 1731 :meth:`bdb.Bdb.runcall`, :meth:`trace.Trace.runfunc` and 1732 :func:`curses.wrapper`. 1733 - *function* in :meth:`unittest.TestCase.addCleanup`. 1734 - *fn* in the :meth:`~concurrent.futures.Executor.submit` method of 1735 :class:`concurrent.futures.ThreadPoolExecutor` and 1736 :class:`concurrent.futures.ProcessPoolExecutor`. 1737 - *callback* in :meth:`contextlib.ExitStack.callback`, 1738 :meth:`contextlib.AsyncExitStack.callback` and 1739 :meth:`contextlib.AsyncExitStack.push_async_callback`. 1740 - *c* and *typeid* in the :meth:`~multiprocessing.managers.Server.create` 1741 method of :class:`multiprocessing.managers.Server` and 1742 :class:`multiprocessing.managers.SharedMemoryServer`. 1743 - *obj* in :func:`weakref.finalize`. 1744 1745 In future releases of Python, they will be :ref:`positional-only 1746 <positional-only_parameter>`. 1747 (Contributed by Serhiy Storchaka in :issue:`36492`.) 1748 1749 1750API and Feature Removals 1751======================== 1752 1753The following features and APIs have been removed from Python 3.8: 1754 1755* Starting with Python 3.3, importing ABCs from :mod:`collections` was 1756 deprecated, and importing should be done from :mod:`collections.abc`. Being 1757 able to import from collections was marked for removal in 3.8, but has been 1758 delayed to 3.9. (See :gh:`81134`.) 1759 1760* The :mod:`macpath` module, deprecated in Python 3.7, has been removed. 1761 (Contributed by Victor Stinner in :issue:`35471`.) 1762 1763* The function :func:`platform.popen` has been removed, after having been 1764 deprecated since Python 3.3: use :func:`os.popen` instead. 1765 (Contributed by Victor Stinner in :issue:`35345`.) 1766 1767* The function :func:`time.clock` has been removed, after having been 1768 deprecated since Python 3.3: use :func:`time.perf_counter` or 1769 :func:`time.process_time` instead, depending 1770 on your requirements, to have well-defined behavior. 1771 (Contributed by Matthias Bussonnier in :issue:`36895`.) 1772 1773* The ``pyvenv`` script has been removed in favor of ``python3.8 -m venv`` 1774 to help eliminate confusion as to what Python interpreter the ``pyvenv`` 1775 script is tied to. (Contributed by Brett Cannon in :issue:`25427`.) 1776 1777* ``parse_qs``, ``parse_qsl``, and ``escape`` are removed from the :mod:`!cgi` 1778 module. They are deprecated in Python 3.2 or older. They should be imported 1779 from the ``urllib.parse`` and ``html`` modules instead. 1780 1781* ``filemode`` function is removed from the :mod:`tarfile` module. 1782 It is not documented and deprecated since Python 3.3. 1783 1784* The :class:`~xml.etree.ElementTree.XMLParser` constructor no longer accepts 1785 the *html* argument. It never had an effect and was deprecated in Python 3.4. 1786 All other parameters are now :ref:`keyword-only <keyword-only_parameter>`. 1787 (Contributed by Serhiy Storchaka in :issue:`29209`.) 1788 1789* Removed the ``doctype()`` method of :class:`~xml.etree.ElementTree.XMLParser`. 1790 (Contributed by Serhiy Storchaka in :issue:`29209`.) 1791 1792* "unicode_internal" codec is removed. 1793 (Contributed by Inada Naoki in :issue:`36297`.) 1794 1795* The ``Cache`` and ``Statement`` objects of the :mod:`sqlite3` module are not 1796 exposed to the user. 1797 (Contributed by Aviv Palivoda in :issue:`30262`.) 1798 1799* The ``bufsize`` keyword argument of :func:`fileinput.input` and 1800 :func:`fileinput.FileInput` which was ignored and deprecated since Python 3.6 1801 has been removed. :issue:`36952` (Contributed by Matthias Bussonnier.) 1802 1803* The functions :func:`sys.set_coroutine_wrapper` and 1804 :func:`sys.get_coroutine_wrapper` deprecated in Python 3.7 have been removed; 1805 :issue:`36933` (Contributed by Matthias Bussonnier.) 1806 1807 1808Porting to Python 3.8 1809===================== 1810 1811This section lists previously described changes and other bugfixes 1812that may require changes to your code. 1813 1814 1815Changes in Python behavior 1816-------------------------- 1817 1818* Yield expressions (both ``yield`` and ``yield from`` clauses) are now disallowed 1819 in comprehensions and generator expressions (aside from the iterable expression 1820 in the leftmost :keyword:`!for` clause). 1821 (Contributed by Serhiy Storchaka in :issue:`10544`.) 1822 1823* The compiler now produces a :exc:`SyntaxWarning` when identity checks 1824 (``is`` and ``is not``) are used with certain types of literals 1825 (e.g. strings, numbers). These can often work by accident in CPython, 1826 but are not guaranteed by the language spec. The warning advises users 1827 to use equality tests (``==`` and ``!=``) instead. 1828 (Contributed by Serhiy Storchaka in :issue:`34850`.) 1829 1830* The CPython interpreter can swallow exceptions in some circumstances. 1831 In Python 3.8 this happens in fewer cases. In particular, exceptions 1832 raised when getting the attribute from the type dictionary are no longer 1833 ignored. (Contributed by Serhiy Storchaka in :issue:`35459`.) 1834 1835* Removed ``__str__`` implementations from builtin types :class:`bool`, 1836 :class:`int`, :class:`float`, :class:`complex` and few classes from 1837 the standard library. They now inherit ``__str__()`` from :class:`object`. 1838 As result, defining the ``__repr__()`` method in the subclass of these 1839 classes will affect their string representation. 1840 (Contributed by Serhiy Storchaka in :issue:`36793`.) 1841 1842* On AIX, :data:`sys.platform` doesn't contain the major version anymore. 1843 It is always ``'aix'``, instead of ``'aix3'`` .. ``'aix7'``. Since 1844 older Python versions include the version number, so it is recommended to 1845 always use ``sys.platform.startswith('aix')``. 1846 (Contributed by M. Felt in :issue:`36588`.) 1847 1848* :c:func:`!PyEval_AcquireLock` and :c:func:`!PyEval_AcquireThread` now 1849 terminate the current thread if called while the interpreter is 1850 finalizing, making them consistent with :c:func:`PyEval_RestoreThread`, 1851 :c:func:`Py_END_ALLOW_THREADS`, and :c:func:`PyGILState_Ensure`. If this 1852 behavior is not desired, guard the call by checking :c:func:`!_Py_IsFinalizing` 1853 or :func:`sys.is_finalizing`. 1854 (Contributed by Joannah Nanjekye in :issue:`36475`.) 1855 1856 1857Changes in the Python API 1858------------------------- 1859 1860* The :func:`os.getcwdb` function now uses the UTF-8 encoding on Windows, 1861 rather than the ANSI code page: see :pep:`529` for the rationale. The 1862 function is no longer deprecated on Windows. 1863 (Contributed by Victor Stinner in :issue:`37412`.) 1864 1865* :class:`subprocess.Popen` can now use :func:`os.posix_spawn` in some cases 1866 for better performance. On Windows Subsystem for Linux and QEMU User 1867 Emulation, the :class:`Popen` constructor using :func:`os.posix_spawn` no longer raises an 1868 exception on errors like "missing program". Instead the child process fails with a 1869 non-zero :attr:`~Popen.returncode`. 1870 (Contributed by Joannah Nanjekye and Victor Stinner in :issue:`35537`.) 1871 1872* The *preexec_fn* argument of * :class:`subprocess.Popen` is no longer 1873 compatible with subinterpreters. The use of the parameter in a 1874 subinterpreter now raises :exc:`RuntimeError`. 1875 (Contributed by Eric Snow in :issue:`34651`, modified by Christian Heimes 1876 in :issue:`37951`.) 1877 1878* The :meth:`imap.IMAP4.logout` method no longer silently ignores arbitrary 1879 exceptions. 1880 (Contributed by Victor Stinner in :issue:`36348`.) 1881 1882* The function :func:`platform.popen` has been removed, after having been deprecated since 1883 Python 3.3: use :func:`os.popen` instead. 1884 (Contributed by Victor Stinner in :issue:`35345`.) 1885 1886* The :func:`statistics.mode` function no longer raises an exception 1887 when given multimodal data. Instead, it returns the first mode 1888 encountered in the input data. (Contributed by Raymond Hettinger 1889 in :issue:`35892`.) 1890 1891* The :meth:`~tkinter.ttk.Treeview.selection` method of the 1892 :class:`tkinter.ttk.Treeview` class no longer takes arguments. Using it with 1893 arguments for changing the selection was deprecated in Python 3.6. Use 1894 specialized methods like :meth:`~tkinter.ttk.Treeview.selection_set` for 1895 changing the selection. (Contributed by Serhiy Storchaka in :issue:`31508`.) 1896 1897* The :meth:`writexml`, :meth:`toxml` and :meth:`toprettyxml` methods of 1898 :mod:`xml.dom.minidom`, and the :meth:`write` method of :mod:`xml.etree`, 1899 now preserve the attribute order specified by the user. 1900 (Contributed by Diego Rojas and Raymond Hettinger in :issue:`34160`.) 1901 1902* A :mod:`dbm.dumb` database opened with flags ``'r'`` is now read-only. 1903 :func:`dbm.dumb.open` with flags ``'r'`` and ``'w'`` no longer creates 1904 a database if it does not exist. 1905 (Contributed by Serhiy Storchaka in :issue:`32749`.) 1906 1907* The ``doctype()`` method defined in a subclass of 1908 :class:`~xml.etree.ElementTree.XMLParser` will no longer be called and will 1909 emit a :exc:`RuntimeWarning` instead of a :exc:`DeprecationWarning`. 1910 Define the :meth:`doctype() <xml.etree.ElementTree.TreeBuilder.doctype>` 1911 method on a target for handling an XML doctype declaration. 1912 (Contributed by Serhiy Storchaka in :issue:`29209`.) 1913 1914* A :exc:`RuntimeError` is now raised when the custom metaclass doesn't 1915 provide the ``__classcell__`` entry in the namespace passed to 1916 ``type.__new__``. A :exc:`DeprecationWarning` was emitted in Python 1917 3.6--3.7. (Contributed by Serhiy Storchaka in :issue:`23722`.) 1918 1919* The :class:`cProfile.Profile` class can now be used as a context 1920 manager. (Contributed by Scott Sanderson in :issue:`29235`.) 1921 1922* :func:`shutil.copyfile`, :func:`shutil.copy`, :func:`shutil.copy2`, 1923 :func:`shutil.copytree` and :func:`shutil.move` use platform-specific 1924 "fast-copy" syscalls (see 1925 :ref:`shutil-platform-dependent-efficient-copy-operations` section). 1926 1927* :func:`shutil.copyfile` default buffer size on Windows was changed from 1928 16 KiB to 1 MiB. 1929 1930* The ``PyGC_Head`` struct has changed completely. All code that touched the 1931 struct member should be rewritten. (See :issue:`33597`.) 1932 1933* The :c:type:`PyInterpreterState` struct has been moved into the "internal" 1934 header files (specifically Include/internal/pycore_pystate.h). An 1935 opaque ``PyInterpreterState`` is still available as part of the public 1936 API (and stable ABI). The docs indicate that none of the struct's 1937 fields are public, so we hope no one has been using them. However, 1938 if you do rely on one or more of those private fields and have no 1939 alternative then please open a BPO issue. We'll work on helping 1940 you adjust (possibly including adding accessor functions to the 1941 public API). (See :issue:`35886`.) 1942 1943* The :meth:`mmap.flush() <mmap.mmap.flush>` method now returns ``None`` on 1944 success and raises an exception on error under all platforms. Previously, 1945 its behavior was platform-dependent: a nonzero value was returned on success; 1946 zero was returned on error under Windows. A zero value was returned on 1947 success; an exception was raised on error under Unix. 1948 (Contributed by Berker Peksag in :issue:`2122`.) 1949 1950* :mod:`xml.dom.minidom` and :mod:`xml.sax` modules no longer process 1951 external entities by default. 1952 (Contributed by Christian Heimes in :issue:`17239`.) 1953 1954* Deleting a key from a read-only :mod:`dbm` database (:mod:`dbm.dumb`, 1955 :mod:`dbm.gnu` or :mod:`dbm.ndbm`) raises :attr:`error` (:exc:`dbm.dumb.error`, 1956 :exc:`dbm.gnu.error` or :exc:`dbm.ndbm.error`) instead of :exc:`KeyError`. 1957 (Contributed by Xiang Zhang in :issue:`33106`.) 1958 1959* Simplified AST for literals. All constants will be represented as 1960 :class:`ast.Constant` instances. Instantiating old classes ``Num``, 1961 ``Str``, ``Bytes``, ``NameConstant`` and ``Ellipsis`` will return 1962 an instance of ``Constant``. 1963 (Contributed by Serhiy Storchaka in :issue:`32892`.) 1964 1965* :func:`~os.path.expanduser` on Windows now prefers the :envvar:`USERPROFILE` 1966 environment variable and does not use :envvar:`HOME`, which is not normally 1967 set for regular user accounts. 1968 (Contributed by Anthony Sottile in :issue:`36264`.) 1969 1970* The exception :class:`asyncio.CancelledError` now inherits from 1971 :class:`BaseException` rather than :class:`Exception` and no longer inherits 1972 from :class:`concurrent.futures.CancelledError`. 1973 (Contributed by Yury Selivanov in :issue:`32528`.) 1974 1975* The function :func:`asyncio.wait_for` now correctly waits for cancellation 1976 when using an instance of :class:`asyncio.Task`. Previously, upon reaching 1977 *timeout*, it was cancelled and immediately returned. 1978 (Contributed by Elvis Pranskevichus in :issue:`32751`.) 1979 1980* The function :func:`asyncio.BaseTransport.get_extra_info` now returns a safe 1981 to use socket object when 'socket' is passed to the *name* parameter. 1982 (Contributed by Yury Selivanov in :issue:`37027`.) 1983 1984* :class:`asyncio.BufferedProtocol` has graduated to the stable API. 1985 1986.. _bpo-36085-whatsnew: 1987 1988* DLL dependencies for extension modules and DLLs loaded with :mod:`ctypes` on 1989 Windows are now resolved more securely. Only the system paths, the directory 1990 containing the DLL or PYD file, and directories added with 1991 :func:`~os.add_dll_directory` are searched for load-time dependencies. 1992 Specifically, :envvar:`PATH` and the current working directory are no longer 1993 used, and modifications to these will no longer have any effect on normal DLL 1994 resolution. If your application relies on these mechanisms, you should check 1995 for :func:`~os.add_dll_directory` and if it exists, use it to add your DLLs 1996 directory while loading your library. Note that Windows 7 users will need to 1997 ensure that Windows Update KB2533623 has been installed (this is also verified 1998 by the installer). 1999 (Contributed by Steve Dower in :issue:`36085`.) 2000 2001* The header files and functions related to pgen have been removed after its 2002 replacement by a pure Python implementation. (Contributed by Pablo Galindo 2003 in :issue:`36623`.) 2004 2005* :class:`types.CodeType` has a new parameter in the second position of the 2006 constructor (*posonlyargcount*) to support positional-only arguments defined 2007 in :pep:`570`. The first argument (*argcount*) now represents the total 2008 number of positional arguments (including positional-only arguments). The new 2009 ``replace()`` method of :class:`types.CodeType` can be used to make the code 2010 future-proof. 2011 2012* The parameter ``digestmod`` for :func:`hmac.new` no longer uses the MD5 digest 2013 by default. 2014 2015Changes in the C API 2016-------------------- 2017 2018* The :c:struct:`PyCompilerFlags` structure got a new *cf_feature_version* 2019 field. It should be initialized to ``PY_MINOR_VERSION``. The field is ignored 2020 by default, and is used if and only if ``PyCF_ONLY_AST`` flag is set in 2021 *cf_flags*. 2022 (Contributed by Guido van Rossum in :issue:`35766`.) 2023 2024* The :c:func:`!PyEval_ReInitThreads` function has been removed from the C API. 2025 It should not be called explicitly: use :c:func:`PyOS_AfterFork_Child` 2026 instead. 2027 (Contributed by Victor Stinner in :issue:`36728`.) 2028 2029* On Unix, C extensions are no longer linked to libpython except on Android 2030 and Cygwin. When Python is embedded, ``libpython`` must not be loaded with 2031 ``RTLD_LOCAL``, but ``RTLD_GLOBAL`` instead. Previously, using 2032 ``RTLD_LOCAL``, it was already not possible to load C extensions which 2033 were not linked to ``libpython``, like C extensions of the standard 2034 library built by the ``*shared*`` section of ``Modules/Setup``. 2035 (Contributed by Victor Stinner in :issue:`21536`.) 2036 2037* Use of ``#`` variants of formats in parsing or building value (e.g. 2038 :c:func:`PyArg_ParseTuple`, :c:func:`Py_BuildValue`, :c:func:`PyObject_CallFunction`, 2039 etc.) without ``PY_SSIZE_T_CLEAN`` defined raises ``DeprecationWarning`` now. 2040 It will be removed in 3.10 or 4.0. Read :ref:`arg-parsing` for detail. 2041 (Contributed by Inada Naoki in :issue:`36381`.) 2042 2043* Instances of heap-allocated types (such as those created with 2044 :c:func:`PyType_FromSpec`) hold a reference to their type object. 2045 Increasing the reference count of these type objects has been moved from 2046 :c:func:`PyType_GenericAlloc` to the more low-level functions, 2047 :c:func:`PyObject_Init` and :c:func:`PyObject_INIT`. 2048 This makes types created through :c:func:`PyType_FromSpec` behave like 2049 other classes in managed code. 2050 2051 :ref:`Statically allocated types <static-types>` are not affected. 2052 2053 For the vast majority of cases, there should be no side effect. 2054 However, types that manually increase the reference count after allocating 2055 an instance (perhaps to work around the bug) may now become immortal. 2056 To avoid this, these classes need to call Py_DECREF on the type object 2057 during instance deallocation. 2058 2059 To correctly port these types into 3.8, please apply the following 2060 changes: 2061 2062 * Remove :c:macro:`Py_INCREF` on the type object after allocating an 2063 instance - if any. 2064 This may happen after calling :c:macro:`PyObject_New`, 2065 :c:macro:`PyObject_NewVar`, :c:func:`PyObject_GC_New`, 2066 :c:func:`PyObject_GC_NewVar`, or any other custom allocator that uses 2067 :c:func:`PyObject_Init` or :c:func:`PyObject_INIT`. 2068 2069 Example: 2070 2071 .. code-block:: c 2072 2073 static foo_struct * 2074 foo_new(PyObject *type) { 2075 foo_struct *foo = PyObject_GC_New(foo_struct, (PyTypeObject *) type); 2076 if (foo == NULL) 2077 return NULL; 2078 #if PY_VERSION_HEX < 0x03080000 2079 // Workaround for Python issue 35810; no longer necessary in Python 3.8 2080 PY_INCREF(type) 2081 #endif 2082 return foo; 2083 } 2084 2085 * Ensure that all custom ``tp_dealloc`` functions of heap-allocated types 2086 decrease the type's reference count. 2087 2088 Example: 2089 2090 .. code-block:: c 2091 2092 static void 2093 foo_dealloc(foo_struct *instance) { 2094 PyObject *type = Py_TYPE(instance); 2095 PyObject_GC_Del(instance); 2096 #if PY_VERSION_HEX >= 0x03080000 2097 // This was not needed before Python 3.8 (Python issue 35810) 2098 Py_DECREF(type); 2099 #endif 2100 } 2101 2102 (Contributed by Eddie Elizondo in :issue:`35810`.) 2103 2104* The :c:macro:`Py_DEPRECATED()` macro has been implemented for MSVC. 2105 The macro now must be placed before the symbol name. 2106 2107 Example: 2108 2109 .. code-block:: c 2110 2111 Py_DEPRECATED(3.8) PyAPI_FUNC(int) Py_OldFunction(void); 2112 2113 (Contributed by Zackery Spytz in :issue:`33407`.) 2114 2115* The interpreter does not pretend to support binary compatibility of 2116 extension types across feature releases, anymore. A :c:type:`PyTypeObject` 2117 exported by a third-party extension module is supposed to have all the 2118 slots expected in the current Python version, including 2119 :c:member:`~PyTypeObject.tp_finalize` (:c:macro:`Py_TPFLAGS_HAVE_FINALIZE` 2120 is not checked anymore before reading :c:member:`~PyTypeObject.tp_finalize`). 2121 2122 (Contributed by Antoine Pitrou in :issue:`32388`.) 2123 2124* The functions :c:func:`!PyNode_AddChild` and :c:func:`!PyParser_AddToken` now accept 2125 two additional ``int`` arguments *end_lineno* and *end_col_offset*. 2126 2127* The :file:`libpython38.a` file to allow MinGW tools to link directly against 2128 :file:`python38.dll` is no longer included in the regular Windows distribution. 2129 If you require this file, it may be generated with the ``gendef`` and 2130 ``dlltool`` tools, which are part of the MinGW binutils package: 2131 2132 .. code-block:: shell 2133 2134 gendef - python38.dll > tmp.def 2135 dlltool --dllname python38.dll --def tmp.def --output-lib libpython38.a 2136 2137 The location of an installed :file:`pythonXY.dll` will depend on the 2138 installation options and the version and language of Windows. See 2139 :ref:`using-on-windows` for more information. The resulting library should be 2140 placed in the same directory as :file:`pythonXY.lib`, which is generally the 2141 :file:`libs` directory under your Python installation. 2142 2143 (Contributed by Steve Dower in :issue:`37351`.) 2144 2145 2146CPython bytecode changes 2147------------------------ 2148 2149* The interpreter loop has been simplified by moving the logic of unrolling 2150 the stack of blocks into the compiler. The compiler emits now explicit 2151 instructions for adjusting the stack of values and calling the 2152 cleaning-up code for :keyword:`break`, :keyword:`continue` and 2153 :keyword:`return`. 2154 2155 Removed opcodes :opcode:`!BREAK_LOOP`, :opcode:`!CONTINUE_LOOP`, 2156 :opcode:`!SETUP_LOOP` and :opcode:`!SETUP_EXCEPT`. Added new opcodes 2157 :opcode:`!ROT_FOUR`, :opcode:`!BEGIN_FINALLY`, :opcode:`!CALL_FINALLY` and 2158 :opcode:`!POP_FINALLY`. Changed the behavior of :opcode:`!END_FINALLY` 2159 and :opcode:`!WITH_CLEANUP_START`. 2160 2161 (Contributed by Mark Shannon, Antoine Pitrou and Serhiy Storchaka in 2162 :issue:`17611`.) 2163 2164* Added new opcode :opcode:`END_ASYNC_FOR` for handling exceptions raised 2165 when awaiting a next item in an :keyword:`async for` loop. 2166 (Contributed by Serhiy Storchaka in :issue:`33041`.) 2167 2168* The :opcode:`MAP_ADD` now expects the value as the first element in the 2169 stack and the key as the second element. This change was made so the key 2170 is always evaluated before the value in dictionary comprehensions, as 2171 proposed by :pep:`572`. (Contributed by Jörn Heissler in :issue:`35224`.) 2172 2173 2174Demos and Tools 2175--------------- 2176 2177Added a benchmark script for timing various ways to access variables: 2178``Tools/scripts/var_access_benchmark.py``. 2179(Contributed by Raymond Hettinger in :issue:`35884`.) 2180 2181Here's a summary of performance improvements since Python 3.3: 2182 2183.. code-block:: none 2184 2185 Python version 3.3 3.4 3.5 3.6 3.7 3.8 2186 -------------- --- --- --- --- --- --- 2187 2188 Variable and attribute read access: 2189 read_local 4.0 7.1 7.1 5.4 5.1 3.9 2190 read_nonlocal 5.3 7.1 8.1 5.8 5.4 4.4 2191 read_global 13.3 15.5 19.0 14.3 13.6 7.6 2192 read_builtin 20.0 21.1 21.6 18.5 19.0 7.5 2193 read_classvar_from_class 20.5 25.6 26.5 20.7 19.5 18.4 2194 read_classvar_from_instance 18.5 22.8 23.5 18.8 17.1 16.4 2195 read_instancevar 26.8 32.4 33.1 28.0 26.3 25.4 2196 read_instancevar_slots 23.7 27.8 31.3 20.8 20.8 20.2 2197 read_namedtuple 68.5 73.8 57.5 45.0 46.8 18.4 2198 read_boundmethod 29.8 37.6 37.9 29.6 26.9 27.7 2199 2200 Variable and attribute write access: 2201 write_local 4.6 8.7 9.3 5.5 5.3 4.3 2202 write_nonlocal 7.3 10.5 11.1 5.6 5.5 4.7 2203 write_global 15.9 19.7 21.2 18.0 18.0 15.8 2204 write_classvar 81.9 92.9 96.0 104.6 102.1 39.2 2205 write_instancevar 36.4 44.6 45.8 40.0 38.9 35.5 2206 write_instancevar_slots 28.7 35.6 36.1 27.3 26.6 25.7 2207 2208 Data structure read access: 2209 read_list 19.2 24.2 24.5 20.8 20.8 19.0 2210 read_deque 19.9 24.7 25.5 20.2 20.6 19.8 2211 read_dict 19.7 24.3 25.7 22.3 23.0 21.0 2212 read_strdict 17.9 22.6 24.3 19.5 21.2 18.9 2213 2214 Data structure write access: 2215 write_list 21.2 27.1 28.5 22.5 21.6 20.0 2216 write_deque 23.8 28.7 30.1 22.7 21.8 23.5 2217 write_dict 25.9 31.4 33.3 29.3 29.2 24.7 2218 write_strdict 22.9 28.4 29.9 27.5 25.2 23.1 2219 2220 Stack (or queue) operations: 2221 list_append_pop 144.2 93.4 112.7 75.4 74.2 50.8 2222 deque_append_pop 30.4 43.5 57.0 49.4 49.2 42.5 2223 deque_append_popleft 30.8 43.7 57.3 49.7 49.7 42.8 2224 2225 Timing loop: 2226 loop_overhead 0.3 0.5 0.6 0.4 0.3 0.3 2227 2228The benchmarks were measured on an 2229`Intel® Core™ i7-4960HQ processor 2230<https://ark.intel.com/content/www/us/en/ark/products/76088/intel-core-i7-4960hq-processor-6m-cache-up-to-3-80-ghz.html>`_ 2231running the macOS 64-bit builds found at 2232`python.org <https://www.python.org/downloads/macos/>`_. 2233The benchmark script displays timings in nanoseconds. 2234 2235 2236Notable changes in Python 3.8.1 2237=============================== 2238 2239Due to significant security concerns, the *reuse_address* parameter of 2240:meth:`asyncio.loop.create_datagram_endpoint` is no longer supported. This is 2241because of the behavior of the socket option ``SO_REUSEADDR`` in UDP. For more 2242details, see the documentation for ``loop.create_datagram_endpoint()``. 2243(Contributed by Kyle Stanley, Antoine Pitrou, and Yury Selivanov in 2244:issue:`37228`.) 2245 2246Notable changes in Python 3.8.2 2247=============================== 2248 2249Fixed a regression with the ``ignore`` callback of :func:`shutil.copytree`. 2250The argument types are now str and List[str] again. 2251(Contributed by Manuel Barkhau and Giampaolo Rodola in :gh:`83571`.) 2252 2253Notable changes in Python 3.8.3 2254=============================== 2255 2256The constant values of future flags in the :mod:`__future__` module 2257are updated in order to prevent collision with compiler flags. Previously 2258``PyCF_ALLOW_TOP_LEVEL_AWAIT`` was clashing with ``CO_FUTURE_DIVISION``. 2259(Contributed by Batuhan Taskaya in :gh:`83743`) 2260 2261Notable changes in Python 3.8.8 2262=============================== 2263 2264Earlier Python versions allowed using both ``;`` and ``&`` as 2265query parameter separators in :func:`urllib.parse.parse_qs` and 2266:func:`urllib.parse.parse_qsl`. Due to security concerns, and to conform with 2267newer W3C recommendations, this has been changed to allow only a single 2268separator key, with ``&`` as the default. This change also affects 2269:func:`!cgi.parse` and :func:`!cgi.parse_multipart` as they use the affected 2270functions internally. For more details, please see their respective 2271documentation. 2272(Contributed by Adam Goldschmidt, Senthil Kumaran and Ken Jin in :issue:`42967`.) 2273 2274Notable changes in Python 3.8.9 2275=============================== 2276 2277A security fix alters the :class:`ftplib.FTP` behavior to not trust the 2278IPv4 address sent from the remote server when setting up a passive data 2279channel. We reuse the ftp server IP address instead. For unusual code 2280requiring the old behavior, set a ``trust_server_pasv_ipv4_address`` 2281attribute on your FTP instance to ``True``. (See :gh:`87451`) 2282 2283Notable changes in Python 3.8.10 2284================================ 2285 2286macOS 11.0 (Big Sur) and Apple Silicon Mac support 2287-------------------------------------------------- 2288 2289As of 3.8.10, Python now supports building and running on macOS 11 2290(Big Sur) and on Apple Silicon Macs (based on the ``ARM64`` architecture). 2291A new universal build variant, ``universal2``, is now available to natively 2292support both ``ARM64`` and ``Intel 64`` in one set of executables. 2293Note that support for "weaklinking", building binaries targeted for newer 2294versions of macOS that will also run correctly on older versions by 2295testing at runtime for missing features, is not included in this backport 2296from Python 3.9; to support a range of macOS versions, continue to target 2297for and build on the oldest version in the range. 2298 2299(Originally contributed by Ronald Oussoren and Lawrence D'Anna in :gh:`85272`, 2300with fixes by FX Coudert and Eli Rykoff, and backported to 3.8 by Maxime Bélanger 2301and Ned Deily) 2302 2303Notable changes in Python 3.8.10 2304================================ 2305 2306urllib.parse 2307------------ 2308 2309The presence of newline or tab characters in parts of a URL allows for some 2310forms of attacks. Following the WHATWG specification that updates :rfc:`3986`, 2311ASCII newline ``\n``, ``\r`` and tab ``\t`` characters are stripped from the 2312URL by the parser in :mod:`urllib.parse` preventing such attacks. The removal 2313characters are controlled by a new module level variable 2314``urllib.parse._UNSAFE_URL_BYTES_TO_REMOVE``. (See :issue:`43882`) 2315 2316 2317Notable changes in Python 3.8.12 2318================================ 2319 2320Changes in the Python API 2321------------------------- 2322 2323Starting with Python 3.8.12 the :mod:`ipaddress` module no longer accepts 2324any leading zeros in IPv4 address strings. Leading zeros are ambiguous and 2325interpreted as octal notation by some libraries. For example the legacy 2326function :func:`socket.inet_aton` treats leading zeros as octal notation. 2327glibc implementation of modern :func:`~socket.inet_pton` does not accept 2328any leading zeros. 2329 2330(Originally contributed by Christian Heimes in :issue:`36384`, and backported 2331to 3.8 by Achraf Merzouki.) 2332 2333Notable security feature in 3.8.14 2334================================== 2335 2336Converting between :class:`int` and :class:`str` in bases other than 2 2337(binary), 4, 8 (octal), 16 (hexadecimal), or 32 such as base 10 (decimal) 2338now raises a :exc:`ValueError` if the number of digits in string form is 2339above a limit to avoid potential denial of service attacks due to the 2340algorithmic complexity. This is a mitigation for :cve:`2020-10735`. 2341This limit can be configured or disabled by environment variable, command 2342line flag, or :mod:`sys` APIs. See the :ref:`integer string conversion 2343length limitation <int_max_str_digits>` documentation. The default limit 2344is 4300 digits in string form. 2345 2346Notable changes in 3.8.17 2347========================= 2348 2349tarfile 2350------- 2351 2352* The extraction methods in :mod:`tarfile`, and :func:`shutil.unpack_archive`, 2353 have a new a *filter* argument that allows limiting tar features than may be 2354 surprising or dangerous, such as creating files outside the destination 2355 directory. 2356 See :ref:`tarfile-extraction-filter` for details. 2357 In Python 3.12, use without the *filter* argument will show a 2358 :exc:`DeprecationWarning`. 2359 In Python 3.14, the default will switch to ``'data'``. 2360 (Contributed by Petr Viktorin in :pep:`706`.) 2361
.jpg){kind=link}