1**************************** 2 What's New In Python 3.6 3**************************** 4 5:Editors: Elvis Pranskevichus <elvis@magic.io>, Yury Selivanov <yury@magic.io> 6 7.. Rules for maintenance: 8 9 * Anyone can add text to this document. Do not spend very much time 10 on the wording of your changes, because your text will probably 11 get rewritten to some degree. 12 13 * The maintainer will go through Misc/NEWS periodically and add 14 changes; it's therefore more important to add your changes to 15 Misc/NEWS than to this file. 16 17 * This is not a complete list of every single change; completeness 18 is the purpose of Misc/NEWS. Some changes I consider too small 19 or esoteric to include. If such a change is added to the text, 20 I'll just remove it. (This is another reason you shouldn't spend 21 too much time on writing your addition.) 22 23 * If you want to draw your new text to the attention of the 24 maintainer, add 'XXX' to the beginning of the paragraph or 25 section. 26 27 * It's OK to just add a fragmentary note about a change. For 28 example: "XXX Describe the transmogrify() function added to the 29 socket module." The maintainer will research the change and 30 write the necessary text. 31 32 * You can comment out your additions if you like, but it's not 33 necessary (especially when a final release is some months away). 34 35 * Credit the author of a patch or bugfix. Just the name is 36 sufficient; the e-mail address isn't necessary. 37 38 * It's helpful to add the bug/patch number as a comment: 39 40 XXX Describe the transmogrify() function added to the socket 41 module. 42 (Contributed by P.Y. Developer in :issue:`12345`.) 43 44 This saves the maintainer the effort of going through the Mercurial log 45 when researching a change. 46 47This article explains the new features in Python 3.6, compared to 3.5. 48Python 3.6 was released on December 23, 2016. See the 49`changelog <https://docs.python.org/3.6/whatsnew/changelog.html>`_ for a full 50list of changes. 51 52.. seealso:: 53 54 :pep:`494` - Python 3.6 Release Schedule 55 56 57Summary -- Release highlights 58============================= 59 60New syntax features: 61 62* :ref:`PEP 498 <whatsnew36-pep498>`, formatted string literals. 63 64* :ref:`PEP 515 <whatsnew36-pep515>`, underscores in numeric literals. 65 66* :ref:`PEP 526 <whatsnew36-pep526>`, syntax for variable annotations. 67 68* :ref:`PEP 525 <whatsnew36-pep525>`, asynchronous generators. 69 70* :ref:`PEP 530 <whatsnew36-pep530>`: asynchronous comprehensions. 71 72 73New library modules: 74 75* :mod:`secrets`: :ref:`PEP 506 -- Adding A Secrets Module To The Standard Library <whatsnew36-pep506>`. 76 77 78CPython implementation improvements: 79 80* The :ref:`dict <typesmapping>` type has been reimplemented to use 81 a :ref:`more compact representation <whatsnew36-compactdict>` 82 based on `a proposal by Raymond Hettinger 83 <https://mail.python.org/pipermail/python-dev/2012-December/123028.html>`_ 84 and similar to the `PyPy dict implementation`_. This resulted in dictionaries 85 using 20% to 25% less memory when compared to Python 3.5. 86 87* Customization of class creation has been simplified with the 88 :ref:`new protocol <whatsnew36-pep487>`. 89 90* The class attribute definition order is 91 :ref:`now preserved <whatsnew36-pep520>`. 92 93* The order of elements in ``**kwargs`` now 94 :ref:`corresponds to the order <whatsnew36-pep468>` in which keyword 95 arguments were passed to the function. 96 97* DTrace and SystemTap :ref:`probing support <whatsnew36-tracing>` has 98 been added. 99 100* The new :ref:`PYTHONMALLOC <whatsnew36-pythonmalloc>` environment variable 101 can now be used to debug the interpreter memory allocation and access 102 errors. 103 104 105Significant improvements in the standard library: 106 107* The :mod:`asyncio` module has received new features, significant 108 usability and performance improvements, and a fair amount of bug fixes. 109 Starting with Python 3.6 the ``asyncio`` module is no longer provisional 110 and its API is considered stable. 111 112* A new :ref:`file system path protocol <whatsnew36-pep519>` has been 113 implemented to support :term:`path-like objects <path-like object>`. 114 All standard library functions operating on paths have been updated to 115 work with the new protocol. 116 117* The :mod:`datetime` module has gained support for 118 :ref:`Local Time Disambiguation <whatsnew36-pep495>`. 119 120* The :mod:`typing` module received a number of 121 :ref:`improvements <whatsnew36-typing>`. 122 123* The :mod:`tracemalloc` module has been significantly reworked 124 and is now used to provide better output for :exc:`ResourceWarning` 125 as well as provide better diagnostics for memory allocation errors. 126 See the :ref:`PYTHONMALLOC section <whatsnew36-pythonmalloc>` for more 127 information. 128 129 130Security improvements: 131 132* The new :mod:`secrets` module has been added to simplify the generation of 133 cryptographically strong pseudo-random numbers suitable for 134 managing secrets such as account authentication, tokens, and similar. 135 136* On Linux, :func:`os.urandom` now blocks until the system urandom entropy 137 pool is initialized to increase the security. See the :pep:`524` for the 138 rationale. 139 140* The :mod:`hashlib` and :mod:`ssl` modules now support OpenSSL 1.1.0. 141 142* The default settings and feature set of the :mod:`ssl` module have been 143 improved. 144 145* The :mod:`hashlib` module received support for the BLAKE2, SHA-3 and SHAKE 146 hash algorithms and the :func:`~hashlib.scrypt` key derivation function. 147 148 149Windows improvements: 150 151* :ref:`PEP 528 <whatsnew36-pep528>` and :ref:`PEP 529 <whatsnew36-pep529>`, 152 Windows filesystem and console encoding changed to UTF-8. 153 154* The ``py.exe`` launcher, when used interactively, no longer prefers 155 Python 2 over Python 3 when the user doesn't specify a version (via 156 command line arguments or a config file). Handling of shebang lines 157 remains unchanged - "python" refers to Python 2 in that case. 158 159* ``python.exe`` and ``pythonw.exe`` have been marked as long-path aware, 160 which means that the 260 character path limit may no longer apply. 161 See :ref:`removing the MAX_PATH limitation <max-path>` for details. 162 163* A ``._pth`` file can be added to force isolated mode and fully specify 164 all search paths to avoid registry and environment lookup. See 165 :ref:`the documentation <windows_finding_modules>` for more information. 166 167* A ``python36.zip`` file now works as a landmark to infer 168 :envvar:`PYTHONHOME`. See :ref:`the documentation <windows_finding_modules>` for 169 more information. 170 171 172.. _PyPy dict implementation: https://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html 173 174 175New Features 176============ 177 178.. _whatsnew36-pep498: 179 180PEP 498: Formatted string literals 181---------------------------------- 182 183:pep:`498` introduces a new kind of string literals: *f-strings*, or 184:ref:`formatted string literals <f-strings>`. 185 186Formatted string literals are prefixed with ``'f'`` and are similar to 187the format strings accepted by :meth:`str.format`. They contain replacement 188fields surrounded by curly braces. The replacement fields are expressions, 189which are evaluated at run time, and then formatted using the 190:func:`format` protocol:: 191 192 >>> name = "Fred" 193 >>> f"He said his name is {name}." 194 'He said his name is Fred.' 195 >>> width = 10 196 >>> precision = 4 197 >>> value = decimal.Decimal("12.34567") 198 >>> f"result: {value:{width}.{precision}}" # nested fields 199 'result: 12.35' 200 201.. seealso:: 202 203 :pep:`498` -- Literal String Interpolation. 204 PEP written and implemented by Eric V. Smith. 205 206 :ref:`Feature documentation <f-strings>`. 207 208 209.. _whatsnew36-pep526: 210 211PEP 526: Syntax for variable annotations 212---------------------------------------- 213 214:pep:`484` introduced the standard for type annotations of function 215parameters, a.k.a. type hints. This PEP adds syntax to Python for annotating 216the types of variables including class variables and instance variables:: 217 218 primes: List[int] = [] 219 220 captain: str # Note: no initial value! 221 222 class Starship: 223 stats: Dict[str, int] = {} 224 225Just as for function annotations, the Python interpreter does not attach any 226particular meaning to variable annotations and only stores them in the 227``__annotations__`` attribute of a class or module. 228 229In contrast to variable declarations in statically typed languages, 230the goal of annotation syntax is to provide an easy way to specify structured 231type metadata for third party tools and libraries via the abstract syntax tree 232and the ``__annotations__`` attribute. 233 234.. seealso:: 235 236 :pep:`526` -- Syntax for variable annotations. 237 PEP written by Ryan Gonzalez, Philip House, Ivan Levkivskyi, Lisa Roach, 238 and Guido van Rossum. Implemented by Ivan Levkivskyi. 239 240 Tools that use or will use the new syntax: 241 `mypy <https://www.mypy-lang.org/>`_, 242 `pytype <https://github.com/google/pytype>`_, PyCharm, etc. 243 244 245.. _whatsnew36-pep515: 246 247PEP 515: Underscores in Numeric Literals 248---------------------------------------- 249 250:pep:`515` adds the ability to use underscores in numeric literals for 251improved readability. For example:: 252 253 >>> 1_000_000_000_000_000 254 1000000000000000 255 >>> 0x_FF_FF_FF_FF 256 4294967295 257 258Single underscores are allowed between digits and after any base 259specifier. Leading, trailing, or multiple underscores in a row are not 260allowed. 261 262The :ref:`string formatting <formatspec>` language also now has support 263for the ``'_'`` option to signal the use of an underscore for a thousands 264separator for floating-point presentation types and for integer 265presentation type ``'d'``. For integer presentation types ``'b'``, 266``'o'``, ``'x'``, and ``'X'``, underscores will be inserted every 4 267digits:: 268 269 >>> '{:_}'.format(1000000) 270 '1_000_000' 271 >>> '{:_x}'.format(0xFFFFFFFF) 272 'ffff_ffff' 273 274.. seealso:: 275 276 :pep:`515` -- Underscores in Numeric Literals 277 PEP written by Georg Brandl and Serhiy Storchaka. 278 279 280.. _whatsnew36-pep525: 281 282PEP 525: Asynchronous Generators 283-------------------------------- 284 285:pep:`492` introduced support for native coroutines and ``async`` / ``await`` 286syntax to Python 3.5. A notable limitation of the Python 3.5 implementation 287is that it was not possible to use ``await`` and ``yield`` in the same 288function body. In Python 3.6 this restriction has been lifted, making it 289possible to define *asynchronous generators*:: 290 291 async def ticker(delay, to): 292 """Yield numbers from 0 to *to* every *delay* seconds.""" 293 for i in range(to): 294 yield i 295 await asyncio.sleep(delay) 296 297The new syntax allows for faster and more concise code. 298 299.. seealso:: 300 301 :pep:`525` -- Asynchronous Generators 302 PEP written and implemented by Yury Selivanov. 303 304 305.. _whatsnew36-pep530: 306 307PEP 530: Asynchronous Comprehensions 308------------------------------------ 309 310:pep:`530` adds support for using ``async for`` in list, set, dict 311comprehensions and generator expressions:: 312 313 result = [i async for i in aiter() if i % 2] 314 315Additionally, ``await`` expressions are supported in all kinds 316of comprehensions:: 317 318 result = [await fun() for fun in funcs if await condition()] 319 320.. seealso:: 321 322 :pep:`530` -- Asynchronous Comprehensions 323 PEP written and implemented by Yury Selivanov. 324 325 326.. _whatsnew36-pep487: 327 328PEP 487: Simpler customization of class creation 329------------------------------------------------ 330 331It is now possible to customize subclass creation without using a metaclass. 332The new ``__init_subclass__`` classmethod will be called on the base class 333whenever a new subclass is created:: 334 335 class PluginBase: 336 subclasses = [] 337 338 def __init_subclass__(cls, **kwargs): 339 super().__init_subclass__(**kwargs) 340 cls.subclasses.append(cls) 341 342 class Plugin1(PluginBase): 343 pass 344 345 class Plugin2(PluginBase): 346 pass 347 348In order to allow zero-argument :func:`super` calls to work correctly from 349:meth:`~object.__init_subclass__` implementations, custom metaclasses must 350ensure that the new ``__classcell__`` namespace entry is propagated to 351``type.__new__`` (as described in :ref:`class-object-creation`). 352 353.. seealso:: 354 355 :pep:`487` -- Simpler customization of class creation 356 PEP written and implemented by Martin Teichmann. 357 358 :ref:`Feature documentation <class-customization>` 359 360 361.. _whatsnew36-pep487-descriptors: 362 363PEP 487: Descriptor Protocol Enhancements 364----------------------------------------- 365 366:pep:`487` extends the descriptor protocol to include the new optional 367:meth:`~object.__set_name__` method. Whenever a new class is defined, the new 368method will be called on all descriptors included in the definition, providing 369them with a reference to the class being defined and the name given to the 370descriptor within the class namespace. In other words, instances of 371descriptors can now know the attribute name of the descriptor in the 372owner class:: 373 374 class IntField: 375 def __get__(self, instance, owner): 376 return instance.__dict__[self.name] 377 378 def __set__(self, instance, value): 379 if not isinstance(value, int): 380 raise ValueError(f'expecting integer in {self.name}') 381 instance.__dict__[self.name] = value 382 383 # this is the new initializer: 384 def __set_name__(self, owner, name): 385 self.name = name 386 387 class Model: 388 int_field = IntField() 389 390 391.. seealso:: 392 393 :pep:`487` -- Simpler customization of class creation 394 PEP written and implemented by Martin Teichmann. 395 396 :ref:`Feature documentation <descriptors>` 397 398 399.. _whatsnew36-pep519: 400 401PEP 519: Adding a file system path protocol 402------------------------------------------- 403 404File system paths have historically been represented as :class:`str` 405or :class:`bytes` objects. This has led to people who write code which 406operate on file system paths to assume that such objects are only one 407of those two types (an :class:`int` representing a file descriptor 408does not count as that is not a file path). Unfortunately that 409assumption prevents alternative object representations of file system 410paths like :mod:`pathlib` from working with pre-existing code, 411including Python's standard library. 412 413To fix this situation, a new interface represented by 414:class:`os.PathLike` has been defined. By implementing the 415:meth:`~os.PathLike.__fspath__` method, an object signals that it 416represents a path. An object can then provide a low-level 417representation of a file system path as a :class:`str` or 418:class:`bytes` object. This means an object is considered 419:term:`path-like <path-like object>` if it implements 420:class:`os.PathLike` or is a :class:`str` or :class:`bytes` object 421which represents a file system path. Code can use :func:`os.fspath`, 422:func:`os.fsdecode`, or :func:`os.fsencode` to explicitly get a 423:class:`str` and/or :class:`bytes` representation of a path-like 424object. 425 426The built-in :func:`open` function has been updated to accept 427:class:`os.PathLike` objects, as have all relevant functions in the 428:mod:`os` and :mod:`os.path` modules, and most other functions and 429classes in the standard library. The :class:`os.DirEntry` class 430and relevant classes in :mod:`pathlib` have also been updated to 431implement :class:`os.PathLike`. 432 433The hope is that updating the fundamental functions for operating 434on file system paths will lead to third-party code to implicitly 435support all :term:`path-like objects <path-like object>` without any 436code changes, or at least very minimal ones (e.g. calling 437:func:`os.fspath` at the beginning of code before operating on a 438path-like object). 439 440Here are some examples of how the new interface allows for 441:class:`pathlib.Path` to be used more easily and transparently with 442pre-existing code:: 443 444 >>> import pathlib 445 >>> with open(pathlib.Path("README")) as f: 446 ... contents = f.read() 447 ... 448 >>> import os.path 449 >>> os.path.splitext(pathlib.Path("some_file.txt")) 450 ('some_file', '.txt') 451 >>> os.path.join("/a/b", pathlib.Path("c")) 452 '/a/b/c' 453 >>> import os 454 >>> os.fspath(pathlib.Path("some_file.txt")) 455 'some_file.txt' 456 457(Implemented by Brett Cannon, Ethan Furman, Dusty Phillips, and Jelle Zijlstra.) 458 459.. seealso:: 460 461 :pep:`519` -- Adding a file system path protocol 462 PEP written by Brett Cannon and Koos Zevenhoven. 463 464 465.. _whatsnew36-pep495: 466 467PEP 495: Local Time Disambiguation 468---------------------------------- 469 470In most world locations, there have been and will be times when local clocks 471are moved back. In those times, intervals are introduced in which local 472clocks show the same time twice in the same day. In these situations, the 473information displayed on a local clock (or stored in a Python datetime 474instance) is insufficient to identify a particular moment in time. 475 476:pep:`495` adds the new *fold* attribute to instances of 477:class:`datetime.datetime` and :class:`datetime.time` classes to differentiate 478between two moments in time for which local times are the same:: 479 480 >>> u0 = datetime(2016, 11, 6, 4, tzinfo=timezone.utc) 481 >>> for i in range(4): 482 ... u = u0 + i*HOUR 483 ... t = u.astimezone(Eastern) 484 ... print(u.time(), 'UTC =', t.time(), t.tzname(), t.fold) 485 ... 486 04:00:00 UTC = 00:00:00 EDT 0 487 05:00:00 UTC = 01:00:00 EDT 0 488 06:00:00 UTC = 01:00:00 EST 1 489 07:00:00 UTC = 02:00:00 EST 0 490 491The values of the :attr:`fold <datetime.datetime.fold>` attribute have the 492value ``0`` for all instances except those that represent the second 493(chronologically) moment in time in an ambiguous case. 494 495.. seealso:: 496 497 :pep:`495` -- Local Time Disambiguation 498 PEP written by Alexander Belopolsky and Tim Peters, implementation 499 by Alexander Belopolsky. 500 501 502.. _whatsnew36-pep529: 503 504PEP 529: Change Windows filesystem encoding to UTF-8 505---------------------------------------------------- 506 507Representing filesystem paths is best performed with str (Unicode) rather than 508bytes. However, there are some situations where using bytes is sufficient and 509correct. 510 511Prior to Python 3.6, data loss could result when using bytes paths on Windows. 512With this change, using bytes to represent paths is now supported on Windows, 513provided those bytes are encoded with the encoding returned by 514:func:`sys.getfilesystemencoding`, which now defaults to ``'utf-8'``. 515 516Applications that do not use str to represent paths should use 517:func:`os.fsencode` and :func:`os.fsdecode` to ensure their bytes are 518correctly encoded. To revert to the previous behaviour, set 519:envvar:`PYTHONLEGACYWINDOWSFSENCODING` or call 520:func:`sys._enablelegacywindowsfsencoding`. 521 522See :pep:`529` for more information and discussion of code modifications that 523may be required. 524 525 526.. _whatsnew36-pep528: 527 528PEP 528: Change Windows console encoding to UTF-8 529------------------------------------------------- 530 531The default console on Windows will now accept all Unicode characters and 532provide correctly read str objects to Python code. ``sys.stdin``, 533``sys.stdout`` and ``sys.stderr`` now default to utf-8 encoding. 534 535This change only applies when using an interactive console, and not when 536redirecting files or pipes. To revert to the previous behaviour for interactive 537console use, set :envvar:`PYTHONLEGACYWINDOWSSTDIO`. 538 539.. seealso:: 540 541 :pep:`528` -- Change Windows console encoding to UTF-8 542 PEP written and implemented by Steve Dower. 543 544 545.. _whatsnew36-pep520: 546 547PEP 520: Preserving Class Attribute Definition Order 548---------------------------------------------------- 549 550Attributes in a class definition body have a natural ordering: the same 551order in which the names appear in the source. This order is now 552preserved in the new class's :attr:`~type.__dict__` attribute. 553 554Also, the effective default class *execution* namespace (returned from 555:ref:`type.__prepare__() <prepare>`) is now an insertion-order-preserving 556mapping. 557 558.. seealso:: 559 560 :pep:`520` -- Preserving Class Attribute Definition Order 561 PEP written and implemented by Eric Snow. 562 563 564.. _whatsnew36-pep468: 565 566PEP 468: Preserving Keyword Argument Order 567------------------------------------------ 568 569``**kwargs`` in a function signature is now guaranteed to be an 570insertion-order-preserving mapping. 571 572.. seealso:: 573 574 :pep:`468` -- Preserving Keyword Argument Order 575 PEP written and implemented by Eric Snow. 576 577 578.. _whatsnew36-compactdict: 579 580New :ref:`dict <typesmapping>` implementation 581--------------------------------------------- 582 583The :ref:`dict <typesmapping>` type now uses a "compact" representation 584based on `a proposal by Raymond Hettinger 585<https://mail.python.org/pipermail/python-dev/2012-December/123028.html>`_ 586which was `first implemented by PyPy 587<https://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html>`_. 588The memory usage of the new :func:`dict` is between 20% and 25% smaller 589compared to Python 3.5. 590 591The order-preserving aspect of this new implementation is considered an 592implementation detail and should not be relied upon (this may change in 593the future, but it is desired to have this new dict implementation in 594the language for a few releases before changing the language spec to mandate 595order-preserving semantics for all current and future Python 596implementations; this also helps preserve backwards-compatibility 597with older versions of the language where random iteration order is 598still in effect, e.g. Python 3.5). 599 600(Contributed by INADA Naoki in :issue:`27350`. Idea 601`originally suggested by Raymond Hettinger 602<https://mail.python.org/pipermail/python-dev/2012-December/123028.html>`_.) 603 604 605.. _whatsnew36-pep523: 606 607PEP 523: Adding a frame evaluation API to CPython 608------------------------------------------------- 609 610While Python provides extensive support to customize how code 611executes, one place it has not done so is in the evaluation of frame 612objects. If you wanted some way to intercept frame evaluation in 613Python there really wasn't any way without directly manipulating 614function pointers for defined functions. 615 616:pep:`523` changes this by providing an API to make frame 617evaluation pluggable at the C level. This will allow for tools such 618as debuggers and JITs to intercept frame evaluation before the 619execution of Python code begins. This enables the use of alternative 620evaluation implementations for Python code, tracking frame 621evaluation, etc. 622 623This API is not part of the limited C API and is marked as private to 624signal that usage of this API is expected to be limited and only 625applicable to very select, low-level use-cases. Semantics of the 626API will change with Python as necessary. 627 628.. seealso:: 629 630 :pep:`523` -- Adding a frame evaluation API to CPython 631 PEP written by Brett Cannon and Dino Viehland. 632 633 634.. _whatsnew36-pythonmalloc: 635 636PYTHONMALLOC environment variable 637--------------------------------- 638 639The new :envvar:`PYTHONMALLOC` environment variable allows setting the Python 640memory allocators and installing debug hooks. 641 642It is now possible to install debug hooks on Python memory allocators on Python 643compiled in release mode using ``PYTHONMALLOC=debug``. Effects of debug hooks: 644 645* Newly allocated memory is filled with the byte ``0xCB`` 646* Freed memory is filled with the byte ``0xDB`` 647* Detect violations of the Python memory allocator API. For example, 648 :c:func:`PyObject_Free` called on a memory block allocated by 649 :c:func:`PyMem_Malloc`. 650* Detect writes before the start of a buffer (buffer underflows) 651* Detect writes after the end of a buffer (buffer overflows) 652* Check that the :term:`GIL <global interpreter lock>` is held when allocator 653 functions of :c:macro:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and 654 :c:macro:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called. 655 656Checking if the GIL is held is also a new feature of Python 3.6. 657 658See the :c:func:`PyMem_SetupDebugHooks` function for debug hooks on Python 659memory allocators. 660 661It is now also possible to force the usage of the :c:func:`malloc` allocator of 662the C library for all Python memory allocations using ``PYTHONMALLOC=malloc``. 663This is helpful when using external memory debuggers like Valgrind on 664a Python compiled in release mode. 665 666On error, the debug hooks on Python memory allocators now use the 667:mod:`tracemalloc` module to get the traceback where a memory block was 668allocated. 669 670Example of fatal error on buffer overflow using 671``python3.6 -X tracemalloc=5`` (store 5 frames in traces):: 672 673 Debug memory block at address p=0x7fbcd41666f8: API 'o' 674 4 bytes originally requested 675 The 7 pad bytes at p-7 are FORBIDDENBYTE, as expected. 676 The 8 pad bytes at tail=0x7fbcd41666fc are not all FORBIDDENBYTE (0xfb): 677 at tail+0: 0x02 *** OUCH 678 at tail+1: 0xfb 679 at tail+2: 0xfb 680 at tail+3: 0xfb 681 at tail+4: 0xfb 682 at tail+5: 0xfb 683 at tail+6: 0xfb 684 at tail+7: 0xfb 685 The block was made by call #1233329 to debug malloc/realloc. 686 Data at p: 1a 2b 30 00 687 688 Memory block allocated at (most recent call first): 689 File "test/test_bytes.py", line 323 690 File "unittest/case.py", line 600 691 File "unittest/case.py", line 648 692 File "unittest/suite.py", line 122 693 File "unittest/suite.py", line 84 694 695 Fatal Python error: bad trailing pad byte 696 697 Current thread 0x00007fbcdbd32700 (most recent call first): 698 File "test/test_bytes.py", line 323 in test_hex 699 File "unittest/case.py", line 600 in run 700 File "unittest/case.py", line 648 in __call__ 701 File "unittest/suite.py", line 122 in run 702 File "unittest/suite.py", line 84 in __call__ 703 File "unittest/suite.py", line 122 in run 704 File "unittest/suite.py", line 84 in __call__ 705 ... 706 707(Contributed by Victor Stinner in :issue:`26516` and :issue:`26564`.) 708 709 710.. _whatsnew36-tracing: 711 712DTrace and SystemTap probing support 713------------------------------------ 714 715Python can now be built ``--with-dtrace`` which enables static markers 716for the following events in the interpreter: 717 718* function call/return 719 720* garbage collection started/finished 721 722* line of code executed. 723 724This can be used to instrument running interpreters in production, 725without the need to recompile specific :ref:`debug builds <debug-build>` or 726providing application-specific profiling/debugging code. 727 728More details in :ref:`instrumentation`. 729 730The current implementation is tested on Linux and macOS. Additional 731markers may be added in the future. 732 733(Contributed by Łukasz Langa in :issue:`21590`, based on patches by 734Jesús Cea Avión, David Malcolm, and Nikhil Benesch.) 735 736 737Other Language Changes 738====================== 739 740Some smaller changes made to the core Python language are: 741 742* A ``global`` or ``nonlocal`` statement must now textually appear 743 before the first use of the affected name in the same scope. 744 Previously this was a :exc:`SyntaxWarning`. 745 746* It is now possible to set a :ref:`special method <specialnames>` to 747 ``None`` to indicate that the corresponding operation is not available. 748 For example, if a class sets :meth:`__iter__` to ``None``, the class 749 is not iterable. 750 (Contributed by Andrew Barnert and Ivan Levkivskyi in :issue:`25958`.) 751 752* Long sequences of repeated traceback lines are now abbreviated as 753 ``"[Previous line repeated {count} more times]"`` (see 754 :ref:`whatsnew36-traceback` for an example). 755 (Contributed by Emanuel Barry in :issue:`26823`.) 756 757* Import now raises the new exception :exc:`ModuleNotFoundError` 758 (subclass of :exc:`ImportError`) when it cannot find a module. Code 759 that currently checks for ImportError (in try-except) will still work. 760 (Contributed by Eric Snow in :issue:`15767`.) 761 762* Class methods relying on zero-argument ``super()`` will now work correctly 763 when called from metaclass methods during class creation. 764 (Contributed by Martin Teichmann in :issue:`23722`.) 765 766 767New Modules 768=========== 769 770.. _whatsnew36-pep506: 771 772secrets 773------- 774 775The main purpose of the new :mod:`secrets` module is to provide an obvious way 776to reliably generate cryptographically strong pseudo-random values suitable 777for managing secrets, such as account authentication, tokens, and similar. 778 779.. warning:: 780 781 Note that the pseudo-random generators in the :mod:`random` module 782 should *NOT* be used for security purposes. Use :mod:`secrets` 783 on Python 3.6+ and :func:`os.urandom` on Python 3.5 and earlier. 784 785.. seealso:: 786 787 :pep:`506` -- Adding A Secrets Module To The Standard Library 788 PEP written and implemented by Steven D'Aprano. 789 790 791Improved Modules 792================ 793 794array 795----- 796 797Exhausted iterators of :class:`array.array` will now stay exhausted even 798if the iterated array is extended. This is consistent with the behavior 799of other mutable sequences. 800 801Contributed by Serhiy Storchaka in :issue:`26492`. 802 803ast 804--- 805 806The new :class:`ast.Constant` AST node has been added. It can be used 807by external AST optimizers for the purposes of constant folding. 808 809Contributed by Victor Stinner in :issue:`26146`. 810 811 812asyncio 813------- 814 815Starting with Python 3.6 the ``asyncio`` module is no longer provisional and its 816API is considered stable. 817 818Notable changes in the :mod:`asyncio` module since Python 3.5.0 819(all backported to 3.5.x due to the provisional status): 820 821* The :func:`~asyncio.get_event_loop` function has been changed to 822 always return the currently running loop when called from coroutines 823 and callbacks. 824 (Contributed by Yury Selivanov in :issue:`28613`.) 825 826* The :func:`~asyncio.ensure_future` function and all functions that 827 use it, such as :meth:`loop.run_until_complete() <asyncio.loop.run_until_complete>`, 828 now accept all kinds of :term:`awaitable objects <awaitable>`. 829 (Contributed by Yury Selivanov.) 830 831* New :func:`~asyncio.run_coroutine_threadsafe` function to submit 832 coroutines to event loops from other threads. 833 (Contributed by Vincent Michel.) 834 835* New :meth:`Transport.is_closing() <asyncio.BaseTransport.is_closing>` 836 method to check if the transport is closing or closed. 837 (Contributed by Yury Selivanov.) 838 839* The :meth:`loop.create_server() <asyncio.loop.create_server>` 840 method can now accept a list of hosts. 841 (Contributed by Yann Sionneau.) 842 843* New :meth:`loop.create_future() <asyncio.loop.create_future>` 844 method to create Future objects. This allows alternative event 845 loop implementations, such as 846 `uvloop <https://github.com/MagicStack/uvloop>`_, to provide a faster 847 :class:`asyncio.Future` implementation. 848 (Contributed by Yury Selivanov in :issue:`27041`.) 849 850* New :meth:`loop.get_exception_handler() <asyncio.loop.get_exception_handler>` 851 method to get the current exception handler. 852 (Contributed by Yury Selivanov in :issue:`27040`.) 853 854* New :meth:`StreamReader.readuntil() <asyncio.StreamReader.readuntil>` 855 method to read data from the stream until a separator bytes 856 sequence appears. 857 (Contributed by Mark Korenberg.) 858 859* The performance of :meth:`StreamReader.readexactly() <asyncio.StreamReader.readexactly>` 860 has been improved. 861 (Contributed by Mark Korenberg in :issue:`28370`.) 862 863* The :meth:`loop.getaddrinfo() <asyncio.loop.getaddrinfo>` 864 method is optimized to avoid calling the system ``getaddrinfo`` 865 function if the address is already resolved. 866 (Contributed by A. Jesse Jiryu Davis.) 867 868* The :meth:`loop.stop() <asyncio.loop.stop>` 869 method has been changed to stop the loop immediately after 870 the current iteration. Any new callbacks scheduled as a result 871 of the last iteration will be discarded. 872 (Contributed by Guido van Rossum in :issue:`25593`.) 873 874* :meth:`Future.set_exception <asyncio.futures.Future.set_exception>` 875 will now raise :exc:`TypeError` when passed an instance of 876 the :exc:`StopIteration` exception. 877 (Contributed by Chris Angelico in :issue:`26221`.) 878 879* New :meth:`loop.connect_accepted_socket() <asyncio.loop.connect_accepted_socket>` 880 method to be used by servers that accept connections outside of asyncio, 881 but that use asyncio to handle them. 882 (Contributed by Jim Fulton in :issue:`27392`.) 883 884* ``TCP_NODELAY`` flag is now set for all TCP transports by default. 885 (Contributed by Yury Selivanov in :issue:`27456`.) 886 887* New :meth:`loop.shutdown_asyncgens() <asyncio.loop.shutdown_asyncgens>` 888 to properly close pending asynchronous generators before closing the 889 loop. 890 (Contributed by Yury Selivanov in :issue:`28003`.) 891 892* :class:`Future <asyncio.Future>` and :class:`Task <asyncio.Task>` 893 classes now have an optimized C implementation which makes asyncio 894 code up to 30% faster. 895 (Contributed by Yury Selivanov and INADA Naoki in :issue:`26081` 896 and :issue:`28544`.) 897 898 899binascii 900-------- 901 902The :func:`~binascii.b2a_base64` function now accepts an optional *newline* 903keyword argument to control whether the newline character is appended to the 904return value. 905(Contributed by Victor Stinner in :issue:`25357`.) 906 907 908cmath 909----- 910 911The new :const:`cmath.tau` (*τ*) constant has been added. 912(Contributed by Lisa Roach in :issue:`12345`, see :pep:`628` for details.) 913 914New constants: :const:`cmath.inf` and :const:`cmath.nan` to 915match :const:`math.inf` and :const:`math.nan`, and also :const:`cmath.infj` 916and :const:`cmath.nanj` to match the format used by complex repr. 917(Contributed by Mark Dickinson in :issue:`23229`.) 918 919 920collections 921----------- 922 923The new :class:`~collections.abc.Collection` abstract base class has been 924added to represent sized iterable container classes. 925(Contributed by Ivan Levkivskyi, docs by Neil Girdhar in :issue:`27598`.) 926 927The new :class:`~collections.abc.Reversible` abstract base class represents 928iterable classes that also provide the :meth:`__reversed__` method. 929(Contributed by Ivan Levkivskyi in :issue:`25987`.) 930 931The new :class:`~collections.abc.AsyncGenerator` abstract base class represents 932asynchronous generators. 933(Contributed by Yury Selivanov in :issue:`28720`.) 934 935The :func:`~collections.namedtuple` function now accepts an optional 936keyword argument *module*, which, when specified, is used for 937the :attr:`~type.__module__` attribute of the returned named tuple class. 938(Contributed by Raymond Hettinger in :issue:`17941`.) 939 940The *verbose* and *rename* arguments for 941:func:`~collections.namedtuple` are now keyword-only. 942(Contributed by Raymond Hettinger in :issue:`25628`.) 943 944Recursive :class:`collections.deque` instances can now be pickled. 945(Contributed by Serhiy Storchaka in :issue:`26482`.) 946 947 948concurrent.futures 949------------------ 950 951The :class:`ThreadPoolExecutor <concurrent.futures.ThreadPoolExecutor>` 952class constructor now accepts an optional *thread_name_prefix* argument 953to make it possible to customize the names of the threads created by the 954pool. 955(Contributed by Gregory P. Smith in :issue:`27664`.) 956 957 958contextlib 959---------- 960 961The :class:`contextlib.AbstractContextManager` class has been added to 962provide an abstract base class for context managers. It provides a 963sensible default implementation for ``__enter__()`` which returns 964``self`` and leaves ``__exit__()`` an abstract method. A matching 965class has been added to the :mod:`typing` module as 966:class:`typing.ContextManager`. 967(Contributed by Brett Cannon in :issue:`25609`.) 968 969 970datetime 971-------- 972 973The :class:`~datetime.datetime` and :class:`~datetime.time` classes have 974the new :attr:`~time.fold` attribute used to disambiguate local time 975when necessary. Many functions in the :mod:`datetime` have been 976updated to support local time disambiguation. 977See :ref:`Local Time Disambiguation <whatsnew36-pep495>` section for more 978information. 979(Contributed by Alexander Belopolsky in :issue:`24773`.) 980 981The :meth:`datetime.strftime() <datetime.datetime.strftime>` and 982:meth:`date.strftime() <datetime.date.strftime>` methods now support 983ISO 8601 date directives ``%G``, ``%u`` and ``%V``. 984(Contributed by Ashley Anderson in :issue:`12006`.) 985 986The :func:`datetime.isoformat() <datetime.datetime.isoformat>` function 987now accepts an optional *timespec* argument that specifies the number 988of additional components of the time value to include. 989(Contributed by Alessandro Cucci and Alexander Belopolsky in :issue:`19475`.) 990 991The :meth:`datetime.combine() <datetime.datetime.combine>` now 992accepts an optional *tzinfo* argument. 993(Contributed by Alexander Belopolsky in :issue:`27661`.) 994 995 996decimal 997------- 998 999New :meth:`Decimal.as_integer_ratio() <decimal.Decimal.as_integer_ratio>` 1000method that returns a pair ``(n, d)`` of integers that represent the given 1001:class:`~decimal.Decimal` instance as a fraction, in lowest terms and 1002with a positive denominator:: 1003 1004 >>> Decimal('-3.14').as_integer_ratio() 1005 (-157, 50) 1006 1007(Contributed by Stefan Krah amd Mark Dickinson in :issue:`25928`.) 1008 1009 1010 1011distutils 1012--------- 1013 1014The ``default_format`` attribute has been removed from 1015``distutils.command.sdist.sdist`` and the ``formats`` 1016attribute defaults to ``['gztar']``. Although not anticipated, 1017any code relying on the presence of ``default_format`` may 1018need to be adapted. See :issue:`27819` for more details. 1019 1020 1021email 1022----- 1023 1024The new email API, enabled via the *policy* keyword to various constructors, is 1025no longer provisional. The :mod:`email` documentation has been reorganized and 1026rewritten to focus on the new API, while retaining the old documentation for 1027the legacy API. (Contributed by R. David Murray in :issue:`24277`.) 1028 1029The :mod:`email.mime` classes now all accept an optional *policy* keyword. 1030(Contributed by Berker Peksag in :issue:`27331`.) 1031 1032The :class:`~email.generator.DecodedGenerator` now supports the *policy* 1033keyword. 1034 1035There is a new :mod:`~email.policy` attribute, 1036:attr:`~email.policy.Policy.message_factory`, that controls what class is used 1037by default when the parser creates new message objects. For the 1038:attr:`email.policy.compat32` policy this is :class:`~email.message.Message`, 1039for the new policies it is :class:`~email.message.EmailMessage`. 1040(Contributed by R. David Murray in :issue:`20476`.) 1041 1042 1043encodings 1044--------- 1045 1046On Windows, added the ``'oem'`` encoding to use ``CP_OEMCP``, and the ``'ansi'`` 1047alias for the existing ``'mbcs'`` encoding, which uses the ``CP_ACP`` code page. 1048(Contributed by Steve Dower in :issue:`27959`.) 1049 1050 1051enum 1052---- 1053 1054Two new enumeration base classes have been added to the :mod:`enum` module: 1055:class:`~enum.Flag` and :class:`~enum.IntFlags`. Both are used to define 1056constants that can be combined using the bitwise operators. 1057(Contributed by Ethan Furman in :issue:`23591`.) 1058 1059Many standard library modules have been updated to use the 1060:class:`~enum.IntFlags` class for their constants. 1061 1062The new :class:`enum.auto` value can be used to assign values to enum 1063members automatically:: 1064 1065 >>> from enum import Enum, auto 1066 >>> class Color(Enum): 1067 ... red = auto() 1068 ... blue = auto() 1069 ... green = auto() 1070 ... 1071 >>> list(Color) 1072 [<Color.red: 1>, <Color.blue: 2>, <Color.green: 3>] 1073 1074 1075faulthandler 1076------------ 1077 1078On Windows, the :mod:`faulthandler` module now installs a handler for Windows 1079exceptions: see :func:`faulthandler.enable`. (Contributed by Victor Stinner in 1080:issue:`23848`.) 1081 1082 1083fileinput 1084--------- 1085 1086:func:`~fileinput.hook_encoded` now supports the *errors* argument. 1087(Contributed by Joseph Hackman in :issue:`25788`.) 1088 1089 1090hashlib 1091------- 1092 1093:mod:`hashlib` supports OpenSSL 1.1.0. The minimum recommend version is 1.0.2. 1094(Contributed by Christian Heimes in :issue:`26470`.) 1095 1096BLAKE2 hash functions were added to the module. :func:`~hashlib.blake2b` 1097and :func:`~hashlib.blake2s` are always available and support the full 1098feature set of BLAKE2. 1099(Contributed by Christian Heimes in :issue:`26798` based on code by 1100Dmitry Chestnykh and Samuel Neves. Documentation written by Dmitry Chestnykh.) 1101 1102The SHA-3 hash functions :func:`~hashlib.sha3_224`, :func:`~hashlib.sha3_256`, 1103:func:`~hashlib.sha3_384`, :func:`~hashlib.sha3_512`, and SHAKE hash functions 1104:func:`~hashlib.shake_128` and :func:`~hashlib.shake_256` were added. 1105(Contributed by Christian Heimes in :issue:`16113`. Keccak Code Package 1106by Guido Bertoni, Joan Daemen, Michaël Peeters, Gilles Van Assche, and 1107Ronny Van Keer.) 1108 1109The password-based key derivation function :func:`~hashlib.scrypt` is now 1110available with OpenSSL 1.1.0 and newer. 1111(Contributed by Christian Heimes in :issue:`27928`.) 1112 1113http.client 1114----------- 1115 1116:meth:`HTTPConnection.request() <http.client.HTTPConnection.request>` and 1117:meth:`~http.client.HTTPConnection.endheaders` both now support 1118chunked encoding request bodies. 1119(Contributed by Demian Brecht and Rolf Krahl in :issue:`12319`.) 1120 1121 1122idlelib and IDLE 1123---------------- 1124 1125The idlelib package is being modernized and refactored to make IDLE look and 1126work better and to make the code easier to understand, test, and improve. Part 1127of making IDLE look better, especially on Linux and Mac, is using ttk widgets, 1128mostly in the dialogs. As a result, IDLE no longer runs with tcl/tk 8.4. It 1129now requires tcl/tk 8.5 or 8.6. We recommend running the latest release of 1130either. 1131 1132'Modernizing' includes renaming and consolidation of idlelib modules. The 1133renaming of files with partial uppercase names is similar to the renaming of, 1134for instance, Tkinter and TkFont to tkinter and tkinter.font in 3.0. As a 1135result, imports of idlelib files that worked in 3.5 will usually not work in 11363.6. At least a module name change will be needed (see idlelib/README.txt), 1137sometimes more. (Name changes contributed by Al Swiegart and Terry Reedy in 1138:issue:`24225`. Most idlelib patches since have been and will be part of the 1139process.) 1140 1141In compensation, the eventual result with be that some idlelib classes will be 1142easier to use, with better APIs and docstrings explaining them. Additional 1143useful information will be added to idlelib when available. 1144 1145New in 3.6.2: 1146 1147Multiple fixes for autocompletion. (Contributed by Louie Lu in :issue:`15786`.) 1148 1149New in 3.6.3: 1150 1151Module Browser (on the File menu, formerly called Class Browser), 1152now displays nested functions and classes in addition to top-level 1153functions and classes. 1154(Contributed by Guilherme Polo, Cheryl Sabella, and Terry Jan Reedy 1155in :issue:`1612262`.) 1156 1157The IDLE features formerly implemented as extensions have been reimplemented 1158as normal features. Their settings have been moved from the Extensions tab 1159to other dialog tabs. 1160(Contributed by Charles Wohlganger and Terry Jan Reedy in :issue:`27099`.) 1161 1162The Settings dialog (Options, Configure IDLE) has been partly rewritten 1163to improve both appearance and function. 1164(Contributed by Cheryl Sabella and Terry Jan Reedy in multiple issues.) 1165 1166New in 3.6.4: 1167 1168The font sample now includes a selection of non-Latin characters so that 1169users can better see the effect of selecting a particular font. 1170(Contributed by Terry Jan Reedy in :issue:`13802`.) 1171The sample can be edited to include other characters. 1172(Contributed by Serhiy Storchaka in :issue:`31860`.) 1173 1174New in 3.6.6: 1175 1176Editor code context option revised. Box displays all context lines up to 1177maxlines. Clicking on a context line jumps the editor to that line. Context 1178colors for custom themes is added to Highlights tab of Settings dialog. 1179(Contributed by Cheryl Sabella and Terry Jan Reedy in :issue:`33642`, 1180:issue:`33768`, and :issue:`33679`.) 1181 1182On Windows, a new API call tells Windows that tk scales for DPI. On Windows 11838.1+ or 10, with DPI compatibility properties of the Python binary 1184unchanged, and a monitor resolution greater than 96 DPI, this should 1185make text and lines sharper. It should otherwise have no effect. 1186(Contributed by Terry Jan Reedy in :issue:`33656`.) 1187 1188New in 3.6.7: 1189 1190Output over N lines (50 by default) is squeezed down to a button. 1191N can be changed in the PyShell section of the General page of the 1192Settings dialog. Fewer, but possibly extra long, lines can be squeezed by 1193right clicking on the output. Squeezed output can be expanded in place 1194by double-clicking the button or into the clipboard or a separate window 1195by right-clicking the button. (Contributed by Tal Einat in :issue:`1529353`.) 1196 1197 1198importlib 1199--------- 1200 1201Import now raises the new exception :exc:`ModuleNotFoundError` 1202(subclass of :exc:`ImportError`) when it cannot find a module. Code 1203that current checks for ``ImportError`` (in try-except) will still work. 1204(Contributed by Eric Snow in :issue:`15767`.) 1205 1206:class:`importlib.util.LazyLoader` now calls 1207:meth:`~importlib.abc.Loader.create_module` on the wrapped loader, removing the 1208restriction that :class:`importlib.machinery.BuiltinImporter` and 1209:class:`importlib.machinery.ExtensionFileLoader` couldn't be used with 1210:class:`importlib.util.LazyLoader`. 1211 1212:func:`importlib.util.cache_from_source`, 1213:func:`importlib.util.source_from_cache`, and 1214:func:`importlib.util.spec_from_file_location` now accept a 1215:term:`path-like object`. 1216 1217 1218inspect 1219------- 1220 1221The :func:`inspect.signature() <inspect.signature>` function now reports the 1222implicit ``.0`` parameters generated by the compiler for comprehension and 1223generator expression scopes as if they were positional-only parameters called 1224``implicit0``. (Contributed by Jelle Zijlstra in :issue:`19611`.) 1225 1226To reduce code churn when upgrading from Python 2.7 and the legacy 1227:func:`inspect.getargspec` API, the previously documented deprecation of 1228:func:`inspect.getfullargspec` has been reversed. While this function is 1229convenient for single/source Python 2/3 code bases, the richer 1230:func:`inspect.signature` interface remains the recommended approach for new 1231code. (Contributed by Nick Coghlan in :issue:`27172`) 1232 1233 1234json 1235---- 1236 1237:func:`json.load` and :func:`json.loads` now support binary input. Encoded 1238JSON should be represented using either UTF-8, UTF-16, or UTF-32. 1239(Contributed by Serhiy Storchaka in :issue:`17909`.) 1240 1241 1242logging 1243------- 1244 1245The new :meth:`WatchedFileHandler.reopenIfNeeded() <logging.handlers.WatchedFileHandler.reopenIfNeeded>` 1246method has been added to add the ability to check if the log file needs to 1247be reopened. 1248(Contributed by Marian Horban in :issue:`24884`.) 1249 1250 1251math 1252---- 1253 1254The tau (*τ*) constant has been added to the :mod:`math` and :mod:`cmath` 1255modules. 1256(Contributed by Lisa Roach in :issue:`12345`, see :pep:`628` for details.) 1257 1258 1259multiprocessing 1260--------------- 1261 1262:ref:`Proxy Objects <multiprocessing-proxy_objects>` returned by 1263:func:`multiprocessing.Manager` can now be nested. 1264(Contributed by Davin Potts in :issue:`6766`.) 1265 1266 1267os 1268-- 1269 1270See the summary of :ref:`PEP 519 <whatsnew36-pep519>` for details on how the 1271:mod:`os` and :mod:`os.path` modules now support 1272:term:`path-like objects <path-like object>`. 1273 1274:func:`~os.scandir` now supports :class:`bytes` paths on Windows. 1275 1276A new :meth:`~os.scandir.close` method allows explicitly closing a 1277:func:`~os.scandir` iterator. The :func:`~os.scandir` iterator now 1278supports the :term:`context manager` protocol. If a :func:`scandir` 1279iterator is neither exhausted nor explicitly closed a :exc:`ResourceWarning` 1280will be emitted in its destructor. 1281(Contributed by Serhiy Storchaka in :issue:`25994`.) 1282 1283On Linux, :func:`os.urandom` now blocks until the system urandom entropy pool 1284is initialized to increase the security. See the :pep:`524` for the rationale. 1285 1286The Linux ``getrandom()`` syscall (get random bytes) is now exposed as the new 1287:func:`os.getrandom` function. 1288(Contributed by Victor Stinner, part of the :pep:`524`) 1289 1290 1291pathlib 1292------- 1293 1294:mod:`pathlib` now supports :term:`path-like objects <path-like object>`. 1295(Contributed by Brett Cannon in :issue:`27186`.) 1296 1297See the summary of :ref:`PEP 519 <whatsnew36-pep519>` for details. 1298 1299 1300pdb 1301--- 1302 1303The :class:`~pdb.Pdb` class constructor has a new optional *readrc* argument 1304to control whether ``.pdbrc`` files should be read. 1305 1306 1307pickle 1308------ 1309 1310Objects that need ``__new__`` called with keyword arguments can now be pickled 1311using :ref:`pickle protocols <pickle-protocols>` older than protocol version 4. 1312Protocol version 4 already supports this case. (Contributed by Serhiy 1313Storchaka in :issue:`24164`.) 1314 1315 1316pickletools 1317----------- 1318 1319:func:`pickletools.dis` now outputs the implicit memo index for the 1320``MEMOIZE`` opcode. 1321(Contributed by Serhiy Storchaka in :issue:`25382`.) 1322 1323 1324pydoc 1325----- 1326 1327The :mod:`pydoc` module has learned to respect the ``MANPAGER`` 1328environment variable. 1329(Contributed by Matthias Klose in :issue:`8637`.) 1330 1331:func:`help` and :mod:`pydoc` can now list named tuple fields in the 1332order they were defined rather than alphabetically. 1333(Contributed by Raymond Hettinger in :issue:`24879`.) 1334 1335 1336random 1337------- 1338 1339The new :func:`~random.choices` function returns a list of elements of 1340specified size from the given population with optional weights. 1341(Contributed by Raymond Hettinger in :issue:`18844`.) 1342 1343 1344re 1345-- 1346 1347Added support of modifier spans in regular expressions. Examples: 1348``'(?i:p)ython'`` matches ``'python'`` and ``'Python'``, but not ``'PYTHON'``; 1349``'(?i)g(?-i:v)r'`` matches ``'GvR'`` and ``'gvr'``, but not ``'GVR'``. 1350(Contributed by Serhiy Storchaka in :issue:`433028`.) 1351 1352Match object groups can be accessed by ``__getitem__``, which is 1353equivalent to ``group()``. So ``mo['name']`` is now equivalent to 1354``mo.group('name')``. (Contributed by Eric Smith in :issue:`24454`.) 1355 1356:class:`~re.Match` objects now support 1357:meth:`index-like objects <object.__index__>` as group 1358indices. 1359(Contributed by Jeroen Demeyer and Xiang Zhang in :issue:`27177`.) 1360 1361 1362readline 1363-------- 1364 1365Added :func:`~readline.set_auto_history` to enable or disable 1366automatic addition of input to the history list. (Contributed by 1367Tyler Crompton in :issue:`26870`.) 1368 1369 1370rlcompleter 1371----------- 1372 1373Private and special attribute names now are omitted unless the prefix starts 1374with underscores. A space or a colon is added after some completed keywords. 1375(Contributed by Serhiy Storchaka in :issue:`25011` and :issue:`25209`.) 1376 1377 1378shlex 1379----- 1380 1381The :class:`~shlex.shlex` has much 1382:ref:`improved shell compatibility <improved-shell-compatibility>` 1383through the new *punctuation_chars* argument to control which characters 1384are treated as punctuation. 1385(Contributed by Vinay Sajip in :issue:`1521950`.) 1386 1387 1388site 1389---- 1390 1391When specifying paths to add to :data:`sys.path` in a ``.pth`` file, 1392you may now specify file paths on top of directories (e.g. zip files). 1393(Contributed by Wolfgang Langner in :issue:`26587`). 1394 1395 1396sqlite3 1397------- 1398 1399:attr:`sqlite3.Cursor.lastrowid` now supports the ``REPLACE`` statement. 1400(Contributed by Alex LordThorsen in :issue:`16864`.) 1401 1402 1403socket 1404------ 1405 1406The :func:`~socket.socket.ioctl` function now supports the 1407:const:`~socket.SIO_LOOPBACK_FAST_PATH` control code. 1408(Contributed by Daniel Stokes in :issue:`26536`.) 1409 1410The :meth:`~socket.socket.getsockopt` constants ``SO_DOMAIN``, 1411``SO_PROTOCOL``, ``SO_PEERSEC``, and ``SO_PASSSEC`` are now supported. 1412(Contributed by Christian Heimes in :issue:`26907`.) 1413 1414The :meth:`~socket.socket.setsockopt` now supports the 1415``setsockopt(level, optname, None, optlen: int)`` form. 1416(Contributed by Christian Heimes in :issue:`27744`.) 1417 1418The socket module now supports the address family 1419:const:`~socket.AF_ALG` to interface with Linux Kernel crypto API. ``ALG_*``, 1420``SOL_ALG`` and :meth:`~socket.socket.sendmsg_afalg` were added. 1421(Contributed by Christian Heimes in :issue:`27744` with support from 1422Victor Stinner.) 1423 1424New Linux constants ``TCP_USER_TIMEOUT`` and ``TCP_CONGESTION`` were added. 1425(Contributed by Omar Sandoval, :issue:`26273`). 1426 1427 1428socketserver 1429------------ 1430 1431Servers based on the :mod:`socketserver` module, including those 1432defined in :mod:`http.server`, :mod:`xmlrpc.server` and 1433:mod:`wsgiref.simple_server`, now support the :term:`context manager` 1434protocol. 1435(Contributed by Aviv Palivoda in :issue:`26404`.) 1436 1437The :attr:`~socketserver.StreamRequestHandler.wfile` attribute of 1438:class:`~socketserver.StreamRequestHandler` classes now implements 1439the :class:`io.BufferedIOBase` writable interface. In particular, 1440calling :meth:`~io.BufferedIOBase.write` is now guaranteed to send the 1441data in full. (Contributed by Martin Panter in :issue:`26721`.) 1442 1443 1444ssl 1445--- 1446 1447:mod:`ssl` supports OpenSSL 1.1.0. The minimum recommend version is 1.0.2. 1448(Contributed by Christian Heimes in :issue:`26470`.) 1449 14503DES has been removed from the default cipher suites and ChaCha20 Poly1305 1451cipher suites have been added. 1452(Contributed by Christian Heimes in :issue:`27850` and :issue:`27766`.) 1453 1454:class:`~ssl.SSLContext` has better default configuration for options 1455and ciphers. 1456(Contributed by Christian Heimes in :issue:`28043`.) 1457 1458SSL session can be copied from one client-side connection to another 1459with the new :class:`~ssl.SSLSession` class. TLS session resumption can 1460speed up the initial handshake, reduce latency and improve performance 1461(Contributed by Christian Heimes in :issue:`19500` based on a draft by 1462Alex Warhawk.) 1463 1464The new :meth:`~ssl.SSLContext.get_ciphers` method can be used to 1465get a list of enabled ciphers in order of cipher priority. 1466 1467All constants and flags have been converted to :class:`~enum.IntEnum` and 1468:class:`~enum.IntFlags`. 1469(Contributed by Christian Heimes in :issue:`28025`.) 1470 1471Server and client-side specific TLS protocols for :class:`~ssl.SSLContext` 1472were added. 1473(Contributed by Christian Heimes in :issue:`28085`.) 1474 1475Added :attr:`ssl.SSLContext.post_handshake_auth` to enable and 1476:meth:`ssl.SSLSocket.verify_client_post_handshake` to initiate TLS 1.3 1477post-handshake authentication. 1478(Contributed by Christian Heimes in :gh:`78851`.) 1479 1480statistics 1481---------- 1482 1483A new :func:`~statistics.harmonic_mean` function has been added. 1484(Contributed by Steven D'Aprano in :issue:`27181`.) 1485 1486 1487struct 1488------ 1489 1490:mod:`struct` now supports IEEE 754 half-precision floats via the ``'e'`` 1491format specifier. 1492(Contributed by Eli Stevens, Mark Dickinson in :issue:`11734`.) 1493 1494 1495subprocess 1496---------- 1497 1498:class:`subprocess.Popen` destructor now emits a :exc:`ResourceWarning` warning 1499if the child process is still running. Use the context manager protocol (``with 1500proc: ...``) or explicitly call the :meth:`~subprocess.Popen.wait` method to 1501read the exit status of the child process. (Contributed by Victor Stinner in 1502:issue:`26741`.) 1503 1504The :class:`subprocess.Popen` constructor and all functions that pass arguments 1505through to it now accept *encoding* and *errors* arguments. Specifying either 1506of these will enable text mode for the *stdin*, *stdout* and *stderr* streams. 1507(Contributed by Steve Dower in :issue:`6135`.) 1508 1509 1510sys 1511--- 1512 1513The new :func:`~sys.getfilesystemencodeerrors` function returns the name of 1514the error mode used to convert between Unicode filenames and bytes filenames. 1515(Contributed by Steve Dower in :issue:`27781`.) 1516 1517On Windows the return value of the :func:`~sys.getwindowsversion` function 1518now includes the *platform_version* field which contains the accurate major 1519version, minor version and build number of the current operating system, 1520rather than the version that is being emulated for the process 1521(Contributed by Steve Dower in :issue:`27932`.) 1522 1523 1524telnetlib 1525--------- 1526 1527:class:`!telnetlib.Telnet` is now a context manager (contributed by 1528Stéphane Wirtel in :issue:`25485`). 1529 1530 1531time 1532---- 1533 1534The :class:`~time.struct_time` attributes :attr:`tm_gmtoff` and 1535:attr:`tm_zone` are now available on all platforms. 1536 1537 1538timeit 1539------ 1540 1541The new :meth:`Timer.autorange() <timeit.Timer.autorange>` convenience 1542method has been added to call :meth:`Timer.timeit() <timeit.Timer.timeit>` 1543repeatedly so that the total run time is greater or equal to 200 milliseconds. 1544(Contributed by Steven D'Aprano in :issue:`6422`.) 1545 1546:mod:`timeit` now warns when there is substantial (4x) variance 1547between best and worst times. 1548(Contributed by Serhiy Storchaka in :issue:`23552`.) 1549 1550 1551tkinter 1552------- 1553 1554Added methods :meth:`~tkinter.Variable.trace_add`, 1555:meth:`~tkinter.Variable.trace_remove` and :meth:`~tkinter.Variable.trace_info` 1556in the :class:`tkinter.Variable` class. They replace old methods 1557:meth:`~tkinter.Variable.trace_variable`, :meth:`~tkinter.Variable.trace`, 1558:meth:`~tkinter.Variable.trace_vdelete` and 1559:meth:`~tkinter.Variable.trace_vinfo` that use obsolete Tcl commands and might 1560not work in future versions of Tcl. 1561(Contributed by Serhiy Storchaka in :issue:`22115`). 1562 1563 1564.. _whatsnew36-traceback: 1565 1566traceback 1567--------- 1568 1569Both the traceback module and the interpreter's builtin exception display now 1570abbreviate long sequences of repeated lines in tracebacks as shown in the 1571following example:: 1572 1573 >>> def f(): f() 1574 ... 1575 >>> f() 1576 Traceback (most recent call last): 1577 File "<stdin>", line 1, in <module> 1578 File "<stdin>", line 1, in f 1579 File "<stdin>", line 1, in f 1580 File "<stdin>", line 1, in f 1581 [Previous line repeated 995 more times] 1582 RecursionError: maximum recursion depth exceeded 1583 1584(Contributed by Emanuel Barry in :issue:`26823`.) 1585 1586 1587tracemalloc 1588----------- 1589 1590The :mod:`tracemalloc` module now supports tracing memory allocations in 1591multiple different address spaces. 1592 1593The new :class:`~tracemalloc.DomainFilter` filter class has been added 1594to filter block traces by their address space (domain). 1595 1596(Contributed by Victor Stinner in :issue:`26588`.) 1597 1598 1599.. _whatsnew36-typing: 1600 1601typing 1602------ 1603 1604Since the :mod:`typing` module is :term:`provisional <provisional API>`, 1605all changes introduced in Python 3.6 have also been 1606backported to Python 3.5.x. 1607 1608The :mod:`typing` module has a much improved support for generic type 1609aliases. For example ``Dict[str, Tuple[S, T]]`` is now a valid 1610type annotation. 1611(Contributed by Guido van Rossum in `Github #195 1612<https://github.com/python/typing/pull/195>`_.) 1613 1614The :class:`typing.ContextManager` class has been added for 1615representing :class:`contextlib.AbstractContextManager`. 1616(Contributed by Brett Cannon in :issue:`25609`.) 1617 1618The :class:`typing.Collection` class has been added for 1619representing :class:`collections.abc.Collection`. 1620(Contributed by Ivan Levkivskyi in :issue:`27598`.) 1621 1622The :const:`typing.ClassVar` type construct has been added to 1623mark class variables. As introduced in :pep:`526`, a variable annotation 1624wrapped in ClassVar indicates that a given attribute is intended to be used as 1625a class variable and should not be set on instances of that class. 1626(Contributed by Ivan Levkivskyi in `Github #280 1627<https://github.com/python/typing/pull/280>`_.) 1628 1629A new :const:`~typing.TYPE_CHECKING` constant that is assumed to be 1630``True`` by the static type checkers, but is ``False`` at runtime. 1631(Contributed by Guido van Rossum in `Github #230 1632<https://github.com/python/typing/issues/230>`_.) 1633 1634A new :func:`~typing.NewType` helper function has been added to create 1635lightweight distinct types for annotations:: 1636 1637 from typing import NewType 1638 1639 UserId = NewType('UserId', int) 1640 some_id = UserId(524313) 1641 1642The static type checker will treat the new type as if it were a subclass 1643of the original type. (Contributed by Ivan Levkivskyi in `Github #189 1644<https://github.com/python/typing/issues/189>`_.) 1645 1646 1647unicodedata 1648----------- 1649 1650The :mod:`unicodedata` module now uses data from `Unicode 9.0.0 1651<https://unicode.org/versions/Unicode9.0.0/>`_. 1652(Contributed by Benjamin Peterson.) 1653 1654 1655unittest.mock 1656------------- 1657 1658The :class:`~unittest.mock.Mock` class has the following improvements: 1659 1660* Two new methods, :meth:`Mock.assert_called() 1661 <unittest.mock.Mock.assert_called>` and :meth:`Mock.assert_called_once() 1662 <unittest.mock.Mock.assert_called_once>` to check if the mock object 1663 was called. 1664 (Contributed by Amit Saha in :issue:`26323`.) 1665 1666* The :meth:`Mock.reset_mock() <unittest.mock.Mock.reset_mock>` method 1667 now has two optional keyword only arguments: *return_value* and 1668 *side_effect*. 1669 (Contributed by Kushal Das in :issue:`21271`.) 1670 1671 1672urllib.request 1673-------------- 1674 1675If a HTTP request has a file or iterable body (other than a 1676bytes object) but no ``Content-Length`` header, rather than 1677throwing an error, :class:`~urllib.request.AbstractHTTPHandler` now 1678falls back to use chunked transfer encoding. 1679(Contributed by Demian Brecht and Rolf Krahl in :issue:`12319`.) 1680 1681 1682urllib.robotparser 1683------------------ 1684 1685:class:`~urllib.robotparser.RobotFileParser` now supports the ``Crawl-delay`` and 1686``Request-rate`` extensions. 1687(Contributed by Nikolay Bogoychev in :issue:`16099`.) 1688 1689 1690venv 1691---- 1692 1693:mod:`venv` accepts a new parameter ``--prompt``. This parameter provides an 1694alternative prefix for the virtual environment. (Proposed by Łukasz Balcerzak 1695and ported to 3.6 by Stéphane Wirtel in :issue:`22829`.) 1696 1697 1698warnings 1699-------- 1700 1701A new optional *source* parameter has been added to the 1702:func:`warnings.warn_explicit` function: the destroyed object which emitted a 1703:exc:`ResourceWarning`. A *source* attribute has also been added to 1704:class:`warnings.WarningMessage` (contributed by Victor Stinner in 1705:issue:`26568` and :issue:`26567`). 1706 1707When a :exc:`ResourceWarning` warning is logged, the :mod:`tracemalloc` module is now 1708used to try to retrieve the traceback where the destroyed object was allocated. 1709 1710Example with the script ``example.py``:: 1711 1712 import warnings 1713 1714 def func(): 1715 return open(__file__) 1716 1717 f = func() 1718 f = None 1719 1720Output of the command ``python3.6 -Wd -X tracemalloc=5 example.py``:: 1721 1722 example.py:7: ResourceWarning: unclosed file <_io.TextIOWrapper name='example.py' mode='r' encoding='UTF-8'> 1723 f = None 1724 Object allocated at (most recent call first): 1725 File "example.py", lineno 4 1726 return open(__file__) 1727 File "example.py", lineno 6 1728 f = func() 1729 1730The "Object allocated at" traceback is new and is only displayed if 1731:mod:`tracemalloc` is tracing Python memory allocations and if the 1732:mod:`warnings` module was already imported. 1733 1734 1735winreg 1736------ 1737 1738Added the 64-bit integer type :data:`REG_QWORD <winreg.REG_QWORD>`. 1739(Contributed by Clement Rouault in :issue:`23026`.) 1740 1741 1742winsound 1743-------- 1744 1745Allowed keyword arguments to be passed to :func:`Beep <winsound.Beep>`, 1746:func:`MessageBeep <winsound.MessageBeep>`, and :func:`PlaySound 1747<winsound.PlaySound>` (:issue:`27982`). 1748 1749 1750xmlrpc.client 1751------------- 1752 1753The :mod:`xmlrpc.client` module now supports unmarshalling 1754additional data types used by the Apache XML-RPC implementation 1755for numerics and ``None``. 1756(Contributed by Serhiy Storchaka in :issue:`26885`.) 1757 1758 1759zipfile 1760------- 1761 1762A new :meth:`ZipInfo.from_file() <zipfile.ZipInfo.from_file>` class method 1763allows making a :class:`~zipfile.ZipInfo` instance from a filesystem file. 1764A new :meth:`ZipInfo.is_dir() <zipfile.ZipInfo.is_dir>` method can be used 1765to check if the :class:`~zipfile.ZipInfo` instance represents a directory. 1766(Contributed by Thomas Kluyver in :issue:`26039`.) 1767 1768The :meth:`ZipFile.open() <zipfile.ZipFile.open>` method can now be used to 1769write data into a ZIP file, as well as for extracting data. 1770(Contributed by Thomas Kluyver in :issue:`26039`.) 1771 1772 1773zlib 1774---- 1775 1776The :func:`~zlib.compress` and :func:`~zlib.decompress` functions now accept 1777keyword arguments. 1778(Contributed by Aviv Palivoda in :issue:`26243` and 1779Xiang Zhang in :issue:`16764` respectively.) 1780 1781 1782Optimizations 1783============= 1784 1785* The Python interpreter now uses a 16-bit wordcode instead of bytecode which 1786 made a number of opcode optimizations possible. 1787 (Contributed by Demur Rumed with input and reviews from 1788 Serhiy Storchaka and Victor Stinner in :issue:`26647` and :issue:`28050`.) 1789 1790* The :class:`asyncio.Future` class now has an optimized C implementation. 1791 (Contributed by Yury Selivanov and INADA Naoki in :issue:`26081`.) 1792 1793* The :class:`asyncio.Task` class now has an optimized 1794 C implementation. (Contributed by Yury Selivanov in :issue:`28544`.) 1795 1796* Various implementation improvements in the :mod:`typing` module 1797 (such as caching of generic types) allow up to 30 times performance 1798 improvements and reduced memory footprint. 1799 1800* The ASCII decoder is now up to 60 times as fast for error handlers 1801 ``surrogateescape``, ``ignore`` and ``replace`` (Contributed 1802 by Victor Stinner in :issue:`24870`). 1803 1804* The ASCII and the Latin1 encoders are now up to 3 times as fast for the 1805 error handler ``surrogateescape`` 1806 (Contributed by Victor Stinner in :issue:`25227`). 1807 1808* The UTF-8 encoder is now up to 75 times as fast for error handlers 1809 ``ignore``, ``replace``, ``surrogateescape``, ``surrogatepass`` (Contributed 1810 by Victor Stinner in :issue:`25267`). 1811 1812* The UTF-8 decoder is now up to 15 times as fast for error handlers 1813 ``ignore``, ``replace`` and ``surrogateescape`` (Contributed 1814 by Victor Stinner in :issue:`25301`). 1815 1816* ``bytes % args`` is now up to 2 times faster. (Contributed by Victor Stinner 1817 in :issue:`25349`). 1818 1819* ``bytearray % args`` is now between 2.5 and 5 times faster. (Contributed by 1820 Victor Stinner in :issue:`25399`). 1821 1822* Optimize :meth:`bytes.fromhex` and :meth:`bytearray.fromhex`: they are now 1823 between 2x and 3.5x faster. (Contributed by Victor Stinner in :issue:`25401`). 1824 1825* Optimize ``bytes.replace(b'', b'.')`` and ``bytearray.replace(b'', b'.')``: 1826 up to 80% faster. (Contributed by Josh Snider in :issue:`26574`). 1827 1828* Allocator functions of the :c:func:`PyMem_Malloc` domain 1829 (:c:macro:`PYMEM_DOMAIN_MEM`) now use the :ref:`pymalloc memory allocator 1830 <pymalloc>` instead of :c:func:`malloc` function of the C library. The 1831 pymalloc allocator is optimized for objects smaller or equal to 512 bytes 1832 with a short lifetime, and use :c:func:`malloc` for larger memory blocks. 1833 (Contributed by Victor Stinner in :issue:`26249`). 1834 1835* :func:`pickle.load` and :func:`pickle.loads` are now up to 10% faster when 1836 deserializing many small objects (Contributed by Victor Stinner in 1837 :issue:`27056`). 1838 1839* Passing :term:`keyword arguments <keyword argument>` to a function has an 1840 overhead in comparison with passing :term:`positional arguments 1841 <positional argument>`. Now in extension functions implemented with using 1842 Argument Clinic this overhead is significantly decreased. 1843 (Contributed by Serhiy Storchaka in :issue:`27574`). 1844 1845* Optimized :func:`~glob.glob` and :func:`~glob.iglob` functions in the 1846 :mod:`glob` module; they are now about 3--6 times faster. 1847 (Contributed by Serhiy Storchaka in :issue:`25596`). 1848 1849* Optimized globbing in :mod:`pathlib` by using :func:`os.scandir`; 1850 it is now about 1.5--4 times faster. 1851 (Contributed by Serhiy Storchaka in :issue:`26032`). 1852 1853* :class:`xml.etree.ElementTree` parsing, iteration and deepcopy performance 1854 has been significantly improved. 1855 (Contributed by Serhiy Storchaka in :issue:`25638`, :issue:`25873`, 1856 and :issue:`25869`.) 1857 1858* Creation of :class:`fractions.Fraction` instances from floats and 1859 decimals is now 2 to 3 times faster. 1860 (Contributed by Serhiy Storchaka in :issue:`25971`.) 1861 1862 1863Build and C API Changes 1864======================= 1865 1866* Python now requires some C99 support in the toolchain to build. 1867 Most notably, Python now uses standard integer types and macros in 1868 place of custom macros like ``PY_LONG_LONG``. 1869 For more information, see :pep:`7` and :issue:`17884`. 1870 1871* Cross-compiling CPython with the Android NDK and the Android API level set to 1872 21 (Android 5.0 Lollipop) or greater runs successfully. While Android is not 1873 yet a supported platform, the Python test suite runs on the Android emulator 1874 with only about 16 tests failures. See the Android meta-issue :issue:`26865`. 1875 1876* The ``--enable-optimizations`` configure flag has been added. Turning it on 1877 will activate expensive optimizations like PGO. 1878 (Original patch by Alecsandru Patrascu of Intel in :issue:`26359`.) 1879 1880* The :term:`GIL <global interpreter lock>` must now be held when allocator 1881 functions of :c:macro:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and 1882 :c:macro:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called. 1883 1884* New :c:func:`Py_FinalizeEx` API which indicates if flushing buffered data 1885 failed. 1886 (Contributed by Martin Panter in :issue:`5319`.) 1887 1888* :c:func:`PyArg_ParseTupleAndKeywords` now supports :ref:`positional-only 1889 parameters <positional-only_parameter>`. Positional-only parameters are 1890 defined by empty names. 1891 (Contributed by Serhiy Storchaka in :issue:`26282`). 1892 1893* ``PyTraceback_Print`` method now abbreviates long sequences of repeated lines 1894 as ``"[Previous line repeated {count} more times]"``. 1895 (Contributed by Emanuel Barry in :issue:`26823`.) 1896 1897* The new :c:func:`PyErr_SetImportErrorSubclass` function allows for 1898 specifying a subclass of :exc:`ImportError` to raise. 1899 (Contributed by Eric Snow in :issue:`15767`.) 1900 1901* The new :c:func:`PyErr_ResourceWarning` function can be used to generate 1902 a :exc:`ResourceWarning` providing the source of the resource allocation. 1903 (Contributed by Victor Stinner in :issue:`26567`.) 1904 1905* The new :c:func:`PyOS_FSPath` function returns the file system 1906 representation of a :term:`path-like object`. 1907 (Contributed by Brett Cannon in :issue:`27186`.) 1908 1909* The :c:func:`PyUnicode_FSConverter` and :c:func:`PyUnicode_FSDecoder` 1910 functions will now accept :term:`path-like objects <path-like object>`. 1911 1912 1913Other Improvements 1914================== 1915 1916* When :option:`--version` (short form: :option:`-V`) is supplied twice, 1917 Python prints :data:`sys.version` for detailed information. 1918 1919 .. code-block:: shell-session 1920 1921 $ ./python -VV 1922 Python 3.6.0b4+ (3.6:223967b49e49+, Nov 21 2016, 20:55:04) 1923 [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] 1924 1925 1926Deprecated 1927========== 1928 1929New Keywords 1930------------ 1931 1932``async`` and ``await`` are not recommended to be used as variable, class, 1933function or module names. Introduced by :pep:`492` in Python 3.5, they will 1934become proper keywords in Python 3.7. Starting in Python 3.6, the use of 1935``async`` or ``await`` as names will generate a :exc:`DeprecationWarning`. 1936 1937 1938Deprecated Python behavior 1939-------------------------- 1940 1941Raising the :exc:`StopIteration` exception inside a generator will now 1942generate a :exc:`DeprecationWarning`, and will trigger a :exc:`RuntimeError` 1943in Python 3.7. See :ref:`whatsnew-pep-479` for details. 1944 1945The :meth:`__aiter__` method is now expected to return an asynchronous 1946iterator directly instead of returning an awaitable as previously. 1947Doing the former will trigger a :exc:`DeprecationWarning`. Backward 1948compatibility will be removed in Python 3.7. 1949(Contributed by Yury Selivanov in :issue:`27243`.) 1950 1951A backslash-character pair that is not a valid escape sequence now generates 1952a :exc:`DeprecationWarning`. Although this will eventually become a 1953:exc:`SyntaxError`, that will not be for several Python releases. 1954(Contributed by Emanuel Barry in :issue:`27364`.) 1955 1956When performing a relative import, falling back on ``__name__`` and 1957``__path__`` from the calling module when ``__spec__`` or 1958``__package__`` are not defined now raises an :exc:`ImportWarning`. 1959(Contributed by Rose Ames in :issue:`25791`.) 1960 1961 1962Deprecated Python modules, functions and methods 1963------------------------------------------------ 1964 1965asynchat 1966~~~~~~~~ 1967 1968The :mod:`!asynchat` has been deprecated in favor of :mod:`asyncio`. 1969(Contributed by Mariatta in :issue:`25002`.) 1970 1971 1972asyncore 1973~~~~~~~~ 1974 1975The :mod:`!asyncore` has been deprecated in favor of :mod:`asyncio`. 1976(Contributed by Mariatta in :issue:`25002`.) 1977 1978 1979dbm 1980~~~ 1981 1982Unlike other :mod:`dbm` implementations, the :mod:`dbm.dumb` module 1983creates databases with the ``'rw'`` mode and allows modifying the database 1984opened with the ``'r'`` mode. This behavior is now deprecated and will 1985be removed in 3.8. 1986(Contributed by Serhiy Storchaka in :issue:`21708`.) 1987 1988 1989distutils 1990~~~~~~~~~ 1991 1992The undocumented ``extra_path`` argument to the 1993``distutils.Distribution`` constructor is now considered deprecated 1994and will raise a warning if set. Support for this parameter will be 1995removed in a future Python release. See :issue:`27919` for details. 1996 1997 1998grp 1999~~~ 2000 2001The support of non-integer arguments in :func:`~grp.getgrgid` has been 2002deprecated. 2003(Contributed by Serhiy Storchaka in :issue:`26129`.) 2004 2005 2006importlib 2007~~~~~~~~~ 2008 2009The :meth:`importlib.machinery.SourceFileLoader.load_module` and 2010:meth:`importlib.machinery.SourcelessFileLoader.load_module` methods 2011are now deprecated. They were the only remaining implementations of 2012:meth:`importlib.abc.Loader.load_module` in :mod:`importlib` that had not 2013been deprecated in previous versions of Python in favour of 2014:meth:`importlib.abc.Loader.exec_module`. 2015 2016The :class:`importlib.machinery.WindowsRegistryFinder` class is now 2017deprecated. As of 3.6.0, it is still added to :data:`sys.meta_path` by 2018default (on Windows), but this may change in future releases. 2019 2020os 2021~~ 2022 2023Undocumented support of general :term:`bytes-like objects <bytes-like object>` 2024as paths in :mod:`os` functions, :func:`compile` and similar functions is 2025now deprecated. 2026(Contributed by Serhiy Storchaka in :issue:`25791` and :issue:`26754`.) 2027 2028re 2029~~ 2030 2031Support for inline flags ``(?letters)`` in the middle of the regular 2032expression has been deprecated and will be removed in a future Python 2033version. Flags at the start of a regular expression are still allowed. 2034(Contributed by Serhiy Storchaka in :issue:`22493`.) 2035 2036ssl 2037~~~ 2038 2039OpenSSL 0.9.8, 1.0.0 and 1.0.1 are deprecated and no longer supported. 2040In the future the :mod:`ssl` module will require at least OpenSSL 1.0.2 or 20411.1.0. 2042 2043SSL-related arguments like ``certfile``, ``keyfile`` and ``check_hostname`` 2044in :mod:`ftplib`, :mod:`http.client`, :mod:`imaplib`, :mod:`poplib`, 2045and :mod:`smtplib` have been deprecated in favor of ``context``. 2046(Contributed by Christian Heimes in :issue:`28022`.) 2047 2048A couple of protocols and functions of the :mod:`ssl` module are now 2049deprecated. Some features will no longer be available in future versions 2050of OpenSSL. Other features are deprecated in favor of a different API. 2051(Contributed by Christian Heimes in :issue:`28022` and :issue:`26470`.) 2052 2053tkinter 2054~~~~~~~ 2055 2056The :mod:`!tkinter.tix` module is now deprecated. :mod:`tkinter` users 2057should use :mod:`tkinter.ttk` instead. 2058 2059.. _whatsnew36-venv: 2060 2061venv 2062~~~~ 2063 2064The ``pyvenv`` script has been deprecated in favour of ``python3 -m venv``. 2065This prevents confusion as to what Python interpreter ``pyvenv`` is 2066connected to and thus what Python interpreter will be used by the virtual 2067environment. (Contributed by Brett Cannon in :issue:`25154`.) 2068 2069 2070xml 2071--- 2072 2073* As mitigation against DTD and external entity retrieval, the 2074 :mod:`xml.dom.minidom` and :mod:`xml.sax` modules no longer process 2075 external entities by default. 2076 (Contributed by Christian Heimes in :gh:`61441`.) 2077 2078 2079Deprecated functions and types of the C API 2080------------------------------------------- 2081 2082Undocumented functions :c:func:`!PyUnicode_AsEncodedObject`, 2083:c:func:`!PyUnicode_AsDecodedObject`, :c:func:`!PyUnicode_AsEncodedUnicode` 2084and :c:func:`!PyUnicode_AsDecodedUnicode` are deprecated now. 2085Use the :ref:`generic codec based API <codec-registry>` instead. 2086 2087 2088Deprecated Build Options 2089------------------------ 2090 2091The ``--with-system-ffi`` configure flag is now on by default on non-macOS 2092UNIX platforms. It may be disabled by using ``--without-system-ffi``, but 2093using the flag is deprecated and will not be accepted in Python 3.7. 2094macOS is unaffected by this change. Note that many OS distributors already 2095use the ``--with-system-ffi`` flag when building their system Python. 2096 2097 2098Removed 2099======= 2100 2101API and Feature Removals 2102------------------------ 2103 2104* Unknown escapes consisting of ``'\'`` and an ASCII letter in 2105 regular expressions will now cause an error. In replacement templates for 2106 :func:`re.sub` they are still allowed, but deprecated. 2107 The :const:`re.LOCALE` flag can now only be used with binary patterns. 2108 2109* ``inspect.getmoduleinfo()`` was removed (was deprecated since CPython 3.3). 2110 :func:`inspect.getmodulename` should be used for obtaining the module 2111 name for a given path. 2112 (Contributed by Yury Selivanov in :issue:`13248`.) 2113 2114* ``traceback.Ignore`` class and ``traceback.usage``, ``traceback.modname``, 2115 ``traceback.fullmodname``, ``traceback.find_lines_from_code``, 2116 ``traceback.find_lines``, ``traceback.find_strings``, 2117 ``traceback.find_executable_lines`` methods were removed from the 2118 :mod:`traceback` module. They were undocumented methods deprecated since 2119 Python 3.2 and equivalent functionality is available from private methods. 2120 2121* The ``tk_menuBar()`` and ``tk_bindForTraversal()`` dummy methods in 2122 :mod:`tkinter` widget classes were removed (corresponding Tk commands 2123 were obsolete since Tk 4.0). 2124 2125* The :meth:`~zipfile.ZipFile.open` method of the :class:`zipfile.ZipFile` 2126 class no longer supports the ``'U'`` mode (was deprecated since Python 3.4). 2127 Use :class:`io.TextIOWrapper` for reading compressed text files in 2128 :term:`universal newlines` mode. 2129 2130* The undocumented ``IN``, ``CDROM``, ``DLFCN``, ``TYPES``, ``CDIO``, and 2131 ``STROPTS`` modules have been removed. They had been available in the 2132 platform specific ``Lib/plat-*/`` directories, but were chronically out of 2133 date, inconsistently available across platforms, and unmaintained. The 2134 script that created these modules is still available in the source 2135 distribution at `Tools/scripts/h2py.py 2136 <https://github.com/python/cpython/blob/v3.6.15/Tools/scripts/h2py.py>`_. 2137 2138* The deprecated ``asynchat.fifo`` class has been removed. 2139 2140 2141Porting to Python 3.6 2142===================== 2143 2144This section lists previously described changes and other bugfixes 2145that may require changes to your code. 2146 2147Changes in 'python' Command Behavior 2148------------------------------------ 2149 2150* The output of a special Python build with defined ``COUNT_ALLOCS``, 2151 ``SHOW_ALLOC_COUNT`` or ``SHOW_TRACK_COUNT`` macros is now off by 2152 default. It can be re-enabled using the ``-X showalloccount`` option. 2153 It now outputs to ``stderr`` instead of ``stdout``. 2154 (Contributed by Serhiy Storchaka in :issue:`23034`.) 2155 2156 2157Changes in the Python API 2158------------------------- 2159 2160* :func:`open() <open>` will no longer allow combining the ``'U'`` mode flag 2161 with ``'+'``. 2162 (Contributed by Jeff Balogh and John O'Connor in :issue:`2091`.) 2163 2164* :mod:`sqlite3` no longer implicitly commits an open transaction before DDL 2165 statements. 2166 2167* On Linux, :func:`os.urandom` now blocks until the system urandom entropy pool 2168 is initialized to increase the security. 2169 2170* When :meth:`importlib.abc.Loader.exec_module` is defined, 2171 :meth:`importlib.abc.Loader.create_module` must also be defined. 2172 2173* :c:func:`PyErr_SetImportError` now sets :exc:`TypeError` when its **msg** 2174 argument is not set. Previously only ``NULL`` was returned. 2175 2176* The format of the :attr:`~codeobject.co_lnotab` attribute of code objects 2177 changed to support 2178 a negative line number delta. By default, Python does not emit bytecode with 2179 a negative line number delta. Functions using :attr:`frame.f_lineno`, 2180 ``PyFrame_GetLineNumber()`` or ``PyCode_Addr2Line()`` are not affected. 2181 Functions directly decoding :attr:`!co_lnotab` should be updated to use a signed 2182 8-bit integer type for the line number delta, but this is only required to 2183 support applications using a negative line number delta. See 2184 ``Objects/lnotab_notes.txt`` for the :attr:`!co_lnotab` format and how to decode 2185 it, and see the :pep:`511` for the rationale. 2186 2187* The functions in the :mod:`compileall` module now return booleans instead 2188 of ``1`` or ``0`` to represent success or failure, respectively. Thanks to 2189 booleans being a subclass of integers, this should only be an issue if you 2190 were doing identity checks for ``1`` or ``0``. See :issue:`25768`. 2191 2192* Reading the :attr:`~urllib.parse.SplitResult.port` attribute of 2193 :func:`urllib.parse.urlsplit` and :func:`~urllib.parse.urlparse` results 2194 now raises :exc:`ValueError` for out-of-range values, rather than 2195 returning :const:`None`. See :issue:`20059`. 2196 2197* The :mod:`!imp` module now raises a :exc:`DeprecationWarning` instead of 2198 :exc:`PendingDeprecationWarning`. 2199 2200* The following modules have had missing APIs added to their :attr:`__all__` 2201 attributes to match the documented APIs: 2202 :mod:`calendar`, :mod:`!cgi`, :mod:`csv`, 2203 :mod:`~xml.etree.ElementTree`, :mod:`enum`, 2204 :mod:`fileinput`, :mod:`ftplib`, :mod:`logging`, :mod:`mailbox`, 2205 :mod:`mimetypes`, :mod:`optparse`, :mod:`plistlib`, :mod:`!smtpd`, 2206 :mod:`subprocess`, :mod:`tarfile`, :mod:`threading` and 2207 :mod:`wave`. This means they will export new symbols when ``import *`` 2208 is used. 2209 (Contributed by Joel Taddei and Jacek Kołodziej in :issue:`23883`.) 2210 2211* When performing a relative import, if ``__package__`` does not compare equal 2212 to ``__spec__.parent`` then :exc:`ImportWarning` is raised. 2213 (Contributed by Brett Cannon in :issue:`25791`.) 2214 2215* When a relative import is performed and no parent package is known, then 2216 :exc:`ImportError` will be raised. Previously, :exc:`SystemError` could be 2217 raised. (Contributed by Brett Cannon in :issue:`18018`.) 2218 2219* Servers based on the :mod:`socketserver` module, including those 2220 defined in :mod:`http.server`, :mod:`xmlrpc.server` and 2221 :mod:`wsgiref.simple_server`, now only catch exceptions derived 2222 from :exc:`Exception`. Therefore if a request handler raises 2223 an exception like :exc:`SystemExit` or :exc:`KeyboardInterrupt`, 2224 :meth:`~socketserver.BaseServer.handle_error` is no longer called, and 2225 the exception will stop a single-threaded server. (Contributed by 2226 Martin Panter in :issue:`23430`.) 2227 2228* :func:`!spwd.getspnam` now raises a :exc:`PermissionError` instead of 2229 :exc:`KeyError` if the user doesn't have privileges. 2230 2231* The :meth:`socket.socket.close` method now raises an exception if 2232 an error (e.g. ``EBADF``) was reported by the underlying system call. 2233 (Contributed by Martin Panter in :issue:`26685`.) 2234 2235* The *decode_data* argument for the :class:`!smtpd.SMTPChannel` and 2236 :class:`!smtpd.SMTPServer` constructors is now ``False`` by default. 2237 This means that the argument passed to 2238 :meth:`!process_message` is now a bytes object by 2239 default, and :meth:`!process_message` will be passed keyword arguments. 2240 Code that has already been updated in accordance with the deprecation 2241 warning generated by 3.5 will not be affected. 2242 2243* All optional arguments of the :func:`~json.dump`, :func:`~json.dumps`, 2244 :func:`~json.load` and :func:`~json.loads` functions and 2245 :class:`~json.JSONEncoder` and :class:`~json.JSONDecoder` class 2246 constructors in the :mod:`json` module are now :ref:`keyword-only 2247 <keyword-only_parameter>`. 2248 (Contributed by Serhiy Storchaka in :issue:`18726`.) 2249 2250* Subclasses of :class:`type` which don't override ``type.__new__`` may no 2251 longer use the one-argument form to get the type of an object. 2252 2253* As part of :pep:`487`, the handling of keyword arguments passed to 2254 :class:`type` (other than the metaclass hint, ``metaclass``) is now 2255 consistently delegated to :meth:`object.__init_subclass__`. This means that 2256 :meth:`type.__new__` and :meth:`type.__init__` both now accept arbitrary 2257 keyword arguments, but :meth:`object.__init_subclass__` (which is called from 2258 :meth:`type.__new__`) will reject them by default. Custom metaclasses 2259 accepting additional keyword arguments will need to adjust their calls to 2260 :meth:`type.__new__` (whether direct or via :class:`super`) accordingly. 2261 2262* In ``distutils.command.sdist.sdist``, the ``default_format`` 2263 attribute has been removed and is no longer honored. Instead, the 2264 gzipped tarfile format is the default on all platforms and no 2265 platform-specific selection is made. 2266 In environments where distributions are 2267 built on Windows and zip distributions are required, configure 2268 the project with a ``setup.cfg`` file containing the following: 2269 2270 .. code-block:: ini 2271 2272 [sdist] 2273 formats=zip 2274 2275 This behavior has also been backported to earlier Python versions 2276 by Setuptools 26.0.0. 2277 2278* In the :mod:`urllib.request` module and the 2279 :meth:`http.client.HTTPConnection.request` method, if no Content-Length 2280 header field has been specified and the request body is a file object, 2281 it is now sent with HTTP 1.1 chunked encoding. If a file object has to 2282 be sent to a HTTP 1.0 server, the Content-Length value now has to be 2283 specified by the caller. 2284 (Contributed by Demian Brecht and Rolf Krahl with tweaks from 2285 Martin Panter in :issue:`12319`.) 2286 2287* The :class:`~csv.DictReader` now returns rows of type 2288 :class:`~collections.OrderedDict`. 2289 (Contributed by Steve Holden in :issue:`27842`.) 2290 2291* The :const:`!crypt.METHOD_CRYPT` will no longer be added to ``crypt.methods`` 2292 if unsupported by the platform. 2293 (Contributed by Victor Stinner in :issue:`25287`.) 2294 2295* The *verbose* and *rename* arguments for 2296 :func:`~collections.namedtuple` are now keyword-only. 2297 (Contributed by Raymond Hettinger in :issue:`25628`.) 2298 2299* On Linux, :func:`ctypes.util.find_library` now looks in 2300 ``LD_LIBRARY_PATH`` for shared libraries. 2301 (Contributed by Vinay Sajip in :issue:`9998`.) 2302 2303* The :class:`imaplib.IMAP4` class now handles flags containing the 2304 ``']'`` character in messages sent from the server to improve 2305 real-world compatibility. 2306 (Contributed by Lita Cho in :issue:`21815`.) 2307 2308* The :func:`mmap.write() <mmap.write>` function now returns the number 2309 of bytes written like other write methods. 2310 (Contributed by Jakub Stasiak in :issue:`26335`.) 2311 2312* The :func:`pkgutil.iter_modules` and :func:`pkgutil.walk_packages` 2313 functions now return :class:`~pkgutil.ModuleInfo` named tuples. 2314 (Contributed by Ramchandra Apte in :issue:`17211`.) 2315 2316* :func:`re.sub` now raises an error for invalid numerical group 2317 references in replacement templates even if the pattern is not 2318 found in the string. The error message for invalid group references 2319 now includes the group index and the position of the reference. 2320 (Contributed by SilentGhost, Serhiy Storchaka in :issue:`25953`.) 2321 2322* :class:`zipfile.ZipFile` will now raise :exc:`NotImplementedError` for 2323 unrecognized compression values. Previously a plain :exc:`RuntimeError` 2324 was raised. Additionally, calling :class:`~zipfile.ZipFile` methods 2325 on a closed ZipFile or calling the :meth:`~zipfile.ZipFile.write` method 2326 on a ZipFile created with mode ``'r'`` will raise a :exc:`ValueError`. 2327 Previously, a :exc:`RuntimeError` was raised in those scenarios. 2328 2329* when custom metaclasses are combined with zero-argument :func:`super` or 2330 direct references from methods to the implicit ``__class__`` closure 2331 variable, the implicit ``__classcell__`` namespace entry must now be passed 2332 up to ``type.__new__`` for initialisation. Failing to do so will result in 2333 a :exc:`DeprecationWarning` in Python 3.6 and a :exc:`RuntimeError` in 2334 Python 3.8. 2335 2336* With the introduction of :exc:`ModuleNotFoundError`, import system consumers 2337 may start expecting import system replacements to raise that more specific 2338 exception when appropriate, rather than the less-specific :exc:`ImportError`. 2339 To provide future compatibility with such consumers, implementers of 2340 alternative import systems that completely replace :func:`__import__` will 2341 need to update their implementations to raise the new subclass when a module 2342 can't be found at all. Implementers of compliant plugins to the default 2343 import system shouldn't need to make any changes, as the default import 2344 system will raise the new subclass when appropriate. 2345 2346 2347Changes in the C API 2348-------------------- 2349 2350* The :c:func:`PyMem_Malloc` allocator family now uses the :ref:`pymalloc allocator 2351 <pymalloc>` rather than the system :c:func:`malloc`. Applications calling 2352 :c:func:`PyMem_Malloc` without holding the GIL can now crash. Set the 2353 :envvar:`PYTHONMALLOC` environment variable to ``debug`` to validate the 2354 usage of memory allocators in your application. See :issue:`26249`. 2355 2356* :c:func:`Py_Exit` (and the main interpreter) now override the exit status 2357 with 120 if flushing buffered data failed. See :issue:`5319`. 2358 2359 2360CPython bytecode changes 2361------------------------ 2362 2363There have been several major changes to the :term:`bytecode` in Python 3.6. 2364 2365* The Python interpreter now uses a 16-bit wordcode instead of bytecode. 2366 (Contributed by Demur Rumed with input and reviews from 2367 Serhiy Storchaka and Victor Stinner in :issue:`26647` and :issue:`28050`.) 2368 2369* The new :opcode:`!FORMAT_VALUE` and :opcode:`BUILD_STRING` opcodes as part 2370 of the :ref:`formatted string literal <whatsnew36-pep498>` implementation. 2371 (Contributed by Eric Smith in :issue:`25483` and 2372 Serhiy Storchaka in :issue:`27078`.) 2373 2374* The new :opcode:`BUILD_CONST_KEY_MAP` opcode to optimize the creation 2375 of dictionaries with constant keys. 2376 (Contributed by Serhiy Storchaka in :issue:`27140`.) 2377 2378* The function call opcodes have been heavily reworked for better performance 2379 and simpler implementation. 2380 The :opcode:`MAKE_FUNCTION`, :opcode:`!CALL_FUNCTION`, 2381 :opcode:`!CALL_FUNCTION_KW` and :opcode:`!BUILD_MAP_UNPACK_WITH_CALL` opcodes 2382 have been modified, the new :opcode:`CALL_FUNCTION_EX` and 2383 :opcode:`!BUILD_TUPLE_UNPACK_WITH_CALL` have been added, and 2384 ``CALL_FUNCTION_VAR``, ``CALL_FUNCTION_VAR_KW`` and ``MAKE_CLOSURE`` opcodes 2385 have been removed. 2386 (Contributed by Demur Rumed in :issue:`27095`, and Serhiy Storchaka in 2387 :issue:`27213`, :issue:`28257`.) 2388 2389* The new :opcode:`SETUP_ANNOTATIONS` and :opcode:`!STORE_ANNOTATION` opcodes 2390 have been added to support the new :term:`variable annotation` syntax. 2391 (Contributed by Ivan Levkivskyi in :issue:`27985`.) 2392 2393 2394Notable changes in Python 3.6.2 2395=============================== 2396 2397New ``make regen-all`` build target 2398----------------------------------- 2399 2400To simplify cross-compilation, and to ensure that CPython can reliably be 2401compiled without requiring an existing version of Python to already be 2402available, the autotools-based build system no longer attempts to implicitly 2403recompile generated files based on file modification times. 2404 2405Instead, a new ``make regen-all`` command has been added to force regeneration 2406of these files when desired (e.g. after an initial version of Python has 2407already been built based on the pregenerated versions). 2408 2409More selective regeneration targets are also defined - see 2410:source:`Makefile.pre.in` for details. 2411 2412(Contributed by Victor Stinner in :issue:`23404`.) 2413 2414.. versionadded:: 3.6.2 2415 2416 2417Removal of ``make touch`` build target 2418-------------------------------------- 2419 2420The ``make touch`` build target previously used to request implicit regeneration 2421of generated files by updating their modification times has been removed. 2422 2423It has been replaced by the new ``make regen-all`` target. 2424 2425(Contributed by Victor Stinner in :issue:`23404`.) 2426 2427.. versionchanged:: 3.6.2 2428 2429 2430Notable changes in Python 3.6.4 2431=============================== 2432 2433The ``PyExc_RecursionErrorInst`` singleton that was part of the public API 2434has been removed as its members being never cleared may cause a segfault 2435during finalization of the interpreter. 2436(Contributed by Xavier de Gaye in :issue:`22898` and :issue:`30697`.) 2437 2438 2439Notable changes in Python 3.6.5 2440=============================== 2441 2442The :func:`locale.localeconv` function now sets temporarily the ``LC_CTYPE`` 2443locale to the ``LC_NUMERIC`` locale in some cases. 2444(Contributed by Victor Stinner in :issue:`31900`.) 2445 2446 2447Notable changes in Python 3.6.7 2448=============================== 2449 2450:mod:`xml.dom.minidom` and :mod:`xml.sax` modules no longer process 2451external entities by default. See also :gh:`61441`. 2452 2453In 3.6.7 the :mod:`tokenize` module now implicitly emits a ``NEWLINE`` token 2454when provided with input that does not have a trailing new line. This behavior 2455now matches what the C tokenizer does internally. 2456(Contributed by Ammar Askar in :issue:`33899`.) 2457 2458Notable changes in Python 3.6.10 2459================================ 2460 2461Due to significant security concerns, the *reuse_address* parameter of 2462:meth:`asyncio.loop.create_datagram_endpoint` is no longer supported. This is 2463because of the behavior of the socket option ``SO_REUSEADDR`` in UDP. For more 2464details, see the documentation for ``loop.create_datagram_endpoint()``. 2465(Contributed by Kyle Stanley, Antoine Pitrou, and Yury Selivanov in 2466:issue:`37228`.) 2467 2468Notable changes in Python 3.6.13 2469================================ 2470 2471Earlier Python versions allowed using both ``;`` and ``&`` as 2472query parameter separators in :func:`urllib.parse.parse_qs` and 2473:func:`urllib.parse.parse_qsl`. Due to security concerns, and to conform with 2474newer W3C recommendations, this has been changed to allow only a single 2475separator key, with ``&`` as the default. This change also affects 2476:func:`!cgi.parse` and :func:`!cgi.parse_multipart` as they use the affected 2477functions internally. For more details, please see their respective 2478documentation. 2479(Contributed by Adam Goldschmidt, Senthil Kumaran and Ken Jin in :issue:`42967`.) 2480 2481Notable changes in Python 3.6.14 2482================================ 2483 2484A security fix alters the :class:`ftplib.FTP` behavior to not trust the 2485IPv4 address sent from the remote server when setting up a passive data 2486channel. We reuse the ftp server IP address instead. For unusual code 2487requiring the old behavior, set a ``trust_server_pasv_ipv4_address`` 2488attribute on your FTP instance to ``True``. (See :gh:`87451`) 2489 2490The presence of newline or tab characters in parts of a URL allows for some 2491forms of attacks. Following the WHATWG specification that updates RFC 3986, 2492ASCII newline ``\n``, ``\r`` and tab ``\t`` characters are stripped from the 2493URL by the parser :func:`urllib.parse` preventing such attacks. The removal 2494characters are controlled by a new module level variable 2495``urllib.parse._UNSAFE_URL_BYTES_TO_REMOVE``. (See :gh:`88048`) 2496