• Home
  • Line#
  • Scopes#
  • Navigate#
  • Raw
  • Download
1****************************
2  What's New In Python 3.5
3****************************
4
5:Editors: Elvis Pranskevichus <elvis@magic.io>, Yury Selivanov <yury@magic.io>
6
7.. Rules for maintenance:
8
9   * Anyone can add text to this document.  Do not spend very much time
10   on the wording of your changes, because your text will probably
11   get rewritten to some degree.
12
13   * The maintainer will go through Misc/NEWS periodically and add
14   changes; it's therefore more important to add your changes to
15   Misc/NEWS than to this file.
16
17   * This is not a complete list of every single change; completeness
18   is the purpose of Misc/NEWS.  Some changes I consider too small
19   or esoteric to include.  If such a change is added to the text,
20   I'll just remove it.  (This is another reason you shouldn't spend
21   too much time on writing your addition.)
22
23   * If you want to draw your new text to the attention of the
24   maintainer, add 'XXX' to the beginning of the paragraph or
25   section.
26
27   * It's OK to just add a fragmentary note about a change.  For
28   example: "XXX Describe the transmogrify() function added to the
29   socket module."  The maintainer will research the change and
30   write the necessary text.
31
32   * You can comment out your additions if you like, but it's not
33   necessary (especially when a final release is some months away).
34
35   * Credit the author of a patch or bugfix.   Just the name is
36   sufficient; the e-mail address isn't necessary.
37
38   * It's helpful to add the bug/patch number as a comment:
39
40   XXX Describe the transmogrify() function added to the socket
41   module.
42   (Contributed by P.Y. Developer in :issue:`12345`.)
43
44   This saves the maintainer the effort of going through the Mercurial log
45   when researching a change.
46
47This article explains the new features in Python 3.5, compared to 3.4.
48Python 3.5 was released on September 13, 2015.  See the
49`changelog <https://docs.python.org/3.5/whatsnew/changelog.html>`_ for a full
50list of changes.
51
52.. seealso::
53
54    :pep:`478` - Python 3.5 Release Schedule
55
56
57Summary -- Release highlights
58=============================
59
60New syntax features:
61
62* :ref:`PEP 492 <whatsnew-pep-492>`, coroutines with async and await syntax.
63* :ref:`PEP 465 <whatsnew-pep-465>`, a new matrix multiplication operator: ``a @ b``.
64* :ref:`PEP 448 <whatsnew-pep-448>`, additional unpacking generalizations.
65
66
67New library modules:
68
69* :mod:`typing`: :ref:`PEP 484 -- Type Hints <whatsnew-pep-484>`.
70* :mod:`zipapp`: :ref:`PEP 441 Improving Python ZIP Application Support
71  <whatsnew-zipapp>`.
72
73
74New built-in features:
75
76* ``bytes % args``, ``bytearray % args``: :ref:`PEP 461 <whatsnew-pep-461>` --
77  Adding ``%`` formatting to bytes and bytearray.
78
79* New :meth:`bytes.hex`, :meth:`bytearray.hex` and :meth:`memoryview.hex`
80  methods. (Contributed by Arnon Yaari in :issue:`9951`.)
81
82* :class:`memoryview` now supports tuple indexing (including multi-dimensional).
83  (Contributed by Antoine Pitrou in :issue:`23632`.)
84
85* Generators have a new ``gi_yieldfrom`` attribute, which returns the
86  object being iterated by ``yield from`` expressions. (Contributed
87  by Benno Leslie and Yury Selivanov in :issue:`24450`.)
88
89* A new :exc:`RecursionError` exception is now raised when maximum
90  recursion depth is reached.  (Contributed by Georg Brandl
91  in :issue:`19235`.)
92
93
94CPython implementation improvements:
95
96* When the ``LC_TYPE`` locale is the POSIX locale (``C`` locale),
97  :py:data:`sys.stdin` and :py:data:`sys.stdout` now use the
98  ``surrogateescape`` error handler, instead of the ``strict`` error handler.
99  (Contributed by Victor Stinner in :issue:`19977`.)
100
101* ``.pyo`` files are no longer used and have been replaced by a more flexible
102  scheme that includes the optimization level explicitly in ``.pyc`` name.
103  (See :ref:`PEP 488 overview <whatsnew-pep-488>`.)
104
105* Builtin and extension modules are now initialized in a multi-phase process,
106  which is similar to how Python modules are loaded.
107  (See :ref:`PEP 489 overview <whatsnew-pep-489>`.)
108
109
110Significant improvements in the standard library:
111
112* :class:`collections.OrderedDict` is now
113  :ref:`implemented in C <whatsnew-ordereddict>`, which makes it
114  4 to 100 times faster.
115
116* The :mod:`ssl` module gained
117  :ref:`support for Memory BIO <whatsnew-sslmemorybio>`, which decouples SSL
118  protocol handling from network IO.
119
120* The new :func:`os.scandir` function provides a
121  :ref:`better and significantly faster way <whatsnew-pep-471>`
122  of directory traversal.
123
124* :func:`functools.lru_cache` has been mostly
125  :ref:`reimplemented in C <whatsnew-lrucache>`, yielding much better
126  performance.
127
128* The new :func:`subprocess.run` function provides a
129  :ref:`streamlined way to run subprocesses <whatsnew-subprocess>`.
130
131* The :mod:`traceback` module has been significantly
132  :ref:`enhanced <whatsnew-traceback>` for improved
133  performance and developer convenience.
134
135
136Security improvements:
137
138* SSLv3 is now disabled throughout the standard library.
139  It can still be enabled by instantiating a :class:`ssl.SSLContext`
140  manually.  (See :issue:`22638` for more details; this change was
141  backported to CPython 3.4 and 2.7.)
142
143* HTTP cookie parsing is now stricter, in order to protect
144  against potential injection attacks. (Contributed by Antoine Pitrou
145  in :issue:`22796`.)
146
147
148Windows improvements:
149
150* A new installer for Windows has replaced the old MSI.
151  See :ref:`using-on-windows` for more information.
152
153* Windows builds now use Microsoft Visual C++ 14.0, and extension modules
154  should use the same.
155
156
157Please read on for a comprehensive list of user-facing changes, including many
158other smaller improvements, CPython optimizations, deprecations, and potential
159porting issues.
160
161
162New Features
163============
164
165.. _whatsnew-pep-492:
166
167PEP 492 - Coroutines with async and await syntax
168------------------------------------------------
169
170:pep:`492` greatly improves support for asynchronous programming in Python
171by adding :term:`awaitable objects <awaitable>`,
172:term:`coroutine functions <coroutine function>`,
173:term:`asynchronous iteration <asynchronous iterable>`,
174and :term:`asynchronous context managers <asynchronous context manager>`.
175
176Coroutine functions are declared using the new :keyword:`async def` syntax::
177
178    >>> async def coro():
179    ...     return 'spam'
180
181Inside a coroutine function, the new :keyword:`await` expression can be used
182to suspend coroutine execution until the result is available.  Any object
183can be *awaited*, as long as it implements the :term:`awaitable` protocol by
184defining the :meth:`__await__` method.
185
186PEP 492 also adds :keyword:`async for` statement for convenient iteration
187over asynchronous iterables.
188
189An example of a rudimentary HTTP client written using the new syntax::
190
191    import asyncio
192
193    async def http_get(domain):
194        reader, writer = await asyncio.open_connection(domain, 80)
195
196        writer.write(b'\r\n'.join([
197            b'GET / HTTP/1.1',
198            b'Host: %b' % domain.encode('latin-1'),
199            b'Connection: close',
200            b'', b''
201        ]))
202
203        async for line in reader:
204            print('>>>', line)
205
206        writer.close()
207
208    loop = asyncio.get_event_loop()
209    try:
210        loop.run_until_complete(http_get('example.com'))
211    finally:
212        loop.close()
213
214
215Similarly to asynchronous iteration, there is a new syntax for asynchronous
216context managers.  The following script::
217
218    import asyncio
219
220    async def coro(name, lock):
221        print('coro {}: waiting for lock'.format(name))
222        async with lock:
223            print('coro {}: holding the lock'.format(name))
224            await asyncio.sleep(1)
225            print('coro {}: releasing the lock'.format(name))
226
227    loop = asyncio.get_event_loop()
228    lock = asyncio.Lock()
229    coros = asyncio.gather(coro(1, lock), coro(2, lock))
230    try:
231        loop.run_until_complete(coros)
232    finally:
233        loop.close()
234
235will output::
236
237    coro 2: waiting for lock
238    coro 2: holding the lock
239    coro 1: waiting for lock
240    coro 2: releasing the lock
241    coro 1: holding the lock
242    coro 1: releasing the lock
243
244Note that both :keyword:`async for` and :keyword:`async with` can only
245be used inside a coroutine function declared with :keyword:`async def`.
246
247Coroutine functions are intended to be run inside a compatible event loop,
248such as the :ref:`asyncio loop <asyncio-event-loop>`.
249
250
251.. note::
252
253   .. versionchanged:: 3.5.2
254      Starting with CPython 3.5.2, ``__aiter__`` can directly return
255      :term:`asynchronous iterators <asynchronous iterator>`.  Returning
256      an :term:`awaitable` object will result in a
257      :exc:`PendingDeprecationWarning`.
258
259      See more details in the :ref:`async-iterators` documentation
260      section.
261
262
263.. seealso::
264
265   :pep:`492` -- Coroutines with async and await syntax
266      PEP written and implemented by Yury Selivanov.
267
268
269.. _whatsnew-pep-465:
270
271PEP 465 - A dedicated infix operator for matrix multiplication
272--------------------------------------------------------------
273
274:pep:`465` adds the ``@`` infix operator for matrix multiplication.
275Currently, no builtin Python types implement the new operator, however, it
276can be implemented by defining :meth:`__matmul__`, :meth:`__rmatmul__`,
277and :meth:`__imatmul__` for regular, reflected, and in-place matrix
278multiplication.  The semantics of these methods is similar to that of
279methods defining other infix arithmetic operators.
280
281Matrix multiplication is a notably common operation in many fields of
282mathematics, science, engineering, and the addition of ``@`` allows writing
283cleaner code::
284
285    S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r)
286
287instead of::
288
289    S = dot((dot(H, beta) - r).T,
290            dot(inv(dot(dot(H, V), H.T)), dot(H, beta) - r))
291
292NumPy 1.10 has support for the new operator::
293
294    >>> import numpy
295
296    >>> x = numpy.ones(3)
297    >>> x
298    array([ 1., 1., 1.])
299
300    >>> m = numpy.eye(3)
301    >>> m
302    array([[ 1., 0., 0.],
303           [ 0., 1., 0.],
304           [ 0., 0., 1.]])
305
306    >>> x @ m
307    array([ 1., 1., 1.])
308
309
310.. seealso::
311
312   :pep:`465` -- A dedicated infix operator for matrix multiplication
313      PEP written by Nathaniel J. Smith; implemented by Benjamin Peterson.
314
315
316.. _whatsnew-pep-448:
317
318PEP 448 - Additional Unpacking Generalizations
319----------------------------------------------
320
321:pep:`448` extends the allowed uses of the ``*`` iterable unpacking
322operator and ``**`` dictionary unpacking operator.  It is now possible
323to use an arbitrary number of unpackings in :ref:`function calls <calls>`::
324
325    >>> print(*[1], *[2], 3, *[4, 5])
326    1 2 3 4 5
327
328    >>> def fn(a, b, c, d):
329    ...     print(a, b, c, d)
330    ...
331
332    >>> fn(**{'a': 1, 'c': 3}, **{'b': 2, 'd': 4})
333    1 2 3 4
334
335Similarly, tuple, list, set, and dictionary displays allow multiple
336unpackings (see :ref:`exprlists` and :ref:`dict`)::
337
338    >>> *range(4), 4
339    (0, 1, 2, 3, 4)
340
341    >>> [*range(4), 4]
342    [0, 1, 2, 3, 4]
343
344    >>> {*range(4), 4, *(5, 6, 7)}
345    {0, 1, 2, 3, 4, 5, 6, 7}
346
347    >>> {'x': 1, **{'y': 2}}
348    {'x': 1, 'y': 2}
349
350.. seealso::
351
352   :pep:`448` -- Additional Unpacking Generalizations
353      PEP written by Joshua Landau; implemented by Neil Girdhar,
354      Thomas Wouters, and Joshua Landau.
355
356
357.. _whatsnew-pep-461:
358
359PEP 461 - percent formatting support for bytes and bytearray
360------------------------------------------------------------
361
362:pep:`461` adds support for the ``%``
363:ref:`interpolation operator <bytes-formatting>` to :class:`bytes`
364and :class:`bytearray`.
365
366While interpolation is usually thought of as a string operation, there are
367cases where interpolation on ``bytes`` or ``bytearrays`` makes sense, and the
368work needed to make up for this missing functionality detracts from the
369overall readability of the code.  This issue is particularly important when
370dealing with wire format protocols, which are often a mixture of binary
371and ASCII compatible text.
372
373Examples::
374
375    >>> b'Hello %b!' % b'World'
376    b'Hello World!'
377
378    >>> b'x=%i y=%f' % (1, 2.5)
379    b'x=1 y=2.500000'
380
381Unicode is not allowed for ``%b``, but it is accepted by ``%a`` (equivalent of
382``repr(obj).encode('ascii', 'backslashreplace')``)::
383
384    >>> b'Hello %b!' % 'World'
385    Traceback (most recent call last):
386      File "<stdin>", line 1, in <module>
387    TypeError: %b requires bytes, or an object that implements __bytes__, not 'str'
388
389    >>> b'price: %a' % '10€'
390    b"price: '10\\u20ac'"
391
392Note that ``%s`` and ``%r`` conversion types, although supported, should
393only be used in codebases that need compatibility with Python 2.
394
395.. seealso::
396
397   :pep:`461` -- Adding % formatting to bytes and bytearray
398      PEP written by Ethan Furman; implemented by Neil Schemenauer and
399      Ethan Furman.
400
401
402.. _whatsnew-pep-484:
403
404PEP 484 - Type Hints
405--------------------
406
407Function annotation syntax has been a Python feature since version 3.0
408(:pep:`3107`), however the semantics of annotations has been left undefined.
409
410Experience has shown that the majority of function annotation
411uses were to provide type hints to function parameters and return values.  It
412became evident that it would be beneficial for Python users, if the
413standard library included the base definitions and tools for type annotations.
414
415:pep:`484` introduces a :term:`provisional module <provisional api>` to
416provide these standard definitions and tools, along with some conventions
417for situations where annotations are not available.
418
419For example, here is a simple function whose argument and return type
420are declared in the annotations::
421
422    def greeting(name: str) -> str:
423        return 'Hello ' + name
424
425While these annotations are available at runtime through the usual
426:attr:`__annotations__` attribute, *no automatic type checking happens at
427runtime*.  Instead, it is assumed that a separate off-line type checker
428(e.g. `mypy <http://mypy-lang.org>`_) will be used for on-demand
429source code analysis.
430
431The type system supports unions, generic types, and a special type
432named :class:`~typing.Any` which is consistent with (i.e. assignable to
433and from) all types.
434
435.. seealso::
436
437   * :mod:`typing` module documentation
438   * :pep:`484` -- Type Hints
439        PEP written by Guido van Rossum, Jukka Lehtosalo, and Łukasz Langa;
440        implemented by Guido van Rossum.
441   * :pep:`483` -- The Theory of Type Hints
442        PEP written by Guido van Rossum
443
444
445.. _whatsnew-pep-471:
446
447PEP 471 - os.scandir() function -- a better and faster directory iterator
448-------------------------------------------------------------------------
449
450:pep:`471` adds a new directory iteration function, :func:`os.scandir`,
451to the standard library.  Additionally, :func:`os.walk` is now
452implemented using ``scandir``, which makes it 3 to 5 times faster
453on POSIX systems and 7 to 20 times faster on Windows systems.  This is
454largely achieved by greatly reducing the number of calls to :func:`os.stat`
455required to walk a directory tree.
456
457Additionally, ``scandir`` returns an iterator, as opposed to returning
458a list of file names, which improves memory efficiency when iterating
459over very large directories.
460
461The following example shows a simple use of :func:`os.scandir` to display all
462the files (excluding directories) in the given *path* that don't start with
463``'.'``. The :meth:`entry.is_file() <os.DirEntry.is_file>` call will generally
464not make an additional system call::
465
466    for entry in os.scandir(path):
467        if not entry.name.startswith('.') and entry.is_file():
468            print(entry.name)
469
470.. seealso::
471
472   :pep:`471` -- os.scandir() function -- a better and faster directory iterator
473      PEP written and implemented by Ben Hoyt with the help of Victor Stinner.
474
475
476.. _whatsnew-pep-475:
477
478PEP 475: Retry system calls failing with EINTR
479----------------------------------------------
480
481An :py:data:`errno.EINTR` error code is returned whenever a system call, that
482is waiting for I/O, is interrupted by a signal.  Previously, Python would
483raise :exc:`InterruptedError` in such cases.  This meant that, when writing a
484Python application, the developer had two choices:
485
486#. Ignore the ``InterruptedError``.
487#. Handle the ``InterruptedError`` and attempt to restart the interrupted
488   system call at every call site.
489
490The first option makes an application fail intermittently.
491The second option adds a large amount of boilerplate that makes the
492code nearly unreadable.  Compare::
493
494    print("Hello World")
495
496and::
497
498    while True:
499        try:
500            print("Hello World")
501            break
502        except InterruptedError:
503            continue
504
505:pep:`475` implements automatic retry of system calls on
506``EINTR``.  This removes the burden of dealing with ``EINTR``
507or :exc:`InterruptedError` in user code in most situations and makes
508Python programs, including the standard library, more robust.  Note that
509the system call is only retried if the signal handler does not raise an
510exception.
511
512Below is a list of functions which are now retried when interrupted
513by a signal:
514
515* :func:`open` and :func:`io.open`;
516
517* functions of the :mod:`faulthandler` module;
518
519* :mod:`os` functions: :func:`~os.fchdir`, :func:`~os.fchmod`,
520  :func:`~os.fchown`, :func:`~os.fdatasync`, :func:`~os.fstat`,
521  :func:`~os.fstatvfs`, :func:`~os.fsync`, :func:`~os.ftruncate`,
522  :func:`~os.mkfifo`, :func:`~os.mknod`, :func:`~os.open`,
523  :func:`~os.posix_fadvise`, :func:`~os.posix_fallocate`, :func:`~os.pread`,
524  :func:`~os.pwrite`, :func:`~os.read`, :func:`~os.readv`, :func:`~os.sendfile`,
525  :func:`~os.wait3`, :func:`~os.wait4`, :func:`~os.wait`,
526  :func:`~os.waitid`, :func:`~os.waitpid`, :func:`~os.write`,
527  :func:`~os.writev`;
528
529* special cases: :func:`os.close` and :func:`os.dup2` now ignore
530  :py:data:`~errno.EINTR` errors; the syscall is not retried (see the PEP
531  for the rationale);
532
533* :mod:`select` functions: :func:`devpoll.poll() <select.devpoll.poll>`,
534  :func:`epoll.poll() <select.epoll.poll>`,
535  :func:`kqueue.control() <select.kqueue.control>`,
536  :func:`poll.poll() <select.poll.poll>`, :func:`~select.select`;
537
538* methods of the :class:`~socket.socket` class: :meth:`~socket.socket.accept`,
539  :meth:`~socket.socket.connect` (except for non-blocking sockets),
540  :meth:`~socket.socket.recv`, :meth:`~socket.socket.recvfrom`,
541  :meth:`~socket.socket.recvmsg`, :meth:`~socket.socket.send`,
542  :meth:`~socket.socket.sendall`, :meth:`~socket.socket.sendmsg`,
543  :meth:`~socket.socket.sendto`;
544
545* :func:`signal.sigtimedwait` and :func:`signal.sigwaitinfo`;
546
547* :func:`time.sleep`.
548
549.. seealso::
550
551   :pep:`475` -- Retry system calls failing with EINTR
552      PEP and implementation written by Charles-François Natali and
553      Victor Stinner, with the help of Antoine Pitrou (the French connection).
554
555
556.. _whatsnew-pep-479:
557
558PEP 479: Change StopIteration handling inside generators
559--------------------------------------------------------
560
561The interaction of generators and :exc:`StopIteration` in Python 3.4 and
562earlier was sometimes surprising, and could conceal obscure bugs.  Previously,
563``StopIteration`` raised accidentally inside a generator function was
564interpreted as the end of the iteration by the loop construct driving the
565generator.
566
567:pep:`479` changes the behavior of generators: when a ``StopIteration``
568exception is raised inside a generator, it is replaced with a
569:exc:`RuntimeError` before it exits the generator frame.  The main goal of
570this change is to ease debugging in the situation where an unguarded
571:func:`next` call raises ``StopIteration`` and causes the iteration controlled
572by the generator to terminate silently. This is particularly pernicious in
573combination with the ``yield from`` construct.
574
575This is a backwards incompatible change, so to enable the new behavior,
576a :term:`__future__` import is necessary::
577
578    >>> from __future__ import generator_stop
579
580    >>> def gen():
581    ...     next(iter([]))
582    ...     yield
583    ...
584    >>> next(gen())
585    Traceback (most recent call last):
586      File "<stdin>", line 2, in gen
587    StopIteration
588
589    The above exception was the direct cause of the following exception:
590
591    Traceback (most recent call last):
592      File "<stdin>", line 1, in <module>
593    RuntimeError: generator raised StopIteration
594
595Without a ``__future__`` import, a :exc:`PendingDeprecationWarning` will be
596raised whenever a :exc:`StopIteration` exception is raised inside a generator.
597
598.. seealso::
599
600   :pep:`479` -- Change StopIteration handling inside generators
601      PEP written by Chris Angelico and Guido van Rossum. Implemented by
602      Chris Angelico, Yury Selivanov and Nick Coghlan.
603
604
605.. _whatsnew-pep-485:
606
607PEP 485: A function for testing approximate equality
608----------------------------------------------------
609
610:pep:`485` adds the :func:`math.isclose` and :func:`cmath.isclose`
611functions which tell whether two values are approximately equal or
612"close" to each other.  Whether or not two values are considered
613close is determined according to given absolute and relative tolerances.
614Relative tolerance is the maximum allowed difference between ``isclose``
615arguments, relative to the larger absolute value::
616
617    >>> import math
618    >>> a = 5.0
619    >>> b = 4.99998
620    >>> math.isclose(a, b, rel_tol=1e-5)
621    True
622    >>> math.isclose(a, b, rel_tol=1e-6)
623    False
624
625It is also possible to compare two values using absolute tolerance, which
626must be a non-negative value::
627
628    >>> import math
629    >>> a = 5.0
630    >>> b = 4.99998
631    >>> math.isclose(a, b, abs_tol=0.00003)
632    True
633    >>> math.isclose(a, b, abs_tol=0.00001)
634    False
635
636.. seealso::
637
638   :pep:`485` -- A function for testing approximate equality
639      PEP written by Christopher Barker; implemented by Chris Barker and
640      Tal Einat.
641
642
643.. _whatsnew-pep-486:
644
645PEP 486: Make the Python Launcher aware of virtual environments
646---------------------------------------------------------------
647
648:pep:`486` makes the Windows launcher (see :pep:`397`) aware of an active
649virtual environment. When the default interpreter would be used and the
650``VIRTUAL_ENV`` environment variable is set, the interpreter in the virtual
651environment will be used.
652
653.. seealso::
654
655    :pep:`486` -- Make the Python Launcher aware of virtual environments
656        PEP written and implemented by Paul Moore.
657
658
659.. _whatsnew-pep-488:
660
661PEP 488: Elimination of PYO files
662---------------------------------
663
664:pep:`488` does away with the concept of ``.pyo`` files. This means that
665``.pyc`` files represent both unoptimized and optimized bytecode. To prevent the
666need to constantly regenerate bytecode files, ``.pyc`` files now have an
667optional ``opt-`` tag in their name when the bytecode is optimized. This has the
668side-effect of no more bytecode file name clashes when running under either
669:option:`-O` or :option:`-OO`. Consequently, bytecode files generated from
670:option:`-O`, and :option:`-OO` may now exist simultaneously.
671:func:`importlib.util.cache_from_source` has an updated API to help with
672this change.
673
674.. seealso::
675
676   :pep:`488` -- Elimination of PYO files
677      PEP written and implemented by Brett Cannon.
678
679
680.. _whatsnew-pep-489:
681
682PEP 489: Multi-phase extension module initialization
683----------------------------------------------------
684
685:pep:`489` updates extension module initialization to take advantage of the
686two step module loading mechanism introduced by :pep:`451` in Python 3.4.
687
688This change brings the import semantics of extension modules that opt-in to
689using the new mechanism much closer to those of Python source and bytecode
690modules, including the ability to use any valid identifier as a module name,
691rather than being restricted to ASCII.
692
693.. seealso::
694
695   :pep:`489` -- Multi-phase extension module initialization
696      PEP written by Petr Viktorin, Stefan Behnel, and Nick Coghlan;
697      implemented by Petr Viktorin.
698
699
700Other Language Changes
701======================
702
703Some smaller changes made to the core Python language are:
704
705* Added the ``"namereplace"`` error handlers.  The ``"backslashreplace"``
706  error handlers now work with decoding and translating.
707  (Contributed by Serhiy Storchaka in :issue:`19676` and :issue:`22286`.)
708
709* The :option:`-b` option now affects comparisons of :class:`bytes` with
710  :class:`int`.  (Contributed by Serhiy Storchaka in :issue:`23681`.)
711
712* New Kazakh ``kz1048`` and Tajik ``koi8_t`` :ref:`codecs <standard-encodings>`.
713  (Contributed by Serhiy Storchaka in :issue:`22682` and :issue:`22681`.)
714
715* Property docstrings are now writable. This is especially useful for
716  :func:`collections.namedtuple` docstrings.
717  (Contributed by Berker Peksag in :issue:`24064`.)
718
719* Circular imports involving relative imports are now supported.
720  (Contributed by Brett Cannon and Antoine Pitrou in :issue:`17636`.)
721
722
723New Modules
724===========
725
726typing
727------
728
729The new :mod:`typing` :term:`provisional <provisional api>` module
730provides standard definitions and tools for function type annotations.
731See :ref:`Type Hints <whatsnew-pep-484>` for more information.
732
733.. _whatsnew-zipapp:
734
735zipapp
736------
737
738The new :mod:`zipapp` module (specified in :pep:`441`) provides an API and
739command line tool for creating executable Python Zip Applications, which
740were introduced in Python 2.6 in :issue:`1739468`, but which were not well
741publicized, either at the time or since.
742
743With the new module, bundling your application is as simple as putting all
744the files, including a ``__main__.py`` file, into a directory ``myapp``
745and running:
746
747.. code-block:: shell-session
748
749    $ python -m zipapp myapp
750    $ python myapp.pyz
751
752The module implementation has been contributed by Paul Moore in
753:issue:`23491`.
754
755.. seealso::
756
757   :pep:`441` -- Improving Python ZIP Application Support
758
759
760Improved Modules
761================
762
763argparse
764--------
765
766The :class:`~argparse.ArgumentParser` class now allows disabling
767:ref:`abbreviated usage <prefix-matching>` of long options by setting
768:ref:`allow_abbrev` to ``False``.  (Contributed by Jonathan Paugh,
769Steven Bethard, paul j3 and Daniel Eriksson in :issue:`14910`.)
770
771
772asyncio
773-------
774
775Since the :mod:`asyncio` module is :term:`provisional <provisional api>`,
776all changes introduced in Python 3.5 have also been backported to Python 3.4.x.
777
778Notable changes in the :mod:`asyncio` module since Python 3.4.0:
779
780* New debugging APIs: :meth:`loop.set_debug() <asyncio.loop.set_debug>`
781  and :meth:`loop.get_debug() <asyncio.loop.get_debug>` methods.
782  (Contributed by Victor Stinner.)
783
784* The proactor event loop now supports SSL.
785  (Contributed by Antoine Pitrou and Victor Stinner in :issue:`22560`.)
786
787* A new :meth:`loop.is_closed() <asyncio.loop.is_closed>` method to
788  check if the event loop is closed.
789  (Contributed by Victor Stinner in :issue:`21326`.)
790
791* A new :meth:`loop.create_task() <asyncio.loop.create_task>`
792  to conveniently create and schedule a new :class:`~asyncio.Task`
793  for a coroutine.  The ``create_task`` method is also used by all
794  asyncio functions that wrap coroutines into tasks, such as
795  :func:`asyncio.wait`, :func:`asyncio.gather`, etc.
796  (Contributed by Victor Stinner.)
797
798* A new :meth:`transport.get_write_buffer_limits() <asyncio.WriteTransport.get_write_buffer_limits>`
799  method to inquire for *high-* and *low-* water limits of the flow
800  control.
801  (Contributed by Victor Stinner.)
802
803* The :func:`~asyncio.async` function is deprecated in favor of
804  :func:`~asyncio.ensure_future`.
805  (Contributed by Yury Selivanov.)
806
807* New :meth:`loop.set_task_factory()
808  <asyncio.loop.set_task_factory>` and
809  :meth:`loop.get_task_factory() <asyncio.loop.get_task_factory>`
810  methods to customize the task factory that :meth:`loop.create_task()
811  <asyncio.loop.create_task>` method uses.  (Contributed by Yury
812  Selivanov.)
813
814* New :meth:`Queue.join() <asyncio.Queue.join>` and
815  :meth:`Queue.task_done() <asyncio.Queue.task_done>` queue methods.
816  (Contributed by Victor Stinner.)
817
818* The ``JoinableQueue`` class was removed, in favor of the
819  :class:`asyncio.Queue` class.
820  (Contributed by Victor Stinner.)
821
822Updates in 3.5.1:
823
824* The :func:`~asyncio.ensure_future` function and all functions that
825  use it, such as :meth:`loop.run_until_complete() <asyncio.loop.run_until_complete>`,
826  now accept all kinds of :term:`awaitable objects <awaitable>`.
827  (Contributed by Yury Selivanov.)
828
829* New :func:`~asyncio.run_coroutine_threadsafe` function to submit
830  coroutines to event loops from other threads.
831  (Contributed by Vincent Michel.)
832
833* New :meth:`Transport.is_closing() <asyncio.BaseTransport.is_closing>`
834  method to check if the transport is closing or closed.
835  (Contributed by Yury Selivanov.)
836
837* The :meth:`loop.create_server() <asyncio.loop.create_server>`
838  method can now accept a list of hosts.
839  (Contributed by Yann Sionneau.)
840
841Updates in 3.5.2:
842
843* New :meth:`loop.create_future() <asyncio.loop.create_future>`
844  method to create Future objects.  This allows alternative event
845  loop implementations, such as
846  `uvloop <https://github.com/MagicStack/uvloop>`_, to provide a faster
847  :class:`asyncio.Future` implementation.
848  (Contributed by Yury Selivanov.)
849
850* New :meth:`loop.get_exception_handler() <asyncio.loop.get_exception_handler>`
851  method to get the current exception handler.
852  (Contributed by Yury Selivanov.)
853
854* New :meth:`StreamReader.readuntil() <asyncio.StreamReader.readuntil>`
855  method to read data from the stream until a separator bytes
856  sequence appears.
857  (Contributed by Mark Korenberg.)
858
859* The :meth:`loop.create_connection() <asyncio.loop.create_connection>`
860  and :meth:`loop.create_server() <asyncio.loop.create_server>`
861  methods are optimized to avoid calling the system ``getaddrinfo``
862  function if the address is already resolved.
863  (Contributed by A. Jesse Jiryu Davis.)
864
865* The :meth:`loop.sock_connect(sock, address) <asyncio.loop.sock_connect>`
866  no longer requires the *address* to be resolved prior to the call.
867  (Contributed by A. Jesse Jiryu Davis.)
868
869
870bz2
871---
872
873The :meth:`BZ2Decompressor.decompress <bz2.BZ2Decompressor.decompress>`
874method now accepts an optional *max_length* argument to limit the maximum
875size of decompressed data. (Contributed by Nikolaus Rath in :issue:`15955`.)
876
877
878cgi
879---
880
881The :class:`~cgi.FieldStorage` class now supports the :term:`context manager`
882protocol.  (Contributed by Berker Peksag in :issue:`20289`.)
883
884
885cmath
886-----
887
888A new function :func:`~cmath.isclose` provides a way to test for approximate
889equality.  (Contributed by Chris Barker and Tal Einat in :issue:`24270`.)
890
891
892code
893----
894
895The :func:`InteractiveInterpreter.showtraceback() <code.InteractiveInterpreter.showtraceback>`
896method now prints the full chained traceback, just like the interactive
897interpreter.  (Contributed by Claudiu Popa in :issue:`17442`.)
898
899
900collections
901-----------
902
903.. _whatsnew-ordereddict:
904
905The :class:`~collections.OrderedDict` class is now implemented in C, which
906makes it 4 to 100 times faster.  (Contributed by Eric Snow in :issue:`16991`.)
907
908:meth:`OrderedDict.items() <collections.OrderedDict.items>`,
909:meth:`OrderedDict.keys() <collections.OrderedDict.keys>`,
910:meth:`OrderedDict.values() <collections.OrderedDict.values>` views now support
911:func:`reversed` iteration.
912(Contributed by Serhiy Storchaka in :issue:`19505`.)
913
914The :class:`~collections.deque` class now defines
915:meth:`~collections.deque.index`, :meth:`~collections.deque.insert`, and
916:meth:`~collections.deque.copy`, and supports the ``+`` and ``*`` operators.
917This allows deques to be recognized as a :class:`~collections.abc.MutableSequence`
918and improves their substitutability for lists.
919(Contributed by Raymond Hettinger in :issue:`23704`.)
920
921Docstrings produced by :func:`~collections.namedtuple` can now be updated::
922
923    Point = namedtuple('Point', ['x', 'y'])
924    Point.__doc__ += ': Cartesian coodinate'
925    Point.x.__doc__ = 'abscissa'
926    Point.y.__doc__ = 'ordinate'
927
928(Contributed by Berker Peksag in :issue:`24064`.)
929
930The :class:`~collections.UserString` class now implements the
931:meth:`__getnewargs__`, :meth:`__rmod__`, :meth:`~str.casefold`,
932:meth:`~str.format_map`, :meth:`~str.isprintable`, and :meth:`~str.maketrans`
933methods to match the corresponding methods of :class:`str`.
934(Contributed by Joe Jevnik in :issue:`22189`.)
935
936
937collections.abc
938---------------
939
940The :meth:`Sequence.index() <collections.abc.Sequence.index>` method now
941accepts *start* and *stop* arguments to match the corresponding methods
942of :class:`tuple`, :class:`list`, etc.
943(Contributed by Devin Jeanpierre in :issue:`23086`.)
944
945A new :class:`~collections.abc.Generator` abstract base class. (Contributed
946by Stefan Behnel in :issue:`24018`.)
947
948New :class:`~collections.abc.Awaitable`, :class:`~collections.abc.Coroutine`,
949:class:`~collections.abc.AsyncIterator`, and
950:class:`~collections.abc.AsyncIterable` abstract base classes.
951(Contributed by Yury Selivanov in :issue:`24184`.)
952
953For earlier Python versions, a backport of the new ABCs is available in an
954external `PyPI package <https://pypi.org/project/backports_abc>`_.
955
956
957compileall
958----------
959
960A new :mod:`compileall` option, :samp:`-j {N}`, allows running *N* workers
961simultaneously to perform parallel bytecode compilation.
962The :func:`~compileall.compile_dir` function has a corresponding ``workers``
963parameter.  (Contributed by Claudiu Popa in :issue:`16104`.)
964
965Another new option, ``-r``, allows controlling the maximum recursion
966level for subdirectories.  (Contributed by Claudiu Popa in :issue:`19628`.)
967
968The ``-q`` command line option can now be specified more than once, in
969which case all output, including errors, will be suppressed.  The corresponding
970``quiet`` parameter in :func:`~compileall.compile_dir`,
971:func:`~compileall.compile_file`, and :func:`~compileall.compile_path` can now
972accept an integer value indicating the level of output suppression.
973(Contributed by Thomas Kluyver in :issue:`21338`.)
974
975
976concurrent.futures
977------------------
978
979The :meth:`Executor.map() <concurrent.futures.Executor.map>` method now accepts a
980*chunksize* argument to allow batching of tasks to improve performance when
981:meth:`~concurrent.futures.ProcessPoolExecutor` is used.
982(Contributed by Dan O'Reilly in :issue:`11271`.)
983
984The number of workers in the :class:`~concurrent.futures.ThreadPoolExecutor`
985constructor is optional now.  The default value is 5 times the number of CPUs.
986(Contributed by Claudiu Popa in :issue:`21527`.)
987
988
989configparser
990------------
991
992:mod:`configparser` now provides a way to customize the conversion
993of values by specifying a dictionary of converters in the
994:class:`~configparser.ConfigParser` constructor, or by defining them
995as methods in ``ConfigParser`` subclasses.  Converters defined in
996a parser instance are inherited by its section proxies.
997
998Example::
999
1000    >>> import configparser
1001    >>> conv = {}
1002    >>> conv['list'] = lambda v: [e.strip() for e in v.split() if e.strip()]
1003    >>> cfg = configparser.ConfigParser(converters=conv)
1004    >>> cfg.read_string("""
1005    ... [s]
1006    ... list = a b c d e f g
1007    ... """)
1008    >>> cfg.get('s', 'list')
1009    'a b c d e f g'
1010    >>> cfg.getlist('s', 'list')
1011    ['a', 'b', 'c', 'd', 'e', 'f', 'g']
1012    >>> section = cfg['s']
1013    >>> section.getlist('list')
1014    ['a', 'b', 'c', 'd', 'e', 'f', 'g']
1015
1016(Contributed by Łukasz Langa in :issue:`18159`.)
1017
1018
1019contextlib
1020----------
1021
1022The new :func:`~contextlib.redirect_stderr` :term:`context manager` (similar to
1023:func:`~contextlib.redirect_stdout`) makes it easier for utility scripts to
1024handle inflexible APIs that write their output to :data:`sys.stderr` and
1025don't provide any options to redirect it::
1026
1027    >>> import contextlib, io, logging
1028    >>> f = io.StringIO()
1029    >>> with contextlib.redirect_stderr(f):
1030    ...     logging.warning('warning')
1031    ...
1032    >>> f.getvalue()
1033    'WARNING:root:warning\n'
1034
1035(Contributed by Berker Peksag in :issue:`22389`.)
1036
1037
1038csv
1039---
1040
1041The :meth:`~csv.csvwriter.writerow` method now supports arbitrary iterables,
1042not just sequences.  (Contributed by Serhiy Storchaka in :issue:`23171`.)
1043
1044
1045curses
1046------
1047
1048The new :func:`~curses.update_lines_cols` function updates the :envvar:`LINES`
1049and :envvar:`COLS` environment variables.  This is useful for detecting
1050manual screen resizing.  (Contributed by Arnon Yaari in :issue:`4254`.)
1051
1052
1053dbm
1054---
1055
1056:func:`dumb.open <dbm.dumb.open>` always creates a new database when the flag
1057has the value ``"n"``.  (Contributed by Claudiu Popa in :issue:`18039`.)
1058
1059
1060difflib
1061-------
1062
1063The charset of HTML documents generated by
1064:meth:`HtmlDiff.make_file() <difflib.HtmlDiff.make_file>`
1065can now be customized by using a new *charset* keyword-only argument.
1066The default charset of HTML document changed from ``"ISO-8859-1"``
1067to ``"utf-8"``.
1068(Contributed by Berker Peksag in :issue:`2052`.)
1069
1070The :func:`~difflib.diff_bytes` function can now compare lists of byte
1071strings.  This fixes a regression from Python 2.
1072(Contributed by Terry J. Reedy and Greg Ward in :issue:`17445`.)
1073
1074
1075distutils
1076---------
1077
1078Both the ``build`` and ``build_ext`` commands now accept a ``-j`` option to
1079enable parallel building of extension modules.
1080(Contributed by Antoine Pitrou in :issue:`5309`.)
1081
1082The :mod:`distutils` module now supports ``xz`` compression, and can be
1083enabled by passing ``xztar`` as an argument to ``bdist --format``.
1084(Contributed by Serhiy Storchaka in :issue:`16314`.)
1085
1086
1087doctest
1088-------
1089
1090The :func:`~doctest.DocTestSuite` function returns an empty
1091:class:`unittest.TestSuite` if *module* contains no docstrings, instead of
1092raising :exc:`ValueError`.  (Contributed by Glenn Jones in :issue:`15916`.)
1093
1094
1095email
1096-----
1097
1098A new policy option :attr:`Policy.mangle_from_ <email.policy.Policy.mangle_from_>`
1099controls whether or not lines that start with ``"From "`` in email bodies are
1100prefixed with a ``">"`` character by generators.  The default is ``True`` for
1101:attr:`~email.policy.compat32` and ``False`` for all other policies.
1102(Contributed by Milan Oberkirch in :issue:`20098`.)
1103
1104A new
1105:meth:`Message.get_content_disposition() <email.message.Message.get_content_disposition>`
1106method provides easy access to a canonical value for the
1107:mailheader:`Content-Disposition` header.
1108(Contributed by Abhilash Raj in :issue:`21083`.)
1109
1110A new policy option :attr:`EmailPolicy.utf8 <email.policy.EmailPolicy.utf8>`
1111can be set to ``True`` to encode email headers using the UTF-8 charset instead
1112of using encoded words.  This allows ``Messages`` to be formatted according to
1113:rfc:`6532` and used with an SMTP server that supports the :rfc:`6531`
1114``SMTPUTF8`` extension.  (Contributed by R. David Murray in
1115:issue:`24211`.)
1116
1117The :class:`mime.text.MIMEText <email.mime.text.MIMEText>` constructor now
1118accepts a :class:`charset.Charset <email.charset.Charset>` instance.
1119(Contributed by Claude Paroz and Berker Peksag in :issue:`16324`.)
1120
1121
1122enum
1123----
1124
1125The :class:`~enum.Enum` callable has a new parameter *start* to
1126specify the initial number of enum values if only *names* are provided::
1127
1128    >>> Animal = enum.Enum('Animal', 'cat dog', start=10)
1129    >>> Animal.cat
1130    <Animal.cat: 10>
1131    >>> Animal.dog
1132    <Animal.dog: 11>
1133
1134(Contributed by Ethan Furman in :issue:`21706`.)
1135
1136
1137faulthandler
1138------------
1139
1140The :func:`~faulthandler.enable`, :func:`~faulthandler.register`,
1141:func:`~faulthandler.dump_traceback` and
1142:func:`~faulthandler.dump_traceback_later` functions now accept file
1143descriptors in addition to file-like objects.
1144(Contributed by Wei Wu in :issue:`23566`.)
1145
1146
1147functools
1148---------
1149
1150.. _whatsnew-lrucache:
1151
1152Most of the :func:`~functools.lru_cache` machinery is now implemented in C, making
1153it significantly faster.  (Contributed by Matt Joiner, Alexey Kachayev, and
1154Serhiy Storchaka in :issue:`14373`.)
1155
1156
1157glob
1158----
1159
1160The :func:`~glob.iglob` and :func:`~glob.glob` functions now support recursive
1161search in subdirectories, using the ``"**"`` pattern.
1162(Contributed by Serhiy Storchaka in :issue:`13968`.)
1163
1164
1165gzip
1166----
1167
1168The *mode* argument of the :class:`~gzip.GzipFile` constructor now
1169accepts ``"x"`` to request exclusive creation.
1170(Contributed by Tim Heaney in :issue:`19222`.)
1171
1172
1173heapq
1174-----
1175
1176Element comparison in :func:`~heapq.merge` can now be customized by
1177passing a :term:`key function` in a new optional *key* keyword argument,
1178and a new optional *reverse* keyword argument can be used to reverse element
1179comparison::
1180
1181    >>> import heapq
1182    >>> a = ['9', '777', '55555']
1183    >>> b = ['88', '6666']
1184    >>> list(heapq.merge(a, b, key=len))
1185    ['9', '88', '777', '6666', '55555']
1186    >>> list(heapq.merge(reversed(a), reversed(b), key=len, reverse=True))
1187    ['55555', '6666', '777', '88', '9']
1188
1189(Contributed by Raymond Hettinger in :issue:`13742`.)
1190
1191
1192http
1193----
1194
1195A new :class:`HTTPStatus <http.HTTPStatus>` enum that defines a set of
1196HTTP status codes, reason phrases and long descriptions written in English.
1197(Contributed by Demian Brecht in :issue:`21793`.)
1198
1199
1200http.client
1201-----------
1202
1203:meth:`HTTPConnection.getresponse() <http.client.HTTPConnection.getresponse>`
1204now raises a :exc:`~http.client.RemoteDisconnected` exception when a
1205remote server connection is closed unexpectedly.  Additionally, if a
1206:exc:`ConnectionError` (of which ``RemoteDisconnected``
1207is a subclass) is raised, the client socket is now closed automatically,
1208and will reconnect on the next request::
1209
1210    import http.client
1211    conn = http.client.HTTPConnection('www.python.org')
1212    for retries in range(3):
1213        try:
1214            conn.request('GET', '/')
1215            resp = conn.getresponse()
1216        except http.client.RemoteDisconnected:
1217            pass
1218
1219(Contributed by Martin Panter in :issue:`3566`.)
1220
1221
1222idlelib and IDLE
1223----------------
1224
1225Since idlelib implements the IDLE shell and editor and is not intended for
1226import by other programs, it gets improvements with every release.  See
1227:file:`Lib/idlelib/NEWS.txt` for a cumulative list of changes since 3.4.0,
1228as well as changes made in future 3.5.x releases. This file is also available
1229from the IDLE :menuselection:`Help --> About IDLE` dialog.
1230
1231
1232imaplib
1233-------
1234
1235The :class:`~imaplib.IMAP4` class now supports the :term:`context manager` protocol.
1236When used in a :keyword:`with` statement, the IMAP4 ``LOGOUT``
1237command will be called automatically at the end of the block.
1238(Contributed by Tarek Ziadé and Serhiy Storchaka in :issue:`4972`.)
1239
1240The :mod:`imaplib` module now supports :rfc:`5161` (ENABLE Extension)
1241and :rfc:`6855` (UTF-8 Support) via the :meth:`IMAP4.enable() <imaplib.IMAP4.enable>`
1242method.  A new :attr:`IMAP4.utf8_enabled <imaplib.IMAP4.utf8_enabled>`
1243attribute tracks whether or not :rfc:`6855` support is enabled.
1244(Contributed by Milan Oberkirch, R. David Murray, and Maciej Szulik in
1245:issue:`21800`.)
1246
1247The :mod:`imaplib` module now automatically encodes non-ASCII string usernames
1248and passwords using UTF-8, as recommended by the RFCs.  (Contributed by Milan
1249Oberkirch in :issue:`21800`.)
1250
1251
1252imghdr
1253------
1254
1255The :func:`~imghdr.what` function now recognizes the
1256`OpenEXR <http://www.openexr.com>`_ format
1257(contributed by Martin Vignali and Claudiu Popa in :issue:`20295`),
1258and the `WebP <https://en.wikipedia.org/wiki/WebP>`_ format
1259(contributed by Fabrice Aneche and Claudiu Popa in :issue:`20197`.)
1260
1261
1262importlib
1263---------
1264
1265The :class:`util.LazyLoader <importlib.util.LazyLoader>` class allows for
1266lazy loading of modules in applications where startup time is important.
1267(Contributed by Brett Cannon in :issue:`17621`.)
1268
1269The :func:`abc.InspectLoader.source_to_code() <importlib.abc.InspectLoader.source_to_code>`
1270method is now a static method.  This makes it easier to initialize a module
1271object with code compiled from a string by running
1272``exec(code, module.__dict__)``.
1273(Contributed by Brett Cannon in :issue:`21156`.)
1274
1275The new :func:`util.module_from_spec() <importlib.util.module_from_spec>`
1276function is now the preferred way to create a new module.  As opposed to
1277creating a :class:`types.ModuleType` instance directly, this new function
1278will set the various import-controlled attributes based on the passed-in
1279spec object.  (Contributed by Brett Cannon in :issue:`20383`.)
1280
1281
1282inspect
1283-------
1284
1285Both the :class:`~inspect.Signature` and :class:`~inspect.Parameter` classes are
1286now picklable and hashable.  (Contributed by Yury Selivanov in :issue:`20726`
1287and :issue:`20334`.)
1288
1289A new
1290:meth:`BoundArguments.apply_defaults() <inspect.BoundArguments.apply_defaults>`
1291method provides a way to set default values for missing arguments::
1292
1293    >>> def foo(a, b='ham', *args): pass
1294    >>> ba = inspect.signature(foo).bind('spam')
1295    >>> ba.apply_defaults()
1296    >>> ba.arguments
1297    OrderedDict([('a', 'spam'), ('b', 'ham'), ('args', ())])
1298
1299(Contributed by Yury Selivanov in :issue:`24190`.)
1300
1301A new class method
1302:meth:`Signature.from_callable() <inspect.Signature.from_callable>` makes
1303subclassing of :class:`~inspect.Signature` easier.  (Contributed
1304by Yury Selivanov and Eric Snow in :issue:`17373`.)
1305
1306The :func:`~inspect.signature` function now accepts a *follow_wrapped*
1307optional keyword argument, which, when set to ``False``, disables automatic
1308following of ``__wrapped__`` links.
1309(Contributed by Yury Selivanov in :issue:`20691`.)
1310
1311A set of new functions to inspect
1312:term:`coroutine functions <coroutine function>` and
1313:term:`coroutine objects <coroutine>` has been added:
1314:func:`~inspect.iscoroutine`, :func:`~inspect.iscoroutinefunction`,
1315:func:`~inspect.isawaitable`, :func:`~inspect.getcoroutinelocals`,
1316and :func:`~inspect.getcoroutinestate`.
1317(Contributed by Yury Selivanov in :issue:`24017` and :issue:`24400`.)
1318
1319The :func:`~inspect.stack`, :func:`~inspect.trace`,
1320:func:`~inspect.getouterframes`, and :func:`~inspect.getinnerframes`
1321functions now return a list of named tuples.
1322(Contributed by Daniel Shahaf in :issue:`16808`.)
1323
1324
1325io
1326--
1327
1328A new :meth:`BufferedIOBase.readinto1() <io.BufferedIOBase.readinto1>`
1329method, that uses at most one call to the underlying raw stream's
1330:meth:`RawIOBase.read() <io.RawIOBase.read>` or
1331:meth:`RawIOBase.readinto() <io.RawIOBase.readinto>` methods.
1332(Contributed by Nikolaus Rath in :issue:`20578`.)
1333
1334
1335ipaddress
1336---------
1337
1338Both the :class:`~ipaddress.IPv4Network` and :class:`~ipaddress.IPv6Network` classes
1339now accept an ``(address, netmask)`` tuple argument, so as to easily construct
1340network objects from existing addresses::
1341
1342    >>> import ipaddress
1343    >>> ipaddress.IPv4Network(('127.0.0.0', 8))
1344    IPv4Network('127.0.0.0/8')
1345    >>> ipaddress.IPv4Network(('127.0.0.0', '255.0.0.0'))
1346    IPv4Network('127.0.0.0/8')
1347
1348(Contributed by Peter Moody and Antoine Pitrou in :issue:`16531`.)
1349
1350A new :attr:`~ipaddress.IPv4Network.reverse_pointer` attribute for the
1351:class:`~ipaddress.IPv4Network` and :class:`~ipaddress.IPv6Network` classes
1352returns the name of the reverse DNS PTR record::
1353
1354    >>> import ipaddress
1355    >>> addr = ipaddress.IPv4Address('127.0.0.1')
1356    >>> addr.reverse_pointer
1357    '1.0.0.127.in-addr.arpa'
1358    >>> addr6 = ipaddress.IPv6Address('::1')
1359    >>> addr6.reverse_pointer
1360    '1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa'
1361
1362(Contributed by Leon Weber in :issue:`20480`.)
1363
1364
1365json
1366----
1367
1368The :mod:`json.tool` command line interface now preserves the order of keys in
1369JSON objects passed in input.  The new ``--sort-keys`` option can be used
1370to sort the keys alphabetically. (Contributed by Berker Peksag
1371in :issue:`21650`.)
1372
1373JSON decoder now raises :exc:`~json.JSONDecodeError` instead of
1374:exc:`ValueError` to provide better context information about the error.
1375(Contributed by Serhiy Storchaka in :issue:`19361`.)
1376
1377
1378linecache
1379---------
1380
1381A new :func:`~linecache.lazycache` function can be used to capture information
1382about a non-file-based module to permit getting its lines later via
1383:func:`~linecache.getline`. This avoids doing I/O until a line is actually
1384needed, without having to carry the module globals around indefinitely.
1385(Contributed by Robert Collins in :issue:`17911`.)
1386
1387
1388locale
1389------
1390
1391A new :func:`~locale.delocalize` function can be used to convert a string into
1392a normalized number string, taking the ``LC_NUMERIC`` settings into account::
1393
1394    >>> import locale
1395    >>> locale.setlocale(locale.LC_NUMERIC, 'de_DE.UTF-8')
1396    'de_DE.UTF-8'
1397    >>> locale.delocalize('1.234,56')
1398    '1234.56'
1399    >>> locale.setlocale(locale.LC_NUMERIC, 'en_US.UTF-8')
1400    'en_US.UTF-8'
1401    >>> locale.delocalize('1,234.56')
1402    '1234.56'
1403
1404(Contributed by Cédric Krier in :issue:`13918`.)
1405
1406
1407logging
1408-------
1409
1410All logging methods (:class:`~logging.Logger` :meth:`~logging.Logger.log`,
1411:meth:`~logging.Logger.exception`, :meth:`~logging.Logger.critical`,
1412:meth:`~logging.Logger.debug`, etc.), now accept exception instances
1413as an *exc_info* argument, in addition to boolean values and exception
1414tuples::
1415
1416    >>> import logging
1417    >>> try:
1418    ...     1/0
1419    ... except ZeroDivisionError as ex:
1420    ...     logging.error('exception', exc_info=ex)
1421    ERROR:root:exception
1422
1423(Contributed by Yury Selivanov in :issue:`20537`.)
1424
1425The :class:`handlers.HTTPHandler <logging.handlers.HTTPHandler>` class now
1426accepts an optional :class:`ssl.SSLContext` instance to configure SSL
1427settings used in an HTTP connection.
1428(Contributed by Alex Gaynor in :issue:`22788`.)
1429
1430The :class:`handlers.QueueListener <logging.handlers.QueueListener>` class now
1431takes a *respect_handler_level* keyword argument which, if set to ``True``,
1432will pass messages to handlers taking handler levels into account.
1433(Contributed by Vinay Sajip.)
1434
1435
1436lzma
1437----
1438
1439The :meth:`LZMADecompressor.decompress() <lzma.LZMADecompressor.decompress>`
1440method now accepts an optional *max_length* argument to limit the maximum
1441size of decompressed data.
1442(Contributed by Martin Panter in :issue:`15955`.)
1443
1444
1445math
1446----
1447
1448Two new constants have been added to the :mod:`math` module: :data:`~math.inf`
1449and :data:`~math.nan`.  (Contributed by Mark Dickinson in :issue:`23185`.)
1450
1451A new function :func:`~math.isclose` provides a way to test for approximate
1452equality. (Contributed by Chris Barker and Tal Einat in :issue:`24270`.)
1453
1454A new :func:`~math.gcd` function has been added.  The :func:`fractions.gcd`
1455function is now deprecated. (Contributed by Mark Dickinson and Serhiy
1456Storchaka in :issue:`22486`.)
1457
1458
1459multiprocessing
1460---------------
1461
1462:func:`sharedctypes.synchronized() <multiprocessing.sharedctypes.synchronized>`
1463objects now support the :term:`context manager` protocol.
1464(Contributed by Charles-François Natali in :issue:`21565`.)
1465
1466
1467operator
1468--------
1469
1470:func:`~operator.attrgetter`, :func:`~operator.itemgetter`,
1471and :func:`~operator.methodcaller` objects now support pickling.
1472(Contributed by Josh Rosenberg and Serhiy Storchaka in :issue:`22955`.)
1473
1474New :func:`~operator.matmul` and :func:`~operator.imatmul` functions
1475to perform matrix multiplication.
1476(Contributed by Benjamin Peterson in :issue:`21176`.)
1477
1478
1479os
1480--
1481
1482The new :func:`~os.scandir` function returning an iterator of
1483:class:`~os.DirEntry` objects has been added.  If possible, :func:`~os.scandir`
1484extracts file attributes while scanning a directory, removing the need to
1485perform subsequent system calls to determine file type or attributes, which may
1486significantly improve performance.  (Contributed by Ben Hoyt with the help
1487of Victor Stinner in :issue:`22524`.)
1488
1489On Windows, a new
1490:attr:`stat_result.st_file_attributes <os.stat_result.st_file_attributes>`
1491attribute is now available.  It corresponds to the ``dwFileAttributes`` member
1492of the ``BY_HANDLE_FILE_INFORMATION`` structure returned by
1493``GetFileInformationByHandle()``.  (Contributed by Ben Hoyt in :issue:`21719`.)
1494
1495The :func:`~os.urandom` function now uses the ``getrandom()`` syscall on Linux 3.17
1496or newer, and ``getentropy()`` on OpenBSD 5.6 and newer, removing the need to
1497use ``/dev/urandom`` and avoiding failures due to potential file descriptor
1498exhaustion.  (Contributed by Victor Stinner in :issue:`22181`.)
1499
1500New :func:`~os.get_blocking` and :func:`~os.set_blocking` functions allow
1501getting and setting a file descriptor's blocking mode (:data:`~os.O_NONBLOCK`.)
1502(Contributed by Victor Stinner in :issue:`22054`.)
1503
1504The :func:`~os.truncate` and :func:`~os.ftruncate` functions are now supported
1505on Windows.  (Contributed by Steve Dower in :issue:`23668`.)
1506
1507There is a new :func:`os.path.commonpath` function returning the longest
1508common sub-path of each passed pathname.  Unlike the
1509:func:`os.path.commonprefix` function, it always returns a valid
1510path::
1511
1512    >>> os.path.commonprefix(['/usr/lib', '/usr/local/lib'])
1513    '/usr/l'
1514
1515    >>> os.path.commonpath(['/usr/lib', '/usr/local/lib'])
1516    '/usr'
1517
1518(Contributed by Rafik Draoui and Serhiy Storchaka in :issue:`10395`.)
1519
1520
1521pathlib
1522-------
1523
1524The new :meth:`Path.samefile() <pathlib.Path.samefile>` method can be used
1525to check whether the path points to the same file as another path, which can
1526be either another :class:`~pathlib.Path` object, or a string::
1527
1528    >>> import pathlib
1529    >>> p1 = pathlib.Path('/etc/hosts')
1530    >>> p2 = pathlib.Path('/etc/../etc/hosts')
1531    >>> p1.samefile(p2)
1532    True
1533
1534(Contributed by Vajrasky Kok and Antoine Pitrou in :issue:`19775`.)
1535
1536The :meth:`Path.mkdir() <pathlib.Path.mkdir>` method now accepts a new optional
1537*exist_ok* argument to match ``mkdir -p`` and :func:`os.makedirs`
1538functionality.  (Contributed by Berker Peksag in :issue:`21539`.)
1539
1540There is a new :meth:`Path.expanduser() <pathlib.Path.expanduser>` method to
1541expand ``~`` and ``~user`` prefixes.  (Contributed by Serhiy Storchaka and
1542Claudiu Popa in :issue:`19776`.)
1543
1544A new :meth:`Path.home() <pathlib.Path.home>` class method can be used to get
1545a :class:`~pathlib.Path` instance representing the user’s home
1546directory.
1547(Contributed by Victor Salgado and Mayank Tripathi in :issue:`19777`.)
1548
1549New :meth:`Path.write_text() <pathlib.Path.write_text>`,
1550:meth:`Path.read_text() <pathlib.Path.read_text>`,
1551:meth:`Path.write_bytes() <pathlib.Path.write_bytes>`,
1552:meth:`Path.read_bytes() <pathlib.Path.read_bytes>` methods to simplify
1553read/write operations on files.
1554
1555The following code snippet will create or rewrite existing file
1556``~/spam42``::
1557
1558    >>> import pathlib
1559    >>> p = pathlib.Path('~/spam42')
1560    >>> p.expanduser().write_text('ham')
1561    3
1562
1563(Contributed by Christopher Welborn in :issue:`20218`.)
1564
1565
1566pickle
1567------
1568
1569Nested objects, such as unbound methods or nested classes, can now be pickled
1570using :ref:`pickle protocols <pickle-protocols>` older than protocol version 4.
1571Protocol version 4 already supports these cases.  (Contributed by Serhiy
1572Storchaka in :issue:`23611`.)
1573
1574
1575poplib
1576------
1577
1578A new :meth:`POP3.utf8() <poplib.POP3.utf8>` command enables :rfc:`6856`
1579(Internationalized Email) support, if a POP server supports it.
1580(Contributed by Milan OberKirch in :issue:`21804`.)
1581
1582
1583re
1584--
1585
1586References and conditional references to groups with fixed length are now
1587allowed in lookbehind assertions::
1588
1589    >>> import re
1590    >>> pat = re.compile(r'(a|b).(?<=\1)c')
1591    >>> pat.match('aac')
1592    <_sre.SRE_Match object; span=(0, 3), match='aac'>
1593    >>> pat.match('bbc')
1594    <_sre.SRE_Match object; span=(0, 3), match='bbc'>
1595
1596(Contributed by Serhiy Storchaka in :issue:`9179`.)
1597
1598The number of capturing groups in regular expressions is no longer limited to
1599100.  (Contributed by Serhiy Storchaka in :issue:`22437`.)
1600
1601The :func:`~re.sub` and :func:`~re.subn` functions now replace unmatched
1602groups with empty strings instead of raising an exception.
1603(Contributed by Serhiy Storchaka in :issue:`1519638`.)
1604
1605The :class:`re.error` exceptions have new attributes,
1606:attr:`~re.error.msg`, :attr:`~re.error.pattern`,
1607:attr:`~re.error.pos`, :attr:`~re.error.lineno`,
1608and :attr:`~re.error.colno`, that provide better context
1609information about the error::
1610
1611    >>> re.compile("""
1612    ...     (?x)
1613    ...     .++
1614    ... """)
1615    Traceback (most recent call last):
1616       ...
1617    sre_constants.error: multiple repeat at position 16 (line 3, column 7)
1618
1619(Contributed by Serhiy Storchaka in :issue:`22578`.)
1620
1621
1622readline
1623--------
1624
1625A new :func:`~readline.append_history_file` function can be used to append
1626the specified number of trailing elements in history to the given file.
1627(Contributed by Bruno Cauet in :issue:`22940`.)
1628
1629
1630selectors
1631---------
1632
1633The new :class:`~selectors.DevpollSelector` supports efficient
1634``/dev/poll`` polling on Solaris.
1635(Contributed by Giampaolo Rodola' in :issue:`18931`.)
1636
1637
1638shutil
1639------
1640
1641The :func:`~shutil.move` function now accepts a *copy_function* argument,
1642allowing, for example, the :func:`~shutil.copy` function to be used instead of
1643the default :func:`~shutil.copy2` if there is a need to ignore file metadata
1644when moving.
1645(Contributed by Claudiu Popa in :issue:`19840`.)
1646
1647The :func:`~shutil.make_archive` function now supports the *xztar* format.
1648(Contributed by Serhiy Storchaka in :issue:`5411`.)
1649
1650
1651signal
1652------
1653
1654On Windows, the :func:`~signal.set_wakeup_fd` function now also supports
1655socket handles.  (Contributed by Victor Stinner in :issue:`22018`.)
1656
1657Various ``SIG*`` constants in the :mod:`signal` module have been converted into
1658:mod:`Enums <enum>`.  This allows meaningful names to be printed
1659during debugging, instead of integer "magic numbers".
1660(Contributed by Giampaolo Rodola' in :issue:`21076`.)
1661
1662
1663smtpd
1664-----
1665
1666Both the :class:`~smtpd.SMTPServer` and :class:`~smtpd.SMTPChannel` classes now
1667accept a *decode_data* keyword argument to determine if the ``DATA`` portion of
1668the SMTP transaction is decoded using the ``"utf-8"`` codec or is instead
1669provided to the
1670:meth:`SMTPServer.process_message() <smtpd.SMTPServer.process_message>`
1671method as a byte string.  The default is ``True`` for backward compatibility
1672reasons, but will change to ``False`` in Python 3.6.  If *decode_data* is set
1673to ``False``, the ``process_message`` method must be prepared to accept keyword
1674arguments.
1675(Contributed by Maciej Szulik in :issue:`19662`.)
1676
1677The :class:`~smtpd.SMTPServer` class now advertises the ``8BITMIME`` extension
1678(:rfc:`6152`) if *decode_data* has been set ``True``.  If the client
1679specifies ``BODY=8BITMIME`` on the ``MAIL`` command, it is passed to
1680:meth:`SMTPServer.process_message() <smtpd.SMTPServer.process_message>`
1681via the *mail_options* keyword.
1682(Contributed by Milan Oberkirch and R.  David Murray in :issue:`21795`.)
1683
1684The :class:`~smtpd.SMTPServer` class now also supports the ``SMTPUTF8``
1685extension (:rfc:`6531`: Internationalized Email).  If the client specified
1686``SMTPUTF8 BODY=8BITMIME`` on the ``MAIL`` command, they are passed to
1687:meth:`SMTPServer.process_message() <smtpd.SMTPServer.process_message>`
1688via the *mail_options* keyword.  It is the responsibility of the
1689``process_message`` method to correctly handle the ``SMTPUTF8`` data.
1690(Contributed by Milan Oberkirch in :issue:`21725`.)
1691
1692It is now possible to provide, directly or via name resolution, IPv6
1693addresses in the :class:`~smtpd.SMTPServer` constructor, and have it
1694successfully connect.  (Contributed by Milan Oberkirch in :issue:`14758`.)
1695
1696
1697smtplib
1698-------
1699
1700A new :meth:`SMTP.auth() <smtplib.SMTP.auth>` method provides a convenient way to
1701implement custom authentication mechanisms. (Contributed by Milan
1702Oberkirch in :issue:`15014`.)
1703
1704The :meth:`SMTP.set_debuglevel() <smtplib.SMTP.set_debuglevel>` method now
1705accepts an additional debuglevel (2), which enables timestamps in debug
1706messages. (Contributed by Gavin Chappell and Maciej Szulik in :issue:`16914`.)
1707
1708Both the :meth:`SMTP.sendmail() <smtplib.SMTP.sendmail>` and
1709:meth:`SMTP.send_message() <smtplib.SMTP.send_message>` methods now
1710support :rfc:`6531` (SMTPUTF8).
1711(Contributed by Milan Oberkirch and R. David Murray in :issue:`22027`.)
1712
1713
1714sndhdr
1715------
1716
1717The :func:`~sndhdr.what` and :func:`~sndhdr.whathdr` functions  now return
1718a :func:`~collections.namedtuple`.  (Contributed by Claudiu Popa in
1719:issue:`18615`.)
1720
1721
1722socket
1723------
1724
1725Functions with timeouts now use a monotonic clock, instead of a system clock.
1726(Contributed by Victor Stinner in :issue:`22043`.)
1727
1728A new :meth:`socket.sendfile() <socket.socket.sendfile>` method allows
1729sending a file over a socket by using the high-performance :func:`os.sendfile`
1730function on UNIX, resulting in uploads being from 2 to 3 times faster than when
1731using plain :meth:`socket.send() <socket.socket.send>`.
1732(Contributed by Giampaolo Rodola' in :issue:`17552`.)
1733
1734The :meth:`socket.sendall() <socket.socket.sendall>` method no longer resets the
1735socket timeout every time bytes are received or sent.  The socket timeout is
1736now the maximum total duration to send all data.
1737(Contributed by Victor Stinner in :issue:`23853`.)
1738
1739The *backlog* argument of the :meth:`socket.listen() <socket.socket.listen>`
1740method is now optional.  By default it is set to
1741:data:`SOMAXCONN <socket.SOMAXCONN>` or to ``128``, whichever is less.
1742(Contributed by Charles-François Natali in :issue:`21455`.)
1743
1744
1745ssl
1746---
1747
1748.. _whatsnew-sslmemorybio:
1749
1750Memory BIO Support
1751~~~~~~~~~~~~~~~~~~
1752
1753(Contributed by Geert Jansen in :issue:`21965`.)
1754
1755The new :class:`~ssl.SSLObject` class has been added to provide SSL protocol
1756support for cases when the network I/O capabilities of :class:`~ssl.SSLSocket`
1757are not necessary or are suboptimal.  ``SSLObject`` represents
1758an SSL protocol instance, but does not implement any network I/O methods, and
1759instead provides a memory buffer interface.  The new :class:`~ssl.MemoryBIO`
1760class can be used to pass data between Python and an SSL protocol instance.
1761
1762The memory BIO SSL support is primarily intended to be used in frameworks
1763implementing asynchronous I/O for which :class:`~ssl.SSLSocket`'s readiness
1764model ("select/poll") is inefficient.
1765
1766A new :meth:`SSLContext.wrap_bio() <ssl.SSLContext.wrap_bio>` method can be used
1767to create a new ``SSLObject`` instance.
1768
1769
1770Application-Layer Protocol Negotiation Support
1771~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1772
1773(Contributed by Benjamin Peterson in :issue:`20188`.)
1774
1775Where OpenSSL support is present, the :mod:`ssl` module now implements
1776the *Application-Layer Protocol Negotiation* TLS extension as described
1777in :rfc:`7301`.
1778
1779The new :meth:`SSLContext.set_alpn_protocols() <ssl.SSLContext.set_alpn_protocols>`
1780can be used to specify which protocols a socket should advertise during
1781the TLS handshake.
1782
1783The new
1784:meth:`SSLSocket.selected_alpn_protocol() <ssl.SSLSocket.selected_alpn_protocol>`
1785returns the protocol that was selected during the TLS handshake.
1786The :data:`~ssl.HAS_ALPN` flag indicates whether ALPN support is present.
1787
1788
1789Other Changes
1790~~~~~~~~~~~~~
1791
1792There is a new :meth:`SSLSocket.version() <ssl.SSLSocket.version>` method to
1793query the actual protocol version in use.
1794(Contributed by Antoine Pitrou in :issue:`20421`.)
1795
1796The :class:`~ssl.SSLSocket` class now implements
1797a :meth:`SSLSocket.sendfile() <ssl.SSLSocket.sendfile>` method.
1798(Contributed by Giampaolo Rodola' in :issue:`17552`.)
1799
1800The :meth:`SSLSocket.send() <ssl.SSLSocket.send>` method now raises either
1801the :exc:`ssl.SSLWantReadError` or :exc:`ssl.SSLWantWriteError` exception on a
1802non-blocking socket if the operation would block. Previously, it would return
1803``0``.  (Contributed by Nikolaus Rath in :issue:`20951`.)
1804
1805The :func:`~ssl.cert_time_to_seconds` function now interprets the input time
1806as UTC and not as local time, per :rfc:`5280`.  Additionally, the return
1807value is always an :class:`int`. (Contributed by Akira Li in :issue:`19940`.)
1808
1809New :meth:`SSLObject.shared_ciphers() <ssl.SSLObject.shared_ciphers>` and
1810:meth:`SSLSocket.shared_ciphers() <ssl.SSLSocket.shared_ciphers>` methods return
1811the list of ciphers sent by the client during the handshake.
1812(Contributed by Benjamin Peterson in :issue:`23186`.)
1813
1814The :meth:`SSLSocket.do_handshake() <ssl.SSLSocket.do_handshake>`,
1815:meth:`SSLSocket.read() <ssl.SSLSocket.read>`,
1816:meth:`SSLSocket.shutdown() <ssl.SSLSocket.shutdown>`, and
1817:meth:`SSLSocket.write() <ssl.SSLSocket.write>` methods of the :class:`~ssl.SSLSocket`
1818class no longer reset the socket timeout every time bytes are received or sent.
1819The socket timeout is now the maximum total duration of the method.
1820(Contributed by Victor Stinner in :issue:`23853`.)
1821
1822The :func:`~ssl.match_hostname` function now supports matching of IP addresses.
1823(Contributed by Antoine Pitrou in :issue:`23239`.)
1824
1825
1826sqlite3
1827-------
1828
1829The :class:`~sqlite3.Row` class now fully supports the sequence protocol,
1830in particular :func:`reversed` iteration and slice indexing.
1831(Contributed by Claudiu Popa in :issue:`10203`; by Lucas Sinclair,
1832Jessica McKellar, and  Serhiy Storchaka in :issue:`13583`.)
1833
1834
1835.. _whatsnew-subprocess:
1836
1837subprocess
1838----------
1839
1840The new :func:`~subprocess.run` function has been added.
1841It runs the specified command and returns a
1842:class:`~subprocess.CompletedProcess` object, which describes a finished
1843process.  The new API is more consistent and is the recommended approach
1844to invoking subprocesses in Python code that does not need to maintain
1845compatibility with earlier Python versions.
1846(Contributed by Thomas Kluyver in :issue:`23342`.)
1847
1848Examples::
1849
1850    >>> subprocess.run(["ls", "-l"])  # doesn't capture output
1851    CompletedProcess(args=['ls', '-l'], returncode=0)
1852
1853    >>> subprocess.run("exit 1", shell=True, check=True)
1854    Traceback (most recent call last):
1855      ...
1856    subprocess.CalledProcessError: Command 'exit 1' returned non-zero exit status 1
1857
1858    >>> subprocess.run(["ls", "-l", "/dev/null"], stdout=subprocess.PIPE)
1859    CompletedProcess(args=['ls', '-l', '/dev/null'], returncode=0,
1860    stdout=b'crw-rw-rw- 1 root root 1, 3 Jan 23 16:23 /dev/null\n')
1861
1862
1863sys
1864---
1865
1866A new :func:`~sys.set_coroutine_wrapper` function allows setting a global
1867hook that will be called whenever a :term:`coroutine object <coroutine>`
1868is created by an :keyword:`async def` function.  A corresponding
1869:func:`~sys.get_coroutine_wrapper` can be used to obtain a currently set
1870wrapper.  Both functions are :term:`provisional <provisional api>`,
1871and are intended for debugging purposes only.  (Contributed by Yury Selivanov
1872in :issue:`24017`.)
1873
1874A new :func:`~sys.is_finalizing` function can be used to check if the Python
1875interpreter is :term:`shutting down <interpreter shutdown>`.
1876(Contributed by Antoine Pitrou in :issue:`22696`.)
1877
1878
1879sysconfig
1880---------
1881
1882The name of the user scripts directory on Windows now includes the first
1883two components of the Python version. (Contributed by Paul Moore
1884in :issue:`23437`.)
1885
1886
1887tarfile
1888-------
1889
1890The *mode* argument of the :func:`~tarfile.open` function now accepts ``"x"``
1891to request exclusive creation.  (Contributed by Berker Peksag in :issue:`21717`.)
1892
1893The :meth:`TarFile.extractall() <tarfile.TarFile.extractall>` and
1894:meth:`TarFile.extract() <tarfile.TarFile.extract>` methods now take a keyword
1895argument *numeric_owner*.  If set to ``True``, the extracted files and
1896directories will be owned by the numeric ``uid`` and ``gid`` from the tarfile.
1897If set to ``False`` (the default, and the behavior in versions prior to 3.5),
1898they will be owned by the named user and group in the tarfile.
1899(Contributed by Michael Vogt and Eric Smith in :issue:`23193`.)
1900
1901The :meth:`TarFile.list() <tarfile.TarFile.list>` now accepts an optional
1902*members* keyword argument that can be set to a subset of the list returned
1903by :meth:`TarFile.getmembers() <tarfile.TarFile.getmembers>`.
1904(Contributed by Serhiy Storchaka in :issue:`21549`.)
1905
1906
1907threading
1908---------
1909
1910Both the :meth:`Lock.acquire() <threading.Lock.acquire>` and
1911:meth:`RLock.acquire() <threading.RLock.acquire>` methods
1912now use a monotonic clock for timeout management.
1913(Contributed by Victor Stinner in :issue:`22043`.)
1914
1915
1916time
1917----
1918
1919The :func:`~time.monotonic` function is now always available.
1920(Contributed by Victor Stinner in :issue:`22043`.)
1921
1922
1923timeit
1924------
1925
1926A new command line option ``-u`` or :samp:`--unit={U}` can be used to specify the time
1927unit for the timer output.  Supported options are ``usec``, ``msec``,
1928or ``sec``.  (Contributed by Julian Gindi in :issue:`18983`.)
1929
1930The :func:`~timeit.timeit` function has a new *globals* parameter for
1931specifying the namespace in which the code will be running.
1932(Contributed by Ben Roberts in :issue:`2527`.)
1933
1934
1935tkinter
1936-------
1937
1938The :mod:`tkinter._fix` module used for setting up the Tcl/Tk environment
1939on Windows has been replaced by a private function in the :mod:`_tkinter`
1940module which makes no permanent changes to environment variables.
1941(Contributed by Zachary Ware in :issue:`20035`.)
1942
1943
1944.. _whatsnew-traceback:
1945
1946traceback
1947---------
1948
1949New :func:`~traceback.walk_stack` and :func:`~traceback.walk_tb`
1950functions to conveniently traverse frame and traceback objects.
1951(Contributed by Robert Collins in :issue:`17911`.)
1952
1953New lightweight classes: :class:`~traceback.TracebackException`,
1954:class:`~traceback.StackSummary`, and :class:`~traceback.FrameSummary`.
1955(Contributed by Robert Collins in :issue:`17911`.)
1956
1957Both the :func:`~traceback.print_tb` and :func:`~traceback.print_stack` functions
1958now support negative values for the *limit* argument.
1959(Contributed by Dmitry Kazakov in :issue:`22619`.)
1960
1961
1962types
1963-----
1964
1965A new :func:`~types.coroutine` function to transform
1966:term:`generator <generator iterator>` and
1967:class:`generator-like <collections.abc.Generator>` objects into
1968:term:`awaitables <awaitable>`.
1969(Contributed by Yury Selivanov in :issue:`24017`.)
1970
1971A new type called :class:`~types.CoroutineType`, which is used for
1972:term:`coroutine` objects created by :keyword:`async def` functions.
1973(Contributed by Yury Selivanov in :issue:`24400`.)
1974
1975
1976unicodedata
1977-----------
1978
1979The :mod:`unicodedata` module now uses data from `Unicode 8.0.0
1980<http://unicode.org/versions/Unicode8.0.0/>`_.
1981
1982
1983unittest
1984--------
1985
1986The :meth:`TestLoader.loadTestsFromModule() <unittest.TestLoader.loadTestsFromModule>`
1987method now accepts a keyword-only argument *pattern* which is passed to
1988``load_tests`` as the third argument.  Found packages are now checked for
1989``load_tests`` regardless of whether their path matches *pattern*, because it
1990is impossible for a package name to match the default pattern.
1991(Contributed by Robert Collins and Barry A. Warsaw in :issue:`16662`.)
1992
1993Unittest discovery errors now are exposed in the
1994:data:`TestLoader.errors <unittest.TestLoader.errors>` attribute of the
1995:class:`~unittest.TestLoader` instance.
1996(Contributed by Robert Collins in :issue:`19746`.)
1997
1998A new command line option ``--locals`` to show local variables in
1999tracebacks.  (Contributed by Robert Collins in :issue:`22936`.)
2000
2001
2002unittest.mock
2003-------------
2004
2005The :class:`~unittest.mock.Mock` class has the following improvements:
2006
2007* The class constructor has a new *unsafe* parameter, which causes mock
2008  objects to raise :exc:`AttributeError` on attribute names starting
2009  with ``"assert"``.
2010  (Contributed by Kushal Das in :issue:`21238`.)
2011
2012* A new :meth:`Mock.assert_not_called() <unittest.mock.Mock.assert_not_called>`
2013  method to check if the mock object was called.
2014  (Contributed by Kushal Das in :issue:`21262`.)
2015
2016The :class:`~unittest.mock.MagicMock` class now supports :meth:`__truediv__`,
2017:meth:`__divmod__` and :meth:`__matmul__` operators.
2018(Contributed by Johannes Baiter in :issue:`20968`, and Håkan Lövdahl
2019in :issue:`23581` and :issue:`23568`.)
2020
2021It is no longer necessary to explicitly pass ``create=True`` to the
2022:func:`~unittest.mock.patch` function when patching builtin names.
2023(Contributed by Kushal Das in :issue:`17660`.)
2024
2025
2026urllib
2027------
2028
2029A new
2030:class:`request.HTTPPasswordMgrWithPriorAuth <urllib.request.HTTPPasswordMgrWithPriorAuth>`
2031class allows HTTP Basic Authentication credentials to be managed so as to
2032eliminate unnecessary ``401`` response handling, or to unconditionally send
2033credentials on the first request in order to communicate with servers that
2034return a ``404`` response instead of a ``401`` if the ``Authorization`` header
2035is not sent. (Contributed by Matej Cepl in :issue:`19494` and Akshit Khurana in
2036:issue:`7159`.)
2037
2038A new *quote_via* argument for the
2039:func:`parse.urlencode() <urllib.parse.urlencode>`
2040function provides a way to control the encoding of query parts if needed.
2041(Contributed by Samwyse and Arnon Yaari in :issue:`13866`.)
2042
2043The :func:`request.urlopen() <urllib.request.urlopen>` function accepts an
2044:class:`ssl.SSLContext` object as a *context* argument, which will be used for
2045the HTTPS connection.  (Contributed by Alex Gaynor in :issue:`22366`.)
2046
2047The :func:`parse.urljoin() <urllib.parse.urljoin>` was updated to use the
2048:rfc:`3986` semantics for the resolution of relative URLs, rather than
2049:rfc:`1808` and :rfc:`2396`.
2050(Contributed by Demian Brecht and Senthil Kumaran in :issue:`22118`.)
2051
2052
2053wsgiref
2054-------
2055
2056The *headers* argument of the :class:`headers.Headers <wsgiref.headers.Headers>`
2057class constructor is now optional.
2058(Contributed by Pablo Torres Navarrete and SilentGhost in :issue:`5800`.)
2059
2060
2061xmlrpc
2062------
2063
2064The :class:`client.ServerProxy <xmlrpc.client.ServerProxy>` class now supports
2065the :term:`context manager` protocol.
2066(Contributed by Claudiu Popa in :issue:`20627`.)
2067
2068The :class:`client.ServerProxy <xmlrpc.client.ServerProxy>` constructor now accepts
2069an optional :class:`ssl.SSLContext` instance.
2070(Contributed by Alex Gaynor in :issue:`22960`.)
2071
2072
2073xml.sax
2074-------
2075
2076SAX parsers now support a character stream of the
2077:class:`xmlreader.InputSource <xml.sax.xmlreader.InputSource>` object.
2078(Contributed by Serhiy Storchaka in :issue:`2175`.)
2079
2080:func:`~xml.sax.parseString` now accepts a :class:`str` instance.
2081(Contributed by Serhiy Storchaka in :issue:`10590`.)
2082
2083
2084zipfile
2085-------
2086
2087ZIP output can now be written to unseekable streams.
2088(Contributed by Serhiy Storchaka in :issue:`23252`.)
2089
2090The *mode* argument of :meth:`ZipFile.open() <zipfile.ZipFile.open>` method now
2091accepts ``"x"`` to request exclusive creation.
2092(Contributed by Serhiy Storchaka in :issue:`21717`.)
2093
2094
2095Other module-level changes
2096==========================
2097
2098Many functions in the :mod:`mmap`, :mod:`ossaudiodev`, :mod:`socket`,
2099:mod:`ssl`, and :mod:`codecs` modules now accept writable
2100:term:`bytes-like objects <bytes-like object>`.
2101(Contributed by Serhiy Storchaka in :issue:`23001`.)
2102
2103
2104Optimizations
2105=============
2106
2107The :func:`os.walk` function has been sped up by 3 to 5 times on POSIX systems,
2108and by 7 to 20 times on Windows.  This was done using the new :func:`os.scandir`
2109function, which exposes file information from the underlying ``readdir`` or
2110``FindFirstFile``/``FindNextFile`` system calls.  (Contributed by
2111Ben Hoyt with help from Victor Stinner in :issue:`23605`.)
2112
2113Construction of ``bytes(int)`` (filled by zero bytes) is faster and uses less
2114memory for large objects. ``calloc()`` is used instead of ``malloc()`` to
2115allocate memory for these objects.
2116(Contributed by Victor Stinner in :issue:`21233`.)
2117
2118Some operations on :mod:`ipaddress` :class:`~ipaddress.IPv4Network` and
2119:class:`~ipaddress.IPv6Network` have been massively sped up, such as
2120:meth:`~ipaddress.IPv4Network.subnets`, :meth:`~ipaddress.IPv4Network.supernet`,
2121:func:`~ipaddress.summarize_address_range`, :func:`~ipaddress.collapse_addresses`.
2122The speed up can range from 3 to 15 times.
2123(Contributed by Antoine Pitrou, Michel Albert, and Markus in
2124:issue:`21486`, :issue:`21487`, :issue:`20826`, :issue:`23266`.)
2125
2126Pickling of :mod:`ipaddress` objects was optimized to produce significantly
2127smaller output.  (Contributed by Serhiy Storchaka in :issue:`23133`.)
2128
2129Many operations on :class:`io.BytesIO` are now 50% to 100% faster.
2130(Contributed by Serhiy Storchaka in :issue:`15381` and David Wilson in
2131:issue:`22003`.)
2132
2133The :func:`marshal.dumps` function is now faster: 65--85% with versions 3
2134and 4, 20--25% with versions 0 to 2 on typical data, and up to 5 times in
2135best cases.
2136(Contributed by Serhiy Storchaka in :issue:`20416` and :issue:`23344`.)
2137
2138The UTF-32 encoder is now 3 to 7 times faster.
2139(Contributed by Serhiy Storchaka in :issue:`15027`.)
2140
2141Regular expressions are now parsed up to 10% faster.
2142(Contributed by Serhiy Storchaka in :issue:`19380`.)
2143
2144The :func:`json.dumps` function was optimized to run with
2145``ensure_ascii=False`` as fast as with ``ensure_ascii=True``.
2146(Contributed by Naoki Inada in :issue:`23206`.)
2147
2148The :c:func:`PyObject_IsInstance` and :c:func:`PyObject_IsSubclass`
2149functions have been sped up in the common case that the second argument
2150has :class:`type` as its metaclass.
2151(Contributed Georg Brandl by in :issue:`22540`.)
2152
2153Method caching was slightly improved, yielding up to 5% performance
2154improvement in some benchmarks.
2155(Contributed by Antoine Pitrou in :issue:`22847`.)
2156
2157Objects from the :mod:`random` module now use 50% less memory on 64-bit
2158builds.  (Contributed by Serhiy Storchaka in :issue:`23488`.)
2159
2160The :func:`property` getter calls are up to 25% faster.
2161(Contributed by Joe Jevnik in :issue:`23910`.)
2162
2163Instantiation of :class:`fractions.Fraction` is now up to 30% faster.
2164(Contributed by Stefan Behnel in :issue:`22464`.)
2165
2166String methods :meth:`~str.find`, :meth:`~str.rfind`, :meth:`~str.split`,
2167:meth:`~str.partition` and the :keyword:`in` string operator are now significantly
2168faster for searching 1-character substrings.
2169(Contributed by Serhiy Storchaka in :issue:`23573`.)
2170
2171
2172Build and C API Changes
2173=======================
2174
2175New ``calloc`` functions were added:
2176
2177* :c:func:`PyMem_RawCalloc`,
2178* :c:func:`PyMem_Calloc`,
2179* :c:func:`PyObject_Calloc`.
2180
2181(Contributed by Victor Stinner in :issue:`21233`.)
2182
2183New encoding/decoding helper functions:
2184
2185* :c:func:`Py_DecodeLocale` (replaced ``_Py_char2wchar()``),
2186* :c:func:`Py_EncodeLocale` (replaced ``_Py_wchar2char()``).
2187
2188(Contributed by Victor Stinner in :issue:`18395`.)
2189
2190A new :c:func:`PyCodec_NameReplaceErrors` function to replace the unicode
2191encode error with ``\N{...}`` escapes.
2192(Contributed by Serhiy Storchaka in :issue:`19676`.)
2193
2194A new :c:func:`PyErr_FormatV` function similar to :c:func:`PyErr_Format`,
2195but accepts a ``va_list`` argument.
2196(Contributed by Antoine Pitrou in :issue:`18711`.)
2197
2198A new :c:data:`PyExc_RecursionError` exception.
2199(Contributed by Georg Brandl in :issue:`19235`.)
2200
2201New :c:func:`PyModule_FromDefAndSpec`, :c:func:`PyModule_FromDefAndSpec2`,
2202and :c:func:`PyModule_ExecDef` functions introduced by :pep:`489` --
2203multi-phase extension module initialization.
2204(Contributed by Petr Viktorin in :issue:`24268`.)
2205
2206New :c:func:`PyNumber_MatrixMultiply` and
2207:c:func:`PyNumber_InPlaceMatrixMultiply` functions to perform matrix
2208multiplication.
2209(Contributed by Benjamin Peterson in :issue:`21176`.  See also :pep:`465`
2210for details.)
2211
2212The :c:member:`PyTypeObject.tp_finalize` slot is now part of the stable ABI.
2213
2214Windows builds now require Microsoft Visual C++ 14.0, which
2215is available as part of `Visual Studio 2015 <https://www.visualstudio.com/>`_.
2216
2217Extension modules now include a platform information tag in their filename on
2218some platforms (the tag is optional, and CPython will import extensions without
2219it, although if the tag is present and mismatched, the extension won't be
2220loaded):
2221
2222* On Linux, extension module filenames end with
2223  ``.cpython-<major><minor>m-<architecture>-<os>.pyd``:
2224
2225  * ``<major>`` is the major number of the Python version;
2226    for Python 3.5 this is ``3``.
2227
2228  * ``<minor>`` is the minor number of the Python version;
2229    for Python 3.5 this is ``5``.
2230
2231  * ``<architecture>`` is the hardware architecture the extension module
2232    was built to run on. It's most commonly either ``i386`` for 32-bit Intel
2233    platforms or ``x86_64`` for 64-bit Intel (and AMD) platforms.
2234
2235  * ``<os>`` is always ``linux-gnu``, except for extensions built to
2236    talk to the 32-bit ABI on 64-bit platforms, in which case it is
2237    ``linux-gnu32`` (and ``<architecture>`` will be ``x86_64``).
2238
2239* On Windows, extension module filenames end with
2240  ``<debug>.cp<major><minor>-<platform>.pyd``:
2241
2242  * ``<major>`` is the major number of the Python version;
2243    for Python 3.5 this is ``3``.
2244
2245  * ``<minor>`` is the minor number of the Python version;
2246    for Python 3.5 this is ``5``.
2247
2248  * ``<platform>`` is the platform the extension module was built for,
2249    either ``win32`` for Win32, ``win_amd64`` for Win64, ``win_ia64`` for
2250    Windows Itanium 64, and ``win_arm`` for Windows on ARM.
2251
2252  * If built in debug mode, ``<debug>`` will be ``_d``,
2253    otherwise it will be blank.
2254
2255* On OS X platforms, extension module filenames now end with ``-darwin.so``.
2256
2257* On all other platforms, extension module filenames are the same as they were
2258  with Python 3.4.
2259
2260
2261Deprecated
2262==========
2263
2264New Keywords
2265------------
2266
2267``async`` and ``await`` are not recommended to be used as variable, class,
2268function or module names.  Introduced by :pep:`492` in Python 3.5, they will
2269become proper keywords in Python 3.7.
2270
2271
2272Deprecated Python Behavior
2273--------------------------
2274
2275Raising the :exc:`StopIteration` exception inside a generator will now generate a silent
2276:exc:`PendingDeprecationWarning`, which will become a non-silent deprecation
2277warning in Python 3.6 and will trigger a :exc:`RuntimeError` in Python 3.7.
2278See :ref:`PEP 479: Change StopIteration handling inside generators <whatsnew-pep-479>`
2279for details.
2280
2281
2282Unsupported Operating Systems
2283-----------------------------
2284
2285Windows XP is no longer supported by Microsoft, thus, per :PEP:`11`, CPython
22863.5 is no longer officially supported on this OS.
2287
2288
2289Deprecated Python modules, functions and methods
2290------------------------------------------------
2291
2292The :mod:`formatter` module has now graduated to full deprecation and is still
2293slated for removal in Python 3.6.
2294
2295The :func:`asyncio.async` function is deprecated in favor of
2296:func:`~asyncio.ensure_future`.
2297
2298The :mod:`smtpd` module has in the past always decoded the DATA portion of
2299email messages using the ``utf-8`` codec.  This can now be controlled by the
2300new *decode_data* keyword to :class:`~smtpd.SMTPServer`.  The default value is
2301``True``, but this default is deprecated.  Specify the *decode_data* keyword
2302with an appropriate value to avoid the deprecation warning.
2303
2304Directly assigning values to the :attr:`~http.cookies.Morsel.key`,
2305:attr:`~http.cookies.Morsel.value` and
2306:attr:`~http.cookies.Morsel.coded_value` of :class:`http.cookies.Morsel`
2307objects is deprecated.  Use the :meth:`~http.cookies.Morsel.set` method
2308instead.  In addition, the undocumented *LegalChars* parameter of
2309:meth:`~http.cookies.Morsel.set` is deprecated, and is now ignored.
2310
2311Passing a format string as keyword argument *format_string* to the
2312:meth:`~string.Formatter.format` method of the :class:`string.Formatter`
2313class has been deprecated.
2314(Contributed by Serhiy Storchaka in :issue:`23671`.)
2315
2316The :func:`platform.dist` and :func:`platform.linux_distribution` functions
2317are now deprecated.  Linux distributions use too many different ways of
2318describing themselves, so the functionality is left to a package.
2319(Contributed by Vajrasky Kok and Berker Peksag in :issue:`1322`.)
2320
2321The previously undocumented ``from_function`` and ``from_builtin`` methods of
2322:class:`inspect.Signature` are deprecated.  Use the new
2323:meth:`Signature.from_callable() <inspect.Signature.from_callable>`
2324method instead. (Contributed by Yury Selivanov in :issue:`24248`.)
2325
2326The :func:`inspect.getargspec` function is deprecated and scheduled to be
2327removed in Python 3.6.  (See :issue:`20438` for details.)
2328
2329The :mod:`inspect` :func:`~inspect.getfullargspec`,
2330:func:`~inspect.getcallargs`, and :func:`~inspect.formatargspec` functions are
2331deprecated in favor of the :func:`inspect.signature` API. (Contributed by Yury
2332Selivanov in :issue:`20438`.)
2333
2334:func:`~inspect.getargvalues` and :func:`~inspect.formatargvalues` functions
2335were inadvertently marked as deprecated with the release of Python 3.5.0.
2336
2337Use of :const:`re.LOCALE` flag with str patterns or :const:`re.ASCII` is now
2338deprecated.  (Contributed by Serhiy Storchaka in :issue:`22407`.)
2339
2340Use of unrecognized special sequences consisting of ``'\'`` and an ASCII letter
2341in regular expression patterns and replacement patterns now raises a
2342deprecation warning and will be forbidden in Python 3.6.
2343(Contributed by Serhiy Storchaka in :issue:`23622`.)
2344
2345The undocumented and unofficial *use_load_tests* default argument of the
2346:meth:`unittest.TestLoader.loadTestsFromModule` method now is
2347deprecated and ignored.
2348(Contributed by Robert Collins and Barry A. Warsaw in :issue:`16662`.)
2349
2350
2351Removed
2352=======
2353
2354API and Feature Removals
2355------------------------
2356
2357The following obsolete and previously deprecated APIs and features have been
2358removed:
2359
2360* The ``__version__`` attribute has been dropped from the email package.  The
2361  email code hasn't been shipped separately from the stdlib for a long time,
2362  and the ``__version__`` string was not updated in the last few releases.
2363
2364* The internal ``Netrc`` class in the :mod:`ftplib` module was deprecated in
2365  3.4, and has now been removed.
2366  (Contributed by Matt Chaput in :issue:`6623`.)
2367
2368* The concept of ``.pyo`` files has been removed.
2369
2370* The JoinableQueue class in the provisional :mod:`asyncio` module was
2371  deprecated in 3.4.4 and is now removed.
2372  (Contributed by A. Jesse Jiryu Davis in :issue:`23464`.)
2373
2374
2375Porting to Python 3.5
2376=====================
2377
2378This section lists previously described changes and other bugfixes
2379that may require changes to your code.
2380
2381
2382Changes in Python behavior
2383--------------------------
2384
2385* Due to an oversight, earlier Python versions erroneously accepted the
2386  following syntax::
2387
2388      f(1 for x in [1], *args)
2389      f(1 for x in [1], **kwargs)
2390
2391  Python 3.5 now correctly raises a :exc:`SyntaxError`, as generator
2392  expressions must be put in parentheses if not a sole argument to a function.
2393
2394
2395Changes in the Python API
2396-------------------------
2397
2398* :pep:`475`: System calls are now retried when interrupted by a signal instead
2399  of raising :exc:`InterruptedError` if the Python signal handler does not
2400  raise an exception.
2401
2402* Before Python 3.5, a :class:`datetime.time` object was considered to be false
2403  if it represented midnight in UTC.  This behavior was considered obscure and
2404  error-prone and has been removed in Python 3.5.  See :issue:`13936` for full
2405  details.
2406
2407* The :meth:`ssl.SSLSocket.send()` method now raises either
2408  :exc:`ssl.SSLWantReadError` or :exc:`ssl.SSLWantWriteError`
2409  on a non-blocking socket if the operation would block.  Previously,
2410  it would return ``0``.  (Contributed by Nikolaus Rath in :issue:`20951`.)
2411
2412* The ``__name__`` attribute of generators is now set from the function name,
2413  instead of being set from the code name. Use ``gen.gi_code.co_name`` to
2414  retrieve the code name. Generators also have a new ``__qualname__``
2415  attribute, the qualified name, which is now used for the representation
2416  of a generator (``repr(gen)``).
2417  (Contributed by Victor Stinner in :issue:`21205`.)
2418
2419* The deprecated "strict" mode and argument of :class:`~html.parser.HTMLParser`,
2420  :meth:`HTMLParser.error`, and the :exc:`HTMLParserError` exception have been
2421  removed.  (Contributed by Ezio Melotti in :issue:`15114`.)
2422  The *convert_charrefs* argument of :class:`~html.parser.HTMLParser` is
2423  now ``True`` by default.  (Contributed by Berker Peksag in :issue:`21047`.)
2424
2425* Although it is not formally part of the API, it is worth noting for porting
2426  purposes (ie: fixing tests) that error messages that were previously of the
2427  form "'sometype' does not support the buffer protocol" are now of the form "a
2428  :term:`bytes-like object` is required, not 'sometype'".
2429  (Contributed by Ezio Melotti in :issue:`16518`.)
2430
2431* If the current directory is set to a directory that no longer exists then
2432  :exc:`FileNotFoundError` will no longer be raised and instead
2433  :meth:`~importlib.machinery.FileFinder.find_spec` will return ``None``
2434  **without** caching ``None`` in :data:`sys.path_importer_cache`, which is
2435  different than the typical case (:issue:`22834`).
2436
2437* HTTP status code and messages from :mod:`http.client` and :mod:`http.server`
2438  were refactored into a common :class:`~http.HTTPStatus` enum.  The values in
2439  :mod:`http.client` and :mod:`http.server` remain available for backwards
2440  compatibility.  (Contributed by Demian Brecht in :issue:`21793`.)
2441
2442* When an import loader defines :meth:`importlib.machinery.Loader.exec_module`
2443  it is now expected to also define
2444  :meth:`~importlib.machinery.Loader.create_module` (raises a
2445  :exc:`DeprecationWarning` now, will be an error in Python 3.6). If the loader
2446  inherits from :class:`importlib.abc.Loader` then there is nothing to do, else
2447  simply define :meth:`~importlib.machinery.Loader.create_module` to return
2448  ``None``.  (Contributed by Brett Cannon in :issue:`23014`.)
2449
2450* The :func:`re.split` function always ignored empty pattern matches, so the
2451  ``"x*"`` pattern worked the same as ``"x+"``, and the ``"\b"`` pattern never
2452  worked.  Now :func:`re.split` raises a warning if the pattern could match
2453  an empty string.  For compatibility, use patterns that never match an empty
2454  string (e.g. ``"x+"`` instead of ``"x*"``).  Patterns that could only match
2455  an empty string (such as ``"\b"``) now raise an error.
2456  (Contributed by Serhiy Storchaka in :issue:`22818`.)
2457
2458* The :class:`http.cookies.Morsel` dict-like interface has been made self
2459  consistent:  morsel comparison now takes the :attr:`~http.cookies.Morsel.key`
2460  and :attr:`~http.cookies.Morsel.value` into account,
2461  :meth:`~http.cookies.Morsel.copy` now results in a
2462  :class:`~http.cookies.Morsel` instance rather than a :class:`dict`, and
2463  :meth:`~http.cookies.Morsel.update` will now raise an exception if any of the
2464  keys in the update dictionary are invalid.  In addition, the undocumented
2465  *LegalChars* parameter of :func:`~http.cookies.Morsel.set` is deprecated and
2466  is now ignored.  (Contributed by Demian Brecht in :issue:`2211`.)
2467
2468* :pep:`488` has removed ``.pyo`` files from Python and introduced the optional
2469  ``opt-`` tag in ``.pyc`` file names. The
2470  :func:`importlib.util.cache_from_source` has gained an *optimization*
2471  parameter to help control the ``opt-`` tag. Because of this, the
2472  *debug_override* parameter of the function is now deprecated. `.pyo` files
2473  are also no longer supported as a file argument to the Python interpreter and
2474  thus serve no purpose when distributed on their own (i.e. sourceless code
2475  distribution). Due to the fact that the magic number for bytecode has changed
2476  in Python 3.5, all old `.pyo` files from previous versions of Python are
2477  invalid regardless of this PEP.
2478
2479* The :mod:`socket` module now exports the :data:`~socket.CAN_RAW_FD_FRAMES`
2480  constant on linux 3.6 and greater.
2481
2482* The :func:`ssl.cert_time_to_seconds` function now interprets the input time
2483  as UTC and not as local time, per :rfc:`5280`.  Additionally, the return
2484  value is always an :class:`int`. (Contributed by Akira Li in :issue:`19940`.)
2485
2486* The ``pygettext.py`` Tool now uses the standard +NNNN format for timezones in
2487  the POT-Creation-Date header.
2488
2489* The :mod:`smtplib` module now uses :data:`sys.stderr` instead of the previous
2490  module-level :data:`stderr` variable for debug output.  If your (test)
2491  program depends on patching the module-level variable to capture the debug
2492  output, you will need to update it to capture sys.stderr instead.
2493
2494* The :meth:`str.startswith` and :meth:`str.endswith` methods no longer return
2495  ``True`` when finding the empty string and the indexes are completely out of
2496  range.  (Contributed by Serhiy Storchaka in :issue:`24284`.)
2497
2498* The :func:`inspect.getdoc` function now returns documentation strings
2499  inherited from base classes.  Documentation strings no longer need to be
2500  duplicated if the inherited documentation is appropriate.  To suppress an
2501  inherited string, an empty string must be specified (or the documentation
2502  may be filled in).  This change affects the output of the :mod:`pydoc`
2503  module and the :func:`help` function.
2504  (Contributed by Serhiy Storchaka in :issue:`15582`.)
2505
2506* Nested :func:`functools.partial` calls are now flattened.  If you were
2507  relying on the previous behavior, you can now either add an attribute to a
2508  :func:`functools.partial` object or you can create a subclass of
2509  :func:`functools.partial`.
2510  (Contributed by Alexander Belopolsky in :issue:`7830`.)
2511
2512Changes in the C API
2513--------------------
2514
2515* The undocumented :c:member:`~PyMemoryViewObject.format` member of the
2516  (non-public) :c:type:`PyMemoryViewObject` structure has been removed.
2517  All extensions relying on the relevant parts in ``memoryobject.h``
2518  must be rebuilt.
2519
2520* The :c:type:`PyMemAllocator` structure was renamed to
2521  :c:type:`PyMemAllocatorEx` and a new ``calloc`` field was added.
2522
2523* Removed non-documented macro :c:macro:`PyObject_REPR` which leaked references.
2524  Use format character ``%R`` in :c:func:`PyUnicode_FromFormat`-like functions
2525  to format the :func:`repr` of the object.
2526  (Contributed by Serhiy Storchaka in :issue:`22453`.)
2527
2528* Because the lack of the :attr:`__module__` attribute breaks pickling and
2529  introspection, a deprecation warning is now raised for builtin types without
2530  the :attr:`__module__` attribute.  This would be an AttributeError in
2531  the future.
2532  (Contributed by Serhiy Storchaka in :issue:`20204`.)
2533
2534* As part of the :pep:`492` implementation, the ``tp_reserved`` slot of
2535  :c:type:`PyTypeObject` was replaced with a
2536  :c:member:`tp_as_async` slot.  Refer to :ref:`coro-objects` for
2537  new types, structures and functions.
2538
2539
2540Notable changes in Python 3.5.4
2541===============================
2542
2543New ``make regen-all`` build target
2544-----------------------------------
2545
2546To simplify cross-compilation, and to ensure that CPython can reliably be
2547compiled without requiring an existing version of Python to already be
2548available, the autotools-based build system no longer attempts to implicitly
2549recompile generated files based on file modification times.
2550
2551Instead, a new ``make regen-all`` command has been added to force regeneration
2552of these files when desired (e.g. after an initial version of Python has
2553already been built based on the pregenerated versions).
2554
2555More selective regeneration targets are also defined - see
2556:source:`Makefile.pre.in` for details.
2557
2558(Contributed by Victor Stinner in :issue:`23404`.)
2559
2560.. versionadded:: 3.5.4
2561
2562
2563Removal of ``make touch`` build target
2564--------------------------------------
2565
2566The ``make touch`` build target previously used to request implicit regeneration
2567of generated files by updating their modification times has been removed.
2568
2569It has been replaced by the new ``make regen-all`` target.
2570
2571(Contributed by Victor Stinner in :issue:`23404`.)
2572
2573.. versionchanged:: 3.5.4
2574
2575