1.. currentmodule:: asyncio 2 3.. _asyncio-dev: 4 5======================= 6Developing with asyncio 7======================= 8 9Asynchronous programming is different from classic "sequential" 10programming. 11 12This page lists common mistakes and traps and explains how 13to avoid them. 14 15 16.. _asyncio-debug-mode: 17 18Debug Mode 19========== 20 21By default asyncio runs in production mode. In order to ease 22the development asyncio has a *debug mode*. 23 24There are several ways to enable asyncio debug mode: 25 26* Setting the :envvar:`PYTHONASYNCIODEBUG` environment variable to ``1``. 27 28* Using the :ref:`Python Development Mode <devmode>`. 29 30* Passing ``debug=True`` to :func:`asyncio.run`. 31 32* Calling :meth:`loop.set_debug`. 33 34In addition to enabling the debug mode, consider also: 35 36* setting the log level of the :ref:`asyncio logger <asyncio-logger>` to 37 :py:data:`logging.DEBUG`, for example the following snippet of code 38 can be run at startup of the application:: 39 40 logging.basicConfig(level=logging.DEBUG) 41 42* configuring the :mod:`warnings` module to display 43 :exc:`ResourceWarning` warnings. One way of doing that is by 44 using the :option:`-W` ``default`` command line option. 45 46 47When the debug mode is enabled: 48 49* asyncio checks for :ref:`coroutines that were not awaited 50 <asyncio-coroutine-not-scheduled>` and logs them; this mitigates 51 the "forgotten await" pitfall. 52 53* Many non-threadsafe asyncio APIs (such as :meth:`loop.call_soon` and 54 :meth:`loop.call_at` methods) raise an exception if they are called 55 from a wrong thread. 56 57* The execution time of the I/O selector is logged if it takes too long to 58 perform an I/O operation. 59 60* Callbacks taking longer than 100ms are logged. The 61 :attr:`loop.slow_callback_duration` attribute can be used to set the 62 minimum execution duration in seconds that is considered "slow". 63 64 65.. _asyncio-multithreading: 66 67Concurrency and Multithreading 68============================== 69 70An event loop runs in a thread (typically the main thread) and executes 71all callbacks and Tasks in its thread. While a Task is running in the 72event loop, no other Tasks can run in the same thread. When a Task 73executes an ``await`` expression, the running Task gets suspended, and 74the event loop executes the next Task. 75 76To schedule a :term:`callback` from another OS thread, the 77:meth:`loop.call_soon_threadsafe` method should be used. Example:: 78 79 loop.call_soon_threadsafe(callback, *args) 80 81Almost all asyncio objects are not thread safe, which is typically 82not a problem unless there is code that works with them from outside 83of a Task or a callback. If there's a need for such code to call a 84low-level asyncio API, the :meth:`loop.call_soon_threadsafe` method 85should be used, e.g.:: 86 87 loop.call_soon_threadsafe(fut.cancel) 88 89To schedule a coroutine object from a different OS thread, the 90:func:`run_coroutine_threadsafe` function should be used. It returns a 91:class:`concurrent.futures.Future` to access the result:: 92 93 async def coro_func(): 94 return await asyncio.sleep(1, 42) 95 96 # Later in another OS thread: 97 98 future = asyncio.run_coroutine_threadsafe(coro_func(), loop) 99 # Wait for the result: 100 result = future.result() 101 102To handle signals and to execute subprocesses, the event loop must be 103run in the main thread. 104 105The :meth:`loop.run_in_executor` method can be used with a 106:class:`concurrent.futures.ThreadPoolExecutor` to execute 107blocking code in a different OS thread without blocking the OS thread 108that the event loop runs in. 109 110There is currently no way to schedule coroutines or callbacks directly 111from a different process (such as one started with 112:mod:`multiprocessing`). The :ref:`Event Loop Methods <asyncio-event-loop>` 113section lists APIs that can read from pipes and watch file descriptors 114without blocking the event loop. In addition, asyncio's 115:ref:`Subprocess <asyncio-subprocess>` APIs provide a way to start a 116process and communicate with it from the event loop. Lastly, the 117aforementioned :meth:`loop.run_in_executor` method can also be used 118with a :class:`concurrent.futures.ProcessPoolExecutor` to execute 119code in a different process. 120 121.. _asyncio-handle-blocking: 122 123Running Blocking Code 124===================== 125 126Blocking (CPU-bound) code should not be called directly. For example, 127if a function performs a CPU-intensive calculation for 1 second, 128all concurrent asyncio Tasks and IO operations would be delayed 129by 1 second. 130 131An executor can be used to run a task in a different thread or even in 132a different process to avoid blocking the OS thread with the 133event loop. See the :meth:`loop.run_in_executor` method for more 134details. 135 136 137.. _asyncio-logger: 138 139Logging 140======= 141 142asyncio uses the :mod:`logging` module and all logging is performed 143via the ``"asyncio"`` logger. 144 145The default log level is :py:data:`logging.INFO`, which can be easily 146adjusted:: 147 148 logging.getLogger("asyncio").setLevel(logging.WARNING) 149 150 151.. _asyncio-coroutine-not-scheduled: 152 153Detect never-awaited coroutines 154=============================== 155 156When a coroutine function is called, but not awaited 157(e.g. ``coro()`` instead of ``await coro()``) 158or the coroutine is not scheduled with :meth:`asyncio.create_task`, asyncio 159will emit a :exc:`RuntimeWarning`:: 160 161 import asyncio 162 163 async def test(): 164 print("never scheduled") 165 166 async def main(): 167 test() 168 169 asyncio.run(main()) 170 171Output:: 172 173 test.py:7: RuntimeWarning: coroutine 'test' was never awaited 174 test() 175 176Output in debug mode:: 177 178 test.py:7: RuntimeWarning: coroutine 'test' was never awaited 179 Coroutine created at (most recent call last) 180 File "../t.py", line 9, in <module> 181 asyncio.run(main(), debug=True) 182 183 < .. > 184 185 File "../t.py", line 7, in main 186 test() 187 test() 188 189The usual fix is to either await the coroutine or call the 190:meth:`asyncio.create_task` function:: 191 192 async def main(): 193 await test() 194 195 196Detect never-retrieved exceptions 197================================= 198 199If a :meth:`Future.set_exception` is called but the Future object is 200never awaited on, the exception would never be propagated to the 201user code. In this case, asyncio would emit a log message when the 202Future object is garbage collected. 203 204Example of an unhandled exception:: 205 206 import asyncio 207 208 async def bug(): 209 raise Exception("not consumed") 210 211 async def main(): 212 asyncio.create_task(bug()) 213 214 asyncio.run(main()) 215 216Output:: 217 218 Task exception was never retrieved 219 future: <Task finished coro=<bug() done, defined at test.py:3> 220 exception=Exception('not consumed')> 221 222 Traceback (most recent call last): 223 File "test.py", line 4, in bug 224 raise Exception("not consumed") 225 Exception: not consumed 226 227:ref:`Enable the debug mode <asyncio-debug-mode>` to get the 228traceback where the task was created:: 229 230 asyncio.run(main(), debug=True) 231 232Output in debug mode:: 233 234 Task exception was never retrieved 235 future: <Task finished coro=<bug() done, defined at test.py:3> 236 exception=Exception('not consumed') created at asyncio/tasks.py:321> 237 238 source_traceback: Object created at (most recent call last): 239 File "../t.py", line 9, in <module> 240 asyncio.run(main(), debug=True) 241 242 < .. > 243 244 Traceback (most recent call last): 245 File "../t.py", line 4, in bug 246 raise Exception("not consumed") 247 Exception: not consumed 248