1.. currentmodule:: asyncio 2 3.. _asyncio-subprocess: 4 5============ 6Subprocesses 7============ 8 9**Source code:** :source:`Lib/asyncio/subprocess.py`, 10:source:`Lib/asyncio/base_subprocess.py` 11 12---------------------------------------- 13 14This section describes high-level async/await asyncio APIs to 15create and manage subprocesses. 16 17.. _asyncio_example_subprocess_shell: 18 19Here's an example of how asyncio can run a shell command and 20obtain its result:: 21 22 import asyncio 23 24 async def run(cmd): 25 proc = await asyncio.create_subprocess_shell( 26 cmd, 27 stdout=asyncio.subprocess.PIPE, 28 stderr=asyncio.subprocess.PIPE) 29 30 stdout, stderr = await proc.communicate() 31 32 print(f'[{cmd!r} exited with {proc.returncode}]') 33 if stdout: 34 print(f'[stdout]\n{stdout.decode()}') 35 if stderr: 36 print(f'[stderr]\n{stderr.decode()}') 37 38 asyncio.run(run('ls /zzz')) 39 40will print:: 41 42 ['ls /zzz' exited with 1] 43 [stderr] 44 ls: /zzz: No such file or directory 45 46Because all asyncio subprocess functions are asynchronous and asyncio 47provides many tools to work with such functions, it is easy to execute 48and monitor multiple subprocesses in parallel. It is indeed trivial 49to modify the above example to run several commands simultaneously:: 50 51 async def main(): 52 await asyncio.gather( 53 run('ls /zzz'), 54 run('sleep 1; echo "hello"')) 55 56 asyncio.run(main()) 57 58See also the `Examples`_ subsection. 59 60 61Creating Subprocesses 62===================== 63 64.. coroutinefunction:: create_subprocess_exec(program, *args, stdin=None, \ 65 stdout=None, stderr=None, limit=None, **kwds) 66 67 Create a subprocess. 68 69 The *limit* argument sets the buffer limit for :class:`StreamReader` 70 wrappers for :attr:`Process.stdout` and :attr:`Process.stderr` 71 (if :attr:`subprocess.PIPE` is passed to *stdout* and *stderr* arguments). 72 73 Return a :class:`~asyncio.subprocess.Process` instance. 74 75 See the documentation of :meth:`loop.subprocess_exec` for other 76 parameters. 77 78 .. versionchanged:: 3.10 79 Removed the *loop* parameter. 80 81 82.. coroutinefunction:: create_subprocess_shell(cmd, stdin=None, \ 83 stdout=None, stderr=None, limit=None, **kwds) 84 85 Run the *cmd* shell command. 86 87 The *limit* argument sets the buffer limit for :class:`StreamReader` 88 wrappers for :attr:`Process.stdout` and :attr:`Process.stderr` 89 (if :attr:`subprocess.PIPE` is passed to *stdout* and *stderr* arguments). 90 91 Return a :class:`~asyncio.subprocess.Process` instance. 92 93 See the documentation of :meth:`loop.subprocess_shell` for other 94 parameters. 95 96 .. important:: 97 98 It is the application's responsibility to ensure that all whitespace and 99 special characters are quoted appropriately to avoid `shell injection 100 <https://en.wikipedia.org/wiki/Shell_injection#Shell_injection>`_ 101 vulnerabilities. The :func:`shlex.quote` function can be used to properly 102 escape whitespace and special shell characters in strings that are going 103 to be used to construct shell commands. 104 105 .. versionchanged:: 3.10 106 Removed the *loop* parameter. 107 108.. note:: 109 110 Subprocesses are available for Windows if a :class:`ProactorEventLoop` is 111 used. See :ref:`Subprocess Support on Windows <asyncio-windows-subprocess>` 112 for details. 113 114.. seealso:: 115 116 asyncio also has the following *low-level* APIs to work with subprocesses: 117 :meth:`loop.subprocess_exec`, :meth:`loop.subprocess_shell`, 118 :meth:`loop.connect_read_pipe`, :meth:`loop.connect_write_pipe`, 119 as well as the :ref:`Subprocess Transports <asyncio-subprocess-transports>` 120 and :ref:`Subprocess Protocols <asyncio-subprocess-protocols>`. 121 122 123Constants 124========= 125 126.. data:: asyncio.subprocess.PIPE 127 128 Can be passed to the *stdin*, *stdout* or *stderr* parameters. 129 130 If *PIPE* is passed to *stdin* argument, the 131 :attr:`Process.stdin <asyncio.subprocess.Process.stdin>` attribute 132 will point to a :class:`StreamWriter` instance. 133 134 If *PIPE* is passed to *stdout* or *stderr* arguments, the 135 :attr:`Process.stdout <asyncio.subprocess.Process.stdout>` and 136 :attr:`Process.stderr <asyncio.subprocess.Process.stderr>` 137 attributes will point to :class:`StreamReader` instances. 138 139.. data:: asyncio.subprocess.STDOUT 140 141 Special value that can be used as the *stderr* argument and indicates 142 that standard error should be redirected into standard output. 143 144.. data:: asyncio.subprocess.DEVNULL 145 146 Special value that can be used as the *stdin*, *stdout* or *stderr* argument 147 to process creation functions. It indicates that the special file 148 :data:`os.devnull` will be used for the corresponding subprocess stream. 149 150 151Interacting with Subprocesses 152============================= 153 154Both :func:`create_subprocess_exec` and :func:`create_subprocess_shell` 155functions return instances of the *Process* class. *Process* is a high-level 156wrapper that allows communicating with subprocesses and watching for 157their completion. 158 159.. class:: asyncio.subprocess.Process 160 161 An object that wraps OS processes created by the 162 :func:`create_subprocess_exec` and :func:`create_subprocess_shell` 163 functions. 164 165 This class is designed to have a similar API to the 166 :class:`subprocess.Popen` class, but there are some 167 notable differences: 168 169 * unlike Popen, Process instances do not have an equivalent to 170 the :meth:`~subprocess.Popen.poll` method; 171 172 * the :meth:`~asyncio.subprocess.Process.communicate` and 173 :meth:`~asyncio.subprocess.Process.wait` methods don't have a 174 *timeout* parameter: use the :func:`wait_for` function; 175 176 * the :meth:`Process.wait() <asyncio.subprocess.Process.wait>` method 177 is asynchronous, whereas :meth:`subprocess.Popen.wait` method 178 is implemented as a blocking busy loop; 179 180 * the *universal_newlines* parameter is not supported. 181 182 This class is :ref:`not thread safe <asyncio-multithreading>`. 183 184 See also the :ref:`Subprocess and Threads <asyncio-subprocess-threads>` 185 section. 186 187 .. coroutinemethod:: wait() 188 189 Wait for the child process to terminate. 190 191 Set and return the :attr:`returncode` attribute. 192 193 .. note:: 194 195 This method can deadlock when using ``stdout=PIPE`` or 196 ``stderr=PIPE`` and the child process generates so much output 197 that it blocks waiting for the OS pipe buffer to accept 198 more data. Use the :meth:`communicate` method when using pipes 199 to avoid this condition. 200 201 .. coroutinemethod:: communicate(input=None) 202 203 Interact with process: 204 205 1. send data to *stdin* (if *input* is not ``None``); 206 2. read data from *stdout* and *stderr*, until EOF is reached; 207 3. wait for process to terminate. 208 209 The optional *input* argument is the data (:class:`bytes` object) 210 that will be sent to the child process. 211 212 Return a tuple ``(stdout_data, stderr_data)``. 213 214 If either :exc:`BrokenPipeError` or :exc:`ConnectionResetError` 215 exception is raised when writing *input* into *stdin*, the 216 exception is ignored. This condition occurs when the process 217 exits before all data are written into *stdin*. 218 219 If it is desired to send data to the process' *stdin*, 220 the process needs to be created with ``stdin=PIPE``. Similarly, 221 to get anything other than ``None`` in the result tuple, the 222 process has to be created with ``stdout=PIPE`` and/or 223 ``stderr=PIPE`` arguments. 224 225 Note, that the data read is buffered in memory, so do not use 226 this method if the data size is large or unlimited. 227 228 .. method:: send_signal(signal) 229 230 Sends the signal *signal* to the child process. 231 232 .. note:: 233 234 On Windows, :py:data:`SIGTERM` is an alias for :meth:`terminate`. 235 ``CTRL_C_EVENT`` and ``CTRL_BREAK_EVENT`` can be sent to processes 236 started with a *creationflags* parameter which includes 237 ``CREATE_NEW_PROCESS_GROUP``. 238 239 .. method:: terminate() 240 241 Stop the child process. 242 243 On POSIX systems this method sends :py:data:`signal.SIGTERM` to the 244 child process. 245 246 On Windows the Win32 API function :c:func:`TerminateProcess` is 247 called to stop the child process. 248 249 .. method:: kill() 250 251 Kill the child process. 252 253 On POSIX systems this method sends :py:data:`SIGKILL` to the child 254 process. 255 256 On Windows this method is an alias for :meth:`terminate`. 257 258 .. attribute:: stdin 259 260 Standard input stream (:class:`StreamWriter`) or ``None`` 261 if the process was created with ``stdin=None``. 262 263 .. attribute:: stdout 264 265 Standard output stream (:class:`StreamReader`) or ``None`` 266 if the process was created with ``stdout=None``. 267 268 .. attribute:: stderr 269 270 Standard error stream (:class:`StreamReader`) or ``None`` 271 if the process was created with ``stderr=None``. 272 273 .. warning:: 274 275 Use the :meth:`communicate` method rather than 276 :attr:`process.stdin.write() <stdin>`, 277 :attr:`await process.stdout.read() <stdout>` or 278 :attr:`await process.stderr.read <stderr>`. 279 This avoids deadlocks due to streams pausing reading or writing 280 and blocking the child process. 281 282 .. attribute:: pid 283 284 Process identification number (PID). 285 286 Note that for processes created by the :func:`create_subprocess_shell` 287 function, this attribute is the PID of the spawned shell. 288 289 .. attribute:: returncode 290 291 Return code of the process when it exits. 292 293 A ``None`` value indicates that the process has not terminated yet. 294 295 A negative value ``-N`` indicates that the child was terminated 296 by signal ``N`` (POSIX only). 297 298 299.. _asyncio-subprocess-threads: 300 301Subprocess and Threads 302---------------------- 303 304Standard asyncio event loop supports running subprocesses from different threads by 305default. 306 307On Windows subprocesses are provided by :class:`ProactorEventLoop` only (default), 308:class:`SelectorEventLoop` has no subprocess support. 309 310On UNIX *child watchers* are used for subprocess finish waiting, see 311:ref:`asyncio-watchers` for more info. 312 313 314.. versionchanged:: 3.8 315 316 UNIX switched to use :class:`ThreadedChildWatcher` for spawning subprocesses from 317 different threads without any limitation. 318 319 Spawning a subprocess with *inactive* current child watcher raises 320 :exc:`RuntimeError`. 321 322Note that alternative event loop implementations might have own limitations; 323please refer to their documentation. 324 325.. seealso:: 326 327 The :ref:`Concurrency and multithreading in asyncio 328 <asyncio-multithreading>` section. 329 330 331Examples 332-------- 333 334An example using the :class:`~asyncio.subprocess.Process` class to 335control a subprocess and the :class:`StreamReader` class to read from 336its standard output. 337 338.. _asyncio_example_create_subprocess_exec: 339 340The subprocess is created by the :func:`create_subprocess_exec` 341function:: 342 343 import asyncio 344 import sys 345 346 async def get_date(): 347 code = 'import datetime; print(datetime.datetime.now())' 348 349 # Create the subprocess; redirect the standard output 350 # into a pipe. 351 proc = await asyncio.create_subprocess_exec( 352 sys.executable, '-c', code, 353 stdout=asyncio.subprocess.PIPE) 354 355 # Read one line of output. 356 data = await proc.stdout.readline() 357 line = data.decode('ascii').rstrip() 358 359 # Wait for the subprocess exit. 360 await proc.wait() 361 return line 362 363 date = asyncio.run(get_date()) 364 print(f"Current date: {date}") 365 366 367See also the :ref:`same example <asyncio_example_subprocess_proto>` 368written using low-level APIs. 369