1.. currentmodule:: asyncio 2 3.. _asyncio-subprocess: 4 5============ 6Subprocesses 7============ 8 9**Source code:** :source:`Lib/asyncio/subprocess.py`, 10:source:`Lib/asyncio/base_subprocess.py` 11 12---------------------------------------- 13 14This section describes high-level async/await asyncio APIs to 15create and manage subprocesses. 16 17.. _asyncio_example_subprocess_shell: 18 19Here's an example of how asyncio can run a shell command and 20obtain its result:: 21 22 import asyncio 23 24 async def run(cmd): 25 proc = await asyncio.create_subprocess_shell( 26 cmd, 27 stdout=asyncio.subprocess.PIPE, 28 stderr=asyncio.subprocess.PIPE) 29 30 stdout, stderr = await proc.communicate() 31 32 print(f'[{cmd!r} exited with {proc.returncode}]') 33 if stdout: 34 print(f'[stdout]\n{stdout.decode()}') 35 if stderr: 36 print(f'[stderr]\n{stderr.decode()}') 37 38 asyncio.run(run('ls /zzz')) 39 40will print:: 41 42 ['ls /zzz' exited with 1] 43 [stderr] 44 ls: /zzz: No such file or directory 45 46Because all asyncio subprocess functions are asynchronous and asyncio 47provides many tools to work with such functions, it is easy to execute 48and monitor multiple subprocesses in parallel. It is indeed trivial 49to modify the above example to run several commands simultaneously:: 50 51 async def main(): 52 await asyncio.gather( 53 run('ls /zzz'), 54 run('sleep 1; echo "hello"')) 55 56 asyncio.run(main()) 57 58See also the `Examples`_ subsection. 59 60 61Creating Subprocesses 62===================== 63 64.. coroutinefunction:: create_subprocess_exec(program, \*args, stdin=None, \ 65 stdout=None, stderr=None, loop=None, \ 66 limit=None, \*\*kwds) 67 68 Create a subprocess. 69 70 The *limit* argument sets the buffer limit for :class:`StreamReader` 71 wrappers for :attr:`Process.stdout` and :attr:`Process.stderr` 72 (if :attr:`subprocess.PIPE` is passed to *stdout* and *stderr* arguments). 73 74 Return a :class:`~asyncio.subprocess.Process` instance. 75 76 See the documentation of :meth:`loop.subprocess_exec` for other 77 parameters. 78 79 .. deprecated-removed:: 3.8 3.10 80 81 The *loop* parameter. 82 83.. coroutinefunction:: create_subprocess_shell(cmd, stdin=None, \ 84 stdout=None, stderr=None, loop=None, \ 85 limit=None, \*\*kwds) 86 87 Run the *cmd* shell command. 88 89 The *limit* argument sets the buffer limit for :class:`StreamReader` 90 wrappers for :attr:`Process.stdout` and :attr:`Process.stderr` 91 (if :attr:`subprocess.PIPE` is passed to *stdout* and *stderr* arguments). 92 93 Return a :class:`~asyncio.subprocess.Process` instance. 94 95 See the documentation of :meth:`loop.subprocess_shell` for other 96 parameters. 97 98 .. important:: 99 100 It is the application's responsibility to ensure that all whitespace and 101 special characters are quoted appropriately to avoid `shell injection 102 <https://en.wikipedia.org/wiki/Shell_injection#Shell_injection>`_ 103 vulnerabilities. The :func:`shlex.quote` function can be used to properly 104 escape whitespace and special shell characters in strings that are going 105 to be used to construct shell commands. 106 107 .. deprecated-removed:: 3.8 3.10 108 109 The *loop* parameter. 110 111.. note:: 112 113 Subprocesses are available for Windows if a :class:`ProactorEventLoop` is 114 used. See :ref:`Subprocess Support on Windows <asyncio-windows-subprocess>` 115 for details. 116 117.. seealso:: 118 119 asyncio also has the following *low-level* APIs to work with subprocesses: 120 :meth:`loop.subprocess_exec`, :meth:`loop.subprocess_shell`, 121 :meth:`loop.connect_read_pipe`, :meth:`loop.connect_write_pipe`, 122 as well as the :ref:`Subprocess Transports <asyncio-subprocess-transports>` 123 and :ref:`Subprocess Protocols <asyncio-subprocess-protocols>`. 124 125 126Constants 127========= 128 129.. data:: asyncio.subprocess.PIPE 130 131 Can be passed to the *stdin*, *stdout* or *stderr* parameters. 132 133 If *PIPE* is passed to *stdin* argument, the 134 :attr:`Process.stdin <asyncio.subprocess.Process.stdin>` attribute 135 will point to a :class:`StreamWriter` instance. 136 137 If *PIPE* is passed to *stdout* or *stderr* arguments, the 138 :attr:`Process.stdout <asyncio.subprocess.Process.stdout>` and 139 :attr:`Process.stderr <asyncio.subprocess.Process.stderr>` 140 attributes will point to :class:`StreamReader` instances. 141 142.. data:: asyncio.subprocess.STDOUT 143 144 Special value that can be used as the *stderr* argument and indicates 145 that standard error should be redirected into standard output. 146 147.. data:: asyncio.subprocess.DEVNULL 148 149 Special value that can be used as the *stdin*, *stdout* or *stderr* argument 150 to process creation functions. It indicates that the special file 151 :data:`os.devnull` will be used for the corresponding subprocess stream. 152 153 154Interacting with Subprocesses 155============================= 156 157Both :func:`create_subprocess_exec` and :func:`create_subprocess_shell` 158functions return instances of the *Process* class. *Process* is a high-level 159wrapper that allows communicating with subprocesses and watching for 160their completion. 161 162.. class:: asyncio.subprocess.Process 163 164 An object that wraps OS processes created by the 165 :func:`create_subprocess_exec` and :func:`create_subprocess_shell` 166 functions. 167 168 This class is designed to have a similar API to the 169 :class:`subprocess.Popen` class, but there are some 170 notable differences: 171 172 * unlike Popen, Process instances do not have an equivalent to 173 the :meth:`~subprocess.Popen.poll` method; 174 175 * the :meth:`~asyncio.subprocess.Process.communicate` and 176 :meth:`~asyncio.subprocess.Process.wait` methods don't have a 177 *timeout* parameter: use the :func:`wait_for` function; 178 179 * the :meth:`Process.wait() <asyncio.subprocess.Process.wait>` method 180 is asynchronous, whereas :meth:`subprocess.Popen.wait` method 181 is implemented as a blocking busy loop; 182 183 * the *universal_newlines* parameter is not supported. 184 185 This class is :ref:`not thread safe <asyncio-multithreading>`. 186 187 See also the :ref:`Subprocess and Threads <asyncio-subprocess-threads>` 188 section. 189 190 .. coroutinemethod:: wait() 191 192 Wait for the child process to terminate. 193 194 Set and return the :attr:`returncode` attribute. 195 196 .. note:: 197 198 This method can deadlock when using ``stdout=PIPE`` or 199 ``stderr=PIPE`` and the child process generates so much output 200 that it blocks waiting for the OS pipe buffer to accept 201 more data. Use the :meth:`communicate` method when using pipes 202 to avoid this condition. 203 204 .. coroutinemethod:: communicate(input=None) 205 206 Interact with process: 207 208 1. send data to *stdin* (if *input* is not ``None``); 209 2. read data from *stdout* and *stderr*, until EOF is reached; 210 3. wait for process to terminate. 211 212 The optional *input* argument is the data (:class:`bytes` object) 213 that will be sent to the child process. 214 215 Return a tuple ``(stdout_data, stderr_data)``. 216 217 If either :exc:`BrokenPipeError` or :exc:`ConnectionResetError` 218 exception is raised when writing *input* into *stdin*, the 219 exception is ignored. This condition occurs when the process 220 exits before all data are written into *stdin*. 221 222 If it is desired to send data to the process' *stdin*, 223 the process needs to be created with ``stdin=PIPE``. Similarly, 224 to get anything other than ``None`` in the result tuple, the 225 process has to be created with ``stdout=PIPE`` and/or 226 ``stderr=PIPE`` arguments. 227 228 Note, that the data read is buffered in memory, so do not use 229 this method if the data size is large or unlimited. 230 231 .. method:: send_signal(signal) 232 233 Sends the signal *signal* to the child process. 234 235 .. note:: 236 237 On Windows, :py:data:`SIGTERM` is an alias for :meth:`terminate`. 238 ``CTRL_C_EVENT`` and ``CTRL_BREAK_EVENT`` can be sent to processes 239 started with a *creationflags* parameter which includes 240 ``CREATE_NEW_PROCESS_GROUP``. 241 242 .. method:: terminate() 243 244 Stop the child process. 245 246 On POSIX systems this method sends :py:data:`signal.SIGTERM` to the 247 child process. 248 249 On Windows the Win32 API function :c:func:`TerminateProcess` is 250 called to stop the child process. 251 252 .. method:: kill() 253 254 Kill the child. 255 256 On POSIX systems this method sends :py:data:`SIGKILL` to the child 257 process. 258 259 On Windows this method is an alias for :meth:`terminate`. 260 261 .. attribute:: stdin 262 263 Standard input stream (:class:`StreamWriter`) or ``None`` 264 if the process was created with ``stdin=None``. 265 266 .. attribute:: stdout 267 268 Standard output stream (:class:`StreamReader`) or ``None`` 269 if the process was created with ``stdout=None``. 270 271 .. attribute:: stderr 272 273 Standard error stream (:class:`StreamReader`) or ``None`` 274 if the process was created with ``stderr=None``. 275 276 .. warning:: 277 278 Use the :meth:`communicate` method rather than 279 :attr:`process.stdin.write() <stdin>`, 280 :attr:`await process.stdout.read() <stdout>` or 281 :attr:`await process.stderr.read <stderr>`. 282 This avoids deadlocks due to streams pausing reading or writing 283 and blocking the child process. 284 285 .. attribute:: pid 286 287 Process identification number (PID). 288 289 Note that for processes created by the :func:`create_subprocess_shell` 290 function, this attribute is the PID of the spawned shell. 291 292 .. attribute:: returncode 293 294 Return code of the process when it exits. 295 296 A ``None`` value indicates that the process has not terminated yet. 297 298 A negative value ``-N`` indicates that the child was terminated 299 by signal ``N`` (POSIX only). 300 301 302.. _asyncio-subprocess-threads: 303 304Subprocess and Threads 305---------------------- 306 307Standard asyncio event loop supports running subprocesses from different threads by 308default. 309 310On Windows subprocesses are provided by :class:`ProactorEventLoop` only (default), 311:class:`SelectorEventLoop` has no subprocess support. 312 313On UNIX *child watchers* are used for subprocess finish waiting, see 314:ref:`asyncio-watchers` for more info. 315 316 317.. versionchanged:: 3.8 318 319 UNIX switched to use :class:`ThreadedChildWatcher` for spawning subprocesses from 320 different threads without any limitation. 321 322 Spawning a subprocess with *inactive* current child watcher raises 323 :exc:`RuntimeError`. 324 325Note that alternative event loop implementations might have own limitations; 326please refer to their documentation. 327 328.. seealso:: 329 330 The :ref:`Concurrency and multithreading in asyncio 331 <asyncio-multithreading>` section. 332 333 334Examples 335-------- 336 337An example using the :class:`~asyncio.subprocess.Process` class to 338control a subprocess and the :class:`StreamReader` class to read from 339its standard output. 340 341.. _asyncio_example_create_subprocess_exec: 342 343The subprocess is created by the :func:`create_subprocess_exec` 344function:: 345 346 import asyncio 347 import sys 348 349 async def get_date(): 350 code = 'import datetime; print(datetime.datetime.now())' 351 352 # Create the subprocess; redirect the standard output 353 # into a pipe. 354 proc = await asyncio.create_subprocess_exec( 355 sys.executable, '-c', code, 356 stdout=asyncio.subprocess.PIPE) 357 358 # Read one line of output. 359 data = await proc.stdout.readline() 360 line = data.decode('ascii').rstrip() 361 362 # Wait for the subprocess exit. 363 await proc.wait() 364 return line 365 366 date = asyncio.run(get_date()) 367 print(f"Current date: {date}") 368 369 370See also the :ref:`same example <asyncio_example_subprocess_proto>` 371written using low-level APIs. 372