1.. currentmodule:: asyncio 2 3.. _asyncio-subprocess: 4 5============ 6Subprocesses 7============ 8 9**Source code:** :source:`Lib/asyncio/subprocess.py`, 10:source:`Lib/asyncio/base_subprocess.py` 11 12---------------------------------------- 13 14This section describes high-level async/await asyncio APIs to 15create and manage subprocesses. 16 17.. _asyncio_example_subprocess_shell: 18 19Here's an example of how asyncio can run a shell command and 20obtain its result:: 21 22 import asyncio 23 24 async def run(cmd): 25 proc = await asyncio.create_subprocess_shell( 26 cmd, 27 stdout=asyncio.subprocess.PIPE, 28 stderr=asyncio.subprocess.PIPE) 29 30 stdout, stderr = await proc.communicate() 31 32 print(f'[{cmd!r} exited with {proc.returncode}]') 33 if stdout: 34 print(f'[stdout]\n{stdout.decode()}') 35 if stderr: 36 print(f'[stderr]\n{stderr.decode()}') 37 38 asyncio.run(run('ls /zzz')) 39 40will print:: 41 42 ['ls /zzz' exited with 1] 43 [stderr] 44 ls: /zzz: No such file or directory 45 46Because all asyncio subprocess functions are asynchronous and asyncio 47provides many tools to work with such functions, it is easy to execute 48and monitor multiple subprocesses in parallel. It is indeed trivial 49to modify the above example to run several commands simultaneously:: 50 51 async def main(): 52 await asyncio.gather( 53 run('ls /zzz'), 54 run('sleep 1; echo "hello"')) 55 56 asyncio.run(main()) 57 58See also the `Examples`_ subsection. 59 60 61Creating Subprocesses 62===================== 63 64.. coroutinefunction:: create_subprocess_exec(program, \*args, stdin=None, \ 65 stdout=None, stderr=None, loop=None, \ 66 limit=None, \*\*kwds) 67 68 Create a subprocess. 69 70 The *limit* argument sets the buffer limit for :class:`StreamReader` 71 wrappers for :attr:`Process.stdout` and :attr:`Process.stderr` 72 (if :attr:`subprocess.PIPE` is passed to *stdout* and *stderr* arguments). 73 74 Return a :class:`~asyncio.subprocess.Process` instance. 75 76 See the documentation of :meth:`loop.subprocess_exec` for other 77 parameters. 78 79 .. deprecated-removed:: 3.8 3.10 80 81 The *loop* parameter. 82 83.. coroutinefunction:: create_subprocess_shell(cmd, stdin=None, \ 84 stdout=None, stderr=None, loop=None, \ 85 limit=None, \*\*kwds) 86 87 Run the *cmd* shell command. 88 89 The *limit* argument sets the buffer limit for :class:`StreamReader` 90 wrappers for :attr:`Process.stdout` and :attr:`Process.stderr` 91 (if :attr:`subprocess.PIPE` is passed to *stdout* and *stderr* arguments). 92 93 Return a :class:`~asyncio.subprocess.Process` instance. 94 95 See the documentation of :meth:`loop.subprocess_shell` for other 96 parameters. 97 98 .. important:: 99 100 It is the application's responsibility to ensure that all whitespace and 101 special characters are quoted appropriately to avoid `shell injection 102 <https://en.wikipedia.org/wiki/Shell_injection#Shell_injection>`_ 103 vulnerabilities. The :func:`shlex.quote` function can be used to properly 104 escape whitespace and special shell characters in strings that are going 105 to be used to construct shell commands. 106 107 .. deprecated-removed:: 3.8 3.10 108 109 The *loop* parameter. 110 111.. note:: 112 113 The default asyncio event loop implementation on **Windows** does not 114 support subprocesses. Subprocesses are available for Windows if a 115 :class:`ProactorEventLoop` is used. 116 See :ref:`Subprocess Support on Windows <asyncio-windows-subprocess>` 117 for details. 118 119.. seealso:: 120 121 asyncio also has the following *low-level* APIs to work with subprocesses: 122 :meth:`loop.subprocess_exec`, :meth:`loop.subprocess_shell`, 123 :meth:`loop.connect_read_pipe`, :meth:`loop.connect_write_pipe`, 124 as well as the :ref:`Subprocess Transports <asyncio-subprocess-transports>` 125 and :ref:`Subprocess Protocols <asyncio-subprocess-protocols>`. 126 127 128Constants 129========= 130 131.. data:: asyncio.subprocess.PIPE 132 133 Can be passed to the *stdin*, *stdout* or *stderr* parameters. 134 135 If *PIPE* is passed to *stdin* argument, the 136 :attr:`Process.stdin <asyncio.subprocess.Process.stdin>` attribute 137 will point to a :class:`StreamWriter` instance. 138 139 If *PIPE* is passed to *stdout* or *stderr* arguments, the 140 :attr:`Process.stdout <asyncio.subprocess.Process.stdout>` and 141 :attr:`Process.stderr <asyncio.subprocess.Process.stderr>` 142 attributes will point to :class:`StreamReader` instances. 143 144.. data:: asyncio.subprocess.STDOUT 145 146 Special value that can be used as the *stderr* argument and indicates 147 that standard error should be redirected into standard output. 148 149.. data:: asyncio.subprocess.DEVNULL 150 151 Special value that can be used as the *stdin*, *stdout* or *stderr* argument 152 to process creation functions. It indicates that the special file 153 :data:`os.devnull` will be used for the corresponding subprocess stream. 154 155 156Interacting with Subprocesses 157============================= 158 159Both :func:`create_subprocess_exec` and :func:`create_subprocess_shell` 160functions return instances of the *Process* class. *Process* is a high-level 161wrapper that allows communicating with subprocesses and watching for 162their completion. 163 164.. class:: asyncio.subprocess.Process 165 166 An object that wraps OS processes created by the 167 :func:`create_subprocess_exec` and :func:`create_subprocess_shell` 168 functions. 169 170 This class is designed to have a similar API to the 171 :class:`subprocess.Popen` class, but there are some 172 notable differences: 173 174 * unlike Popen, Process instances do not have an equivalent to 175 the :meth:`~subprocess.Popen.poll` method; 176 177 * the :meth:`~asyncio.subprocess.Process.communicate` and 178 :meth:`~asyncio.subprocess.Process.wait` methods don't have a 179 *timeout* parameter: use the :func:`wait_for` function; 180 181 * the :meth:`Process.wait() <asyncio.subprocess.Process.wait>` method 182 is asynchronous, whereas :meth:`subprocess.Popen.wait` method 183 is implemented as a blocking busy loop; 184 185 * the *universal_newlines* parameter is not supported. 186 187 This class is :ref:`not thread safe <asyncio-multithreading>`. 188 189 See also the :ref:`Subprocess and Threads <asyncio-subprocess-threads>` 190 section. 191 192 .. coroutinemethod:: wait() 193 194 Wait for the child process to terminate. 195 196 Set and return the :attr:`returncode` attribute. 197 198 .. note:: 199 200 This method can deadlock when using ``stdout=PIPE`` or 201 ``stderr=PIPE`` and the child process generates so much output 202 that it blocks waiting for the OS pipe buffer to accept 203 more data. Use the :meth:`communicate` method when using pipes 204 to avoid this condition. 205 206 .. coroutinemethod:: communicate(input=None) 207 208 Interact with process: 209 210 1. send data to *stdin* (if *input* is not ``None``); 211 2. read data from *stdout* and *stderr*, until EOF is reached; 212 3. wait for process to terminate. 213 214 The optional *input* argument is the data (:class:`bytes` object) 215 that will be sent to the child process. 216 217 Return a tuple ``(stdout_data, stderr_data)``. 218 219 If either :exc:`BrokenPipeError` or :exc:`ConnectionResetError` 220 exception is raised when writing *input* into *stdin*, the 221 exception is ignored. This condition occurs when the process 222 exits before all data are written into *stdin*. 223 224 If it is desired to send data to the process' *stdin*, 225 the process needs to be created with ``stdin=PIPE``. Similarly, 226 to get anything other than ``None`` in the result tuple, the 227 process has to be created with ``stdout=PIPE`` and/or 228 ``stderr=PIPE`` arguments. 229 230 Note, that the data read is buffered in memory, so do not use 231 this method if the data size is large or unlimited. 232 233 .. method:: send_signal(signal) 234 235 Sends the signal *signal* to the child process. 236 237 .. note:: 238 239 On Windows, :py:data:`SIGTERM` is an alias for :meth:`terminate`. 240 ``CTRL_C_EVENT`` and ``CTRL_BREAK_EVENT`` can be sent to processes 241 started with a *creationflags* parameter which includes 242 ``CREATE_NEW_PROCESS_GROUP``. 243 244 .. method:: terminate() 245 246 Stop the child process. 247 248 On POSIX systems this method sends :py:data:`signal.SIGTERM` to the 249 child process. 250 251 On Windows the Win32 API function :c:func:`TerminateProcess` is 252 called to stop the child process. 253 254 .. method:: kill() 255 256 Kill the child. 257 258 On POSIX systems this method sends :py:data:`SIGKILL` to the child 259 process. 260 261 On Windows this method is an alias for :meth:`terminate`. 262 263 .. attribute:: stdin 264 265 Standard input stream (:class:`StreamWriter`) or ``None`` 266 if the process was created with ``stdin=None``. 267 268 .. attribute:: stdout 269 270 Standard output stream (:class:`StreamReader`) or ``None`` 271 if the process was created with ``stdout=None``. 272 273 .. attribute:: stderr 274 275 Standard error stream (:class:`StreamReader`) or ``None`` 276 if the process was created with ``stderr=None``. 277 278 .. warning:: 279 280 Use the :meth:`communicate` method rather than 281 :attr:`process.stdin.write() <stdin>`, 282 :attr:`await process.stdout.read() <stdout>` or 283 :attr:`await process.stderr.read <stderr>`. 284 This avoids deadlocks due to streams pausing reading or writing 285 and blocking the child process. 286 287 .. attribute:: pid 288 289 Process identification number (PID). 290 291 Note that for processes created by the :func:`create_subprocess_shell` 292 function, this attribute is the PID of the spawned shell. 293 294 .. attribute:: returncode 295 296 Return code of the process when it exits. 297 298 A ``None`` value indicates that the process has not terminated yet. 299 300 A negative value ``-N`` indicates that the child was terminated 301 by signal ``N`` (POSIX only). 302 303 304.. _asyncio-subprocess-threads: 305 306Subprocess and Threads 307---------------------- 308 309Standard asyncio event loop supports running subprocesses from different threads by 310default. 311 312On Windows subprocesses are provided by :class:`ProactorEventLoop` only (default), 313:class:`SelectorEventLoop` has no subprocess support. 314 315On UNIX *child watchers* are used for subprocess finish waiting, see 316:ref:`asyncio-watchers` for more info. 317 318 319.. versionchanged:: 3.8 320 321 UNIX switched to use :class:`ThreadedChildWatcher` for spawning subprocesses from 322 different threads without any limitation. 323 324 Spawning a subprocess with *inactive* current child watcher raises 325 :exc:`RuntimeError`. 326 327Note that alternative event loop implementations might have own limitations; 328please refer to their documentation. 329 330.. seealso:: 331 332 The :ref:`Concurrency and multithreading in asyncio 333 <asyncio-multithreading>` section. 334 335 336Examples 337-------- 338 339An example using the :class:`~asyncio.subprocess.Process` class to 340control a subprocess and the :class:`StreamReader` class to read from 341its standard output. 342 343.. _asyncio_example_create_subprocess_exec: 344 345The subprocess is created by the :func:`create_subprocess_exec` 346function:: 347 348 import asyncio 349 import sys 350 351 async def get_date(): 352 code = 'import datetime; print(datetime.datetime.now())' 353 354 # Create the subprocess; redirect the standard output 355 # into a pipe. 356 proc = await asyncio.create_subprocess_exec( 357 sys.executable, '-c', code, 358 stdout=asyncio.subprocess.PIPE) 359 360 # Read one line of output. 361 data = await proc.stdout.readline() 362 line = data.decode('ascii').rstrip() 363 364 # Wait for the subprocess exit. 365 await proc.wait() 366 return line 367 368 date = asyncio.run(get_date()) 369 print(f"Current date: {date}") 370 371 372See also the :ref:`same example <asyncio_example_subprocess_proto>` 373written using low-level APIs. 374