1.. currentmodule:: asyncio 2 3.. _asyncio-subprocess: 4 5============ 6Subprocesses 7============ 8 9**Source code:** :source:`Lib/asyncio/subprocess.py`, 10:source:`Lib/asyncio/base_subprocess.py` 11 12---------------------------------------- 13 14This section describes high-level async/await asyncio APIs to 15create and manage subprocesses. 16 17.. _asyncio_example_subprocess_shell: 18 19Here's an example of how asyncio can run a shell command and 20obtain its result:: 21 22 import asyncio 23 24 async def run(cmd): 25 proc = await asyncio.create_subprocess_shell( 26 cmd, 27 stdout=asyncio.subprocess.PIPE, 28 stderr=asyncio.subprocess.PIPE) 29 30 stdout, stderr = await proc.communicate() 31 32 print(f'[{cmd!r} exited with {proc.returncode}]') 33 if stdout: 34 print(f'[stdout]\n{stdout.decode()}') 35 if stderr: 36 print(f'[stderr]\n{stderr.decode()}') 37 38 asyncio.run(run('ls /zzz')) 39 40will print:: 41 42 ['ls /zzz' exited with 1] 43 [stderr] 44 ls: /zzz: No such file or directory 45 46Because all asyncio subprocess functions are asynchronous and asyncio 47provides many tools to work with such functions, it is easy to execute 48and monitor multiple subprocesses in parallel. It is indeed trivial 49to modify the above example to run several commands simultaneously:: 50 51 async def main(): 52 await asyncio.gather( 53 run('ls /zzz'), 54 run('sleep 1; echo "hello"')) 55 56 asyncio.run(main()) 57 58See also the `Examples`_ subsection. 59 60 61Creating Subprocesses 62===================== 63 64.. coroutinefunction:: create_subprocess_exec(program, *args, stdin=None, \ 65 stdout=None, stderr=None, limit=None, **kwds) 66 67 Create a subprocess. 68 69 The *limit* argument sets the buffer limit for :class:`StreamReader` 70 wrappers for :attr:`Process.stdout` and :attr:`Process.stderr` 71 (if :const:`subprocess.PIPE` is passed to *stdout* and *stderr* arguments). 72 73 Return a :class:`~asyncio.subprocess.Process` instance. 74 75 See the documentation of :meth:`loop.subprocess_exec` for other 76 parameters. 77 78 .. versionchanged:: 3.10 79 Removed the *loop* parameter. 80 81 82.. coroutinefunction:: create_subprocess_shell(cmd, stdin=None, \ 83 stdout=None, stderr=None, limit=None, **kwds) 84 85 Run the *cmd* shell command. 86 87 The *limit* argument sets the buffer limit for :class:`StreamReader` 88 wrappers for :attr:`Process.stdout` and :attr:`Process.stderr` 89 (if :const:`subprocess.PIPE` is passed to *stdout* and *stderr* arguments). 90 91 Return a :class:`~asyncio.subprocess.Process` instance. 92 93 See the documentation of :meth:`loop.subprocess_shell` for other 94 parameters. 95 96 .. important:: 97 98 It is the application's responsibility to ensure that all whitespace and 99 special characters are quoted appropriately to avoid `shell injection 100 <https://en.wikipedia.org/wiki/Shell_injection#Shell_injection>`_ 101 vulnerabilities. The :func:`shlex.quote` function can be used to properly 102 escape whitespace and special shell characters in strings that are going 103 to be used to construct shell commands. 104 105 .. versionchanged:: 3.10 106 Removed the *loop* parameter. 107 108.. note:: 109 110 Subprocesses are available for Windows if a :class:`ProactorEventLoop` is 111 used. See :ref:`Subprocess Support on Windows <asyncio-windows-subprocess>` 112 for details. 113 114.. seealso:: 115 116 asyncio also has the following *low-level* APIs to work with subprocesses: 117 :meth:`loop.subprocess_exec`, :meth:`loop.subprocess_shell`, 118 :meth:`loop.connect_read_pipe`, :meth:`loop.connect_write_pipe`, 119 as well as the :ref:`Subprocess Transports <asyncio-subprocess-transports>` 120 and :ref:`Subprocess Protocols <asyncio-subprocess-protocols>`. 121 122 123Constants 124========= 125 126.. data:: asyncio.subprocess.PIPE 127 :module: 128 129 Can be passed to the *stdin*, *stdout* or *stderr* parameters. 130 131 If *PIPE* is passed to *stdin* argument, the 132 :attr:`Process.stdin <asyncio.subprocess.Process.stdin>` attribute 133 will point to a :class:`StreamWriter` instance. 134 135 If *PIPE* is passed to *stdout* or *stderr* arguments, the 136 :attr:`Process.stdout <asyncio.subprocess.Process.stdout>` and 137 :attr:`Process.stderr <asyncio.subprocess.Process.stderr>` 138 attributes will point to :class:`StreamReader` instances. 139 140.. data:: asyncio.subprocess.STDOUT 141 :module: 142 143 Special value that can be used as the *stderr* argument and indicates 144 that standard error should be redirected into standard output. 145 146.. data:: asyncio.subprocess.DEVNULL 147 :module: 148 149 Special value that can be used as the *stdin*, *stdout* or *stderr* argument 150 to process creation functions. It indicates that the special file 151 :data:`os.devnull` will be used for the corresponding subprocess stream. 152 153 154Interacting with Subprocesses 155============================= 156 157Both :func:`create_subprocess_exec` and :func:`create_subprocess_shell` 158functions return instances of the *Process* class. *Process* is a high-level 159wrapper that allows communicating with subprocesses and watching for 160their completion. 161 162.. class:: asyncio.subprocess.Process 163 :module: 164 165 An object that wraps OS processes created by the 166 :func:`create_subprocess_exec` and :func:`create_subprocess_shell` 167 functions. 168 169 This class is designed to have a similar API to the 170 :class:`subprocess.Popen` class, but there are some 171 notable differences: 172 173 * unlike Popen, Process instances do not have an equivalent to 174 the :meth:`~subprocess.Popen.poll` method; 175 176 * the :meth:`~asyncio.subprocess.Process.communicate` and 177 :meth:`~asyncio.subprocess.Process.wait` methods don't have a 178 *timeout* parameter: use the :func:`~asyncio.wait_for` function; 179 180 * the :meth:`Process.wait() <asyncio.subprocess.Process.wait>` method 181 is asynchronous, whereas :meth:`subprocess.Popen.wait` method 182 is implemented as a blocking busy loop; 183 184 * the *universal_newlines* parameter is not supported. 185 186 This class is :ref:`not thread safe <asyncio-multithreading>`. 187 188 See also the :ref:`Subprocess and Threads <asyncio-subprocess-threads>` 189 section. 190 191 .. coroutinemethod:: wait() 192 193 Wait for the child process to terminate. 194 195 Set and return the :attr:`returncode` attribute. 196 197 .. note:: 198 199 This method can deadlock when using ``stdout=PIPE`` or 200 ``stderr=PIPE`` and the child process generates so much output 201 that it blocks waiting for the OS pipe buffer to accept 202 more data. Use the :meth:`communicate` method when using pipes 203 to avoid this condition. 204 205 .. coroutinemethod:: communicate(input=None) 206 207 Interact with process: 208 209 1. send data to *stdin* (if *input* is not ``None``); 210 2. closes *stdin*; 211 3. read data from *stdout* and *stderr*, until EOF is reached; 212 4. wait for process to terminate. 213 214 The optional *input* argument is the data (:class:`bytes` object) 215 that will be sent to the child process. 216 217 Return a tuple ``(stdout_data, stderr_data)``. 218 219 If either :exc:`BrokenPipeError` or :exc:`ConnectionResetError` 220 exception is raised when writing *input* into *stdin*, the 221 exception is ignored. This condition occurs when the process 222 exits before all data are written into *stdin*. 223 224 If it is desired to send data to the process' *stdin*, 225 the process needs to be created with ``stdin=PIPE``. Similarly, 226 to get anything other than ``None`` in the result tuple, the 227 process has to be created with ``stdout=PIPE`` and/or 228 ``stderr=PIPE`` arguments. 229 230 Note, that the data read is buffered in memory, so do not use 231 this method if the data size is large or unlimited. 232 233 .. versionchanged:: 3.12 234 235 *stdin* gets closed when `input=None` too. 236 237 .. method:: send_signal(signal) 238 239 Sends the signal *signal* to the child process. 240 241 .. note:: 242 243 On Windows, :py:const:`~signal.SIGTERM` is an alias for :meth:`terminate`. 244 ``CTRL_C_EVENT`` and ``CTRL_BREAK_EVENT`` can be sent to processes 245 started with a *creationflags* parameter which includes 246 ``CREATE_NEW_PROCESS_GROUP``. 247 248 .. method:: terminate() 249 250 Stop the child process. 251 252 On POSIX systems this method sends :py:const:`~signal.SIGTERM` to the 253 child process. 254 255 On Windows the Win32 API function :c:func:`!TerminateProcess` is 256 called to stop the child process. 257 258 .. method:: kill() 259 260 Kill the child process. 261 262 On POSIX systems this method sends :py:data:`SIGKILL` to the child 263 process. 264 265 On Windows this method is an alias for :meth:`terminate`. 266 267 .. attribute:: stdin 268 269 Standard input stream (:class:`StreamWriter`) or ``None`` 270 if the process was created with ``stdin=None``. 271 272 .. attribute:: stdout 273 274 Standard output stream (:class:`StreamReader`) or ``None`` 275 if the process was created with ``stdout=None``. 276 277 .. attribute:: stderr 278 279 Standard error stream (:class:`StreamReader`) or ``None`` 280 if the process was created with ``stderr=None``. 281 282 .. warning:: 283 284 Use the :meth:`communicate` method rather than 285 :attr:`process.stdin.write() <stdin>`, 286 :attr:`await process.stdout.read() <stdout>` or 287 :attr:`await process.stderr.read() <stderr>`. 288 This avoids deadlocks due to streams pausing reading or writing 289 and blocking the child process. 290 291 .. attribute:: pid 292 293 Process identification number (PID). 294 295 Note that for processes created by the :func:`create_subprocess_shell` 296 function, this attribute is the PID of the spawned shell. 297 298 .. attribute:: returncode 299 300 Return code of the process when it exits. 301 302 A ``None`` value indicates that the process has not terminated yet. 303 304 A negative value ``-N`` indicates that the child was terminated 305 by signal ``N`` (POSIX only). 306 307 308.. _asyncio-subprocess-threads: 309 310Subprocess and Threads 311---------------------- 312 313Standard asyncio event loop supports running subprocesses from different threads by 314default. 315 316On Windows subprocesses are provided by :class:`ProactorEventLoop` only (default), 317:class:`SelectorEventLoop` has no subprocess support. 318 319On UNIX *child watchers* are used for subprocess finish waiting, see 320:ref:`asyncio-watchers` for more info. 321 322 323.. versionchanged:: 3.8 324 325 UNIX switched to use :class:`ThreadedChildWatcher` for spawning subprocesses from 326 different threads without any limitation. 327 328 Spawning a subprocess with *inactive* current child watcher raises 329 :exc:`RuntimeError`. 330 331Note that alternative event loop implementations might have own limitations; 332please refer to their documentation. 333 334.. seealso:: 335 336 The :ref:`Concurrency and multithreading in asyncio 337 <asyncio-multithreading>` section. 338 339 340Examples 341-------- 342 343An example using the :class:`~asyncio.subprocess.Process` class to 344control a subprocess and the :class:`StreamReader` class to read from 345its standard output. 346 347.. _asyncio_example_create_subprocess_exec: 348 349The subprocess is created by the :func:`create_subprocess_exec` 350function:: 351 352 import asyncio 353 import sys 354 355 async def get_date(): 356 code = 'import datetime; print(datetime.datetime.now())' 357 358 # Create the subprocess; redirect the standard output 359 # into a pipe. 360 proc = await asyncio.create_subprocess_exec( 361 sys.executable, '-c', code, 362 stdout=asyncio.subprocess.PIPE) 363 364 # Read one line of output. 365 data = await proc.stdout.readline() 366 line = data.decode('ascii').rstrip() 367 368 # Wait for the subprocess exit. 369 await proc.wait() 370 return line 371 372 date = asyncio.run(get_date()) 373 print(f"Current date: {date}") 374 375 376See also the :ref:`same example <asyncio_example_subprocess_proto>` 377written using low-level APIs. 378