Lines Matching full:process
1 :mod:`multiprocessing` --- Process-based parallelism
5 :synopsis: Process-based parallelism.
51 to push tasks to a background process without blocking execution of the
52 calling process. Compared to using the :class:`~multiprocessing.pool.Pool`
54 the submission of work to the underlying process pool to be separated from
58 The :class:`Process` class
61 In :mod:`multiprocessing`, processes are spawned by creating a :class:`Process`
62 object and then calling its :meth:`~Process.start` method. :class:`Process`
66 from multiprocessing import Process
72 p = Process(target=f, args=('bob',))
76 To show the individual process IDs involved, here is an expanded example::
78 from multiprocessing import Process
84 print('parent process:', os.getppid())
85 print('process id:', os.getpid())
93 p = Process(target=f, args=('bob',))
108 to start a process. These *start methods* are
111 The parent process starts a fresh Python interpreter process. The
112 child process will only inherit those resources necessary to run
113 the process object's :meth:`~Process.run` method. In particular,
114 unnecessary file descriptors and handles from the parent process
115 will not be inherited. Starting a process using this method is
121 The parent process uses :func:`os.fork` to fork the Python
122 interpreter. The child process, when it begins, is effectively
123 identical to the parent process. All resources of the parent are
124 inherited by the child process. Note that safely forking a
125 multithreaded process is problematic.
131 a server process is started. From then on, whenever a new process
132 is needed, the parent process connects to the server and requests
133 that it fork a new process. The fork server process is single
153 start a *resource tracker* process which tracks the unlinked named
158 Usually there should be none, but if a process was killed by a signal
177 p = mp.Process(target=foo, args=(q,))
198 p = ctx.Process(target=foo, args=(q,))
231 from multiprocessing import Process, Queue
238 p = Process(target=f, args=(q,))
243 Queues are thread and process safe.
250 from multiprocessing import Process, Pipe
258 p = Process(target=f, args=(child_conn,))
277 that only one process prints to standard output at a time::
279 from multiprocessing import Process, Lock
292 Process(target=f, args=(lock, num)).start()
313 from multiprocessing import Process, Value, Array
324 p = Process(target=f, args=(num, arr))
339 objects will be process and thread-safe.
345 **Server process**
347 A manager object returned by :func:`Manager` controls a server process which
357 from multiprocessing import Process, Manager
370 p = Process(target=f, args=(d, l))
382 Server process managers are more flexible than using shared memory objects
416 res = pool.apply_async(f, (20,)) # runs in *only* one process
420 res = pool.apply_async(os.getpid, ()) # runs in *only* one process
421 print(res.get(timeout=1)) # prints the PID of that process
440 process which created it.
457 Process PoolWorker-1:
458 Process PoolWorker-2:
459 Process PoolWorker-3:
469 stop the parent process somehow.)
479 :class:`Process` and exceptions
482 .. class:: Process(group=None, target=None, name=None, args=(), kwargs={}, \
485 Process objects represent activity that is run in a separate process. The
486 :class:`Process` class has equivalents of all the methods of
493 called. *name* is the process name (see :attr:`name` for more details).
496 the keyword-only *daemon* argument sets the process :attr:`daemon` flag
498 inherited from the creating process.
505 base class constructor (:meth:`Process.__init__`) before doing anything else
506 to the process.
513 Method representing the process's activity.
520 Using a list or tuple as the *args* argument passed to :class:`Process`
525 >>> from multiprocessing import Process
526 >>> p = Process(target=print, args=[1])
529 >>> p = Process(target=print, args=(1,))
535 Start the process's activity.
537 This must be called at most once per process object. It arranges for the
538 object's :meth:`run` method to be invoked in a separate process.
543 blocks until the process whose :meth:`join` method is called terminates.
545 Note that the method returns ``None`` if its process terminates or if the
546 method times out. Check the process's :attr:`exitcode` to determine if
549 A process can be joined many times.
551 A process cannot join itself because this would cause a deadlock. It is
552 an error to attempt to join a process before it has been started.
556 The process's name. The name is a string used for identification purposes
562 'Process-N\ :sub:`1`:N\ :sub:`2`:...:N\ :sub:`k`' is constructed, where
567 Return whether the process is alive.
569 Roughly, a process object is alive from the moment the :meth:`start`
570 method returns until the child process terminates.
574 The process's daemon flag, a Boolean value. This must be set before
577 The initial value is inherited from the creating process.
579 When a process exits, it attempts to terminate all of its daemonic child
582 Note that a daemonic process is not allowed to create child processes.
583 Otherwise a daemonic process would leave its children orphaned if it gets
584 terminated when its parent process exits. Additionally, these are **not**
588 In addition to the :class:`threading.Thread` API, :class:`Process` objects
593 Return the process ID. Before the process is spawned, this will be
598 The child's exit code. This will be ``None`` if the process has not yet
611 The process's authentication key (a byte string).
613 When :mod:`multiprocessing` is initialized the main process is assigned a
616 When a :class:`Process` object is created, it will inherit the
617 authentication key of its parent process, although this may be changed by
625 the process ends.
639 Terminate the process. On Unix this is done using the ``SIGTERM`` signal;
643 Note that descendant processes of the process will *not* be terminated --
648 If this method is used when the associated process is using a pipe or
650 become unusable by other process. Similarly, if the process has
662 Close the :class:`Process` object, releasing all resources associated
663 with it. :exc:`ValueError` is raised if the underlying process
665 other methods and attributes of the :class:`Process` object will
672 the process that created the process object.
674 Example usage of some of the methods of :class:`Process`:
679 >>> p = multiprocessing.Process(target=time.sleep, args=(1000,))
681 <Process ... initial> False
684 <Process ... started> True
688 <Process ... stopped exitcode=-SIGTERM> False
760 However, objects enqueued by the same process will always be in
765 If a process is killed using :meth:`Process.terminate` or :func:`os.kill`
767 likely to become corrupted. This may cause any other process to get an
772 As mentioned above, if a child process has put items on a queue (and it has
774 <multiprocessing.Queue.cancel_join_thread>`), then that process will
777 This means that if you try joining that process you may get a deadlock unless
779 consumed. Similarly, if the child process is non-daemonic then the parent
780 process may hang on exit when it tries to join all its non-daemonic children.
803 Returns a process shared queue implemented using a pipe and a few
804 locks/semaphores. When a process first puts an item on the queue a feeder
875 process. The background thread will quit once it has flushed all buffered
885 By default if a process is not the creator of the queue then on exit it
886 will attempt to join the queue's background thread. The process can call
892 the background thread from being joined automatically when the process
898 It is really only there if you need the current process to exit
974 Return list of all live children of the current process.
983 This number is not equivalent to the number of CPUs the current process can
995 Return the :class:`Process` object corresponding to the current process.
1001 Return the :class:`Process` object corresponding to the parent process of
1002 the :func:`current_process`. For the main process, ``parent_process`` will
1016 from multiprocessing import Process, freeze_support
1023 Process(target=f).start()
1078 Set the path of the Python interpreter to use when starting a child process.
1242 receives, which can be a security risk unless you can trust the process
1252 If a process is killed while it is trying to read or write to a pipe then
1305 Once a process or thread has acquired a lock, subsequent attempts to
1306 acquire it from any process or thread will block until it is released;
1307 any process or thread may release it. The concepts and behaviors of
1347 Release a lock. This can be called from any process or thread, not only
1348 the process or thread which originally acquired the lock.
1357 recursive lock must be released by the process or thread that acquired it.
1358 Once a process or thread has acquired a recursive lock, the same process
1359 or thread may acquire it again without blocking; that process or thread
1375 lock is in an unlocked state (not owned by any process or thread) unless
1376 the lock is already owned by the current process or thread. The current
1377 process or thread then takes ownership of the lock (if it does not
1386 process or thread, the current process or thread does not take ownership
1389 current process or thread takes ownership and the recursion level is
1401 owned by any process or thread) and if any other processes or threads
1404 nonzero, the lock remains locked and owned by the calling process or
1407 Only call this method when the calling process or thread owns the lock.
1408 An :exc:`AssertionError` is raised if this method is called by a process
1467 by a lock, so it will not necessarily be "process-safe".
1500 "process-safe".
1521 this will refer to a location in the address space of a specific process.
1523 process and trying to dereference the pointer from the second process may
1560 process-safe synchronization wrapper may be returned instead of a raw ctypes
1569 "process-safe".
1576 process-safe synchronization wrapper may be returned instead of a raw ctypes
1584 "process-safe".
1595 Return a process-safe wrapper object for a ctypes object which uses *lock* to
1625 process::
1627 from multiprocessing import Process, Lock
1650 p = Process(target=modify, args=(n, x, s, A))
1679 different machines. A manager object controls a server process which manages
1688 object corresponds to a spawned child process and has methods which will
1692 :synopsis: Share data between process with shared objects.
1695 their parent process exits. The manager classes are defined in the
1703 that the manager object refers to a started manager process.
1705 *address* is the address on which the manager process listens for new
1709 validity of incoming connections to the server process. If
1719 *shutdown_timeout* is a timeout in seconds used to wait until the process
1721 shutdown times out, the process is terminated. If terminating the process
1722 also times out, the process is killed.
1747 Connect a local manager object to a remote manager process::
1755 Stop the process used by the manager. This is only available if
1756 :meth:`start` has been used to start the server process.
1795 *typeid* which can be used to tell the server process to create a new
1807 server process (if it has not already started) and then returns the
1811 manager's server process if it was not already started.
1987 >>> from multiprocessing import Process, Queue
1989 >>> class Worker(Process):
2012 in a different process. The shared object is said to be the *referent* of the
2117 in the manager's process.
2125 process then this is converted into a :exc:`RemoteError` exception and is
2166 A shared object gets deleted from the manager process when there are no longer
2170 Process Pools
2181 A process pool object which controls a pool of worker processes to which jobs
2188 If *initializer* is not ``None`` then each worker process will call
2191 *maxtasksperchild* is the number of tasks a worker process can complete
2192 before it will exit and be replaced with a fresh worker process, to enable
2203 the process which created the pool.
2209 can lead to the process hanging on finalization.
2228 process spawned to replace the old one. The *maxtasksperchild*
2262 the process pool as separate tasks. The (approximate) size of these
2304 only one worker process is the order guaranteed to be "correct".)
2386 result = pool.apply_async(f, (10,)) # evaluate "f(10)" asynchronously in a single process
2528 * the :attr:`~multiprocessing.Process.sentinel` attribute of a
2529 :class:`~multiprocessing.Process` object.
2592 from multiprocessing import Process, Pipe, current_process
2606 p = Process(target=foo, args=(w,))
2609 # p is the only process which owns a handle for it. This
2663 :class:`~multiprocessing.Process`). This value will be automatically inherited by
2664 any :class:`~multiprocessing.Process` object that the current process creates.
2665 This means that (by default) all processes of a multi-process program will share
2676 package does not use process shared locks so it is possible (depending on the
2690 parent process's logger -- any other customization of the logger will not be
2710 [INFO/SyncManager-...] child process calling self.run()
2749 If *initializer* is not ``None`` then each worker process will call
2806 On Unix when a process finishes but has not been joined it becomes a zombie.
2807 There should never be very many because each time a new process starts (or
2810 process's :meth:`Process.is_alive <multiprocessing.Process.is_alive>` will
2811 join the process. Even so it is probably good
2820 Instead you should arrange the program so that a process which
2822 from an ancestor process.
2826 Using the :meth:`Process.terminate <multiprocessing.Process.terminate>`
2827 method to stop a process is liable to
2829 currently being used by the process to become broken or unavailable to other
2833 :meth:`Process.terminate <multiprocessing.Process.terminate>` on processes
2838 Bear in mind that a process that has put items in a queue will wait before
2840 the underlying pipe. (The child process can call the
2846 process is joined. Otherwise you cannot be sure that processes which have
2852 from multiprocessing import Process, Queue
2859 p = Process(target=f, args=(queue,))
2869 On Unix using the *fork* start method, a child process can make
2870 use of a shared resource created in a parent process using a
2872 argument to the constructor for the child process.
2876 child process is still alive the object will not be garbage
2877 collected in the parent process. This might be important if some
2879 parent process.
2883 from multiprocessing import Process, Lock
2891 Process(target=f).start()
2895 from multiprocessing import Process, Lock
2903 Process(target=f, args=(lock,)).start()
2911 in the :meth:`multiprocessing.Process._bootstrap` method --- this resulted
2946 Ensure that all arguments to :meth:`Process.__init__` are picklable.
2947 Also, if you subclass :class:`~multiprocessing.Process` then make sure that
2948 instances will be picklable when the :meth:`Process.start
2949 <multiprocessing.Process.start>` method is called.
2953 Bear in mind that if code run in a child process tries to access a global
2955 in the parent process at the time that :meth:`Process.start
2956 <multiprocessing.Process.start>` was called.
2967 process).
2973 from multiprocessing import Process
2978 p = Process(target=foo)
2984 from multiprocessing import Process, freeze_support, set_start_method
2992 p = Process(target=foo)