1.. highlightlang:: c 2 3 4.. _initialization: 5 6***************************************** 7Initialization, Finalization, and Threads 8***************************************** 9 10 11Initializing and finalizing the interpreter 12=========================================== 13 14 15.. c:function:: void Py_Initialize() 16 17 .. index:: 18 single: Py_SetProgramName() 19 single: PyEval_InitThreads() 20 single: PyEval_ReleaseLock() 21 single: PyEval_AcquireLock() 22 single: modules (in module sys) 23 single: path (in module sys) 24 module: __builtin__ 25 module: __main__ 26 module: sys 27 triple: module; search; path 28 single: PySys_SetArgv() 29 single: PySys_SetArgvEx() 30 single: Py_Finalize() 31 32 Initialize the Python interpreter. In an application embedding Python, this 33 should be called before using any other Python/C API functions; with the 34 exception of :c:func:`Py_SetProgramName`, :c:func:`Py_SetPythonHome`, :c:func:`PyEval_InitThreads`, 35 :c:func:`PyEval_ReleaseLock`, and :c:func:`PyEval_AcquireLock`. This initializes 36 the table of loaded modules (``sys.modules``), and creates the fundamental 37 modules :mod:`__builtin__`, :mod:`__main__` and :mod:`sys`. It also initializes 38 the module search path (``sys.path``). It does not set ``sys.argv``; use 39 :c:func:`PySys_SetArgvEx` for that. This is a no-op when called for a second time 40 (without calling :c:func:`Py_Finalize` first). There is no return value; it is a 41 fatal error if the initialization fails. 42 43 44.. c:function:: void Py_InitializeEx(int initsigs) 45 46 This function works like :c:func:`Py_Initialize` if *initsigs* is ``1``. If 47 *initsigs* is ``0``, it skips initialization registration of signal handlers, which 48 might be useful when Python is embedded. 49 50 .. versionadded:: 2.4 51 52 53.. c:function:: int Py_IsInitialized() 54 55 Return true (nonzero) when the Python interpreter has been initialized, false 56 (zero) if not. After :c:func:`Py_Finalize` is called, this returns false until 57 :c:func:`Py_Initialize` is called again. 58 59 60.. c:function:: void Py_Finalize() 61 62 Undo all initializations made by :c:func:`Py_Initialize` and subsequent use of 63 Python/C API functions, and destroy all sub-interpreters (see 64 :c:func:`Py_NewInterpreter` below) that were created and not yet destroyed since 65 the last call to :c:func:`Py_Initialize`. Ideally, this frees all memory 66 allocated by the Python interpreter. This is a no-op when called for a second 67 time (without calling :c:func:`Py_Initialize` again first). There is no return 68 value; errors during finalization are ignored. 69 70 This function is provided for a number of reasons. An embedding application 71 might want to restart Python without having to restart the application itself. 72 An application that has loaded the Python interpreter from a dynamically 73 loadable library (or DLL) might want to free all memory allocated by Python 74 before unloading the DLL. During a hunt for memory leaks in an application a 75 developer might want to free all memory allocated by Python before exiting from 76 the application. 77 78 **Bugs and caveats:** The destruction of modules and objects in modules is done 79 in random order; this may cause destructors (:meth:`__del__` methods) to fail 80 when they depend on other objects (even functions) or modules. Dynamically 81 loaded extension modules loaded by Python are not unloaded. Small amounts of 82 memory allocated by the Python interpreter may not be freed (if you find a leak, 83 please report it). Memory tied up in circular references between objects is not 84 freed. Some memory allocated by extension modules may not be freed. Some 85 extensions may not work properly if their initialization routine is called more 86 than once; this can happen if an application calls :c:func:`Py_Initialize` and 87 :c:func:`Py_Finalize` more than once. 88 89 90Process-wide parameters 91======================= 92 93 94.. c:function:: void Py_SetProgramName(char *name) 95 96 .. index:: 97 single: Py_Initialize() 98 single: main() 99 single: Py_GetPath() 100 101 This function should be called before :c:func:`Py_Initialize` is called for 102 the first time, if it is called at all. It tells the interpreter the value 103 of the ``argv[0]`` argument to the :c:func:`main` function of the program. 104 This is used by :c:func:`Py_GetPath` and some other functions below to find 105 the Python run-time libraries relative to the interpreter executable. The 106 default value is ``'python'``. The argument should point to a 107 zero-terminated character string in static storage whose contents will not 108 change for the duration of the program's execution. No code in the Python 109 interpreter will change the contents of this storage. 110 111 112.. c:function:: char* Py_GetProgramName() 113 114 .. index:: single: Py_SetProgramName() 115 116 Return the program name set with :c:func:`Py_SetProgramName`, or the default. 117 The returned string points into static storage; the caller should not modify its 118 value. 119 120 121.. c:function:: char* Py_GetPrefix() 122 123 Return the *prefix* for installed platform-independent files. This is derived 124 through a number of complicated rules from the program name set with 125 :c:func:`Py_SetProgramName` and some environment variables; for example, if the 126 program name is ``'/usr/local/bin/python'``, the prefix is ``'/usr/local'``. The 127 returned string points into static storage; the caller should not modify its 128 value. This corresponds to the :makevar:`prefix` variable in the top-level 129 :file:`Makefile` and the ``--prefix`` argument to the :program:`configure` 130 script at build time. The value is available to Python code as ``sys.prefix``. 131 It is only useful on Unix. See also the next function. 132 133 134.. c:function:: char* Py_GetExecPrefix() 135 136 Return the *exec-prefix* for installed platform-*dependent* files. This is 137 derived through a number of complicated rules from the program name set with 138 :c:func:`Py_SetProgramName` and some environment variables; for example, if the 139 program name is ``'/usr/local/bin/python'``, the exec-prefix is 140 ``'/usr/local'``. The returned string points into static storage; the caller 141 should not modify its value. This corresponds to the :makevar:`exec_prefix` 142 variable in the top-level :file:`Makefile` and the ``--exec-prefix`` 143 argument to the :program:`configure` script at build time. The value is 144 available to Python code as ``sys.exec_prefix``. It is only useful on Unix. 145 146 Background: The exec-prefix differs from the prefix when platform dependent 147 files (such as executables and shared libraries) are installed in a different 148 directory tree. In a typical installation, platform dependent files may be 149 installed in the :file:`/usr/local/plat` subtree while platform independent may 150 be installed in :file:`/usr/local`. 151 152 Generally speaking, a platform is a combination of hardware and software 153 families, e.g. Sparc machines running the Solaris 2.x operating system are 154 considered the same platform, but Intel machines running Solaris 2.x are another 155 platform, and Intel machines running Linux are yet another platform. Different 156 major revisions of the same operating system generally also form different 157 platforms. Non-Unix operating systems are a different story; the installation 158 strategies on those systems are so different that the prefix and exec-prefix are 159 meaningless, and set to the empty string. Note that compiled Python bytecode 160 files are platform independent (but not independent from the Python version by 161 which they were compiled!). 162 163 System administrators will know how to configure the :program:`mount` or 164 :program:`automount` programs to share :file:`/usr/local` between platforms 165 while having :file:`/usr/local/plat` be a different filesystem for each 166 platform. 167 168 169.. c:function:: char* Py_GetProgramFullPath() 170 171 .. index:: 172 single: Py_SetProgramName() 173 single: executable (in module sys) 174 175 Return the full program name of the Python executable; this is computed as a 176 side-effect of deriving the default module search path from the program name 177 (set by :c:func:`Py_SetProgramName` above). The returned string points into 178 static storage; the caller should not modify its value. The value is available 179 to Python code as ``sys.executable``. 180 181 182.. c:function:: char* Py_GetPath() 183 184 .. index:: 185 triple: module; search; path 186 single: path (in module sys) 187 188 Return the default module search path; this is computed from the program name 189 (set by :c:func:`Py_SetProgramName` above) and some environment variables. 190 The returned string consists of a series of directory names separated by a 191 platform dependent delimiter character. The delimiter character is ``':'`` 192 on Unix and Mac OS X, ``';'`` on Windows. The returned string points into 193 static storage; the caller should not modify its value. The list 194 :data:`sys.path` is initialized with this value on interpreter startup; it 195 can be (and usually is) modified later to change the search path for loading 196 modules. 197 198 .. XXX should give the exact rules 199 200 201.. c:function:: const char* Py_GetVersion() 202 203 Return the version of this Python interpreter. This is a string that looks 204 something like :: 205 206 "1.5 (#67, Dec 31 1997, 22:34:28) [GCC 2.7.2.2]" 207 208 .. index:: single: version (in module sys) 209 210 The first word (up to the first space character) is the current Python version; 211 the first three characters are the major and minor version separated by a 212 period. The returned string points into static storage; the caller should not 213 modify its value. The value is available to Python code as ``sys.version``. 214 215 216.. c:function:: const char* Py_GetPlatform() 217 218 .. index:: single: platform (in module sys) 219 220 Return the platform identifier for the current platform. On Unix, this is 221 formed from the "official" name of the operating system, converted to lower 222 case, followed by the major revision number; e.g., for Solaris 2.x, which is 223 also known as SunOS 5.x, the value is ``'sunos5'``. On Mac OS X, it is 224 ``'darwin'``. On Windows, it is ``'win'``. The returned string points into 225 static storage; the caller should not modify its value. The value is available 226 to Python code as ``sys.platform``. 227 228 229.. c:function:: const char* Py_GetCopyright() 230 231 Return the official copyright string for the current Python version, for example 232 233 ``'Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam'`` 234 235 .. index:: single: copyright (in module sys) 236 237 The returned string points into static storage; the caller should not modify its 238 value. The value is available to Python code as ``sys.copyright``. 239 240 241.. c:function:: const char* Py_GetCompiler() 242 243 Return an indication of the compiler used to build the current Python version, 244 in square brackets, for example:: 245 246 "[GCC 2.7.2.2]" 247 248 .. index:: single: version (in module sys) 249 250 The returned string points into static storage; the caller should not modify its 251 value. The value is available to Python code as part of the variable 252 ``sys.version``. 253 254 255.. c:function:: const char* Py_GetBuildInfo() 256 257 Return information about the sequence number and build date and time of the 258 current Python interpreter instance, for example :: 259 260 "#67, Aug 1 1997, 22:34:28" 261 262 .. index:: single: version (in module sys) 263 264 The returned string points into static storage; the caller should not modify its 265 value. The value is available to Python code as part of the variable 266 ``sys.version``. 267 268 269.. c:function:: void PySys_SetArgvEx(int argc, char **argv, int updatepath) 270 271 .. index:: 272 single: main() 273 single: Py_FatalError() 274 single: argv (in module sys) 275 276 Set :data:`sys.argv` based on *argc* and *argv*. These parameters are 277 similar to those passed to the program's :c:func:`main` function with the 278 difference that the first entry should refer to the script file to be 279 executed rather than the executable hosting the Python interpreter. If there 280 isn't a script that will be run, the first entry in *argv* can be an empty 281 string. If this function fails to initialize :data:`sys.argv`, a fatal 282 condition is signalled using :c:func:`Py_FatalError`. 283 284 If *updatepath* is zero, this is all the function does. If *updatepath* 285 is non-zero, the function also modifies :data:`sys.path` according to the 286 following algorithm: 287 288 - If the name of an existing script is passed in ``argv[0]``, the absolute 289 path of the directory where the script is located is prepended to 290 :data:`sys.path`. 291 - Otherwise (that is, if *argc* is 0 or ``argv[0]`` doesn't point 292 to an existing file name), an empty string is prepended to 293 :data:`sys.path`, which is the same as prepending the current working 294 directory (``"."``). 295 296 .. note:: 297 It is recommended that applications embedding the Python interpreter 298 for purposes other than executing a single script pass ``0`` as *updatepath*, 299 and update :data:`sys.path` themselves if desired. 300 See `CVE-2008-5983 <https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-5983>`_. 301 302 On versions before 2.6.6, you can achieve the same effect by manually 303 popping the first :data:`sys.path` element after having called 304 :c:func:`PySys_SetArgv`, for example using:: 305 306 PyRun_SimpleString("import sys; sys.path.pop(0)\n"); 307 308 .. versionadded:: 2.6.6 309 310 .. XXX impl. doesn't seem consistent in allowing ``0``/``NULL`` for the params; 311 check w/ Guido. 312 313 314.. c:function:: void PySys_SetArgv(int argc, char **argv) 315 316 This function works like :c:func:`PySys_SetArgvEx` with *updatepath* set to ``1``. 317 318 319.. c:function:: void Py_SetPythonHome(char *home) 320 321 Set the default "home" directory, that is, the location of the standard 322 Python libraries. See :envvar:`PYTHONHOME` for the meaning of the 323 argument string. 324 325 The argument should point to a zero-terminated character string in static 326 storage whose contents will not change for the duration of the program's 327 execution. No code in the Python interpreter will change the contents of 328 this storage. 329 330 331.. c:function:: char* Py_GetPythonHome() 332 333 Return the default "home", that is, the value set by a previous call to 334 :c:func:`Py_SetPythonHome`, or the value of the :envvar:`PYTHONHOME` 335 environment variable if it is set. 336 337 338.. _threads: 339 340Thread State and the Global Interpreter Lock 341============================================ 342 343.. index:: 344 single: GIL 345 single: global interpreter lock 346 single: interpreter lock 347 single: lock, interpreter 348 349The Python interpreter is not fully thread-safe. In order to support 350multi-threaded Python programs, there's a global lock, called the :term:`global 351interpreter lock` or :term:`GIL`, that must be held by the current thread before 352it can safely access Python objects. Without the lock, even the simplest 353operations could cause problems in a multi-threaded program: for example, when 354two threads simultaneously increment the reference count of the same object, the 355reference count could end up being incremented only once instead of twice. 356 357.. index:: single: setcheckinterval() (in module sys) 358 359Therefore, the rule exists that only the thread that has acquired the 360:term:`GIL` may operate on Python objects or call Python/C API functions. 361In order to emulate concurrency of execution, the interpreter regularly 362tries to switch threads (see :func:`sys.setcheckinterval`). The lock is also 363released around potentially blocking I/O operations like reading or writing 364a file, so that other Python threads can run in the meantime. 365 366.. index:: 367 single: PyThreadState 368 single: PyThreadState 369 370The Python interpreter keeps some thread-specific bookkeeping information 371inside a data structure called :c:type:`PyThreadState`. There's also one 372global variable pointing to the current :c:type:`PyThreadState`: it can 373be retrieved using :c:func:`PyThreadState_Get`. 374 375Releasing the GIL from extension code 376------------------------------------- 377 378Most extension code manipulating the :term:`GIL` has the following simple 379structure:: 380 381 Save the thread state in a local variable. 382 Release the global interpreter lock. 383 ... Do some blocking I/O operation ... 384 Reacquire the global interpreter lock. 385 Restore the thread state from the local variable. 386 387This is so common that a pair of macros exists to simplify it:: 388 389 Py_BEGIN_ALLOW_THREADS 390 ... Do some blocking I/O operation ... 391 Py_END_ALLOW_THREADS 392 393.. index:: 394 single: Py_BEGIN_ALLOW_THREADS 395 single: Py_END_ALLOW_THREADS 396 397The :c:macro:`Py_BEGIN_ALLOW_THREADS` macro opens a new block and declares a 398hidden local variable; the :c:macro:`Py_END_ALLOW_THREADS` macro closes the 399block. These two macros are still available when Python is compiled without 400thread support (they simply have an empty expansion). 401 402When thread support is enabled, the block above expands to the following code:: 403 404 PyThreadState *_save; 405 406 _save = PyEval_SaveThread(); 407 ...Do some blocking I/O operation... 408 PyEval_RestoreThread(_save); 409 410.. index:: 411 single: PyEval_RestoreThread() 412 single: PyEval_SaveThread() 413 414Here is how these functions work: the global interpreter lock is used to protect the pointer to the 415current thread state. When releasing the lock and saving the thread state, 416the current thread state pointer must be retrieved before the lock is released 417(since another thread could immediately acquire the lock and store its own thread 418state in the global variable). Conversely, when acquiring the lock and restoring 419the thread state, the lock must be acquired before storing the thread state 420pointer. 421 422.. note:: 423 Calling system I/O functions is the most common use case for releasing 424 the GIL, but it can also be useful before calling long-running computations 425 which don't need access to Python objects, such as compression or 426 cryptographic functions operating over memory buffers. For example, the 427 standard :mod:`zlib` and :mod:`hashlib` modules release the GIL when 428 compressing or hashing data. 429 430 431.. _gilstate: 432 433Non-Python created threads 434-------------------------- 435 436When threads are created using the dedicated Python APIs (such as the 437:mod:`threading` module), a thread state is automatically associated to them 438and the code showed above is therefore correct. However, when threads are 439created from C (for example by a third-party library with its own thread 440management), they don't hold the GIL, nor is there a thread state structure 441for them. 442 443If you need to call Python code from these threads (often this will be part 444of a callback API provided by the aforementioned third-party library), 445you must first register these threads with the interpreter by 446creating a thread state data structure, then acquiring the GIL, and finally 447storing their thread state pointer, before you can start using the Python/C 448API. When you are done, you should reset the thread state pointer, release 449the GIL, and finally free the thread state data structure. 450 451The :c:func:`PyGILState_Ensure` and :c:func:`PyGILState_Release` functions do 452all of the above automatically. The typical idiom for calling into Python 453from a C thread is:: 454 455 PyGILState_STATE gstate; 456 gstate = PyGILState_Ensure(); 457 458 /* Perform Python actions here. */ 459 result = CallSomeFunction(); 460 /* evaluate result or handle exception */ 461 462 /* Release the thread. No Python API allowed beyond this point. */ 463 PyGILState_Release(gstate); 464 465Note that the :c:func:`PyGILState_\*` functions assume there is only one global 466interpreter (created automatically by :c:func:`Py_Initialize`). Python 467supports the creation of additional interpreters (using 468:c:func:`Py_NewInterpreter`), but mixing multiple interpreters and the 469:c:func:`PyGILState_\*` API is unsupported. 470 471Another important thing to note about threads is their behaviour in the face 472of the C :c:func:`fork` call. On most systems with :c:func:`fork`, after a 473process forks only the thread that issued the fork will exist. That also 474means any locks held by other threads will never be released. Python solves 475this for :func:`os.fork` by acquiring the locks it uses internally before 476the fork, and releasing them afterwards. In addition, it resets any 477:ref:`lock-objects` in the child. When extending or embedding Python, there 478is no way to inform Python of additional (non-Python) locks that need to be 479acquired before or reset after a fork. OS facilities such as 480:c:func:`pthread_atfork` would need to be used to accomplish the same thing. 481Additionally, when extending or embedding Python, calling :c:func:`fork` 482directly rather than through :func:`os.fork` (and returning to or calling 483into Python) may result in a deadlock by one of Python's internal locks 484being held by a thread that is defunct after the fork. 485:c:func:`PyOS_AfterFork` tries to reset the necessary locks, but is not 486always able to. 487 488 489High-level API 490-------------- 491 492These are the most commonly used types and functions when writing C extension 493code, or when embedding the Python interpreter: 494 495.. c:type:: PyInterpreterState 496 497 This data structure represents the state shared by a number of cooperating 498 threads. Threads belonging to the same interpreter share their module 499 administration and a few other internal items. There are no public members in 500 this structure. 501 502 Threads belonging to different interpreters initially share nothing, except 503 process state like available memory, open file descriptors and such. The global 504 interpreter lock is also shared by all threads, regardless of to which 505 interpreter they belong. 506 507 508.. c:type:: PyThreadState 509 510 This data structure represents the state of a single thread. The only public 511 data member is :c:type:`PyInterpreterState \*`:attr:`interp`, which points to 512 this thread's interpreter state. 513 514 515.. c:function:: void PyEval_InitThreads() 516 517 .. index:: 518 single: PyEval_ReleaseLock() 519 single: PyEval_ReleaseThread() 520 single: PyEval_SaveThread() 521 single: PyEval_RestoreThread() 522 523 Initialize and acquire the global interpreter lock. It should be called in the 524 main thread before creating a second thread or engaging in any other thread 525 operations such as :c:func:`PyEval_ReleaseLock` or 526 ``PyEval_ReleaseThread(tstate)``. It is not needed before calling 527 :c:func:`PyEval_SaveThread` or :c:func:`PyEval_RestoreThread`. 528 529 .. index:: single: Py_Initialize() 530 531 This is a no-op when called for a second time. It is safe to call this function 532 before calling :c:func:`Py_Initialize`. 533 534 .. index:: module: thread 535 536 .. note:: 537 538 When only the main thread exists, no GIL operations are needed. This is a 539 common situation (most Python programs do not use threads), and the lock 540 operations slow the interpreter down a bit. Therefore, the lock is not 541 created initially. This situation is equivalent to having acquired the lock: 542 when there is only a single thread, all object accesses are safe. Therefore, 543 when this function initializes the global interpreter lock, it also acquires 544 it. Before the Python :mod:`_thread` module creates a new thread, knowing 545 that either it has the lock or the lock hasn't been created yet, it calls 546 :c:func:`PyEval_InitThreads`. When this call returns, it is guaranteed that 547 the lock has been created and that the calling thread has acquired it. 548 549 It is **not** safe to call this function when it is unknown which thread (if 550 any) currently has the global interpreter lock. 551 552 This function is not available when thread support is disabled at compile time. 553 554 555.. c:function:: int PyEval_ThreadsInitialized() 556 557 Returns a non-zero value if :c:func:`PyEval_InitThreads` has been called. This 558 function can be called without holding the GIL, and therefore can be used to 559 avoid calls to the locking API when running single-threaded. This function is 560 not available when thread support is disabled at compile time. 561 562 .. versionadded:: 2.4 563 564 565.. c:function:: PyThreadState* PyEval_SaveThread() 566 567 Release the global interpreter lock (if it has been created and thread 568 support is enabled) and reset the thread state to *NULL*, returning the 569 previous thread state (which is not *NULL*). If the lock has been created, 570 the current thread must have acquired it. (This function is available even 571 when thread support is disabled at compile time.) 572 573 574.. c:function:: void PyEval_RestoreThread(PyThreadState *tstate) 575 576 Acquire the global interpreter lock (if it has been created and thread 577 support is enabled) and set the thread state to *tstate*, which must not be 578 *NULL*. If the lock has been created, the current thread must not have 579 acquired it, otherwise deadlock ensues. (This function is available even 580 when thread support is disabled at compile time.) 581 582 583.. c:function:: PyThreadState* PyThreadState_Get() 584 585 Return the current thread state. The global interpreter lock must be held. 586 When the current thread state is *NULL*, this issues a fatal error (so that 587 the caller needn't check for *NULL*). 588 589 590.. c:function:: PyThreadState* PyThreadState_Swap(PyThreadState *tstate) 591 592 Swap the current thread state with the thread state given by the argument 593 *tstate*, which may be *NULL*. The global interpreter lock must be held 594 and is not released. 595 596 597.. c:function:: void PyEval_ReInitThreads() 598 599 This function is called from :c:func:`PyOS_AfterFork` to ensure that newly 600 created child processes don't hold locks referring to threads which 601 are not running in the child process. 602 603 604The following functions use thread-local storage, and are not compatible 605with sub-interpreters: 606 607.. c:function:: PyGILState_STATE PyGILState_Ensure() 608 609 Ensure that the current thread is ready to call the Python C API regardless 610 of the current state of Python, or of the global interpreter lock. This may 611 be called as many times as desired by a thread as long as each call is 612 matched with a call to :c:func:`PyGILState_Release`. In general, other 613 thread-related APIs may be used between :c:func:`PyGILState_Ensure` and 614 :c:func:`PyGILState_Release` calls as long as the thread state is restored to 615 its previous state before the Release(). For example, normal usage of the 616 :c:macro:`Py_BEGIN_ALLOW_THREADS` and :c:macro:`Py_END_ALLOW_THREADS` macros is 617 acceptable. 618 619 The return value is an opaque "handle" to the thread state when 620 :c:func:`PyGILState_Ensure` was called, and must be passed to 621 :c:func:`PyGILState_Release` to ensure Python is left in the same state. Even 622 though recursive calls are allowed, these handles *cannot* be shared - each 623 unique call to :c:func:`PyGILState_Ensure` must save the handle for its call 624 to :c:func:`PyGILState_Release`. 625 626 When the function returns, the current thread will hold the GIL and be able 627 to call arbitrary Python code. Failure is a fatal error. 628 629 .. versionadded:: 2.3 630 631 632.. c:function:: void PyGILState_Release(PyGILState_STATE) 633 634 Release any resources previously acquired. After this call, Python's state will 635 be the same as it was prior to the corresponding :c:func:`PyGILState_Ensure` call 636 (but generally this state will be unknown to the caller, hence the use of the 637 GILState API). 638 639 Every call to :c:func:`PyGILState_Ensure` must be matched by a call to 640 :c:func:`PyGILState_Release` on the same thread. 641 642 .. versionadded:: 2.3 643 644 645.. c:function:: PyThreadState* PyGILState_GetThisThreadState() 646 647 Get the current thread state for this thread. May return ``NULL`` if no 648 GILState API has been used on the current thread. Note that the main thread 649 always has such a thread-state, even if no auto-thread-state call has been 650 made on the main thread. This is mainly a helper/diagnostic function. 651 652 .. versionadded:: 2.3 653 654 655The following macros are normally used without a trailing semicolon; look for 656example usage in the Python source distribution. 657 658 659.. c:macro:: Py_BEGIN_ALLOW_THREADS 660 661 This macro expands to ``{ PyThreadState *_save; _save = PyEval_SaveThread();``. 662 Note that it contains an opening brace; it must be matched with a following 663 :c:macro:`Py_END_ALLOW_THREADS` macro. See above for further discussion of this 664 macro. It is a no-op when thread support is disabled at compile time. 665 666 667.. c:macro:: Py_END_ALLOW_THREADS 668 669 This macro expands to ``PyEval_RestoreThread(_save); }``. Note that it contains 670 a closing brace; it must be matched with an earlier 671 :c:macro:`Py_BEGIN_ALLOW_THREADS` macro. See above for further discussion of 672 this macro. It is a no-op when thread support is disabled at compile time. 673 674 675.. c:macro:: Py_BLOCK_THREADS 676 677 This macro expands to ``PyEval_RestoreThread(_save);``: it is equivalent to 678 :c:macro:`Py_END_ALLOW_THREADS` without the closing brace. It is a no-op when 679 thread support is disabled at compile time. 680 681 682.. c:macro:: Py_UNBLOCK_THREADS 683 684 This macro expands to ``_save = PyEval_SaveThread();``: it is equivalent to 685 :c:macro:`Py_BEGIN_ALLOW_THREADS` without the opening brace and variable 686 declaration. It is a no-op when thread support is disabled at compile time. 687 688 689Low-level API 690------------- 691 692All of the following functions are only available when thread support is enabled 693at compile time, and must be called only when the global interpreter lock has 694been created. 695 696 697.. c:function:: PyInterpreterState* PyInterpreterState_New() 698 699 Create a new interpreter state object. The global interpreter lock need not 700 be held, but may be held if it is necessary to serialize calls to this 701 function. 702 703 704.. c:function:: void PyInterpreterState_Clear(PyInterpreterState *interp) 705 706 Reset all information in an interpreter state object. The global interpreter 707 lock must be held. 708 709 710.. c:function:: void PyInterpreterState_Delete(PyInterpreterState *interp) 711 712 Destroy an interpreter state object. The global interpreter lock need not be 713 held. The interpreter state must have been reset with a previous call to 714 :c:func:`PyInterpreterState_Clear`. 715 716 717.. c:function:: PyThreadState* PyThreadState_New(PyInterpreterState *interp) 718 719 Create a new thread state object belonging to the given interpreter object. 720 The global interpreter lock need not be held, but may be held if it is 721 necessary to serialize calls to this function. 722 723 724.. c:function:: void PyThreadState_Clear(PyThreadState *tstate) 725 726 Reset all information in a thread state object. The global interpreter lock 727 must be held. 728 729 730.. c:function:: void PyThreadState_Delete(PyThreadState *tstate) 731 732 Destroy a thread state object. The global interpreter lock need not be held. 733 The thread state must have been reset with a previous call to 734 :c:func:`PyThreadState_Clear`. 735 736 737.. c:function:: PyObject* PyThreadState_GetDict() 738 739 Return a dictionary in which extensions can store thread-specific state 740 information. Each extension should use a unique key to use to store state in 741 the dictionary. It is okay to call this function when no current thread state 742 is available. If this function returns *NULL*, no exception has been raised and 743 the caller should assume no current thread state is available. 744 745 .. versionchanged:: 2.3 746 Previously this could only be called when a current thread is active, and *NULL* 747 meant that an exception was raised. 748 749 750.. c:function:: int PyThreadState_SetAsyncExc(long id, PyObject *exc) 751 752 Asynchronously raise an exception in a thread. The *id* argument is the thread 753 id of the target thread; *exc* is the exception object to be raised. This 754 function does not steal any references to *exc*. To prevent naive misuse, you 755 must write your own C extension to call this. Must be called with the GIL held. 756 Returns the number of thread states modified; this is normally one, but will be 757 zero if the thread id isn't found. If *exc* is :const:`NULL`, the pending 758 exception (if any) for the thread is cleared. This raises no exceptions. 759 760 .. versionadded:: 2.3 761 762 763.. c:function:: void PyEval_AcquireThread(PyThreadState *tstate) 764 765 Acquire the global interpreter lock and set the current thread state to 766 *tstate*, which should not be *NULL*. The lock must have been created earlier. 767 If this thread already has the lock, deadlock ensues. 768 769 :c:func:`PyEval_RestoreThread` is a higher-level function which is always 770 available (even when thread support isn't enabled or when threads have 771 not been initialized). 772 773 774.. c:function:: void PyEval_ReleaseThread(PyThreadState *tstate) 775 776 Reset the current thread state to *NULL* and release the global interpreter 777 lock. The lock must have been created earlier and must be held by the current 778 thread. The *tstate* argument, which must not be *NULL*, is only used to check 779 that it represents the current thread state --- if it isn't, a fatal error is 780 reported. 781 782 :c:func:`PyEval_SaveThread` is a higher-level function which is always 783 available (even when thread support isn't enabled or when threads have 784 not been initialized). 785 786 787.. c:function:: void PyEval_AcquireLock() 788 789 Acquire the global interpreter lock. The lock must have been created earlier. 790 If this thread already has the lock, a deadlock ensues. 791 792 .. warning:: 793 This function does not change the current thread state. Please use 794 :c:func:`PyEval_RestoreThread` or :c:func:`PyEval_AcquireThread` 795 instead. 796 797 798.. c:function:: void PyEval_ReleaseLock() 799 800 Release the global interpreter lock. The lock must have been created earlier. 801 802 .. warning:: 803 This function does not change the current thread state. Please use 804 :c:func:`PyEval_SaveThread` or :c:func:`PyEval_ReleaseThread` 805 instead. 806 807 808Sub-interpreter support 809======================= 810 811While in most uses, you will only embed a single Python interpreter, there 812are cases where you need to create several independent interpreters in the 813same process and perhaps even in the same thread. Sub-interpreters allow 814you to do that. You can switch between sub-interpreters using the 815:c:func:`PyThreadState_Swap` function. You can create and destroy them 816using the following functions: 817 818 819.. c:function:: PyThreadState* Py_NewInterpreter() 820 821 .. index:: 822 module: builtins 823 module: __main__ 824 module: sys 825 single: stdout (in module sys) 826 single: stderr (in module sys) 827 single: stdin (in module sys) 828 829 Create a new sub-interpreter. This is an (almost) totally separate environment 830 for the execution of Python code. In particular, the new interpreter has 831 separate, independent versions of all imported modules, including the 832 fundamental modules :mod:`builtins`, :mod:`__main__` and :mod:`sys`. The 833 table of loaded modules (``sys.modules``) and the module search path 834 (``sys.path``) are also separate. The new environment has no ``sys.argv`` 835 variable. It has new standard I/O stream file objects ``sys.stdin``, 836 ``sys.stdout`` and ``sys.stderr`` (however these refer to the same underlying 837 file descriptors). 838 839 The return value points to the first thread state created in the new 840 sub-interpreter. This thread state is made in the current thread state. 841 Note that no actual thread is created; see the discussion of thread states 842 below. If creation of the new interpreter is unsuccessful, *NULL* is 843 returned; no exception is set since the exception state is stored in the 844 current thread state and there may not be a current thread state. (Like all 845 other Python/C API functions, the global interpreter lock must be held before 846 calling this function and is still held when it returns; however, unlike most 847 other Python/C API functions, there needn't be a current thread state on 848 entry.) 849 850 .. index:: 851 single: Py_Finalize() 852 single: Py_Initialize() 853 854 Extension modules are shared between (sub-)interpreters as follows: the first 855 time a particular extension is imported, it is initialized normally, and a 856 (shallow) copy of its module's dictionary is squirreled away. When the same 857 extension is imported by another (sub-)interpreter, a new module is initialized 858 and filled with the contents of this copy; the extension's ``init`` function is 859 not called. Note that this is different from what happens when an extension is 860 imported after the interpreter has been completely re-initialized by calling 861 :c:func:`Py_Finalize` and :c:func:`Py_Initialize`; in that case, the extension's 862 ``initmodule`` function *is* called again. 863 864 .. index:: single: close() (in module os) 865 866 867.. c:function:: void Py_EndInterpreter(PyThreadState *tstate) 868 869 .. index:: single: Py_Finalize() 870 871 Destroy the (sub-)interpreter represented by the given thread state. The given 872 thread state must be the current thread state. See the discussion of thread 873 states below. When the call returns, the current thread state is *NULL*. All 874 thread states associated with this interpreter are destroyed. (The global 875 interpreter lock must be held before calling this function and is still held 876 when it returns.) :c:func:`Py_Finalize` will destroy all sub-interpreters that 877 haven't been explicitly destroyed at that point. 878 879 880Bugs and caveats 881---------------- 882 883Because sub-interpreters (and the main interpreter) are part of the same 884process, the insulation between them isn't perfect --- for example, using 885low-level file operations like :func:`os.close` they can 886(accidentally or maliciously) affect each other's open files. Because of the 887way extensions are shared between (sub-)interpreters, some extensions may not 888work properly; this is especially likely when the extension makes use of 889(static) global variables, or when the extension manipulates its module's 890dictionary after its initialization. It is possible to insert objects created 891in one sub-interpreter into a namespace of another sub-interpreter; this should 892be done with great care to avoid sharing user-defined functions, methods, 893instances or classes between sub-interpreters, since import operations executed 894by such objects may affect the wrong (sub-)interpreter's dictionary of loaded 895modules. 896 897Also note that combining this functionality with :c:func:`PyGILState_\*` APIs 898is delicate, because these APIs assume a bijection between Python thread states 899and OS-level threads, an assumption broken by the presence of sub-interpreters. 900It is highly recommended that you don't switch sub-interpreters between a pair 901of matching :c:func:`PyGILState_Ensure` and :c:func:`PyGILState_Release` calls. 902Furthermore, extensions (such as :mod:`ctypes`) using these APIs to allow calling 903of Python code from non-Python created threads will probably be broken when using 904sub-interpreters. 905 906 907Asynchronous Notifications 908========================== 909 910A mechanism is provided to make asynchronous notifications to the main 911interpreter thread. These notifications take the form of a function 912pointer and a void pointer argument. 913 914 915.. c:function:: int Py_AddPendingCall(int (*func)(void *), void *arg) 916 917 .. index:: single: Py_AddPendingCall() 918 919 Schedule a function to be called from the main interpreter thread. On 920 success, ``0`` is returned and *func* is queued for being called in the 921 main thread. On failure, ``-1`` is returned without setting any exception. 922 923 When successfully queued, *func* will be *eventually* called from the 924 main interpreter thread with the argument *arg*. It will be called 925 asynchronously with respect to normally running Python code, but with 926 both these conditions met: 927 928 * on a :term:`bytecode` boundary; 929 * with the main thread holding the :term:`global interpreter lock` 930 (*func* can therefore use the full C API). 931 932 *func* must return ``0`` on success, or ``-1`` on failure with an exception 933 set. *func* won't be interrupted to perform another asynchronous 934 notification recursively, but it can still be interrupted to switch 935 threads if the global interpreter lock is released. 936 937 This function doesn't need a current thread state to run, and it doesn't 938 need the global interpreter lock. 939 940 .. warning:: 941 This is a low-level function, only useful for very special cases. 942 There is no guarantee that *func* will be called as quick as 943 possible. If the main thread is busy executing a system call, 944 *func* won't be called before the system call returns. This 945 function is generally **not** suitable for calling Python code from 946 arbitrary C threads. Instead, use the :ref:`PyGILState API<gilstate>`. 947 948 .. versionadded:: 2.7 949 950 951.. _profiling: 952 953Profiling and Tracing 954===================== 955 956.. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org> 957 958 959The Python interpreter provides some low-level support for attaching profiling 960and execution tracing facilities. These are used for profiling, debugging, and 961coverage analysis tools. 962 963Starting with Python 2.2, the implementation of this facility was substantially 964revised, and an interface from C was added. This C interface allows the 965profiling or tracing code to avoid the overhead of calling through Python-level 966callable objects, making a direct C function call instead. The essential 967attributes of the facility have not changed; the interface allows trace 968functions to be installed per-thread, and the basic events reported to the trace 969function are the same as had been reported to the Python-level trace functions 970in previous versions. 971 972 973.. c:type:: int (*Py_tracefunc)(PyObject *obj, PyFrameObject *frame, int what, PyObject *arg) 974 975 The type of the trace function registered using :c:func:`PyEval_SetProfile` and 976 :c:func:`PyEval_SetTrace`. The first parameter is the object passed to the 977 registration function as *obj*, *frame* is the frame object to which the event 978 pertains, *what* is one of the constants :const:`PyTrace_CALL`, 979 :const:`PyTrace_EXCEPTION`, :const:`PyTrace_LINE`, :const:`PyTrace_RETURN`, 980 :const:`PyTrace_C_CALL`, :const:`PyTrace_C_EXCEPTION`, or 981 :const:`PyTrace_C_RETURN`, and *arg* depends on the value of *what*: 982 983 +------------------------------+--------------------------------------+ 984 | Value of *what* | Meaning of *arg* | 985 +==============================+======================================+ 986 | :const:`PyTrace_CALL` | Always *NULL*. | 987 +------------------------------+--------------------------------------+ 988 | :const:`PyTrace_EXCEPTION` | Exception information as returned by | 989 | | :func:`sys.exc_info`. | 990 +------------------------------+--------------------------------------+ 991 | :const:`PyTrace_LINE` | Always *NULL*. | 992 +------------------------------+--------------------------------------+ 993 | :const:`PyTrace_RETURN` | Value being returned to the caller, | 994 | | or *NULL* if caused by an exception. | 995 +------------------------------+--------------------------------------+ 996 | :const:`PyTrace_C_CALL` | Function object being called. | 997 +------------------------------+--------------------------------------+ 998 | :const:`PyTrace_C_EXCEPTION` | Function object being called. | 999 +------------------------------+--------------------------------------+ 1000 | :const:`PyTrace_C_RETURN` | Function object being called. | 1001 +------------------------------+--------------------------------------+ 1002 1003 1004.. c:var:: int PyTrace_CALL 1005 1006 The value of the *what* parameter to a :c:type:`Py_tracefunc` function when a new 1007 call to a function or method is being reported, or a new entry into a generator. 1008 Note that the creation of the iterator for a generator function is not reported 1009 as there is no control transfer to the Python bytecode in the corresponding 1010 frame. 1011 1012 1013.. c:var:: int PyTrace_EXCEPTION 1014 1015 The value of the *what* parameter to a :c:type:`Py_tracefunc` function when an 1016 exception has been raised. The callback function is called with this value for 1017 *what* when after any bytecode is processed after which the exception becomes 1018 set within the frame being executed. The effect of this is that as exception 1019 propagation causes the Python stack to unwind, the callback is called upon 1020 return to each frame as the exception propagates. Only trace functions receives 1021 these events; they are not needed by the profiler. 1022 1023 1024.. c:var:: int PyTrace_LINE 1025 1026 The value passed as the *what* parameter to a trace function (but not a 1027 profiling function) when a line-number event is being reported. 1028 1029 1030.. c:var:: int PyTrace_RETURN 1031 1032 The value for the *what* parameter to :c:type:`Py_tracefunc` functions when a 1033 call is returning without propagating an exception. 1034 1035 1036.. c:var:: int PyTrace_C_CALL 1037 1038 The value for the *what* parameter to :c:type:`Py_tracefunc` functions when a C 1039 function is about to be called. 1040 1041 1042.. c:var:: int PyTrace_C_EXCEPTION 1043 1044 The value for the *what* parameter to :c:type:`Py_tracefunc` functions when a C 1045 function has raised an exception. 1046 1047 1048.. c:var:: int PyTrace_C_RETURN 1049 1050 The value for the *what* parameter to :c:type:`Py_tracefunc` functions when a C 1051 function has returned. 1052 1053 1054.. c:function:: void PyEval_SetProfile(Py_tracefunc func, PyObject *obj) 1055 1056 Set the profiler function to *func*. The *obj* parameter is passed to the 1057 function as its first parameter, and may be any Python object, or *NULL*. If 1058 the profile function needs to maintain state, using a different value for *obj* 1059 for each thread provides a convenient and thread-safe place to store it. The 1060 profile function is called for all monitored events except the line-number 1061 events. 1062 1063 1064.. c:function:: void PyEval_SetTrace(Py_tracefunc func, PyObject *obj) 1065 1066 Set the tracing function to *func*. This is similar to 1067 :c:func:`PyEval_SetProfile`, except the tracing function does receive line-number 1068 events. 1069 1070.. c:function:: PyObject* PyEval_GetCallStats(PyObject *self) 1071 1072 Return a tuple of function call counts. There are constants defined for the 1073 positions within the tuple: 1074 1075 +-------------------------------+-------+ 1076 | Name | Value | 1077 +===============================+=======+ 1078 | :const:`PCALL_ALL` | 0 | 1079 +-------------------------------+-------+ 1080 | :const:`PCALL_FUNCTION` | 1 | 1081 +-------------------------------+-------+ 1082 | :const:`PCALL_FAST_FUNCTION` | 2 | 1083 +-------------------------------+-------+ 1084 | :const:`PCALL_FASTER_FUNCTION`| 3 | 1085 +-------------------------------+-------+ 1086 | :const:`PCALL_METHOD` | 4 | 1087 +-------------------------------+-------+ 1088 | :const:`PCALL_BOUND_METHOD` | 5 | 1089 +-------------------------------+-------+ 1090 | :const:`PCALL_CFUNCTION` | 6 | 1091 +-------------------------------+-------+ 1092 | :const:`PCALL_TYPE` | 7 | 1093 +-------------------------------+-------+ 1094 | :const:`PCALL_GENERATOR` | 8 | 1095 +-------------------------------+-------+ 1096 | :const:`PCALL_OTHER` | 9 | 1097 +-------------------------------+-------+ 1098 | :const:`PCALL_POP` | 10 | 1099 +-------------------------------+-------+ 1100 1101 :const:`PCALL_FAST_FUNCTION` means no argument tuple needs to be created. 1102 :const:`PCALL_FASTER_FUNCTION` means that the fast-path frame setup code is used. 1103 1104 If there is a method call where the call can be optimized by changing 1105 the argument tuple and calling the function directly, it gets recorded 1106 twice. 1107 1108 This function is only present if Python is compiled with :const:`CALL_PROFILE` 1109 defined. 1110 1111.. _advanced-debugging: 1112 1113Advanced Debugger Support 1114========================= 1115 1116.. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org> 1117 1118 1119These functions are only intended to be used by advanced debugging tools. 1120 1121 1122.. c:function:: PyInterpreterState* PyInterpreterState_Head() 1123 1124 Return the interpreter state object at the head of the list of all such objects. 1125 1126 .. versionadded:: 2.2 1127 1128 1129.. c:function:: PyInterpreterState* PyInterpreterState_Next(PyInterpreterState *interp) 1130 1131 Return the next interpreter state object after *interp* from the list of all 1132 such objects. 1133 1134 .. versionadded:: 2.2 1135 1136 1137.. c:function:: PyThreadState * PyInterpreterState_ThreadHead(PyInterpreterState *interp) 1138 1139 Return the pointer to the first :c:type:`PyThreadState` object in the list of 1140 threads associated with the interpreter *interp*. 1141 1142 .. versionadded:: 2.2 1143 1144 1145.. c:function:: PyThreadState* PyThreadState_Next(PyThreadState *tstate) 1146 1147 Return the next thread state object after *tstate* from the list of all such 1148 objects belonging to the same :c:type:`PyInterpreterState` object. 1149 1150 .. versionadded:: 2.2 1151 1152