1.. highlightlang:: c 2 3 4.. _extending-intro: 5 6****************************** 7Extending Python with C or C++ 8****************************** 9 10It is quite easy to add new built-in modules to Python, if you know how to 11program in C. Such :dfn:`extension modules` can do two things that can't be 12done directly in Python: they can implement new built-in object types, and they 13can call C library functions and system calls. 14 15To support extensions, the Python API (Application Programmers Interface) 16defines a set of functions, macros and variables that provide access to most 17aspects of the Python run-time system. The Python API is incorporated in a C 18source file by including the header ``"Python.h"``. 19 20The compilation of an extension module depends on its intended use as well as on 21your system setup; details are given in later chapters. 22 23.. note:: 24 25 The C extension interface is specific to CPython, and extension modules do 26 not work on other Python implementations. In many cases, it is possible to 27 avoid writing C extensions and preserve portability to other implementations. 28 For example, if your use case is calling C library functions or system calls, 29 you should consider using the :mod:`ctypes` module or the `cffi 30 <https://cffi.readthedocs.org>`_ library rather than writing custom C code. 31 These modules let you write Python code to interface with C code and are more 32 portable between implementations of Python than writing and compiling a C 33 extension module. 34 35 36.. _extending-simpleexample: 37 38A Simple Example 39================ 40 41Let's create an extension module called ``spam`` (the favorite food of Monty 42Python fans...) and let's say we want to create a Python interface to the C 43library function :c:func:`system`. [#]_ This function takes a null-terminated 44character string as argument and returns an integer. We want this function to 45be callable from Python as follows:: 46 47 >>> import spam 48 >>> status = spam.system("ls -l") 49 50Begin by creating a file :file:`spammodule.c`. (Historically, if a module is 51called ``spam``, the C file containing its implementation is called 52:file:`spammodule.c`; if the module name is very long, like ``spammify``, the 53module name can be just :file:`spammify.c`.) 54 55The first line of our file can be:: 56 57 #include <Python.h> 58 59which pulls in the Python API (you can add a comment describing the purpose of 60the module and a copyright notice if you like). 61 62.. note:: 63 64 Since Python may define some pre-processor definitions which affect the standard 65 headers on some systems, you *must* include :file:`Python.h` before any standard 66 headers are included. 67 68All user-visible symbols defined by :file:`Python.h` have a prefix of ``Py`` or 69``PY``, except those defined in standard header files. For convenience, and 70since they are used extensively by the Python interpreter, ``"Python.h"`` 71includes a few standard header files: ``<stdio.h>``, ``<string.h>``, 72``<errno.h>``, and ``<stdlib.h>``. If the latter header file does not exist on 73your system, it declares the functions :c:func:`malloc`, :c:func:`free` and 74:c:func:`realloc` directly. 75 76The next thing we add to our module file is the C function that will be called 77when the Python expression ``spam.system(string)`` is evaluated (we'll see 78shortly how it ends up being called):: 79 80 static PyObject * 81 spam_system(PyObject *self, PyObject *args) 82 { 83 const char *command; 84 int sts; 85 86 if (!PyArg_ParseTuple(args, "s", &command)) 87 return NULL; 88 sts = system(command); 89 return Py_BuildValue("i", sts); 90 } 91 92There is a straightforward translation from the argument list in Python (for 93example, the single expression ``"ls -l"``) to the arguments passed to the C 94function. The C function always has two arguments, conventionally named *self* 95and *args*. 96 97For module functions, the *self* argument is *NULL* or a pointer selected while 98initializing the module (see :c:func:`Py_InitModule4`). For a method, it would 99point to the object instance. 100 101The *args* argument will be a pointer to a Python tuple object containing the 102arguments. Each item of the tuple corresponds to an argument in the call's 103argument list. The arguments are Python objects --- in order to do anything 104with them in our C function we have to convert them to C values. The function 105:c:func:`PyArg_ParseTuple` in the Python API checks the argument types and 106converts them to C values. It uses a template string to determine the required 107types of the arguments as well as the types of the C variables into which to 108store the converted values. More about this later. 109 110:c:func:`PyArg_ParseTuple` returns true (nonzero) if all arguments have the right 111type and its components have been stored in the variables whose addresses are 112passed. It returns false (zero) if an invalid argument list was passed. In the 113latter case it also raises an appropriate exception so the calling function can 114return *NULL* immediately (as we saw in the example). 115 116 117.. _extending-errors: 118 119Intermezzo: Errors and Exceptions 120================================= 121 122An important convention throughout the Python interpreter is the following: when 123a function fails, it should set an exception condition and return an error value 124(usually a *NULL* pointer). Exceptions are stored in a static global variable 125inside the interpreter; if this variable is *NULL* no exception has occurred. A 126second global variable stores the "associated value" of the exception (the 127second argument to :keyword:`raise`). A third variable contains the stack 128traceback in case the error originated in Python code. These three variables 129are the C equivalents of the Python variables ``sys.exc_type``, 130``sys.exc_value`` and ``sys.exc_traceback`` (see the section on module 131:mod:`sys` in the Python Library Reference). It is important to know about them 132to understand how errors are passed around. 133 134The Python API defines a number of functions to set various types of exceptions. 135 136The most common one is :c:func:`PyErr_SetString`. Its arguments are an exception 137object and a C string. The exception object is usually a predefined object like 138:c:data:`PyExc_ZeroDivisionError`. The C string indicates the cause of the error 139and is converted to a Python string object and stored as the "associated value" 140of the exception. 141 142Another useful function is :c:func:`PyErr_SetFromErrno`, which only takes an 143exception argument and constructs the associated value by inspection of the 144global variable :c:data:`errno`. The most general function is 145:c:func:`PyErr_SetObject`, which takes two object arguments, the exception and 146its associated value. You don't need to :c:func:`Py_INCREF` the objects passed 147to any of these functions. 148 149You can test non-destructively whether an exception has been set with 150:c:func:`PyErr_Occurred`. This returns the current exception object, or *NULL* 151if no exception has occurred. You normally don't need to call 152:c:func:`PyErr_Occurred` to see whether an error occurred in a function call, 153since you should be able to tell from the return value. 154 155When a function *f* that calls another function *g* detects that the latter 156fails, *f* should itself return an error value (usually *NULL* or ``-1``). It 157should *not* call one of the :c:func:`PyErr_\*` functions --- one has already 158been called by *g*. *f*'s caller is then supposed to also return an error 159indication to *its* caller, again *without* calling :c:func:`PyErr_\*`, and so on 160--- the most detailed cause of the error was already reported by the function 161that first detected it. Once the error reaches the Python interpreter's main 162loop, this aborts the currently executing Python code and tries to find an 163exception handler specified by the Python programmer. 164 165(There are situations where a module can actually give a more detailed error 166message by calling another :c:func:`PyErr_\*` function, and in such cases it is 167fine to do so. As a general rule, however, this is not necessary, and can cause 168information about the cause of the error to be lost: most operations can fail 169for a variety of reasons.) 170 171To ignore an exception set by a function call that failed, the exception 172condition must be cleared explicitly by calling :c:func:`PyErr_Clear`. The only 173time C code should call :c:func:`PyErr_Clear` is if it doesn't want to pass the 174error on to the interpreter but wants to handle it completely by itself 175(possibly by trying something else, or pretending nothing went wrong). 176 177Every failing :c:func:`malloc` call must be turned into an exception --- the 178direct caller of :c:func:`malloc` (or :c:func:`realloc`) must call 179:c:func:`PyErr_NoMemory` and return a failure indicator itself. All the 180object-creating functions (for example, :c:func:`PyInt_FromLong`) already do 181this, so this note is only relevant to those who call :c:func:`malloc` directly. 182 183Also note that, with the important exception of :c:func:`PyArg_ParseTuple` and 184friends, functions that return an integer status usually return a positive value 185or zero for success and ``-1`` for failure, like Unix system calls. 186 187Finally, be careful to clean up garbage (by making :c:func:`Py_XDECREF` or 188:c:func:`Py_DECREF` calls for objects you have already created) when you return 189an error indicator! 190 191The choice of which exception to raise is entirely yours. There are predeclared 192C objects corresponding to all built-in Python exceptions, such as 193:c:data:`PyExc_ZeroDivisionError`, which you can use directly. Of course, you 194should choose exceptions wisely --- don't use :c:data:`PyExc_TypeError` to mean 195that a file couldn't be opened (that should probably be :c:data:`PyExc_IOError`). 196If something's wrong with the argument list, the :c:func:`PyArg_ParseTuple` 197function usually raises :c:data:`PyExc_TypeError`. If you have an argument whose 198value must be in a particular range or must satisfy other conditions, 199:c:data:`PyExc_ValueError` is appropriate. 200 201You can also define a new exception that is unique to your module. For this, you 202usually declare a static object variable at the beginning of your file:: 203 204 static PyObject *SpamError; 205 206and initialize it in your module's initialization function (:c:func:`initspam`) 207with an exception object (leaving out the error checking for now):: 208 209 PyMODINIT_FUNC 210 initspam(void) 211 { 212 PyObject *m; 213 214 m = Py_InitModule("spam", SpamMethods); 215 if (m == NULL) 216 return; 217 218 SpamError = PyErr_NewException("spam.error", NULL, NULL); 219 Py_INCREF(SpamError); 220 PyModule_AddObject(m, "error", SpamError); 221 } 222 223Note that the Python name for the exception object is :exc:`spam.error`. The 224:c:func:`PyErr_NewException` function may create a class with the base class 225being :exc:`Exception` (unless another class is passed in instead of *NULL*), 226described in :ref:`bltin-exceptions`. 227 228Note also that the :c:data:`SpamError` variable retains a reference to the newly 229created exception class; this is intentional! Since the exception could be 230removed from the module by external code, an owned reference to the class is 231needed to ensure that it will not be discarded, causing :c:data:`SpamError` to 232become a dangling pointer. Should it become a dangling pointer, C code which 233raises the exception could cause a core dump or other unintended side effects. 234 235We discuss the use of ``PyMODINIT_FUNC`` as a function return type later in this 236sample. 237 238The :exc:`spam.error` exception can be raised in your extension module using a 239call to :c:func:`PyErr_SetString` as shown below:: 240 241 static PyObject * 242 spam_system(PyObject *self, PyObject *args) 243 { 244 const char *command; 245 int sts; 246 247 if (!PyArg_ParseTuple(args, "s", &command)) 248 return NULL; 249 sts = system(command); 250 if (sts < 0) { 251 PyErr_SetString(SpamError, "System command failed"); 252 return NULL; 253 } 254 return PyLong_FromLong(sts); 255 } 256 257 258.. _backtoexample: 259 260Back to the Example 261=================== 262 263Going back to our example function, you should now be able to understand this 264statement:: 265 266 if (!PyArg_ParseTuple(args, "s", &command)) 267 return NULL; 268 269It returns *NULL* (the error indicator for functions returning object pointers) 270if an error is detected in the argument list, relying on the exception set by 271:c:func:`PyArg_ParseTuple`. Otherwise the string value of the argument has been 272copied to the local variable :c:data:`command`. This is a pointer assignment and 273you are not supposed to modify the string to which it points (so in Standard C, 274the variable :c:data:`command` should properly be declared as ``const char 275*command``). 276 277The next statement is a call to the Unix function :c:func:`system`, passing it 278the string we just got from :c:func:`PyArg_ParseTuple`:: 279 280 sts = system(command); 281 282Our :func:`spam.system` function must return the value of :c:data:`sts` as a 283Python object. This is done using the function :c:func:`Py_BuildValue`, which is 284something like the inverse of :c:func:`PyArg_ParseTuple`: it takes a format 285string and an arbitrary number of C values, and returns a new Python object. 286More info on :c:func:`Py_BuildValue` is given later. :: 287 288 return Py_BuildValue("i", sts); 289 290In this case, it will return an integer object. (Yes, even integers are objects 291on the heap in Python!) 292 293If you have a C function that returns no useful argument (a function returning 294:c:type:`void`), the corresponding Python function must return ``None``. You 295need this idiom to do so (which is implemented by the :c:macro:`Py_RETURN_NONE` 296macro):: 297 298 Py_INCREF(Py_None); 299 return Py_None; 300 301:c:data:`Py_None` is the C name for the special Python object ``None``. It is a 302genuine Python object rather than a *NULL* pointer, which means "error" in most 303contexts, as we have seen. 304 305 306.. _methodtable: 307 308The Module's Method Table and Initialization Function 309===================================================== 310 311I promised to show how :c:func:`spam_system` is called from Python programs. 312First, we need to list its name and address in a "method table":: 313 314 static PyMethodDef SpamMethods[] = { 315 ... 316 {"system", spam_system, METH_VARARGS, 317 "Execute a shell command."}, 318 ... 319 {NULL, NULL, 0, NULL} /* Sentinel */ 320 }; 321 322Note the third entry (``METH_VARARGS``). This is a flag telling the interpreter 323the calling convention to be used for the C function. It should normally always 324be ``METH_VARARGS`` or ``METH_VARARGS | METH_KEYWORDS``; a value of ``0`` means 325that an obsolete variant of :c:func:`PyArg_ParseTuple` is used. 326 327When using only ``METH_VARARGS``, the function should expect the Python-level 328parameters to be passed in as a tuple acceptable for parsing via 329:c:func:`PyArg_ParseTuple`; more information on this function is provided below. 330 331The :const:`METH_KEYWORDS` bit may be set in the third field if keyword 332arguments should be passed to the function. In this case, the C function should 333accept a third ``PyObject *`` parameter which will be a dictionary of keywords. 334Use :c:func:`PyArg_ParseTupleAndKeywords` to parse the arguments to such a 335function. 336 337The method table must be passed to the interpreter in the module's 338initialization function. The initialization function must be named 339:c:func:`initname`, where *name* is the name of the module, and should be the 340only non-\ ``static`` item defined in the module file:: 341 342 PyMODINIT_FUNC 343 initspam(void) 344 { 345 (void) Py_InitModule("spam", SpamMethods); 346 } 347 348Note that PyMODINIT_FUNC declares the function as ``void`` return type, 349declares any special linkage declarations required by the platform, and for C++ 350declares the function as ``extern "C"``. 351 352When the Python program imports module :mod:`spam` for the first time, 353:c:func:`initspam` is called. (See below for comments about embedding Python.) 354It calls :c:func:`Py_InitModule`, which creates a "module object" (which is 355inserted in the dictionary ``sys.modules`` under the key ``"spam"``), and 356inserts built-in function objects into the newly created module based upon the 357table (an array of :c:type:`PyMethodDef` structures) that was passed as its 358second argument. :c:func:`Py_InitModule` returns a pointer to the module object 359that it creates (which is unused here). It may abort with a fatal error for 360certain errors, or return *NULL* if the module could not be initialized 361satisfactorily. 362 363When embedding Python, the :c:func:`initspam` function is not called 364automatically unless there's an entry in the :c:data:`_PyImport_Inittab` table. 365The easiest way to handle this is to statically initialize your 366statically-linked modules by directly calling :c:func:`initspam` after the call 367to :c:func:`Py_Initialize`:: 368 369 int 370 main(int argc, char *argv[]) 371 { 372 /* Pass argv[0] to the Python interpreter */ 373 Py_SetProgramName(argv[0]); 374 375 /* Initialize the Python interpreter. Required. */ 376 Py_Initialize(); 377 378 /* Add a static module */ 379 initspam(); 380 381 ... 382 383An example may be found in the file :file:`Demo/embed/demo.c` in the Python 384source distribution. 385 386.. note:: 387 388 Removing entries from ``sys.modules`` or importing compiled modules into 389 multiple interpreters within a process (or following a :c:func:`fork` without an 390 intervening :c:func:`exec`) can create problems for some extension modules. 391 Extension module authors should exercise caution when initializing internal data 392 structures. Note also that the :func:`reload` function can be used with 393 extension modules, and will call the module initialization function 394 (:c:func:`initspam` in the example), but will not load the module again if it was 395 loaded from a dynamically loadable object file (:file:`.so` on Unix, 396 :file:`.dll` on Windows). 397 398A more substantial example module is included in the Python source distribution 399as :file:`Modules/xxmodule.c`. This file may be used as a template or simply 400read as an example. 401 402 403.. _compilation: 404 405Compilation and Linkage 406======================= 407 408There are two more things to do before you can use your new extension: compiling 409and linking it with the Python system. If you use dynamic loading, the details 410may depend on the style of dynamic loading your system uses; see the chapters 411about building extension modules (chapter :ref:`building`) and additional 412information that pertains only to building on Windows (chapter 413:ref:`building-on-windows`) for more information about this. 414 415If you can't use dynamic loading, or if you want to make your module a permanent 416part of the Python interpreter, you will have to change the configuration setup 417and rebuild the interpreter. Luckily, this is very simple on Unix: just place 418your file (:file:`spammodule.c` for example) in the :file:`Modules/` directory 419of an unpacked source distribution, add a line to the file 420:file:`Modules/Setup.local` describing your file:: 421 422 spam spammodule.o 423 424and rebuild the interpreter by running :program:`make` in the toplevel 425directory. You can also run :program:`make` in the :file:`Modules/` 426subdirectory, but then you must first rebuild :file:`Makefile` there by running 427':program:`make` Makefile'. (This is necessary each time you change the 428:file:`Setup` file.) 429 430If your module requires additional libraries to link with, these can be listed 431on the line in the configuration file as well, for instance:: 432 433 spam spammodule.o -lX11 434 435 436.. _callingpython: 437 438Calling Python Functions from C 439=============================== 440 441So far we have concentrated on making C functions callable from Python. The 442reverse is also useful: calling Python functions from C. This is especially the 443case for libraries that support so-called "callback" functions. If a C 444interface makes use of callbacks, the equivalent Python often needs to provide a 445callback mechanism to the Python programmer; the implementation will require 446calling the Python callback functions from a C callback. Other uses are also 447imaginable. 448 449Fortunately, the Python interpreter is easily called recursively, and there is a 450standard interface to call a Python function. (I won't dwell on how to call the 451Python parser with a particular string as input --- if you're interested, have a 452look at the implementation of the :option:`-c` command line option in 453:file:`Modules/main.c` from the Python source code.) 454 455Calling a Python function is easy. First, the Python program must somehow pass 456you the Python function object. You should provide a function (or some other 457interface) to do this. When this function is called, save a pointer to the 458Python function object (be careful to :c:func:`Py_INCREF` it!) in a global 459variable --- or wherever you see fit. For example, the following function might 460be part of a module definition:: 461 462 static PyObject *my_callback = NULL; 463 464 static PyObject * 465 my_set_callback(PyObject *dummy, PyObject *args) 466 { 467 PyObject *result = NULL; 468 PyObject *temp; 469 470 if (PyArg_ParseTuple(args, "O:set_callback", &temp)) { 471 if (!PyCallable_Check(temp)) { 472 PyErr_SetString(PyExc_TypeError, "parameter must be callable"); 473 return NULL; 474 } 475 Py_XINCREF(temp); /* Add a reference to new callback */ 476 Py_XDECREF(my_callback); /* Dispose of previous callback */ 477 my_callback = temp; /* Remember new callback */ 478 /* Boilerplate to return "None" */ 479 Py_INCREF(Py_None); 480 result = Py_None; 481 } 482 return result; 483 } 484 485This function must be registered with the interpreter using the 486:const:`METH_VARARGS` flag; this is described in section :ref:`methodtable`. The 487:c:func:`PyArg_ParseTuple` function and its arguments are documented in section 488:ref:`parsetuple`. 489 490The macros :c:func:`Py_XINCREF` and :c:func:`Py_XDECREF` increment/decrement the 491reference count of an object and are safe in the presence of *NULL* pointers 492(but note that *temp* will not be *NULL* in this context). More info on them 493in section :ref:`refcounts`. 494 495.. index:: single: PyObject_CallObject() 496 497Later, when it is time to call the function, you call the C function 498:c:func:`PyObject_CallObject`. This function has two arguments, both pointers to 499arbitrary Python objects: the Python function, and the argument list. The 500argument list must always be a tuple object, whose length is the number of 501arguments. To call the Python function with no arguments, pass in NULL, or 502an empty tuple; to call it with one argument, pass a singleton tuple. 503:c:func:`Py_BuildValue` returns a tuple when its format string consists of zero 504or more format codes between parentheses. For example:: 505 506 int arg; 507 PyObject *arglist; 508 PyObject *result; 509 ... 510 arg = 123; 511 ... 512 /* Time to call the callback */ 513 arglist = Py_BuildValue("(i)", arg); 514 result = PyObject_CallObject(my_callback, arglist); 515 Py_DECREF(arglist); 516 517:c:func:`PyObject_CallObject` returns a Python object pointer: this is the return 518value of the Python function. :c:func:`PyObject_CallObject` is 519"reference-count-neutral" with respect to its arguments. In the example a new 520tuple was created to serve as the argument list, which is :c:func:`Py_DECREF`\ 521-ed immediately after the :c:func:`PyObject_CallObject` call. 522 523The return value of :c:func:`PyObject_CallObject` is "new": either it is a brand 524new object, or it is an existing object whose reference count has been 525incremented. So, unless you want to save it in a global variable, you should 526somehow :c:func:`Py_DECREF` the result, even (especially!) if you are not 527interested in its value. 528 529Before you do this, however, it is important to check that the return value 530isn't *NULL*. If it is, the Python function terminated by raising an exception. 531If the C code that called :c:func:`PyObject_CallObject` is called from Python, it 532should now return an error indication to its Python caller, so the interpreter 533can print a stack trace, or the calling Python code can handle the exception. 534If this is not possible or desirable, the exception should be cleared by calling 535:c:func:`PyErr_Clear`. For example:: 536 537 if (result == NULL) 538 return NULL; /* Pass error back */ 539 ...use result... 540 Py_DECREF(result); 541 542Depending on the desired interface to the Python callback function, you may also 543have to provide an argument list to :c:func:`PyObject_CallObject`. In some cases 544the argument list is also provided by the Python program, through the same 545interface that specified the callback function. It can then be saved and used 546in the same manner as the function object. In other cases, you may have to 547construct a new tuple to pass as the argument list. The simplest way to do this 548is to call :c:func:`Py_BuildValue`. For example, if you want to pass an integral 549event code, you might use the following code:: 550 551 PyObject *arglist; 552 ... 553 arglist = Py_BuildValue("(l)", eventcode); 554 result = PyObject_CallObject(my_callback, arglist); 555 Py_DECREF(arglist); 556 if (result == NULL) 557 return NULL; /* Pass error back */ 558 /* Here maybe use the result */ 559 Py_DECREF(result); 560 561Note the placement of ``Py_DECREF(arglist)`` immediately after the call, before 562the error check! Also note that strictly speaking this code is not complete: 563:c:func:`Py_BuildValue` may run out of memory, and this should be checked. 564 565You may also call a function with keyword arguments by using 566:c:func:`PyObject_Call`, which supports arguments and keyword arguments. As in 567the above example, we use :c:func:`Py_BuildValue` to construct the dictionary. :: 568 569 PyObject *dict; 570 ... 571 dict = Py_BuildValue("{s:i}", "name", val); 572 result = PyObject_Call(my_callback, NULL, dict); 573 Py_DECREF(dict); 574 if (result == NULL) 575 return NULL; /* Pass error back */ 576 /* Here maybe use the result */ 577 Py_DECREF(result); 578 579 580.. _parsetuple: 581 582Extracting Parameters in Extension Functions 583============================================ 584 585.. index:: single: PyArg_ParseTuple() 586 587The :c:func:`PyArg_ParseTuple` function is declared as follows:: 588 589 int PyArg_ParseTuple(PyObject *arg, char *format, ...); 590 591The *arg* argument must be a tuple object containing an argument list passed 592from Python to a C function. The *format* argument must be a format string, 593whose syntax is explained in :ref:`arg-parsing` in the Python/C API Reference 594Manual. The remaining arguments must be addresses of variables whose type is 595determined by the format string. 596 597Note that while :c:func:`PyArg_ParseTuple` checks that the Python arguments have 598the required types, it cannot check the validity of the addresses of C variables 599passed to the call: if you make mistakes there, your code will probably crash or 600at least overwrite random bits in memory. So be careful! 601 602Note that any Python object references which are provided to the caller are 603*borrowed* references; do not decrement their reference count! 604 605Some example calls:: 606 607 int ok; 608 int i, j; 609 long k, l; 610 const char *s; 611 int size; 612 613 ok = PyArg_ParseTuple(args, ""); /* No arguments */ 614 /* Python call: f() */ 615 616:: 617 618 ok = PyArg_ParseTuple(args, "s", &s); /* A string */ 619 /* Possible Python call: f('whoops!') */ 620 621:: 622 623 ok = PyArg_ParseTuple(args, "lls", &k, &l, &s); /* Two longs and a string */ 624 /* Possible Python call: f(1, 2, 'three') */ 625 626:: 627 628 ok = PyArg_ParseTuple(args, "(ii)s#", &i, &j, &s, &size); 629 /* A pair of ints and a string, whose size is also returned */ 630 /* Possible Python call: f((1, 2), 'three') */ 631 632:: 633 634 { 635 const char *file; 636 const char *mode = "r"; 637 int bufsize = 0; 638 ok = PyArg_ParseTuple(args, "s|si", &file, &mode, &bufsize); 639 /* A string, and optionally another string and an integer */ 640 /* Possible Python calls: 641 f('spam') 642 f('spam', 'w') 643 f('spam', 'wb', 100000) */ 644 } 645 646:: 647 648 { 649 int left, top, right, bottom, h, v; 650 ok = PyArg_ParseTuple(args, "((ii)(ii))(ii)", 651 &left, &top, &right, &bottom, &h, &v); 652 /* A rectangle and a point */ 653 /* Possible Python call: 654 f(((0, 0), (400, 300)), (10, 10)) */ 655 } 656 657:: 658 659 { 660 Py_complex c; 661 ok = PyArg_ParseTuple(args, "D:myfunction", &c); 662 /* a complex, also providing a function name for errors */ 663 /* Possible Python call: myfunction(1+2j) */ 664 } 665 666 667.. _parsetupleandkeywords: 668 669Keyword Parameters for Extension Functions 670========================================== 671 672.. index:: single: PyArg_ParseTupleAndKeywords() 673 674The :c:func:`PyArg_ParseTupleAndKeywords` function is declared as follows:: 675 676 int PyArg_ParseTupleAndKeywords(PyObject *arg, PyObject *kwdict, 677 char *format, char *kwlist[], ...); 678 679The *arg* and *format* parameters are identical to those of the 680:c:func:`PyArg_ParseTuple` function. The *kwdict* parameter is the dictionary of 681keywords received as the third parameter from the Python runtime. The *kwlist* 682parameter is a *NULL*-terminated list of strings which identify the parameters; 683the names are matched with the type information from *format* from left to 684right. On success, :c:func:`PyArg_ParseTupleAndKeywords` returns true, otherwise 685it returns false and raises an appropriate exception. 686 687.. note:: 688 689 Nested tuples cannot be parsed when using keyword arguments! Keyword parameters 690 passed in which are not present in the *kwlist* will cause :exc:`TypeError` to 691 be raised. 692 693.. index:: single: Philbrick, Geoff 694 695Here is an example module which uses keywords, based on an example by Geoff 696Philbrick (philbrick@hks.com):: 697 698 #include "Python.h" 699 700 static PyObject * 701 keywdarg_parrot(PyObject *self, PyObject *args, PyObject *keywds) 702 { 703 int voltage; 704 char *state = "a stiff"; 705 char *action = "voom"; 706 char *type = "Norwegian Blue"; 707 708 static char *kwlist[] = {"voltage", "state", "action", "type", NULL}; 709 710 if (!PyArg_ParseTupleAndKeywords(args, keywds, "i|sss", kwlist, 711 &voltage, &state, &action, &type)) 712 return NULL; 713 714 printf("-- This parrot wouldn't %s if you put %i Volts through it.\n", 715 action, voltage); 716 printf("-- Lovely plumage, the %s -- It's %s!\n", type, state); 717 718 Py_INCREF(Py_None); 719 720 return Py_None; 721 } 722 723 static PyMethodDef keywdarg_methods[] = { 724 /* The cast of the function is necessary since PyCFunction values 725 * only take two PyObject* parameters, and keywdarg_parrot() takes 726 * three. 727 */ 728 {"parrot", (PyCFunction)keywdarg_parrot, METH_VARARGS | METH_KEYWORDS, 729 "Print a lovely skit to standard output."}, 730 {NULL, NULL, 0, NULL} /* sentinel */ 731 }; 732 733:: 734 735 void 736 initkeywdarg(void) 737 { 738 /* Create the module and add the functions */ 739 Py_InitModule("keywdarg", keywdarg_methods); 740 } 741 742 743.. _buildvalue: 744 745Building Arbitrary Values 746========================= 747 748This function is the counterpart to :c:func:`PyArg_ParseTuple`. It is declared 749as follows:: 750 751 PyObject *Py_BuildValue(char *format, ...); 752 753It recognizes a set of format units similar to the ones recognized by 754:c:func:`PyArg_ParseTuple`, but the arguments (which are input to the function, 755not output) must not be pointers, just values. It returns a new Python object, 756suitable for returning from a C function called from Python. 757 758One difference with :c:func:`PyArg_ParseTuple`: while the latter requires its 759first argument to be a tuple (since Python argument lists are always represented 760as tuples internally), :c:func:`Py_BuildValue` does not always build a tuple. It 761builds a tuple only if its format string contains two or more format units. If 762the format string is empty, it returns ``None``; if it contains exactly one 763format unit, it returns whatever object is described by that format unit. To 764force it to return a tuple of size 0 or one, parenthesize the format string. 765 766Examples (to the left the call, to the right the resulting Python value): 767 768.. code-block:: none 769 770 Py_BuildValue("") None 771 Py_BuildValue("i", 123) 123 772 Py_BuildValue("iii", 123, 456, 789) (123, 456, 789) 773 Py_BuildValue("s", "hello") 'hello' 774 Py_BuildValue("ss", "hello", "world") ('hello', 'world') 775 Py_BuildValue("s#", "hello", 4) 'hell' 776 Py_BuildValue("()") () 777 Py_BuildValue("(i)", 123) (123,) 778 Py_BuildValue("(ii)", 123, 456) (123, 456) 779 Py_BuildValue("(i,i)", 123, 456) (123, 456) 780 Py_BuildValue("[i,i]", 123, 456) [123, 456] 781 Py_BuildValue("{s:i,s:i}", 782 "abc", 123, "def", 456) {'abc': 123, 'def': 456} 783 Py_BuildValue("((ii)(ii)) (ii)", 784 1, 2, 3, 4, 5, 6) (((1, 2), (3, 4)), (5, 6)) 785 786 787.. _refcounts: 788 789Reference Counts 790================ 791 792In languages like C or C++, the programmer is responsible for dynamic allocation 793and deallocation of memory on the heap. In C, this is done using the functions 794:c:func:`malloc` and :c:func:`free`. In C++, the operators ``new`` and 795``delete`` are used with essentially the same meaning and we'll restrict 796the following discussion to the C case. 797 798Every block of memory allocated with :c:func:`malloc` should eventually be 799returned to the pool of available memory by exactly one call to :c:func:`free`. 800It is important to call :c:func:`free` at the right time. If a block's address 801is forgotten but :c:func:`free` is not called for it, the memory it occupies 802cannot be reused until the program terminates. This is called a :dfn:`memory 803leak`. On the other hand, if a program calls :c:func:`free` for a block and then 804continues to use the block, it creates a conflict with re-use of the block 805through another :c:func:`malloc` call. This is called :dfn:`using freed memory`. 806It has the same bad consequences as referencing uninitialized data --- core 807dumps, wrong results, mysterious crashes. 808 809Common causes of memory leaks are unusual paths through the code. For instance, 810a function may allocate a block of memory, do some calculation, and then free 811the block again. Now a change in the requirements for the function may add a 812test to the calculation that detects an error condition and can return 813prematurely from the function. It's easy to forget to free the allocated memory 814block when taking this premature exit, especially when it is added later to the 815code. Such leaks, once introduced, often go undetected for a long time: the 816error exit is taken only in a small fraction of all calls, and most modern 817machines have plenty of virtual memory, so the leak only becomes apparent in a 818long-running process that uses the leaking function frequently. Therefore, it's 819important to prevent leaks from happening by having a coding convention or 820strategy that minimizes this kind of errors. 821 822Since Python makes heavy use of :c:func:`malloc` and :c:func:`free`, it needs a 823strategy to avoid memory leaks as well as the use of freed memory. The chosen 824method is called :dfn:`reference counting`. The principle is simple: every 825object contains a counter, which is incremented when a reference to the object 826is stored somewhere, and which is decremented when a reference to it is deleted. 827When the counter reaches zero, the last reference to the object has been deleted 828and the object is freed. 829 830An alternative strategy is called :dfn:`automatic garbage collection`. 831(Sometimes, reference counting is also referred to as a garbage collection 832strategy, hence my use of "automatic" to distinguish the two.) The big 833advantage of automatic garbage collection is that the user doesn't need to call 834:c:func:`free` explicitly. (Another claimed advantage is an improvement in speed 835or memory usage --- this is no hard fact however.) The disadvantage is that for 836C, there is no truly portable automatic garbage collector, while reference 837counting can be implemented portably (as long as the functions :c:func:`malloc` 838and :c:func:`free` are available --- which the C Standard guarantees). Maybe some 839day a sufficiently portable automatic garbage collector will be available for C. 840Until then, we'll have to live with reference counts. 841 842While Python uses the traditional reference counting implementation, it also 843offers a cycle detector that works to detect reference cycles. This allows 844applications to not worry about creating direct or indirect circular references; 845these are the weakness of garbage collection implemented using only reference 846counting. Reference cycles consist of objects which contain (possibly indirect) 847references to themselves, so that each object in the cycle has a reference count 848which is non-zero. Typical reference counting implementations are not able to 849reclaim the memory belonging to any objects in a reference cycle, or referenced 850from the objects in the cycle, even though there are no further references to 851the cycle itself. 852 853The cycle detector is able to detect garbage cycles and can reclaim them so long 854as there are no finalizers implemented in Python (:meth:`__del__` methods). 855When there are such finalizers, the detector exposes the cycles through the 856:mod:`gc` module (specifically, the :attr:`~gc.garbage` variable in that module). 857The :mod:`gc` module also exposes a way to run the detector (the 858:func:`~gc.collect` function), as well as configuration 859interfaces and the ability to disable the detector at runtime. The cycle 860detector is considered an optional component; though it is included by default, 861it can be disabled at build time using the :option:`!--without-cycle-gc` option 862to the :program:`configure` script on Unix platforms (including Mac OS X) or by 863removing the definition of ``WITH_CYCLE_GC`` in the :file:`pyconfig.h` header on 864other platforms. If the cycle detector is disabled in this way, the :mod:`gc` 865module will not be available. 866 867 868.. _refcountsinpython: 869 870Reference Counting in Python 871---------------------------- 872 873There are two macros, ``Py_INCREF(x)`` and ``Py_DECREF(x)``, which handle the 874incrementing and decrementing of the reference count. :c:func:`Py_DECREF` also 875frees the object when the count reaches zero. For flexibility, it doesn't call 876:c:func:`free` directly --- rather, it makes a call through a function pointer in 877the object's :dfn:`type object`. For this purpose (and others), every object 878also contains a pointer to its type object. 879 880The big question now remains: when to use ``Py_INCREF(x)`` and ``Py_DECREF(x)``? 881Let's first introduce some terms. Nobody "owns" an object; however, you can 882:dfn:`own a reference` to an object. An object's reference count is now defined 883as the number of owned references to it. The owner of a reference is 884responsible for calling :c:func:`Py_DECREF` when the reference is no longer 885needed. Ownership of a reference can be transferred. There are three ways to 886dispose of an owned reference: pass it on, store it, or call :c:func:`Py_DECREF`. 887Forgetting to dispose of an owned reference creates a memory leak. 888 889It is also possible to :dfn:`borrow` [#]_ a reference to an object. The 890borrower of a reference should not call :c:func:`Py_DECREF`. The borrower must 891not hold on to the object longer than the owner from which it was borrowed. 892Using a borrowed reference after the owner has disposed of it risks using freed 893memory and should be avoided completely. [#]_ 894 895The advantage of borrowing over owning a reference is that you don't need to 896take care of disposing of the reference on all possible paths through the code 897--- in other words, with a borrowed reference you don't run the risk of leaking 898when a premature exit is taken. The disadvantage of borrowing over owning is 899that there are some subtle situations where in seemingly correct code a borrowed 900reference can be used after the owner from which it was borrowed has in fact 901disposed of it. 902 903A borrowed reference can be changed into an owned reference by calling 904:c:func:`Py_INCREF`. This does not affect the status of the owner from which the 905reference was borrowed --- it creates a new owned reference, and gives full 906owner responsibilities (the new owner must dispose of the reference properly, as 907well as the previous owner). 908 909 910.. _ownershiprules: 911 912Ownership Rules 913--------------- 914 915Whenever an object reference is passed into or out of a function, it is part of 916the function's interface specification whether ownership is transferred with the 917reference or not. 918 919Most functions that return a reference to an object pass on ownership with the 920reference. In particular, all functions whose function it is to create a new 921object, such as :c:func:`PyInt_FromLong` and :c:func:`Py_BuildValue`, pass 922ownership to the receiver. Even if the object is not actually new, you still 923receive ownership of a new reference to that object. For instance, 924:c:func:`PyInt_FromLong` maintains a cache of popular values and can return a 925reference to a cached item. 926 927Many functions that extract objects from other objects also transfer ownership 928with the reference, for instance :c:func:`PyObject_GetAttrString`. The picture 929is less clear, here, however, since a few common routines are exceptions: 930:c:func:`PyTuple_GetItem`, :c:func:`PyList_GetItem`, :c:func:`PyDict_GetItem`, and 931:c:func:`PyDict_GetItemString` all return references that you borrow from the 932tuple, list or dictionary. 933 934The function :c:func:`PyImport_AddModule` also returns a borrowed reference, even 935though it may actually create the object it returns: this is possible because an 936owned reference to the object is stored in ``sys.modules``. 937 938When you pass an object reference into another function, in general, the 939function borrows the reference from you --- if it needs to store it, it will use 940:c:func:`Py_INCREF` to become an independent owner. There are exactly two 941important exceptions to this rule: :c:func:`PyTuple_SetItem` and 942:c:func:`PyList_SetItem`. These functions take over ownership of the item passed 943to them --- even if they fail! (Note that :c:func:`PyDict_SetItem` and friends 944don't take over ownership --- they are "normal.") 945 946When a C function is called from Python, it borrows references to its arguments 947from the caller. The caller owns a reference to the object, so the borrowed 948reference's lifetime is guaranteed until the function returns. Only when such a 949borrowed reference must be stored or passed on, it must be turned into an owned 950reference by calling :c:func:`Py_INCREF`. 951 952The object reference returned from a C function that is called from Python must 953be an owned reference --- ownership is transferred from the function to its 954caller. 955 956 957.. _thinice: 958 959Thin Ice 960-------- 961 962There are a few situations where seemingly harmless use of a borrowed reference 963can lead to problems. These all have to do with implicit invocations of the 964interpreter, which can cause the owner of a reference to dispose of it. 965 966The first and most important case to know about is using :c:func:`Py_DECREF` on 967an unrelated object while borrowing a reference to a list item. For instance:: 968 969 void 970 bug(PyObject *list) 971 { 972 PyObject *item = PyList_GetItem(list, 0); 973 974 PyList_SetItem(list, 1, PyInt_FromLong(0L)); 975 PyObject_Print(item, stdout, 0); /* BUG! */ 976 } 977 978This function first borrows a reference to ``list[0]``, then replaces 979``list[1]`` with the value ``0``, and finally prints the borrowed reference. 980Looks harmless, right? But it's not! 981 982Let's follow the control flow into :c:func:`PyList_SetItem`. The list owns 983references to all its items, so when item 1 is replaced, it has to dispose of 984the original item 1. Now let's suppose the original item 1 was an instance of a 985user-defined class, and let's further suppose that the class defined a 986:meth:`__del__` method. If this class instance has a reference count of 1, 987disposing of it will call its :meth:`__del__` method. 988 989Since it is written in Python, the :meth:`__del__` method can execute arbitrary 990Python code. Could it perhaps do something to invalidate the reference to 991``item`` in :c:func:`bug`? You bet! Assuming that the list passed into 992:c:func:`bug` is accessible to the :meth:`__del__` method, it could execute a 993statement to the effect of ``del list[0]``, and assuming this was the last 994reference to that object, it would free the memory associated with it, thereby 995invalidating ``item``. 996 997The solution, once you know the source of the problem, is easy: temporarily 998increment the reference count. The correct version of the function reads:: 999 1000 void 1001 no_bug(PyObject *list) 1002 { 1003 PyObject *item = PyList_GetItem(list, 0); 1004 1005 Py_INCREF(item); 1006 PyList_SetItem(list, 1, PyInt_FromLong(0L)); 1007 PyObject_Print(item, stdout, 0); 1008 Py_DECREF(item); 1009 } 1010 1011This is a true story. An older version of Python contained variants of this bug 1012and someone spent a considerable amount of time in a C debugger to figure out 1013why his :meth:`__del__` methods would fail... 1014 1015The second case of problems with a borrowed reference is a variant involving 1016threads. Normally, multiple threads in the Python interpreter can't get in each 1017other's way, because there is a global lock protecting Python's entire object 1018space. However, it is possible to temporarily release this lock using the macro 1019:c:macro:`Py_BEGIN_ALLOW_THREADS`, and to re-acquire it using 1020:c:macro:`Py_END_ALLOW_THREADS`. This is common around blocking I/O calls, to 1021let other threads use the processor while waiting for the I/O to complete. 1022Obviously, the following function has the same problem as the previous one:: 1023 1024 void 1025 bug(PyObject *list) 1026 { 1027 PyObject *item = PyList_GetItem(list, 0); 1028 Py_BEGIN_ALLOW_THREADS 1029 ...some blocking I/O call... 1030 Py_END_ALLOW_THREADS 1031 PyObject_Print(item, stdout, 0); /* BUG! */ 1032 } 1033 1034 1035.. _nullpointers: 1036 1037NULL Pointers 1038------------- 1039 1040In general, functions that take object references as arguments do not expect you 1041to pass them *NULL* pointers, and will dump core (or cause later core dumps) if 1042you do so. Functions that return object references generally return *NULL* only 1043to indicate that an exception occurred. The reason for not testing for *NULL* 1044arguments is that functions often pass the objects they receive on to other 1045function --- if each function were to test for *NULL*, there would be a lot of 1046redundant tests and the code would run more slowly. 1047 1048It is better to test for *NULL* only at the "source:" when a pointer that may be 1049*NULL* is received, for example, from :c:func:`malloc` or from a function that 1050may raise an exception. 1051 1052The macros :c:func:`Py_INCREF` and :c:func:`Py_DECREF` do not check for *NULL* 1053pointers --- however, their variants :c:func:`Py_XINCREF` and :c:func:`Py_XDECREF` 1054do. 1055 1056The macros for checking for a particular object type (``Pytype_Check()``) don't 1057check for *NULL* pointers --- again, there is much code that calls several of 1058these in a row to test an object against various different expected types, and 1059this would generate redundant tests. There are no variants with *NULL* 1060checking. 1061 1062The C function calling mechanism guarantees that the argument list passed to C 1063functions (``args`` in the examples) is never *NULL* --- in fact it guarantees 1064that it is always a tuple. [#]_ 1065 1066It is a severe error to ever let a *NULL* pointer "escape" to the Python user. 1067 1068.. Frank Stajano: 1069 A pedagogically buggy example, along the lines of the previous listing, would 1070 be helpful here -- showing in more concrete terms what sort of actions could 1071 cause the problem. I can't very well imagine it from the description. 1072 1073 1074.. _cplusplus: 1075 1076Writing Extensions in C++ 1077========================= 1078 1079It is possible to write extension modules in C++. Some restrictions apply. If 1080the main program (the Python interpreter) is compiled and linked by the C 1081compiler, global or static objects with constructors cannot be used. This is 1082not a problem if the main program is linked by the C++ compiler. Functions that 1083will be called by the Python interpreter (in particular, module initialization 1084functions) have to be declared using ``extern "C"``. It is unnecessary to 1085enclose the Python header files in ``extern "C" {...}`` --- they use this form 1086already if the symbol ``__cplusplus`` is defined (all recent C++ compilers 1087define this symbol). 1088 1089 1090.. _using-capsules: 1091 1092Providing a C API for an Extension Module 1093========================================= 1094 1095.. sectionauthor:: Konrad Hinsen <hinsen@cnrs-orleans.fr> 1096 1097 1098Many extension modules just provide new functions and types to be used from 1099Python, but sometimes the code in an extension module can be useful for other 1100extension modules. For example, an extension module could implement a type 1101"collection" which works like lists without order. Just like the standard Python 1102list type has a C API which permits extension modules to create and manipulate 1103lists, this new collection type should have a set of C functions for direct 1104manipulation from other extension modules. 1105 1106At first sight this seems easy: just write the functions (without declaring them 1107``static``, of course), provide an appropriate header file, and document 1108the C API. And in fact this would work if all extension modules were always 1109linked statically with the Python interpreter. When modules are used as shared 1110libraries, however, the symbols defined in one module may not be visible to 1111another module. The details of visibility depend on the operating system; some 1112systems use one global namespace for the Python interpreter and all extension 1113modules (Windows, for example), whereas others require an explicit list of 1114imported symbols at module link time (AIX is one example), or offer a choice of 1115different strategies (most Unices). And even if symbols are globally visible, 1116the module whose functions one wishes to call might not have been loaded yet! 1117 1118Portability therefore requires not to make any assumptions about symbol 1119visibility. This means that all symbols in extension modules should be declared 1120``static``, except for the module's initialization function, in order to 1121avoid name clashes with other extension modules (as discussed in section 1122:ref:`methodtable`). And it means that symbols that *should* be accessible from 1123other extension modules must be exported in a different way. 1124 1125Python provides a special mechanism to pass C-level information (pointers) from 1126one extension module to another one: Capsules. A Capsule is a Python data type 1127which stores a pointer (:c:type:`void \*`). Capsules can only be created and 1128accessed via their C API, but they can be passed around like any other Python 1129object. In particular, they can be assigned to a name in an extension module's 1130namespace. Other extension modules can then import this module, retrieve the 1131value of this name, and then retrieve the pointer from the Capsule. 1132 1133There are many ways in which Capsules can be used to export the C API of an 1134extension module. Each function could get its own Capsule, or all C API pointers 1135could be stored in an array whose address is published in a Capsule. And the 1136various tasks of storing and retrieving the pointers can be distributed in 1137different ways between the module providing the code and the client modules. 1138 1139Whichever method you choose, it's important to name your Capsules properly. 1140The function :c:func:`PyCapsule_New` takes a name parameter 1141(:c:type:`const char \*`); you're permitted to pass in a *NULL* name, but 1142we strongly encourage you to specify a name. Properly named Capsules provide 1143a degree of runtime type-safety; there is no feasible way to tell one unnamed 1144Capsule from another. 1145 1146In particular, Capsules used to expose C APIs should be given a name following 1147this convention:: 1148 1149 modulename.attributename 1150 1151The convenience function :c:func:`PyCapsule_Import` makes it easy to 1152load a C API provided via a Capsule, but only if the Capsule's name 1153matches this convention. This behavior gives C API users a high degree 1154of certainty that the Capsule they load contains the correct C API. 1155 1156The following example demonstrates an approach that puts most of the burden on 1157the writer of the exporting module, which is appropriate for commonly used 1158library modules. It stores all C API pointers (just one in the example!) in an 1159array of :c:type:`void` pointers which becomes the value of a Capsule. The header 1160file corresponding to the module provides a macro that takes care of importing 1161the module and retrieving its C API pointers; client modules only have to call 1162this macro before accessing the C API. 1163 1164The exporting module is a modification of the :mod:`spam` module from section 1165:ref:`extending-simpleexample`. The function :func:`spam.system` does not call 1166the C library function :c:func:`system` directly, but a function 1167:c:func:`PySpam_System`, which would of course do something more complicated in 1168reality (such as adding "spam" to every command). This function 1169:c:func:`PySpam_System` is also exported to other extension modules. 1170 1171The function :c:func:`PySpam_System` is a plain C function, declared 1172``static`` like everything else:: 1173 1174 static int 1175 PySpam_System(const char *command) 1176 { 1177 return system(command); 1178 } 1179 1180The function :c:func:`spam_system` is modified in a trivial way:: 1181 1182 static PyObject * 1183 spam_system(PyObject *self, PyObject *args) 1184 { 1185 const char *command; 1186 int sts; 1187 1188 if (!PyArg_ParseTuple(args, "s", &command)) 1189 return NULL; 1190 sts = PySpam_System(command); 1191 return Py_BuildValue("i", sts); 1192 } 1193 1194In the beginning of the module, right after the line :: 1195 1196 #include "Python.h" 1197 1198two more lines must be added:: 1199 1200 #define SPAM_MODULE 1201 #include "spammodule.h" 1202 1203The ``#define`` is used to tell the header file that it is being included in the 1204exporting module, not a client module. Finally, the module's initialization 1205function must take care of initializing the C API pointer array:: 1206 1207 PyMODINIT_FUNC 1208 initspam(void) 1209 { 1210 PyObject *m; 1211 static void *PySpam_API[PySpam_API_pointers]; 1212 PyObject *c_api_object; 1213 1214 m = Py_InitModule("spam", SpamMethods); 1215 if (m == NULL) 1216 return; 1217 1218 /* Initialize the C API pointer array */ 1219 PySpam_API[PySpam_System_NUM] = (void *)PySpam_System; 1220 1221 /* Create a Capsule containing the API pointer array's address */ 1222 c_api_object = PyCapsule_New((void *)PySpam_API, "spam._C_API", NULL); 1223 1224 if (c_api_object != NULL) 1225 PyModule_AddObject(m, "_C_API", c_api_object); 1226 } 1227 1228Note that ``PySpam_API`` is declared ``static``; otherwise the pointer 1229array would disappear when :func:`initspam` terminates! 1230 1231The bulk of the work is in the header file :file:`spammodule.h`, which looks 1232like this:: 1233 1234 #ifndef Py_SPAMMODULE_H 1235 #define Py_SPAMMODULE_H 1236 #ifdef __cplusplus 1237 extern "C" { 1238 #endif 1239 1240 /* Header file for spammodule */ 1241 1242 /* C API functions */ 1243 #define PySpam_System_NUM 0 1244 #define PySpam_System_RETURN int 1245 #define PySpam_System_PROTO (const char *command) 1246 1247 /* Total number of C API pointers */ 1248 #define PySpam_API_pointers 1 1249 1250 1251 #ifdef SPAM_MODULE 1252 /* This section is used when compiling spammodule.c */ 1253 1254 static PySpam_System_RETURN PySpam_System PySpam_System_PROTO; 1255 1256 #else 1257 /* This section is used in modules that use spammodule's API */ 1258 1259 static void **PySpam_API; 1260 1261 #define PySpam_System \ 1262 (*(PySpam_System_RETURN (*)PySpam_System_PROTO) PySpam_API[PySpam_System_NUM]) 1263 1264 /* Return -1 on error, 0 on success. 1265 * PyCapsule_Import will set an exception if there's an error. 1266 */ 1267 static int 1268 import_spam(void) 1269 { 1270 PySpam_API = (void **)PyCapsule_Import("spam._C_API", 0); 1271 return (PySpam_API != NULL) ? 0 : -1; 1272 } 1273 1274 #endif 1275 1276 #ifdef __cplusplus 1277 } 1278 #endif 1279 1280 #endif /* !defined(Py_SPAMMODULE_H) */ 1281 1282All that a client module must do in order to have access to the function 1283:c:func:`PySpam_System` is to call the function (or rather macro) 1284:c:func:`import_spam` in its initialization function:: 1285 1286 PyMODINIT_FUNC 1287 initclient(void) 1288 { 1289 PyObject *m; 1290 1291 m = Py_InitModule("client", ClientMethods); 1292 if (m == NULL) 1293 return; 1294 if (import_spam() < 0) 1295 return; 1296 /* additional initialization can happen here */ 1297 } 1298 1299The main disadvantage of this approach is that the file :file:`spammodule.h` is 1300rather complicated. However, the basic structure is the same for each function 1301that is exported, so it has to be learned only once. 1302 1303Finally it should be mentioned that Capsules offer additional functionality, 1304which is especially useful for memory allocation and deallocation of the pointer 1305stored in a Capsule. The details are described in the Python/C API Reference 1306Manual in the section :ref:`capsules` and in the implementation of Capsules (files 1307:file:`Include/pycapsule.h` and :file:`Objects/pycapsule.c` in the Python source 1308code distribution). 1309 1310.. rubric:: Footnotes 1311 1312.. [#] An interface for this function already exists in the standard module :mod:`os` 1313 --- it was chosen as a simple and straightforward example. 1314 1315.. [#] The metaphor of "borrowing" a reference is not completely correct: the owner 1316 still has a copy of the reference. 1317 1318.. [#] Checking that the reference count is at least 1 **does not work** --- the 1319 reference count itself could be in freed memory and may thus be reused for 1320 another object! 1321 1322.. [#] These guarantees don't hold when you use the "old" style calling convention --- 1323 this is still found in much existing code. 1324 1325