python/doc/building.qbk

[chapter Building and Testing
    [quickbook 1.7]
    [authors [Abrahams, David]]
    [copyright 2002 - 2015 David Abrahams, Stefan Seefeld]
    [id building]
]
[/ Copyright David Abrahams 2006. Distributed under the Boost
 / Software License, Version 1.0. (See accompanying
 / file LICENSE_1_0.txt or copy at
 / http://www.boost.org/LICENSE_1_0.txt)
 /]

[section Requirements]

Boost.Python requires [@http://www.python.org/2.2 Python 2.2]
[footnote Note that although we tested earlier versions of Boost.Python
 with Python 2.2, and we don't *think* we've done anything to break
 compatibility, this release of Boost.Python may not have been tested
 with versions of Python earlier than 2.4, so we're not 100% sure that
 python 2.2 and 2.3 are supported.] *or* [@http://www.python.org newer].

[endsect]
[section Background]

There are two basic models for combining C++ and Python:

* [@http://www.python.org/doc/current/ext/intro.html extending],
  in which the end-user launches the Python interpreter
  executable and imports Python “extension modules” written in C++.
  Think of taking a library written in C++ and giving it a Python
  interface so Python programmers can use it.  From Python, these
  modules look just like regular Python modules.

* [@http://www.python.org/doc/current/ext/embedding.html embedding],
  in which the end-user launches a program written
  in C++ that in turn invokes the Python interpreter as a library
  subroutine.  Think of adding scriptability to an existing
  application.

The key distinction between extending and embedding is the location
of the C++ `main()` function: in the Python interpreter executable,
or in some other program, respectively.  Note that even when
embedding Python in another program, [@http://www.python.org/doc/current/ext/extending-with-embedding.html extension modules are often
the best way to make C/C++ functionality accessible to Python
code], so the use of extension modules is really at the heart of
both models.

Except in rare cases, extension modules are built as
dynamically-loaded libraries with a single entry point, which means
you can change them without rebuilding either the other extension
modules or the executable containing `main()`.

[endsect]
[section No-Install Quickstart]

There is no need to “install Boost” in order to get started using
Boost.Python.  These instructions use _bb_ projects,
which will build those binaries as soon as they're needed.  Your
first tests may take a little longer while you wait for
Boost.Python to build, but doing things this way will save you from
worrying about build intricacies like which library binaries to use
for a specific compiler configuration and figuring out the right
compiler options to use yourself.

[note Of course it's possible to use other build systems to
   build Boost.Python and its extensions, but they are not
   officially supported by Boost.  Moreover *99% of all “I can't
   build Boost.Python” problems come from trying to use another
   build system* without first following these instructions.

   If you want to use another system anyway, we suggest that you
   follow these instructions, and then invoke `bjam` with the

   `-a -o`\ /filename/

   options to dump the build commands it executes to a file, so
   you can see what your alternate build system needs to do.]

[section Basic Procedure]

1. Get Boost; see sections 1 and 2 of the _gsg_.

2. Get the `bjam` build driver.  See section 5 of the _gsg_.

3. cd into the `example/quickstart/` directory of your
   Boost.Python installation, which contains a small example project.

4. Invoke `bjam`.  Replace the “\ `stage`\ “ argument from the
   example invocation from section 5 of the _gsg_ with “\ `test`\ ,“ to
   build all the test targets.  Also add the argument “\ `--verbose-test`\ ”
   to see the output generated by the tests when they are run.
   On Windows, your `bjam` invocation might look something like:
   ``
     C:\\...\\quickstart> bjam toolset=msvc --verbose-test test
   ``
   and on Unix variants, perhaps,
   ``
     .../quickstart$ bjam toolset=gcc --verbose-test test
   ``

[note For the sake of concision, the rest of this guide will use
   unix-style forward slashes in pathnames instead of the
   backslashes with which Windows users may be more familiar.  The forward
   slashes should work everywhere except in
   [@http://www.boost.org/more/getting_started/windows.html#command-prompt
   Command Prompt] windows, where you should use backslashes.]

If you followed this procedure successfully, you will have built an
extension module called `extending` and tested it by running a
Python script called `test_extending.py`.  You will also have
built and run a simple application called `embedding` that embeds
python.

[endsect]
[section In Case of Trouble]

If you're seeing lots of compiler and/or linker error messages,
it's probably because Boost.Build is having trouble finding your
Python installation.  You might want to pass the
`--debug-configuration` option to `bjam` the first few times
you invoke it, to make sure that Boost.Build is correctly locating
all the parts of your Python installation.  If it isn't, consider
[link building.configuring_boost_build Configuring Boost.Build]
as detailed below.

If you're still having trouble, Someone on one of the following
mailing lists may be able to help:

* The _bb_list_ for issues related to Boost.Build
* The _bp_list_ for issues specifically related to Boost.Python

[endsect]
[section In Case Everything Seemed to Work]

Rejoice!  If you're new to Boost.Python, at this point it might be
a good idea to ignore build issues for a while and concentrate on
learning the library by going through the _tutorial_ and perhaps
some of the _reference_, trying out what you've
learned about the API by modifying the quickstart project.

[endsect]
[section Modifying the Example Project]

If you're content to keep your extension module forever in one
source file called `extending.cpp`, inside your Boost.Python
distribution, and import it forever as `extending`, then you can
stop here.  However, it's likely that you will want to make a few
changes.  There are a few things you can do without having to learn
_bb_ in depth.

The project you just built is specified in two files in the current
directory: `boost-build.jam`, which tells `bjam` where it can
find the interpreted code of the Boost build system, and
`Jamroot`, which describes the targets you just built.  These
files are heavily commented, so they should be easy to modify.
Take care, however, to preserve whitespace.  Punctuation such as
`;` will not be recognized as intended by `bjam` if it is not
surrounded by whitespace.

[section Relocate the Project]

You'll probably want to copy this project elsewhere so you can
change it without modifying your Boost distribution.  To do that,
simply

a. copy the entire `example/quickstart/` directory
   into a new directory.

b. In the new copies of `boost-build.jam` and `Jamroot`, locate
   the relative path near the top of the file that is clearly
   marked by a comment, and edit that path so that it refers to the
   same directory your Boost distribution as it referred to when
   the file was in its original location in the
   `example/quickstart/` directory.

For example, if you moved the project from
`/home/dave/boost_1_34_0/libs/python/example/quickstart` to
`/home/dave/my-project`, you could change the first path in
`boost-build.jam` from
``
  ../../../../tools/build/src
``
to
``
  /home/dave/boost_1_34_0/tools/build/src
``
and change the first path in `Jamroot` from
``
  ../../../..
``
to
``
  /home/dave/boost_1_34_0
``

[endsect]
[section Add New or Change Names of Existing Source Files]

The names of additional source files involved in building your
extension module or embedding application can be listed in
`Jamroot` right alongside `extending.cpp` or `embedding.cpp`
respectively.  Just be sure to leave whitespace around each
filename:
``
  … file1.cpp file2.cpp file3.cpp …
``
Naturally, if you want to change the name of a source file you can
tell Boost.Build about it by editing the name in `Jamroot`.

[endsect]
[section Change the Name of your Extension Module]

The name of the extension module is determined by two things:

# the name in `Jamroot` immediately following `python-extension`, and
# the name passed to `BOOST_PYTHON_MODULE` in `extending.cpp`.

To change the name of the extension module from `extending` to
`hello`, you'd edit `Jamroot`, changing
``
  python-extension extending : extending.cpp ;
``
to
``
  python-extension hello : extending.cpp ;
``
and you'd edit extending.cpp, changing

``
  BOOST_PYTHON_MODULE(extending)
``
to
``
  BOOST_PYTHON_MODULE(hello)
``
[endsect]
[endsect]
[endsect]
[section Installing Boost.Python on your System]

Since Boost.Python is a separately-compiled (as opposed to
`header-only`) library, its user relies on the services of a
Boost.Python library binary.

If you need a regular installation of the Boost.Python library
binaries on your system, the _gsg_ will
walk you through the steps of creating one.  If building binaries
from source, you might want to supply the `--with-python`
argument to `bjam` (or the `--with-libraries=python` argument
to `configure`), so only the Boost.Python binary will be built,
rather than all the Boost binaries.

[endsect]
[section Configuring Boost.Build]

As described in the [@http://www.boost.org/build/doc/html/bbv2/overview/configuration.html Boost.Build Reference Manual], a file called
`user-config.jam` in your home directory is used to
specify the tools and libraries available to the build system. You
may need to create or edit `user-config.jam` to tell Boost.Build
how to invoke Python, `#include` its headers, and link with its
libraries.

[note If you are using a unix-variant OS and you ran Boost's
   `configure` script, it may have generated a
   `user-config.jam` for you. [footnote `configure` overwrites the existing
   `user-config.jam` in your home directory (if any) after making a backup of
   the old version.] If your `configure`\ /\ `make` sequence was successful and
   Boost.Python binaries were built, your `user-config.jam` file is probably already
   correct.]

If you have one fairly “standard” python installation for your
platform, you might not need to do anything special to describe it.  If
you haven't configured python in `user-config.jam` (and you don't
specify `--without-python` on the Boost.Build command line),
Boost.Build will automatically execute the equivalent of

``
  import toolset : using ;
  using python ;
``
which automatically looks for Python in the most likely places.
However, that only happens when using the Boost.Python project file
(e.g. when referred to by another project as in the quickstart
method).  If instead you are linking against separately-compiled
Boost.Python binaries, you should set up a `user-config.jam` file
with at least the minimal incantation above.

[section Python Configuration Parameters]

If you have several versions of Python installed, or Python is
installed in an unusual way, you may want to supply any or all of
the following optional parameters to `using python`.

[variablelist
 [[version]

  [the version of Python to use.  Should be in Major.Minor
  format, for example, `2.3`.  Do not include the subminor
  version (i.e. *not* `2.5.1`).  If you have multiple Python
  versions installed, the version will usually be the only
  configuration argument required.]]

 [[cmd-or-prefix]

  [preferably, a command that invokes a Python interpreter.
  Alternatively, the installation prefix for Python libraries and
  header files.  Only use the alternative formulation if there is
  no appropriate Python executable available.]]

 [[*includes*]

  [the `#include` paths for Python headers.  Normally the correct
  path(s) will be automatically deduced from `version` and/or
  `cmd-or-prefix`.]]

 [[*libraries*]

  [the path to Python library binaries.  On MacOS/Darwin,
  you can also pass the path of the Python framework.  Normally the
  correct path(s) will be automatically deduced from `version`
  and/or `cmd-or-prefix`.]]

 [[*condition*]

  [if specified, should be a set of Boost.Build
  properties that are matched against the build configuration when
  Boost.Build selects a Python configuration to use.  See examples
  below for details.]]

 [[*extension-suffix*]

  [A string to append to the name of extension
  modules before the true filename extension.  You almost certainly
  don't need to use this.  Usually this suffix is only used when
  targeting a Windows debug build of Python, and will be set
  automatically for you based on the value of the
  [link building.python_debugging_builds <python-debugging>] feature.
  However, at least one Linux distribution (Ubuntu Feisty Fawn) has
  a specially configured [@https://wiki.ubuntu.com/PyDbgBuilds <python-dbg>]
  package that claims to use such a suffix.]]
  ]

[endsect]
[section Examples]

Note that in the examples below, case and *especially whitespace* are
significant.

* If you have both python 2.5 and python 2.4 installed,
  `user-config.jam` might contain

  ``
    using python : 2.5 ;  # Make both versions of Python available
    using python : 2.4 ;  # To build with python 2.4, add python=2.4
                          # to your command line.
  ``
  The first version configured (2.5) becomes the default.  To build
  against python 2.4, add `python=2.4` to the `bjam` command line.

* If you have python installed in an unusual location, you might
  supply the path to the interpreter in the `cmd-or-prefix`
  parameter:

  ``
    using python : : /usr/local/python-2.6-beta/bin/python ;
  ``

* If you have a separate build of Python for use with a particular
  toolset, you might supply that toolset in the `condition`
  parameter:

  ``
    using python ;  # use for most toolsets

    # Use with Intel C++ toolset
    using python
         : # version
         : c:\\Devel\\Python-2.5-IntelBuild\\PCBuild\\python # cmd-or-prefix
         : # includes
         : # libraries
         : <toolset>intel # condition
         ;
  ``

* If you have downloaded the Python sources and built both the
  normal and the [link building.python_debugging_builds "python debugging"]
  builds from source on Windows, you might see:

  ``
    using python : 2.5 : C:\\src\\Python-2.5\\PCBuild\\python ;
    using python : 2.5 : C:\\src\\Python-2.5\\PCBuild\\python_d
      : # includes
      : # libs
      : <python-debugging>on ;
  ``
* You can set up your user-config.jam so a bjam built under Windows
  can build/test both Windows and Cygwin_ python extensions.  Just pass
  `<target-os>cygwin` in the `condition` parameter
  for the cygwin python installation:

  ``
    # windows installation
    using python ;

    # cygwin installation
    using python : : c:\\cygwin\\bin\\python2.5 : : : <target-os>cygwin ;
  ``
  when you put target-os=cygwin in your build request, it should build
  with the cygwin version of python: [#flavor]_

  ``
    bjam target-os=cygwin toolset=gcc
  ``
  This is supposed to work the other way, too (targeting windows
  python with a [@http://cygwin.com Cygwin] bjam) but it seems as though the support in
  Boost.Build's toolsets for building that way is broken at the
  time of this writing.

* Note that because of [@http://zigzag.cs.msu.su/boost.build/wiki/AlternativeSelection
  the way Boost.Build currently selects target alternatives], you might have be very
  explicit in your build requests.  For example, given:

  ``
    using python : 2.5 ; # a regular windows build
    using python : 2.4 : : : : <target-os>cygwin ;
  ``
  building with
  ``
    bjam target-os=cygwin
  ``

  will yield an error.  Instead, you'll need to write

  ``
    bjam target-os=cygwin/python=2.4
  ``

[endsect]
[endsect]
[section Choosing a Boost.Python Library Binary]

If—instead of letting Boost.Build construct and link with the right
libraries automatically—you choose to use a pre-built Boost.Python
library, you'll need to think about which one to link with.  The
Boost.Python binary comes in both static and dynamic flavors.  Take
care to choose the right flavor for your application. [footnote
Information about how to identify the static and dynamic builds of Boost.Python on
[@http://boost.org/more/getting_started/windows.html#library-naming Windows] /
[@http://boost.org/more/getting_started/unix-variants.html#library-naming Unix variants]]

[section The Dynamic Binary]

The dynamic library is the safest and most-versatile choice:

* A single copy of the library code is used by all extension
  modules built with a given toolset. [footnote Because of the way most \*nix platforms
  share symbols among dynamically-loaded objects, I'm not certain
  that extension modules built with different compiler toolsets
  will always use different copies of the Boost.Python library
  when loaded into the same Python instance.  Not using different
  libraries could be a good thing if the compilers have compatible
  ABIs, because extension modules built with the two libraries
  would be interoperable.  Otherwise, it could spell disaster,
  since an extension module and the Boost.Python library would
  have different ideas of such things as class layout. I would
  appreciate someone doing the experiment to find out what
  happens.]

* The library contains a type conversion registry.  Because one
  registry is shared among all extension modules, instances of a
  class exposed to Python in one dynamically-loaded extension
  module can be passed to functions exposed in another such module.

[endsect]
[section The Static Binary]

It might be appropriate to use the static Boost.Python library in
any of the following cases:

* You are _extending_ python and the types exposed in your
  dynamically-loaded extension module don't need to be used by any
  other Boost.Python extension modules, and you don't care if the
  core library code is duplicated among them.

* You are _embedding_ python in your application and either:

  * You are targeting a Unix variant OS other than MacOS or AIX,
    where the dynamically-loaded extension modules can “see” the
    Boost.Python library symbols that are part of the executable.

  * Or, you have statically linked some Boost.Python extension
    modules into your application and you don't care if any
    dynamically-loaded Boost.Python extension modules are able to
    use the types exposed by your statically-linked extension
    modules (and vice-versa).

[endsect]
[endsect]
[section `#include` Issues]

1. If you should ever have occasion to `#include "python.h"`
   directly in a translation unit of a program using Boost.Python,
   use `#include "boost/python/detail/wrap_python.hpp"` instead.
   It handles several issues necessary for use with Boost.Python,
   one of which is mentioned in the next section.

2. Be sure not to `#include` any system headers before
   `wrap_python.hpp`.  This restriction is actually imposed by
   Python, or more properly, by Python's interaction with your
   operating system.  See
   [@http://docs.python.org/ext/simpleExample.html] for details.

[endsect]
[section Python Debugging Builds]

Python can be built in a special “python debugging” configuration
that adds extra checks and instrumentation that can be very useful
for developers of extension modules.  The data structures used by
the debugging configuration contain additional members, so *a
Python executable built with python debugging enabled cannot be
used with an extension module or library compiled without it, and
vice-versa.*

Since pre-built “python debugging” versions of the Python
executable and libraries are not supplied with most distributions
of Python, [footnote On Unix and similar platforms, a debugging python and associated libraries are built by adding --with-pydebug when configuring the Python build. On Windows, the debugging version of Python is generated by the "Win32 Debug" target of the Visual Studio project in the PCBuild subdirectory of a full Python source code distribution.] and we didn't want to force our users
to build them, Boost.Build does not automatically enable python
debugging in its `debug` build variant (which is the default).
Instead there is a special build property called
`python-debugging` that, when used as a build property, will
define the right preprocessor symbols and select the right
libraries to link with.

On unix-variant platforms, the debugging versions of Python's data
structures will only be used if the symbol `Py_DEBUG` is defined.
On many windows compilers, when extension modules are built with
the preprocessor symbol `_DEBUG`, Python defaults to force
linking with a special debugging version of the Python DLL.  Since
that symbol is very commonly used even when Python is not present,
Boost.Python temporarily undefines `_DEBUG` when `Python.h`
is #included from `boost/python/detail/wrap_python.hpp` - unless
`BOOST_DEBUG_PYTHON` is defined.  The upshot is that if you want
“python debugging”and you aren't using Boost.Build, you should make
sure `BOOST_DEBUG_PYTHON` is defined, or python debugging will be
suppressed.

[endsect]
[section Testing Boost.Python]

To run the full test suite for Boost.Python, invoke `bjam` in the
`test` subdirectory of your Boost.Python distribution.

[endsect]
[section Notes for MinGW (and Cygwin with -mno-cygwin) GCC Users]

If you are using a version of Python prior to 2.4.1 with a MinGW
prior to 3.0.0 (with binutils-2.13.90-20030111-1), you will need to
create a MinGW-compatible version of the Python library; the one
shipped with Python will only work with a Microsoft-compatible
linker. Follow the instructions in the “Non-Microsoft” section of
the “Building Extensions: Tips And Tricks” chapter in
[@https://docs.python.org/2/install/index.html Installing Python Modules]
to create `libpythonXX.a`, where `XX` corresponds to the major and minor
version numbers of your Python installation.

[endsect]