Personal tools
User menu

Knowledge Base/CPP/Boost Python Step By Step

From Thalesians

Jump to: navigation, search

THIS PAGE IS CURRENTLY UNDER CONSTRUCTION! THIS PAGE IS CURRENTLY UNDER CONSTRUCTION! THIS PAGE IS CURRENTLY UNDER CONSTRUCTION!

Contents

Acknowledgements

We would like to thank the following individuals for their help and guidance on the mailing lists c++-sig@python.org, boost-build@lists.boost.org and boost-users@lists.boost.org: David Abrahams, Andrea Denzler, Jurko Gospodnetić, Ralf W. Grosse-Kunstleve, Vladimir Prus, Sohail Somani, Troy D. Straszheim, Roman Yakovenko.

Introduction

In this tutorial we consider Boost.Python — a library for building hybrid systems in Python and C++. Historically, most of the high quality scientific computing and quantitative finance packages are written in C++, yet it is far more convenient to code higher lever logic in Python. Boost.Python attempts to resolve this difficulty. It is a framework for interfacing Python and C++ and it is part of Boost.

Despite the editorial "we", as things stand, this tutorial was written entirely by Paul Bilokon, which is disgraceful. But this doesn't have to be the case and if you would like to contribute and possibly become a co-author, please contact me (us).

What is Python?

Python is a general-purpose, very high-level programming language. It was first released by Guido van Rossum in 1991. Nowadays the language has an open, community-based development model managed by the non-profit Python Software Foundation.

According to the official website, "Python is a dynamic object-oriented programming language that can be used for many kinds of software development. It offers strong support for integration with other languages and tools, comes with extensible standard libraries, and can be learned in a few days. Many Python programmers report substantial productivity gains and feel the language encourages the development of higher quality, more maintainable code."

Python versus C++

C++ is a general-purpose programming language regarded as middle-level, as it comprises a combination of both high-level and low-level language features. It is a much lower-level programming language than Python.

C++ places considerable burden on the programmer by requiring that he delve into the intricacies of memory management, exception safety and the like. The resulting code is complex, verbose and often difficult to read. The learning curve for C++ is steep and there is a high barrier to entry.

Python, with its "programming for everyone" ideology, is ideally suited for scientific computing and quantitative finance, as it can be quickly adopted by people with no formal background in programming. Yet historically most of such systems were developed in C++.

Thus we have a gap that needs to be bridged, the gap between the core C++ code and Python. This is where Boost.Python comes to the rescue.

Boost.Python

According to the QuickStart, "The Boost Python Library is a framework for interfacing Python and C++. It allows you to quickly and seamlessly expose C++ classes functions and objects to Python, and vice-versa, using no special tools — just your C++ compiler. It is designed to wrap C++ interfaces non-intrusively, so that you should not have to change the C++ code at all in order to wrap it, making Boost.Python ideal for exposing 3rd-party libraries to Python. The library's use of advanced metaprogramming techniques simplifies its syntax for users, so that wrapping code takes on the look of a kind of declarative interface definition language (IDL)."

Alternatives

It must be noted that Boost.Python is not the only technology for creating hybrid systems in Python and C++. There is also the Simplified Wrapper and Interface Generator (SWIG). SWIG connects programs written in C and C++ with a variety of high-level programming languages, including Python.

It is rather difficult to compare Boost.Python and SWIG and these technologies are rather different. However, you will find one such comparison here.

The Need for This Tutorial

Despite the existence of a number of Internet resources, the knowledge base for Boost.Python cannot always keep up with the development effort. Nor do we always find all the implementation details in the project's official documentation. Much of Boost.Python is still undocumented. Moreover, some of the ticks of the trade have to be discovered either by consulting the mailing lists (especially C++-sig) or by means of active experimentation. The latter is often painful and constitutes, to quote Peter Norton out of context, "an extravagant waste of human talent".

Most of the content of this tutorial overlaps with that of C++-sig and other mailing lists. However, the information on the mailing lists often lacks systematisation and the answers to specific questions are often difficult to find. Here we attempt to add more structure to this information and make it more discoverable.

In this tutorial we endeavour to teach the reader to use Boost.Python step by step, exactly how we discovered it. This tutorial is by no means complete as we are still learning Boost.Python. You contributions are most welcome.

Prerequisites

Let us make sure that we are properly prepared before we proceed. Although in the sequel we are talking about the prerequisites on a Windows system there is certainly no reason why you cannot use this tutorial on a *nix or any other system. If you have any system specific wisdom to share, please do not hesitate to contact the authors.

Overview of Software Requirements

At the very least, we should have the following software installed:

The direct download links for this software at the time of writing are:

Just in case, we have also set up mirrors for some of the above:

Not that this is terribly important, on our computer the installation directories are:

  • Microsoft Visual Studio 2005: C:\Program Files\Microsoft Visual Studio 8
  • Boost C++ Libraries: E:\dev\external\Boost\boost_1_35_0
  • BJam: E:\dev\external\Boost\boost_1_35_0\bjam
  • Active Python: C:\Programs\Win32\Python\V2.5.2.2

We shall keep referring to these directories in the sequel. If on your system they are different, you should make the appropriate amendments.

Although we have tested our examples on Microsoft Visual Studio 2005, they are almost certain to work with 2003 and 2008. The readers' feedback is welcome. However, we cannot guarantee this, so we will keep referring to Microsoft Visual Studio 2005.

Boost C++ libraries can be downloaded free of charge. We have tested the examples with ver. 1.34.1 and 1.35.0. Other versions may work just as well.

BJam may need to be downloaded separately. It is distributed as an archive containing two files: bjam.exe and LICENSE_1_0.txt. We put them under the directory E:\dev\external\Boost\boost_1_35_0\bjam.

ActiveState ActivePython is one of the most popular Python ports on Windows systems. Unfortunately, we were unable to get the 64-bit ActivePython to work with Boost.Python (please see the notes at the end of this tutorial), therefore we recommend that you use the 32-bit edition even on 64-bit systems.

Installation

Microsoft Visual Studio 2005

Follow the standard installation procedure for this product and install it to C:\Program Files\Microsoft Visual Studio 8

Boost C++ Libraries

  • Download boost_1_35_0.zip from Boost.org or our mirror.
  • Create the directory E:\dev\external\Boost\boost_1_35_0
  • Decompress boost_1_35_0.zip into E:\dev\external\Boost\boost_1_35_0, so that you have
    • E:\dev\external\Boost\boost_1_35_0\boost
    • E:\dev\external\Boost\boost_1_35_0\doc
    • E:\dev\external\Boost\boost_1_35_0\libs
    • ...

BJam

  • Download boost-jam-3.1.16-1-ntx86.zip from Boost.org or our mirror.
  • Create the directory E:\dev\external\Boost\boost_1_35_0\bjam
  • Decompress boost-jam-3.1.16-1-ntx86.zip into E:\dev\external\Boost\boost_1_35_0\bjam, so that you have
    • E:\dev\external\Boost\boost_1_35_0\bjam\bjam.exe
    • E:\dev\external\Boost\boost_1_35_0\bjam\LICENSE_1_0.txt

ActiveState ActivePython

  • Download ActivePython-2.5.2.2-win32-x86.msi from ActiveState or our mirror.
  • Follow the installation procedure and install ActiveState ActivePython in full to C:\Programs\Win32\Python\V2.5.2.2

Build

Boost C++ Libraries

Before we can go any further, we need to build Boost C++ libraries. There are several ways to build them, but we shall stick with BJam. This is because it is nototiously difficult to build Boost.Python extensions with any build tool other than BJam. In the official Tutorial Introduction we find the following passage:

"Besides bjam, there are of course other ways to get your module built. What's written here should not be taken as "the one and only way". There are of course other build tools apart from bjam.

"Take note however that the preferred build tool for Boost.Python is bjam. There are so many ways to set up the build incorrectly. Experience shows that 90% of the "I can't build Boost.Python" problems come from people who had to use a different tool."

Enough said, we are using BJam, which is a build engine with its own interpreted language. It was made for Boost and is part of Boost.Build.

Configuring BJam

BJam is configured using user-config.jam. This file is found in the top-level directory of Boost.Build installation. In our case, it is

E:\dev\external\Boost\boost_1_35_0\tools\build\v2

You can either edit this file there or create a local copy in your home directory and change it there. On Windows, you can find out what your home directory is by running

echo "%HOMEDRIVE%%HOMEPATH%"

We must tell BJam about Microsoft Visual C++ and about Python. Therefore we need to append the following to user-config.jam:

# Required for Boost.Python: ##############################################

# MSVC configuration
using msvc : 8.0 : "C:/Program Files/Microsoft Visual Studio 8/VC/bin/cl.exe" ;

# Python configuration
using python : 2.5 : "C:/Programs/Win32/Python/V2.5.2.2" ;

###########################################################################

This will tell BJam to use Microsoft Visual Studio 2005 (if you are using Microsoft Viusual Studio 2003, you should replace "8.0" with "7.2" above and inform it that Python 2.5 is installed under C:/Programs/Win32/Python/V2.5.2.2. Of course, you should substitute in the actual path to Python on your system.

If the toolsets are not considered as above BJam may not work and you may see some warnings:

warning: No toolsets are configured.
warning: Configuring default toolset "msvc".
warning: If the default is wrong, your build may not work correctly.
warning: Use the "--toolset=xxxxx" option to override our guess.
warning: For more configuration options, please consult
warning: http://boost.org/boost-build2/doc/html/bbv2/advanced/configuration.html

To see the command line options for BJam, you can run

E:\dev\external\boost\boost_1_35_0\bjam\bjam.exe --help

For future reference, here is the result of the above command:

Boost.Build V2 (Milestone 12)
Boost.Jam 03.1.16

Project-specific help:

  Project has jamfile at ../Jamroot

Usage:

  bjam [options] [install|stage]

  Builds and installs Boost.

Targets and Related Options:

  install                 Install headers and compiled library files to the
  =======                 configured locations (below).

  --prefix=<PREFIX>       Install architecture independent files here.
                          Default; C:\Boost on Win32
                          Default; /usr/local on Unix. Linux, etc.

  --exec-prefix=<EPREFIX> Install architecture dependent files here.
                          Default; <PREFIX>

  --libdir=<DIR>          Install library files here.
                          Default; <EPREFIX>/lib

  --includedir=<HDRDIR>   Install header files here.
                          Default; <PREFIX>/include

  stage                   Build and install only compiled library files
  =====                   to the stage directory.

  --stagedir=<STAGEDIR>   Install library files here
                          Default; ./stage

Other Options:

  --build-type=<type>     Build the specified pre-defined set of variations
                          of the libraries. Note, that which variants get
                          built depends on what each library supports.

                              minimal (default) - Builds the single
                              "release" version of the libraries. This
                              release corresponds to specifying:
                              "release <threading>multi <link>shared
                              <runtime-link>shared" as the Boost.Build
                              variant to build.

                              complete - Attempts to build all possible
                              variations.

  --build-dir=DIR         Build in this location instead of building
                          within the distribution tree. Recommended!

  --toolset=toolset       Indicates the toolset to build with.

  --show-libraries        Displays the list of Boost libraries that require
                          build and installation steps, then exit.

  --layout=<layout>       Determines whether to choose library names 
                          and header locations such that multiple 
                          versions of Boost or multiple compilers can
                          be used on the same system.

                              versioned (default) - Names of boost
                              binaries include the Boost version
                              number and the name and version of the
                              compiler.  Boost headers are installed
                              in a subdirectory of <HDRDIR> whose
                              name contains the Boost version
                              number.

                              system - Binaries names do not include
                              the Boost version number or the name
                              and version number of the compiler.
                              Boost headers are installed directly
                              into <HDRDIR>.  This option is
                              intended for system integrators who
                              are building distribution packages.

  --buildid=ID            Adds the specified ID to the name of built
                          libraries.  The default is to not add anything.

  --help                  This message.

  --with-<library>        Build and install the specified <library>
                          If this option is used, only libraries
                          specified using this option will be built.

  --without-<library>     Do not build, stage, or install the specified
                          <library>. By default, all libraries are built.

Configuration help:

  Configuration file at H:\user-config.jam

 This file is used to configure your Boost.Build installation. Please read
 the user manual to find out where to put it.

General command line usage:

    bjam [options] [properties] [targets]

  Options, properties and targets can be specified in any order.
      
Important Options:

  * --clean Remove targets instead of building
  * -a Rebuild everything
  * -n Don't execute the commands, only print them
  * -d+2 Show commands as they are executed
  * -d0 Supress all informational messages
  * -q Stop at first error
  * --debug-configuration Diagnose configuration
  * --debug-building Report which targets are built with what properties
  * --debug-generator Diagnose generator search/execution

Further Help:

  The following options can be used to obtain additional documentation.

  * --help-options Print more obscure command line options.
  * --help-internal Boost.Build implementation details.
  * --help-doc-options Implementation details doc formatting.
Note

We shall be running BJam quite often. To save us typing E:\dev\external\boost\boost_1_35_0\bjam\bjam.exe we can add E:\dev\external\boost\boost_1_35_0\bjam to the Path. On Windows XP, click on Start > Settings > Control Panel. "Switch to Classic View", if necessary, then select System > Advanced > Environment Variables. Under "User variables for..." select PATH, then click edit. Append

;"E:\dev\external\boost\boost_1_35_0\bjam"

to Variable value. Then click "OK" several times. Now we can type bjam.exe at the command line prompt.

Building Boost C++ Libraries

We are now ready to build Boost. Go to the Boost directory:

cd "E:\dev\external\boost\boost_1_35_0"

Then run BJam:

"E:\dev\external\boost\boost_1_35_0\bjam\bjam.exe" --build-type=complete --toolset=msvc-8.0 stage

You should see the following output:

warning: Graph library does not contain optional GraphML reader.
note: to enable GraphML support, set EXPAT_INCLUDE and EXPAT_LIBPATH to the
note: directories containing the Expat headers and libraries, respectively.
warning: skipping optional Message Passing Interface (MPI) library.
note: to enable MPI support, add "using mpi ;" to user-config.jam.
note: to suppress this message, pass "--without-mpi" to bjam.
note: otherwise, you can safely ignore this message.
Building Boost.Regex with the optional Unicode/ICU support disabled.
Please refer to the Boost.Regex documentation for more information
(don't panic: this is a strictly optional feature).
Skipping build of:  ./stage  <build>no in common properties
Skipping build of:  ./stage  <build>no in common properties
Skipping build of:  ./stage  <build>no in common properties
Skipping build of:  ./stage  <build>no in common properties
...patience...
...patience...
...found 5454 targets...
...updating 1783 targets...
MkDir1 stage
MkDir1 stage\lib
MkDir1 bin.v2
MkDir1 bin.v2\libs
MkDir1 bin.v2\libs\serialization
MkDir1 bin.v2\libs\serialization\build
MkDir1 bin.v2\libs\serialization\build\msvc-8.0
MkDir1 bin.v2\libs\serialization\build\msvc-8.0\debug
MkDir1 bin.v2\libs\serialization\build\msvc-8.0\debug\threading-multi
compile-c-c++ bin.v2\libs\serialization\build\msvc-8.0\debug\threading-multi\basic_archive.obj
basic_archive.cpp
compile-c-c++ bin.v2\libs\serialization\build\msvc-8.0\debug\threading-multi\basic_iarchive.obj
basic_iarchive.cpp
compile-c-c++ bin.v2\libs\serialization\build\msvc-8.0\debug\threading-multi\basic_iserializer.obj
basic_iserializer.cpp
.
.
.

Et cetera. The warnings and notes above can safely be ignored. And eventually you will see:

.
.
.
common.hard-link stage\lib\libboost_signals-vc80-s.lib
Could Not Find E:\dev\external\boost\boost_1_35_0\stage\lib\libboost_signals-vc80-s.lib
        1 file(s) copied.
common.hard-link stage\lib\libboost_prg_exec_monitor-vc80-s.lib
Could Not Find E:\dev\external\boost\boost_1_35_0\stage\lib\libboost_prg_exec_monitor-vc80-s.lib
        1 file(s) copied.
common.hard-link stage\lib\libboost_test_exec_monitor-vc80-s.lib
Could Not Find E:\dev\external\boost\boost_1_35_0\stage\lib\libboost_test_exec_monitor-vc80-s.lib
        1 file(s) copied.
common.hard-link stage\lib\libboost_unit_test_framework-vc80-s.lib
Could Not Find E:\dev\external\boost\boost_1_35_0\stage\lib\libboost_unit_test_framework-vc80-s.lib
        1 file(s) copied.
common.hard-link stage\lib\libboost_wave-vc80-s.lib
Could Not Find E:\dev\external\boost\boost_1_35_0\stage\lib\libboost_wave-vc80-s.lib
        1 file(s) copied.
...updated 1783 targets...

As a result, two new directories will be created under E:\dev\external\boost\boost_1_35_0:

  • E:\dev\external\boost\boost_1_35_0\bin.v2 (thic can be overridden with --build-dir=<DIR>),
  • E:\dev\external\boost\boost_1_35_0\stage (this can be overridden with --stagedir=<STAGEDIR>).

The former is the build directory. It contains the generated library files, object files, and other auxiliary files produced by the build process:

E:\dev\external\boost\boost_1_35_0\bin.v2\libs
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\debug
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\debug\link-static
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\debug\link-static\threading-multi
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\debug\link-static\threading-multi\date_generators.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\debug\link-static\threading-multi\date_generators.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\debug\link-static\threading-multi\greg_month.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\debug\link-static\threading-multi\greg_month.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\debug\link-static\threading-multi\greg_weekday.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\debug\link-static\threading-multi\greg_weekday.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\debug\link-static\threading-multi\libboost_date_time-vc80-mt-gd-1_35.lib
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\debug\link-static\threading-multi\libboost_date_time-vc80-mt-gd-1_35.lib.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\debug\threading-multi
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\debug\threading-multi\boost_date_time-vc80-mt-gd-1_35.dll
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\debug\threading-multi\boost_date_time-vc80-mt-gd-1_35.dll.manifest
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\debug\threading-multi\boost_date_time-vc80-mt-gd-1_35.dll.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\debug\threading-multi\boost_date_time-vc80-mt-gd-1_35.exp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\debug\threading-multi\boost_date_time-vc80-mt-gd-1_35.lib
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\debug\threading-multi\boost_date_time-vc80-mt-gd-1_35.pdb
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\debug\threading-multi\date_generators.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\debug\threading-multi\date_generators.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\debug\threading-multi\greg_month.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\debug\threading-multi\greg_month.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\debug\threading-multi\greg_weekday.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\debug\threading-multi\greg_weekday.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\release
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\release\link-static
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\release\link-static\threading-multi
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\release\link-static\threading-multi\date_generators.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\release\link-static\threading-multi\date_generators.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\release\link-static\threading-multi\greg_month.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\release\link-static\threading-multi\greg_month.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\release\link-static\threading-multi\greg_weekday.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\release\link-static\threading-multi\greg_weekday.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\release\link-static\threading-multi\libboost_date_time-vc80-mt-1_35.lib
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\release\link-static\threading-multi\libboost_date_time-vc80-mt-1_35.lib.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\release\threading-multi
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\release\threading-multi\boost_date_time-vc80-mt-1_35.dll
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\release\threading-multi\boost_date_time-vc80-mt-1_35.dll.manifest
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\release\threading-multi\boost_date_time-vc80-mt-1_35.dll.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\release\threading-multi\boost_date_time-vc80-mt-1_35.exp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\release\threading-multi\boost_date_time-vc80-mt-1_35.lib
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\release\threading-multi\date_generators.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\release\threading-multi\date_generators.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\release\threading-multi\greg_month.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\release\threading-multi\greg_month.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\release\threading-multi\greg_weekday.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\date_time\build\msvc-8.0\release\threading-multi\greg_weekday.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\link-static
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\link-static\threading-multi
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\link-static\threading-multi\libboost_filesystem-vc80-mt-gd-1_35.lib
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\link-static\threading-multi\libboost_filesystem-vc80-mt-gd-1_35.lib.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\link-static\threading-multi\operations.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\link-static\threading-multi\operations.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\link-static\threading-multi\path.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\link-static\threading-multi\path.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\link-static\threading-multi\portability.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\link-static\threading-multi\portability.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\link-static\threading-multi\utf8_codecvt_facet.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\link-static\threading-multi\utf8_codecvt_facet.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\threading-multi
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\threading-multi\boost_filesystem-vc80-mt-gd-1_35.dll
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\threading-multi\boost_filesystem-vc80-mt-gd-1_35.dll.manifest
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\threading-multi\boost_filesystem-vc80-mt-gd-1_35.dll.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\threading-multi\boost_filesystem-vc80-mt-gd-1_35.exp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\threading-multi\boost_filesystem-vc80-mt-gd-1_35.lib
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\threading-multi\boost_filesystem-vc80-mt-gd-1_35.pdb
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\threading-multi\operations.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\threading-multi\operations.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\threading-multi\path.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\threading-multi\path.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\threading-multi\portability.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\threading-multi\portability.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\threading-multi\utf8_codecvt_facet.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\debug\threading-multi\utf8_codecvt_facet.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release\link-static
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release\link-static\threading-multi
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release\link-static\threading-multi\libboost_filesystem-vc80-mt-1_35.lib
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release\link-static\threading-multi\libboost_filesystem-vc80-mt-1_35.lib.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release\link-static\threading-multi\operations.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release\link-static\threading-multi\operations.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release\link-static\threading-multi\path.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release\link-static\threading-multi\path.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release\link-static\threading-multi\portability.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release\link-static\threading-multi\portability.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release\link-static\threading-multi\utf8_codecvt_facet.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release\link-static\threading-multi\utf8_codecvt_facet.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release\threading-multi
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release\threading-multi\boost_filesystem-vc80-mt-1_35.dll
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release\threading-multi\boost_filesystem-vc80-mt-1_35.dll.manifest
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release\threading-multi\boost_filesystem-vc80-mt-1_35.dll.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release\threading-multi\boost_filesystem-vc80-mt-1_35.exp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release\threading-multi\boost_filesystem-vc80-mt-1_35.lib
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release\threading-multi\operations.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release\threading-multi\operations.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release\threading-multi\path.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release\threading-multi\path.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release\threading-multi\portability.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release\threading-multi\portability.obj.rsp
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release\threading-multi\utf8_codecvt_facet.obj
E:\dev\external\boost\boost_1_35_0\bin.v2\libs\filesystem\build\msvc-8.0\release\threading-multi\utf8_codecvt_facet.obj.rsp
.
.
.

The second directory, E:\dev\external\boost\boost_1_35_0\stage, contains the library files alone (no intermediate object files), in a flat structure (i.e. there are no subdirectories for each build variant):

E:\dev\external\boost\boost_1_35_0\stage\lib
E:\dev\external\boost\boost_1_35_0\stage\lib\boost_date_time-vc80-mt-1_35.dll
E:\dev\external\boost\boost_1_35_0\stage\lib\boost_date_time-vc80-mt-1_35.lib
E:\dev\external\boost\boost_1_35_0\stage\lib\boost_date_time-vc80-mt-gd-1_35.dll
E:\dev\external\boost\boost_1_35_0\stage\lib\boost_date_time-vc80-mt-gd-1_35.lib
E:\dev\external\boost\boost_1_35_0\stage\lib\boost_date_time-vc80-mt-gd.lib
E:\dev\external\boost\boost_1_35_0\stage\lib\boost_date_time-vc80-mt.lib
E:\dev\external\boost\boost_1_35_0\stage\lib\boost_filesystem-vc80-mt-1_35.dll
E:\dev\external\boost\boost_1_35_0\stage\lib\boost_filesystem-vc80-mt-1_35.lib
E:\dev\external\boost\boost_1_35_0\stage\lib\boost_filesystem-vc80-mt-gd-1_35.dll
E:\dev\external\boost\boost_1_35_0\stage\lib\boost_filesystem-vc80-mt-gd-1_35.lib
E:\dev\external\boost\boost_1_35_0\stage\lib\boost_filesystem-vc80-mt-gd.lib
E:\dev\external\boost\boost_1_35_0\stage\lib\boost_filesystem-vc80-mt.lib
E:\dev\external\boost\boost_1_35_0\stage\lib\boost_graph-vc80-mt-1_35.dll
E:\dev\external\boost\boost_1_35_0\stage\lib\boost_graph-vc80-mt-1_35.lib
E:\dev\external\boost\boost_1_35_0\stage\lib\boost_graph-vc80-mt-gd-1_35.dll
E:\dev\external\boost\boost_1_35_0\stage\lib\boost_graph-vc80-mt-gd-1_35.lib
E:\dev\external\boost\boost_1_35_0\stage\lib\boost_graph-vc80-mt-gd.lib
E:\dev\external\boost\boost_1_35_0\stage\lib\boost_graph-vc80-mt.lib
E:\dev\external\boost\boost_1_35_0\stage\lib\boost_iostreams-vc80-mt-1_35.dll
E:\dev\external\boost\boost_1_35_0\stage\lib\boost_iostreams-vc80-mt-1_35.lib
E:\dev\external\boost\boost_1_35_0\stage\lib\boost_iostreams-vc80-mt-gd-1_35.dll
E:\dev\external\boost\boost_1_35_0\stage\lib\boost_iostreams-vc80-mt-gd-1_35.lib
E:\dev\external\boost\boost_1_35_0\stage\lib\boost_iostreams-vc80-mt-gd.lib
E:\dev\external\boost\boost_1_35_0\stage\lib\boost_iostreams-vc80-mt.lib
.
.
.

These libraries have been staged. In other words, they have been copied to an intermediate location from a build directory, so they are ready to be deployed. (According to Wikipedia, a staging area is a location where organisms, people, vehicles, equipment or material are assembled prior to their use.)

Let's take a closer look at the options for the command that produced these directory trees. For the sake of convenience, we shall repeat it below:

"E:\dev\external\boost\boost_1_35_0\bjam\bjam.exe" --build-type=complete --toolset=msvc-8.0 stage

stage tells BJam to build and install only the compiled library files — not the header files — to the stage directory, which, since we are not specifying --stagedir=<STAGEDIR> is set to the default value, namely E:\dev\external\boost\boost_1_35_0\stage. The header files stay where they are, i.e. under E:\dev\external\boost\boost_1_35_0\boost. That's all we need. We can always add E:\dev\external\boost\boost_1_35_0 to the include directories list and access the individual header files as, for example, #include <boost/any.hpp>, which is, of course, E:\dev\external\boost\boost_1_35_0\boost\any.hpp.

--toolset=msvc-8.0 tells BJam to use the Visual Studio 2005 toolkit (C++ compiler, linker, etc.) that we have previously configured.

--build-type=complete tells BJam to attempt to build all possible variations of the libraries, which includes all combinations of:

  • variant: release, debug;
  • threading: single, multi;
  • link: static, shared;
  • runtime-link: static, shared.

There are four parameters, with two possible values for each, which would give us 16 combinations. However, in reality fewer combinations will be built (typically 8), because some of these combinations are incompatible or don't make sense. (This explanation is courtesy of Andrea Denzler.)

+-- E:\dev\external\boost\boost_1_35_0\build\boost\bin.v2\libs\date_time\build\msvc-8.0
    |
    +-- debug
    |   |
    |   +-- link-static
    |   |   |
    |   |   +-- runtime-link-static:  libboost_date_time-vc80-sgd-1_35.lib
    |   |   |   |                     (variant=debug, threading=single, link=static, runtime-link=static)
    |   |   |   |
    |   |   |   +-- threading-multi:  libboost_date_time-vc80-mt-sgd-1_35.lib
    |   |   |                         (variant=debug, threading=multi, link=static, runtime-link=static)
    |   |   |
    |   |   +-- threading-multi:      libboost_date_time-vc80-mt-gd-1_35.lib
    |   |                             (variant=debug, threading=multi, link=static, runtime-link=shared)
    |   |
    |   +-- threading-multi:          boost_date_time-vc80-mt-gd-1_35.dll
    |                                 (variant=debug, threading=multi, link=shared, runtime-link=shared)
    |
    +-- release
        |
        +-- link-static
        |   |
        |   +-- runtime-link-static:  libboost_date_time-vc80-s-1_35.lib
        |   |   |                     (variant=release, threading=single, link=static, runtime-link=static)
        |   |   |
        |   |   +-- threading-multi:  libboost_date_time-vc80-mt-s-1_35.lib
        |   |                         (variant=release, threading=multi, link=static, runtime-link=static)
        |   |
        |   +-- threading-multi:      libboost_date_time-vc80-mt-1_35.lib
        |                             (variant=release, threading=multi, link=static, runtime-link=shared)
        |
        +-- threading-multi:          boost_date_time-vc80-mt-1_35.dll
                                      (variant=release, threading=multi, link=shared, runtime-link=shared)

It is worth noting that the build variations are also reflected in the names of the resulting library files (*.libs and/or *.dlls), as explained in Boost: Getting Started. The library file names start with lib, except for DLLs (built with link=shared), for which this prefix is omitted. This is followed by the library name which is always of the form boost_name, e.g. boost_date_time. This in turn is followed by the toolset tag, in our case -vc80 for Visual Studio 2005. Then, the threading tag: if the library is built with threading=multi, we get -mt for "multithreading", otherwise, for threading=single libraries, this tag is omitted. Then, we have the ABI tag, -d. It encodes the details that affect the library's interoperability with other compiled code. For each such feature, a single letter is added to the tag:

  • s — use this library when linking statically to the C++ standard library and compiler runtime support libraries;
  • g — use this library when using debug versions of the standard and runtime support libraries;
  • y — use this library when using a special debug build of Python (more on this later);
  • d — use this library when building a debug version of your code;
  • p — use this library when using the STLPort standard library rather than the default one supplied with your compiler;
  • n — use this library when using STLPort's deprecated "native iostreams" feature.

Building Boost.Python Examples

Building the Boost.Python Tutorial in Its Original Location

This is the simplest example included with Boost.Python and, as such, the most instructive. At some point you may wish to consult the original documentation on this tutorial.

Let us build it. Change the directory to

E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial

and run

"E:\dev\external\boost\boost_1_35_0\bjam\bjam.exe" variant=debug,release threading=single,multi

In this case we are specifying explicitly that we want to build the debug and release variants, with single and multi threading.

The console output looks as follows:

warning: Graph library does not contain optional GraphML reader.
note: to enable GraphML support, set EXPAT_INCLUDE and EXPAT_LIBPATH to the
note: directories containing the Expat headers and libraries, respectively.
warning: skipping optional Message Passing Interface (MPI) library.
note: to enable MPI support, add "using mpi ;" to user-config.jam.
note: to suppress this message, pass "--without-mpi" to bjam.
note: otherwise, you can safely ignore this message.
Building Boost.Regex with the optional Unicode/ICU support disabled.
Please refer to the Boost.Regex documentation for more information
(don't panic: this is a strictly optional feature).
...patience...
...found 1145 targets...
...updating 22 targets...
MkDir1 bin
MkDir1 bin\msvc-8.0
MkDir1 bin\msvc-8.0\debug
MkDir1 bin\msvc-8.0\debug\threading-multi
compile-c-c++ bin\msvc-8.0\debug\threading-multi\hello.obj
hello.cpp
msvc.link.dll bin\msvc-8.0\debug\threading-multi\hello_ext.pyd
   Creating library bin\msvc-8.0\debug\threading-multi\hello_ext.lib and object
bin\msvc-8.0\debug\threading-multi\hello_ext.exp
MkDir1 bin\hello.test
MkDir1 bin\hello.test\msvc-8.0
MkDir1 bin\hello.test\msvc-8.0\debug
MkDir1 bin\hello.test\msvc-8.0\debug\threading-multi
capture-output bin\hello.test\msvc-8.0\debug\threading-multi\hello
        1 file(s) copied.
**passed** bin\hello.test\msvc-8.0\debug\threading-multi\hello.test
MkDir1 bin\msvc-8.0\release
MkDir1 bin\msvc-8.0\release\threading-multi
compile-c-c++ bin\msvc-8.0\release\threading-multi\hello.obj
hello.cpp
msvc.link.dll bin\msvc-8.0\release\threading-multi\hello_ext.pyd
   Creating library bin\msvc-8.0\release\threading-multi\hello_ext.lib and object bin\msvc-8.0\release\threading-multi\hello_ext.exp
MkDir1 bin\hello.test\msvc-8.0\release
MkDir1 bin\hello.test\msvc-8.0\release\threading-multi
capture-output bin\hello.test\msvc-8.0\release\threading-multi\hello
        1 file(s) copied.
**passed** bin\hello.test\msvc-8.0\release\threading-multi\hello.test
...updated 22 targets...

This results in the creation of the following directory tree:

E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\hello.test
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\hello.test\msvc-8.0
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\hello.test\msvc-8.0\debug
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\hello.test\msvc-8.0\debug\threading-multi
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\hello.test\msvc-8.0\debug\threading-multi\hello
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\hello.test\msvc-8.0\debug\threading-multi\hello.output
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\hello.test\msvc-8.0\debug\threading-multi\hello.test
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\hello.test\msvc-8.0\release
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\hello.test\msvc-8.0\release\threading-multi
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\hello.test\msvc-8.0\release\threading-multi\hello
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\hello.test\msvc-8.0\release\threading-multi\hello.output
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\hello.test\msvc-8.0\release\threading-multi\hello.test
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\msvc-8.0
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\msvc-8.0\debug
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\msvc-8.0\debug\threading-multi
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\msvc-8.0\debug\threading-multi\hello.obj
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\msvc-8.0\debug\threading-multi\hello.obj.rsp
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\msvc-8.0\debug\threading-multi\hello_ext.exp
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\msvc-8.0\debug\threading-multi\hello_ext.lib
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\msvc-8.0\debug\threading-multi\hello_ext.pdb
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\msvc-8.0\debug\threading-multi\hello_ext.pyd.manifest
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\msvc-8.0\debug\threading-multi\hello_ext.pyd.rsp
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\msvc-8.0\release
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\msvc-8.0\release\threading-multi
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\msvc-8.0\release\threading-multi\hello.obj
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\msvc-8.0\release\threading-multi\hello.obj.rsp
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\msvc-8.0\release\threading-multi\hello_ext.exp
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\msvc-8.0\release\threading-multi\hello_ext.lib
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\msvc-8.0\release\threading-multi\hello_ext.pyd.manifest
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\msvc-8.0\release\threading-multi\hello_ext.pyd.rsp

E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\Jamroot is the Jamroot file containing instructions that Bjam uses to build this project:

# Copyright David Abrahams 2006. Distributed under the Boost
# Software License, Version 1.0. (See accompanying
# file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)

# Specify the path to the Boost project.  If you move this project,
# adjust this path to refer to the Boost root directory.
use-project boost
  : ../../../.. ;

# Set up the project-wide requirements that everything uses the
# boost_python library from the project whose global ID is
# /boost/python.
project
  : requirements <library>/boost/python//boost_python ;

# Declare the three extension modules.  You can specify multiple
# source files after the colon separated by spaces.
python-extension hello_ext : hello.cpp ;

# A little "rule" (function) to clean up the syntax of declaring tests
# of these extension modules.
local rule run-test ( test-name : sources + )
{
    import testing ;
    testing.make-test run-pyd : $(sources) : : $(test-name) ;
}

# Declare test targets
run-test hello : hello_ext hello.py ;

Here we define a project that has the Boost.Python library as one of its requirements. The most important line is

python-extension hello_ext : hello.cpp ;

which tells BJam to build the Python extension hello_ext from hello.cpp. We also define a unit test using a custom local rule run-test (which calls the standard testing.make-test rule in module testing).

Thus there are really two build targets, the Python extension hello_ext and the unit test. Once we've run BJam, they appear under

E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\msvc-8.0

and

E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\hello.test

respectively.

We get the build variants debug and release for each, but only the threading-multi variation gets built. This is because the build variation threading=single turns out to be incompatible with our build configuration.

But did you notice a problem? The actual Python extension, hello_ext.pyd, which was produced from hello.cpp is nowhere to be found. There is a corresponding *.lib file, hello_ext.lib, but we need hello_ext.pyd (which is really a DLL with the *.dll extension changed to *.pyd) in order to run the extension from our Python code. It is all the more puzzling, given that hello_ext.pyd appears in BJam output above:

msvc.link.dll bin\msvc-8.0\debug\threading-multi\hello_ext.pyd

The answer (courtesy of Dave Abrahams) is somewhat counterintuitive. By default Boost.Build deletes some binaries used in testing when the test passes. Thus when we got to the test target, defined by

# Declare test targets
run-test hello : hello_ext hello.py ;

in Jamroot, hello_ext.pyd was removed.

We have two options: remove the test target and rebuild, or re-run BJam with the option --preserve-test-targets:

"E:\dev\external\boost\boost_1_35_0\bjam\bjam.exe" variant=debug,release --preserve-test-targets

Now we can see our Python extension:

E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\msvc-8.0\debug\threading-multi\hello_ext.pyd
E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\msvc-8.0\release\threading-multi\hello_ext.pyd

Let's try running hello_ext.pyd from a Python program. Take a look at E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\hello.py:

#  Copyright Joel de Guzman 2002-2007. Distributed under the Boost
#  Software License, Version 1.0. (See accompanying file LICENSE_1_0.txt
#  or copy at http://www.boost.org/LICENSE_1_0.txt)
#  Hello World Example from the tutorial

import hello_ext
print hello_ext.greet()

Notice that hello_ext is imported as a "normal" Python module. However, hello_ext.pyd must be on our PYTHONPATH for hello.py to find it. Or hello_ext.pyd and hello.py must be in the same directory. So let's copy E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\hello.py to E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\msvc-8.0\release\threading-multi where we have the release variant of hello_ext.pyd. Try running

python hello.py

More likely than not, we get a message box with title "python.exe - Unable To Locate Component" and message "This application has failed to start because boost_python_vc80-mt-1_35.dll was not found. Re-installing the application may fix this problem."

We discover that the appropriate variant of Boost.Python library must be on our PATH. We locate boost_python-vc80-mt-1_35.dll under E:\dev\external\boost\boost_1_35_0\stage\lib. Then we can either add E:\dev\external\boost\boost_1_35_0\stage\lib to the PATH environment variable or simply copy it to E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin\msvc-8.0\release\threading-multi. We do the latter.

Now we are happy:

E:\...msvc-8.0\release\threading-multi>python hello.py
hello, world

From hello.py we are calling the function greet() defined in E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\hello.cpp:

char const* greet()
{
   return "hello, world";
}

And wrapped into a Python module using the following Boost.Python constructions, also in hello.cpp:

BOOST_PYTHON_MODULE(hello_ext)
{
    using namespace boost::python;
    def("greet", greet);
}
Building the Boost.Python Tutorial in Another Location

Since we don't normally develop under the E:\dev\external\boost\boost_1_35_0 directory, our next step is to build the Boost.Python Tutorial elsewhere. We shall see that some changes are required.

If you have already built the Boost.Python Tutorial in situ, delete the directory E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial\bin along with all its contents.

We create the following directory:

mkdir E:\dev\BoostPythonTutorial

Then copy the contents of E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial to that directory, so that we end up with

E:\dev\BoostPythonTutorial\hello.cpp
E:\dev\BoostPythonTutorial\hello.py
E:\dev\BoostPythonTutorial\Jamroot

If we cd into E:\dev\BoostPythonTutorial and try calling E:\dev\external\boost\boost_1_35_0\bjam\bjam.exe, we'll see the following:

E:\dev\BoostPythonTutorial>E:\dev\external\boost\boost_1_35_0\bjam\bjam.exe
Unable to load Boost.Build: could not find "boost-build.jam"
---------------------------------------------------------------
Attempted search from E:\dev\BoostPythonTutorial up to the root

Please consult the documentation at 'http://www.boost.org'.

Somehow we've lost boost-build.jam along the way. In fact there is a boost-build.jam under E:\dev\external\boost\boost_1_35_0\libs\python\example and we were OK when we called BJam from E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial as BJam searches for this file up to the root, i.e. it checks

E:\dev\external\boost\boost_1_35_0\libs\python\example\tutorial
E:\dev\external\boost\boost_1_35_0\libs\python\example
E:\dev\external\boost\boost_1_35_0\libs\python
E:\dev\external\boost\boost_1_35_0\libs
E:\dev\external\boost\boost_1_35_0
E:\dev\external\boost
E:\dev\external
E:\dev
E:\

in that order. (Actually it would stop at E:\dev\external\boost\boost_1_35_0\libs\python\example, since the file is there.)

So we copy E:\dev\external\boost\boost_1_35_0\libs\python\example\boost-build.jam to E:\dev\BoostPythonTutorial. Yet we still have problems:

E:\dev\BoostPythonTutorial>E:\dev\external\boost\boost_1_35_0\bjam\bjam.exe
Unable to load Boost.Build: could not find build system.
---------------------------------------------------------
E:\dev\BoostPythonTutorial\boost-build.jam attempted to load the build system by
 invoking

   'boost-build ../../../tools/build/v2 ;'

but we were unable to find "bootstrap.jam" in the specified directory
or in BOOST_BUILD_PATH (searching E:\dev\BoostPythonTutorial\../../../tools/build/v2).

Please consult the documentation at 'http://www.boost.org'.

Have a look at E:\dev\BoostPythonTutorial\boost-build.jam:

# Copyright David Abrahams 2006. Distributed under the Boost
# Software License, Version 1.0. (See accompanying
# file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)

# Edit this path to point at the tools/build/v2 subdirectory of your
# Boost installation.  Absolute paths work, too.
boost-build ../../../tools/build/v2 ;

The relative path to E:\dev\external\boost\boost_1_35_0\tools\build\v2 is no longer correct, so we replace it with the absolute path:

# Copyright David Abrahams 2006. Distributed under the Boost
# Software License, Version 1.0. (See accompanying
# file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)

# Edit this path to point at the tools/build/v2 subdirectory of your
# Boost installation.  Absolute paths work, too.
boost-build "E:\dev\external\boost\boost_1_35_0\tools\build\v2" ;

This time we get

E:\dev\BoostPythonTutorial>E:\dev\external\boost\boost_1_35_0\bjam\bjam.exe
Unable to load Boost.Build: could not find build system.
---------------------------------------------------------
E:\dev\BoostPythonTutorial\boost-build.jam attempted to load the build system by
 invoking

   'boost-build E:devexternalboostboost_1_35_0toolsbuildv2 ;'

but we were unable to find "bootstrap.jam" in the specified directory
or in BOOST_BUILD_PATH (searching E:\dev\BoostPythonTutorial\E:devexternalboostboost_1_35_0toolsbuildv2).

Please consult the documentation at 'http://www.boost.org'.

We learn our lesson: BJam expects slashes, not backslashes:

# Copyright David Abrahams 2006. Distributed under the Boost
# Software License, Version 1.0. (See accompanying
# file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)

# Edit this path to point at the tools/build/v2 subdirectory of your
# Boost installation.  Absolute paths work, too.
boost-build "E:/dev/external/boost/boost_1_35_0/tools/build/v2" ;

What's the result this time?

E:/dev/external/boost/boost_1_35_0/tools/build/v2/build\project.jam:266: in find-jamfile from module project
error: Unable to load Jamfile.
error: Could not find a Jamfile in directory '../../../..'.
error: Attempted to find it with pattern '[Bb]uild.jam [Jj]amfile.v2 [Jj]amfile [Jj]amfile.jam'.
error: Please consult the documentation at 'http://www.boost.org'.
E:/dev/external/boost/boost_1_35_0/tools/build/v2/build\project.jam:287: in load-jamfile from module project
E:/dev/external/boost/boost_1_35_0/tools/build/v2/build\project.jam:63: in project.load from module project
E:/dev/external/boost/boost_1_35_0/tools/build/v2/build\project.jam:720: in project.use from module project
E:/dev/external/boost/boost_1_35_0/tools/build/v2/build\project.jam:90: in load-used-projects from module project
E:/dev/external/boost/boost_1_35_0/tools/build/v2/build\project.jam:74: in load from module project
E:/dev/external/boost/boost_1_35_0/tools/build/v2/build\project.jam:167: in project.find from module project
E:/dev/external/boost/boost_1_35_0/tools/build/v2\build-system.jam:516: in load from module build-system
E:\dev\external\boost\boost_1_35_0\tools\build\v2/kernel\modules.jam:267: in import from module modules
E:\dev\external\boost\boost_1_35_0\tools\build\v2/kernel/bootstrap.jam:132: in boost-build from module
E:\dev\BoostPythonTutorial\boost-build.jam:7: in module scope from module

This simply tells us that there is another relative path that we need to replace with an absolute path, this time in E:\dev\BoostPythonTutorial\Jamroot:

# Specify the path to the Boost project.  If you move this project,
# adjust this path to refer to the Boost root directory.
use-project boost
  : ../../../.. ;

We replace it with

# Specify the path to the Boost project.  If you move this project,
# adjust this path to refer to the Boost root directory.
use-project boost
  : "E:/dev/external/boost/boost_1_35_0" ;

BJam finally succeeds:

E:\dev\BoostPythonTutorial>t\boost_1_35_0\bjam\bjam.exe --preserve-test-targets
warning: Graph library does not contain optional GraphML reader.
note: to enable GraphML support, set EXPAT_INCLUDE and EXPAT_LIBPATH to the
note: directories containing the Expat headers and libraries, respectively.
warning: skipping optional Message Passing Interface (MPI) library.
note: to enable MPI support, add "using mpi ;" to user-config.jam.
note: to suppress this message, pass "--without-mpi" to bjam.
note: otherwise, you can safely ignore this message.
Building Boost.Regex with the optional Unicode/ICU support disabled.
Please refer to the Boost.Regex documentation for more information
(don't panic: this is a strictly optional feature).
...patience...
...found 1104 targets...
...updating 13 targets...
MkDir1 bin
MkDir1 bin\msvc-8.0
MkDir1 bin\msvc-8.0\debug
MkDir1 bin\msvc-8.0\debug\threading-multi
compile-c-c++ bin\msvc-8.0\debug\threading-multi\hello.obj
hello.cpp
msvc.link.dll bin\msvc-8.0\debug\threading-multi\hello_ext.pyd
   Creating library bin\msvc-8.0\debug\threading-multi\hello_ext.lib and object bin\msvc-8.0\debug\threading-multi\hello_ext.exp
MkDir1 bin\hello.test
MkDir1 bin\hello.test\msvc-8.0
MkDir1 bin\hello.test\msvc-8.0\debug
MkDir1 bin\hello.test\msvc-8.0\debug\threading-multi
capture-output bin\hello.test\msvc-8.0\debug\threading-multi\hello
        1 file(s) copied.
**passed** bin\hello.test\msvc-8.0\debug\threading-multi\hello.test
...updated 13 targets...

And the results of the built appear under E:\dev\BoostPythonTutorial\bin.

Setting Up a BJam Project

In this tutorial, we shall have a simple directory structure:

+-- BoostPythonStepByStep
    |
    +-- Example00_HelloWorld
    |   |
    |   +-- bin
    |       |
    |       +-- msvc-8.0
    |           |
    |           +-- debug
    |           |   |
    |           |   +-- threading-multi
    |           |
    |           +-- release
    |               |
    |               +-- threading-multi
    |
    +-- Example01_DataAndFunctions
    |   |
    |   +-- bin
    |       |
    |       +-- msvc-8.0
    |           |
    |           +-- debug
    |           |   |
    |           |   +-- threading-multi
    |           |
    |           +-- release
    |               |
    |               +-- threading-multi
    |
    +-- Example02_Arguments
    |   |
    |   +-- bin
    |       |
    |       +-- msvc-8.0
    |           |
    |           +-- debug
    |           |   |
    |           |   +-- threading-multi
    |           |
    |           +-- release
    |               |
    |               +-- threading-multi
    |
    +-- Example03_ReturnValues
    |   |
    |   +-- bin
    |       |
    |       +-- msvc-8.0
    |           |
    |           +-- debug
    |           |   |
    |           |   +-- threading-multi
    |           |
    |           +-- release
    |               |
    |               +-- threading-multi
    |
    +-- ...

It doesn't matter where you put the directory BoostPythonStepByStep. This will be the root of our development directory tree.

The bin directories and everything underneath will be created by the BJam build process. However, although we have set up BJam, there are still a couple of things we need to do to get it to work for our project. We need to create the files boost-build.jam and Jamroot under BoostPythonStepByStep and Jamfile under each Example??_* directory:

+-- BoostPythonStepByStep
    |
    +-- boost-build.jam
    |
    +-- Jamroot
    |
    +-- Example00_HelloWorld
    |   |
    |   +-- Jamfile
    |   |
    |   +-- bin
    |       |
    |       +-- msvc-8.0
    |           |
    |           +-- debug
    |           |   |
    |           |   +-- threading-multi
    |           |
    |           +-- release
    |               |
    |               +-- threading-multi
    |
    +-- Example01_DataAndFunctions
    |   |
    |   +-- Jamfile
    |   |
    |   +-- bin
    |       |
    |       +-- msvc-8.0
    |           |
    |           +-- debug
    |           |   |
    |           |   +-- threading-multi
    |           |
    |           +-- release
    |               |
    |               +-- threading-multi
    |
    +-- Example02_Arguments
    |   |
    |   +-- Jamfile
    |   |
    |   +-- bin
    |       |
    |       +-- msvc-8.0
    |           |
    |           +-- debug
    |           |   |
    |           |   +-- threading-multi
    |           |
    |           +-- release
    |               |
    |               +-- threading-multi
    |
    +-- Example03_ReturnValues
    |   |
    |   +-- Jamfile
    |   |
    |   +-- bin
    |       |
    |       +-- msvc-8.0
    |           |
    |           +-- debug
    |           |   |
    |           |   +-- threading-multi
    |           |
    |           +-- release
    |               |
    |               +-- threading-multi
    |
    +-- ...

boost-build.jam contains a single line:

boost-build E:/dev/external/Boost/boost_1_35_0/tools/build/v2 ;

Strictly speaking, boost-build.jam needs to be in the root directory of the project (as is the case) or in any of its parent directories. This file is required to initialise Boost.Build. The boost-build line tells BJam where Boost.Build files are located.

Why can it be placed in any of the project's parent directories? Because this file is often shared by multiple projects. Strictly speaking we could "factor it out" of our projects and put it somewhere like E:/dev.

Boost.Python Step by Step

Hello, World!

Boost.Python DLLs

Go to the directory

...\BoostPythonStepByStep\HelloWorld\bin\msvc-8.0\release\threading-multi

and run

python hello.py

You may see the following message:

This application has failed to start because
boost_python-vc80-mt-1_35.dll was not found. Re-installing the
application may fix this problem.

This is because the Boost Python DLLs cannot be found. In order for them to be found, you may need to add the directory that contains them (search under your Boost intallation, in my case it is E:\dev\external\Boost\boost_1_35_0\bin.v2\libs\python\build\msvc-8.0\debug\threading-multi for boost_python-vc80-mt-gd-1_35.dll and E:\dev\external\Boost\boost_1_35_0\bin.v2\libs\python\build\msvc-8.0\release\threading-multi for boost_python-vc80-mt-1_35.dll) to your PATH.

Alternatively, you may copy the DLLs to the directory containing hello.pyd, i.e. to

...\BoostPythonStepByStep\HelloWorld\bin\msvc-8.0\release\threading-multi

Both alternatives have their shortcomings. Since our *.pyd files will appear in multiple directories, copying boost_python-*.dll to each of them is rather painful. Therefore we shall the Boost DLL directories to the PATH. The readers will have to decide how to distribute the DLLs with their applications.

Notes

Capturing the Output of BJam

The output of BJam can often be verbose. You may wish to use the following command to capture the messages in a file called bjam-out.txt:

bjam.exe debug release --debug-configuration > bjam-out.txt 2>&1

Please note the --debug-configuration option. This option causes BJam to produce additional diagnostic output when it is parsing its configuration. This output is often valuable.

This will capture both the STDOUT and, most importantly, STDERR.

Using the 64-bit ActivePython

I attempted to use the 64-bit ActivePython ver. 2.5.2.2 with Boost.Python. Unfortunately I was unable to compile Boost.Python with it. On running BJam I got numerous linker error messages:

...patience...
...patience...
...found 4154 targets...
...updating 4 targets...
msvc.link.dll bin.v2\libs\python\build\msvc-8.0\debug\threading-multi\boost_python-vc80-mt-gd-1_35.dll
   Creating library bin.v2\libs\python\build\msvc-8.0\debug\threading-multi\boost_python-vc80-mt-gd-1_35.lib and object bin.v2\libs\python\build\msvc-8.0\debug\threading-multi\boost_python-vc80-mt-gd-1_35.exp
function.obj : error LNK2001: unresolved external symbol __imp__PyErr_Format
numeric.obj : error LNK2019: unresolved external symbol __imp__PyErr_Format referenced in function "void __cdecl boost::python::numeric::`anonymous namespace'::throw_load_failure(void)" (?throw_load_failure@?A0x10c448e9@numeric@python@boost@@YAXXZ)
from_python.obj : error LNK2001: unresolved external symbol __imp__PyErr_Format
registry.obj : error LNK2001: unresolved external symbol __imp__PyErr_Format
class.obj : error LNK2001: unresolved external symbol __imp__PyErr_Format
numeric.obj : error LNK2001: unresolved external symbol __imp__PyExc_ImportError
numeric.obj : error LNK2019: unresolved external symbol __imp__PyErr_Clear referenced in function "bool __cdecl boost::python::numeric::`anonymous namespace'::load(bool)" (?load@?A0x10c448e9@numeric@python@boost@@YA_N_N@Z)
class.obj : error LNK2001: unresolved external symbol __imp__PyErr_Clear
function.obj : error LNK2001: unresolved external symbol __imp__PyErr_Clear
object_protocol.obj : error LNK2001: unresolved external symbol __imp__PyErr_Clear
numeric.obj : error LNK2019: unresolved external symbol __imp__PyCallable_Check referenced in function "bool __cdecl boost::python::numeric::`anonymous namespace'::load(bool)" (?load@?A0x10c448e9@numeric@python@boost@@YA_N_N@Z)
class.obj : error LNK2001: unresolved external symbol __imp__PyCallable_Check
object_protocol.obj : error LNK2001: unresolved external symbol __imp__PyType_IsSubtype
numeric.obj : error LNK2019: unresolved external symbol __imp__PyType_IsSubtype referenced in function "bool __cdecl boost::python::numeric::`anonymous namespace'::load(bool)" (?load@?A0x10c448e9@numeric@python@boost@@YA_N_N@Z)
class.obj : error LNK2001: unresolved external symbol __imp__PyType_IsSubtype
function.obj : error LNK2001: unresolved external symbol __imp__PyType_IsSubtype
builtin_converters.obj : error LNK2001: unresolved external symbol __imp__PyType_IsSubtype
life_support.obj : error LNK2001: unresolved external symbol __imp__PyType_Type
numeric.obj : error LNK2001: unresolved external symbol __imp__PyType_Type
enum.obj : error LNK2001: unresolved external symbol __imp__PyType_Type
class.obj : error LNK2001: unresolved external symbol __imp__PyType_Type
function.obj : error LNK2001: unresolved external symbol __imp__PyType_Type

and more along the same lines. So far I haven't resolved this problem. Perhaps the readers know what is going on. The 32-bit ActivePython ver. 2.5.2.2 works pretty well.

Further Reading and Resources

There are a number of useful resources on Boost.Python, to name but a few:

  • This page was last modified on 21 August 2008, at 13:24.
  • This page has been accessed 123,227 times.