Boost Build System

Table of Contents

Introduction

Assumptions

The requirements are driven by several basic assumptions:

Requirements

This build system was designed to satisfy the following requirements:

Installing Jam

Jam can be found at http://www.perforce.com/jam/jam.html. If you are installing on Windows NT, please read this note. You may need to use the ``SUBST hack'' described at the bottom of the message to bootstrap Jam. Also, when defining the MSVCNT, BCCROOT, etc. environment variables described here, please remember that they are case-sensitive.

Initiating a Build

Currently, the build system must be invoked with the -fallyourbase-path option, where allyourbase-path is the path to the allyourbase.jam file supplied in this directory. When the system matures, this file will be compiled directly into Jam as the Jambase, and the -f option will no longer be needed. The environment variables neccessary for bootstrapping Jam are not needed once it has been built.

Here are some sample Boost Jam invocations:
Command Line(s) Effects
jam -fallyourbase-path -sTOOLS=gcc my_target default (debug) BUILD of my_targetwith GCC
jam -fallyourbase-path -sTOOLS="msvc gcc" default-build all with msvc and gcc
set TOOLS=msvc
jam -fallyourbase-path 		Set an NT environment variable to always build with MSVC
default-build all.
jam -fallyourbase-path -sBUILD=release release build all with default TOOLS:
jam -fallyourbase-path -sBUILD="debug release" debug and release build all.

Support Files

To use the build system, the following must be located in your project's root directory, or in directories given by the path variables indicated below:
Filename(s) Optional Path Variable Meaning
toolset-tools.jam BOOST_BUILD_INSTALLATION Feature-to-command-line mapping for toolset.
features.jam BOOST_BUILD_INSTALLATION Abstract toolset feature descriptions.
boost-base.jam BOOST_BASE_DIR Boost build system-specific rule definitions.
The boost-base.jam file is temporary, and will eventually be compiled into our Jam executable.

Basic Design and Terminology

This section gives an overview of the way that the system works, outlining the system's capabilities and overall design. It also introduces the terminology and concepts neccessary to understand the sections on writing Jamfiles and command-line invocations.

Projects and Subprojects

A project is a source directory tree containing a Jamfile. The root directory of the project is known as the project root. The root directory of a project may contain a Jamrules file, which contains project-specific Jam code. If the Jamrules file is not present when Jam is invoked, a warning will be issued.

Subdirectories containing Jamfiles are called subproject directories. Each such Jamfile describes a subproject.

The build system installation directory is a directory containing Jam files describing compilers and build variants. The installation directory can be specified explicitly by setting the variable BOOST_BUILD_INSTALLATION. If the installation directory is not specified, it is the same as the project root, and BOOST_BUILD_INSTALLATION is set to refer to that directory.

An Example Jamfile

Here is an example of a simple subproject Jamfile. In this example, it is assumed that the user has set BOOST_PARENT_DIRECTORY, either as an environment variable, on the command-line or in the project's Jamrules file: 
subproject foo/bar/baz ; # path to here from project root

# A static library called 'baz'
lib baz : baz1.cpp baz2.cpp # C++ sources
          parser/src/baz4.ll # Lex->C++ sources
          parser/src/baz5.yy  # Yacc->C++ sources
        : <include>$(BOOST_PARENT_DIRECTORY)  # Put boost in #include path
    ;

# An executable called 'test'
exe test : <lib>baz # use the 'baz' library
           baz_test.cpp   # C++ source
         : <include>$(BOOST_PARENT_DIRECTORY)
    ;

That's it! The build system takes care of the rest. If the you want to be able to build all subprojects from the project root directory, you can add a Jamfile at the root: 

# Read subproject Jamfiles
subinclude foo/bar/baz foo/bar/...;
subinclude a/b/c ... ; # more subincludes

Targets

Each Jamfile describes one or more main targets.

Each main target is an abstract description of one or more built targets which are expressions of the corresponding main target under particular compilers and build variants. Intermediate files such as .o/.obj files generated by compiling .cpp files as a consequence of building a main target are also referred to as built targets. The term build directory tree refers to the location of built target files.

For each main target, there is a corresponding location in the build directory tree known as the target's build root, where all intermediate and final targets resulting from that main target are located.

Features and Properties

A feature is a normalized description of an individual build parameter, such as whether inlining is enabled. Each feature usually corresponds to a command-line option of one or more build tools. Features come in two varieties:

  1. Simple features can take on any of several predetermined values. For example, the feature optimization might take one of the values off, speed, or space. Simple features have a default value. Most features fall into this category.
  2. Free features can take on any number of user-specified values simultaneously. For example, the define feature for a release build might have the values NDEBUG and BOOST_RELEASE_BUILD.

A feature-value pair is known as a build property, or simply property. The prefixes Simple and free apply to properties in an analogous way to features.

Build Variants

A build variant, or simply variant is a named set of build properties describing how targets should be built for each compiler. Built targets for distinct build variants and compilers are generated in separate parts of the build directory tree, known as the variant directories. For example, a (sub)project with main targets foo and bar, compiled with both GCC and KAI for debug and release variants might generate the following structure (target directories in bold). 

 bin
 +-foo  <--- foo's build root
 | +-gcc
 | | +-debug
 | | `-release
 | `-kai
 |   +-debug
 |   `-release
 `-bar  <--- bar's build root
   +-gcc
   | +-debug
   | `-release
   `-kai
     +-debug
     `-release

Subvariants

When a target is built with simple properties that don't exactly match those specified in a build variant, the non-matching features are called subvariant features and the target is located in a subvariant directory beneath the directory of the base variant. This can occur for two reasons:

  1. Some features are only relevant to certain compilers. When relevant simple features have no value specified in the build variant, a value must be chosen. Even when the default value is used, the target is generated into a subvariant directory. For example, the runtime-link feature may be unspecified in the debug variant, but relevant to MSVC. In that case, a fragment of the target tree might look like: 
     bin
 +-foo  <--- foo's build root
 | +-msvc
 | | +-debug
 . . . `-runtime-link-dynamic
 . . .
    Because the default value of runtime-link is dynamic, when the debug variant is requested, the runtime-link-dynamic subvariant of foo is built.

  2. It is possible to request (either on the command-line, or as part of a main target description) that particular subvariants be built. For example, it may be desirable to generate builds that link to the runtime both statically and dynamically. In that case, both subvariant directories in the example above would be generated: 
     bin
 +-foo  <--- foo's build root
 | +-msvc
 | | +-debug
 . . . +-runtime-link-dynamic
 . . . `-runtime-link-static
 . . .
In no case will targets be built directly into bin/foo/msvc/debug, since the debug variant doesn't include the runtime-link feature, which is relevant to MSVC.

When a subvariant includes multiple subvariant features, targets are built into a subvariant directory whose path is determined by concatenating the properties sorted in order of their feature names. For example, the borland compiler, which uses different libraries depending on whether the target is a console or GUI program, might create the following structure for a DLL: 

 bin
 +-foo  <--- foo's build root
 | +-msvc
 | | +-debug
 | | | +-runtime-link-dynamic
 | | | | +-user-interface-console
 | | | | `-user-interface-gui
 . . . `-runtime-link-static
 . . .   +-user-interface-console
 . . .   `-user-interface-gui

Any configuration of properties for which a target is built, whether base variant or subvariant, is known as a build configuration, or simply a build.

Dependent Targets

When a main target depends on the product of a second main target (as when an executable depends on and links to a static library), each build configuration of the dependent target is depends on the same build of the dependee. Because only simple features participate in build identity, the dependent and dependee targets may have completely different free features. This puts the onus on the user for ensuring link-compatibility when certain free properties are used. For example, when assert() is used in header files, the preprocessor symbol NDEBUG can impact link-compatibility of separate compilation units. This danger can be minimized by encapsulating such feature differences inside of build variants.

Usage

This section describes how to start a build from the command-line and how to write project and subproject Jamfiles. It also describes the other files written in the Jam language: build-tool specification files, feature descriptions files.

Some terminology

In the Jam language, a name can be divided into two parts. If the name starts with a '<' symbol and contains a '>' symbol, the characters between the '<' and the first '>', inclusive, is known as grist (if the name doesn't match this format, the grist is empty). The rest of the name is known as the ungristed part. In Jam, Grist is added to user-specified target names from different subprojects to distinguish them from one another in case the same target name is used twice. The Boost build system also takes full advantage of Jam's ability to divide strings on grist boundaries, sometimes concatenating multiple gristed elements at the beginning of a string.

SubProject Jamfiles

This section describes how to write a Jamfile for a subproject.

The subproject rule

A subproject's Jamfile begins with an invocation of the subproject rule that specifies the subproject's location relative to the top of the project tree: 

subproject path-from-top ;

The subproject rule tells the build system where to place built targets from the subproject in case ALL_LOCATE_TARGET is used to specify the build directory tree.

Describing Main Targets

A main target is described using the following syntax: 

target-type name : sources
    [ : requirements [ : default-BUILD ] ] ;

NOTE: for simple features in both requirements and default-BUILD, more-specific qualification overrides less-specific.

Example

This artificially complex example shows how an executable called "foo" might be described in a Jamfile. The executable is composed of the sources ./foo.cpp and ./src/bar.cpp (specified relative to the directory in which the Jamfile resides), and the built target which results from building the target baz as described in ../bazlib/Jamfile. 

exe foo : foo.cpp src/bar.cpp <lib>../bazlib/baz
    ## Requirements ##
    : <include>../bazlib/include 
      <define>BUILDING_FOO=1
      <release><define>FOO_RELEASE
      <msvc><*><define>FOO_MSVC
      <msvc><release><define>FOO_MSVC_RELEASE
      <gcc><*><optimization>off
      <gcc><release><optimization>space
      <threading>multi
    
    ## default-BUILD ##
    : debug release
      <debug><runtime-link>static/dynamic
    ;

The requirements section:

The default-BUILD section:

Toolset Description Files

Toolset descriptions are located in the project's root directory, or a directory specified by BOOST_BUILD_INSTALLATION, which may be set in a Jamfile or the project's Jamrules file. Each file is called toolset-name-tools.jam, where toolset-name is the name of the toolset. The toolset description file has two main jobs:

  1. redefine the following rules: These rules should simply invoke the action part of a rule whose name is uniquely defined for the toolset. For example, 
    rule C++-action
{
    msvc-C++-action $(<) : $(>) ;
}

actions msvc-C++-action
{
    cl -nologo -GX -c -U$(UNDEFS) -D$(DEFINES) $(CFLAGS) $(C++FLAGS) -I$(HDRS) -I$(STDHDRS) -Fo$(<) -Tp$(>)
}
    Note that Link-action may require special care: on platforms where the global variable gEXPORT_SUFFIX(DLL) is defined (e.g. Windows), the first argument may have two elements when linking a shared library. The first is the shared library target, and the second is the import library target, with suffix given by $(gEXPORT_SUFFIX(DLL)). It will always have a third argument which is either ``EXE'' or ``DLL''. This can be used to dispatch to different actions for linking DLLs and EXEs if neccessary, but usually it will be easier to take advantage of the special <target-type> feature, which will have the same value using the flags rule described below.
  2. Translate build settings given in the global gBUILD_PROPERTIES variable into something that can be used by the toolset. The build system provides the flags rule to help translate build properties into elements of global variables which are later attached to targets so that they can affect the build actions. The flags rule is used as follows: 
    flags toolset variable condition [: value...]
    The parameters are:

    Semantics only affect targets built with the specified toolset, and depend on the target's build configuration:

    1. if any specified property-set is a subset of the target's build properties, the values specified in $(3) will be appended once to variable.
    2. The value of each specified feature that participates in the target's build properaties is appended to variable. In either case, the variable will be set "on" the target so it may be used in the build actions.

    Example

    The description of the flags rule above is actually more complicated than it sounds. For example, the following line might be used to specify how optimization can be turned off for MSVC: 

    flags msvc CFLAGS <optimization>off : /Od ;
    It says that the string /Od should be added to the global CFLAGS variable whenever a build configuration includes the property <optimization>off.

    Similarly, in the following example, 

    flags msvc CFLAGS <runtime-build>release/<runtime-link>dynamic/<threading>multi : /MD ;
    we add /MD to the CFLAGS variable when all of the specified conditions are satisfied. We could grab all of the values of the free feature <include> in the HDRS variable as follows: 
    flags msvc HDRS <include> ;

    The use of these variables should be apparent from the declaration of actions msvc-C++-action in the previous section.

    Feature Descriptions

    Internals

    Jam Fundamentals

    This section is derived from the official Jam documentation and from my experience using it and reading the Jambase rules. I repeat the information here mostly because it is essential to understanding and using Jam, but is not consolidated in a single place. Some of it is missing from the official documentation altogether. I hope it will be useful to anyone wishing to become familiar with Jam and the Boost build system.

    Please also read The Jam language reference for the additional details, and the Jam release notes for a brief description of recent, but fundamental changes to the Jam language without which you will probably not understand any of the build system code. In particular, note that the return statement does not affect control flow.

    Global Variables

    This section describes some of the global variables used by the build system. Please note that some parts of the system (particularly those in allyourbase.jam) are heavily based on the Jambase file supplied with Jam, and as such do not follow the conventions described below.

    Global variables used in the build system fall into three categories:

    Please note that the build system commonly takes advantage of Jam's Dynamic Scoping feature (see the local command in the "Flow of Control" section below the link target) to temporarily "change" a global variable by declaring a local of the same name.

    Variables Associated with SubProject Identity

    Grist Variables

    © Copyright David Abrahams 2001. Permission to copy, use, modify, sell and distribute this document is granted provided this copyright notice appears in all copies. This document is provided "as is" without express or implied warranty, and with no claim as to its suitability for any purpose.

    Revised 3 June, 2001