Wrapping up new design

From: David Abrahams (david.abrahams_at_[hidden])
Date: 2002-01-01 15:49:58

Sorry if things have been a bit quiet lately. I'm just trying to keep
all of these balls in the air at once...

Anyway, I've been thinking about what's missing that would allow us to
make rapid progress. It seems to me that we need a high-level view of
how the build process will go. The rest is "just" detail.

I've managed to write things down at the highest level here, but I am stuck
on a few open questions which I'll bring up in a separate message to follow.

--------

First, let's outline the constraints that come from the build
tool. The main tasks of any build system using Jam are:

1. Establish targets
2. Generate build actions for the targets
2a. Attach variables to the targets which will be substituted into build
actions
3. Establish dependency relationships between the targets
4. Establish target locations.

I'll refer to steps 1, 2, and 3 as "building the dependency graph".
There are also some details to handle header scanning, but the list
above covers most of it.

OK, now my vision for how the build system works:

1. Jambase is read. BOOST_BUILD_PATH is set, so none of the default
Jambase code takes effect. Instead, we search for the file called
boost-build.jam on BOOST_BUILD_PATH and read that.

** the current Jambase gives an error about FTJam toolset
definitions etc., if BOOST_BUILD_PATH is not set and the toolset
definition is not set either. Probably that message should be
extended to say something about setting BOOST_BUILD_PATH so that
people are not confused. The FTJam toolset definitions aren't
needed unless you're building Jam itself.

2. boost-build.jam first loads two modules, site-config.jam and
user-config.jam. We can put empty versions in the Boost.Build
directory so that there is no error if users don't put their own
somewhere earlier in BOOST_BUILD_PATH (we'll re-use that trick a
lot). The Actually, these should probably contain /LOTS/ of
commented-out code which does basic configuration jobs. What do
these things look like? For one thing, I'd like to avoid having the
user set a bunch of obscure global variables. Let's do this through
rule invocations (just meant to be an example, not neccessarily
committed to this syntax):

# ---- Sample site-config.jam file ----
# loads the msvc module, which registers it as an available
# toolset No special toolset location/configuration information
# is given, so it is assumed that the toolset is already set up
# (e.g. VCVARS32.BAT has been called), or that it's installed in
# its standard location.
using msvc ;

# As above, but tells the system that we have two versions of the
# gcc toolset installed, in the specified locations, with 2.95.3
# being the default. The 'using' rule loads the given module and
# calls its "configure" method with the rest of its
# arguments. How a module treats configuration information, of
# course, is up to the module.
using gcc : 2.95.3 /usr/local/gcc-2.95.3
3.0.2 /usr/local/gcc-3.0.2
;

# Same idea as above.
using stlport : 4.5 ~/stlport-4.5
4.6b2 ~/stlport-4.6b2
;

# does what ALL_LOCATE_TARGET currently does
locate-built-targets bigdrive:/dave/builds ;

3. The system reads the Jamfile in the invocation directory. Jamfiles
contain mostly /declarative/ rule invocations. Declarative rules
build data structures describing targets (and other things,
e.g. toolsets in toolset description files). Of course a Jamfile
may also import modules and invoke rules that do more heavy-duty
work. There are several important reason to use declarative rules
in Jamfiles. First, the current system which constructs a
dependency/action graph as each descriptive rule is read is quite
inefficient for large projects. Secondly, we are unable to make any
delayed decisions based on the entire contents of the jamfile. For
example, we might want to do something completely different with
the target descriptions, e.g. generate makefiles. Finally, it is
too easy for the rules invoked and variables set by the user in a
Jamfile to interfere with the build process.

4. The system traverses the set of top-level targets and generates
the dependency graph based on the expanded build description (the
algorithm for expanding build descriptions is given at the bottom
of

http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl?Boost.Build_D
esign_Proposal).
OK, so I've tossed off most of the work of the build system in one
sentence. The rest of this document deals with that in more detail.

The basic algorithm is as follows:

Determine the subproject requirements
For each main target in the Jamfile:
Generate the targets which correspond to source files
Generate the subvariant targets of the main target
For each subvariant of the main target
build the dependency graph between the source targets and
the subvariant target

Determining the subproject requirements

This part is simple. The user tells us about the subproject
using the following rules:

project.project ( project-id : requirements * : default-build * )

Declares a project or subproject. A subproject's id is a path,
starting with the project id of which it is a subproject. The
requirements and default build apply to any targets described in
the Jamfile which do not explicitly declare others. A project
rule invocation is mandatory in any Jamfile in a project which
includes subprojects or uses other projects.

project.jamfile-location( root-to-jamfile )

Declares the location of this Jamfile with respect to the
project root, in case the path given in the project rule does
not describe the location of the jamfile.

project.source-location( root-to-source )

Declares that relative paths in this Jamfile are all specified
relative to the specified directory. Thus, a project with this
structure:

root
+- build
| `- Jamfile
`- src
+- foo.c
`- bar.c

might have the following Jamfile:

project.project foobar ;
project.jamfile-location build
project.source-location src ;

exe foobar : foo.c bar.c ;

Generating Targets

Target names given by users in a Jamfile aren't, in general, the
names of targets that will actually be used by the build system,
since elements such as subvariant grist come into play. In order to
keep the system usable, however, ungristed names are "claimed" by
the system for NOTFILE targets which correspond to the user's
notion of what should be built. When multiple subvariant builds
have been requested these NOTFILE targets will depend on several
built target files.

...