Questions re SCons conversion

From: Andreas Fredriksson (deplinenoise_at_[hidden])
Date: 2005-10-05 01:40:47

Hi list,

sorry for this rather lengthy mail, but I thought it better to give
too much information rather than too little :-)

I'm pondering the thought of converting a fairly large project to use
bjam v2 instead of SCons, perhaps mostly because of performance. A
no-act SCons invokation on a freshly-built project with everything in
disk cache on Linux takes about 10 seconds on a modern PC while the
corresponding times for Windows are much worse. I assume this has to
do with disk caching differences.

So I'm looking for ballpark figures from this list on what kind of
performance I could expect from boost.build and whether there are some
features of our current SCons system that would prove difficult to
implement. There are about 3 million lines of code in total, split
into 20-something modules including 3rd party libraries.

To simpify the job of adding new subsystems, I've actually made a
meta-system on top of SCons which reads up a simplified descriptions
(source lists and a type selections in the simplest case) and converts
them to a SCons target tree internally. This allows a programmer
without build system skills to add his new subproject in a matter of
seconds:

dir: mysubsys/
file: mysubsys.build:
name = 'mysubsys'
auto_sources = 1 # default recursive glob w/ platform-specific dirs
type = 'shared_library'

This will create SCons targets for a shared library with our
world-global settings that will automatically glob sources and add
platform-specific patterns so that any platform-specific code is
compiled in from directories named p_<major_platform>
p_<minor_platform> (e.g. p_unix, p_macosx or p_linux).

An important goal is to keep the boilerplate to a minumum and allow
the user to write code instead of build system files.

So here are my questions:

1) Is there a good way to add global defines based on the target O/S?
Our sourcebase expects a number of preprocessor declarations such as
FOO_UNIX, FOO_UNIX_LINUX, etc. My current attempt is to set these in
the top container project (main Jamfile) via a series of if-else
blocks.

2) I thought Boost.Build's project concept would match pretty closely
to the system I've described above and I came up with the following
Jamfile template to be placed in each subsystem directory (from the
top of my head):

project /subsys/foo
: requirements [ helpers.mk_dll_export FOO_API ]
: usage-requirements [ helpers.mk_dll_import FOO_API ]
;

# glob *.cpp and p_<platform>/*.cpp to get source list;
# also depend on another subsystem
lib foo : /subsys/bar [ helpers.glob_sources ] ;

The helpers module has a number of rules that can build _declspec(..)
and __attribute__((visibility("default"))) defines automatically given
a define name.

Does this look acceptable or am I missing something?

I've noticed that Boost.Build (and Jam for that matter) doesn't apply
grist based on paths in the source list in a Jamfile; so if there are
several subdirectories beneath a subsystem's directory they will all
be flattened in the object directory; i.e. a/a.cpp b/b.cpp -> a.o b.o.
Won't this break when there are several files with the same name?

3) A feature of SCons we like is that we can easily install all shared
libraries and binaries to a BIN directory from where we debug and
otherwise work with the project. I couldn't get Boost.Build to install
a base library and one or more libraries that depend on that base
library, because it complains that multiple targets do the same thing.

I assume BB is trying to install the base library from looking at the
requirements of the second library or something, but I'm unsure of how
to fix this. Any suggestions?

4) Given a design that would work similar to the SCons design, can I
do any sort of rough guess about the build system overhead and
performance? I'm specifically interested in small-change times, when a
small number of source files have changed.

Best regards,
Andreas

--
"Those who live by the sword get shot by those who don't."