Re: bjam rationale...

From: Larry Evans (cppljevans_at_[hidden])
Date: 2004-12-16 12:50:18

Whew! This must have taken a lot of time and thought. Thanks.

On 12/16/2004 06:30 AM, David Abrahams wrote:
> Tompa wrote:
[snip]

>>Fine, thank you! But, hmmm... What is it in that list of
>>requirements that bjam does so much better than gnumake?

[snip]

> Q1.0: A developer adding a new library or test program must only
> have to add simple entries naming the source files to a text file,
> and not have to know anything about platform specific files. ...

Any platform specific files can be included with

include $(PLATFORM).imk

where PLATFORM is set by testing which platform (I assume this
means operating system in order to decide the suffixes for
executables and object files and directory name separators,
etc.) I'm assuming the test for platform can be done by invoking
a script function via the $(shell *command*), where *command*
is the actual script command.

> A1.0: With GNU Make a developer needs to write the specific flags
> and options used by all the compilers and operating systems (s)he is
> targeting into her build actions. With Boost.Build, the developer
> writes largely platform- independent build specifications.

include $(PLATFORM).$(COMPILER).imk

could include the compiler commands for compiler, $(COMPILER),
and operating system, $(PLATFORM).

>
> Q1.5: ...The developer should not have to supply header dependency
> information.
>
> A1.5: Without help from external tools, GNU Make is not capable of
> scanning for header dependencies, so the user has to use external tools
> like perl scripts to handle that job.

True, but maybe that's not a big disadvantage. For example, gcc can
produce this information in a .d file with the -MD compiler option,
and I believe most compilers have similar flags to produce the same
information. So one might say bjam reinvents the wheel with this
feature. OTOH, I understand that the dependency information provided
by bjam is always up-to-date since it does the analysis each time.
OTOOH, this extra scanning requires, I assume, a lot of time.

>
> Q2. There should be a very high likelihood of builds succeeding on
> all platforms if a build succeeds on any platform. In other words, a
> developer must not be required to have access to many platforms or
> compilers to ensure correct builds

>
> A2. See A1.0. Yes, it is possible to write "cross-platform" and
> "cross-compiler" Makefiles. I have seen such systems, though, and
> they: 1. have always been limited to a small number of platforms and
> compilers and 2. have been ugly an unmaintainable hacks because of
> the expressive limitations of the make language.

The following part of a Makefile:

<---------
HOW=gcc
#HOW=icc
COMPILER.gcc=/usr/local/gcc-3.4.1/bin/g++
COMPILER.icc=icc
COMPILE.gcc=$(COMPILER.gcc) -c -Wall -ftemplate-depth-100 -O0 -fno-inline
COMPILE.icc=$(COMPILER.icc) -c -xc++ -O0 -Ob0
LINK.gcc = $(COMPILER.gcc)
LINK.icc = $(COMPILER.icc)
...
$(SRC).$(HOW).o: $(SRC).cpp
$(COMPILE.$(HOW)) $(INCS) $(SRC).cpp -o $@

>---------

Allows easy selection of the compiler. A similar scheme could be used
to make the command specific to the platform. The syntax is obviously
not as clear as bjam's Jamfile's, but I think it still is
maintainable. After all, adding a new compiler just means adding a
new instance of the COMPILE.* macros. The user making the change
would, of course, have to know the relevant commands for the specific
compiler, but this is the same for bjam. And with bjam, when I wanted
to add como to the set of compilers, I had to search through several
source files before I figured I needed to modify ~/user-config.jam and
needed to add tools/como.jam or something like that.

{ BTW, I think the $(COMPILE.$(HOW)) usage is gnumake specific.}

In addition, the $(SRC).$(HOW).o in the above Makefile part could be
extended to emulate the way bjam selects the output directory as
follows:

$(BUILD)/$(WHAT)/$(HOW)/$(SRC).o

where WHAT could be the location w.r.t. some root source directory and
BUILD would be the build-dir, in bjam.v2 terms.

[snip]

> A3. In other words, it should be possible to plug in support for a
> new operating system or compiler without modifying the project
> descriptions (makefiles/Jamfiles, whatever). With GNU make that
> isn't supported natively and is difficult to implement.

I think it could be done with the:

include $(PLATFORM).$(COMPILER).imk

scheme mentioned earlier. And BTW again, I think this `include`
command is gnumake specific too.

> Q4. The build should rely only on tools native to the platform and
> compiler, or supplied via the boost download.

> A4. This appears to be less of a concern for Boost developers than it
> was, but for various reasons people were at the time unwilling or unable
> to install "external" tools like GNU Make, Perl, Python, etc. Somehow
> building bjam from relatively simple codebase was acceptable.

> Q5. The details of how the build is done for a particular platform or
> compiler should be appropriate for that platform.

> A5. GNU make provides no native support for controlling these details
> without intruding on Makefiles/Jamfiles, and the previously mentioned
> limitations of the make programming language make implementing it
> difficult and ugly.

> Q6. It should be possible to build multiple variants (e.g.
> debug/release) of a single target.

This could be done by including a file located in a directory whose
path is composed of the variants. For example:

HOW=$(COMPILER)/$(VARIANT)
HOW.path=./how_variants/$(HOW)
include $(HOW.path)/how.imk

where:

VARIANT = "debug" or "release" or whatever the user chooses as long
as he provides the corresponding how.imk file to be included.

> Q7. It should be possible to build multiple variants of multiple targets
> with multiple compilers from a single build command.

> A7. Same answer, multiplied by 10.

I don't think make can do this from a single command, unless the
command could be parsed to detect multiple variants/targets and then
make invoked with each combination. This would have to be done by
some script file, AFAICT.

[snip]

> Q9. Support for dynamic and static linking should be included.

Again, this could be done by encoding the type of link in the VARIANT
macro mentioned after Q6 above. However, instead of just release or
debug, there would be macros,

LINK-TYPE # either "shared" or "static"
DEV-PHASE # either "debug" or "release"

and VARIANT would be:

VARIANT=$(LINK-TYPE)/$(DEV-PHASE)

[snip]

> Q13. It should be possible to build targets into a directory
> unrelated to the source directories (they may be read-only)

> A14. I *think* GNU make can handle this easily enough if properly
> engineered to do so. But while we're in the neighborhood, Google
> for "recursive make considered harmful"

My above response containing $(BUILD) suggests how this could be done
without recursive makes.