Re: [boost] Trying to learn build system

From: David Abrahams (david.abrahams_at_[hidden])
Date: 2002-07-07 10:59:48

----- Original Message -----
From: "Gennadiy Rozental" <rogeeff_at_[hidden]>

> Hi,
>
> I am trying to start using Boost.Build system. After several days of
reading
> and playing I must admit that I still completely lost - I do not
understand
> how this thing is working. I was not able to run the tests I wanted. I am
> not able to write "makefile" (Jamfile?) myself and I am almost completely
> clueless how to write toolset description for my Sun box ( BTW how do we
> produce Compiler results on Sun without build toolset for sunpro?). I may
> need some help.
>
>   Ok. Starting with "Building Boost Libraries" page and following the
> instructions there I was able to build some offline libs on my NT box.
Now I
> wanted to create Compiler status page. I did not find any reference how
to
> do this. In fact I did not find any reference how to compile and run any
> tests at all. Should I look in different place?

I'm really confused about your confusion, because I was under the
impression that you and Beman were working together on this. In particular,
Beman has been working on some kind of .bat file which post-processes the
result of running "bjam test" in boost's status subdirectory.  This is the
only way I currently know of to create .html status pages from the bjam
regression-testing output. Up till now, regression testing has not been
done with Boost.Build, but with Jens' regr.bat program as described in
http://groups.yahoo.com/group/jamboost/message/1353. That also explains why
you can't find a SunPro toolset file -- we don't have one AFAIK.

>    Next I was trying to reveal how it is done. Very quickly I realized
that
> there are a lot of things behind the scene and these "things" include
some
> agreements, global variables and scripts written in some language that I
do
> not understand (In fact I was not able to find language description
> anywhere, only pieces here and there). So I address my eyes to Boost
Build
> System documentation.

That information is contained in the combination of
http://www.perforce.com/jam/jam.html and
http://www.boost.org/tools/build/build_system.htm.

In particular, as the Build System docs attest at the end of
http://www.boost.org/tools/build/build_system.htm#jam_fundamentals, you
will probably not understand anything unless you read
http://public.perforce.com/public/jam/src/Jam.html.

> Synopsis, status and TOC I understand. Getting started already a bit
> confusing since it has a section "Setting Jam Variables", while both
terms
> "Jam" and "Jam variables are not introduced yet. Next we see "An Example
> Jamfile" which is again an example of things that not defined yet.

It helps if you read up on the system from Perforce on which this is based.

> For a new
> user like me it is a bit confusing. Though again maybe I should have read
> something else before. Support Files section seems incomplete (there some
> other files that looks like support).

You're right. We're not anxious to try to make this part of the
documentation much better as we're moving very close to having a first
release of Boost.Build v2, which will obsolete all of this.

> Looking inside of these files making things even worse. What are these
> files: part of Jam, Part of Build.Build, what do I need to know about
them?

They're part of Boost.Build.

> BTW unit_test.jam listed here But never ever mention in any other place

Don't worry about it, in that case.

> More over "meanings" are unclear for
> unfamiliar user (unfamiliar with terms feature, rules toolset and so on).

Features should be explained here:
http://www.boost.org/tools/build/build_system.htm#features
"Rules" are explained at
http://www.boost.org/tools/build/build_system.htm#jam_fundamentals
A "toolset" is a set of compatible related tools for building programs
(e.g. C++ compiler, linker).

Please let me know if there's anything more I can do to clear that up.

> "Basic Design and Terminology" bring some hope. But statement about
Jamrules
> file is unclear: what is "Jam code", what the difference between Jamfile
and
> Jamrules file?

See http://public.perforce.com/public/jam/src/Jamfile.html.

However, be aware that significant core changes have been made to Boost
Jam, and that mostly the information on the Jambase is not relevant.

> Target section defined term "main target". In what sense it is main?

That's just a name I picked; I couldn't think of a better one.

> And
> what means definition of "build targets" as an  expressions of the
> corresponding main target under particular compilers and build variants?

The term is "built target". The point is that if my Jamfile says:

    exe Foo : a.cpp b.cpp ;

There is an abstract idea of the target called Foo, which might corresponds
to a number of files which actually get constructed for, e.g. debug and
release build variants and for various toolsets. The actually-constructed
files are the "built targets".

> "Features and Properties" section left me wondering how features are
related
> to my  Jamfiles, how to use and define them. As far as I understand
> features.jam list some predefined features. Is there complete description
> for all of them?

No.

> But what is most important I did not get how defining the
> feature affect the build system. I presume it something in ??-tools
files,

Yes.

> but syntax unclear to me.

Doesn't the description of the "flags" rule at
http://www.boost.org/tools/build/build_system.htm#toolset_description say
it all?

> Pararagraphs about variant and subvariant I did not understand
completely.
> What is a "variant": term, feature, rule?

http://www.boost.org/tools/build/build_system.htm#variants:
"A build variant, or simply variant is a named set of build properties
describing how targets should be built"

How can I describe it more clearly?

> How it is related/mentioned in
> Jamfiles?

In a Jamfile you might optionally include it in the default-BUILD section
of a main target specification.

> What are the variant statements in features.jam?

Those are the declarations of known build variants.

> How is it related to global variable BUILD?

$(BUILD) may include the names of one or more variants to be built.

> "subvariant" paragraph left me clueless also.

Ask a specific question, or I can't help.

> Usage:
>
> Global variables section list only 3 variables. Is there complete
> description of all global variables.

No. Another thing we don't want to invest in too much because v2 will not
rely much on global variables. However, much of the information you would
ever need is in the table here:
http://www.boost.org/tools/build/#Configuring

> sub-project jamfiles section recommend to use project-root  rule if
Jamfile
> exist in a project rule directory. How is it know the path to the root
then?

That's a mistake in the documentation. It should say to use project-root
instead of subproject in any Jamfile located at the root of the project
tree. See the Jamfile in the root of the boost tree for an example.

> And why subproject path is used only of ALL_LOCATE_TARGET is set?

Because it's not needed otherwise: the build directory tree is a bin/
subdirectory of the Jam invocation directory.

> "Describing Main Targets":
> What means: "Additional main target rules are possible"? Did you mean it
> could be more than one main target?

No, it means that you can define new rules other than exe, dll, etc., which
describe main targets.

> What are the features I can use in default-BUILD section (other then
> <suppress> mentioned for example) ?

All of the features listed in features.jam.

> And  how do they affect build variants?

Explicitly-specified features override any features specified by build
variants.

> "Describing Stage Targets"
> Could you clarify what is <tag>? Feature? And How is it used with
examples.
> Could it be several tags?

I leave this one to Rene Rivera; "stage" targets are his feature.

> "Toolset Description Files"
> What is $(<), $(<)?

See http://public.perforce.com/public/jam/src/Jam.html#language

> What are UNDEFS C++FLAGS and so on - global variables?

Yes.

> If yes, where could I
> find complete list?

Nowhere, sorry :(

> Could you clarify again why do I need another level of indirection when
> implementing C++-action rule?

I'm not actually sure it's neccessary, but in any case it's an artifact of
old limitations of Jam which don't apply any more.

> What are the argument specification for the rules I need to redefine.

Most of them are:

     ( targets * : sources * )

except for

    Link-action ( targets * : sources * : target-type )

> Who is defining the variable gBUILD_PROPERTIES and when it should be
used?

The build system does it; you should almost never need to touch it or worry
about it.

> Why do I need  to mention toolset name for flags rule? I am already in
> toolset file.

Poor design.

> What if condition is just a <feature> without value. When flag will be
set?

Whenever the feature is present in the build properties.

> In general before moving to the Internals, I would like to remark that
there
> are a lot of thing (rules, features, GLOBAL_VARIABLES) that are not
> mentioned in doc. I remarked that there are about 200 rules defined in
both
> ..base.jam files. Where could I find a description of them?

In the in-code comments.

> Internals.
> As I sad before language description seems incomplete (descriptive). I
could
> not find strict syntax specification. Does you assume familiarity with
some
> other document?

http://public.perforce.com/public/jam/src/Jam.html

> "Jam fundamentals"
> What are "actions"? Which built-in rules exist?

See the section called "Built-in Rules" below
http://public.perforce.com/public/jam/src/Jam.html#actionmods

and the list of extensions at
http://www.boost.org/tools/build/build_system.htm#core_extensions

> Which built-in variables
> exist and what their meanings?

?? There's no mention of "built-in variables" in any of my docs.

> What are the local variables?

?? Look in the source of the rules for local variables.

> Without this
> knowledge what you writing is unclear and I could not move further.
>
> Hope you will give some pointers,

I hope this helped, at least a little.

If it's any consolation, Rene has been working on an interactive help
system for Boost.Build v2 which will automatically extract documentation
from the comments, for those who need help with internals.

-Dave