Boost-Build :

dave_at_[hidden]

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com
 --==-=-= 
Date: Mon, 16 Jun 2003 15:12:53 +0400
From: Vladimir Prus <ghost_at_[hidden]>
To: David Abrahams <dave_at_[hidden]>,
Steven Knight <knight_at_[hidden]>
Cc: Ali Azarbayejani <ali_at_[hidden]>
Subject: Re: [scons-devel] SCons BOF at O'Reilly OSCon 2003
Message-Id: <200306161512.53685.ghost_at_[hidden]>
References: <Pine.LNX.4.44.0306131141300.4806-100000_at_[hidden]>
<uk7bpj1dx.fsf_at_[hidden]>
MIME-Version: 1.0
David Abrahams wrote:
> >> We actually only discussed Scons integration in private, but there's
> >> been a lot of discussion of architecture. There's some disagreement
> >> about whether the Boost.Build v2 notion of targets is compatible with
> >> Scons'.
> >
> > Some example(s) of the way(s) in which people are concerned they're not
> > compatible would be helpful.
>
> I think I'll have to leave Volodya to comment on this.
I don't think we ever made a specific list of things different from Scons, or 
desirable from Scons, so I'll try to compose it right now.
1. Features. Those are quite important and nifty things. I believe Scons is 
using Environment for the same purpose?
2. Main target. In Boost.Build, "main target" --- i.e. target specified by 
user --- can generate a big number of variants. Actually, different sets of 
real files. If I understand correctly, target defined by user can be built in 
only one way by Scons.
3. Main targets in Boost.Build can have "usage-requirements" --- properties 
which are applied to dependents. That's covered in docs at
http://boost.sf.net/boost-build2
4. We have a notion of "project", which can be referred by "id". This gives 
great convenience. I can write
exe util : util.cpp @/boost/filesystem/filesystem ;
which will use Boost filesystem library. And I don't care when the library is 
really located --- it's specified by some high-level configuration code.
There are two more thing, that I'm not sure about.
1. How SCons finds the transformation from sources to desired type. I've tried 
looking at Builder.py, but did not understood much. 
2. How SCons handles toolset differences? Are variables like INCLUDES 
translated by each tool into relevant command line switches? Is there some 
common framework for such transformations?
So far, given all the differences above, we though it's better to implement 
Python version of Boost.Build as a layer on top of SCons. We'll write the 
classes to represent main target and projects, and the 'generate' method of 
such classes will take set of properties and return SCons Node instances --- 
which will be built by Scons after that. The scheme is straigforward, the 
interface to Scons is very lean, but I'm not sure if deeper integration would 
yeild more benefits.
For example, at the moment, V2 implements precisely the same scheme, except 
that we use 'virtual-target' class instead of scons.Node. There is discussion 
if both classes ('main-target' and 'virtual-target') should have common base. 
Recast in Scons terms, it would mean that Node should have 
'generate-subvariant(property-set)' method. On regular nodes it will return 
self. On nodes corresponding to Boost.Build main target, it will construct 
the needed subvariant and return regular nodes for that subvariant.
That's one thing were changes to Scons would benefit Boost.Build. I'm sure 
there are more...
BTW, Steven, supposing that Boost.Build is rewritten in Python one day, what 
form of intergration with SCons would you prefer? I suspect that complete 
merger will be very hard both from UI view and technically (and probably 
politically)
- Volodya
 --==-=-= 
Date: Mon, 16 Jun 2003 09:32:09 -0500 (CDT)
From: Steven Knight <knight_at_[hidden]>
To: Vladimir Prus <ghost_at_[hidden]>
cc: David Abrahams <dave_at_[hidden]>,
Ali Azarbayejani <ali_at_[hidden]>
Subject: Re: [scons-devel] SCons BOF at O'Reilly OSCon 2003
Message-ID: <Pine.LNX.4.44.0306160822530.20778-100000_at_[hidden]>
MIME-Version: 1.0
Hi Volodya--
> > >> We actually only discussed Scons integration in private, but there's
> > >> been a lot of discussion of architecture. There's some disagreement
> > >> about whether the Boost.Build v2 notion of targets is compatible with
> > >> Scons'.
> > >
> > > Some example(s) of the way(s) in which people are concerned they're not
> > > compatible would be helpful.
> >
> > I think I'll have to leave Volodya to comment on this.
> 
> I don't think we ever made a specific list of things different from Scons, or 
> desirable from Scons, so I'll try to compose it right now.
> 
> 1. Features. Those are quite important and nifty things. I believe Scons is 
> using Environment for the same purpose?
Yes (based on my quick scan through the documentation). An Environment
is where you set up how you want one or more products to be built: use
*this* compiler, *this* version of yacc, these flags, these include
paths, these libraries, etc...
> 2. Main target. In Boost.Build, "main target" --- i.e. target specified by 
> user --- can generate a big number of variants. Actually, different sets of 
> real files. If I understand correctly, target defined by user can be built in 
> only one way by Scons.
I think this would be covered by Alias nodes in SCons. If you have
multiple variants that you want built under control of one target name
that the user can specify on the command line, you could set it up like
so:
env.Alias('libs', ['build/sunos/lib',
'build/linux/lib',
'build/win32/lib'])
An Alias just expands to whatever other targets are specified--real
files, directories, other Aliases, whatever.
> 3. Main targets in Boost.Build can have "usage-requirements" --- properties 
> which are applied to dependents. That's covered in docs at
> 
> http://boost.sf.net/boost-build2
So the basic idea is, if I link a program with a library, it may have
some attributes (-D options or the like) that will automatically
propagate to the way my program's .o files are built? Or just to the
way the library is linked with the resulting executable?
If I understand this correctly, these would be best handled by building
the libraries and dependent programs with the same Environment. This
makes it explicit, not automatically propagated (which may be a feature
or a bug, depending on your point of view).
> 4. We have a notion of "project", which can be referred by "id". This gives 
> great convenience. I can write
> 
> exe util : util.cpp @/boost/filesystem/filesystem ;
> 
> which will use Boost filesystem library. And I don't care when the library is 
> really located --- it's specified by some high-level configuration code.
The syntax above is pretty opaque; I don't understand what it's really
doing under the hood. What does this turn into on an executed command
line?
If it's a matter of searching for a library in an Autoconf-like way,
we've just recently added an SConf subsystem that can do that.
> There are two more thing, that I'm not sure about.
> 
> 1. How SCons finds the transformation from sources to desired type. I've tried 
> looking at Builder.py, but did not understood much. 
The target type isn't purely intuited from the source type; it's
explicitly chosen by the builder that's called:
env.Program('foo', [psources]) # build 'foo' or 'foo.exe'
env.Library('foo', [lsources]) # build 'libfoo.a' or 'foo.lib'
Does that answer the question, or did I misunderstand?
> 2. How SCons handles toolset differences? Are variables like INCLUDES 
> translated by each tool into relevant command line switches? Is there some 
> common framework for such transformations?
Right now, INCLUDES is translated into -I or /I options by common code.
We're at the limits of what that code can do because of the differences
in how different tool chains handling #include "..." and the like. The
architecture is there for toolchain-specific translations of INCLUDES
into options, but no one has yet had a compelling enough reason to code
up a specific override for a tool.
> So far, given all the differences above, we though it's better to implement 
> Python version of Boost.Build as a layer on top of SCons. We'll write the 
> classes to represent main target and projects, and the 'generate' method of 
> such classes will take set of properties and return SCons Node instances --- 
> which will be built by Scons after that. The scheme is straigforward, the 
> interface to Scons is very lean, but I'm not sure if deeper integration would 
> yeild more benefits.
I agree. In fact, we've tried to maintain a clean separation in the
SCons architecture between the underlying build engine (the Nodes,
Scanners, Builders and the like) and the wrapping interface. In our
case, the interface we supply happens to be Python scripts, but we
*want* people to be able to write other interfaces around the build
engine, and we're very committed to the idea of enhancing the build
engine, if necessary, to support other interfaces such as yours.
For example, Stephen Kennedy has put a continuous-integration interface
around it, and Asko Kauppi even has a Lua interface (although I think
he translates into SConscript files instead of using the build engine
directly).
> For example, at the moment, V2 implements precisely the same scheme, except 
> that we use 'virtual-target' class instead of scons.Node. There is discussion 
> if both classes ('main-target' and 'virtual-target') should have common base. 
> Recast in Scons terms, it would mean that Node should have 
> 'generate-subvariant(property-set)' method. On regular nodes it will return 
> self. On nodes corresponding to Boost.Build main target, it will construct 
> the needed subvariant and return regular nodes for that subvariant.
Odds are, if something is useful for your interface, other interfaces
might find it useful, too, so I'll be very inclined to work with you
on extending the Node class (and other parts of the build engine) to
accomodate your requirements.
> That's one thing were changes to Scons would benefit Boost.Build. I'm sure 
> there are more...
Right. In practice, I was anticipating that people working on alternate
interfaces around the build engine would prototype the functionality
they need (by subclassing things like Nodes, for example), and then work
out with us how to to migrate that functionality into the SCons build
engine proper, if that's appropriate.
> BTW, Steven, supposing that Boost.Build is rewritten in Python one day, what 
> form of intergration with SCons would you prefer? I suspect that complete 
> merger will be very hard both from UI view and technically (and probably 
> politically)
Architecturally, I think we're all right with the separation between the
build engine and the external interface (modulo the fact that the build
engine is not really documented as a separate interface right now).
Packaging-wise, we also distribute SCons in some "local" packages that
are designed to be dropped in and delivered with other software that
wants to use SCons but doesn't want to force everyone to install it.
One way would be to just use that package to deliver your interface
to your own schedule. Then you can upgrade the build engine when you
choose, and not be bound by our release schedule. My guess is that
would be the most politically palatable way...?
Alternatively, if you wanted to, I'd be happy to merge an integrated
Boost.Build interface into the packages we deliver. Then anyone
installing SCons would just have multiple interfaces to choose from
(two--or more--for the price of one!). I'm already considering doing
this with Asko Kauppi's Lua interface. If that's better because it save
time and effort, great.
I guess it comes down to: I'm more than happy with whatever way makes
the most sense for Boost.Build. We've tried to build in the right
flexibility in the SCons architecture and packaging so we can make
it work with any other software's requirements, be they technical,
political, or otherwise...
Let me know how I can continue helping. Thanks!
--SK
 --==-=-= 
Date: Mon, 16 Jun 2003 11:33:12 -0400
From: David Abrahams <dave_at_[hidden]>
To: Steven Knight <knight_at_[hidden]>
Cc: Vladimir Prus <ghost_at_[hidden]>, Ali Azarbayejani <ali_at_[hidden]>
Subject: Re: [scons-devel] SCons BOF at O'Reilly OSCon 2003
Message-ID: <uhe6qm407.fsf_at_[hidden]>
References: <Pine.LNX.4.44.0306160822530.20778-100000_at_[hidden]>
MIME-Version: 1.0
Wow, great discussion. I'm really sorry we didn't do this on the
Jamboost list now... anyone mind if we move?
Steven Knight <knight_at_[hidden]> writes:
>> 1. Features. Those are quite important and nifty things. I believe Scons is 
>> using Environment for the same purpose?
>
> Yes (based on my quick scan through the documentation). An
> Environment is where you set up how you want one or more products to
> be built: use *this* compiler, *this* version of yacc, these flags,
> these include paths, these libraries, etc...
I always imagined that many features might end up being translated
into Environment settings, but I guess another possibility is that we
just bypass the Environment so that its "smarts" don't get in the way
:(. If we want people who specify features to have a uniform way to
express them, and if we don't think the Environment is going to cover
all of our needs, we may have to do that. I'd rather that we're all
able capitalize on one-another's knowledge of tools and platforms,
though.
>> 2. Main target. In Boost.Build, "main target" --- i.e. target specified by 
>> user --- can generate a big number of variants. Actually, different sets of 
>> real files. If I understand correctly, target defined by user can be built in 
>> only one way by Scons.
>
> I think this would be covered by Alias nodes in SCons. If you have
> multiple variants that you want built under control of one target name
> that the user can specify on the command line, you could set it up like
> so:
>
> env.Alias('libs', ['build/sunos/lib',
> 'build/linux/lib',
> 'build/win32/lib'])
>
> An Alias just expands to whatever other targets are specified--real
> files, directories, other Aliases, whatever.
IMO as Ali has been suggesting in another thread (I *wish* this were
on Jamboost!) we don't really need Main target to be a real Scons
target node at all.
>> 3. Main targets in Boost.Build can have "usage-requirements" --- properties 
>> which are applied to dependents. That's covered in docs at
>> 
>> http://boost.sf.net/boost-build2
>
> So the basic idea is, if I link a program with a library, it may have
> some attributes (-D options or the like) that will automatically
> propagate to the way my program's .o files are built? 
Yes! More likely still, -I options.
> Or just to the way the library is linked with the resulting
> executable?
Either one is possible.
> If I understand this correctly, these would be best handled by
> building the libraries and dependent programs with the same
> Environment. 
That won't happen except in special cases. In general there are
differences between how libraries and their dependents need to be
built. This is just a mechanism for propagating the known required
similarities.
> This makes it explicit, not automatically propagated (which may be a
> feature or a bug, depending on your point of view).
People who want to do everything explicitly can use a lower-level
build tool ;-)
>> 4. We have a notion of "project", which can be referred by
>> "id". This gives great convenience. I can write
>> 
>> exe util : util.cpp @/boost/filesystem/filesystem ;
>> 
>> which will use Boost filesystem library. And I don't care when the library is 
>> really located --- it's specified by some high-level configuration code.
>
> The syntax above is pretty opaque; I don't understand what it's really
> doing under the hood. What does this turn into on an executed command
> line?
What it means at a high level is, "build an executable called util
which uses as sources util.cpp, and the library identified as
'filesystem' in the 'filesystem' subproject of the top-level project
whose name is 'boost'". How that gets translated into a command-line
depends on lots of factors, including the toolset(s) in use.
> If it's a matter of searching for a library in an Autoconf-like way,
> we've just recently added an SConf subsystem that can do that.
Nope, this is a whole different thing, though that sounds really
useful.
IMO, our project abstraction should also be a layer of our own which
we manage on top of Scons' low-level build engine.
>> There are two more thing, that I'm not sure about.
>> 
>> 1. How SCons finds the transformation from sources to desired
>> type. I've tried looking at Builder.py, but did not understood
>> much.
>
> The target type isn't purely intuited from the source type; it's
> explicitly chosen by the builder that's called:
>
> env.Program('foo', [psources]) # build 'foo' or 'foo.exe'
> env.Library('foo', [lsources]) # build 'libfoo.a' or 'foo.lib'
>
> Does that answer the question, or did I misunderstand?
We do the same thing. I think Volodya is asking about how
intermediate targets are determined. This question is related closely
to the generators_prototype.py script I referenced earlier. IOW, when
psources contains '.cpp' files, what mechanism decides that '.o' files
will be built from them and chooses a particular linker to assembe the
executable?
>> 2. How SCons handles toolset differences? Are variables like
>> INCLUDES translated by each tool into relevant command line
>> switches? Is there some common framework for such transformations?
>
> Right now, INCLUDES is translated into -I or /I options by common
> code.
If you're gonna do it with common code you can just use -I ;-). All
those windows compilers respond to '-' just as well as '/'.
> We're at the limits of what that code can do because of the
> differences in how different tool chains handling #include "..." and
> the like. The architecture is there for toolchain-specific
> translations of INCLUDES into options, but no one has yet had a
> compelling enough reason to code up a specific override for a tool.
That was just an example. There are lots of other common options,
such as enabling/disabling debug symbols, optimizations, ... I think
Volodya's question is whether there's a general framework for handling
these things.
>> So far, given all the differences above, we though it's better to
>> implement Python version of Boost.Build as a layer on top of
>> SCons. We'll write the classes to represent main target and
>> projects, and the 'generate' method of such classes will take set
>> of properties and return SCons Node instances --- which will be
>> built by Scons after that. The scheme is straigforward, the
>> interface to Scons is very lean, but I'm not sure if deeper
>> integration would yeild more benefits.
>
> I agree. In fact, we've tried to maintain a clean separation in the
> SCons architecture between the underlying build engine (the Nodes,
> Scanners, Builders and the like) and the wrapping interface. In our
> case, the interface we supply happens to be Python scripts, but we
> *want* people to be able to write other interfaces around the build
> engine, and we're very committed to the idea of enhancing the build
> engine, if necessary, to support other interfaces such as yours.
That is wonderful news!
> For example, Stephen Kennedy has put a continuous-integration
> interface
What does that mean?
> around it, and Asko Kauppi even has a Lua interface (although I think
> he translates into SConscript files instead of using the build engine
> directly).
>
>> For example, at the moment, V2 implements precisely the same scheme, except 
>> that we use 'virtual-target' class instead of scons.Node. There is discussion 
>> if both classes ('main-target' and 'virtual-target') should have common base. 
>> Recast in Scons terms, it would mean that Node should have 
>> 'generate-subvariant(property-set)' method. On regular nodes it will return 
>> self. On nodes corresponding to Boost.Build main target, it will construct 
>> the needed subvariant and return regular nodes for that subvariant.
>
> Odds are, if something is useful for your interface, other interfaces
> might find it useful, too, so I'll be very inclined to work with you
> on extending the Node class (and other parts of the build engine) to
> accomodate your requirements.
That is really wonderful to hear as well.
>> That's one thing were changes to Scons would benefit Boost.Build. I'm sure 
>> there are more...
>
> Right. In practice, I was anticipating that people working on alternate
> interfaces around the build engine would prototype the functionality
> they need (by subclassing things like Nodes, for example), and then work
> out with us how to to migrate that functionality into the SCons build
> engine proper, if that's appropriate.
>
>> BTW, Steven, supposing that Boost.Build is rewritten in Python one day, what 
>> form of intergration with SCons would you prefer? I suspect that complete 
>> merger will be very hard both from UI view and technically (and probably 
>> politically)
>
> Architecturally, I think we're all right with the separation between
> the build engine and the external interface (modulo the fact that
> the build engine is not really documented as a separate interface
> right now).
>
> Packaging-wise, we also distribute SCons in some "local" packages
> that are designed to be dropped in and delivered with other software
> that wants to use SCons but doesn't want to force everyone to
> install it. One way would be to just use that package to deliver
> your interface to your own schedule. Then you can upgrade the build
> engine when you choose, and not be bound by our release schedule.
> My guess is that would be the most politically palatable way...?
I guess I don't understand the implications. Could you be more
specific?
> Alternatively, if you wanted to, I'd be happy to merge an integrated
> Boost.Build interface into the packages we deliver. Then anyone
> installing SCons would just have multiple interfaces to choose from
> (two--or more--for the price of one!). I'm already considering doing
> this with Asko Kauppi's Lua interface. If that's better because it save
> time and effort, great.
>
> I guess it comes down to: I'm more than happy with whatever way makes
> the most sense for Boost.Build. We've tried to build in the right
> flexibility in the SCons architecture and packaging so we can make
> it work with any other software's requirements, be they technical,
> political, or otherwise...
>
> Let me know how I can continue helping. Thanks!
Wow, that is the true spirit of cooperation. Thanks for being you,
Steven.
-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com
 --==-=-= 
Date: Mon, 16 Jun 2003 11:18:09 -0500 (CDT)
From: Steven Knight <knight_at_[hidden]>
To: David Abrahams <dave_at_[hidden]>
cc: Vladimir Prus <ghost_at_[hidden]>, Ali Azarbayejani <ali_at_[hidden]>
Subject: Re: [scons-devel] SCons BOF at O'Reilly OSCon 2003
Message-ID: <Pine.LNX.4.44.0306161041010.20778-100000_at_[hidden]>
MIME-Version: 1.0
Hey David--
> Wow, great discussion. I'm really sorry we didn't do this on the
> Jamboost list now... anyone mind if we move?
No problem on my account. Feel free to forward past messages of mine as
appropriate.
> >> 1. Features. Those are quite important and nifty things. I believe Scons is 
> >> using Environment for the same purpose?
> >
> > Yes (based on my quick scan through the documentation). An
> > Environment is where you set up how you want one or more products to
> > be built: use *this* compiler, *this* version of yacc, these flags,
> > these include paths, these libraries, etc...
> 
> I always imagined that many features might end up being translated
> into Environment settings, but I guess another possibility is that we
> just bypass the Environment so that its "smarts" don't get in the way
> :(.
Hmm, maybe I gave you the wrong impression. Environments are actually
pretty dumb, they're basically just dictionaries of values that get
plugged in to how you build things. They're also *the* way to interact
with the SCons build engine.
> If we want people who specify features to have a uniform way to
> express them, and if we don't think the Environment is going to cover
> all of our needs, we may have to do that. I'd rather that we're all
> able capitalize on one-another's knowledge of tools and platforms,
> though.
I think that's covered. The tools that we support are each in a module
that contain the information about how that tool needs to be built.
What we don't do right now is tie the tools in different tool chains
together as tightly as I'd like. It's *theoretically* possible, for
example, that a given build run will configure the MinGW compiler and
the Visual Studio linker in the same Environment. In practice, it's not
a problem because if your PATH finds the MinGW compiler first, it'd be
really, really weird for it to not find the corresponding linker first,
too...
Nevertheless, this is something I've wanted us to clean up, but it's
working well enough now that it's never been a high priority.
> >> 2. Main target. In Boost.Build, "main target" --- i.e. target specified by 
> >> user --- can generate a big number of variants. Actually, different sets of 
> >> real files. If I understand correctly, target defined by user can be built in 
> >> only one way by Scons.
> >
> > I think this would be covered by Alias nodes in SCons. If you have
> > multiple variants that you want built under control of one target name
> > that the user can specify on the command line, you could set it up like
> > so:
> >
> > env.Alias('libs', ['build/sunos/lib',
> > 'build/linux/lib',
> > 'build/win32/lib'])
> >
> > An Alias just expands to whatever other targets are specified--real
> > files, directories, other Aliases, whatever.
> 
> IMO as Ali has been suggesting in another thread (I *wish* this were
> on Jamboost!) we don't really need Main target to be a real Scons
> target node at all.
Cool.
> >> 3. Main targets in Boost.Build can have "usage-requirements" --- properties 
> >> which are applied to dependents. That's covered in docs at
> >> 
> >> http://boost.sf.net/boost-build2
> >
> > If I understand this correctly, these would be best handled by
> > building the libraries and dependent programs with the same
> > Environment. 
> 
> That won't happen except in special cases. In general there are
> differences between how libraries and their dependents need to be
> built. This is just a mechanism for propagating the known required
> similarities.
Cool, so your interface would probably just have to track how you want
them propagated, and then set up Environments appropriately.
> >> 4. We have a notion of "project", which can be referred by
> >> "id". This gives great convenience. I can write
> >> 
> >> exe util : util.cpp @/boost/filesystem/filesystem ;
> >> 
> >> which will use Boost filesystem library. And I don't care when the library is 
> >> really located --- it's specified by some high-level configuration code.
> >
> > If it's a matter of searching for a library in an Autoconf-like way,
> > we've just recently added an SConf subsystem that can do that.
> 
> Nope, this is a whole different thing, though that sounds really
> useful.
> 
> IMO, our project abstraction should also be a layer of our own which
> we manage on top of Scons' low-level build engine.
Agreed.
> >> 1. How SCons finds the transformation from sources to desired
> >> type. I've tried looking at Builder.py, but did not understood
> >> much.
> >
> > The target type isn't purely intuited from the source type; it's
> > explicitly chosen by the builder that's called:
> >
> > env.Program('foo', [psources]) # build 'foo' or 'foo.exe'
> > env.Library('foo', [lsources]) # build 'libfoo.a' or 'foo.lib'
> >
> > Does that answer the question, or did I misunderstand?
> 
> We do the same thing. I think Volodya is asking about how
> intermediate targets are determined. This question is related closely
> to the generators_prototype.py script I referenced earlier. IOW, when
> psources contains '.cpp' files, what mechanism decides that '.o' files
> will be built from them and chooses a particular linker to assembe the
> executable?
Builders have a 'src_builder' attribute that can be set to one or
more other Builders that can be used to generate input source files.
Builders also have 'suffix' and 'src_suffix' attributes that can be set
to the target and (list of) source file suffixes.
A Builder like the Program() builder has an Object() builder as its
'src_builder', and '.o' (or '.obj') as its 'src_suffix'. The Object()
builder also has a list of known src_suffixes (.c, .cpp, .s, etc.) that
get added to it as different tools are configured/discovered, and it can
in turn have a list of 'src_builders' that know how to build .c files
from .y or .l, etc.
So you can basically hook up Builders arbitrarily using 'src_builder',
and when env.Program() is invoked, we walk back through the list of
src_builders until we find a chain that leads back to the specified
source suffixes. So you end up just listing the input source files:
env.Program('foo', ['f1.o', 'f2.c', 'f3.y', 'f4.s'])
And the build engine works out the internal details based on how the
builders are configured.
> >> 2. How SCons handles toolset differences? Are variables like
> >> INCLUDES translated by each tool into relevant command line
> >> switches? Is there some common framework for such transformations?
> >
> > Right now, INCLUDES is translated into -I or /I options by common
> > code.
> 
> If you're gonna do it with common code you can just use -I ;-). All
> those windows compilers respond to '-' just as well as '/'.
How about the OS/2 icc compiler? Regardless, the actual prefix involved
is configurable in the Environment.
> > We're at the limits of what that code can do because of the
> > differences in how different tool chains handling #include "..." and
> > the like. The architecture is there for toolchain-specific
> > translations of INCLUDES into options, but no one has yet had a
> > compelling enough reason to code up a specific override for a tool.
> 
> That was just an example. There are lots of other common options,
> such as enabling/disabling debug symbols, optimizations, ... I think
> Volodya's question is whether there's a general framework for handling
> these things.
Yes, there is a framwork for this.
We have a separate module for each of the tools that we support,
each module with two interface functions: one searches for the tool
and returns a value that says, "Yes, they have compiler X installed
in a PATH that this Environment can get to;" the other actually
initializes an Environment with all of the appropriate values so that
the Environment can use the tool, creating any necessary Builders or
doing anything else that's required to set up things properly.
> > For example, Stephen Kennedy has put a continuous-integration
> > interface
> 
> What does that mean?
Kind of like AntHill, something that reads up the configuration files
(SConscripts) just once, and then can build and rebuild in response to,
I don't know, signals or checkins or something. He clears out the state
of the Nodes and does the next rebuild without having to restart and
reread the configs all the time.
> > Packaging-wise, we also distribute SCons in some "local" packages
> > that are designed to be dropped in and delivered with other software
> > that wants to use SCons but doesn't want to force everyone to
> > install it. One way would be to just use that package to deliver
> > your interface to your own schedule. Then you can upgrade the build
> > engine when you choose, and not be bound by our release schedule.
> > My guess is that would be the most politically palatable way...?
> 
> I guess I don't understand the implications. Could you be more
> specific?
Basically, we've created a package that's designed to be dropped in to
other packages and used from your local directory, instead of being
installed in a system-wide directory. You could baseline on SCons build
engine version 0.90, say, by just dropping it in to your source tree and
packaging and shipping it with everything else. Then you can upgrade
the build engine version whenever it suits you, and not have to worry
about *your* installation breaking just because we shipped a new version
of SCons. You can keep shipping 0.90 as long as it suits your purposes.
> > Alternatively, if you wanted to, I'd be happy to merge an integrated
> > Boost.Build interface into the packages we deliver. Then anyone
> > installing SCons would just have multiple interfaces to choose from
> > (two--or more--for the price of one!). I'm already considering doing
> > this with Asko Kauppi's Lua interface. If that's better because it save
> > time and effort, great.
> >
> > I guess it comes down to: I'm more than happy with whatever way makes
> > the most sense for Boost.Build. We've tried to build in the right
> > flexibility in the SCons architecture and packaging so we can make
> > it work with any other software's requirements, be they technical,
> > political, or otherwise...
> >
> > Let me know how I can continue helping. Thanks!
> 
> Wow, that is the true spirit of cooperation. Thanks for being you,
> Steven.
Well, I guess if somebody has to be me, it might as well be me... :-)
Thanks for the interest. This sounds like it could be really cool to
bring this stuff together. What's prompting the move in this direction,
anyway?
(BTW, are you guys aware that Ralf W. Grosse-Kunstleve is already using
or shipping Boost, I believe, with an SCons-based build system for
cctbx, his Computational Crystallography ToolBox? I guess he has about
twenty lines of Python code that parse up the .jam or .bjam or whatever
files and turn them into calls into the SCons build engine. Might be a
useful starting point to look at how someone else has done some of the
basic stuff.)
--SK
 --==-=-=-- --=-=-= 
-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com
 --=-=-=--