Subject: Re: [Boost-build] Compiling multiple sources
From: Steven Watanabe (watanabesj_at_[hidden])
Date: 2014-03-12 14:53:57
On 03/12/2014 09:19 AM, Phillip Seaver wrote:
> In this somewhat-ancient thread, I asked about the possibility of
> compiling multiple source files with the same command-line:
> This use case isn't as frequent for me as just changing a few files and
> recompiling, but if I change a primary header in a library, it can cause
> many files to be recompiled (or if I bring in a new version of a
> third-party library, e.g.).
> I ran a test with a library with 48 files, running the compiler
> directly. With a .bat file containing one command-line per file, it
> took about 8.6 seconds. With a single command-line containing all the
> source files, it took about 5.3 seconds.
I'm not sure that this would be a good thing
even if it were possible. Have you considered
the impact on parallel builds? Batching compiles
like this is going to reduce parallelism.
> I know I'm probably an outlier in how I use boost-build. A few of my
> projects use wxWidgets, and we compile it and all the third-party
> libraries we use (and that it uses), so we have some fairly large projects.
I don't think that b2's dependency graph is capable
of representing this.
# First of all, the combining needs to be done
# by b2. It can't be handled by the msvc
# toolset, because we need to include only
# sources that need to be built.
# action modifiers:
# together - combine actions (merging $(>)) for equal values of $(<).
# piecemeal - split into multiple actions (splitting $(>)) to
# avoid overflowing the command line limits.
# updated - only include targets that have been updated in $(>)
# - $(>) is the sources of an action.
# - $(<) is the targets produced by an action.
actions piecemeal together updated compile.c++.combined
# For together to work, we need to create a target
# for each set of build options. This allows b2
# to determine which groups of actions can be merged.
rule get-combined-target ( obj-target )
local options = # get some kind of hash of the variables
return <$(options)>$(target-directory) ;
# Finally, we can add
rule compile.c++ ( target : sources * : properties * )
local group-target = [ get-combined-target $(target) ] ;
compile.c++.combined $(group-target) : $(sources) ;
# Now we're left with a bit of a problem.
# We can't consider $(target) to be updated
# until after $(group-target) has been updated,
# but at the same time, we don't want to force
# add DEPENDS $(target) : $(group-target) ;
# Further, we really want each source to be
# included if the corresponding target is
Boost-Build list run by bdawes at acm.org, david.abrahams at rcn.com, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk