Re: [Boost-build] Compiling multiple sources

Subject: Re: [Boost-build] Compiling multiple sources
From: Steven Watanabe (watanabesj_at_[hidden])
Date: 2014-03-12 14:53:57

AMDG

On 03/12/2014 09:19 AM, Phillip Seaver wrote:
> In this somewhat-ancient thread, I asked about the possibility of
> compiling multiple source files with the same command-line: 
> http://lists.boost.org/boost-build/2007/07/16817.php
> 
> This use case isn't as frequent for me as just changing a few files and
> recompiling, but if I change a primary header in a library, it can cause
> many files to be recompiled (or if I bring in a new version of a
> third-party library, e.g.).
> 
> I ran a test with a library with 48 files, running the compiler
> directly.  With a .bat file containing one command-line per file, it
> took about 8.6 seconds.  With a single command-line containing all the
> source files, it took about 5.3 seconds.
> 

I'm not sure that this would be a good thing
even if it were possible.  Have you considered
the impact on parallel builds?  Batching compiles
like this is going to reduce parallelism.

> I know I'm probably an outlier in how I use boost-build.  A few of my
> projects use wxWidgets, and we compile it and all the third-party
> libraries we use (and that it uses), so we have some fairly large projects.
> 

I don't think that b2's dependency graph is capable
of representing this.

# First of all, the combining needs to be done
# by b2.  It can't be handled by the msvc
# toolset, because we need to include only
# sources that need to be built.
#
# action modifiers:
#   together  - combine actions (merging $(>)) for equal values of $(<).
#   piecemeal - split into multiple actions (splitting $(>)) to
#               avoid overflowing the command line limits.
#   updated   - only include targets that have been updated in $(>)
#
# Notes:
#  - $(>) is the sources of an action.
#  - $(<) is the targets produced by an action.
actions piecemeal together updated compile.c++.combined
{
}

# For together to work, we need to create a target
# for each set of build options.  This allows b2
# to determine which groups of actions can be merged.
rule get-combined-target ( obj-target )
{
  local options = # get some kind of hash of the variables
  return <$(options)>$(target-directory) ;
}

# Finally, we can add
rule compile.c++ ( target : sources * : properties * )
{
  local group-target = [ get-combined-target $(target) ] ;
  compile.c++.combined $(group-target) : $(sources) ;
}

# Now we're left with a bit of a problem.
# We can't consider $(target) to be updated
# until after $(group-target) has been updated,
# but at the same time, we don't want to force
# add DEPENDS $(target) : $(group-target) ;
# Further, we really want each source to be
# included if the corresponding target is
# updated.

In Christ,
Steven Watanabe