Boost logo

Boost-Build :

Subject: [Boost-build] RFC: problems lazy loading modules referenced using source target-ids
From: Jurko Gospodnetiæ (jurko.gospodnetic_at_[hidden])
Date: 2012-08-07 06:05:31

   Hi all.

   I need some more opinions on what to do with a Boost Build problem
related to lazy loading project modules explicitly referenced using
source target-ids, e.g as in '../some/other/project//source'.

   A text-book Boost Build run does the following:
     1. Loads the main project module, its parent modules and
recursively any additional project modules referenced from there (e.g.
via the use-project rule). These loaded modules contain different
feature declarations.
     2. Processes the build request command-line (all the features used
in the command-line need to be declared by this time).
     3. Constructs the virtual-target/action tree needed to construct
the requested main targets and the declared features are used in this
process, especially non-optional features.

   However, things get complicated if explicit project module references
are allowed in source ids, e.g.:
   exe some-exe : ../some/other/project//source ;

   Such additional modules get loaded in phase 3 instead of phase 1,
assuming nothing caused them to get loaded earlier.

   This then may (and does!) cause problems if such lazy-loaded modules
define new non-optional features. Such features are then taken into
consideration when constructing later targets, but were not taken into
consideration when constructing earlier ones. There may also be other
issues caused by this lazy loading that I am not aware of.

   I'm attaching a test case demonstrating one such issue, prepared to
be run as a regular internal Boost Build test case. It prepares a
project where, if you build its ccc and ddd targets in order, it fails
due to two separate virtual-targets getting prepared for a single actual
produced file (each has a bit different property set). If you reverse
the order in which these two targets are built - everything goes well.
Also, if you load its 'a' project module directly using the use-project
rule - everything builds fine no matter which order the targets are
built in.

   Another issue caused by the same underlying problem was reported
recently on the list by Norbert Wenzel (see the 'Build error when adding
Boost libraries to own projects' thread).

   Debugging issues caused by this gets real messy and requires one to
delve deep into the Boost Build internals which does not seem like a
'Good Thing'.

   I now ask 'what can be done about this'?

   Some ideas:

--- ignore the issue ---
   Don't like this as we are leaving behind a problem that people are
going to keep running into time and time again, causing them to waste a
lot of time debugging the issue and possibly even give up on using Boost
Build all together.

--- disallow explicit project module references in source target-ids ---
   Alternative is to reference the project module earlier using the
use-project rule, assigning a project-id for it and then use that
project-id to reference targets from that project. Bad thing about this
is that we are possibly complicating simpler build scenarios and
potentially slowing down more complex one with use this to lazy load
project modules only when and id actually needed.

--- disallow declaring features in modules loaded due to being
explicitly references in a used source target id ---
   The only problems caused by such lazy loaded modules that I can
currently see are caused by them declaring features. We could possibly
report an error if this is enountered and allow such lazy module loading
for projects not declaring additional features.

--- disallow declaring non-optional features in modules loaded due to
being explicitly references in a used source target id ---
   It is even possible (I am not 100% certain of this though) that
declaring optional features (i.e. those that do not get their default
value automatically added to all built targets) does not cause any
problems and can be allowed. Just food for though...

   Thanks in advance for your input...

   Best regards,
     Jurko Gospodnetiæ

   There is also a bit more general design change that could be done at
a later time which I find like a nice idea, and that is to make features
project-specific (or at least allow them to be declared as such). Then
we could allow project-specific features to be declared as well but all
this is a separate issue.

Boost-Build list run by bdawes at, david.abrahams at, gregod at, cpdaniel at, john at