Re: [Boost-build] allowing targets to fail

Subject: Re: [Boost-build] allowing targets to fail
From: Steven Watanabe (watanabesj_at_[hidden])
Date: 2017-04-08 17:31:15

AMDG

On 04/08/2017 11:00 AM, Stefan Seefeld via Boost-build wrote:
> On 08.04.2017 11:28, Steven Watanabe via Boost-build wrote:
>> 
>>   Yes.  In theory, if check-target-builds has
>> to update multiple targets it can build them
>> in parallel.  However, it's usually used to
>> build one target at a time.  I've considered
>> ways to allow check-target-builds to be fully
>> asynchronous and allow these checks to build
>> in parallel, but this would require almost a
>> total rewrite.
> 
> (I'm experimenting with a new Python frontend, only reusing the existing
> engine. There, I'm able to call `engine.update_now(...)` from within an
> action, which seems to get close to what you are describing.)

  It doesn't.  What is necessary is a non-blocking version
of UPDATE_NOW that notifies you when the targets have
finished updating.

>>>> All properties including check-target-build are handled
>>>> in phase 2.
>>> I see ! That means that the target argument passed to the
>>> "check-target-build" ("foo" in my case above) isn't itself considered a
>>> regular target, or somehow gets special treatment. Right ?
>>   This is just a high level view.  The implementation
>> allows both generating targets and building targets
>> to be handled on a per-target basis.  The special treatment
>> is just that foo gets processed immediately instead of
>> being built later.
> 
> Can you qualify "immediately" and "later" a little more ?

  Immediately means that we call UPDATE_NOW
directly.  Later means that we add the target
to dependency graph and assume that someone
else will call UPDATE_NOW.

> Normal targets
> (those being built "later", that is) are only built if either explicitly
> requested or needed by explicitly requested dependent targets.
> But, for these "immediate" targets this doesn't seem to be possible, or
> does it ?

  Boost.Build only generates the actual targets
which are needed.  check-target-builds will only
be evaluated if we are building the target that
uses it.

> If these immediate targets are used while evaluating which
> (regular) targets need building, it seems impossible to figure out
> whether they need to be built or not. Thus, it sounds like immediate
> targets are always unconditionally built.

  Immediate targets are only built if they are out-of-date,
just like any other target.

> (This also corresponds to what
> Rene suggested earlier.) Am I understanding this correctly ? (I now see
> that even if I run `.../b2 clean` these immediate targets are built (or
> taken from cache). I somehow had missed that earlier.)
> 

Right.  clean interacts somewhat poorly with check-target-builds.

> Speaking of caching: How does b2 handle that ? Is the caching tied to
> all "immediate" targets ?

  check-target-builds saves the result in a special
file which records whether it passed or failed.

> And more generally: how does b2 recognize
> "immediate" targets ? Structurally (by being referenced in property
> expressions, say), or by type (e.g. by virtue of the check-target-build
> rule) ?
> 

  b2 itself doesn't care.  It happily updates targets
whenever you call UPDATE_NOW.

>>> Are there any
>>> restrictions on what "foo" might be ? (In the above case it's itself a
>>> target created with the "obj" meta-target rule, so I'd assume it could
>>> be anything.
>>>
>> It can be any target.
>>
>>> Conceptually this seems like a two-phase build,
>>   It's actually more of a N-phase build.
>> The targets used by check-target-builds
>> can themselves use check-target-builds.
>> The process of building the targets used
>> by check-target-builds is identical to the
>> main build.
> 
> What mechanism detects these dependencies (and thus triggers
> prerequisite targets to be built) ? I assumed all this scheduling only
> happens in phase 3 (in your original description), but it now seems
> that's not the case.
> 

  You're thinking about it wrong.  b2 doesn't break
things up into multiple passes.  All the engine does
is to update targets when you ask it to (which you
can do as many times as you want).  check-target-builds
requests updating a target when it is evaluated.

In Christ,
Steven Watanabe