Re: [Boost-build] allowing targets to fail

Subject: Re: [Boost-build] allowing targets to fail
From: Stefan Seefeld (stefan_at_[hidden])
Date: 2017-04-08 17:00:35

On 08.04.2017 11:28, Steven Watanabe via Boost-build wrote:
> AMDG
>
> On 04/08/2017 08:46 AM, Stefan Seefeld via Boost-build wrote:
>> On 08.04.2017 10:13, Steven Watanabe via Boost-build wrote:
>>> Phase 1: Load Jamfiles
>>> Phase 2: Generate actual targets
>>> Phase 3: Build targets
>> OK. And the parallel execution (requested via the "-j" option) would
>> only affect phase 3 ?
>>
>   Yes.  In theory, if check-target-builds has
> to update multiple targets it can build them
> in parallel.  However, it's usually used to
> build one target at a time.  I've considered
> ways to allow check-target-builds to be fully
> asynchronous and allow these checks to build
> in parallel, but this would require almost a
> total rewrite.

(I'm experimenting with a new Python frontend, only reusing the existing
engine. There, I'm able to call `engine.update_now(...)` from within an
action, which seems to get close to what you are describing.)
>>> All properties including check-target-build are handled
>>> in phase 2.
>> I see ! That means that the target argument passed to the
>> "check-target-build" ("foo" in my case above) isn't itself considered a
>> regular target, or somehow gets special treatment. Right ?
>   This is just a high level view.  The implementation
> allows both generating targets and building targets
> to be handled on a per-target basis.  The special treatment
> is just that foo gets processed immediately instead of
> being built later.

Can you qualify "immediately" and "later" a little more ? Normal targets
(those being built "later", that is) are only built if either explicitly
requested or needed by explicitly requested dependent targets.
But, for these "immediate" targets this doesn't seem to be possible, or
does it ? If these immediate targets are used while evaluating which
(regular) targets need building, it seems impossible to figure out
whether they need to be built or not. Thus, it sounds like immediate
targets are always unconditionally built. (This also corresponds to what
Rene suggested earlier.) Am I understanding this correctly ? (I now see
that even if I run `.../b2 clean` these immediate targets are built (or
taken from cache). I somehow had missed that earlier.)

Speaking of caching: How does b2 handle that ? Is the caching tied to
all "immediate" targets ? And more generally: how does b2 recognize
"immediate" targets ? Structurally (by being referenced in property
expressions, say), or by type (e.g. by virtue of the check-target-build
rule) ?

>> Are there any
>> restrictions on what "foo" might be ? (In the above case it's itself a
>> target created with the "obj" meta-target rule, so I'd assume it could
>> be anything.
>>
> It can be any target.
>
>> Conceptually this seems like a two-phase build,
>   It's actually more of a N-phase build.
> The targets used by check-target-builds
> can themselves use check-target-builds.
> The process of building the targets used
> by check-target-builds is identical to the
> main build.

What mechanism detects these dependencies (and thus triggers
prerequisite targets to be built) ? I assumed all this scheduling only
happens in phase 3 (in your original description), but it now seems
that's not the case.

>> where a first pass
>> filters out all those "check-target-build" targets and builds those, to
>> evaluate dependent properties (what other effects may these have other
>> than defining properties ?).
>   It's implemented as a <conditional> which can
> only affect properties.  However, properties
> can affect anything.

Thanks,
        Stefan

-- 
      ...ich hab' noch einen Koffer in Berlin...