Re: [Boost-build] feature, properties, variants, and all the rest

Subject: Re: [Boost-build] feature, properties, variants, and all the rest
From: Stefan Seefeld (stefan_at_[hidden])
Date: 2017-09-29 15:13:00

On 25.09.2017 19:49, Steven Watanabe via Boost-build wrote:
> AMDG
>
> On 09/25/2017 01:26 PM, Stefan Seefeld via Boost-build wrote:
>> Hi Steven,
>>
>> I hope you don't mind me following up on this long and ongoing
>> discussion. It's really helpful for me to understand not only the
>> existing code but also the underlying rationale of the design.
>>
>> On 04.08.2017 00:41, Steven Watanabe via Boost-build wrote:
>>>   There's no clear priority for usage-requirements
>>> from two different dependencies.  As a result,
>>> if two dependencies have contradictory usage
>>> requirements, it can only be a hard error, as
>>> neither one can override the other. 
>> Yeah, I agree. But in the absence of any such contradictions, I'm still
>> not convinced of the need to restrict features to be applied "late" to
>> have to be "free".
>> (I'm not trying to be difficult. Rather, I'd like to establish a minimal
>> set of working rules, in the hopes to make the overall interface
>> simpler. For users having to remember that only certain features may be
>> set as "usage requirements" seems more complex than necessary.)
>>
>   It's a bit of a trade-off.  Do you want to have
> stricter requirements that guarantee that you
> won't see errors later on or looser requirements that
> let you do anything that doesn't create an actual problem.
> For Boost.Build there are simply too many ways for it
> to fail, so it's better to be strict.

That makes sense.

>> <snip>
>> Now I'm still struggling with the internal workflow in bjam:
>>
>> make0() is used to bind all targets and determine a target's "fate",
>> which is then later used in make1() to determine what to do. If I want
>> to remove the restriction on "free" features above, it means that a
>> target's boundname (filename) may be set later in the process (after
>> prerequisite targets are updated). Therefore I need to either move some
>> of the logic from make0() into make1() (and its subcalls), or I need to
>> repeat it.
>>
>> Can you advise me whether that's possible at all, or whether that would
>> create logical issues in the workflow ? Notably, if I bind a new
>> (file-)name to a target sometime in a make1x() call, I'd need to
>> (re-)run the logic from make0, steps 4g and 4h. Would that be possible
>> at all ? (Right now no make0() call happens once the first make1()
>> invocation has started.)
>>
>   The interaction between SEARCH and LOCATE requires
> the boundname of targets with an explicit LOCATE to
> be set early.
>   SEARCH on <x>file1 = /path1 /path2 ;
>   LOCATE on <x/path1>file1 = /path1 ;
> The boundname of <x>file1 is /path1/file1
> and it gets a hard dependency on <x/path1>file1

This reminds me of a (slightly) tangential question: how are header
dependencies handled by bjam ? In particular:

* it appears as if source code is scanned without any true
pre-processing. Doesn't this lead to false positives (such as
conditional include directives) ?
* how are include directives mapped (bound) to actual files ? Do you
interact with the selected compiler to determine the absolute path ?
* it seems to me that letting the compiler itself do this work (the
typical "make depend" idiom, using commands akin to `gcc -M`) would
solve these issues. What are the reasons bjam doesn't use that strategy ?

>   In addition, it's impossible to determine
> a target's fate without knowing its location,
> as you need an actual file to check the timestamp.
> To make this work, you'd need to merge make0 into
> make1 completely.  This creates another problem,
> because the cycle detection in make0 requires a
> strict depth-first search.

I see "scc" related code in multiple locations, not just in make0().
Assuming that that's the code used to detect cycles, how does it work,
and why isn't it a local call in make0 that
simply raises an error if a cycle is detected ? In fact, why do cycles
merely generate warnings, rather than errors ? Is that merely to support
cyclic header dependencies (which would be another argument to use
something like `gcc -M`, as that would already break up cycles)

>   make1 handles asynchronous
> execution, so the actual traversal order is
> non-deterministic.  (This is probably solvable,
> but will make the algorithm even more complex)

Slightly. The only issue I see right now (assuming the cycle detection
to be solved separately) is with temporaries. If the binding happens
late, there needs to be a provision to postpone the computation of a
temporary's fate until the parent has been bound, as only then the
timestamp comparison can take place. (This obviously implies that it
shouldn't be possible for temporaries to update features, as that would
require temporaries to be force-updated for the data flow to be
well-defined).

        Stefan

-- 
      ...ich hab' noch einen Koffer in Berlin...