Re: [Boost-build] dynamically editing the bjam dependency graph

Subject: Re: [Boost-build] dynamically editing the bjam dependency graph
From: Stefan Seefeld (stefan_at_[hidden])
Date: 2017-09-02 01:00:53

On 09/01/2017 08:49 PM, Steven Watanabe via Boost-build wrote:
> AMDG
>
> On 09/01/2017 07:55 AM, Stefan Seefeld via Boost-build wrote:
>> On 08/31/2017 11:39 PM, Steven Watanabe via Boost-build wrote:
>>> On 08/31/2017 09:10 PM, Stefan Seefeld via Boost-build wrote:
>>>> I'm still struggling with implementing the dynamic editing of the bjam
>>>> dependency graph (as outlined in
>>>> https://github.com/stefanseefeld/faber/wiki/Workflow). While I believe I
>>>> have worked out the Python frontend, the bjam backend isn't quite
>>>> cooperating the way I expect.
>>>> Consider this test code:
>>>>
>>>   The reason this fails is that dependencies
>>> are processed once by make1a, when the target
>>> is first pushed into the stack.  This happens
>>> before 'gen' is executed, so the additional
>>> dependencies that it adds are ignored.  Handling
>>> this correctly probably requires significant
>>> changes to make1.
>> If I understand correctly, the new dependencies need to be updated if
>> their dependants have a fate other than T_FATE_INIT, in which case
>> T_STATE_MAKE1A needs to be submitted for them.
>>
>> So, whenever a new dependency is registered, I need to look at the fate
>> of the target, and if that's already being processed, I submit the
>> sources, too. That's what the attached patch does, and it does indeed
>> make my little test do what I expect.
>>
>> Does this approach sound reasonable ? Am I overlooking some cases ?
>>
> - What happens if the dependencies are added after a
>   target has completed updating?

It should raise an error.

> - Can it hang when the added dependencies create a cycle?

Can the cycle be detected ? (There seems to be some code in the engine
to do that.)

> - How does this interact with cyclic #includes of generated headers?

I have no idea yet. As I mentioned, I (intend to) handle include parsing
quite differently from b2, i.e. I scan for include directives and submit
the found headers as prerequisites via calls to `builtin_depends`. How
do you expect this to interact ? Do you expect any difficulties ?

> - Is it correct for dependencies added in between two calls
>   to UPDATE_NOW?

Why not ? In fact, I don't expect multiple calls to UPDATE_NOW. There is
no API that exposes that to users. With my current design there simply
is no need for it. config checks etc. are all simply artefacts like any
others that are declared and executed as per the (dynamic) dependency graph.

        Stefan

-- 
      ...ich hab' noch einen Koffer in Berlin...