Re: [Boost-build] question(s) about bjam code II

Subject: Re: [Boost-build] question(s) about bjam code II
From: Stefan Seefeld (stefan_at_[hidden])
Date: 2017-10-20 11:57:38

On 20.10.2017 01:32, Steven Watanabe via Boost-build wrote:
> AMDG
>
> On 10/19/2017 10:34 PM, Stefan Seefeld via Boost-build wrote:
>> I have been following your advice and tried to merge the "make0()" function call 
>> to compute a target's fate into the state machine implemented in make1.c, to 
>> support a more dynamic build process. In general this is working great (i.e. I 
>> can now freely mix config-like targets that compute parameters that are later 
>> used in the build, with other targets). However, there is one set of use-cases 
>> that's still causing me some headache: actions that update more than one target, 
>> i.e. that require other targets to be "rebuilt".
>>
>> While I understand the situation (as well as the implementation with 
>> "force_rebuilds()") well, I'm still struggling with some of the code:
>>
>> Why is the loop in 
>> https://github.com/boostorg/build/blob/develop/src/engine/make1.c#L1220-L1227 
>> launching all the targets that are to be updated by the current command, and 
>> then letting that command wait for them to be ready ? (I see the corresponding 
>> "unlock" code is 
>> https://github.com/boostorg/build/blob/develop/src/engine/make1.c#L939-L948, but 
>> I don't quite understand that part of the workflow.)
>>
>   It's basically dealing with the fact that there
> isn't a one-to-one correspondence between targets
> and actions.  What we need to schedule is the actions,
> but the dependencies are expressed in terms of targets.
> The rule is that an action can be started when all
> the dependencies of all the targets it creates are
> up-to-date.
>   The code in question was introduced in this commit:
> https://github.com/boostorg/build/commit/c1395b49614ed5c4b0958355ffaa0f794fb760a2
> which replaces an older implementation which used INCLUDES.
> (I don't remember the details of why INCLUDES caused
> problems off the top of my head.)
>   Also, keep in mind that these were fixes that were
> put in much later than the original code.  The
> original Perforce Jam mishandled actions
> that produced multiple targets quite badly.
> The mismatch between dependencies/targets/actions
> is quite annoying to deal with, and the fix
> to make it work is a bit of a kludge.

Right, indeed. At first sight, the above logic seemed strange ("update
all the targets before running the action that updates them"), but I
suppose it makes sense, as "update" doesn't really mean to complete
them; just get them to the state where the action can be performed.

I have to admit that I find that state engine in make1.c both ingenious
and challenging, especially after I augmented it by additional steps to
deal with the dynamic dependency graph manipulation. I'm constantly
switching between my computer screen and a whiteboard, trying to
formalize the state transitions (complicated by the asynchronous
handling of commands).

But, I'm overall very happy with the results, as I believe the logic
expressed in in the higher layers of the build logic (especially the
fabscripts themselves) is more intuitive and straight-forward. I hope to
stabilize this quickly, so I can ask for a public review.

Thanks again for your ongoing help !

Stefan

-- 
      ...ich hab' noch einen Koffer in Berlin...