Re: [Boost-build] Dependencies dropped during concurrent builds

Subject: Re: [Boost-build] Dependencies dropped during concurrent builds
From: Vladimir Prus (ghost_at_[hidden])
Date: 2013-11-02 01:37:35

On 02.11.2013 03:13, Steven Watanabe wrote:
> AMDG
>
> On 11/01/2013 03:50 PM, Nogradi, Chris wrote:
>>
>> On Friday, November 01, 2013 4:04 PM, Steven Watanabe wrote:
>>> This patch should fix it.  Are there any more problems?
>>>
>>
>> Yes it does fix the issue.  Thanks!  I am trying a number of scenarios before I conclude that it is good.  There are a couple other product builds that need to be tested also.  I'll report what I find.
>>
>> BTW, thanks for the performance improvements that you (and others) have made in the past year to bb, they have been substantial.
>>
>> I do have a question regarding your enhancement to property.refine where you had removed some 'premature optimizations which hurt performance' in svn83854.  Well I had made some changes to this rule to also remove sub-features of features that it had removed, otherwise the overridden feature would be removed but its sub-features would remain and clash with the sub-features specified in the requirements.  But your change suggests that I should not be doing that here.  Or should I? I was doing this before:
>>
>
> That's a good point.  Do you have a test case?
>
>>              local required-value = $(__require__$(p:G)) ;
>>              if $(required-value)
>>              {
>>                  if $(p:G=) != $(required-value)
>>                  {
>>                      result += $(p:G)$(required-value) ;
>>                  }
>>                  else
>>                  {
>>                      result += $(p) ;
>>                  }
>>              }
>>              else
>>              {
>>   --->           local subfeature = [ MATCH <(.*)-(.*):.*> : $(p:G) ] ;
>>   --->           local required-value = $(__require__<$(subfeature[1])>) ;
>>   --->           if ! $(required-value) || $(required-value) = $(subfeature[2])
>>   --->           {
>>                      result += $(p) ;
>>   --->           }
>>              }
>>
>> Also, your change makes a reasonable difference when high level targets have propagated requirements attached to them.  Before the change, adding a propagated property to each dll (we can have up to 40) produced by our builds would add 1/2 second to the no-op build time (for a total of 20 seconds). But even after the change, in order to get good performance on incremental builds, I still had to put the propagated properties as close as possible to where they are needed.  I am not sure I understand why they have such a large impact on the build. Can you comment?  In my current state (being careful with propagated requirements), the change does save me 3 seconds on no-op builds.
>>
>
> Property handling in Boost.Build is very inefficient.
> I've done what I could, but it really needs to be
> rewritten completely.

Do you have specific design in mind? Besides optimization, which would certainly needed,
it would be nice to change the algorithm for metataget generation so that we first pick toolchain
using user-provided properties, and then add other properties inferred from this toolchain,
so that for example address-model is always set to some value.

- Volodya