Re: [Boost-build] Dependencies dropped during concurrent builds

Subject: Re: [Boost-build] Dependencies dropped during concurrent builds
From: Steven Watanabe (watanabesj_at_[hidden])
Date: 2013-10-31 11:50:38

AMDG

On 10/31/2013 06:24 AM, Nogradi, Chris wrote:
> Steve,
> 
> Sorry about that, I retired for the evening.
> 
> Well that target is a hack.  In order to work around the problem that you are fixing, I had to break up a large generator into many and in order to run the script that generates the large number of targets once, I used a temp file to see if I had run it already in the action.  Here is the generator:
> 
> rule gen_files ( targets * : sources * : properties * )
> {
>    ...
>    JAM_SEMAPHORE on $(targets) = <s>verify-gen-semaphore ;
>    local temp = $(targets[1]:G)verify_temp ;
>    NOCARE $(temp) ;
>    LOCATE on $(temp) = [ on $(targets[1]) return $(LOCATE) ] ;
>    TEMP on $(targets) = $(temp) ;
>    DEPENDS $(temp) :  $(sources) ;
>    DEPENDS $(targets) :  $(temp) ;
>    common.RmTemps $(temp) : $(temp) ;
> }
> 

Can $(targets) ever have more than one target here?
If so, then the problem is definitely the
JAM_SEMAPHORE.  (gen_files now waits until
all its targets are ready to update.  JAM_SEMAPHORE
prevents more than one target from being ready
at the same time.).  Ugh.  I'm going to have to
rewrite the semaphore code, too.

> Here is the action:
> 
> .RM = [ common.rm-command ] ;
> .IF_EXISTS = "if exist" ;
> .CREATE = [ common.file-creation-command ] ;
> 
> actions gen_files bind TEMP
> {
>     $(.IF_EXISTS) "$(TEMP)" exit 0
>     $(.CREATE) "$(TEMP)"
>     "$(.path)verify_code_gen.py" -x $(>[2]) -d $(>[1]) -e -s $(>[1]:D) -a $(<[1]:D) --lf @($(<[1]:S=.libs:W):E=$(.src-path)$(INCLUDES:J=;);$(IMP_INCLUDES:J=;))
> }
> 
> Anyways, I reverting this ugliness and now the generator looks something like this (modules can be up to 30 or so strings):
> 
> for local mod in $(modules)
> {
>     .targets += VRFY_PUB_GEN_H($(mod)_pub_verify_%) VRFY_PUB_GEN_TEST_H($(mod)_pub_verify_test_%)
>                 VRFY_PRV_GEN_H($(mod:L)_prv_verify_%) VRFY_PRV_GEN_TEST_H($(mod:L)_prv_verify_test_%)
>                 VRFY_GEN_C($(mod:L)_verify_%) VRFY_GEN_TEST_C($(mod:L)_verify_test_%) ;
> }
> generators.register-composing verify_limits.gen_files : VRFY_DICT CTYPE_DEFS : $(.targets) ;
> 
> When I run this version I get similar behavior to what I had originally reported to you that the build completes with no failures but not all targets are built:
> 
> ...patience...
> ...patience...
> ...patience...
> ...patience...
> ...patience...
> ...found 123386 targets...
> ...updating 5684 targets...
> <snip>
> ...updated 5500 targets...
> 
> If I run again, I get:
> 
> ...patience...
> ...patience...
> ...patience...
> ...patience...
> ...patience...
> ...patience...
> ...found 133400 targets...
> ...updating 184 targets...
> 
> And nothing is built.
> 
> However if I now try to build one of the dlls, it actually builds it (-d+12 attached in out file).  Then if I invoke the entire build, it actually finishes and produces all the dlls and the final exe (out2 attached).
> 
> Thanks for your help diagnosing this.
> 

It looks like the problem is sys_resource.dll.
Is there anything unusual about this dll?

In Christ,
Steven Watanabe