[Boost-build] dynamically editing the bjam dependency graph

Subject: [Boost-build] dynamically editing the bjam dependency graph
From: Stefan Seefeld (stefan_at_[hidden])
Date: 2017-09-01 03:10:41

Hi,

I'm still struggling with implementing the dynamic editing of the bjam
dependency graph (as outlined in
https://github.com/stefanseefeld/faber/wiki/Workflow). While I believe I
have worked out the Python frontend, the bjam backend isn't quite
cooperating the way I expect.
Consider this test code:

```
echo = action('echo', 'echo $(<) $(>)')
c = artefact('c', attrs=notfile)

def generate(*args, **kwds):
    """generate the graph for c."""
    a = rule('a', recipe=echo, attrs=notfile|always)
    a1 = rule('a1', a, recipe=echo, attrs=notfile|always)
    a2 = rule('a2', a1, recipe=echo, attrs=notfile|always)
    rule(c, a2, recipe=echo, attrs=notfile)
    bjam.print_dependency_graph(c.qname)
   
b = rule('b', recipe=action('gen', generate), attrs=notfile|always)
c = depend(c, b)
bjam.print_dependency_graph(c.qname)

default = c
```

The goal is to ultimately build 'c' from 'a' using the chain a -> a1 ->
a2 -> c.
However, the exact intermediate steps aren't known at the start (when
the code above is parsed). Instead, the immediate dependency graph is b
-> c. Updating 'b' is done calling the 'generate' function above, which
will inject the actions to generate the above dependency graph.

The code above contains two calls to "bjam.print_dependency_graph()"
that prints the DG for 'c', once after the code above is parsed, and a
second time after the remaining actions are submitted. So, the first
time this yields:

```
   0 Name: c
         : NOTFILE
         : Depends on b (init) (max time)
    1 Name: b
          : NOTFILE TOUCHED
```
while the second yields
```
-> 0 Name: c
         : Updating it
         : NOTFILE
         : Depends on b (touched) (max time)
         : Depends on a2 (init) (max time)
->  1 Name: b
          : Been touched, updating it
          : NOTFILE TOUCHED
    1 Name: a2
          : NOTFILE TOUCHED
          : Depends on a1 (init) (max time)
     2 Name: a1
           : NOTFILE TOUCHED
           : Depends on a (init) (max time)
      3 Name: a
            : NOTFILE TOUCHED
```
which looks correct (this is indeed output generated by
https://github.com/boostorg/build/blob/develop/src/engine/make.c#L784).

However, only the "echo c" action is actually executed (after "gen B",
of course). This of course is wrong: I would expect

```
gen b
echo a1
echo a2
echo c
```
Any idea why that may not be happening ? I have injected "printf()"
statements into make1a(), make1b(), and make1c() (printing the target
name), to attempt to trace a bit what is going on. I have attached the
full log to this mail, and would very much appreciate any help in
figuring out what is going on, i.e. why the dependency graph isn't
properly executed, despite the (seemingly) correct DG printout above.

Many thanks,
        Stefan

PS: I'm using "builtin_depends()"
(https://github.com/boostorg/build/blob/develop/src/engine/builtins.c#L543)
to declare dependencies.

-- 
      ...ich hab' noch einen Koffer in Berlin...