Proto for Boost mailing list page: Re: [proto] Thoughts on traversing proto expressions and reusing grammar

Subject: Re: [proto] Thoughts on traversing proto expressions and reusing grammar
From: Thomas Heller (thom.heller_at_[hidden])
Date: 2010-10-05 01:57:24

Eric Niebler wrote:

> On 10/4/2010 12:20 PM, Thomas Heller wrote:
>> On Mon, Oct 4, 2010 at 8:53 PM, joel falcou
>> <joel.falcou-CYrOEBX6jiM_at_[hidden]> wrote:
>>> On 04/10/10 20:45, Eric Niebler wrote:
>>>>
>>>> I'm not opposed to such a thing being in Proto, but I (personally)
>>>> don't feel a strong need. I'd be more willing if I saw a more strongly
>>>> motivating example. I believe Joel Falcou invented something similar.
>>>> Joel, what was your use scenario?
>>>>
>>>
>>> NT2 ;)
>>>
>>> More specifically, all our transform are built the same way:
>>> visit the tree, dispatch on visitor type + tag and act accordignly.
>>> It was needed for us cause the grammar could NOT have been written by
>>> hand as we supprot 200+ functions on nt2 terminal. All our code is
>>> somethign like "for each node, do Foo" with variable Foo depending on
>>> the pass and duplicating
>>> the grammar was a no-no.
>>>
>>> We ended up with somethign like this, except without switch_ (which I
>>> like btw), so we
>>> can easily add new transform on the AST from the external view point of
>>> user who
>>> didn't have to know much proto. As I only had to define one grammar (the
>>> visitor) and only specialisation of the
>>> visitor for some tag, it compiled fast and that was what we wanted.
>>>
>>> Thomas, why not showign the split example ? It's far better than this
>>> one and I remember I and Eric
>>> weren't able to write it usign grammar/transform back in the day.
>> 
>> The split example was one of the motivating examples, that is correct,
>> though it suffers the exact points Eric is criticizing.
>> The split example was possible because i added some new transforms
>> which proto currently misses, but i didn't want to shoot out all my
>> ammunition just yet :P
>> But since you ask for it:
>> 
http://github.com/sithhell/boosties/blob/master/proto/libs/proto/test/splitter.cpp
> 
> Can you describe in words or add some comments? It's not immediately
> obvious what this code does.

Sure, please reload that site and follow my line to line description.

line 57 to 68:
    We want to determine whether our current expression contains some split 
    expression. that is what the fold does.
line 69 to 97:
    Fold returned true, which means, we have to do two things, and join the 
    results of 1) and 2):
    1) replace the split(...) with a placeholder, and place it to our 
       resulting vector
       line 70 to 81:
       Transform the children of our expression: if it is a split, replace 
       it with placeholder, if not, just leave it be.
    2) insert the "..." (which is the subexpression we want to split from 
       our main expression into our resulting vector
       line 82 to 96:
       traverse the children again, if we encounter a split expression, 
       insert the children of this expression into our fusion vector, if not 
       return the state (which is our fusion vector)
       Note 1: The traversal of our tree happens here, we pass the result of 
       the recursive call to split_grammar to either join or push_back (line 
       90 and 91)
       Note 2: This is a Bottom-Up traversal.
line 98:
    The children of our current node have no split expressions, just leave 
    them be, and pack into a fusion vector, the reason for that is because  
    of we call split_grammar with an expression containing no split at all, 
    and call fusion::as_vector on the result, it will unpack the children of 
    the expression.

I hope that makes the code clearer. The hardest part of this exercise was 
not to write down the grammar and the transforms, but to actually change the 
children of an arbitrary expression:
 
>> the new thing i added is transform_expr, which works like
>> fusion::transform: It creates the expression again, with transformed
>> child nodes (the child nodes get transformed according to the transform
>> you specify as template parameter
> 
> How is that different than what the pass_through transform does?

It works slightly different than pass_through:

transform_expr calls
when<_, Fun>::template impl<typename result_of::child_c<Expr, N>::type,
State, Data>
while pass_through calls
Grammar::proto_childN::template impl<typename result_of::child_c<Expr,
N>::type, State, Data>

Truth is, I could not see how I could use pass_through to do what I wanted, 
and still fail to see what pass_through really does. Another truth is, that 
i initially wanted to call transform_expr pass_through until I realized that 
it is already there.
Perhaps you can sched some light on why pass_through is implemented that 
way.