Proto for Boost mailing list page: Re: [proto] Thoughts on traversing proto expressions and reusing grammar

Subject: Re: [proto] Thoughts on traversing proto expressions and reusing grammar
From: Eric Niebler (eric_at_[hidden])
Date: 2010-10-13 16:46:40

On 10/13/2010 11:54 AM, Thomas Heller wrote:
> On Wednesday 13 October 2010 20:15:55 Eric Niebler wrote:
>> Possibly. But now, you're explicitly constructing a terminal with a
>> reference_wrapper as a value. Proto doesn't know how to treat
>> reference_wrapper when evaluating Phoenix expression because you haven't
>> told it how. You'll need to fit this into your evaluation strategy
>> somehow. It will be a good test to see how truly extensible your
>> proposed mechanism is.
> 
> Ok, i have to admit, it gets a little messy here. We probably would like to 
> specialize our generic evaluator based on the terminal tag.
> Which takes care of: a) regular terminals (as in captured by value) b) 
> things like reference wrapper c) placeholders as you suggested them.
> I don't have a good solution to that problem. Anyone else?

Not sure yet. This needs thought.

> The only thing i can think of right know is to introduce a special tag for a 
> reference. It would be great if we could make proto::terminals with different 
> tags, or let's not call them terminals but nullary expressions.

Proto already lets you define nullary expressions with custom tags.
Might be handy here.

<snip>
>> Simple. They become:
>>
>>   typedef
>>     actor<proto::terminal<phoenix_placeholder<mpl::int_<0> > >::type>
>>   arg1_type;
>>
>>   arg1_type const arg1 = {};
>>
>> And then your default evaluator changes to handle placeholders:
>>
>>     template <typename Tag>
>>     struct generic_evaluator
>>       : proto::or_<
>>             proto::when<
>>                 proto::terminal<phoenix_placeholder<proto::_> >
>>               , fusion_at(proto::_value, proto::_state)
>>             >
>>           , proto::_default<eval_grammar>
>>         >
>>     {};
>>
>> or something. If placeholder are part of the core, there's nothing
>> stopping you from doing this. You can also handle reference_wrappers
>> here, but that feels like a hack to me. There should be a way for
>> end-users to customize how Phoenix handles terminal types like
>> reference_wrapper.
> 
> The problem i am having with approach to tackle placeholders, is that i 
> still can't see how this would solve the grand "placeholder unification" 
> problem

There are two aspects of "placeholder unification":

1) Using phoenix placeholders in other EDSLs.
2) Using placeholders from other EDSL in Phoenix.

(1) is solved by making the phoenix domain a sub-domain of
proto::default_domain. (2) is not yet solved, but can be trivially done
so by changing the above evaluator to:

struct Placeholder
  : proto::when<
        proto::and_<
            proto::terminal<proto::_>
          , proto::if_<boost::is_placeholder<proto::_value>()>
        >
      , fusion_at(proto::_value, proto::_state)
    >
{};

template<class Tag>
struct generic_evaluator>
  : proto::or_<
        Placeholder
      , _default<eval_grammar>
    >
{};

Note the use of some hook-able boost::is_placeholder trait.

<snip>
>>> However, it can be easily added (this is actually where the visitor
>>> plays out its strength :)).
>>> In line 47 i defined phoenix_grammar, which can be specialized on tags
>>> to define a better grammar for one specific tag.
>>
>> I don't get that. There needs to be a phoenex grammar. One, and only
>> one. I don't see why it's a template, what the tag type is for, or how
>> one could use a tag to define a "better" phoenix grammar. Because
>> there's only one. Either an expression is a Phoenix grammar or it isn't,
>> right?
> 
> I disagree here.
> End users might want to extend phoenix (the EDSL itself), by introducing new 
> expressions (right now by introducing a new tag). Now, the user wants to 
> specifiy what valid expressions look like that can be formed with his new 
> expression type. Here i currently can imagine two different scenarios:
> First, the user just wants to extend our "regular" grammar. This can be 
> achieved by something like this:
> 
> template <>
> struct phoenix_grammar<my_new_super_cool_tag>
>   : // some proto grammar which specifies how valid expressions look like
> 
> The second scenario i can imagine, that someone wants to build some kind of 
> other language reusing phoenix parts, imagine someone who is so obsessed 
> with functional programming, and doesn't like the current approach and wants 
> a pure functional programming EDSL. Thus he wants to disallow certain 
> expressions. The current visitor design makes that perfectly possible!
> (I hope this makes things clearer)

No, I'm afraid it doesn't. Extending phoenix by introducing additional
tags doesn't necessitate parameterizing the *whole* grammar on a tag.
The old design handled that scenario just fine by using the openly
extensible proto::switch_. Just one grammar, and users could hook it easily.

And the second scenario also doesn't suggest to me that the grammar
needs parameterization. It suggests to me that someone who wants a
radical customization of Phoenix grammar should make their domain a
sub-domain of Phoenix, reuse the parts they want, and define the grammar
of their new EDSL.

>>> You can then check with (admitting this doesn't work yet, but i plan to
>>> implement this soon):
>>> proto::matches<phoenix_visitor<proto::_>, some_expr>
>>> If an expression is a valid lambda. Note, that you don't really have to
>>> care about the algorithm (or transform) and just check that the data
>>> you are passing is valid.
>>
>> That made no sense to me. Care to try again?
> 
> Sure.If you look at my code, there is a construct like this:
> 
> template <template <typename> class Visitor> struct phoenix_visitor
>     : proto::visitor<Visitor, phoenix_grammar> {};
> 
> phoenix_grammar now has the definitions on what a valid phoenix expression is 
> (defined by whoever through the tag dispatch). the only part missing is the 
> Visitor. But for validating visitors we don't really care what transforms 
> will be applied to our expression. Thus the proto::_ placeholder for 
> visitor.

(I'll ignore for now the fact that you can't use proto::_ as a template
template parameter.)

This is all very convoluted. After days of looking and much discussion,
I'm afraid I still don't get it. I think it's a bad sign that so far
I've heard no simple, comprehensible, high-level description of the
architecture of this design.

Diving once more into the code,

-- 
Eric Niebler
BoostPro Computing
http://www.boostpro.com