Proto for Boost mailing list page: Re: [proto] [phoenix3] New design proposal

Subject: Re: [proto] [phoenix3] New design proposal
From: Thomas Heller (thom.heller_at_[hidden])
Date: 2010-10-20 01:30:08

On Wednesday 20 October 2010 05:19:17 Joel de Guzman wrote:
> On 10/20/2010 12:08 AM, Eric Niebler wrote:
> > On 10/19/2010 1:33 AM, Thomas Heller wrote:
> >> On Tue, Oct 19, 2010 at 6:21 AM, Joel de Guzman wrote:
> >>> Can we also focus on one very specific use-case that demonstrates
> >>> the motivation behind the need of such a refactoring and why the
> >>> old(er) design is not sufficient? I'd really want to sync up with
> >>> you guys.
> >> 
> >> With the old design (the one which is currently in the gsoc svn
> >> sandbox) I had problems with defining what phoenix expressions really
> >> are. We had at least two types of expressions. First were the ones we
> >> reused from proto (plus, multiplies, function and so on), Second were
> >> these proto::function constructs which had a funcwrap<T>  struct and
> >> an env placeholder. This env placeholder just wastes a valuable slot
> >> for potential arguments. The second point why this design is not
> >> good, is that data and behaviour is not separated. The T in funcwrap
> >> defines how the phoenix expression will get evaluated.
> >> 
> >> This design solves this two problems: Data and behaviour are cleanly
> >> separated. Additionally we end up with only one type of expressions:
> >> A expression is a structure which has a tag, and a variable list of
> >> children. You define what what a valid expression is by extending the
> >> phoenix_algorithm template through specialisation for your tag. The
> >> Actions parameter is responsible for evaluating the expression. By
> >> template parametrisation of this parameter we allow users to easily
> >> define their own evaluation schemes without worrying about the
> >> validity of the phoenix expression. This is fixed by the meta grammar
> >> class.
> > 
> > What Thomas said. We realized that for Phoenix to be extensible at the
> > lowest level, we'd need to document its intermediate form: the Proto
> > tree. That way folks have the option to use Proto transforms on it.
> > (There are higher-level customization points that don't expose Proto,
> > but I'm talking about real gear-heads here.)
> > 
> > There were ugly things about the intermediate form we wanted to clean
> > up before we document it. That started the discussion. Then the
> > discussion turned to, "Can a user just change a semantic actions here
> > and there without having to redefine the whole Phoenix grammar in
> > Proto, which is totally non-trivial?" I forget offhand what the use
> > case was, but it seemed a reasonable thing to want to do in general.
> > So as Thomas says, the goal is two-fold: (a) a clean-up of the
> > intermediate form ahead of its documentation, and (b) a way to easily
> > plug in user-defined semantic actions without changing the grammar.
> > 
> > I think these changes effect the way to define new Phoenix syntactic
> > constructs, so it's worth doing a before-and-after comparison of the
> > extensibility mechanisms. Thomas, can you send around such a
> > comparison? How hard is it to add a new statement, for instance?
> 
> Yes, exactly, that's what I want. Anyway, while I'd still want to see
> this, I looked at the code and I like it, except for some nits here
> and there (especially naming). More on that later.

Yep, naming was one of the major problems i had ;)

I don't have much time now. But here is a side by side comparision of the 
old and new design:
http://www.scribd.com/doc/39713335/Design-ion

When looking at it, please keep in mind, that with the new design, we can 
reuse for_expr in a proto grammar. Additionally, we can reuse it in any 
extendable Actions class. While in the old design, everything is wrapped
behind a proto::function<funcwrap<for_eval>, ....>. Reusing it, for example
for generating debug output, is not directly supported by the old design. 
While in the new design, we have a unified, documented way on how to interact 
with out phoenix expression AST. Additionally, keep in mind, that in the old 
design, everything basically was a phoenix expression. The grammar wasn't 
really definining what can be used and what not. While in the new design, we 
explicitly say: here is the expression, this is how we want to use it.

Sorry, I am in kind of a hurry right now.
Hope that helped this far.

Thomas