Preliminary submission: Boost.Proto DSEL Framework
As discussed here recently, Boost.Proto is an expression template (ET) framework for building domain-specific embedded languages (DSEL). The purpose of this preliminary submission is to solicit feedback prior to a full submission (and to buy myself a little time to polish the docs/tests/examples :-). << Marketing Blurb >> Boost.Proto eases the development of domain-specific embedded languages (DSELs). Use Proto to define the primitives of your mini-language and let Proto handle the operator overloading and the construction of the expression parse tree. Immediately evaluate the expression tree by passing it a function object. Or transform the expression tree by defining the meta-grammar of your mini-language, decorated with an assortment of tree transforms provided by Proto or defined by you. Then use the meta-grammar to give your users short and readable syntax errors for invalid expressions! An expression template library developed with Proto is declarative and readable. In short, Proto is a DSEL for defining DSELs. << Hello, World >> Here's the required hello world example in proto. It builds an expression template and evaluates it. (The docs have more interesting examples.) #include <iostream> #include <boost/xpressive/proto/proto.hpp> #include <boost/xpressive/proto/context.hpp> #include <boost/typeof/std/ostream.hpp> using namespace boost; proto::terminal< std::ostream & >::type cout_ = { std::cout }; template< typename Expr > void evaluate( Expr const & expr ) { proto::default_context ctx; proto::eval(expr, ctx); } int main() { evaluate( cout_ << "hello" << ',' << " world" ); return 0; } << Documentation (such as it is) >> http://boost-sandbox.sourceforge.net/libs/proto/doc/html/index.html << Download >> proto.zip at http://boost-consulting.com/vault/index.php?&directory=Template%20Metaprogra... << Requirements >> Tested with Boost RC 1.34 and HEAD. Feedback is most welcome. -- Eric Niebler Boost Consulting www.boost-consulting.com
On 04/04/2007 02:18 AM, Eric Niebler wrote:
As discussed here recently, Boost.Proto is an expression template (ET) framework for building domain-specific embedded languages (DSEL). The purpose of this preliminary submission is to solicit feedback prior to a full submission (and to buy myself a little time to polish the docs/tests/examples :-). [snip]
<< Documentation (such as it is) >>
http://boost-sandbox.sourceforge.net/libs/proto/doc/html/index.html
On page: http://boost-sandbox.sourceforge.net/libs/proto/doc/html/boost_proto/user_s_... the sentence: Notice that in order to build a Proto expression template, at least one argument in the expression must be a Proto-ified. In this case, that is done with the terminal<> class template, which is used to wrap std::cout. was, at first, unclear to me because evaluate just had one argument; however, "one argument in expression" suggests there could be more than one. The following would be clearer to me: Notice that in order to build a Proto expression template, at least one subexpression of the whole expression must be a Proto-ified. In the case of: cout_ << "hello" << ',' << " world" this one Proto-ified subexpression is cout_, which is the Proto-ification of std::cout via wrapping by the terminal<> class template. I hope that's right. If not, then I still don't know what the sentence means :( [snip]
Larry Evans wrote:
On page:
http://boost-sandbox.sourceforge.net/libs/proto/doc/html/boost_proto/user_s_...
the sentence:
Notice that in order to build a Proto expression template, at least one argument in the expression must be a Proto-ified. In this case, that is done with the terminal<> class template, which is used to wrap std::cout.
was, at first, unclear to me because evaluate just had one argument; however, "one argument in expression" suggests there could be more than one.
The following would be clearer to me:
Notice that in order to build a Proto expression template, at least one subexpression of the whole expression must be a Proto-ified. In the case of:
cout_ << "hello" << ',' << " world"
this one Proto-ified subexpression is cout_, which is the Proto-ification of std::cout via wrapping by the terminal<> class template.
I hope that's right. If not, then I still don't know what the sentence means :(
Yeesh, I was being very imprecise. Your's is an improvement, but it's still not completely accurate. It needs to say that for any /operator/ in an expression, at least one operand must be proto-ified. Otherwise, proto's operator overloads will not be found. -- Eric Niebler Boost Consulting www.boost-consulting.com
On 04/04/2007 01:20 PM, Eric Niebler wrote:
Larry Evans wrote: [snip]
Notice that in order to build a Proto expression template, at least one subexpression of the whole expression must be a Proto-ified.
[snip] Yeesh, I was being very imprecise. Your's is an improvement, but it's still not completely accurate. It needs to say that for any /operator/ in an expression, at least one operand must be proto-ified. Otherwise, proto's operator overloads will not be found.
My understanding of "subexpression" is any operand of an operator. If that definition of subexpression is used, then I think were saying the same thing.
On 04/04/2007 02:18 AM, Eric Niebler wrote:
As discussed here recently, Boost.Proto is an expression template (ET) framework for building domain-specific embedded languages (DSEL). The purpose of this preliminary submission is to solicit feedback prior to a full submission (and to buy myself a little time to polish the docs/tests/examples :-).
Proto uses or_ and and_ for alternatives and sequences. It seems that alt_ and seq_ would be closer to the spirit names. Any reason why those names were preferred over alt_ and seq_?
Larry Evans wrote:
On 04/04/2007 02:18 AM, Eric Niebler wrote:
As discussed here recently, Boost.Proto is an expression template (ET) framework for building domain-specific embedded languages (DSEL). The purpose of this preliminary submission is to solicit feedback prior to a full submission (and to buy myself a little time to polish the docs/tests/examples :-).
Proto uses or_ and and_ for alternatives and sequences. It seems that alt_ and seq_ would be closer to the spirit names. Any reason why those names were preferred over alt_ and seq_?
Because for what they do, or_ and and_ are the right names. :-) You realize that these do not create expression templates, right? They are part of proto's meta-grammar facility. If you want to know if an expression type E matches grammar G1 or G2, you use: proto::matches<E, proto::or_<G1, G2> > That's a compile-time boolean test. Proto::and_ behaves similarly. Like mpl::or_ and mpl::and_ (and like || and &&) these do short-circuit evaluation. HTH, -- Eric Niebler Boost Consulting www.boost-consulting.com
On 04/04/2007 01:14 PM, Eric Niebler wrote:
Larry Evans wrote: [snip]
Proto uses or_ and and_ for alternatives and sequences. It seems that alt_ and seq_ would be closer to the spirit names. Any reason why those names were preferred over alt_ and seq_?
Because for what they do, or_ and and_ are the right names. :-) You realize that these do not create expression templates, right?
Yes, but I guess I need to reread about expression templates.
They are part of proto's meta-grammar facility. If you want to know if an expression type E matches grammar G1 or G2, you use:
proto::matches<E, proto::or_<G1, G2> >
That's a compile-time boolean test. Proto::and_ behaves similarly. Like mpl::or_ and mpl::and_ (and like || and &&) these do short-circuit evaluation.
HTH,
That sort of helps. I guess I was just predisposed to thinking of them as doing something similar to spirit's alternative<L,R> and sequence<L,R> templates; however, I guess not. Maybe my confusion is somehow related to the possiblity of have placeholder's as arguments to the templates. I understand that this is related to proto's ability to do transform of the template expressions, OTOH, spirit doesn't transform templates, only runtime values (via the parse member function). In short, I need to study proto some more :(
Larry Evans wrote:
On 04/04/2007 01:14 PM, Eric Niebler wrote:
They are part of proto's meta-grammar facility. If you want to know if an expression type E matches grammar G1 or G2, you use:
proto::matches<E, proto::or_<G1, G2> >
That's a compile-time boolean test. Proto::and_ behaves similarly. Like mpl::or_ and mpl::and_ (and like || and &&) these do short-circuit evaluation.
HTH,
That sort of helps. I guess I was just predisposed to thinking of them as doing something similar to spirit's alternative<L,R> and sequence<L,R> templates; however, I guess not. Maybe my confusion is somehow related to the possiblity of have placeholder's as arguments to the templates. I understand that this is related to proto's ability to do transform of the template expressions, OTOH, spirit doesn't transform templates, only runtime values (via the parse member function).
In short, I need to study proto some more :(
Proto has some novel ways of doing things, so your confusion is not surprising. But this is a good opportunity for me to dig into Proto's philosophy. In Spirit-1, alternate<> and sequence<> are containers. They *are* the expression template. They also have domain-specific behaviors (eg. parse() member functions) that make the expression template do that thing that Spirit does (parse). In proto, there is one container: expr<>. Proto's operator overloads glom smaller expr<>'s into larger ones, building a tree. By default, that tree has no behaviors (no parse() member function, for example).[*] So once you've built this tree, you'll need to do something with it. Proto gives you a couple of choices. One is proto::eval(), which lets you walk the tree and do domain-specific things at each node. Another choice is to transform the tree into another object that has the behaviors you want. For this, you write a domain-specific tree transformation by writing out your DSEL's grammar and attaching transforms to your grammar's rules. This is where proto::or_ and proto::and_ come in. Imagine in your DSEL, only terminals of type int and char* are allowed. Then you might define a Terminal rule in your grammar as follows: struct Terminal : proto::or_< proto::terminal<int> , proto::terminal<char*> > {}; When you write your tree transform, you might want to attach some transformation to the Terminal rule to, for example, turn char* into spirit::string_parser, or whatever. Proto's meta-grammar facilities are good for more than just tree transformations, by the way. You can use them with proto::matches<> to make compile-time decisions based on the structure of an expression. You can even use a proto meta-grammar to control proto's operator overloading! When you do that, only those operators which build valid expressions, as determined by your meta-grammar, are considered. [*] There is a way in proto to add domain-specific behaviors, like extra member functions, directly to the expression tree. It's called proto::extends<>. Just so you know. -- Eric Niebler Boost Consulting www.boost-consulting.com
On 04/04/2007 02:49 PM, Eric Niebler wrote: [snip]
behaviors you want. For this, you write a domain-specific tree transformation by writing out your DSEL's grammar and attaching transforms to your grammar's rules.
This is where proto::or_ and proto::and_ come in. Imagine in your DSEL, only terminals of type int and char* are allowed. Then you might define a Terminal rule in your grammar as follows:
struct Terminal : proto::or_< proto::terminal<int> , proto::terminal<char*> > {};
When you write your tree transform, you might want to attach some transformation to the Terminal rule to, for example, turn char* into spirit::string_parser, or whatever.
I'm still unclear about and_. I've looked at matches.hpp where both or_ and and_ are defined and it looks like, as their names suggest, that they are predicates of some sort. Now I can understand how or_ works as you described above for defining a grammar's terminals; however, I'm unclear about and_. Is an and_ instantiation a possible 2nd arg to matches and, in this case, matches tests it's first argument to see that it satisfies both arguments to the and_?
Proto's meta-grammar facilities are good for more than just tree transformations, by the way. You can use them with proto::matches<> to make compile-time decisions based on the structure of an expression.
So matches<Expr,Grammar> tests Expr to see if it matches the meta-grammar, Grammar? IOW, and_<X,Y> and or_<X,Y> are possible values of Grammar in matches<Expr,Grammar>?
Larry Evans wrote:
On 04/04/2007 02:49 PM, Eric Niebler wrote: [snip]
behaviors you want. For this, you write a domain-specific tree transformation by writing out your DSEL's grammar and attaching transforms to your grammar's rules.
This is where proto::or_ and proto::and_ come in. Imagine in your DSEL, only terminals of type int and char* are allowed. Then you might define a Terminal rule in your grammar as follows:
struct Terminal : proto::or_< proto::terminal<int> , proto::terminal<char*> > {};
When you write your tree transform, you might want to attach some transformation to the Terminal rule to, for example, turn char* into spirit::string_parser, or whatever.
I'm still unclear about and_. I've looked at matches.hpp where both or_ and and_ are defined and it looks like, as their names suggest, that they are predicates of some sort. Now I can understand how or_ works as you described above for defining a grammar's terminals; however, I'm unclear about and_. Is an and_ instantiation a possible 2nd arg to matches and, in this case, matches tests it's first argument to see that it satisfies both arguments to the and_?
That's correct. and_ isn't as commonly needed as or_, but it's still handy. There is also if_, which evaluates an mpl lambda. So, for instance, here's a rule in a meta-grammar that matches scalar terminals: struct ScalarTerminal : proto::and_< proto::terminal<proto::_> , proto::if_<is_scalar<proto::result_of::arg<mpl::_> > > > {};
Proto's meta-grammar facilities are good for more than just tree transformations, by the way. You can use them with proto::matches<> to make compile-time decisions based on the structure of an expression.
So matches<Expr,Grammar> tests Expr to see if it matches the meta-grammar, Grammar? IOW, and_<X,Y> and or_<X,Y> are possible values of Grammar in matches<Expr,Grammar>?
Yes. And types that inherit from them, like ScalarTerminal above. -- Eric Niebler Boost Consulting www.boost-consulting.com
On 04/04/2007 02:18 AM, Eric Niebler wrote:
As discussed here recently, Boost.Proto is an expression template (ET) framework for building domain-specific embedded languages (DSEL). The
[snip]
Feedback is most welcome.
proto/libs/xpressive/proto/doc/html/boost_proto/user_s_guide/calculator.html contains: The expr<> template makes up the nodes in expression trees. The first template parameter is the node type; in this case, proto::tag::terminal. That means that _1 is a leaf-node in the expression tree. The second template parameter is a list of children types. Terminals will always have only one type in the type list. The last parameter is the number of types in the list. It is deduced from the type list, so you won't need to worry about it. which suggests a leaf-node can have children. I thought leaf-nodes had no children. What am I missing? OOPS. Maybe a leaf-node, as you're defining it can have no other proto-expressions as children but it can have other types of children. Is that right?
Larry Evans wrote:
On 04/04/2007 02:18 AM, Eric Niebler wrote:
As discussed here recently, Boost.Proto is an expression template (ET) framework for building domain-specific embedded languages (DSEL). The
[snip]
Feedback is most welcome.
proto/libs/xpressive/proto/doc/html/boost_proto/user_s_guide/calculator.html
contains:
The expr<> template makes up the nodes in expression trees. The first template parameter is the node type; in this case, proto::tag::terminal. That means that _1 is a leaf-node in the expression tree. The second template parameter is a list of children types. Terminals will always have only one type in the type list. The last parameter is the number of types in the list. It is deduced from the type list, so you won't need to worry about it.
which suggests a leaf-node can have children. I thought leaf-nodes had no children. What am I missing? OOPS. Maybe a leaf-node, as you're defining it can have no other proto-expressions as children but it can have other types of children. Is that right?
That's right, expr<tag::terminal, ...> is special in that it is the only node type that is allowed to have a non-expr<> as a child. Even though leaf nodes in a tree are special, it simplifies much code to reuse the expr<> type this way, rather than to define a separate terminal<> template. -- Eric Niebler Boost Consulting www.boost-consulting.com
Hi, I have looked at the examples and they all look very readable. I am now missing a facility to make more extended checks. I am not sure if I completly understood the syntax of the proto dsl. Here is a small matrix-vector example: struct RowVector : proto::or_< proto::terminal< row_vector<_> > , proto::terminal< dyn_row_vector > >{}; struct ColumnVector : proto::or_< proto::terminal< col_vector<_> > , proto::terminal< dyn_col_vector > >{}; struct MatrixExpr : proto::or_< proto::multiply<ColumnVector,RowVector>
{};
struct ScalarExpr : proto::or_< proto::terminal< float> , proto::multiply<RowVector,ColumnVector>
{};
In this case col_vector and row_vector could be like boost_array, with a compile time dimension, while dyn_col_vector and dyn_row_vector behave more like std::vector. Now I would like to forbid the mulitplication operator at compile time if both dimensions are compile time ones, and do not match. In all other cases the check has to be executed during runtime. How can I do that with proto? Regards Andreas Pokorny
Andreas Pokorny wrote:
Hi,
I have looked at the examples and they all look very readable. I am now missing a facility to make more extended checks.
I am not sure if I completly understood the syntax of the proto dsl. Here is a small matrix-vector example:
struct RowVector : proto::or_< proto::terminal< row_vector<_> > , proto::terminal< dyn_row_vector > >{};
struct ColumnVector : proto::or_< proto::terminal< col_vector<_> > , proto::terminal< dyn_col_vector > >{};
struct MatrixExpr : proto::or_< proto::multiply<ColumnVector,RowVector>
{};
struct ScalarExpr : proto::or_< proto::terminal< float> , proto::multiply<RowVector,ColumnVector>
{};
In this case col_vector and row_vector could be like boost_array, with a compile time dimension, while dyn_col_vector and dyn_row_vector behave more like std::vector.
Now I would like to forbid the mulitplication operator at compile time if both dimensions are compile time ones, and do not match. In all other cases the check has to be executed during runtime.
How can I do that with proto?
I would create a predicate like dim_equal_to, like this: template<typename A, typename B> struct dim_equal_to : mpl::or_< mpl::equal_to<A,B> , mpl::equal_to<A,long_<-1> > , mpl::equal_to<B,long_<-1> > > {}; template<typename T> struct dim_of; template<typename Dim> // Assumes Dim is mpl::long_<N> struct dim_of<row_vector<Dim> > : Dim {}; template<typename Dim> // Assumes Dim is mpl::long_<N> struct dim_of<col_vector<Dim> > : Dim {}; template<typename T> struct dim_of<dyn_row_vector> : long_<-1> {}; template<typename T> struct dim_of<dyn_col_vector> : long_<-1> {}; And I would change MatrixExpr to look something like (untested!): struct MatrixExpr : proto::and_< proto::multiply<ColumnVector,RowVector> , proto::if_< dim_equal_to< dim_of< proto::result_of::arg<proto::result_of::left<mpl::_> > > , dim_of< proto::result_of::arg<proto::result_of::right<mpl::_> > > > > > {}; ScalarExpr would get a similar treatment. Some observations. In your code, you have:
struct MatrixExpr : proto::or_< proto::multiply<ColumnVector,RowVector>
{};
In this case, the proto::or_ is superfluous, and can just be removed. And in this expression:
proto::terminal< row_vector<_> >
note that the _ is proto::_, not mpl::_, and that it will only match types, not integers. Hence the comment above about assuming that Dim is mpl::long_<N>. HTH, (I'm leaving today for a week-long vacation. I'll be on email only off and on. Sorry 'bout that.) -- Eric Niebler Boost Consulting www.boost-consulting.com
On 04/04/2007 02:18 AM, Eric Niebler wrote: [snip]
<< Documentation (such as it is) >>
http://boost-sandbox.sourceforge.net/libs/proto/doc/html/index.html
[snip] On: libs/xpressive/proto/doc/html/boost_proto/user_s_guide/pattern_matching_and_meta_grammars.html under: Other Valid Patterns an argument to and_ is called an "alternate". Shouldn't that term just be used for the arguments to or_? Maybe the phrase: each argument of the and_ must match in order for the and_ to match would be clearer.
Larry Evans wrote:
On:
libs/xpressive/proto/doc/html/boost_proto/user_s_guide/pattern_matching_and_meta_grammars.html
under:
Other Valid Patterns
an argument to and_ is called an "alternate". Shouldn't that term just be used for the arguments to or_? Maybe the phrase:
each argument of the and_ must match in order for the and_ to match
would be clearer.
Yep, thanks. (I'm leaving today for a week-long vacation. I'll be on email only off and on. Sorry 'bout that.) -- Eric Niebler Boost Consulting www.boost-consulting.com
On 04/04/2007 02:18 AM, Eric Niebler wrote:
<< Requirements >>
Tested with Boost RC 1.34 and HEAD.
Feedback is most welcome.
This just a headsup. With the attached, I get: gcc.compile.c++ ../../../../../../bin.v2/sandbox/lje/libs/xpressive/proto/example/gcc-4.1/debug/class_var_incomplete_test.o ../../../../../proto/boost/xpressive/proto/expr.hpp: In instantiation of 'boost::proto::expr<boost::proto::tag::terminal, boost::proto::args1<inp<inp_0> >, 1l>': class_var_incomplete_test.cpp:49: instantiated from 'inp<inp_0>' class_var_incomplete_test.cpp:63: instantiated from here ../../../../../proto/boost/xpressive/proto/expr.hpp:105: error: 'boost::proto::expr<Tag, Args, 1l>::arg0' has incomplete type class_var_incomplete_test.cpp:39: error: declaration of 'struct inp<inp_0>' "/usr/bin/g++-4.1" -ftemplate-depth-128 -O0 -fno-inline -Wall -g -fPIC -DBOOST_ALL_NO_LIB=1 -I"../../../.." -I"../../../../../.." -I"../../../../../proto" -c -o "../../../../../../bin.v2/sandbox/lje/libs/xpressive/proto/example/gcc-4.1/debug/class_var_incomplete_test.o" "class_var_incomplete_test.cpp" What's odd is if proto::terminal is *not* used, it compiles OK *or* if non-nested inp_0_val is declared, it compiles OK.
Larry Evans wrote:
On 04/04/2007 02:18 AM, Eric Niebler wrote:
<< Requirements >>
Tested with Boost RC 1.34 and HEAD.
Feedback is most welcome.
This just a headsup. With the attached, I get:
gcc.compile.c++ ../../../../../../bin.v2/sandbox/lje/libs/xpressive/proto/example/gcc-4.1/debug/class_var_incomplete_test.o
I don't think this is a bug in proto. I get the same error with: #include <utility> struct inp { typedef std::pair<inp, inp> type; static const type c; }; int main() { void const *p = &(inp::c); return p != 0; } (Tested with msvc-8.0.) -- Eric Niebler Boost Consulting www.boost-consulting.com
On 04/17/2007 02:36 PM, Eric Niebler wrote: [snip]
I don't think this is a bug in proto. I get the same error with:
#include <utility>
struct inp { typedef std::pair<inp, inp> type; static const type c; };
int main() { void const *p = &(inp::c); return p != 0; }
(Tested with msvc-8.0.)
Maybe msvc-8.0 shouldn't get the same error. I tried that code with g++ and got no error. However, I did try the attached, which contains just the essense of the proto code, and reproduced the error with g++ but not with como. So, I'm thinking g++ has a bug.
participants (3)
-
Andreas Pokorny -
Eric Niebler -
Larry Evans