|
|
Boost : |
From: Eric Niebler (eric_at_[hidden])
Date: 2008-03-28 16:56:40
David Abrahams wrote:
>>> * What happens if your type has a generalized operator? ::
>>>
>>> namespace fu
>>> {
>>> struct zero {};
>>> #if 1
>>> template <class T> T operator+(T x,zero) { return x; }
>>> #else
>>> double operator+(double x,zero) { return x; }
>>> #endif
>>> }
>>>
>>> int main()
>>> {
>>> // Define a calculator context, where _1 is 45 and _2 is 50
>>> calculator_context ctx( 45, 50 );
>>>
>>> // Create an arithmetic expression and immediately evaluate it
>>> double d = proto::eval( (_2 - _1) / _2 * 100 + fu::zero(), ctx );
>>>
>>> // This prints "10"
>>> std::cout << d << std::endl;
>>> }
>>>
>>> Answer: a nasty error message (at least on g++). Anything we can
>>> do to improve it (just wondering)?
>>
>> It's similar to what happens in e.g. a linear algebra domain where
>> vector terminals want to define += that actually does work as opposed to
>> build expression trees. In that case, you'll need to disable proto's
>> operator overloads with a grammar. Otherwise, the operators are ambiguous.
>>
>> I'm can't think of anything better.
>
> In this case what I really wanted was to disable the existing operator.
Proto's operator? The way to do that is with a grammar. 'Course I don't
show that until the very end.
> Think of zero as a non-lazy type like double, that you might want to use
> in a lazy context.
>
> Anyway, I was only aiming at "can we improve the error message," rather
> than, "can we actually make this work," although I can imagine some
> approaches to the latter.
I'm curious what you have in mind here.
>>> * is there a reason we need ``ref_`` as opposed to using true
>>> references? (just curious, but the docs don't answer this
>>> question).
>>
>> It's not strictly necessary, and in branches/proto/v3 there's a version
>> of proto that uses true references. I found it complicated the the
>> implementation and causes a bunch of unnecessary remove_reference<>
>> instantiations.
>
> If the existing implementation isn't too complicated, you could always
> add an internal "convert T& to ref_<T>" step, just to keep ref_ out of
> the users' face.
You mean, just to keep ref_ out of the type of an expression? It would
add some code complexity in some places (e.g., given expr<Tag,Args,N>, I
wouldn't be able to simply get the tag type of the 0th child as
Args::arg0::proto_tag), but it might be worth it. ref_ really is a
detail that shouldn't show up in error messages and in debuggers. It
also contributes to debug symbol name length.
>> I need to document the naming idioms. proto::foo() is a function,
>> proto::result_of::foo is a metafunction that calculates the return type,
>> proto::functional::foo is the equivalent function object, and (where
>> relevant) proto::transform::foo is the primitive transform, and
>> proto::_foo is an alias for proto::transform::foo.
>>
>> Now that you know, do you have any suggestions for improvements?
>
> Hmm, that's hard to keep track of. If you dopped the alias it would be
> a little simpler. Maybe s/functional/function_obj/ or even
> s/functional/object/...
Which alias would you drop? If I had to drop one, I'd drop everything in
the transform namespace, and just use _foo. If I had to type
transform::childN everywhere, I think I'd shoot myself.
I don't like "object" ... it carries no semantic weight. How about
s/functional/functor/?
> wait, isn't proto::foo in the above case actually an instance of
> proto::functional::foo?
It shouldn't be, no.
>>> * http://boost-sandbox.sourceforge.net/libs/proto/doc/html/boost_proto/users_guide/expression_construction/tags_and_meta_functions.html
>> ...
>>> - What purpose do the <>s serve in the table if you're not going
>>> to show the metafunction signature?
>> Elsewhere in the docs, I've used "foo()" to emphasize that foo is a
>> function, and "foo<>" to emphasize that foo is a template. Not sure if
>> it's a worthwhile convention.
>
> I think as long as you're going to put the <>s in, it would help a lot
> to remind people how the thing is used, so I would add arguments.
Just in this table, or everywhere?
>>> - Does it use the same tag and metafunction no matter the arity
>>> of a function call?
>> Not sure what you mean. What function call?
>
> I'm talking about the tag::function row.
Oh, yes. The function<> grammar/metafunction is variadic. So is
nary_expr<>. Those are the only two.
>>> - What namespaces are the metafunctions in?
>> boost::proto
>
> My point was that there's no guide at the beginning that tells me how to
> interpret such names. I would prefer in most cases to see everything
> written as though
>
> namespace proto = boost::proto;
>
> were in effect, so there would be a lot more qualification.
OK.
>>> - The ``DomainOrArg`` argument to ``result_of::make_expr`` is
>>> confusing. I don't see a correponding argument to the function
>>> object. I might not have been confused by this except that you
>>> seem to use that argument in the very next example.
>> The make_expr function object does have an optional Domain template
>> parameter:
>>
>> template<typename Tag, typename Domain = default_domain>
>> struct make_expr : callable
>>
>> What I'm not showing is an overload of the proto::make_expr() free
>> function that doesn't require you to specify the domain.
>
> Can you do anything to clear up the confusion?
I think for the users' guide, I should describe the usages and leave the
signatures for the reference. That should make this clearer.
>>> - I don't know how well or badly it would work with the rest of
>>> the library, but I'm thinkin' in cases like this one it might
>>> be possible to save the user some redundant typing::
>>>
>>> // One terminal held by reference:
>>> int i = 0;
>>>
>>> typedef
>>> proto::result_of::make_expr<
>>> MyTag
>>> , int & // <-- Note reference here
>>> , char
>>> >::type
>>> expr_type;
>>>
>>> expr_type expr = proto::make_expr<MyTag>(boost::ref(i), 'a');
>>>
>>> I'm thinking the result of ``proto::make_expr<...>(...)`` could hold
>>> everything by reference, but the type of
>>> ``proto::result_of::make_expr< ... >`` would hold things by the
>>> specified type.
>> And rely on an implicit conversion between expression types?
>
> Yes.
>
>> I've tried to avoid that. Figuring what is convertible to what can be
>> expensive, and it's hard to know whether you are paying that cost in
>> your code or not because the conversions are implicit.
>
> I don't understand, sorry. This doesn't look like something that
> requires any type checking that wouldn't happen anyway.
If I understand what you're suggesting, each expr<T,Args,N> would have
the following conversion operator:
template<typename Args2>
operator expr<T,Args2,N>() const
{
expr<T,Args2,N> that = {
this->arg0
, this->arg1
, ...
};
return that;
}
Is that right? This is recursive, because this->arg0 might be an expr<>
that needs to be implicitly converted, etc. This makes me uncomfortable.
Whenever I return an expr<>, I don't know whether an implicit
conversion is happening, how expensive it might be at runtime, or how
many unnecessary templates are instantiated at compile time. I can't
search my code for these conversions, either -- they're invisible. My
only recourse is to comment out the conversion operator and see what breaks.
My gut tells me to leave it out.
>>> - Is all this time spent on ``make_expr`` really appropriate at
>>> this early stage of the tutorial? Seems to me we *ought* to be
>>> able to do a lot of more sophisticated things with the library
>>> before we start into the nitty-gritty of building expressions
>>> explicitly (i.e. framework details). No?
>> You're probably right. Currently, the users' guide is neatly divided
>> into 5 sections: construction, evaluation, introspection, transformation
>> and extension.
>
> That sounds like an excellent structure for a reference section :-)
Interesting suggestion.
>> That means I have to exhaustively cover construction
>> before I get to anything else -- even make_expr() which is rather
>> esoteric. I suppose I should rethink the overall structure. Suggestions
>> welcome.
>
> Walk us through practical examples in increasing order of
> sophistication, showing the most useful features first and the more
> esoteric ones laater.
Sure.
>>> - Is this the first time you're showing us how to build a simple
>>> lazy function? That should come *much, much* earlier.
>> Not everything can come first!
>
> Does it sound like I'm saying that about everything?
Should have put a winky face there .. it was tongue-in-cheek.
>>> * what tells the library to substitute the function's template
>>> parameter there?
>> Not sure I understand the question.
>
> What is the underlying rule that says how this works? Does the library
> substitute the function's template parameter into all first arguments,
> or what?
I'll clarify.
>>> * http://boost-sandbox.sourceforge.net/libs/proto/doc/html/boost_proto/users_guide/expression_evaluation/proto_eval.html
...
>> You were misled by the fact that proto::eval is an instance of
>> proto::functional::eval. It shouldn't be. Better to make it a free
>> function like the others. No reason why it shouldn't be find-able with
>> ADL.
>
> OK. Well, the larger point is that the idioms need to be explained up
> front so people don't have to create a series of wrong hypotheses like I
> did before they understand the patterns you're using.
Right.
>>> * http://boost-sandbox.sourceforge.net/libs/proto/doc/html/boost_proto/users_guide/expression_evaluation/canned_contexts.html
>> ...
>>> - That said, a really good motivating case for using matches<>
>>> seems to be missing.
>> I should talk about how it can be used to improve error messages by
>> validating expressions at API boundaries.
>
> Yep. In general I've found enable_if's impact on error messages to be
> disappointing (a long list of the overloads that didn't match, with
> little explanation). It usually turns out to be better to match with an
> overload that generates a specific error message.
Agreed. This, and static function dispatching are the primary use cases
for matches<>.
>>> * http://boost-sandbox.sourceforge.net/libs/proto/doc/html/boost_proto/users_guide/expression_introspection/defining_dsel_grammars.html
>> ...
>
> To start with, there's one very basic thing we won't be able to do: name
> template specializations that cannot be instantiated, like
>
> vector<foo(bar)>
>
> just for DSEL purposes.
Ugh! I suppose this would work around the issue:
vector<call<foo(bar)>>
as long as call<> happened to be Regular.
>>> - ``when< grammar, transform >`` seems like it would be better
>>> named ``replace< grammar, transform >`` or something.
>> Why do you say that?
>
> My understanding was that tree transformations might match parts of the
> tree and replace them with transformed versions, while leaving other
> subtrees untouched.
To make an analogy with the runtime world, I don't think of a function
call as "replacing" its arguments with the function's return value.
> I'm not sure I believe that there's a better name than "when," but this
> shows you how I was thinking about what you wrote, anyway.
I once called it "case_", but then there was an impedance mismatch with
the switch_ grammar element. I'm still happiest with "when".
>> Visitor is just a blob of mutable data, whatever you want. The type of
>> the visitor usually doesn't change during the transformation. None of
>> proto's built in transforms touch it in any way --- it is passed through
>> unchanged.
>
> Oh, oh... please don't call that a visitor then! It's just auxilliary
> data; you might call it "data" or "aux". When I see "visitor" I
> automatically think of an object with associated operations that are
> applied at each traversed element of a structure.
Good point. (The name "visitor" in Proto is historical ... in xpressive,
it really was a visitor.) The Haskell guys call this a "context", but
for obvious reasons I don't want to call it that. "data" or "aux" would
work.
>>> - I think ::
>>>
>>> when< unary_expr< _, CalcArity >, CalcArity(_arg) >
>>>
>>> should be spelled ::
>>>
>>> when< unary_expr< _, CalcArity >, CalcArity(_arg(_)) >
>> CalcArity(_arg) and CalcArity(_arg(_)) are synonyms today. Do you feel
>> that the (_) should be required? (_) is optional because _arg is a
>> so-called primitive transform, for which expr, state, and visitor can be
>> implicit. It's not just syntactic sugar -- it's less work for Proto.
>
> In that case, no, I don't feel it should be required. However, I think
> this library is hard enough to grok that consistency of notation should
> be a primary goal in the user guide, so I would use the more consistent
> spelling. Can you get most of the efficiency back with a single
> template specialization on <_arg(_)> ? That's often the case in such
> situations.
Sort of. I would specialize when<X,Y(_)>, but I can't make an assumption
here about whether Y is callable or not. It would still be an
improvement, tho.
>>> - I think you're missing a good example of why you'd do two
>>> different evaluations on the same expr. A classic might be
>>> regular expression evaluation and symbolic differentiation.
>>> E.g., ::
>>>
>>> x = eval(_1 * _1, normal(7)); // 49
>>> y = eval(_1 * _1, differentiate(7)); // 14
>> Symbolic differentiation?
>
> d(x*x)/dx = 2x
Duh. Of course. Good suggestion.
>>> * http://boost-sandbox.sourceforge.net/libs/proto/doc/html/boost_proto/users_guide/expression_transformation/canned_transforms.html
>> ...
>>> - "It can be used without needing to explicitly specify any
>>> arguments to the transform." Huh? What kind of arguments?
>> This is why _arg is a synonym for _arg(_). In this case, "_" is an
>> argument to the transform.
>
> Okay. I tend to think of placeholders as not being actual arguments.
I'm grasping for the right terminology here.
>> Because _arg is "callable", you can leave off
>> _ and also _state and _visitor. They are implicit.
>
> The point being that the language confused me. Do what you will with
> that fact; clarifying 50% of the things I complain about may make the
> other 50% understandable to me.
I don't yet know how best to talk about these things. An improvement
would be to write a glossary of terms, use terms consistently, and refer
people to the glossary frequently.
>>> 2. Can we replace ``_make_terminal(_arg)`` above with
>>> ``_expr``? If not, why not?
>> No, _expr is analogous to lambda::_1 ... whatever the first argument is,
>> return that. (_state is analogous to lambda::_2 and _visitor is
>> analogous to lambda::_3 --
>
> Oooooh! Please, a statement up front about that!
That would help people familiar with Lambda, but might just confuse
people who don't. I can say the same thing without referring to Lambda, tho.
>> they return the state and visitor
>> parameters). So _expr(_arg) would be the same as _arg. It wouldn't
>> create a new expression node, which is what _make_terminal(_arg) does.
>>
>> _make_terminal is a typedef for functional::make_expr<tag::terminal>. I
>> don't think I say that anywhere, though. :-/
>
> There's a lot of that going around ;-)
Yeah, documentation is hard.
>>> * http://boost-sandbox.sourceforge.net/libs/proto/doc/html/boost_proto/users_guide/expression_transformation/canned_transforms/and_or_not.html
>> ...
>>> - Does ``and_< T0,T1, ... Tn >`` *really* only apply ``T``\ *n*?
>> Yes. It can't apply all the transforms. The result of applying the first
>> transform might have a type that the second transform can't make sense
>> of. Transforms don't chain that way.
>
> They could be chained by and_, could they not?
No, not in any meaningful way. Consider:
and_<
when<G1, T1>
,when<G2, T2>
>
T1 requires that any expression passed to it matches G1. Likewise for T2
and G2. Now, imagine that the result of applying T1 is some type that
*doesn't* match G2 ... it might not even be an expression! If you tried
to pass it to T2, you would be violating T2's preconditions. T2's
transform will likely fail.
>> The default transform for and_ is admittedly a bit arbitrary and not
>> terribly useful. But you can always override it with proto::when
>
> I'm lost. Why override it when you don't have to use the arbitrary and
> not-terribly-useful thing in the first place?
Is it clearer now?
>>> - wow, you totally lost me on this page. I can't understand why
>>> the stated behaviors of these transforms make sense
>>> (e.g. correspond to their names), and I can't understand why
>>> the usage of ``and_`` in ``UnwrapReference`` is an example of a
>>> transform and not a grammar element. The outer ``or_`` is a
>>> grammar element, right? When ``or_`` is used as a grammar
>>> element, aren't its arguments considered grammar elements also?
>>
>> and_, or_ and not_ are both grammar elements and transforms.
>
> Yes, I'm aware of that duality. My understanding is that how they are
> treated depends on context. My point is that in the UnwrapReference
> example, and_ is treated as a grammar element and not as a transform.
It is treated as *both* a grammar element and a transform. It's easier
to see with or_
struct UnwrapReference
: or_<
// ... stuff
>
{};
Here, or_ is very clearly being used as a grammar element. But when you
apply UnwrapReference's transform, you're actually applying or_'s
transform, because UnwrapReference inherits its transform from or_.
The case of and_ is the same:
struct UnwrapReference
: or_<
// Pass through terminals that are not
// reference_wrappers unchanged:
and_<
terminal<_>
, not_<if_<is_reference_wrapper<_arg>()> >
>
// ... other stuff
>
{};
or_'s transform is simply to apply the transform associated with
whichever subgrammar matched. So if the and_ subgrammar matched, its
transform is applied. Hence, and_ is being used both as a grammar and as
a transform.
> That's doubly true because given its sub-elements, even the default
> transform associated with and_ has no interesting effects: whether it
> returned the result of the first, the last, or chained the transforms,
> you'd get the same answer.
In this case, yes. The only interesting thing being demonstrated here is
that and_ can be legally used where a transform is needed.
>>> * http://boost-sandbox.sourceforge.net/libs/proto/doc/html/boost_proto/users_guide/expression_transformation/canned_transforms/call.html
>> ...
>>> - This business of handling transforms specially when they can
>>> accept 3 arguments is hard to understand. Aside from the fact
>>> that special cases tend to make things harder to use, it's not
>>> clear what it's really doing or why. I guess you're saying
>>> that state and visitor are implicitly passed through when they
>>> can be?
>> Yes, and the expression, too.
>
> It would be nice to have a logical overview up front that describes the
> transformation as a uniform process of operating on these three values
> at every matched node in the tree, and that you have these three
> specially-named placeholders that pick up those values... unless their
> values are overridden by supplying a round lambda expression.
I think I see what you're getting at. I may follow up with you offline
at some point.
>>> I can understand why you'd want something like that,
>>> but let's look at this more closely:
>>>
>>> "For callable transforms that take 0, 1, or 2 arguments,
>>> special handling is done to see if the transform actually
>>> expects 3 arguments..."
>>>
>>> Do you really mean, "for callable transforms that are *passed*
>>> 0, 1, or 2 arguments...?" Or maybe it's something more
>>> complicated, like "for callable transforms that *are written as
>>> though* they take 0, 1, or 2 arguments...?"
>> Yes, the latter.
>
> OK, please clarify the language. My point is that I had to work all
> this stuff out for myself by writing about it.
Sure.
>> The following transform are synonyms:
>>
>> _arg
>> _arg()
>> _arg(_)
>> _arg(_,_state)
>> _arg(_,_state,_visitor)
>>
>> That is true not just for _arg but for any primitive transform. And for
>> completeness (or just to make it more confusing?) you can use _expr
>> instead of _ and it means the same thing here.
>>
>> As I say above, using _arg instead of _arg(_,_state,_visitor) is more
>> than just sugar. It's less work for Proto.
>
> Great; the picture is becoming clearer. Let's get that clarity into the
> documentation.
Yes, I can't imagine why I didn't just say this in the first place. :-/
>>> - Again the use of ``proto::callable`` without a prior
>>> explanation... oh! there it is, finally, in the footnote of the
>>> example! If you check for "callable with 3 args" using
>>> metaprogramming tricks anyway, why not do the same for
>>> "callable" in general?
>> Not sure I understand the questions. Getting a little bleary-eyed myself.
>
> The point is that, given everything you've written so far, at this point
> I wonder why I have to specialize is_callable (or derive a class from
> proto::callable) when you have (and use) a way to detect callability.
Ah. Those tricks I use to detect callability would cause types to
instantiate. That wouldn't work for std::vector<foo(bar)>.
>> Or the "call" transform can be renamed
>> "apply", and "function" can be "call".
>
> That might be better.
Elsewhere, I use names from <functional>, like "multiplies" and
"divides". So ... "calls"? Ick.
>> I don't care.
>
> me neither ;-)
:-P
>>> - Translating the example into terms I understand::
>>>
>>> make_pair(_arg(_arg1), _arg(_arg2))
>>>
>>> becomes ::
>>>
>>> make_pair(_value(_left(_)), _value(_right(_)))
>>>
>>> which looks a bit better to me.
>>
>> typedef _arg _value;
>>
>> You're good to go! :-)
>
> I realize that; I'm suggesting the 2nd way makes an easier-to-grasp
> presentation.
I'm of 2 minds about this. First, I want to document "best practices",
and if the first is better than the second, I don't want to be
encouraging the second.
The other issue is rather subtle: proto::_ is not a placeholder like
mpl::_ is. It's a transform. Ditto for _left and _right. It's important
for users to understand that they can write their own transforms and use
them instead. There's nothing magic about proto::_ and no good reason to
use it like this. _value(_left(_)) may *look* nice, but IMO it
encourages people to think about this in a limiting way.
>> Consider a transform such as:
>>
>> fusion::single_view< add_const<_> >( _ )
>>
>> Do you see now? Gotta look for the nested ::type in add_const *after*
>> the placeholder (er, nested transform) has been evaluated.
>
> Sure. What I mean is that the option to have no nested ::type is merely
> convenient, and not even all that reliable in most cases (how do you
> know your vector<>'s implementor didn't add a member ::type)?
Oh, I see. You're suggesting to *always* require a nested ::type. Sure.
But it's very convenient as-is.
> if you just used boost::result_of in the documentation, I think maybe
> you'd skirt the whole issue.
Good point.
>>> * http://boost-sandbox.sourceforge.net/libs/proto/doc/html/boost_proto/users_guide/expression_transformation/canned_transforms/pass_through.html
>>>
>>> The example doesn't even name pass_through directly. Can we do
>>> better?
>> I've never used pass_through explicitly, but I use it all the time
>> implicitly, as in this example.
>
> Then maybe it doesn't deserve so much space in the user guide.
But it's quite possibly the most important transform.
>>> * http://boost-sandbox.sourceforge.net/libs/proto/doc/html/boost_proto/users_guide/expression_extension/inhibiting_overloads.html
>> ...
>>> - It's not clear to me why you need all this fancy footwork to
>>> define a special meaning for ``operator[]``. Isn't that what
>>> contexts are for?
>> So for:
>>
>> ( v2 + v3 )[ 2 ];
>>
>> ... you're saying to let proto build the expression tree representing
>> the array access, and evaluate it lazily with a context. Seems reasonable.
>
> Does that undermine the whole example?
Oh, I remember now:
double d = (v2 + v3)[2];
Either op[] is special, or else there is an implicit conversion to
double. I opted for the former.
>>> - But I have to say, that use of the grammar to restrict the
>>> allowed operators is *way cool*. I just think it should have
>>> been shown *way earlier* ;-).
>> I can't show *all* the good bits first.
>
> I actually disagree, for some reasonable definition of "first."
>
>> Then there'd be no reason to read further! :-)
>
> It sounds a bit like you're saying it's your goal in the user guide to
> eventually teach the reader everything about the library, whether he wants
> to learn the details or not. I don't think you should be disappointed
> if she stops after she's learned to use the library powerfully but long
> before she learns many of the details she won't need.
I was being tongue-in-cheek again. :-) I'll use <tongue
location="cheek"/> tags in the future.
>>> * http://boost-sandbox.sourceforge.net/libs/proto/doc/html/boost_proto/users_guide/expression_extension/expression_generators.html
>>>
>>> - "After Proto has calculated a new expression type, it checks
>>> the domains of the children expressions. They must match."
>>>
>>> Does this impair DSEL interoperability?
>> It does. Thanks for bringing that issue up; I had forgotten about it.
>> It's an open design question. What is the domain of an expression such
>> as (A + B) where A and B are in different domains? There needs to be an
>> arbitration mechanism, but Proto doesn't have one yet. I don't know what
>> that would look like.
>
> This is important, because interoperability is one of the most
> powerful arguments for DSELs. I think there are a few cases, at least,
> where you can make some default decisions. For example A(B) and A[B]
> both ought to be handled in A's domain in most cases.
Yes, that's how it works currently.
> Other more symmetric expressions probably need some explicit wrappers:
>
> A + in_domain<A>(B)
>
> or something.
Ah! That makes sense, is very explicit, and would be very easy to
implement. Thanks.
>> And I really don't want to have to explain PETE in Proto's docs.
>
> I was thinking more along the lines of "_this_ roughly corresponds to
> _that_ but look how much easier _this other thing_ is in Proto."
OK
-- Eric Niebler Boost Consulting www.boost-consulting.com
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk