Re: [boost] Yap's formal review is starting now!

Subject: Re: [boost] Yap's formal review is starting now!
From: Brook Milligan (brook_at_[hidden])
Date: 2018-02-07 20:50:52

> On Feb 5, 2018, at 8:53 AM, Louis Dionne via Boost <boost_at_[hidden]> wrote:
> 
> The formal review of Zach Laine's Yap library starts Monday, February 5th
> and
> ends on Wednesday, February 14th.

I have not heard any discussion of Yap yet, so I’ll jump in and start it off.

My review will come later, but for the moment I want to raise an issue for discussion.

First, a bit of context.  I have been using Yap in production for the last year (thanks Zach for your help along the way).  In my application, I am dealing with arbitrary, user-coded expression trees.  Some of the terminals can be function objects that in turn evaluate other user-coded expression trees.  Ideally, evaluation of these expressions would work for any types wrapped in the appropriate expression classes.  Indeed, this is the case, which is great.

However, I also have a common use case that requires changing how the evaluation works depending on context.  For example, it would be useful to write something analogous to evaluate(expr,context) with a stateful context object that would influence how certain terminals are evaluated.  

The official stance on this [1] is that one should instead use transform(expr,xform), where xform could play the role of context above because transforms can “do anything”, including evaluate the expression using contextual information contained within the xform object.

The problem I see with using transform() is that the entire recursion through the expression tree that is provided by the implementation of evaluate() must be duplicated in some way by the user in order to implement the necessary overloads in xform.  This is because “if xform(N) is a well-formed C++ expression, that xform(N) is used, and no nodes under N in expr are visited” [2].  Therefore, the user has to provide whatever recursion is needed to mimic what evaluate() would otherwise do.

This situation raises several questions.  It is my hope that this post will raise a discussion about the appropriate design considerations at work.

- Is context-dependent evaluation a use-case that is valuable to support by the library?  Based on my experience, the answer is clearly yes, but perhaps others wish to weigh in.

- Is it appropriate for a library to require users to reimplement something as intricate as this recursion in order to support a use case like that?

- Is it appropriate for Yap to have something like evaluate_with_context(eval,context,…) that would implement a recursion similar to evaluate() but allow a context argument to control the evaluation at appropriate customization points.  Note, the variadic part is for placeholders, which are supported by evaluate(), and not really part of the issue.  Again, from my experience it seems that implementing this once correctly in the library would save much pain on the users’ part.

I hope this will stimulate some discussion and look forward to seeing where it goes.

Cheers,
Brook

[1] Often when you create an expression, you will want to evaluate it in different contexts, changing its evaluation -- or even entire meaning -- in each context. evaluate() Is wrong for this task, since it only takes values for substitution into placeholders. In these situations, you should instead use an explicit transform that evaluates your expression. https://tzlaine.github.io/yap/doc/html/boost_yap__proposed_/manual/tutorial/transforming_expressions/explicit_transformations.html

[2] https://tzlaine.github.io/yap/doc/html/boost_yap__proposed_/manual/tutorial/transforming_expressions/explicit_transformations.html