Re: [boost] Re: FC++ Formal review is now in progress (2/13-2/23)

From: Brian McNamara (lorgon_at_[hidden])
Date: 2004-02-19 17:03:56

On Thu, Feb 19, 2004 at 02:47:25PM -0500, Brian McNamara wrote:
> I think I may see a nice way to handle reference-issues in FC++ using
> boost::ref. 
> 
> I need to think about it a little more, though.  Will post more later.

Ok, I think I've got it now.  Bear with me.

First off, there is one design constraint ("DC") in FC++ that cannot be
changed.  Functoids must be able to parameters by value.  This:

   // From 4.3 of the lambda docs
   int i = 1; int j = 2; 
   (_1 + _2)(i, j); // ok
   (_1 + _2)(1, 2); // error (!)

is not acceptable for FC++; in functional programming, you typically
write code as one giant expression, and having to stop to declare
statements and named variables is anathema.  When programming inside the
FC++ framework, practically everything is a functoid; whereas the issue
above is only an occasional minor annoyance with boost::lambda, it would
be a total killer with FC++.  So that's just one issue that always has
to be kept in the back of your mind.  

One other quick note: throughout this message, I am abusing terminology,
and use "by value" to mean either by-value or by-const-&, and use "by
reference" to mean only by-non-const-&.

Ok, moving on...

Currently in FC++, "full functoids" are classes which wrap up "basic
functoids" and imbue them with a number of extra features (currying,
infix, lambda, etc.).  It occurs to me that "a way to deal with
references" can be considered another one of these "features" which full
functoids can do the work of supporting.

(Aside: full functoids in FC++ in some sense do the same kind of work as
bind() in boost.  The reason you can't say "f(x,_1)" but instead have
to say "bind(f,x,_1)" in boost  is analogous to  the reason that "f(x)"
doesn't work for binary basic functoids but does work for binary full
functoids in FC++.  The main difference is that in boost, the "extra
smarts" are typically added at the call site, whereas in FC++, the
"extra smarts" are typically added at the function definition point.)

In FC++, a polymorphic functoid is typically defined something like this

   struct Foo {
      // sig information
      template <class T, class U>
      T operator()( const T& x, const U& y ) { ... }
   };
   typedef full2<Foo> foo_type;
   foo_type foo;   // foo is a full functoid

and only "by value" parameters are supported.  Now, if we add some more
smarts to full functoids, we could imagine doing this:

   struct Foo {
      // sig information            // (A)
      template <class T, class U>
      T operator()( T& x, const U& y ) { ... }   // note T param
   };
   typedef full2<Foo> foo_type;     // (B)
   foo_type foo;   // foo is a full functoid

Now we have a reference parameter.  Provided that somewhere at either
point (A) or point (B) we communicate this information (that the first
parameter is by-reference) so that the full2 wrapper class is aware of
it, we can delegate the "smarts" of dealing with references to the full2
class.  Good.

The idea is that references can be passed using ref(), as they sometimes
are in Boost.  The refs (reference_wrapper objects) get passed around by
value until at some point we actually reach some underlying basic
functoid which is expecting a reference.  Its fullN wrapper knows this,
and unwraps the reference at the last moment for the basic functoid to
consume.

Passing a non-ref-wrapped argument to a functoid that expects that
argument by-reference is a compile-time error.

Here is an example of how things would work, based on an example from
earlier.  Suppose I want to write the higher-order functoid app:

   app( f, x ) == f(x)

Suppose I want to be able to apply it to functoids with these
signatures:

   //    int f(int);
   //    int g(int&);

Here's what we end up with

   int x;
   app(_,3)(f);        // fine (works now)
   app(_,x)(f);        // fine (works now)
   app(_,3)(g);        // compile-time error
   app(_,x)(g);        // compile-time error   // (1)
   app(_,ref(x))(g);   // works
   
   f(3);               // fine (works now)
   f(x);               // fine (works now)
   g(3);               // compile-time error
   g(x);               // compile-time error   // (2)
   g(ref(x));          // works

I think this is a system which is self-consistent, allows for
references, does not have different behavior depending upon whether
first-order or higher-order functions are used, and works with the
design constraint (DC) mentioned at the top of this message.

There are two main ways this works different from boost::lambda.
At (1) (and maybe also the line above), boost would create a copy and
then modify the copy.  In the system I envision, this would be an error.
(This is the desirable behavior, IMO.)

At (2), even though g is the very function which expects a reference,
you must still use ref() to pass its argument.  I think this is an
inescapable consequence of the combination of DC and the issues
involving higher-order functions described earlier in the thread.

Overall though, I think it looks like it will work, and it seems to me
to be self-consistent and not contain any "special cases".  I would
appreciate feedback along these lines (especially if people can see
some important issue I am missing; I would not be surprised if I have
overlooked some "fatal flaw").

(One case I do know needs more thought is:
   f(ref(x)); 
Note that f() is not "expecting" a reference.)

Assuming it works, do people "like it"?  At some level, it is actually
not too different from the way things are done now in FC++ to get
mutable parameters (use a pointer).  The advantages are
 - the explicit pointers are gone (no more worrying about 'null' issues
   and confounding the "intent" of parameters in the interface)
 - the machinery to do the "dereferencing" is hidden (in current FC++,
   you had to do it explicitly yourself--a pain)
The main disadvantage is that
 - you still need to be explicit when passing arguments by reference
but this is sometimes even true in boost::lambda (e.g. in bind()
expressions).  But now the trust is back in the hands of the programmer;
you can write functoids involving references and pass references around,
and it is your responsibility to deal with object lifetimes and such.
The syntax/mechanism is closer to what happens in boost::lambda now, so
it should be familiar.

I'd appreciate any comments you-all have; especially from those of you
with a deep understanding of the technical details.

-- 
-Brian McNamara (lorgon_at_[hidden])