Re: [boost] New Library Proposal: dual_state

From: Jost, Andrew (Andrew_Jost_at_[hidden])
Date: 2005-07-12 23:46:37

> -----Original Message-----
> From: boost-bounces_at_[hidden] 
> [mailto:boost-bounces_at_[hidden]] On Behalf Of Fernando Cacciola
> Sent: Tuesday, July 12, 2005 1:11 AM
> To: boost_at_[hidden]
> Subject: Re: [boost] New Library Proposal: dual_state
> 

Now we're getting somewhere.  Your post has led me in a new direction
so, rather than answer the previous message point by point, I'll just
use a few comments as anchors for my explanation.  If I leave any
question unanswered, please feel free to point it out. 

> 
> Hi Andrew,
> 
> > [SNIP]
> >
> > *GUARANTEED OBJECT DELIVERY*
> > First and foremost, dual_state is guaranteed to always 
> deliver a valid 
> > object (or reference), even if this object (or reference) must be 
> > conjured from nowhere.  This is in direct contrast to 
> Boost.optional, 
> > which maintains that
> >
> Well, it never ocurred to me to let optional<> provide default values.
> I just can't think right now of any situation were I would use it.
> Can you post some *real life* examples?
> (the example you posted is not good enough because as it is 
> one would argue wthat the optional/dual_state wrapper is not 
> needed to begin with)

Here is a real example.  Imagine a researcher with many instruments
collecting data.  To make it concrete, let's say we have a geologist
with dozens of instruments spread across some mountain summit.  When he
pushes enter on his laptop, all the instruments take a measurement and
transmit it to the laptop.  Or not.  Because sometimes instruments fail,
or just fail to respond.  Which is why he's using Boost.Optional to
store the measurements.  So that solves the problem and we can all go
home, right?  But wait!  Once we get off the mountain there is more work
to do.  Our geologist friend still must retrieve and analyze his data.
He will load it into containers and call functions.  One of those
functions is a member named of the class trend named trend::loadPoint.
This member accepts data points into the class then, after all the
points are loaded, a call to trend::calculate returns some useful
number.  To do its work, trend must account for missed measurements, so
loadPoint accepts a signal -- namely, any negative value -- to indicate
no measurement.  For the sake of argument, we'll assume that "trend" is
located in a library the geologist cannot change.  His code might
resemble the following example program:

// -- begin
#include <boost/optional/optional.hpp>
#include <boost/none.hpp>
#include <iostream>
#include <vector>

typedef long double data_point;
typedef boost::optional<data_point> opt_dp;
typedef std::vector< opt_dp > data_set;

// the real "trend" is in a library -- this definition is for
illustration only
class trend {
public:
    void loadPoint( data_point x ) {
        if( x<0 ) {
            cout << "no measurement" << endl; 
        } else {
            cout << "point: " << x << endl;
        }
    }
};

int main() {

    data_set d;
    trend tr;

    // make up some data
    d.push_back(5);
    d.push_back(boost::none);
    d.push_back(10);

    // call loadPoint
    for( data_set::iterator p = d.begin(); p != d.end(); ++p ) {
        tr.loadPoint( *p ? p->get() : -1 );
    }
    
    return 0;
}
// -- end

Okay.  This works, but what about that ugly call to loadPoint?  The
problem is that the library function is expecting a signal value, not a
Boost.Optional!  Boost.Optional helped us manage data collection, but
does not help us call trend::loadPoint.  Let's look at another example.

Say I'm working on a project that persists some configuration data.
Momentarily forgetting that Boost.Program_options exists, I decide to
use a simple model in which I map string parameter names to string
values.  Values are allowed to be non-existent, so the map structure
might be declared as follows:

// -- begin
typedef boost::optional<string> param_map_value;
typedef std::map<string, param_map_value> param_map;
param_map pMap;
// -- end

After our program runs for a while, and after the user has set a few
persistable parameters, it is time to quit, so the program opens a
configuration file and begins writing the contents of pMap to that file,
which should look like this:

// -- begin
# CONFIG FILE
PARAM_A     VALUE_A
PARAM_B     VALUE_B
PARAM_C     <undef>
...
// -- end

In the particular case above, the user has apparently set values for the
"A" and "B" parameters, but not for "C".  We might write such a file
with the following code:

// -- begin
// assume cf_out is an open ofstream
for( param_map::iterator p = pMap.begin(); p != pMap.end(); ++p ) {
    cf_out << p->first << " " << ( p->second ? p->second.get() :
"<undef>" ) << endl;
}
// -- end

This illustrates the same problem as before, namely that it requires us
to translate an empty Boost.Optional into something else.  This
translation point is where I see the need for guaranteed object
delivery, for it seems that Boost.Optional and guaranteed object
delivery are two ends of the same pipeline: at the head, we obtain (or
fail to obtain) a value, and in the process escape the need for a signal
value; at the tail, we use the values we've obtained, possibly in a
context that EXPECTS signal values.  Requires them.  So far we've used
only the trinary operator to perform this translation, but is there a
better way?

> 
> As David Abraham said, however, IF it turned out to be useful 
> (I can't see it right now) it seems that such an "optional 
> with default" could be a specialized form of optional<>.
>

> Off the top of my head, a default value could be supported in 
> at least 3 ways:
> 
> (1) Storing an actual default value in each optional instance.
> 
> (2) Parameterizing optional<> with a static factory that can 
> be called to provide such defauts.
> 
> (3) Using T's default constructor (if any) to create such a value.
> 
> Notice that optional<> does not require T to be DefaultConstructible.
>

Several people were quick to point out a key conceptual flaw in the
original dual_state concept.  By delivering only default-constructed
objects, dual_state is an arbitrarily circumscribed implementation of a
more general, and powerful, model.  Suggestions (1) and (2), proposed in
the above clip, are a bit better because they let us specify the default
value, but a deeper problem remains.  It is that by combining the
default value with the optional-like storage object, we commit the
mortal sin of joining metadata with the real data it describes.  It's
the same reason we use iterators to serially access containers, rather
than absurdly assigning some "current_element" state to the container
itself.  Returning to the trend::loadPoint example, that function
expects the signal value to be a negative number.  The next function
might expect 0.  A third function, yet another signal.  The default
value is not a property of our data, rather it is a property of the
functions that USE our data.

In light of this, a better implementation of guaranteed delivery would
make use of template function objects that I'll call "adapters".  The
role of an adapter would be to evaluate a given optional object (I'll
refer to anything resembling Boost.Optional as an "optional" object)
into either the valid object it contains, or a default value, which is
probably defined in the adapter's template specification or constructor.
Adapters separate our optional objects from the defaults we might want
to use in place of uninitialized T objects.  Note that they are
incompatible with noncopyable T objects.

Using adapters, the call to trend::loadPoint in the first example might
look like this:

// -- begin
// call trend::loadPoint
opt_dp::adapter f(-1);
for( data_set::iterator p = d.begin(); p != d.end(); ++p ) {
    tr.loadPoint( f(*p) );
}
// -- end

The second example would look like this:

// -- begin
// assume cf_out is an open ofstream
param_map_value::adapter f("<undef>");
for( param_map::iterator p = pMap.begin(); p != pMap.end(); ++p ) {
    cf_out << p->first << " " << f(p->second) << endl;
}
// -- end

With a little creativity, programmers could make further use of the STL
to craft strikingly elegant solutions.  It seems to me adapters would
make a useful and appropriate addition to our toolbox, improving the
flexibility with which we can access copyable T objects contained in an
optional container.

But, why do I keep saying optional instead of Boost.Optional?  To see,
we must think about how an adapter interacts with the optional object on
which it operates.  I won't spell out the fairly obvious point that
adapters should return references rather than whole T objects, but if
this is so, then an adapter that operates on an empty optional object
must ask that optional to, first, construct a new T object, then return
a reference to that new object.  Based on this, we can say a few things
about an optional object, X, on which an adapter operates:

        (a) X must have valid EMPTY and FULL states.
        (b) There must exist for X a valid EMPTY state, S, in which X
privately holds an object, x, of type T.
        (c) If X is EMPTY (including state S), then altering or
replacing the internal x without setting the state to FULL is a "const
this" operation.

As it currently is, Boost.Optional, which we can think of as a stack
with fixed capacity 1 and variable size, is incompatible with adapters,
but we could certainly define a class derived from Boost.Optional that
meets the above requirements.  Perhaps, for contrast it could be called
"Required"?  User's might choose to use Required, sacrificing
compatibility with noncopyable objects, in cases where they think
adapters will come in handy.

This message is already too long, so I'll reserve my additional thoughts
until others have a chance to respond.

-Andy