Re: [boost] [outcome] On the design and documentation

Subject: Re: [boost] [outcome] On the design and documentation
From: Vicente J. Botet Escriba (vicente.botet_at_[hidden])
Date: 2017-05-26 07:03:41

Le 26/05/2017 à 08:22, Thomas Heller a écrit :
> On 05/25/2017 07:28 AM, Vicente J. Botet Escriba wrote:
>> Le 24/05/2017 à 21:44, Thomas Heller via Boost a écrit :
>
>>>
>>> First of all, I don't agree with the strong (conceptual) relationship
>>> between
>>> optional (be it boost:: or std::experimental) in such a way that
>>> expected is a
>>> generalization of it. From what I understand, the purpose of
>>> expected/outcome/
>>> result is to be used as a mechanism to return the result of a
>>> function. As
>>> such it should also expose the semantics of it. Fortunately, we
>>> already have a
>>> (asynchronous) return object standardized (std::future). And this is
>>> my basic
>>> disagreement here: Why not model expected in the lines of future? With
>>> the
>>> main points being:
>>>   - expected being movable only
>>>   - expected<T,E>::value() should always return by value and
>>> invalidate this.
>>>   - (I would really prefer the color .get though ;))
>>> So the question to Vicente/Niall is: what is the motivation to make it
>>> "optional-ish"? Do we have use cases which make this design 
>>> undesirable?
>> expected is a generalization of optional as it is synchronous and could
>> return more information about why the the value is not there.
>
> Right, I can see this argument now, let me try to rephrase it a little 
> bit (correct me if I got something wrong please):
> We want to be able to have a mechanism to return (or store) a value 
> which might not be there, and we need to know why it is not there. The 
> class that's currently available which almost does what we want is 
> optional, in fact, it is already used in such situations, so what we 
> miss is the possible error. So here we are, and then we naturally end 
> up with something like variant<T, E>. Makes perfect sense.
>
> My line of thought was mostly influenced by the property of being 
> solely used as a return object. And well, we already have the 
> asynchronous return objects, so why not go with something a 
> synchronous return object which represents a similar interface/semantics.
>
> With that being said, I am still not sure if the result of both ways 
> to look at it should converged into the same result.
>
>> expected could be seen as the ready storage of a future.
>> future::get block until the future is ready and then returns by
>> reference :)
>
> Except not quite ;)
> excepted indeed sounds like the perfect fit for the value store in the 
> share state. The only problem here, is that it really requires some 
> kind of empty (or uninitialized) since the value is only computed some 
> time in the future (this is motivation #1 for the proposed default 
> constructed semantics), having either a default constructed T or E 
> doesn't make sense in that scenario.
> So it is more like a variant<monostate, T, E>.
 From a high level I will see it as optional<expected<T,exception_pr>>. 
We optional because we have in addition the not-yet-ready state. Once 
the future becomes ready we have expected<T,exception_ptr>.
I understand that the implementation optional<expected<T,exception_pr>> 
could be variant<monostate, T, exception_ptr>. The implementation of 
expected<T,exception_pr> could store as well variant<T,exception_pr>, 
but this makes difficult to extract the ready value as 
variant<T,exception_pr> is not convertible from variant<monostate, 
T,exception_pr>. So there are two possibilities for future::then
* status-quo pass the future as parameter of the continuation
* make it possible to cast variant<T,exception_pr> from 
variant<monostate, T,exception_pr> and pass expected to the continuation

I proposed other future functions (next and catch_exception) long time 
ago to manage with the success and failure branches, so that we don't 
add any constraint in the way the contents is stored.

Resuming, you are right expected<T,E> will not help as a value type 
representing the ready state without major changes.
>
> The weak point in my proposed interface would indeed be the 
> destructive .value() functions, since a shared state needs to be able 
> to obtain it's value multiple times (in the case of shared_future).
>
>>
>> I want to ask you, what would be the liabilities of an expected that is
>> copyable?
>
> That's a good question. When initially thinking about it, I was very 
> deep into the semantics of asynchronous return objects, which use a 
> common shared state which presents an entirely different problem. So I 
> was always thinking in terms of: Does a copy share the result, like 
> shared_future, or does it have to be unique? I guess this doesn't 
> apply to expected and keeping the value semantics of the underlying 
> types makes most sense.
> I guess there is nothing wrong with expected being copyable if T and E 
> are copyable.
Glad to see we agree.
>
>> We don't have a problem returning be reference, why would we like to
>> return by value?
>
> Mainly to represent the fact that we have a return value from a 
> function. There is always one return, if you want to alias it, bind it 
> to a named variable.
I see this as tuples and structured binding. We return a tuple/expected 
and we are able to get the contents by reference. No need to copy while 
you want to access to the elements.
>
>> Why do you prefer get? what do you get with get? How will you name the
>> function that give you access to the value of a PossiblyValued type?
>
> It's really just a different color. The preference to get is coming 
> from my mental model of mine with expected being more like a 
> asynchronous return object (future calls it get as well). There are 
> other types (with similar purpose) using the name get for their 
> accessor: shared_ptr, unique_ptr, tuple and variant.
+ tuple

These get functions are not uniform. Some can throw some not. Some 
return a pointer some return a reference. Some are narrow some are wide.
Should we use a different name for each flavor? I  believe we should, 
but it is too late.

If you had to name a function that takes a unique_ptr and returns the 
pointee value or throw if nullptr, how would you name it.
I'll name it value() even if value is not a name.

Note that future::get  does two things: wait until ready and then 
returns a reference to the success value or throw.
In some way future::get could be named future::wait_value. Too late also.

>
>>>
>>> By having these constraints, expected of course needs to have an
>>> uninitialized
>>> state. As such we'd have the three observers: valid(): true when
>>> has_value()
>>> || has_error(), false otherwise (for example default constructed,
>>> invalidated), has_value() and has_error().
>> Sorry but, not. We don't need such state. This is something future
>> needs, but not expected.
>
> This state is the logical conclusion coming from the destructive value 
> function. As a result, one could just as well reuse it for default 
> construction. If there is consensus that value returns by reference, I 
> would go for result not being default constructible and have 
> never-empty guarantees.
I don't follows you here. How the fact that expected<T,E> could be 
default constructible and returning by value or by reference are related.
>
>> I we decided to default construct to an uninitialized state, I wouldn't
>> support to show to the user this state via any observable function, but
>> via UB (as for chrono::duration).
>
> That's fine for me as well...
I've changed my mind since I wrote this as my arguments were wrong.
Either expected doesn't provide a default constructor or if it does we 
need to default it to T.
>
>>>
>>> Second, I think Niall raised a valid about outcome being a framework 
>>> for
>>> interoperability (completely orthogonal to the first point). However,
>>> I totally
>>> miss this from the proposed library, most pressing are non intrusive
>>> mechanisms. For that purpose I postulate, that a mechanism to transform
>>> between different unexpected results, that is: various error codes etc.
>>> However, for that to work, one would of course need a properly defined
>>> concept,
>>> for example, as Vicente suggested "EitherValue", and a mechanism to
>>> coerce one
>>> error type into another, maybe through ADL, or traits specialization or
>>> whatever.
>> There will be such a proposal as a generalization of Nullable based on
>> what I named PossiblyValued..
>> https://github.com/viboes/std-make/blob/master/doc/proposal/nullable/D0196R3.md 
>>
>
> That's a good starting point!
Comments welcome (privately or on the std--proposal ML)
>
>>
>>
>>> With that in place, one could simply define the different EitherValue
>>> types,
>>> there is no need that everything needs to be in the form of
>>> "basic_XXX". For
>>> the library under review, this would be perfectly sufficient:
>>> template <typename T, typename E>
>>> class expected;
>>> template <typename T>
>>> using result = expected<T, extended_error_info>;
>>> template <typename T>
>>> using outcome = expected<T, variant<extended_error_info, 
>>> exception_ptr>>;
>> We will need to have a specialization of expected<T, varaint<E...>> as
>> we have an index for variant.
>> The revision 2 of the expected proposal talks of a expected<T, E1, ...,
>> En>.
>>>
>>> That is, given that we have either a value or unexpected, we can 
>>> convert
>>> expected<T, E1> to expected<U, E2> if T is convertible to U and E1
>>> "coercable"
>>> (with whichever mechanism) to E2.
>> I have added this conversion constructor recently to the expected
>> proposal as the result of my understanding of the need of Outcome and I
>> hope we will discuss it in Toronto.
>
> Cool!
:)
>
>>>
>>> If we then have a generic mechanism to get from a (possibly user
>>> defined "E")
>>> to an exception, I completely miss the point of the outcome template.
>> And why not to throw E?
>
> throw E; is certainly a nice default, but it really should be 
> customisable. For example, for std::exception_ptr, you probably want 
> to throw the underlying exception?
I've a problem by fixing some hardcoded customization as for 
exception_ptr and error_code as some are suggesting, and don't allowing 
the user to customize their errors.
Maybe this is not the best, but it could be what we are able to agree on.
My expected implementation allows the user to customize the exception to 
throw. I did it this way because we need it for exception_ptr and people 
wanted to have other exceptions than bad_expected_access<E>.
> What if you have exceptions disabled, do you want to give users the 
> chance to implement whatever they want?
For a Boost library I will follow what Boost.Exception proposes already.
>
>>>
>>> All optimizations can then easily be put as implementation details and
>>> the
>>> generic expected<T, E> will probably suffice for most use cases, for
>>> everything
>>> else, we can implement special types which conform to our concepts and
>>> implement the error conversion mechanisms. This will most likely also
>>> work
>>> with different APIs/ABIs.
>> The main problem is that we don't have here the generic interface and it
>> is for this reason we are discussing on the details of a concrete class.
>> The original expected proposal has fmap, bind, catch_error functions. We
>> have removed them form expected since the last revision, but we need now
>> to have a generic interface for those functions.
>
> Those should be the next step once we have the underlying concepts ready.
> I was made aware of a very interesting experiment recently ... mainly 
> to use the mechanisms defined in the coroutine TS for optional.
I'm aware also of a customizing expected<T,E> with the coroutine TS. I 
can share a POC I got ( I don't remember from how) if people are 
interested in.
> Ignoring that co_await sounds awkward when dealing with PossiblyValued 
> objects, it almost gives you everything you need to handle those objects
> with the nice syntax people usually refer to as "monadic".
Right. This is something we should have if  expected and coroutine TS 
are accepted. Maybe we could add *try* for PossibleValued types.
>
>>
>> For me expected should have the minimum, everything else should be
>> associated to a specific concept, as Nullable, PossiblyValued,
>> MonadError, SumType.
>>
>> https://github.com/viboes/std-make/blob/master/doc/proposal/monads/Monads.md 
>>
>>
>> https://github.com/viboes/std-make/tree/master/include/experimental/fundamental/v3/nullable 
>>
>>
>> https://github.com/viboes/std-make/tree/master/include/experimental/fundamental/v3/monad 
>>
>>
>> https://github.com/viboes/std-make/tree/master/include/experimental/fundamental/v3/sum_type 
>>
>
> I guess we are not fundamentally disagreeing. I would really like to 
> see "monad" to vanish again from our vocabulary though, while it is 
> certainly a nice underlying theory, C++ just misses too much to ever 
> really define Monads properly without making fun itself (Type 
> Categories?, Concepts are not really the same...).
Agreed. We can not define type classes with concepts as there is some 
times a universal quantification. Anyway, we are able to provide a 
monadic interface in C++. We have it already in Boost.Hana.

Thanks for your the feedback,
Vicente