[Boost-announce] [boost][review] Outcome Review Report

Subject: [Boost-announce] [boost][review] Outcome Review Report
From: charleyb123 . (charleyb123_at_[hidden])
Date: 2017-06-06 12:34:11

# Boost.Outcome REVIEW REPORT

In conclusion to the Outcome Library Review (19-May to 02-June, 2017):  The
library is REJECTED.

Rejection is principally due to:

  (1) Unresolved key behavior at conclusion of the review period
  (2) Misunderstanding/disagreement regarding library’s intended purpose
  (3) Unclear precedent/implications from accepting this library in its
current configuration into the Boost distribution

The following weighed heavily to consider accepting the library into Boost:

  (a) The library demonstrates solutions to fundamental issues (error and
exception handling)
  (b) The library effectively addresses constraints to serve the needs of
several user-bases
  (c) New paradigms/idioms are enabled
  (d) Implementation is of high quality

Some 20+ participants sent 850+ email messages to the public list
discussing the Outcome library, plus additional (non-trivial) off-list
discussion.  Public reviews sent to the list were from:

  *- Paul Bristow -- ACCEPT (Tue-23-May)
  *- Deniz Bahadir -- ACCEPT (Wed-24-May)
  *- Robert Ramey -- REJECT (Fri-26-May)
  *- Peter Dimov -- REJECT (Sun-28-May)
  *- Emil Dotchevski -- REJECT (Sun-28-May)
  *- Vicente J. Botet Escriba -- REJECT (Tue-30-May)
  *- Andrzej Krzemienski -- REJECT (Wed-31-May)
  *- Gavin Lambert -- REJECT (May 31)
  *- Bjorn Reese -- REJECT (Fri-02-June)

The remainder of this report is a summary of discussion with select issues
specific to the Outcome library, plus some possible topics for further
consideration by the Boost Community and Boost Steering Committee.

This further elaboration is not intended to be comprehensive regarding
issues raised during the review;  Rather, it is to:

  (a) Provide historical “summary/context” for the level of discussion
exhibited during this review, and which contributed to the final decision;
  (b) Highlight issues that may provide assistance in considering
“readiness” for a later review for an evolved form of this library (or for
a similar library intending to address the same domain);
  (c) Provide “discussion-points” for the Boost Community and Boost
Steering Committee regarding possible future evolution or guidance
associated with Boost tooling, distribution, and individual library
dependencies (related to topics raised during this review).

The conclusions will be:

  (1) Outcome Library Author:  Consider re-submitting for review when...
      (a) Key behavior/design issues are resolved (see below)
      (b) Integration plan with Boost is more clear (and consistent with
Boost Steering Committee Guidance)

  (2) Boost Community / Boost Steering Committee:  Consider providing
guidance to future library submissions regarding...
      (a) Attempted dual-license, this submission attempted:
        *- BSL+Apache2 dual-license
      (b) “Extra-build-steps”, this submission attempted:
        *- Custom preprocessing to “generate-source” (to significantly
improve compile-time speed) (Note:  During review, a template-based
solution was proposed as a possible replacement mechanism, but code-gen
should probably be explicitly addressed)
      (c) Class/namespace versioning, this submission attempted:
        *- Stable ABI/versioning mechanism through a framework provided
with this submission (to enable multiple versions of this library used by
differing third-party libraries to correctly coexist within the same binary)
      (d) Use of outside source; use of git (git-submodules), this
submission attempted:
        *- Dependency to an outside (git-submodule) lib archive (gsl-lite:
https://github.com/martinmoene/gsl-lite.git )
      (e) Use of (non-test/Jamfile) for test, this submission attempted:
        *- Custom test mechanism (because “Boost.Test” won’t work with C++
exceptions disabled; lib needed “boost-lite” to run the test suite)
      (f) Use of CMake, Dependency upon Boost, this submission attempted:
        *- No Boost dependency (but provided a “boost-lite” shim to coexist
in user-workspaces with-or-without Boost, providing common cmake machinery
and test-running)

# SUMMARY OVERVIEW

As demonstrated through the volume and depth of discussion, the library
attracted wide interest, which is consistent with a perception that it
attempts to address an active need.

Comments suggested the library’s purpose (enabling public APIs to pass
<value|error|exception>) was useful, and that enabling a shared handling of
<error|exception> to be somewhat novel.  Several participants further
suggested that the library attempts to serve a role as a “fundamental”
library:  to provide vocabulary types and pose new idioms, to be used in
public APIs as a common <error|exception> exchange mechanism.

One reviewer commented that the library’s documentation under-sold the
library’s motivation, suggesting it compelling for:

  *- Cases where errors cannot be propagated by throwing exceptions (e.g.,
within callbacks where an error value is provided [e.g., ASIO]);
  *- When passing values between threads, such as in an Actor model where a
message queue handles return values (e.g., pass outcome<T> instead of an
std::error_code to handle <error|exception> in addition to desired <T>)

Many/most reviewers that voted REJECT commented that they would like a
similar library to be accepted, or perhaps an evolved form of this library
after necessary changes (see below).

To accept the importance of the domain which the library attempts to
address, one need not agree with the design nor implementation of Outcome
(indeed, the library is rejected in its current form at the conclusion of
this review).  Rather, this summary assertion is that the Outcome proposal
successfully demonstrates a need for such a fundamental (and
community-endorsed) solution (or direction) for error handling.

# ISSUE:  Unresolved key behavior at conclusion of the review period

Key behavior (or design) issues that did not reach consensus by the end of
the review period:

  (1) Narrow vs. Wide contracts -- consensus is not reached
  (2) Empty-state (allowed?) remains controversial
  (3) Ring-buffer appears uncontroversial; but perhaps needs additional
scrutiny
      (a) Separate library implementation (for reuse elsewhere, and to
enable evolution without touching the ‘result<>’/’outcome<>’ API)?
      (b) Bigger buffer?

Some of the difficulties in this evaluation likely trace to the
(in-progress) evolution of the proposed ‘std::expected<>’ (also discussed
during this review).  A future Outcome review would likely benefit from the
clarity provided by fully resolved semantics for ‘std::expected<>’.

# ISSUE:  Misunderstanding/disagreement regarding library’s intended purpose

The Outcome library attempts to define a paradigm and propose idioms in
<error|exception> handling, where few can successfully argue that current
practice, libraries, and idioms are sufficient.

The review concluded that Outcome proposes a new paradigm that is
demonstrated to be useful and effective.  However, disagreement exists on
the nature of that framework:

  PURPOSE:  Outcome is a unified/universal <error|exception> handling
framework for use in public APIs;

    (a) ...enabling user-customization to address domain-specific
constraints
...versus...
    (b) ...for possible future consideration in a C++ Standards Track,
likely within <system_error>

The Library Author intended (a), while multiple reviewers advocated for
(b).  This disagreement-in-vision resulted in many difficult discussions
during the review, and ultimately lack of consensus for several important
API and design decisions (due to the corresponding implied future
limitations).

For example:  Static-checked vs. Runtime-checked behavior.  The
library-as-presented attempts to serve both, citing orthogonal user bases
with differing (latency) constraints, such as “with-or-without” runtime
overhead, and “with-or-without” Undefined Behavior (UB).  Current
implementation enables integration of arbitrary third-party libraries (each
with their own preferred error handling strategy and tradeoffs), with
zero-cost conversion and inter-operation.  However, this *significantly*
complicates the design:  The user composes ‘result<>’ and ‘outcome<>’ types
with regards to module-specific static or runtime type safety and
performance requirements, which (transparently) propagate through the
library internals and across threads and public APIs to external modules.

In contrast, a C++ Standards Track implementation is likely much simpler,
introducing one-or-two vocabulary types merely parameterized on ‘<T>’,
providing a simplified (and “standard”) mechanism to bridge
<T|error|exception> transport across third-party module boundaries.

For example, in advocating for a “Standards-Track” vocabulary type,
reviewers argued for a mechanism that would enable user-libraries to
obsolete the duplicate “throw/no-throw” APIs with a single API:

    R filesystem_api(); // throws filesystem_error
    R filesystem_api( error_code& ec ) noexcept;
...becomes...
    outcome<R> filesystem_api();  // <R|error|exception>

The contrast between the two visions can more distinctly be termed:

  (a) “Multi-Modal” Usage -- Many varieties of usage style within the API
is necessary, because there are a multitude of ways for library
abstractions to be used in composition patterns due to domain-specific
constraints (likely with some idioms not yet discovered, or not yet in
widespread use)

  (b) “Single-Modal” Usage -- One (single) clean design and use model is
encouraged

The Library Author states that normal C++ design principles should prefer
(b), but due to the vocabulary-type nature of the problem and
domain-specific constraints encountered by real-world users, this
particular library was forcefully evolved into (a).

It is perhaps ironic that the suitability of a proposed library as
providing vocabulary types and possibly evolving into a future standard
demands that the Library Author change the library’s stated purpose due to
this potential popularity; but that seems consistent with the current
circumstance.  The Library Author (of course) should have authority to
characterize the actual intended use-case and resulting behavior provided
through the authored work; with the Review Process providing analysis and
feedback regarding possible missed opportunity resulting from those
decisions.  However, this review’s conclusion seems to encounter the
curious scenario where the vision itself remains in conflict.

Of course, these very differing outlooks resulted in some apparent
“impasses” or circular discussions regarding type-safety (and
number-of-types), wide or narrow contracts, optimization opportunities,
“too-big” (or “cluttered”) interfaces, implicit or explicit conversion, the
need for iterator and visitor patterns on ‘outcome<T>’, and the “empty”
state and sentinel values.

In support of the “Multi-Modal” design, the Library Author posits three
main use patterns exist for error-handling objects-returned, and that a
single design can serve only two at best:

(Quoting the Library Author):

> <quote>
> (1) User wants failure reason type to be local to only the code to which
> it applies. So think custom E type, nonconvertible to any other E type, and
> E is never error_code nor exception_ptr. They specifically want to keep
> failure handling local a small patch of code e.g. constexpr evaluation
> contexts.
>
> All-narrow observers make sense for this use case as object usage is
> tightly integrated to the code using it, and failure is handled entirely
> locally.
>
>  (2) User wants failure reason type to be globally understood and will be
> choosing either E = std::error_code, or E = std::exception_ptr. They
> specifically want failure handling to be able to traverse any or all code,
> just like C++ exception throws. This is where Outcome was primarily aimed
> at.
>
> All-wide observers make sense for this use case, as due to handling a wide
> degree of uncertainty, most failure handling code is *generic* i.e. we test
> for some specific failure causes, and for everything else we either ignore
> or hand it off to other code to handle.
>
>  (3) User wants to write functional programming logic using the basic
> vocabulary of Maybe, Either and i/o monads and basic operators of bind,
> fmap, do etc and probably some subset or refinement of Hana for the
> collections monads, though my GSoC student may be making the Ranges TS a
> choice here as well later this summer.
>
> All-narrow observers make sense for this use case as the monadic operators
> ensure your function will never be called with the wrong state.
> </quote>

The Outcome library uses a, “single implementation, multiple personalities”
to solve this problem, and makes the intentional decision to attempt
targeting (2).

# ISSUE:  Unclear precedent/implications from including this library in its
current configuration into the Boost distribution

The library in its current configuration is perceived as setting precedent
for changes to the Boost distribution, which reviewers commented added
complexity to their analysis.

Specific topics:
  *- Dual-license (BSL and Apache2) confused reviewers
  *- Add bjam support (author agreed)
  *- Packaging relied upon git-submodules to an outside lib archive
(gsl-lite, https://github.com/martinmoene/gsl-lite.git)
  *- Library does tricks with the preprocessor to avoid significant
compile-time costs; but these tricks were commented by some reviewers as
being unnecessarily complex.
      *- The Author found these tricks necessary to reach reasonable
compile metrics for large projects using Outcome in their public APIs, and
believes some mechanism must address this build-speed-need.  A proposed
metaprogramming mechanism may eliminate the need for these preprocessor
tricks in the future.

# FINAL THOUGHTS

In the very active years of “Classic Boost” (such as 2004-2008),
fundamental libraries had a good chance of becoming vocabulary types that
attempt to establish new Best Practice for the industry (and not merely be
“just another library”).  This awareness drove much of the review
discussion, and underscored the importance of experimentation and “lessons
learned” to result in a library that offered:  (1) behavior and idioms that
are understood; and (2) sufficient experience to make use recommendation to
the wider user base in contrast to existing alternative approaches.

Consistent with this historical context, it should be relatively
uncontroversial to recognize that the Outcome library does not reach that
threshold at this time (so it is rejected).

However, this review also suggests:

  (1) A similar library is needed;
  (2) The community lacks agreement on direction (e.g., “Multi-Modal” vs.
“Single-Modal”)

In support of experimentation and to expand on our “lessons-learned”, it
seems somewhat obvious that Outcome provides a strongly-implemented
Multi-Modal low-latency library that will assist the community in gaining
experience with these idioms proposed.

Since conclusion of the review period, discussion continues (on and
off-list) regarding possible evolution to the Outcome library, related to
feedback provided during this review, and related to discussions of design
evolution for pertinent libraries (e.g., possible changes to
std::expected<>, and evolution of a never-empty ‘variant2<>’).  It is
promising that such a strong library as Outcome continues to receive
critical attention and evolution by multiple interested parties.

Great thanks to the Boost Community for their tremendous efforts,
disciplined review, and detailed exploration of topics in considering this
library submission.

Sincerest appreciation to Niall Douglas (Author of the Outcome Library) for
pushing the envelope for low-latency <error|exception> handling and for
submitting his work to the Boost Review process.

-- charley (Outcome Review Manager)

If there be any in this assembly, any dear
friend of Caesar's, to him I say that Brutus' love
to Caesar was no less than his. If then that friend
demand why Brutus rose against Caesar, this is my
answer: not that I loved Caesar less, but that I loved
Rome more. (3.2.19-24)