# Boost.Outcome REVIEW REPORT
In conclusion to the Outcome Library Review (19-May to 02-June, 2017): The library is REJECTED.
Rejection is principally due to:
(1) Unresolved key behavior at conclusion of the review period
(2) Misunderstanding/disagreement regarding library’s intended purpose
(3) Unclear precedent/implications from accepting this library in its current configuration into the Boost distribution
The following weighed heavily to consider accepting the library into Boost:
(a) The library demonstrates solutions to fundamental issues (error and exception handling)
(b) The library effectively addresses constraints to serve the needs of several user-bases
(c) New paradigms/idioms are enabled
(d) Implementation is of high quality
Some 20+ participants sent 850+ email messages to the public list discussing the Outcome library, plus additional (non-trivial) off-list discussion. Public reviews sent to the list were from:
*- Paul Bristow -- ACCEPT (Tue-23-May)
*- Deniz Bahadir -- ACCEPT (Wed-24-May)
*- Robert Ramey -- REJECT (Fri-26-May)
*- Peter Dimov -- REJECT (Sun-28-May)
*- Emil Dotchevski -- REJECT (Sun-28-May)
*- Vicente J. Botet Escriba -- REJECT (Tue-30-May)
*- Andrzej Krzemienski -- REJECT (Wed-31-May)
*- Gavin Lambert -- REJECT (May 31)
*- Bjorn Reese -- REJECT (Fri-02-June)
The remainder of this report is a summary of discussion with select issues specific to the Outcome library, plus some possible topics for further consideration by the Boost Community and Boost Steering Committee.
This further elaboration is not intended to be comprehensive regarding issues raised during the review; Rather, it is to:
(a) Provide historical “summary/context” for the level of discussion exhibited during this review, and which contributed to the final decision;
(b) Highlight issues that may provide assistance in considering “readiness” for a later review for an evolved form of this library (or for a similar library intending to address the same domain);
(c) Provide “discussion-points” for the Boost Community and Boost Steering Committee regarding possible future evolution or guidance associated with Boost tooling, distribution, and individual library dependencies (related to topics raised during this review).
The conclusions will be:
(1) Outcome Library Author: Consider re-submitting for review when...
(a) Key behavior/design issues are resolved (see below)
(b) Integration plan with Boost is more clear (and consistent with Boost Steering Committee Guidance)
(2) Boost Community / Boost Steering Committee: Consider providing guidance to future library submissions regarding...
(a) Attempted dual-license, this submission attempted:
*- BSL+Apache2 dual-license
(b) “Extra-build-steps”, this submission attempted:
*- Custom preprocessing to “generate-source” (to significantly improve compile-time speed) (Note: During review, a template-based solution was proposed as a possible replacement mechanism, but code-gen should probably be explicitly addressed)
(c) Class/namespace versioning, this submission attempted:
*- Stable ABI/versioning mechanism through a framework provided with this submission (to enable multiple versions of this library used by differing third-party libraries to correctly coexist within the same binary)
(d) Use of outside source; use of git (git-submodules), this submission attempted:
(e) Use of (non-test/Jamfile) for test, this submission attempted:
*- Custom test mechanism (because “Boost.Test” won’t work with C++ exceptions disabled; lib needed “boost-lite” to run the test suite)
(f) Use of CMake, Dependency upon Boost, this submission attempted:
*- No Boost dependency (but provided a “boost-lite” shim to coexist in user-workspaces with-or-without Boost, providing common cmake machinery and test-running)
# SUMMARY OVERVIEW
As demonstrated through the volume and depth of discussion, the library attracted wide interest, which is consistent with a perception that it attempts to address an active need.
Comments suggested the library’s purpose (enabling public APIs to pass <value|error|exception>) was useful, and that enabling a shared handling of <error|exception> to be somewhat novel. Several participants further suggested that the library attempts to serve a role as a “fundamental” library: to provide vocabulary types and pose new idioms, to be used in public APIs as a common <error|exception> exchange mechanism.
One reviewer commented that the library’s documentation under-sold the library’s motivation, suggesting it compelling for:
*- Cases where errors cannot be propagated by throwing exceptions (e.g., within callbacks where an error value is provided [e.g., ASIO]);
*- When passing values between threads, such as in an Actor model where a message queue handles return values (e.g., pass outcome<T> instead of an std::error_code to handle <error|exception> in addition to desired <T>)
Many/most reviewers that voted REJECT commented that they would like a similar library to be accepted, or perhaps an evolved form of this library after necessary changes (see below).
To accept the importance of the domain which the library attempts to address, one need not agree with the design nor implementation of Outcome (indeed, the library is rejected in its current form at the conclusion of this review). Rather, this summary assertion is that the Outcome proposal successfully demonstrates a need for such a fundamental (and community-endorsed) solution (or direction) for error handling.
# ISSUE: Unresolved key behavior at conclusion of the review period
Key behavior (or design) issues that did not reach consensus by the end of the review period:
(1) Narrow vs. Wide contracts -- consensus is not reached
(2) Empty-state (allowed?) remains controversial
(3) Ring-buffer appears uncontroversial; but perhaps needs additional scrutiny
(a) Separate library implementation (for reuse elsewhere, and to enable evolution without touching the ‘result<>’/’outcome<>’ API)?
(b) Bigger buffer?
Some of the difficulties in this evaluation likely trace to the (in-progress) evolution of the proposed ‘std::expected<>’ (also discussed during this review). A future Outcome review would likely benefit from the clarity provided by fully resolved semantics for ‘std::expected<>’.
# ISSUE: Misunderstanding/disagreement regarding library’s intended purpose
The Outcome library attempts to define a paradigm and propose idioms in <error|exception> handling, where few can successfully argue that current practice, libraries, and idioms are sufficient.
The review concluded that Outcome proposes a new paradigm that is demonstrated to be useful and effective. However, disagreement exists on the nature of that framework:
PURPOSE: Outcome is a unified/universal <error|exception> handling framework for use in public APIs;
(a) ...enabling user-customization to address domain-specific constraints
...versus...
(b) ...for possible future consideration in a C++ Standards Track, likely within <system_error>
The Library Author intended (a), while multiple reviewers advocated for (b). This disagreement-in-vision resulted in many difficult discussions during the review, and ultimately lack of consensus for several important API and design decisions (due to the corresponding implied future limitations).
For example: Static-checked vs. Runtime-checked behavior. The library-as-presented attempts to serve both, citing orthogonal user bases with differing (latency) constraints, such as “with-or-without” runtime overhead, and “with-or-without” Undefined Behavior (UB). Current implementation enables integration of arbitrary third-party libraries (each with their own preferred error handling strategy and tradeoffs), with zero-cost conversion and inter-operation. However, this *significantly* complicates the design: The user composes ‘result<>’ and ‘outcome<>’ types with regards to module-specific static or runtime type safety and performance requirements, which (transparently) propagate through the library internals and across threads and public APIs to external modules.
In contrast, a C++ Standards Track implementation is likely much simpler, introducing one-or-two vocabulary types merely parameterized on ‘<T>’, providing a simplified (and “standard”) mechanism to bridge <T|error|exception> transport across third-party module boundaries.
For example, in advocating for a “Standards-Track” vocabulary type, reviewers argued for a mechanism that would enable user-libraries to obsolete the duplicate “throw/no-throw” APIs with a single API:
R filesystem_api(); // throws filesystem_error
R filesystem_api( error_code& ec ) noexcept;
...becomes...
outcome<R> filesystem_api(); // <R|error|exception>
The contrast between the two visions can more distinctly be termed:
(a) “Multi-Modal” Usage -- Many varieties of usage style within the API is necessary, because there are a multitude of ways for library abstractions to be used in composition patterns due to domain-specific constraints (likely with some idioms not yet discovered, or not yet in widespread use)
(b) “Single-Modal” Usage -- One (single) clean design and use model is encouraged
The Library Author states that normal C++ design principles should prefer (b), but due to the vocabulary-type nature of the problem and domain-specific constraints encountered by real-world users, this particular library was forcefully evolved into (a).
It is perhaps ironic that the suitability of a proposed library as providing vocabulary types and possibly evolving into a future standard demands that the Library Author change the library’s stated purpose due to this potential popularity; but that seems consistent with the current circumstance. The Library Author (of course) should have authority to characterize the actual intended use-case and resulting behavior provided through the authored work; with the Review Process providing analysis and feedback regarding possible missed opportunity resulting from those decisions. However, this review’s conclusion seems to encounter the curious scenario where the vision itself remains in conflict.
Of course, these very differing outlooks resulted in some apparent “impasses” or circular discussions regarding type-safety (and number-of-types), wide or narrow contracts, optimization opportunities, “too-big” (or “cluttered”) interfaces, implicit or explicit conversion, the need for iterator and visitor patterns on ‘outcome<T>’, and the “empty” state and sentinel values.
In support of the “Multi-Modal” design, the Library Author posits three main use patterns exist for error-handling objects-returned, and that a single design can serve only two at best:
(Quoting the Library Author):
<quote>
(1) User wants failure reason type to be local to only the code to which it applies. So think custom E type, nonconvertible to any other E type, and E is never error_code nor exception_ptr. They specifically want to keep failure handling local a small patch of code e.g. constexpr evaluation contexts.
All-narrow observers make sense for this use case as object usage is tightly integrated to the code using it, and failure is handled entirely locally.
(2) User wants failure reason type to be globally understood and will be choosing either E = std::error_code, or E = std::exception_ptr. They specifically want failure handling to be able to traverse any or all code, just like C++ exception throws. This is where Outcome was primarily aimed at.
All-wide observers make sense for this use case, as due to handling a wide degree of uncertainty, most failure handling code is *generic* i.e. we test for some specific failure causes, and for everything else we either ignore or hand it off to other code to handle.
(3) User wants to write functional programming logic using the basic vocabulary of Maybe, Either and i/o monads and basic operators of bind, fmap, do etc and probably some subset or refinement of Hana for the collections monads, though my GSoC student may be making the Ranges TS a choice here as well later this summer.
All-narrow observers make sense for this use case as the monadic operators ensure your function will never be called with the wrong state.
</quote>
The Outcome library uses a, “single implementation, multiple personalities” to solve this problem, and makes the intentional decision to attempt targeting (2).
# ISSUE: Unclear precedent/implications from including this library in its current configuration into the Boost distribution
The library in its current configuration is perceived as setting precedent for changes to the Boost distribution, which reviewers commented added complexity to their analysis.
Specific topics:
*- Dual-license (BSL and Apache2) confused reviewers
*- Add bjam support (author agreed)
*- Library does tricks with the preprocessor to avoid significant compile-time costs; but these tricks were commented by some reviewers as being unnecessarily complex.
*- The Author found these tricks necessary to reach reasonable compile metrics for large projects using Outcome in their public APIs, and believes some mechanism must address this build-speed-need. A proposed metaprogramming mechanism may eliminate the need for these preprocessor tricks in the future.
# FINAL THOUGHTS
In the very active years of “Classic Boost” (such as 2004-2008), fundamental libraries had a good chance of becoming vocabulary types that attempt to establish new Best Practice for the industry (and not merely be “just another library”). This awareness drove much of the review discussion, and underscored the importance of experimentation and “lessons learned” to result in a library that offered: (1) behavior and idioms that are understood; and (2) sufficient experience to make use recommendation to the wider user base in contrast to existing alternative approaches.
Consistent with this historical context, it should be relatively uncontroversial to recognize that the Outcome library does not reach that threshold at this time (so it is rejected).
However, this review also suggests:
(1) A similar library is needed;
(2) The community lacks agreement on direction (e.g., “Multi-Modal” vs. “Single-Modal”)
In support of experimentation and to expand on our “lessons-learned”, it seems somewhat obvious that Outcome provides a strongly-implemented Multi-Modal low-latency library that will assist the community in gaining experience with these idioms proposed.
Since conclusion of the review period, discussion continues (on and off-list) regarding possible evolution to the Outcome library, related to feedback provided during this review, and related to discussions of design evolution for pertinent libraries (e.g., possible changes to std::expected<>, and evolution of a never-empty ‘variant2<>’). It is promising that such a strong library as Outcome continues to receive critical attention and evolution by multiple interested parties.
Great thanks to the Boost Community for their tremendous efforts, disciplined review, and detailed exploration of topics in considering this library submission.
Sincerest appreciation to Niall Douglas (Author of the Outcome Library) for pushing the envelope for low-latency <error|exception> handling and for submitting his work to the Boost Review process.
-- charley (Outcome Review Manager)
If there be any in this assembly, any dear
friend of Caesar's, to him I say that Brutus' love
to Caesar was no less than his. If then that friend
demand why Brutus rose against Caesar, this is my
answer: not that I loved Caesar less, but that I loved
Rome more. (3.2.19-24)