Re: Error handling benchmarks: LEAF, Boost Outcome

From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2019-11-27 12:46:49

On 27/11/2019 00:01, Emil Dotchevski via Boost wrote:
> I wrote a benchmark program for LEAF and Boost Outcome. Here are the
> results:
> 
> https://github.com/zajo/leaf/blob/master/benchmark/benchmark.md

These results seem reasonable if you are calling 32 functions deep, and
those functions do no work.

Returning large sized objects where you cause RVO to be disabled is by
definition a benchmark of memcpy().

In real world code, the C++ compiler works very hard to avoid calling
deep stacks of small functions. It's rare in the real world. Returning
large sized objects from functions also tends to be rare in the real world.

Outcome has known poor codegen. I am working on a new implementation
where if both T and E are trivially copyable or move relocatable, you
get union-based storage and much improved codegen. This should narrow
the gap considerably over the null case.

But in truth, when I test the new implementation in an existing
Outcome-based codebase, I find no statistically observable difference.
If you're doing any real work at all, Outcome can be 10x less efficient
and it gets lost by other more dominant work.

I thank Emil for doing this work, and for sharing the results with me
before he published them. I concur with the benchmarks observed for what
was tested.

Niall