Re: [boost] [review][mp11] Formal review of Mp11

Subject: Re: [boost] [review][mp11] Formal review of Mp11
From: Vinnie Falco (vinnie.falco_at_[hidden])
Date: 2017-07-15 15:01:39

This is my review of the mp11 library.

I. DESIGN
---------

The design of the library is well organized and well thought out. The
rationale of prepending `mp_` to all identifiers makes sense and
results in less confusion. It lets lets you write

    using namespace boost::mp11;

with little chance of stepping on anything in the enclosing namespace.
Function names are consistent with their run-time equivalents. The
decision to recognize almost any class template as a list is brilliant
and speaks to the power of the library to express ideas in flexible
and concise ways.

II. IMPLEMENTATION
------------------

I looked at each public header file. Its really amazing how C++11
allows the compact expression of metafunctions using parameter packs;
this is reflected in mp11's implementation. Many of the functions are
simple, intuitive one-liners. There are helpful comments such as this,
to aid users who get compilation errors:

<https://github.com/pdimov/mp11/blob/8e1d23b11f25c3864593b6fc5a8449306e17ea23/include/boost/mp11/list.hpp#L28>

Everything is `constexpr` where possible, and the author has
implemented straightforward optimizations to reduce compilation times
for the heavier functions, such as these:

<https://github.com/pdimov/mp11/blob/8e1d23b11f25c3864593b6fc5a8449306e17ea23/include/boost/mp11/detail/mp_count.hpp#L46>

<https://github.com/pdimov/mp11/blob/8e1d23b11f25c3864593b6fc5a8449306e17ea23/include/boost/mp11/detail/mp_append.hpp#L81>

A casual inspection of the source files should be enough to give any
potential user confidence in the quality of the library (assuming they
are somewhat familiar with the domain). The implementation is in sharp
contrast to the "template soup" I usually encounter when looking at
Boost source code. For example, here's Hana:

<https://github.com/boostorg/hana/blob/e507494b5f7f51026e8afa317b3ca2c058a4d290/include/boost/hana/any_of.hpp#L58>

and here's mp11

<https://github.com/pdimov/mp11/blob/8e1d23b11f25c3864593b6fc5a8449306e17ea23/include/boost/mp11/algorithm.hpp#L861>

III. DOCUMENTATION
------------------

The documentation is spartan and not for a beginner like me. There is
little explanation on many of the functions (most are single
sentences), and a dearth of examples. Most of the reference is a
laundry-list of valid expressions with minimal exposition. I found
this in a surprising contrast to the linked articles, which were
brilliantly written, engaging, informative, and exciting. The author
explained that more examples are coming for the reference items.

Here's an example of the disparity. The documentation for mp_all has a
lot of exposition relative to other reference items, and it includes a
good chunk of example code:

    mp_all<T…>

    mp_all<T…> is mp_true if mp_to_bool<U> is mp_true for all types U
    in T…, mp_false otherwise. Same as mp_and, but does not perform
    short-circuit evaluation. mp_and<mp_false, void> is mp_false, but
    mp_all<mp_false, void> is an error because void does not have a
    nested value. The upside is that mp_all is potentially faster and does
    not mask substitution failures as mp_and does.

    Code Example 62. mp_all behavior

<https://github.com/pdimov/mp11/blob/8e1d23b11f25c3864593b6fc5a8449306e17ea23/doc/mp11/function.adoc#mp_allt>

Now compare that with the documentation for mp_replace_at_c:

    mp_replace_at_c<L, I, W>

    Replaces the element of L at zero-based index I with W and returns
the result.

<https://github.com/pdimov/mp11/blob/8e1d23b11f25c3864593b6fc5a8449306e17ea23/doc/mp11/algorithm.adoc#mp_replace_at_cl-i-w>

I realize that seasoned metaprogrammers will likely have no trouble at
all understanding these terse descriptions and lack of examples but
they aren't doing beginners any favors.

There are things missing from the documentation that I would expect or
hope to find, as some of them may be important to potential users and
others are just good marketing which could help to revive the dying
Boost:

* Comparison to other libraries
  - How is this different from Hana? MPL?

* Limits
  - Which functions work with almost limitless type lists?
  - Which functions might require care in the size of type lists?

* Speed
  - Benchmarks. The doc could link to http://metaben.ch/
  - Which functions might become slow?

The type list limits are compiler-dependent so exact numbers aren't
expected, but at least guidance for where someone might encounter
problems could be provided to aid in the inevitable troubleshooting.

In my opinion, the documentation under-sells the library, I think more
could be done to increase the conversion ratio (the fraction of users
who decide to use the library after reading the documentation). I can
see that metaprogramming gurus who read the Overview will immediately
see the advantage of allowing L to be any class template, but I rather
doubt that beginners will see that - this should be explained in a bit
more direct terms. Here's a rough draft with some sizzle:

    "mp11 is easier to use than other metaprogramming libraries because
    it leverages C++11 and allows the use of any class template to express
    a type list. Programs that use mp11 over other libraries will be easier
    to read, compile faster, and require fewer lines of code."

IV. PRACTICAL USE
-----------------

The library appears useful for its intended purpose. I looked through
my own code to find metafunctions that could be expressed using mp11
instead. The result of this work can be found here:

<https://github.com/vinniefalco/Beast/tree/mp11>
<https://github.com/vinniefalco/Beast/commit/d0385e70fa45c675c617dcd192426e059bc7941d>

I stumbled a bit in the documentation trying to figure out how to form
my expressions but once I did the library performed wonderfully. I
liked how `mp_repeat_c` was a more general replacement of my
`repeat_tuple`.

I spent about two hours with it. Its a shame that so few of Beast's
metafunctions fell into the domain of problems that mp11 solves but
that's an issue with my library and not mp11. I am somewhat familiar
with the problem domain, having done some light manipulation of type
lists.

V. VERDICT
----------

I have seen various metaprogramming libraries over the years, such as
MPL and Hana. Neither of them jumped out at me as something easy to
use or immediately helpful. Despite all of my aforementioned
complaints regarding the documentation, mp11 remains the ONLY
metaprogramming library I have ever had success at in applying to my
own programs, and the only metaprogramming library I would consider
taking as a dependency.

For this reason, mp11 should be ACCEPTED.

Thanks