Boost logo

Boost :

Subject: Re: [boost] [preprocessor] Warning: Incoming
From: Paul Mensonides (pmenso57_at_[hidden])
Date: 2011-06-26 20:11:49

On Sun, 26 Jun 2011 11:31:55 -0400, Edward Diener wrote:

> On 6/26/2011 10:57 AM, Lorenzo Caminiti wrote:

>> If the pp-lib needs to be changed, why not change it radically to add
>> ground-up variadic support? (This is not a rhetoric question. I am
>> sincerely asking as there might be very good reasons not to add
>> ground-up variadic support to Boost.Preprocessor.)
> I am not sure what Paul meant by "ground-up" variadic support, nor how
> you take it, but the variadic support in pp-lib is very much the same as
> in my VMD library with a few added enhancements based on Paul's
> knowledge of preprocessor programming. The overall intent in pp-lib was
> the same as VMD: to integrate the use of variadic macros with pp-lib's
> richer data types ( tuple, array, list, seq ) and to enhance the use of
> pp-lib tuples with variadic macro support.

This is a good summary of the changes.

What I meant by "ground-up" would be effectively re-implementing
Boost.Preprocessor from scratch with variadic macros and placemarkers in
mind. For example, with decent preprocessors (something we don't have as
a rule), this would drastically reduce the number of use cases for
BOOST_PP_TUPLE_ELEM. The use case I'm specifically thinking of at the
moment is the encoding and decoding of auxiliary data or state data
passed through algorithms provided by the library. E.g.

#define ADD(x, y) WHILE(PRED, OP, (x, y))

#define PRED(d, state) \
    TUPLE_ELEM(2, 1, state) \
#define OP(d, state) \
    (INC(TUPLE_ELEM(2, 0, state)), DEC(TUPLE_ELEM(2, 1, state))) \

This one type of scenario decreases the efficiency and drastically
decreases the readability of the code because the TUPLE_ELEM (or similar)
clutter required in the implementation of both PRED and OP is either
pervasive or results in more macro definitions.

Ground-up support would imply that WHILE (and every other higher-order
macro in the library) allow variadic state such that the example becomes:

#define ADD(x, y) WHILE(PRED, OP, x, y)

#define PRED(d, x, y) y
#define OP(d, x, y) INC(x), DEC(y)

The amount of clutter, even in this trivial example, is significantly
reduced resulting in smaller, more efficient, and more readable code.

However, even doing this requires breaking interfaces because existing
arguments are in the wrong orders in many places. For example,
SEQ_FOLD_LEFT's interface is SEQ_FOLD_LEFT(op, state, seq) where op's
interface must be op(s, state, elem). This would need to be SEQ_FOLD_LEFT
(op, (a1, a2, ...)(b1, b2, ...), s1, s2, ...) where op's interface must
"compatible" with op(s, e1, e2, ..., s1, s2, ...). In this case, that's
variadic accumulation state (i.e. fold state) together with potentially n-
ary sequence data (in this case, not variadic sequence data such as (1)
(1, 2)(1, 2, 3), but sequence data that is unary (1)(1)(1), binary (1, 2)
(1, 2)(1, 2), or whatever so long as each element has the same "arity").

All of that aside, there is no way to generalize recursion on broken
compilers--particularly popular ones such as VC++. The lack of this
leads to an implementation explosion. Every reentrant algorithm in
Boost.Preprocessor is implemented with a huge set of macros. Often
times, they are implemented with multiple huge sets of macros which are
different in that they contain differing workarounds for various
compilers. Furthermore, many other higher-order algorithms are not
reentrant because they are not implemented that way (and are instead
implemented in terms of the ones which are). This inherently leads to
implementation bubbling which is particularly costly with the
preprocessor since there is no direct recursion and because the resulting
error messages can be as bad as template-related error messages.

With decent preprocessors, on the other hand, recursion is both
generalizable and extensible. For example, all higher-order algorithms
in Chaos are reentrant and none of them are implemented via huge sets of
algorithm-specific macros. At the end of the day, the lack of
generalizable recursion is even more fundamental and cripples
Boost.Preprocessor even more than lack of ground-up support for variadic
macros. I consider the entire methodology of Boost.Preprocessor to be
garbage due to widespread, aspectual workarounds. Because of the above,
I have no incentive to put major effort into minor improvements
particularly when some vendor is too lazy to fix their preprocessor
(which is a minor implementation in comparison to the underlying compiler
and optimizer) or has too much of a "the standard is whatever we say it
is" mentality.

Paul Mensonides

Boost list run by bdawes at, gregod at, cpdaniel at, john at