Re: [boost] [Preprocessor] Adding variadic macros support

Subject: Re: [boost] [Preprocessor] Adding variadic macros support
From: Paul Mensonides (pmenso57_at_[hidden])
Date: 2010-11-27 22:26:01

On Sat, 27 Nov 2010 11:57:41 -0500, Edward Diener wrote:

> I actually just downloaded the tar.gz file. While I could get Tortoise
> CVS and use it to keep up to date, I have gotten lazy enough where since
> its not in SVN ( or Git ) I tend not to want to have to deal with CVS
> anymore. But that's my problem.

I didn't know that there was a tar.gz file.  AFAIR, I've never put one 
up.  The project has existed since before Sourceforge had Subversion 
services.  At that time, Boost was hosted there in a CVS repository.

> What are the chances that you would be willing to propose Chaos for
> Boost even though it would work with only a subset of compilers, and
> specifically not with VC++ ? I personally think this would still be
> valuable for the authors of Boost libraries and for end-users, but I
> could understand your unwillingness to do so or other Boost developers
> and/or end-users unwillingness to accept Chaos into Boost because it
> does not work for a number of C++ compilers. But IMO that Chaos would
> provide easier preprocessing programming for even a subset of C++
> implementations would make it worth using.

I'm not against it except that I will not compromise Chaos' fundamental 
principle--reference implementation with no workarounds.  It is past time 
that compilers improve and/or get fixed rather than constant hacks 
permeating everything.  Implementing a correct preprocessor is no where 
near as hard as implementing the rest of a modern C++ compiler.

>> The most obvious (considering the context) is support for variadics/
>> placemarkers from the ground up.  Probably more importantly, Chaos
>> generalizes recursion and all higher-order constructs are reentrant
>> without replication of their implementations.
> 
> That is an excellent advantage in simplification over current Boost PP
> despite the fact that you have made Boost PP quite usable even with the
> recursion difficulties.

To a degree.  However, in the pp-lib, there are a few reentrant 
constructs, each having their own "state".  Macros built on top of those 
are generally _not_ reentrant.  I hear about difficulties caused by that 
comparatively frequently.  The problem is fundamental and discourages 
reuse (both in the library and outside of it).

> I totally agree and made a point of saying in my variadic_macro_data
> library that variadic macros real advantage over Boost PP sequences is
> largely in familiarity/ease of syntax for end-users. I am glad you have
> also found their usage in data structures valuable.

I would say that "sequences" ala (a)(b)(c) have become the most effective 
preprocessor data structure used in Boost.  Chaos supports non-unary 
sequences natively and variadic sequences "indirectly".  I.e. the higher-
order sequence algorithms in Chaos directly call user-supplied 
operations, predicates, etc., with the contents of the sequence element.  
E.g.

#define macro(s, x, y, c) + c * (x - y)

CHAOS_PP_EXPR(CHAOS_PP_SEQ_FOR_EACH(
    macro, (1, 2)(3, 4)(5, 6), 7
))

=> + 7 * (1 - 2) + 7 * (3 - 4) + 7 * (5 - 6)

More care has to be taken when the sequence itself is truly variadic 
(meaning the "arity" of elements in the sequence is different from one 
element to another) such as (1)(2, 3)(4, 5, 6).  Chaos has alternative 
higher-order macros for those types of sequences (which it calls, 
unsurprisingly, "variadic sequences").  E.g.

#define macro(s, e, c) + c * e

CHAOS_PP_EXPR(CHAOS_PP_VARIADIC_SEQ_FOR_EACH(
    macro, (1)(2, 3)(4, 5, 6), 7
))

=> + 7 * (1) + 7 * (2, 3) + 7 * (4, 5, 6)

The basic difference is that the elements of the sequence are passed to 
user-defined operations (etc.) as "tuples" whose contents are the 
element.  E.g.

CHAOS_PP_EXPR(
    CHAOS_PP_SEQ_FOR_EACH(
        a, (1, 2)(3, 4)(5, 6)
    )
    CHAOS_PP_VARIADIC_SEQ_FOR_EACH(
        b, (1)(2, 3)(4, 5, 6)
    )
)

=> a(2, 1, 2) a(2, 3, 4) a(2, 5, 6) b(2, (1)) b(2, (2, 3)) b(2, (4, 5, 6))

More generally:

CHAOS_PP_EXPR(
    CHAOS_PP_FOR_EACH(
        a, (CHAOS_PP_SEQ) (1)(2)(3)
    )
    CHAOS_PP_FOR_EACH(
        b, (CHAOS_PP_TUPLE) (1, 2, 3)
    )
    CHAOS_PP_FOR_EACH(
        c, (CHAOS_PP_LIST) (1, (2, (3, ...)))
    )
    CHAOS_PP_FOR_EACH(
        d, (CHAOS_PP_VARIADIC_SEQ) (1)(2)(3)
    )
)

I.e. a sequence is the only data structure defined by Chaos that can 
carry non-unary (or variadic) content.

-----

// with lambda...

>> CHAOS_PP_EXPR(CHAOS_PP_ENUM(
>>      5,
>>      CHAOS_PP_PRIMITIVE_CAT_(class T, CHAOS_PP_ARG(1))
>>          CHAOS_PP_WHEN_(CHAOS_PP_ARG(1))(
>>              CHAOS_PP_CAT_(= T, CHAOS_PP_DEC_(CHAOS_PP_ARG(1)))
>>          )
>> ))

// without lambda...

>> #define _(s, n) \
>>      class T ## n \
>>          CHAOS_PP_WHEN(n)(= CHAOS_PP_CAT(T, CHAOS_PP_DEC(n))) \
>>      /**/
>>
>> CHAOS_PP_EXPR(CHAOS_PP_ENUM(5, _))
>>
>> #undef _
> 
> If lambdas are syntactically easier to use I would still encourage you
> to keep them in Chaos. While I am never against techniques which
> increase compile-time speed I strongly feel that ease of use in
> programming is far more important than waiting longer for a compilation
> to finish.

I'm not sure that they _are_ syntactically easier.  I cannot define 
placeholders like _1, _2, and _3 (at least, not permanently).  I find 
CHAOS_PP_ARG(n) makes things more wordy, not less.  I have a facility 
intended to "temporarily" define placeholders like _1, _2, and _3...

#include CHAOS_PP_PLACEHOLDERS(1)

CHAOS_PP_EXPR(CHAOS_PP_ENUM(
      5,
      CHAOS_PP_PRIMITIVE_CAT_(class T, _1)
          CHAOS_PP_WHEN_(_1)(
              CHAOS_PP_CAT_(= T, CHAOS_PP_DEC_(_1))
          )
))

#include CHAOS_PP_PLACEHOLDERS(0)

...but that simply trades one type of line (defining and undefining a 
macro) for another (pair of includes).  Also, because the preprocessor 
doesn't provide tools to examine arbitrary tokens, the mechanism requires 
a lot of stuff to be "escaped".  I.e. you can't have

template<class T = _1, class U = _2> class XYZ { };

because the mechanism cannot find the _1 and _2.  Instead, you have to 
have something (minimally) like

#include CHAOS_PP_PLACEHOLDERS(1)

CHAOS_PP_EXPR(CHAOS_PP_SEQ_FOR_EACH(
    CHAOS_PP_ESCAPE(template<class T =) _1
        CHAOS_PP_ESCAPE(, class U =) _2> class XYZ { };,
    (A, B)(X, Y)(P, Q)
))

#include CHAOS_PP_PLACEHOLDERS(0)

I do not find that better than the alternative of just:

#define _(s, a, b) \
    template<class T = a, class U = b> class XYZ { }; \
    /**/

CHAOS_PP_EXPR(CHAOS_PP_SEQ_FOR_EACH(
    _, (A, B)(X, Y)(P, Q)
))

#undef _

So, they might not be too bad in terms of expressions in the language 
(i.e. macro invocations, etc.), but they aren't that great at 
representing output forms (i.e. the underlying language, C++-proper in 
this case) which is, at the end of the day, mostly the point.