Re: [boost] [Preprocessor] Adding variadic macros support

Subject: Re: [boost] [Preprocessor] Adding variadic macros support
From: Edward Diener (eldiener_at_[hidden])
Date: 2010-11-28 08:08:15

On 11/27/2010 10:26 PM, Paul Mensonides wrote:
> On Sat, 27 Nov 2010 11:57:41 -0500, Edward Diener wrote:
>
>> I actually just downloaded the tar.gz file. While I could get Tortoise
>> CVS and use it to keep up to date, I have gotten lazy enough where since
>> its not in SVN ( or Git ) I tend not to want to have to deal with CVS
>> anymore. But that's my problem.
>
> I didn't know that there was a tar.gz file.  AFAIR, I've never put one
> up.  The project has existed since before Sourceforge had Subversion
> services.  At that time, Boost was hosted there in a CVS repository.

If you could get the Chaos files in a subversion repository I think it 
would be helpful. I have nothing in particular against CVS other than 
the fact that like most programmers I find subversion much more flexible 
to use.

>
>> What are the chances that you would be willing to propose Chaos for
>> Boost even though it would work with only a subset of compilers, and
>> specifically not with VC++ ? I personally think this would still be
>> valuable for the authors of Boost libraries and for end-users, but I
>> could understand your unwillingness to do so or other Boost developers
>> and/or end-users unwillingness to accept Chaos into Boost because it
>> does not work for a number of C++ compilers. But IMO that Chaos would
>> provide easier preprocessing programming for even a subset of C++
>> implementations would make it worth using.
>
> I'm not against it except that I will not compromise Chaos' fundamental
> principle--reference implementation with no workarounds.  It is past time
> that compilers improve and/or get fixed rather than constant hacks
> permeating everything.  Implementing a correct preprocessor is no where
> near as hard as implementing the rest of a modern C++ compiler.

I would not think anyone would want you to manipulate Chaos for 
compilers with broken preprocessing. The whole idea of Chaos in Boost 
should be that it strictly requires a compiler with a correct C++ 
standard preprocessor. As others have commented in this thread, it might 
even be an impetus for compilers which have preprocessor bugs to fix 
them. As you mention, implementing a correct preprocessor is easier than 
implementing correct C++ in general.

>
>>> The most obvious (considering the context) is support for variadics/
>>> placemarkers from the ground up.  Probably more importantly, Chaos
>>> generalizes recursion and all higher-order constructs are reentrant
>>> without replication of their implementations.
>>
>> That is an excellent advantage in simplification over current Boost PP
>> despite the fact that you have made Boost PP quite usable even with the
>> recursion difficulties.
>
> To a degree.  However, in the pp-lib, there are a few reentrant
> constructs, each having their own "state".  Macros built on top of those
> are generally _not_ reentrant.  I hear about difficulties caused by that
> comparatively frequently.  The problem is fundamental and discourages
> reuse (both in the library and outside of it).

Understood.

>
>> I totally agree and made a point of saying in my variadic_macro_data
>> library that variadic macros real advantage over Boost PP sequences is
>> largely in familiarity/ease of syntax for end-users. I am glad you have
>> also found their usage in data structures valuable.
>
> I would say that "sequences" ala (a)(b)(c) have become the most effective
> preprocessor data structure used in Boost.  Chaos supports non-unary
> sequences natively and variadic sequences "indirectly".  I.e. the higher-
> order sequence algorithms in Chaos directly call user-supplied
> operations, predicates, etc., with the contents of the sequence element.

I have always found Boost PP sequences to be more than adequate enough 
to be used internally as opposed to variadic macro syntax. My main 
impetus for the variadic_macro_data library was to allow macros with 
variadic data to be specified for end-users for whom AMACRO(x,y,z) feels 
and looks more natural than AMACRO((x),(y),(z)). Other than that Boost 
PP sequences ( and arrays, lists, and tuples ) have much richer 
functionality than variadic data. But I think it was a very good thing 
that the C++ standard committee recognized that variadic macros were 
necessary from an ease of use standpoint, and of course Boost PP is not 
part of the C++ standard.

> E.g.
>
> #define macro(s, x, y, c) + c * (x - y)
>
> CHAOS_PP_EXPR(CHAOS_PP_SEQ_FOR_EACH(
>      macro, (1, 2)(3, 4)(5, 6), 7
> ))
>
> =>  + 7 * (1 - 2) + 7 * (3 - 4) + 7 * (5 - 6)

Very cool !

>
> More care has to be taken when the sequence itself is truly variadic
> (meaning the "arity" of elements in the sequence is different from one
> element to another) such as (1)(2, 3)(4, 5, 6).  Chaos has alternative
> higher-order macros for those types of sequences (which it calls,
> unsurprisingly, "variadic sequences").  E.g.
>
> #define macro(s, e, c) + c * e
>
> CHAOS_PP_EXPR(CHAOS_PP_VARIADIC_SEQ_FOR_EACH(
>      macro, (1)(2, 3)(4, 5, 6), 7
> ))
>
> =>  + 7 * (1) + 7 * (2, 3) + 7 * (4, 5, 6)
>
> The basic difference is that the elements of the sequence are passed to
> user-defined operations (etc.) as "tuples" whose contents are the
> element.  E.g.
>
> CHAOS_PP_EXPR(
>      CHAOS_PP_SEQ_FOR_EACH(
>          a, (1, 2)(3, 4)(5, 6)
>      )
>      CHAOS_PP_VARIADIC_SEQ_FOR_EACH(
>          b, (1)(2, 3)(4, 5, 6)
>      )
> )
>
> =>  a(2, 1, 2) a(2, 3, 4) a(2, 5, 6) b(2, (1)) b(2, (2, 3)) b(2, (4, 5, 6))
>
> More generally:
>
> CHAOS_PP_EXPR(
>      CHAOS_PP_FOR_EACH(
>          a, (CHAOS_PP_SEQ) (1)(2)(3)
>      )
>      CHAOS_PP_FOR_EACH(
>          b, (CHAOS_PP_TUPLE) (1, 2, 3)
>      )
>      CHAOS_PP_FOR_EACH(
>          c, (CHAOS_PP_LIST) (1, (2, (3, ...)))
>      )
>      CHAOS_PP_FOR_EACH(
>          d, (CHAOS_PP_VARIADIC_SEQ) (1)(2)(3)
>      )
> )
>
> I.e. a sequence is the only data structure defined by Chaos that can
> carry non-unary (or variadic) content.

The choice of data structures in Boost PP is a big plus so I would 
expect the same from Chaos.

>
> -----
>
> // with lambda...
>
>>> CHAOS_PP_EXPR(CHAOS_PP_ENUM(
>>>       5,
>>>       CHAOS_PP_PRIMITIVE_CAT_(class T, CHAOS_PP_ARG(1))
>>>           CHAOS_PP_WHEN_(CHAOS_PP_ARG(1))(
>>>               CHAOS_PP_CAT_(= T, CHAOS_PP_DEC_(CHAOS_PP_ARG(1)))
>>>           )
>>> ))
>
> // without lambda...
>
>>> #define _(s, n) \
>>>       class T ## n \
>>>           CHAOS_PP_WHEN(n)(= CHAOS_PP_CAT(T, CHAOS_PP_DEC(n))) \
>>>       /**/
>>>
>>> CHAOS_PP_EXPR(CHAOS_PP_ENUM(5, _))
>>>
>>> #undef _

In a way the lambda use above is syntactically easier because one does 
not have to break up the usage into two pieces. Often syntactically 
easier does not mean shorter but more understandable in the way 
something is constructed.

>>
>> If lambdas are syntactically easier to use I would still encourage you
>> to keep them in Chaos. While I am never against techniques which
>> increase compile-time speed I strongly feel that ease of use in
>> programming is far more important than waiting longer for a compilation
>> to finish.
>
> I'm not sure that they _are_ syntactically easier.  I cannot define
> placeholders like _1, _2, and _3 (at least, not permanently).  I find
> CHAOS_PP_ARG(n) makes things more wordy, not less.  I have a facility
> intended to "temporarily" define placeholders like _1, _2, and _3...
>
> #include CHAOS_PP_PLACEHOLDERS(1)
>
> CHAOS_PP_EXPR(CHAOS_PP_ENUM(
>        5,
>        CHAOS_PP_PRIMITIVE_CAT_(class T, _1)
>            CHAOS_PP_WHEN_(_1)(
>                CHAOS_PP_CAT_(= T, CHAOS_PP_DEC_(_1))
>            )
> ))
>
> #include CHAOS_PP_PLACEHOLDERS(0)
>
> ...but that simply trades one type of line (defining and undefining a
> macro) for another (pair of includes).  Also, because the preprocessor
> doesn't provide tools to examine arbitrary tokens, the mechanism requires
> a lot of stuff to be "escaped".  I.e. you can't have
>
> template<class T = _1, class U = _2>  class XYZ { };
>
> because the mechanism cannot find the _1 and _2.  Instead, you have to
> have something (minimally) like
>
> #include CHAOS_PP_PLACEHOLDERS(1)
>
> CHAOS_PP_EXPR(CHAOS_PP_SEQ_FOR_EACH(
>      CHAOS_PP_ESCAPE(template<class T =) _1
>          CHAOS_PP_ESCAPE(, class U =) _2>  class XYZ { };,
>      (A, B)(X, Y)(P, Q)
> ))
>
> #include CHAOS_PP_PLACEHOLDERS(0)
>
> I do not find that better than the alternative of just:
>
> #define _(s, a, b) \
>      template<class T = a, class U = b>  class XYZ { }; \
>      /**/
>
> CHAOS_PP_EXPR(CHAOS_PP_SEQ_FOR_EACH(
>      _, (A, B)(X, Y)(P, Q)
> ))
>
> #undef _
>
> So, they might not be too bad in terms of expressions in the language
> (i.e. macro invocations, etc.), but they aren't that great at
> representing output forms (i.e. the underlying language, C++-proper in
> this case) which is, at the end of the day, mostly the point.

It is up to you whether or not you find lambda syntax within Chaos 
easier to use and/or flexible enough to justify their inclusion in 
Chaos. My general point is that unless compiler time increases to a very 
large extent when creating C++ constructs, I find that programmers 
complaining about excess compiler time when using C++ ( or any other 
language ) are off base. In today's programming world valuable 
programming work can almost always be done while a programmer waits for 
a compile/build to finish.