Re: [boost] [preprocessor] [config] Support for variadic macros

Subject: Re: [boost] [preprocessor] [config] Support for variadic macros
From: Paul Mensonides (pmenso57_at_[hidden])
Date: 2010-08-07 03:12:10

On 8/6/2010 5:11 AM, Wolf Lammen wrote:
>
>> someone there has submitted a workaround for this bug. I've tried it,
>> it looks sufficient to implement macros like BOOST_PP_TUPLE_*, but
>> variadic, without the "size" parameter for MSVC (and C99/C++0x)
>>
>
> Hi,
>
> Variadic macros are part of the C99 standard. Judging from the preprocessor sources, their developers Mr Mensonides (and perhaps Mr Karvonen) seem to have painfully avoided breaking any rule of the C90 standard. For example, there are never empty macro arguments passed, although this would have simplified code in several instances.
>
> So, using variadic macros will break downward compatibility.
>
> If you think it is time to leave C90 behind, it might be worth evaluating one of the preprocessor engines Chaos (Mensonides) or Order (Karvonen) developed several years ago.
>
> Cheers
>
> Wolf Lammen

The only way to solve Edward's problem without relying on non-portable 
implementation artifacts is to use variadic macros.  If you have 
variadics available, you can discriminate between streams of 
preprocessing tokens which begin with a left parenthesis (and contain a 
matching right parenthesis later) and streams of tokens which do not 
begin with a left parenthesis.

AFAIK, given an arbitrary input stream (with the exception that 
unmatched parentheses are disallowed), there is no way to distinguish 
between the case where the parenthesized expression is the entire 
expression versus the the case where the parenthesized expression is 
only at the beginning of the expression.  E.g. the difference between 
(a, b, c) and something like a C-style cast (int)abc.  If you limit the 
input, you can do more.  Specifically, given variadics and placeholders, 
you can detect and remove a leading parenthesized sequence of 
preprocessing tokens leaving a possibly empty sequence of preprocessing 
tokens, but you cannot construct a fully general-purpose macro that 
detects whether a sequence of preprocessing tokens is empty.  The 
closest approximation to a such a macro is one that the input is not 
allowed to terminate with the name of a function-like macro.  With stuff 
from chaos-pp:

// test.cpp

#include <chaos/preprocessor/control/inline_when.h>
#include <chaos/preprocessor/detection/is_empty.h>
#include <chaos/preprocessor/detection/is_variadic.h>
#include <chaos/preprocessor/logical/bitand.h>
#include <chaos/preprocessor/tuple/eat.h>
#include <chaos/preprocessor/tuple/rem.h>

#define TEST(...) \
   CHAOS_PP_INLINE_WHEN( \
     CHAOS_PP_BITAND \
       (CHAOS_PP_IS_VARIADIC(__VA_ARGS__)) \
       (CHAOS_PP_IS_EMPTY_NON_FUNCTION(CHAOS_PP_EAT __VA_ARGS__)) \
   )(CHAOS_PP_REM) __VA_ARGS__ \
   /**/

TEST(a, b, c)  // a, b, c
TEST((int)abc) // (int)abc
TEST((a, x))   // a, x

gcc -E -P -std=c++0x -I $CHAOS_ROOT -DCHAOS_PP_VARIADICS test.cpp

If preprocessors were better, meaning that they actually closely 
approximated the standard(s), there are methods that can be used to deal 
with this that don't require variadics or any special handling within 
the macro.  Unfortunately, doing such things requires much a much better 
preprocessor than the one in VC++ (which is a horrible piece of crap).  E.g.

// test.cpp

#include <chaos/preprocessor/punctuation/comma.h>
#include <chaos/preprocessor/punctuation/paren.h>
#include <chaos/preprocessor/recursion/rail.h>

#define M1(x) M2(x)
#define M2(x) M3(x)
#define M3(x) x

#define L CHAOS_PP_UNSAFE_RAIL(CHAOS_PP_LPAREN)()
#define R CHAOS_PP_UNSAFE_RAIL(CHAOS_PP_RPAREN)()
#define C CHAOS_PP_UNSAFE_RAIL(CHAOS_PP_COMMA)()

M1(123) // 123

CHAOS_PP_WALL( M1(C R L C) ) // , ) ( ,

gcc -E -P -std=c++0x -I $CHAOS_ROOT test.cpp

Using similar stuff, chaos-pp already has some non-invasive facilities 
designed to pass around things like type names that come from templates 
with multiple arguments.

#include <chaos/preprocessor/facilities/type.h>
#include <chaos/preprocessor/recursion/rail.h>

#define A(x) B(x)
#define B(x) C(x)
#define C(x) x

CHAOS_PP_WALL(A(
   std::vector<CHAOS_PP_TYPE(std::pair<int, int>)>
))
// std::vector<std::pair<int, int>>

gcc -E -P -std=c++0x -I $CHAOS_ROOT -DCHAOS_PP_VARIADICS test.cpp

In fact, the macros through which such things are passed need not be 
designed to handle them:

// test.cpp

#include <boost/preprocessor/punctuation/comma_if.hpp>
#include <boost/preprocessor/seq/for_each_i.hpp>

#include <chaos/preprocessor/facilities/type.h>
#include <chaos/preprocessor/recursion/rail.h>

#define MACRO(r, data, i, elem) BOOST_PP_COMMA_IF(i) elem

#define TYPELIST(seq) \
   CHAOS_PP_WALL( \
     typelist< \
       BOOST_PP_SEQ_FOR_EACH_I( \
         MACRO, ~, seq \
       ) \
     > \
   ) \
   /**/

TYPELIST(
   (int)
   (double)
   (CHAOS_PP_TYPE(std::pair<int, int>))
   (std::complex<double>)
)
// typelist<int, double, std::pair<int, int>, std::complex<double> >

gcc -E -P -std=c++0x -I $BOOST_ROOT -I $CHAOS_ROOT -DCHAOS_PP_VARIADICS 
test.cpp

What's interesting to note in this case is that variadic content is 
being passed _through_ a library (the Boost pp-lib) that is not designed 
to handle variadic content _and_ being buried in the middle of arbitrary 
output.  (Similar stuff can be done without variadics, but the encoding 
is much more verbose.)

The basic problem with the Boost pp-lib is that it has to be 
portable--even to preprocessors that are foundationally broken (such as 
MSVC).  Because of that, it really represents the lowest common 
denominator of what can be done with the preprocessor stably across so 
many targets.  This is particularly true because the metaprogramming 
libraries in Boost are used heavily by other "regular" Boost libraries.

In the case of VC++, some uses of variadics could be made to work with 
enough effort, but variadics themselves are not yet portable enough in 
C++ compilers to deploy as a required API in Boost.

OTOH, there are preprocessors out there that can handle the far more 
advanced tricks that (e.g.) chaos-pp uses.  GCC, EDG-based compilers, 
Wave, and a few others, for example, can all handle virtually all of 
chaos-pp.

Regards,
Paul Mensonides