Re: [boost] PP lambda?

From: Paul Mensonides (pmenso57_at_[hidden])
Date: 2003-04-10 22:29:28

Aleksey Gurtovoy wrote:

>> The major problem with this is making it work on buggy preprocessors
>> -- which is either not possible or so difficult that it wouldn't be
>> worth it.
>
> That's OK at least with me in terms of that I am more interested in a
> long-term outcome of the PP library's evolution. Also, it's always
> easier to approach vendors when you have something public you can
> point
> them to and say "this doesn't work". And then there is always Wave (if
> I haven't mentioned it yet, I am really excited about this work!).

Yes, it is a great piece of work.  Thanks again to Hartmut Kaiser and Spirit!

[...]

>> Third, the parametric lambda facility is implemented with variadics
>> and placemarkers, which are part of C99, but not C++.  This was a
>> deliberate design decision as the syntax is much cleaner for the user
>> then the non-variadic version was that I had implemented previously.
>
> That's understandable too; these extensions are already available in
> many contemporary compilers anyway.

Yes, and I'm trying to push it. ;)

>> For your example above, it looks something like this (and I'm using
>> a different macro prefix so that the strict pp-lib can co-exist with
>> the CVS pp-lib):
>>
>> CHAOS_PP_EXPR(CHAOS_PP_REPEAT(
>>     5
>>   , ( CHAOS_PP_CAT_( (1), (0) ), =, (0) )
>>   , int x
>> ))
>
>
> Not bad at all. Can we somehow get rid of the extra commas and
> have the "canonical" placeholder names, e.g.
>
> CHAOS_PP_EXPR(CHAOS_PP_REPEAT(
>       5
>     , ( CHAOS_PP_CAT_(_1, _0) = _0 )
>     , int x
>     ))

Not without making it worse.  Two fundamental problems exist here.  First, the
"canonical" placeholders.  The only way that I could tell what they are is with
concatenation.  That would disallow direct input of operators, since
token-pasting that produces multiple tokens is undefined behavior.  (I'm
currently working on a paper for the committee to explicitly make this
well-defined behavior, because it causes all sorts of problems.  I can use the
credentials of as many people as possible in this endeavour, because, after all,
we're dealing with "extending" the preprocessor here.)  I suppose I could define
_1, _2, etc., but that is not a good idea for obvious reasons.  The second
problem with the above is that I can do absolutely nothing with the text between
two operators (e.g. operators that are not commas or parentheses).  Such as the
following:

+ text +

I cannot even get at "text" at all in order to deal with it.  The resulting
syntax that I have above is the result of weighing options for several weeks
(You can ask Hartmut about that.)  Nothing is perfect, but the above is the most
"general".  For example, the CHAOS_PP_REPEAT macro is overloaded to accept
either a regular target macro ala DECL or a lambda expression like the above.
The same is true with every other library primitive that calls user-defined
operations.  You can even mix and match, for example, CHAOS_PP_WHILE takes a
predicate and a state mutating operation.  Either, both, or none can be a lambda
expression.  Further, the lambda expression itself can generate recursive calls
to whatever it wants (it can even produce itself).  Likewise, if designed
properly, user-defined macros can "directly" recurse--even through themselves.
E.g. "repeat" can use "macro" can use "repeat" can use "macro", etc..

>> BTW, the CHAOS_PP_EXPR above has nothing to do with the lambda
>> expression in parameter 2.  That is part of a recursive abstraction
>> that is, by itself, several orders of magnitude more powerful than
>> the CVS pp-lib.  E.g. the CVS pp-lib supports three dimensions of
>> repetition with BOOST_PP_REPEAT, and requires over 256 macros to
>> implement each one.  The strict pp-lib needs only about **three or
>> four** macros to support up to 512 dimensions with a maximum of 512
>> total repetitions.  The same is true for WHILE, FOR, ENUM, various
>> types of FOLD_RIGHT and FOLD_LEFT for different data structures,
>> and even the FOR_EACH_PRODUCT algorithms.  Basically, you have
>> full-fledged, generalized recursion and full algorithmic support.
>
> Way too cool!
>
> Somewhat out-of-line thought - how about BOOST_P2 for the new prefix?

At this point, it must be sufficiently different, because it is, in many ways,
fundamentally not compatible.  If and when major vendors get with the program
so-to-speak, I will change the prefix back to BOOST_PP.  However, the "strict"
version represents, by definition, absolutely _no_ hacks _at all_.  It is also
going to be a test bed for new facilities etc., and will be more volatile than
the CVS pp-lib can afford to be as far as compatibility goes.  As such, for now
it exists as a separate entity, but it must be sufficiently separate so that the
two do not get confused, yet can still be used side-by-side.  There are many
things that are different.  For instance, the entire library is implemented from
the ground up to support variadics/placemarkers if using C99 (or C++ with those
features).  This alters the signatures a quite a few library primitives that use
a data or state parameter.  REPEAT is an example that doesn't have to change.

REPEAT(count, macro, data)
                     ^^^^

In C99, this is:

REPEAT(count, macro, ...)

And the variadic parameters are passed straight through the mechanism and dealt
with directly on the other end--which eliminates the need for excessive
TUPLE_ELEM types of calls:

#define DECL(s, n, id, init, ...) \
    __VA_ARGS__ id ## n = init; \
    /**/

CHAOS_PP_EXPR(CHAOS_PP_REPEAT(
    5, DECL, x, "Aleksey Gurtovoy", std::string<char, my_traits<char> >
))

std::string<char, my_traits<char> > x0 = "Aleksey Gurtovoy";
std::string<char, my_traits<char> > x1 = "Aleksey Gurtovoy";
std::string<char, my_traits<char> > x2 = "Aleksey Gurtovoy";
std::string<char, my_traits<char> > x3 = "Aleksey Gurtovoy";
std::string<char, my_traits<char> > x4 = "Aleksey Gurtovoy";

...as an example.

>> The major purpose of the "strict" pp-lib is to show what _could/can_
>> be done if preprocessors would get their act together and to give me
>> a clean implementation to use as a reference implementation.  The
>> "strict" pp-lib literally blows the CVS pp-lib out of the water.
>>
>> So, the answer to the question, "can it be done?", is "yes."
>
> That's really good news.
>
> Do you think it's possible to archive cleaner syntax for it within
> C99, though? The one you cited is still a little bit too contrived,
> IMO - too
> many brackets/extra punctuation to be easily readable.

The fundamental problem is that these types of things are undefined (in both C99
and C++):

+ ## +=

ID ## (

However, I could ignore all "empty" arguments and have a special escape sequence
to indicate "empty", if I did that, then I could have the "lambda" versions of
macros, such as CHAOS_PP_CAT_ (with the trailing underscore) generate extra
commas before and after, which would all this at best:

( CHAOS_PP_CAT_(class T, (0)) = CHAOS_PP_ARG(0) )

However, it will start to become difficult to remember when commas are necessary
and when they aren't.  Ultimately, the best I can do *requires* token-pasting of
arbitrary tokens, and it will never be what I'd consider perfect--but it will be
useable.

>> Can it be done on buggy preprocessors, particularly VC++ and MWCW?
>> Unlikely.  Can it be done without variadic macros?  Yes, but it makes
>> for ugly, hard-to-read lambda expressions, which make it much easier
>> just to define the target macro instead.
>
> I wonder how the latter ones look like.

I'll give you a hint, it is a four letter word that ends with 'rap' and starts
with 'c'.

Regards,
Paul Mensonides