Re: [boost] Preprocessor Library

From: Paul Mensonides (pmenso57_at_[hidden])
Date: 2002-04-04 03:04:39

> Basically, we are trying to come up with a best way to utilize powerful
> capabilities of Vesa Karvonen's PREPROCESSOR library
> (http://www.boost.org/libs/preprocessor/doc/index.htm) without imposing the
> downsides of preprocessor-based code generation (such as "real pain to
> debug" :) on users of our libraries (and, of course, ourselves).
>
> > If that's it, I have some ideas from my own stuff.
>
> I am interested.

Pardon the length of my reply, I have to lay some background...

I haven't really used the preprocessor library, but I have some stuff that I
created independently that do the same type of things...so here are some
possibilities...

Firstly, the preprocessor meta-programming facilities should be easily turned on
and off.  That way you can localize the macro names to a small amount of source.
I accomplished this with a reference counted inclusion mechanism.  (note:  my
personal library is called 'chaos')

// primitive operations

#define CONCAT(A, B) CONCAT_PRIMITIVE(A, B)
#define CONCAT_PRIMITIVE(A, B) A ## B

#define STRINGIZE(A) STRINGIZE_PRIMITIVE(A)
#define STRINGIZE_PRIMITIVE(A) #A

// library version location abstraction

#define CHAOS(header) STRINGIZE(CONCAT(chaos3/, header))

#define CHAOS_ENABLE(header) CHAOS(CONCAT(rc/++, header))
#define CHAOS_DISABLE(header) CHAOS(CONCAT(rc/--, header))

Then for each prepreprocessor facility, I have a matched pair of '++' and '--'
files that do nothing but manage the reference counting... (incrementing sample)

// [++incr.h]

#ifndef CHAOS_INCR_H
    #define CHAOS_INCR_H 1
    #ifndef CHAOS_PREPROCESSOR_GUARD
        #define CHAOS_PREPROCESSOR_GUARD
    #endif
    #include CHAOS(incr.h)
    #undef CHAOS
#else
    #if CHAOS_INCR_H >= 50
        #error chaos:  inclusion limit exceeded on "incr.h"
    #elif CHAOS_INCR_H == 1
        #undef CHAOS_INCR_H
        #define CHAOS_INCR_H 2
    #elif CHAOS_INCR_H == 2
        #undef CHAOS_INCR_H
        #define CHAOS_INCR_H 3
    // ...
    #endif
#endif

// [--incr.h]

#ifndef CHAOS_INCR_H
    #error chaos:  "incr.h" is not enabled
#else
    #if CHAOS_INCR_H > 50
        #error chaos:  user-tampering detected on macro CHAOS_INCR_H
    #elif CHAOS_INCR_H == 1
        #define CHAOS_PREPROCESSOR_GUARD
        #undef CHAOS_INCR_H
        #include CHAOS(incr.h)
        #undef CHAOS_PREPROCESSOR_GUARD
    #elif CHAOS_INCR_H == 2
        #undef CHAOS_INCR_H
        #define CHAOS_INCR_H 1
    #elif CHAOS_INCR_H == 3
        #undef CHAOS_INCR_H
        #define CHAOS_INCR_H 2
    // ...
    #endif
#endif

// [incr.h]

#ifndef CHAOS_PREPROCESSOR_GUARD
#error chaos:  invalid inclusion of "incr.h"
#endif

#ifdef CHAOS_INCR_H

// interface

#define INCR(N) CONCAT(CHAOS_INCR_, N)

// implementation

#define CHAOS_INCR_0 1
#define CHAOS_INCR_1 2
#define CHAOS_INCR_2 3
// ...

#else

// disable

#undef INCR

#undef CHAOS_INCR_0
#undef CHAOS_INCR_1
#undef CHAOS_INCR_2
// ...

#endif

This way, I can easily use facilities then turn them off.  The locality of names
means that I don't need to use extensive "library-name" prefixes on all user-use
macros.  So, for instance, I'd use the above mechanism... (note:  I use the
trigraph to distinguish the abnormal include usage.)

#include "chaos.h" // basic primitives

??=include CHAOS_ENABLE(incr.h)
    // expands to:  #include "chaos3/rc/++incr.h"

// use INCR

??=include CHAOS_DISABLE(incr.h)
    // expands to:  #include "chaos3/rc/--incr.h"

Secondly, for something like the linearization that I think you were talking
about above, I have a similar facility.  The interesting part is that the
mechanism I use uses the inclusion mechanism to perform the top-level expansion.
I simply created a series of header files like the following...

// [make_1.h] (special case for lower-bound)

#ifdef CHAOS_LBOUND
    #if CHAOS_LBOUND < 1
        MACRO(0)
        MACRO(1)
    #elif CHAOS_LBOUND == 1
        MACRO(1)
    #endif
#else
    MACRO(1)
#endif

// [make_2.h]

#include "make_1.h"

#ifdef CHAOS_LBOUND
    #if CHAOS_LBOUND <= 2
        MACRO(2)
    #endif
#else
    MACRO(2)
#endif

// [make_3.h]
// (same as "make_2.h" with the numbers changed)

// ...

Then I have a helper macro...

#define MAKE(C) CHAOS(CONCAT_EX(3, (linearize/make/make_, C, .h)))
  // note that CONCAT_EX(3 ...
  // is just a ternary concatenation primitive

So, say I use the above facility to implement a typelist system...

#define CHAOS_TYPELIST_LIMIT 50

template<class T, class U = null_t> struct typelist_t {
    typedef T head;
    typedef U tail;
};

template<int> struct typelist;

template<> struct typelist<1> {
    template<class T1> struct args {
        typedef typelist_t<T1> type;
    };
};

??=include CHAOS_ENABLE(linearize.h)

#define MACRO(C) \
    template<> struct typelist<C> { \
        template<LIST_CLASS_T_DEF(C)> struct args { \
            typedef typelist_t<T1, \
                typename typelist<C - 1> \
                ::args<LIST_T_MINUS_FIRST(C)>::type> \
                type; \
        }; \
    };

#define CHAOS_LBOUND 2
#include MAKE(CHAOS_TYPELIST_LIMIT)

??=include CHAOS_DISABLE(linearize.h)

// function-type extraction

#define CHAOS_EXTRACT_LIMIT 50

template<class T> struct incomplete : type_to_type<T> { };

template<class T>
struct strip_incomplete
    : type_to_type<T> { };

template<class T>
struct strip_incomplete< incomplete<T> >
    : type_to_type<T> { };

template<class, template<int> class output> class extract;

??=include CHAOS_ENABLE(linearize.h)

#define STRIP(N) typename strip_incomplete<CONCAT(T, N)>::type

#define MACRO(C) \
    template<class R, LIST_CLASS_T(C), template<int> class output> \
    struct extract<R (LIST_T(C)), output> { \
        typedef typename output<C> \
            ::template args<LIST_USER(STRIP, C, COMMA)> \
            type; \
    };

#include MAKE(CHAOS_EXTRACT_LIMIT)

??=include CHAOS_DISABLE(linearize.h)

// new TYPELIST macro

#define TYPELIST(ARGS) extract<void ARGS, typelist>::type::type

Now I have a TYPELIST macro that is used thusly... TYPELIST((int, int, int)).
When, and if, we get __VA_ARGS__ from C99 in C++, I will change it to:

#define TYPELIST(...) extract<void (__VA_ARGS__), typelist>::type::type

Then I'll finally have:  TYPELIST(int, int, int)

I use the 'extraction' facility for several things like ->* closures and tuples,
etc..

Utimately, I run everything through a preprocessor pass first and then compile
the output, so with Comeau C++ on Win32 I have a batch file like the
following...

como --A --preprocess somefile.cpp > pp.somefile.cpp
como --A pp.somefile.cpp

The nice thing is that using the inclusion mechanism for the top-level
repetition is significantly faster on Comeau C++.  Also, each macro expansion,
MACRO(1) MACRO(2), etc., is on a separate line, which makes debugging it
significantly easier.  You can actually get useful error messages if you compile
the preprocessed output in a separate step, instead of getting something like
this...

error:  not enough parameters for macro (or something)
MACRO(2)
^

--which obviously isn't to useful.  Please note that I'm using my own facilities
above, not Boost's, but the meanings for various macro-names should be
self-explanatory.  Also, note that I used the name 'MACRO' because nobody in
there right mind should use that name. :)  It is undefined every time
"linearize.h" is enabled or disabled automatically.

So basically, I guess my points are that the preprocessor library should be
reference counted to improve localization.  I.e. the following works fine...

??=include CHAOS_ENABLE(incr.h) // rc goes to 1 (incr enabled)
??=include CHAOS_ENABLE(linearize.h)

    ??=include CHAOS_ENABLE(incr.h) // rc goes to 2

// use linearize facilities and INCR

??=include CHAOS_DISABLE(linearize.h)
??=include CHAOS_DISABLE(incr.h) // rc goes to 1

// use INCR

??=include CHAOS_DISABLE(incr.h) // rc goes to 0 (incr disabled)

Secondly, the top level expansion can be done with the inclusion mechanism
rather than macro replacement, which generates more debuggeable (sp?) output.

And lastly, it would be nice to have a file which defines primitives to abstract
the location of the rest of the Boost libraries.  That way, if you use a new
library version and want to keep the old one too, you just need to change one
file.

#define BOOST(header) STRINGIZE(CONCAT(boost_v5/, header))

Maybe you guys have looked at this type of stuff before.  As I said before, I'm
new here. :)

Paul Mensonides