[boost] Notes on Cpp (possible new idiom?)

From: Paul Mensonides (pmenso57_at_[hidden])
Date: 2002-05-27 17:51:25

There are several major flaws in the C/C++ preprocessor.  Several of
them appear in the documentation to the Boost.Preprocessor library:

http://www.boost.org/libs/preprocessor/doc/known_problems_with_cpp.htm

Unfortunately, that list doesn't address the two most annoying parts of the
preprocessor when it is used for meta-programming:

1.  Recursion is not allowed, so it must be faked by huge lists of macros that
control some other 'recursion'.

Two classic examples of this are concatenation and while loops.  In order to use
an argument-expanded concatentation macro inside another, more than one
concatenation macro is required.  For example...

#define AA 10
#define BB CONCAT(A, A)
#define CC CONCAT(B, B)
#define DD CONCAT(C, C)

This will fail to expand to '10' because no CONCAT's are expanded after the
first.  The typical solution to this problem is to define several CONCAT
macros...

#define AA 10
#define BB CONCAT_3RD(A, A)
#define CC CONCAT_2ND(B, B)
#define DD CONCAT_1ST(C, C)

This will work, but it is tedious.  Also, this method fixes unilaterally the
exact concatentation macro that is used in each macro, so any macro that uses
this macro must also know that.  To clarify this point, I'll use a while-loop
example.  In order to produce the following with a preprocessor equivalent...

while (...) {
    while (...) {

    }
}

...one must use two different 'WHILE' controllers--maybe WHILE_1ST and
WHILE_2ND.  When used directly (like the above) this does not present a major
problem, but when used indirectly it can lead to incomprehensible error
messages.  For instance, if the above example was changed to the following
(psuedo-C++)...

int get_length() {
    while (...) {

    }
    return result;
}

while (...) {
    get_length();
}

...it is not as clear that a 'WHILE' is being used in the implementation of
'get_length'.  In the Boost.Preprocessor library, there is a ton of
documentation saying 'uses BOOST_PP_WHILE' and there are many alternative
implementations that 'can be used inside BOOST_PP_WHILE' (e.g. BOOST_PP_MAX and
BOOST_PP_MAX_D).  This is annoying, because the user must know exactly what is
being used all the way down to the deepest crevices of the implementation of the
library macros that he is using.

2.  Debugging.  When a macro is accidentally used inside itself, it will not
expand.  At that point the preprocessor will continue to expand everything else
until the result is unintelligible.  This makes errors of this nature extremely
hard to find and correct.

If, for instance, one failed to use BOOST_PP_MAX_D and instead used BOOST_PP_MAX
inside their own BOOST_PP_WHILE loop, the output would likely be a terrible
mess, and it would not likely be clear where the error is.

There seems to be a way to alleviate these problems somewhat, though I am not
sure whether other preprocessors will accept it--I have tested it on VC++7 and
Comeau C++, but I would appreciate it if others tested it on other platforms.

The solution (seemly) revolves around using the limitations of the preprocessor
against the preprocessor itself (trust me, it really *is* a battle against the
preprocessor).  The following implementation abstracts the 'depth' of a serious
of macros from their use and also provides a somewhat helpful error message if
things go awry.  The following is an 'abstracted' CONCAT implementation:

----- begin sample -----

// 3 different CONCAT's

# define CONCAT_1ST(a, b) PRIMITIVE_CONCAT_1ST(a, b)
#    define PRIMITIVE_CONCAT_1ST(a, b) a ## b

# define CONCAT_2ND(a, b) PRIMITIVE_CONCAT_2ND(a, b)
#    define PRIMITIVE_CONCAT_2ND(a, b) a ## b

# define CONCAT_3RD(a, b) PRIMITIVE_CONCAT_3RD(a, b)
#    define PRIMITIVE_CONCAT_3RD(a, b) a ## b

// private CONCAT used only by the implementation of
// various 'verification' macros.

# define PRIVATE_CONCAT(a, b) PRIMITIVE_PRIVATE_CONCAT(a, b)
#    define PRIMITIVE_PRIVATE_CONCAT(a, b) a ## b

// 'verification' macros

# define VERIFY_CONCAT_1ST() PRIVATE_CONCAT(CONCAT_, CONCAT_1ST(RES_, 1))()
#    define CONCAT_RES_1() CONCAT_1ST
#    define CONCAT_CONCAT_1ST(a, b) DUMMY_CONCAT_1ST
#    define DUMMY_CONCAT_1ST() VERIFY_CONCAT_2ND() // CONCAT_2ND

# define VERIFY_CONCAT_2ND() PRIVATE_CONCAT(CONCAT_, CONCAT_2ND(RES_, 2))()
#    define CONCAT_RES_2() CONCAT_2ND
#    define CONCAT_CONCAT_2ND(a, b) DUMMY_CONCAT_2ND
#    define DUMMY_CONCAT_2ND() VERIFY_CONCAT_3RD() // CONCAT_3RD

# define VERIFY_CONCAT_3RD() PRIVATE_CONCAT(CONCAT_, CONCAT_3RD(RES_, 3))()
#    define CONCAT_RES_3() CONCAT_3RD
#    define CONCAT_CONCAT_3RD(a, b) DUMMY_CONCAT_3RD
#    define DUMMY_CONCAT_3RD() (ERROR_NO_AVAILABLE_CONCAT)

// front-end

# define CONCAT() VERIFY_CONCAT_1ST()

// sample usage

# ifndef MAKE_ERROR
#    define AA 10
# else
#    define AA CONCAT()(1, 0)
# endif

# define BB CONCAT()(A, A)
# define CC CONCAT()(B, B)
# define DD CONCAT()(C, C)

# include <iostream>

int main() {
    std::cout << DD << &std::endl;
    return 0;
}

----- end sample -----

If MAKE_ERROR is not defined, both VC++7 and Comeau C++ compile this fine and
output 10.  However, if MAKE_ERROR is defined, an attempt is made to use one
more CONCAT than the number that exist.  Comeau C++ outputs the following if
MAKE_ERROR
is defined:

error: identifier "ERROR_NO_AVAILABLE_CONCAT" is undefined
    std::cout << DD << &std::endl;
                 ^

Though that isn't a perfect error message, it is better than nothing.

VC++7, on the other hand, compiles without complaint even when MAKE_ERROR is
defined.  I believe that that is incorrect--even though it produces the
'expected' output of '10'.

This same type of thing can be used for things like BOOST_PP_REPEAT,
BOOST_PP_REPEAT_2ND, and BOOST_PP_REPEAT_3RD (and other things like it).  With
this method, macros that used it would automatically adjust to the right 'depth'
and can give some kind of useful error string if there are no available depths.

For example, instead of...

BOOST_PP_REPEAT(count, macro, data)
BOOST_PP_REPEAT_2ND(count, macro, data)
-and-
BOOST_PP_REPEAT_3RD(count, macro, data)

...a uniform automatically adjusting interface could be used:

BOOST_PP_ABSTRACT_REPEAT()(count, macro, data)

...that can be used all over the place.

What do you Boosters think?  Any feedback would be helpful, and I would
appreciate it if the others tested the sample code on their platforms.

Paul Mensonides