Boost logo

Boost :

From: Douglas Gregor (gregod_at_[hidden])
Date: 2001-06-29 01:10:23


I believe that the preprocessor library should be accepted into Boost.

As a reasonable testcase for the preprocessor library, I converted
Boost.Function to use the preprocessor library. I'll relate my experiences
when performing this conversion first, and then I have a set of specific
comments.

----------Conversion of boost/function/functionN.hpp -----------
Boost.Function has several numbered function classes (function0, function1,
etc.) that each perform the same functionality, but for different numbers of
arguments. To create these function classes, there is a pattern file
"boost/function/function_template.hpp" that contains all of the code, but
relies on macro definitions for everything that changes depending on the
number of arguments: class names, argument lists, template parameter lists,
etc. Before the preprocessor library, there was a set of files
boost/function/function0.hpp, boost/function/function1.hpp, etc. that were
generated by a simple Perl script. Each file looks something like this:

----
#define BOOST_FUNCTION_TEMPLATE_PARMS typename T1, typename T2, typename T3, 
typename T4, typename T5
#define BOOST_FUNCTION_TEMPLATE_ARGS T1, T2, T3, T4, T5
#define BOOST_FUNCTION_PARMS T1 a1, T2 a2, T3 a3, T4 a4, T5 a5
#define BOOST_FUNCTION_ARGS a1, a2, a3, a4, a5
#define BOOST_FUNCTION_FUNCTION function5
// on and on and on
---
The goal in using the preprocessor library was to eliminate the need for 
using the Perl script to autogenerate these files. Instead, merely defining 
the number of arguments desired and then including 
boost/function/function_template.hpp. The number of arguments is defined by 
the macro BOOST_FUNCTION_NUM_ARGS. This conversion was relatively simple, and 
the preprocessor code for generating the above macros looks something like 
this:
--
#define BOOST_FUNCTION_TEMPLATE_PARMS     \
  BOOST_PREPROCESSOR_ENUM_SHIFTED_PARAMS( \
    BOOST_FUNCTION_NUM_ARGS_P1,           \
    typename T)
#define BOOST_FUNCTION_PARM(i,A,B)                     \
  BOOST_PREPROCESSOR_COMMA_IF(i)                       \
  BOOST_PREPROCESSOR_CAT(T,BOOST_PREPROCESSOR_INC(i)) \
  BOOST_PREPROCESSOR_CAT(a,BOOST_PREPROCESSOR_INC(i)) 
#define BOOST_FUNCTION_PARMS \
  BOOST_PREPROCESSOR_REPEAT(BOOST_FUNCTION_NUM_ARGS,BOOST_FUNCTION_PARM,A,B)
#define BOOST_FUNCTION_ARGS               \
  BOOST_PREPROCESSOR_ENUM_SHIFTED_PARAMS( \
    BOOST_FUNCTION_NUM_ARGS_P1,           \
    a)
#define BOOST_FUNCTION_TEMPLATE_ARGS      \
  BOOST_PREPROCESSOR_ENUM_SHIFTED_PARAMS( \
    BOOST_FUNCTION_NUM_ARGS_P1,           \
    T)
#define BOOST_FUNCTION_FUNCTION \
  BOOST_PREPROCESSOR_CAT(function,BOOST_FUNCTION_NUM_ARGS)
----
Notice that BOOST_FUNCTION_NUM_ARGS_P1 is used where one would expect 
BOOST_FUNCTION_NUM_ARGS? The _P1 stands for "plus one", because, to my 
surprise, BOOST_PREPROCESSOR_ENUM_SHIFTED_PARAMS emits copies for values 
1..(n-1) when given the value "n" for counting, so I had to increment the 
number of parameters to get enough template parameters. I find it very odd 
that the unshifted versions emit terms for values in the range 0..(n-1), but 
the shifted emits terms for values in the range 1..(n-1) instead of the 
shifted range 1..n. 
Handling the list of function parameters (e.g., T1 a1, T2 a2, etc) was 
relatively simple. The rest of the macros used by 
boost/function/function_template.hpp were trivial.
At this point (with some other simple macro definitions needed for 
Boost.Function), I was able to build Boost.Function numbered objects for any 
number of arguments. So far so good, so on to the tough part...
----------Conversion of boost/function.hpp----------
This file has a construct that requires a "nested loop" of preprocessor 
output. Essentially, Boost.Function will support up to "N" parameters. 
However, the user may supply 0 <= M <= N parameters. A mapping is performed 
to take the M parameters as given to the boost::function class template and 
return one of the numbered variants created above. For instance, for
boost::function<void, int, int> there are two parameters, so I would need to 
get boost::function2<void, int, int>. For boost::function<void, int> there is 
only one parameter, so I would need to retrieve boost::function1<void, int>. 
Without partial specialization, one way to handle this is to count the number 
of arguments used, specialize a template with that number, and use a member 
template to get the actual class. Here is the pattern that needs to be 
instantiated (N arguments possible, M actual)
--
template<> 
struct real_get_function_impl<M> {
  template<
    typename R,
    typename T1,
    typename T2,
    ...
    typename TN>
  struct params {
    typedef functionM<R,T1,T2,...,TM> type;
  }
};
--
To instantiate this pattern for a single M, we need to loop over the 
"typename Ti" term N times, and the "Tj" term M times. Since we need to loop 
over all 0 <= M <= N, we require BOOST_PREPROCESSOR_2ND_REPEAT for the outer 
loop. In the end, the code to instantiate one of these patterns looks like 
this:
--
#define REAL_GET_FUNCTION_IMPL_SPEC(i,A,B)                      \
  template<>                                                    \
  struct real_get_function_impl<i>                              \
  {                                                             \
    template<                                                   \
      typename R BOOST_PREPROCESSOR_IF(BOOST_FUNCTION_NUM_ARGS, \
                   BOOST_PREPROCESSOR_COMMA,                    \
                   BOOST_PREPROCESSOR_EMPTY)()                  \
      BOOST_PREPROCESSOR_ENUM_SHIFTED_PARAMS(                   \
        BOOST_FUNCTION_NUM_ARGS_P1,                             \
        typename T),                                            \
      typename Policy,                                          \
      typename Mixin,                                           \
      typename Allocator                                        \
    >                                                           \
    struct params {                                             \
      typedef BOOST_PREPROCESSOR_CAT(function,i)<R              \
                BOOST_PREPROCESSOR_IF(i,                        \
                  BOOST_PREPROCESSOR_COMMA,                     \
                  BOOST_PREPROCESSOR_EMPTY)()                   \
                BOOST_PREPROCESSOR_ENUM_SHIFTED_PARAMS(         \
                  BOOST_PREPROCESSOR_INC(i),                    \
                  T),                                           \
                Policy, Mixin, Allocator> type;                 \
    };                                                          \
  };
--
This took me about an hour to construct. Most of that time was spent chasing 
down the extra "()" strings that occurred in the output. The culprit? I 
didn't realize the difference bettween BOOST_PREPROCESSOR_UNUSED and 
BOOST_PREPROCESSOR_EMPTY. The format is just replaced with space, whereas the 
latter needs to be evaluated (i.e., BOOST_PREPROCESSOR_EMPTY()) to get the 
empty space. Since BOOST_PREPROCESSOR_IF evaluates its result, 
BOOST_PREPROCESSOR_EMPTY must be used. This was a _huge_ hurdle and I humbly 
request an addition to the documentation. 
Anyway, the REAL_GET_FUNCTION_IMPL_SPEC macro creates one pattern, so we use 
the second repeat macro to generate all of the specializations we need:
--
BOOST_PREPROCESSOR_2ND_REPEAT(
  BOOST_FUNCTION_NUM_ARGS_P1,
  REAL_GET_FUNCTION_IMPL_SPEC,
  A,B)
--
----------Specific Comments--------
Usage & headers:
	- The enumeration of shifted parameters should go from 1 to n, not 1 to 
(n-1). The current behavior is quite surprising and I don't see any rationale 
behind it.
	- Perhaps a "enumerate shifted parameters with defaults" should be added, to 
be consistent with the unshifted version.
	- A header file "boost/preprocessor.hpp" that pulls in all of the 
preprocessor code would be most appreciated. I found that I was including 
almost every file in the library individually when converting Boost.Function. 
	- Each of the headers needs a reference to http://www.boost.org
Documentation:
	- A reference-style section that lists each user-level macro and describes 
its parameters and expansion would be very useful. I found myself looking to 
the code to see what the parameters to BOOST_PREPROCESSOR_REPEAT were.
	- The macro names in the tutorial don't have the BOOST_PREPROCESSOR_ prefix, 
but they probably should.
	- A discussion of BOOST_PREPROCESSOR_UNUSED vs. BOOST_PREPROCESSOR_EMPTY 
would be invaluable. Even if a a single sentence in the descriptions of the 
macros they would be used with (i.e., BOOST_PREPROCESSOR_IF) stating which 
one to use would be helpful.
	- In the "Notes" for the example "Use a Local Macro to avoid small scale 
repetition", there is a typo in "defining more and different kind of 
operators" ("kind" should be plural).
	Doug

Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk