[boost] File-Iteration Synopsis

From: Paul Mensonides (pmenso57_at_[hidden])
Date: 2002-06-20 00:09:33

The files have been added to the CVS.  Please remember that these files are
somewhat preliminary.  They should be fully functional, but some names might
change based on initial use.

What follows is a relatively detailed explanation of the mechanism...

The key idea of file-iteration is use of a file like it is macro.  Just as
macros can have arguments, so can files--in the form of macros.  In other words,
macros are to files what arguments are to macros.

What the file-iteration does is repeatedly include a file, supplying various
arguments to file as macros, starting with a lower bound and continuing until it
reaches an upper bound.  Each time, it supplies the file with the current
iteration number.

The primary purpose of the file-iteration mechanism is to support 'vertical
repetition.'  Macro-recursion already supports 'horizontal repetition,' but
there are several downsides to that approach (as well as several upsides).  In a
sense, file-iteration fills in some of the gaps in the iteration support offered
by macro-recursion.

There are two major constructs provided by the library for this purpose.  The
first is a local iteration mechanism which iterates over a user-defined macro.
The other, more complex, mechanism iterates over files to multiple depths.

The simple mechanism is useful for iterations of small amounts of text, and use
of it is fairly simple.  The user must provide, at minimum, a unary macro to
expand and the upper bound of the iteration.  The lower bound may also be
supplied, but it is optional (the library defaults to 0).

The macro to be expanded *must* have the name BOOST_PP_LOCAL_MACRO and have a
unary signature.  This macro will be expanded with numbers ranging from
BOOST_PP_LOCAL_START to BOOST_PP_LOCAL_FINISH.  This is accomplished by
*including* BOOST_PP_LOCAL_ITERATE(), which implements the iteration.  All of
these macros will automatically be undefined by the library following the
iteration.

So, a simple use of the mechanism would be as follows:

#include <boost/preprocessor/iterate.hpp>

template<int> struct sample;

#define BOOST_PP_LOCAL_MACRO(n) \
    template<> struct sample<n> { };

#define BOOST_PP_LOCAL_FINISH 10

#include BOOST_PP_LOCAL_ITERATE()

This will generate specializations of the 'sample' template from 0 to 10 as
follows:

template<> struct sample<0> { };
template<> struct sample<1> { };
template<> struct sample<2> { };
// .. etc.
template<> struct sample<10> { };

Also note that it is possible for boundaries to be simple arithmetic expressions
that the preprocessor can evaluate.  For example:

#define MY_LIBRARY_LIMIT 15
#define BOOST_PP_LOCAL_FINISH MY_LIBRARY_LIMIT - 1

This is possible because the arguments are evaluated by #if directives.

Remember that the BOOST_PP_LOCAL_START, BOOST_PP_LOCAL_FINISH, and
BOOST_PP_LOCAL_MACRO are all undefined following the iteration.  If you wish to
maintain those values, you must define them as other macros as follows:

#define MY_LIBRARY_LOWER_BOUND 1
#define MY_LIBRARY_UPPER_BOUND 15

#define MY_LIBRARY_MACRO(n) \
    template<> struct sample<n> { };

#define BOOST_PP_LOCAL_START MY_LIBRARY_LOWER_BOUND
#define BOOST_PP_LOCAL_FINISH MY_LIBRARY_UPPER_BOUND

#define BOOST_PP_LOCAL_MACRO(n) MY_LIBRARY_MACRO(n)

The second and more involved mechanism uses files rather than macros.

In order to use the second mechanism, again two things *must* be defined--an
upper bound for the iteration and a filename to iterate over.  A lower bound can
also be supplied, but again it defaults to 0.  Setting the upper and lower
boundaries is a bit more complex than in the first mechanism.

#define BOOST_PP_ITERATION_BOUND 1
#include BOOST_PP_SET_ITERATION_START()

#define BOOST_PP_ITERATION_BOUND 100
#include BOOST_PP_SET_ITERATION_FINISH()

Notice that there are only two macro definitions, but there are also two
*includes*.  This is necessary in order to abstract the depth of iteration (more
on this below).

Setting the filename is slightly easier.

#define BOOST_PP_FILENAME_1 "somefile.h"

It is possible for the file specified to be the current file, but I will get
into that later.

At this point all that needs to be done is to start the iteration:

#include BOOST_PP_ITERATE()

This will cause the file specified by BOOST_PP_FILENAME_1 to be included
repeatedly from the lower bound to the upper bound.  Each time, the current
iteration is passed to the file in the form of a macro:  BOOST_PP_ITERATION().

A simple example follows:

// file1.hpp
#include <boost/preprocessor/comma_if.hpp>
#include <boost/preprocessor/repeat.hpp>

#include <boost/preprocessor/iterate.hpp>

template<int> struct sample;

#define PARAM(i, data) BOOST_PP_COMMA_IF(i) class T ## i

#define BOOST_PP_ITERATION_BOUND 1
#include BOOST_PP_SET_ITERATION_START()

#define BOOST_PP_ITERATION_BOUND 10
#include BOOST_PP_SET_ITERATION_FINISH()

#define BOOST_PP_FILENAME_1 "file2.hpp"
#include BOOST_PP_ITERATE()

#undef PARAM

// file2
template<> struct sample<BOOST_PP_ITERATION()> {
    template<BOOST_PP_REPEAT(BOOST_PP_ITERATION(), PARAM, nil)>
    struct args { };
};

This will generate the following:

template<> struct sample<1> {
    template<class T0>
    struct args { };
};

template<> struct sample<2> {
    template<class T0, class T1>
    struct args { };
};

// etc. up to 10 arguments

Inside the iterated file, the lower and upper bounds of the iteration are
available via the macros BOOST_PP_ITERATION_START() and
BOOST_PP_ITERATION_FINISH() respectively.

As I said earlier, it is possible to use the current file as the target of the
iteration, but for obvious reasons, care must be taken to prevent infinite
inclusion of the same file.  To help with this situation, the library defines
the macro BOOST_PP_IS_ITERATING whenever it begins an iteration and undefines it
when it finishes.  So the the basic structure of such a file would be like this:

// file3.hpp
#if !defined(BOOST_PP_IS_ITERATING)
    // contents of file1.hpp above go here
    // with only this different:
    #define BOOST_PP_FILENAME_1 "file3.hpp"
#else
    // contents of file2.hpp above go here
#endif

It is *imperative* that the filename *not* be the predefined macro __FILE__.
This will not work because lazy-evaluation would cause the iteration
implementation to try to include itself.

The library also defines a useful debugging aid in the form of a line directive
expansion.  This will append the current iteration number(s) to the filename.
It is used, in an iterated section, as follows:

#line BOOST_PP_LINE(__LINE__, file3.hpp) // without quotes

Using the __LINE__ macro will cause the line number to get off by one, because
the #line directive specifies what the *following* line number will be.  This
discrepancy is minor compared to manually keeping track of the real line number
at that point.  Line directives will be generated as follows:

#line __LINE__ "file3.hpp[1]"

#line __LINE__ "file3.hpp[2]"

// etc.

So the above can be amended as follows...

// file3.hpp
#if !defined(BOOST_PP_IS_ITERATING)
    // contents of file1.hpp above go here
    // with only this different:
    #define BOOST_PP_FILENAME_1 "file3.hpp"
#else
    #line BOOST_PP_LINE(__LINE__, file3.hpp)
    // contents of file2.hpp above go here
#endif

Multidimensional iteration is also supported by the library.  The current
dimension of iteration is available via the BOOST_PP_ITERATION_DEPTH() macro.
So a multidimensional iteration in one file would look something like this:

#if !defined(BOOST_PP_IS_ITERATING)
    // normal file
#elif BOOST_PP_ITERATION_DEPTH() == 1
    // iterated section for the 1st depth
#elif BOOST_PP_ITERATION_DEPTH() == 2
    // iterated section for the 2nd depth
#endif

Everything is the same for subsequent depths of iteration--except a different
macro must be defined for the filename.  For instance, to begin a second level
of iteration, BOOST_PP_FILENAME_2 must be defined rather than
BOOST_PP_FILENAME_1.  Otherwise, the usage is the same.

Inside of nested iterations however, there is more information available.  For
instance, the outer iteration index that the current iteration is nested in is
available via the BOOST_PP_FRAME_ITERATION(1).  The iteration indexes further
back are available via BOOST_PP_FRAME_ITERATION(2), BOOST_PP_FRAME_ITERATION(3),
etc..  BOOST_PP_FRAME_ITERATION(0) is the same value as BOOST_PP_ITERATION().

The lower and upper bounds of outer iterations are similarly available via the
BOOST_PP_FRAME_START(i) and BOOST_PP_FRAME_FINISH(i).  As before,
BOOST_PP_FRAME_START(0) is identical to BOOST_PP_ITERATION_START().

If a BOOST_PP_LINE 'directive' appears in a nested iteration, *every* available
iteration index is appended to the filename.  E.g.

#line __LINE__ "file.hpp[1][2][3]"

This specifies that the file is on the '3rd' iteration of the '2nd' iteration of
the '1st' iteration.

Because the filename cannot be abstracted like the upper and lower bounds, a
file that can be used in an arbitrary position (depth-wise) that iterates again
must manually abstract the filename to be iterated itself:

#if BOOST_PP_ITERATION_DEPTH() == 1
    #define BOOST_PP_FILENAME_2 "file.hpp"
#elif BOOST_PP_ITERATION_DEPTH() == 2
    #define BOOST_PP_FILENAME_3 "file.hpp"
#elif BOOST_PP_ITERATION_DEPTH() == 3
    #define BOOST_PP_FILENAME_4 "file.hpp"
#else
    #define BOOST_PP_FILENAME_5 "file.hpp"
#endif

Here is an example that uses multiple dimensions of iteration:

// file4.hpp

#if !defined(BOOST_PP_IS_ITERATING)

#ifndef FILE4_HPP
#define FILE4_HPP

    #include <boost/preprocessor/cat.hpp>
    #include <boost/preprocessor/comma_if.hpp>
    #include <boost/preprocessor/repeat.hpp>
    #include <boost/preprocessor/iterate.hpp>

    template<int> struct type_tuple;

    #define PARAM(n, unused) BOOST_PP_COMMA_IF(n) class BOOST_PP_CAT(T, n)

    #define BOOST_PP_ITERATION_BOUND 1
    #include BOOST_PP_SET_ITERATION_START()

    #define BOOST_PP_ITERATION_BOUND 15
    #include BOOST_PP_SET_ITERATION_FINISH()

    #define BOOST_PP_FILENAME_1 "file4.hpp"
    #include BOOST_PP_ITERATE()

#endif

#elif BOOST_PP_ITERATION_DEPTH() == 1
    #line BOOST_PP_LINE(__LINE__, file4.hpp)
    #define C BOOST_PP_ITERATION()

    template<> struct type_tuple<C> {
        template<BOOST_PP_REPEAT(C, PARAM, unused)>
        struct args {
            template<int, int D = 0> struct index;

            #define BOOST_PP_ITERATION_BOUND 0
            #include BOOST_PP_SET_ITERATION_START()

            #define BOOST_PP_ITERATION_BOUND C - 1
            #include BOOST_PP_SET_ITERATION_FINISH()

            #define BOOST_PP_FILENAME_2 BOOST_PP_FILENAME_1
            #include BOOST_PP_ITERATE()

        };
    };

    #undef C
#else
    #line BOOST_PP_LINE(__LINE__, file4.hpp)

            template<int D> struct index<BOOST_PP_ITERATION()> {
                typedef BOOST_PP_CAT(T, BOOST_PP_ITERATION()) type;
            };

#endif

This will generate structures like the following:

template<> struct type_tuple<3> {
    template<class T0, class T1, class T2>
    struct args {
        template<int, int D = 0> struct index;
        template<int D> struct index<0> {
            typedef T0 type;
        };
        template<int D> struct index<1> {
            typedef T1 type;
        };
        template<int D> struct index<2> {
            typedef T2 type;
        };
    };
};

This structure might be used as follows:

typedef type_tuple<3>::args<int, short, double> types;

types::index<0> x = 0;
types::index<1> y = 0;
types::index<2> z = 0.0;

Feel free to request clarifications or to make comments.

Paul Mensonides