THE BOOST TYPEOF LIBRARY

Motivation

Why typeof?

Native typeof support

Why typeof emulation?

Example

The 3 participating parties

Features supported

STL support

Registration

Usage

What needs to be registered?

Compilers supported

Limits

Known limitations

Contributed by:

Acknowledgements

 

Motivation

Why typeof?

Today many template libraries supply object generators to simplify object creation by utilizing the C++ template argument deduction facility.  Consider std::pair.  In order to instantiate this class template and create a temporary object of this instantiation, one has to supply both type parameters and value parameters:

 

std::pair<int, double>(5, 3.14159);

 

To avoid this duplication, STL supplies the std::make_pair object generator.  When it is used, the types of template parameters are deduced from supplied function arguments:

 

std::make_pair(5, 3.14159);

 

For the temporary objects it is enough.  However, when a named object needs to be allocated, the problem appears again:

 

std::pair<int, double> p(5, 3.14159);

 

The object generator no longer helps:

 

std::pair<int, double> p = std::make_pair(5, 3.14159);

 

It would be nice to deduce the type of the object (on the left) from the expression it is initialized with (on the right), but the current C++ syntax doesn’t allow for this.

 

The above example demonstrates the essence of the problem but doesn’t demonstrate its scale.  Many libraries, especially expression template libraries, create objects of really complicated types, and go a long way to hide this complexity behind object generators.  Consider a nit Boost.Lambda functor:

 

_1 > 15 && _2 < 20

 

If one wanted to allocate a named copy of such an innocently looking functor, she would have to specify something like this:

 

lambda_functor<
      lambda_functor_base<
            logical_action<and_action>,
            tuple<
                  lambda_functor<
                        lambda_functor_base<
                              relational_action<greater_action>,
                              tuple<
                                    lambda_functor<placeholder<1> >,
                                    int const
                              >
                        >
                  >,
                  lambda_functor<
                        lambda_functor_base<
                              relational_action<less_action>,
                              tuple<
                                    lambda_functor<placeholder<2> >,
                                    int const
                              >
                        >
                  >
            >
      >

f = _1 > 15 && _2 < 20;

 

Not exactly elegant.  To solve this problem, the C++ standard committee is considering a few additions to the standard language, such as typeof/decltype and auto (see http://www.osl.iu.edu/~jajarvi/publications/papers/decltype_n1478.pdf).

 

The typeof operator (or decltype, which is a slightly different flavor of typeof) allows one to determine the type of an expression at compile time.  Using typeof, the above example can be simplified drastically:

 

typeof(_1 > 15 && _2 < 20) f = _1 > 15 && _2 < 20;

 

Much better, but there still is duplication.  The auto type solves the rest of the problem:

 

auto f = _1 > 15 && _2 < 20; 

Native typeof support

Many compilers support typeof already, most noticeable GCC and Metrowerks.

 

Igor Chesnokov recently discovered a method that allows to implement typeof on the VC series of compilers.  It uses a bug in the Microsoft compiler that allows a nested class of base to be defined in a class derived from base:

 

template<int ID> struct typeof_access
{
   struct id2type; // not defined
};

template<class T, int ID> struct typeof_register : typeof_access
{
   // define base's nested class here
   struct typeof_access::id2type   {
       typedef T type;
   };
};

//Type registration function 
typeof_register<T,compile-time-constant> register_type(const T&);

//Actually register type by instantiating typeof_register for the correct type
sizeof(register_type(
some-type)); 

//Use the base class to access the type.
typedef typeof_access::id2type::type type; 

 

This method has also been adapted to VC7.0, where the nested class is a template class that is specialized in the derived class. The loopholes have been fixed in VC8.0 though, so on this compiler this method doesn’t work.

 

For this and many other compilers neither native typeof support nor the trick described above is an option.  For such compilers there has to be other way of implementing typeof.

Why typeof emulation?

According to a rough estimate, at the time of this writing (June 2004) the introduction of the typeof, auto, etc., into the C++ standard is not going to happen at least within a year.  Even after it’s done, some time still has to pass before most compilers implement this feature.  But even after that, there always are legacy compilers to support (for example now, in 2004, many people are still using VC6, long after VC7.x became available).

 

Considering extreme usefulness of the feature right now, it seems to make sense to try to emulate it at a library level.  Such emulation will inevitably have major drawbacks, such as additional responsibilities put on the library users, slower compiles, etc., but in the end, depending on the context, the benefits will likely outweigh.

 

The Typeof library implements such emulation utilizing template and preprocessor meta-programming facilities provided by Boost MPL and Preprocessor libraries.  

Example

Here is how one could re-write the above examples using BOOST_AUTO:

 

Boost.Typeof

typeof

BOOST_AUTO(p, std::make_pair(5, 3.14159)); 

auto p = std::make_pair(5, 3.14159);

BOOST_AUTO(f, _1 > 15 && _2 < 20);

auto f = _1 > 15 && _2 < 20; 

 

Looks like a pretty good approximation.  However, there is some cost.  Since, unlike the compiler, the library doesn’t know much about types the expression is combined of, the user must help by providing some information.  For the first example something like following has to be specified:

 

#include BOOST_TYPEOF_INCREMENT_REGISTRATION_GROUP()
BOOST_TYPEOF_REGISTER_TEMPLATE(std::pair, 2);

 

The first line:

 

#include BOOST_TYPEOF_INCREMENT_REGISTRATION_GROUP()

 

does pretty much what it says: increments the BOOST_TYPEOF_REGISTRATION_GROUP symbolic constant.  This constant is used by all registration macros, and must be defined to some unique integer in every source or header file that contains any registration.  Incrementing it insures that it is, indeed, unique.

 

The second line registers std::pair as a template with two type parameters. 

 

Once any types and templates are registered, the library can handle any combination of those.  Since all the fundamental types are pre-registered by the typeof library, the above registration makes it possible to also write something like this:

 

BOOST_AUTO(p, 
  std::make_pair(
    std::make_pair(5, 3.14159),
    std::make_pair(2.71828, 6)));

 

The more complicated Lambda example above requires some more registration:

 

#include BOOST_TYPEOF_INCREMENT_REGISTRATION_GROUP()
 
BOOST_TYPEOF_REGISTER_TEMPLATE(boost::tuples::tuple, 2);
BOOST_TYPEOF_REGISTER_TEMPLATE(boost::lambda::lambda_functor, 1);
BOOST_TYPEOF_REGISTER_TEMPLATE(boost::lambda::lambda_functor_base, 2);
BOOST_TYPEOF_REGISTER_TEMPLATE(boost::lambda::relational_action, 1);
BOOST_TYPEOF_REGISTER_TEMPLATE(boost::lambda::logical_action, 1);
BOOST_TYPEOF_REGISTER_TEMPLATE(boost::lambda::other_action, 1);
BOOST_TYPEOF_REGISTER_TYPE(boost::lambda::greater_action);
BOOST_TYPEOF_REGISTER_TYPE(boost::lambda::less_action);
BOOST_TYPEOF_REGISTER_TYPE(boost::lambda::and_action);
BOOST_TYPEOF_REGISTER_TEMPLATE(boost::lambda::placeholder, (int));

 

The 3 participating parties

It may seem that the price for the ability to discover the expression’s type is too high: rather large amount of registration is required.  However note that all of the above registration is done only once, and after that, any combination of the registered types and templates would be handled.  Moreover, this registration is typically done not by the end-user, but rather by a layer on top of some library (in this example -- Boost.Lambda).

 

When thinking about this, it’s helpful to consider three parties: the typeof facility, the library (probably built on expression templates principle), and the end-user.  The typeof facility is responsible for registering fundamental types.  The library can register its own types and templates.

 

In the best-case scenario, if the expressions always consist of only fundamental types and library-defined types and templates, a library author can achieve the impression that the typeof is natively supported for her library.  On the other hand, the more often expressions contain user-defined types, the more responsibility is put on the end-user, and therefore the less attractive this approach becomes. 

 

Thus, the ratio of user-defined types in the expressions should be the main factor to consider when deciding whether or not to apply this facility.

Features supported

The Boost.Typeof  library pre-registers fundamental types.  For these types, and for any other types/templates registered by the user library or end-user, any combination of the following is supported:

 

§         Pointers;

§         References (except top-level);

§         Consts (except top-level);

§         Volatiles (except top-level);

§         Arrays;

§         Functions (compliant compilers only), function pointers, and references;

§         Pointers to member functions;

§         Pointers to data members.

 

For example the following type:

 

int& (*)(const volatile char*, double[5], void(*)(short))

 

is supported right away, and something like:

 

void (MyClass::*)(int MyClass::*, MyClass[10]) const

 

is supported provided MyClass is registered.

STL support

The library also provides registration files for most STL classes/templates.  These files are located in the std subdirectory, and named after corresponding STL headers.  These files are not included by the typeof system and have to be explicitly included by the user, as needed:

 

#include <boost/typeof/std/functional.hpp>
BOOST_AUTO(fun, std::bind2nd(std::less<int>(), 21)); //create named function object for future use.

Registration

Registration may be done by either a library author or an end-user.  The purpose of the registration is to tell the typeof system about a given type or template, so that it can be handled.

 

Two macros are provided to handle the registration:

 

§         BOOST_TYPEOF_REGISTER_TYPE(Name);

§         BOOST_TYPEOF_REGISTER_TEMPLATE(Name, Params);

 

Both macros rely on the integer value defined by BOOST_TYPEOF_REGISTRATION_GROUP.  This value should be unique for each file, since it is used in conjunction with __LINE__ preprocessor macro to generate unique integer identifiers to assign to types and templates.  To establish a unique value for BOOST_TYPEOF_REGISTRATION_GROUP, the following line should be specified before the actual registration is done:

 

#include BOOST_TYPEOF_INCREMENT_REGISTRATION_GROUP()

 

A type is registered by simply supplying its name to the BOOST_TYPEOF_REGISTER_TYPE macro:

 

BOOST_TYPEOF_REGISTER_TYPE(MyType); 

 

A template is registered with the BOOST_TYPEOF_REGISTER_TEMPLATE macro.  The first parameter to this macro is the name of the template.  The second parameter describes the parameters of the template, and may come in one of two forms:

 

If the template has only type parameters, then just the number of parameters can be supplied:

 

template<class T, class U>
class MyTemplate;
 
BOOST_TYPEOF_REGISTER_TEMPLATE(MyTemplate, 2)

 

In more general case, the parameters have to be described with a preprocessor sequence:

 

template<class T, unsigned int n>
class MyTemplate;
 
BOOST_TYPEOF_REGISTER_TEMPLATE(MyTemplate, (class)(unsigned int))

 

The library allows registration of templates with type, integral, and template template parameters:

 

§         A type template parameter is described by the (class) or (typename) sequence element

§         A template parameter of a well-known integral type can be described by simply supplying its type, like (unsigned int).  The following well-known integral types are supported: 

o       [signed/unsigned] char

o       [unsigned] short

o       [unsigned] int

o       [unsigned] long

o       unsigned

o       bool

o       size_t

§         Other integral types, such as enums, need to be described explicitly with the BOOST_TYPEOF_INTEGRAL macro, like (BOOST_TYPEOF_INTEGRAL(MyEnum))

§         Template template parameters are described with the BOOST_TYPEOF_TEMPLATE macro, like: (BOOST_TYPEOF_TEMPLATE((class)(unsigned int))).  In case of all type parameters this can be shortened to something like (BOOST_TYPEOF_TEMPLATE(2)).  The nested template template parameters are not supported.

 

For example:

 

enum MyEnum {one, two, three};
 
template<class T, unsigned int n>
class A;
 
template<template<class, unsigned int> class Tpl, MyEnum e>
class B;
 
BOOST_TYPEOF_REGISTER_TEMPLATE(A, (class)(unsigned int))
 
BOOST_TYPEOF_REGISTER_TEMPLATE(B,
      (BOOST_TYPEOF_TEMPLATE((class)(unsigned int)))
      (BOOST_TYPEOF_INTEGRAL(MyEnum))
)

 

The library also supports dependent template parameters:

 

template<class T, T n>
struct A;
 
BOOST_TYPEOF_REGISTER_TEMPLATE(A,
      (class)
      (BOOST_TYPEOF_INTEGRAL(P0))
);

 

A template is encoded by combining its integer identifier with results of encoding its parameters.  If a template has default parameters, it might be a good idea to register it more than once:

 

BOOST_TYPEOF_REGISTER_TEMPLATE(std::set, 3);
BOOST_TYPEOF_REGISTER_TEMPLATE(std::set, 1);

 

The above would result in two different pairs of template specializations, the latter one – more specialized.  Consider std::set<int[, std::less<int>, std::allocator<int> ]>.  In general this type would be encoded by 6 integers:

 

std::set, int, std::less, int, std::allocator, int

 

However, because the second registration was done, in this particular case only two integers would be required, thus saving on default parameters:

 

std::set, int

Usage

The usage is pretty straightforward.  BOOST_TYPEOF is used to discover the type of the expression, like following:

 

BOOST_TYPEOF(f) f1; // compare with: typeof(f) f1;
f1 = f;


BOOST_AUTO is used to allocate a named object initialized with a given expression such as:

 

BOOST_AUTO(f, _1 > 15 && _2 < 20); // compare with: auto f = _1 > 15 && _2 < 20;

 

A (const) reference can be allocated like this:

 

const int& foo()
{
      static int n;
      return n;
}
 
//later:
BOOST_AUTO(const& var, foo()); // compare with: auto const& var = foo();

 

BOOST_TYPEOF_TPL and BOOST_AUTO_TPL, accordingly, should be used in templates when the expression depends on template parameters.  These macros take care of adding or avoiding the keyword “typename”, depending on the compiler, as well as on whether the native typeof support or emulation is being used.

 

Another macro, BOOST_LVALUE_TYPEOF, is an attempt to emulate the “decltype”.  Its rules, however, are directly built on the rules of binding a reference, and therefore differ from the ones of real “decltype”:

 

expr

Decltype

BOOST_LVALUE _TYPEOF

int i;

Int

int&

const int ci = 0;

const int

const int&

int& ri = I

int&

int&

const int& cri = ci;

const int&

const int&

int foo();

Int

int

const int foo();

const int

const int&

int& foo();

int&

int&

const int& foo();

const int&

const int&

21

int

compiler-dependent

int(21)

int

int


Of all these rules, the highlighted one seems to be the most unfortunate.

What needs to be registered?

It is possible to take advantage of the compiler when registering types for the typeof library. Even though there is currently no direct support for typeof in the language, the compiler is aware of what the type of an expression is, and gives an error if it encounters an expression that has not been handled correctly. In the typeof context, this error message will contain clues to what types needs to be registered with the typeof library in order for BOOST_TYPEOF to work.

struct X {};
 
template<typename A,bool B>
struct Y {};

std::pair<X,Y<int,true> > a;

BOOST_AUTO(a,b);

 

We get the following error message from VC7.1

 

error C2504: 'boost::type_of::`anonymous-namespace'::encode_type_impl<V,Type_Not_Registered_With_Typeof_System>' : base
class undefined
      with
      [
            V=boost::type_of::`anonymous-namespace'::encode_type_impl<boost::mpl::vector0<boost::mpl::na>,std::pair<X,Y<int,true>>>::V0,
            Type_Not_Registered_With_Typeof_System=X
      ]

 

Inspecting this error message, we see that the compiler complains about X

 

BOOST_TYPEOF_REGISTER_TYPE(X); //register X with the typeof system


Recompiling, we get a new error message from VC7.1

 

error C2504: 'boost::type_of::`anonymous-namespace'::encode_type_impl<V,Type_Not_Registered_With_Typeof_System>' : base
class undefined
      with
      [
            V=boost::type_of::`anonymous-namespace'::encode_type_impl<boost::mpl::vector0<boost::mpl::na>,std::pair<X,Y<int,true>>>::V1,
            Type_Not_Registered_With_Typeof_System=Y<int,true>
      ]

 

Inspecting this error message, we see that the compiler complains about Y<int,true>.
Since Y is a template, and contains integral constants, we need to take more care when registering:

BOOST_TYPEOF_REGISTER_TEMPLATE(Y,(typename)(bool)); //register template class Y

 

It is a good idea to look up the exact definition of Y when it contains integral constants. For simple template classes containing only typenames, you can rely solely on the compiler error.


The above code now compiles.

This technique can be used to get an overview of which types needs to be registered for a given project in order to support typeof.

Compilers supported

The system has been tested with the following compilers:

§         MSVC 6.5/7.0/7.1/8.0;

§         GCC 3.4.2

Limits

§         The number of nodes in the expression is limited by BOOST_TYPEOF_LIMIT_SIZE, by default 50.  This number can be increased, but, depending on the compiler, there is upper limit to this.

§         The number of function parameters supported is limited by BOOST_TYPEOF_LIMIT_FUNCTION_ARITY, by default 10.  This applies to functions, function pointers and references, and to pointers to member functions.

Known limitations

Nested template template parameters are not supported, like:

 

template<template<template<class> class> class Tpl>
class A; // can’t register!

 

Classes and templates nested inside other templates also can’t be registered because of the issue of nondeduced context.  This limitation is most noticeable with regards to standard iterators in Dinkumware STL, which are implemented as nested classes.  Instead, instantiations can be registered:

 

BOOST_TYPEOF_REGISTER_TYPE(std::list<int>::const_iterator)

Contributed by:

§         Compliant compilers         --          Arkadiy Vertleyb, Peder Holt

§         MSVC 6.5, 7.0, 7.1           --          Igor Chesnokov, Peder Holt

Acknowledgements

The idea of representing a type as multiple compile-time integers, and passing these integers across function boundaries using sizeof(), was taken from Steve Dewhurst’s article “A Bitwise typeof Operator”, CUJ 2002.

 

Special thank you to Paul Mensonides, Vesa Karvonen, and Aleksey Gurtovoy for the Boost Preprocessor Library and MPL.  Without these two libraries, this typeof implementation would not exist. 

 

The following people provided support, gave valuable comments, or in any other way contributed to the library development (in alphabetical order):

 

§         David Abrahams;

§         Andrey Beliakov;

§         Joel de Guzman;

§         Daniel James;

§         Vesa Karvonen;

§         Paul Mensonides;

§         Alexander Nasonov;

§         Martin Wille.

 

Copyright © 2004 - 2005 Arkadiy Vertleyb

Copyright © 2005 Peder Holt