Re: [boost] BOOST_PP_ITERATE for huge mpl and fusion

Subject: Re: [boost] BOOST_PP_ITERATE for huge mpl and fusion
From: Aleksey Gurtovoy (agurtovoy_at_[hidden])
Date: 2012-03-20 02:55:32

Hi Christopher,

On Wed, 07 Mar 2012 06:08:01 -0600, Hite, Christopher  
<Christopher.Hite_at_[hidden]> wrote:

> "Not to 50!!!" - Princess Bride

:)

> Things like mpl::vector and fusion::map have a configured limit like  
> BOOST_MPL_LIMIT_VECTOR_SIZE.

Just to clarify, BOOST_MPL_LIMIT_VECTOR_SIZE determines the maximum arity  
of the vector's "constructor" in its variadic form (i.e. vector<>).  
Somewhat contrary to its name, it does *not* determine the vector's  
maximum size. Rather, the latter is governed by what's available to the  
library on the compiler side:

  - on compilers that support typeof [1], there is really no predefined  
limit for the vector's size other than general compiler limitations on the  
template nesting depth and the likes.

  - on compilers w/o typeof support the default limit is 50, as you've  
discovered. The original rationale for that limit is cited in one of the  
posts you are linking to below:

      Increasing the default limit to, let's say, 100 elements means plus  
~120 KB to the
      size of the distribution, and I'm not sure the need is widespread  
enough to make
      everybody pay for it.

    Given the current size of the distribution nowadays, this looks like a  
non-issue.

>
> The documentation is dodgy and doesn't mention that while it is  
> configurable you can't really configure it to be more than 50.

The docs on this topic are definitely lacking. Let me know if this post  
clarifies things for you, and I'll rework it into a doc page.

> Many people have stumbled here:
> http://boost.2283326.n4.nabble.com/msm-Boost-MPL-vector-limit-size-exceeded-td2600696.html
> http://lists.boost.org/Archives/boost/2004/08/70900.php
> http://comments.gmane.org/gmane.comp.lib.boost.user/55679
> http://lists.boost.org/Archives/boost/2010/09/170443.php
> Appearantly there are some scripts to generate these headers, but they  
> may not work anymore.

They do, at least on Windows, and they are here:  
http://svn.boost.org/svn/boost/trunk/libs/mpl/preprocessed/

>
> I had to rush hack things in my code to make several containers go over  
> 50 in one compilation scope.
>
> #define ULTIMATE_N 70
> #include <boost/mpl/vector/vector50.hpp>
>
> namespace boost { namespace mpl {
>
> #   define BOOST_PP_ITERATION_PARAMS_1 \
> 	(3,(51, ULTIMATE_N, <boost/mpl/vector/aux_/numbered.hpp>))
> #   include BOOST_PP_ITERATE()
>
> }}
>

This is actually a legit way of going over the predefined limit; there is  
really nothing wrong with putting this in a header and including it where  
you need vectors over 50 elements. The only downside of using this instead  
of pre-generated preprocessed headers is potential loss of compilation  
speed, emphasis on potential.

> I'd really like to fix this in boost.  AFAICT all I have to do is find  
> all *50.hpp files and replace the 50 with the appropriate config limit.
>
> I count 74 instances of this stuff going on!  Mostly it's phoenix, mpl,  
> and fusion.
>
> So to wrap it up my questions are:
> 1) Do the scripts still work?  Does anyone use them?  Are they regularly  
> tested?

The scripts still work, they are used when a fix is made in one of the  
headers affecting the preprocessed output, and no, they are not regularly  
tested, but they also don't regularly break :).

> 2) Does anyone see a problem with my simple fix?

I can't really speak for the other libraries, but I have no problem with  
bumping up the default MPL limit to, say, 100.

[1] ... and decltype, once you apply a four-lines patch.

-- 
Aleksey Gurtovoy
MetaCommunications Engineering