Re: [boost] BOOST_PP_ITERATE for huge mpl and fusion

Subject: Re: [boost] BOOST_PP_ITERATE for huge mpl and fusion
From: Hite, Christopher (Christopher.Hite_at_[hidden])
Date: 2012-03-20 07:35:21

Aleksey Gurtovoy wrote:
> Things like mpl::vector and fusion::map have a configured limit like 
> BOOST_MPL_LIMIT_VECTOR_SIZE.

> Just to clarify, BOOST_MPL_LIMIT_VECTOR_SIZE determines the maximum arity of the vector's "constructor" in its variadic form (i.e. vector<>).  
> Somewhat contrary to its name, it does *not* determine the vector's maximum size. Rather, the latter is governed by what's available to the library on the compiler side:

>   - on compilers that support typeof [1], there is really no predefined limit for the vector's size other than general compiler limitations on the template nesting depth and the likes.

>   - on compilers w/o typeof support the default limit is 50, as you've discovered. The original rationale for that limit is cited in one of the posts you are linking to below:

>       Increasing the default limit to, let's say, 100 elements means plus ~120 KB to the
>       size of the distribution, and I'm not sure the need is widespread enough to make
>       everybody pay for it.

>     Given the current size of the distribution nowadays, this looks like a non-issue.

> The docs on this topic are definitely lacking. Let me know if this post clarifies things for you, and I'll rework it into a doc page.
Thanks, those few lines there already help a lot!

I know you're not responsible for fusion, but I also don't know if it needs the "constructor".

> > I had to rush hack things in my code to make several containers go 
> > over 50 in one compilation scope.

> This is actually a legit way of going over the predefined limit; there is really nothing wrong with putting this in a header and including it where you need vectors over 50 elements. The only downside of using this instead of pre-generated preprocessed headers is potential loss of compilation > > speed, emphasis on potential.
It may work for MPL I just upgraded boost and fusion now checks the limit and complains:

boost_1_49_0/boost/fusion/container/map/detail/preprocessed/as_map.hpp:21:2: error: #error "FUSION_MAX_MAP_SIZE out of bounds for preprocessed headers"

I spent about an hour trying to hack around that.  It looks like I'll either have to change boost or copy paste a good deal of fusion's details into a header.

I realize you're not repsonsible for fusion, but as far as I can tell the other libraries copied your techniques and if we figure this out for MPL we can apply it to them too.

I think we could fix all of these in the library by just changing the loops like:

#   define BOOST_PP_ITERATION_PARAMS_1 \
    (3,(41, 50, <boost/mpl/map/aux_/numbered.hpp>))

to 

#   define BOOST_PP_ITERATION_PARAMS_1 \
    (3,(41, BOOST_MPL_LIMIT_MAP_SIZE, <boost/mpl/map/aux_/numbered.hpp>))

so the last file loops as often as the user wants.  It just seems like you snapped defeat from the jaws of victory by putting the limit in.  Is there something I missed like some compiler's preprocessor not handling that variable?

Would it require you to change your scripts?

> They do, at least on Windows, and they are here:  
> http://svn.boost.org/svn/boost/trunk/libs/mpl/preprocessed/
...
> The scripts still work, they are used when a fix is made in one of the headers affecting the preprocessed output, and no, they are not regularly tested, but they also don't regularly break :).

Are you talking about boost_1_49_0/libs/mpl/preprocessed/preprocess.cmd ?  Where do I put the N?

> I can't really speak for the other libraries, but I have no problem with bumping up the default MPL limit to, say, 100.
That would take care of my project, but I'd hate to come back later and ask you for more.

> [1] ... and decltype, once you apply a four-lines patch.
Perhaps I should go fight for c++11.  Would the four-lines be #ifdef'ed in future versions of boost?

Thanks for helping me!

Chris