Re: [Boost-Users] Re: Preprocessor library question

From: Tanton Gibbs (thgibbs_at_[hidden])
Date: 2002-12-19 18:56:34

Thanks for your response, I'll look through it and try to get a full grasp
on everything you said.

Basically, my problem is this

I have to read in a file of fixed length records, manipulate them (e.g.,
sort, dedup, group, some custom processing ) and then write them out.  There
are about 40M records, so I want it to be as fast as possible.  If I know
the length of the record fields ahead of time, it is MUCH more efficient.
However, I would like the field lengths to be customizable at runtime...I
thought if I could precompute the program for various values of the 3
fields, then I could have the best of both worlds.  However, I forgot about
all the template instantiations...perhaps they'll just have to recompile
when they want a new field length.

Tanton
----- Original Message -----
From: "Paul Mensonides" <yg-boost-users_at_[hidden]>
To: <boost-users_at_[hidden]>
Sent: Thursday, December 19, 2002 1:56 AM
Subject: [Boost-Users] Re: Preprocessor library question

>
> "Tanton Gibbs" <thgibbs_at_[hidden]> wrote in message
> news:024e01c2a725$6814d720$b048cc18_at_Corp.Acxiom.net...
> > I'm trying to create a template class based on user input.  For example:
> >
> > template< int i, int j, int k > class T {};
> >
> > int main() {
> >   int i, j, k;
> >   cin >> i >> j >> k;
> >   T<i, j, k> t;
> > }
> >
> > Naturally, this is not possible, as i, j, and k must be constants.
> Therefore, I would like to limit the maximum value of i, j, and k to 100
and
> create all permutations for T<i,,j,k> at compile time.  Basically,
something
> like this:
>
> Hi Tanton,
>
> You realize that this will give you a 101*101*101 permutation right?
While
> the pp-lib can generate it, that is a pretty big block of code to be
inside
> a single function.  I would try to separate it into separate functions if
> possible:
>
> template<int i, int j, int k> struct bound {
>     static void target() {
>         T<i, j, k> t;
>         // ...
>     }
> };
>
> typedef void (* pf)();
>
> template<int i, int j> pf bind(int k) {
>     switch (k) {
>         case 0:
>             return &bound<i, j, 0>::target;
>             break;
>         case 1:
>             return &bound<i, j, 1>::target;
>             break;
>
>         // ...
>
>         default:
>             return 0;
>     }
> }
>
> template<int i> pf bind(int j, int k) {
>     switch (j) {
>         case 0:
>             return bind<i, 0>(k);
>         case 1:
>             return bind<i, 1>(k);
>
>         // ...
>
>         default:
>             return 0;
>     }
> }
>
> pf bind(int i, int j, int k) {
>     switch (i) {
>         case 0:
>             return bind<0>(j, k);
>         case 1:
>             return bind<1>(j, k);
>
>         // ...
>
>         default:
>             return 0;
>     }
> }
>
> int main() {
>     int i, j, k;
>     std::cin >> i >> j >> k;
>     bind(i, j, k)();
>     // ...
> }
>
> Will something like that work for you?  Something similar to this is *way*
> better than having an if structure with 101*101*101 conditional tests.  If
> so, here is an implementation:
>
> #include <iostream>
> #include <boost/preprocessor/iteration/local.hpp>
>
> #define LIMIT 100
>
> template<int i, int j, int k> struct bound {
>     static void target() {
>         std::cout << i << ' ' << j << ' ' << k << &std::endl;
>         return;
>     }
> };
>
> void overflow() {
>     std::cout << "overflow" << &std::endl;
>     return;
> }
>
> typedef void (* pbound_f)();
>
> template<int i, int j> pbound_f bind(int k) {
>     switch (k) {
>         #define BOOST_PP_LOCAL_MACRO(k) \
>             case k: \
>                 return &bound<i, j, k>::target; \
>             /**/
>         #define BOOST_PP_LOCAL_LIMITS (0, LIMIT)
>         #include BOOST_PP_LOCAL_ITERATE()
>         default:
>             return &overflow;
>     }
> }
>
> template<int i> pbound_f bind(int j, int k) {
>     switch (j) {
>         #define BOOST_PP_LOCAL_MACRO(j) \
>             case j: \
>                 return bind<i, j>(k); \
>             /**/
>         #define BOOST_PP_LOCAL_LIMITS (0, LIMIT)
>         #include BOOST_PP_LOCAL_ITERATE()
>         default:
>             return &overflow;
>     }
> }
>
> pbound_f bind(int i, int j, int k) {
>     switch (i) {
>         #define BOOST_PP_LOCAL_MACRO(i) \
>             case i: \
>                 return bind<i>(j, k); \
>             /**/
>         #define BOOST_PP_LOCAL_LIMITS (0, LIMIT)
>         #include BOOST_PP_LOCAL_ITERATE()
>         default:
>             return &overflow;
>     }
> }
>
> int main() {
>     int i, j, k;
>     std::cin >> i >> j >> k;
>     bind(i, j, k)();
>     return 0;
> }
>
> This works, but takes forever to compile--at least on Comeau C++.  The
> reason is not the use of the pp-lib though (that is fast), it is because
of
> the sheer amount of template instantiations.  E.g. you get 101
> instantiations of the one-template-argument "bind," 10,201 instantiations
of
> the two-template-argument "bind," and 1,030,301 instantiations of the
> "target" function (+ 1,030,301 instantiations of "T").  Even if you use
the
> "if/else if" statements (or nested switches), you still going to get
> instantiations of "T" in the millions.  That means that you have to 1)
bite
> the bullet and give up trying to do whatever you're doing with
compile-time
> values or 2) seriously reduce the maximum value (to about 10, for
example).
> In any case, the pp-lib can generate this, direct nested switches, or even
> the permutations along the lines of "if/else if."  The problem is massive
> instantiation and massive code bloat no matter how you do it.  IMHO, it
just
> isn't worth it.
>
> Another idea is to have an array of function pointers:
>
> pbound_f array[101][101][101] = { ... };
>
> ...and initialize the elements with instantiations of some fully bound
> function template (the pp-lib can, of course, do this as well).  And use
it
> like this:
>
> array[i][j][k]();
>
> This will only avoid the first 10,302 instantiations though.  You are
still
> going to get template instantiation in the millions.  What are you trying
to
> do exactly?  Maybe there is a better way.
>
> > Naturally, I don't want to type all that in myself, so I was hoping the
> boost preprocessor library could do it for me.  If someone knows of a way
to
> do it, please let me know.
>
> Regards,
> Paul Mensonides
>
>
>
>
>
>
>
> Info: <http://www.boost.org>
> Wiki: <http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl>
> Unsubscribe: <mailto:boost-users-unsubscribe_at_[hidden]>
>
>
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>
>
>
>