Re: [boost] [fusion] improving compile times

Subject: Re: [boost] [fusion] improving compile times
From: John Bytheway (jbytheway+boost_at_[hidden])
Date: 2009-06-02 17:42:43

Eric Niebler wrote:
> Larry Evans wrote:
>> On 06/01/09 20:26, Eric Niebler wrote:
>>> I'm attaching a simple patch to vector_n_chooser.hpp that replaces
>>> some template metaprogramming with preprocessor metaprogramming in
>>> the interest of improving compile times. I found this hotspot through
>>> profiling, 
>> [snip]
>> Eric,
>>
>> Could you post your benchmark code that showed the improvement in
>> compile speed?  I'd like to eventually try it with a variadic template
>> compiler version of fusion vector.
> 
> I confess that I'm not actually benchmarking compile speed; rather, I'm
> benchmarking the number of template instantiations as reported by
> Steven's template profiler. I'm profiling TMP-heavy code like some of
> Proto's and xpressive's tests and cherry-picking the worst offenders.
> The Fusion vector_n_chooser patch knocked off 100's of template
> instantiations, for instance.

Be careful.  I once spent a couple of days trying to speed up some
compilations that were taking ~5mins per file.  All my efforts to reduce
template instantiations had no effect on the compile time.  Eventually I
realised that ccache was actually caching the compiles and my timing was
ignoring the compile step entirely; those minutes were being spent in
the preprocessor.  I rewrote some preprocessor metaprogramming, changing
an algorithm from O(n^4) to O(n) (at the expense of some runtime
memory), and it all went away.

I later had to abandon the project because the template metaprogramming
was too memory-hungry, but it was a valuable lesson.  I don't suppose
anyone's written a preprocessing metaprogramming profiler yet?

John