Re: [boost] [intrusive] Interface for hooking into client code improvable?

From: Lassi Tuura (lassi.tuura_at_[hidden])
Date: 2007-08-11 09:03:52

Hi,

> It's not the first time I've seen this, and I still do not understand
> what the belief of template meta-programming causing bloat is based  
> on.

Maybe I can help with that.  For simple toys and well-contained  
programs, the compiler can do a lot to inline, as was kindly shown by  
someone.  But take an application of ours as an example from somewhat  
more real software world.

An average run pulls in several hundred shared libraries and has a  
memory foot print of about 500 MB.  Of the 500 MB, ~100 MB is for  
text segments (machine code).  Of the 100 MB, several tens of  
megabytes is redundant duplicate symbols, for example template  
instantiations and out-of-line versions of inline functions.  There's  
1 MB of code for (numerous copies of) a single template function.   
For a more real measure of redundancy, you'd have to add on top code  
that was actually inlined.

Put another way, we estimate 10-25% of the entire program memory  
footprint is various degrees of _code_ redundancy.

But it gets worse.  The "useful" and "useless redundant code"  
interleave in the shared libraries.  For each duplicated symbol the  
dynamic linker picks a "representative" symbol; the others will never  
be used.  You won't know which ones the representative ones will be  
until at run time.  The result: in memory code is like Swiss cheese,  
some hundred bytes of used code, some hundred bytes of unused code,  
used code, unused code.  Not only do we have lots of memory committed  
to code, we make terrible use of it and have poor page locality.

And surprise, some of the bigger bottlenecks plaguing our programs  
are L1 instruction cache pressure and ITLB misses.  40% of L2 cache  
accesses are for instructions -- and each and every L1 instruction  
cache miss stalls the CPU.  Solving the code bloat is obviously  
making its way up on our performance optimisation priorities.   
Haphazard use of templates and inlining definitely make things  
worse.  Hope this helps understand why.

While we all hope compiler technology to improve, the reality check  
is that the impact factor of 20 years of compiler research is  
trivially measured: it's the difference between -O0 vs. -O3.  If your  
program falls into the class where that makes a world of difference,  
good for you.  Most programs in this world are not in that class, and  
you start engineering and making trade-offs.

Lassi