[Boost-users] Re: compilation time [was: Re: multi_index : performance issue]

From: Joaquin M Lopez Munoz (joaquin_at_[hidden])
Date: 2004-12-21 15:44:54

Stanislas RENAN <stanislas.renan <at> nsktechnologies.com> writes:

> I've made the changes and see no noticeable difference in figures
> 
> before :
> multi 16
> STL   0
> multi 422
> STL    468
> 
> after :
> Insertion
> multi = 16
> STL   = 15
> Recherche
> multi = 438
> STL   = 484
> 
> switching to release mode, I give the arimthmic average value of 5 runs
> because figures are small (so from notes and calculations) :
> multi :  3
> STL   :  0
> multi :  137
> STL   :  131
> 
> I see no real difference in performance between both of them (the
> accuracy of the measures is not very good).
> 
> Here are the changes in the code :
>    std::multimap<int, employee *> m_byAge;
>    typedef std::multimap<int, employee *>::iterator EMAgeIt;
> 
> and
> 
>      m_byAge.insert(std::make_pair<int, employee *>(e2->age(), e2));

You seem to have made the same changes as I, yet
our numbers do not coindice. Could you please send me your
changed code? Thanks!
Searching should be roughly the same, anyway, but I
expect insertion to be noticeably faster in the case
of multi_index_container.

> 
> The next problem I have now is compilation time.
> This example has taken 50s to compile on my computer in debug mode, in
> its first version.
> 
> Now, with the 2 debug defines commented, and the multimap it takes :
> - 41s to compile in release mode with MI
> - 3s to compile with all BOOST references enclosed in #if defined() that
> evaluates to false.
> So, on this example, it's 13 times longer.
> 
> I wonder why compilation time is so long, and if I can do something to
> make things faster.
> I'm getting upset with compilation times on a very small test, so I
> wonder what it's gonna be in real programs.
> Maybe it's an absolute addition, just 40s, whatever the number of
> references to MI (any BOOST ?) objects.
> But even if it's right, if each file compiled suffers from this additon,
> it could make compilation time very long.
> That may be considered an acceptable price.
> 
> We've got an application here that take several hours to compile. If we 
> use BOOST library, is this (long) time gonna be 13 times longer ?
> That would probably be an unacceptable price.

I'd tend to think you would more likely pay a 
roughly fixed penalty per header, instead of a x13
factor in compilation times, especially if your code
takes already long to compile. But this may well be
wishful thinking; please read below.

> 
> Maybe there is a document or a discussion regarding this issue ? I've
> found nothing neither on the web site nor on the list.
> Or am I missing something anew ?
> 

Unfortunately, this is a known issue, and one for which
I have no easy workaround. Boost.MultiIndex is heavily
templatized, and this makes the compiler work real hard.
I've been reported that MSVC 7.0 and 7.1 (aka .NET 2002
and 2003) are faster when dealing with Boost.MultiIndex,
but I guess switching compiler is not an option to you.

Another possibility (not tested) is that you include
Boost.MultiIndex headers inside your precompiled header
stdafx.h. MSVC++ 6.0 precompiled headers support is a
little fragile, so I won't be surprised if this resulted
in spurious crashing and/or the need for full rebuilds.
In any case, if you try this option please let me know.

To sum it up, compilation times can be a barrier for
adoption, especially in somewhat older compilers, and
you are wise to test this before embarking into using
Boost.MultiIndex. If you finally decide to drop it,
at least keep in mind this is going to be better with
newer compilers. To my discharge, allow me to say that
this is the best I've been able to come up with: after
all, template stuff was not thought in the late nineties
to be as extensively abused as it is now.

Joaquín M López Muñoz
Telefónica, Investigación y Desarrollo