Boost logo

Boost :

Subject: Re: [boost] [proto] Looong compile times and other issues
From: Eric Niebler (eric_at_[hidden])
Date: 2011-09-11 15:11:46

On 9/11/2011 1:44 PM, Joel Falcou wrote:
> Le 11/09/2011 19:18, John Maddock a écrit :
>> So.... I thought I had all the expression template stuff pretty much
>> done, and then I tried some real world use cases (compiling all the
>> students_t distribution functions with my extended-precision FP type)
>> and everything fell apart:
>> * VC10 wouldn't compile the code at all - more or less runs the system
>> out of swap space (takes about 10 minutes or more!), then exits with an
>> internal compiler error (if I break the code down into it's parts and
>> instantiate each part separately it does compile though - that's not a
>> solution though!).
>> * GCC-4.4.x fails to compile the code due to clashes between
>> boost::math::complement (a function) and boost::proto::complement (a
>> class). I suspect this is an old gcc bug (finding structures via ADL) -
>> I guess the solution is to not derive my number type from a proto-type
>> so ADL can't find proto:: classes? Or will I hit this from some other
>> unforeseen lookup?

This is not a gcc bug. At least, the gcc developers would say so. There
is some ambiguity in the standard about whether ADL should find classes
in addition to functions -- despite of the fact that finding classes is
never what the user wants.

John, where does complement show up unqualified in your code?

>> * GCC-4.5.0 Fails with an internal compiler error :-(
>> * GCC-4.6.0 Builds the code OK, but takes a long time - though possibly
>> just barely acceptable.
>> I suspect this is a "triple template" problem:
>> * proto is complex template library.
>> * my number class is a fairly large/complex template in it's own right.
>> * the code above gets instantiated from deep within Boost.Math's
>> internals.
>> Other than using proto::switch in the grammar (which I'm doing already),
>> is there anything I can do to reduce the template load on the proto side
>> of things?
>> Otherwise since I'm only using a tiny fraction of proto's capabilities,
>> the only other option I can see is to rip it out and replace with a
>> mini-proto designed to minimalize template instantiations within this
>> particular use case.
> What kind of transform do you use ? I foudn out that colalpsing complex
> trasnform into primitive one eased the problem.

Right. The nice transform syntax that allows composition via function
types is rather expensive at compile time. It's great for knocking
together a DSEL quickly, but once you have it all working, you can
replace them one by one with the equivalent primitive transforms, like
Joel suggests.

> Next to that :
> - improve CT through forward declaration. proto is a bit woobey on this
> side and I think it may help if we split proto files in more grain
> fashion

Joel, can you give an example of where this might help>

> - fusion is also a big hitter : lots of PP and lots of forced
> instanciation instead of lazy specialization.

I avoid fusion, sadly. Proto is not built on top of fusion for just this
reason. Proto is expensive enough all by itself. :-P

> As for the ADL, can your terminals be made PODs ? We can see if adding
> ADL barriers in proto helps

Proto already has ADL barriers. The expression types are in
boost::proto::exprns_, for instance. I would have to see the code to
know what gcc is complaining about.

> For the ICE, is there any hints in the core dump ?

There usually isn't, AFAIK.

John, if you can post your code publicly, perhaps one of the proto
cognoscenti can help.

Eric Niebler
BoostPro Computing

Boost list run by bdawes at, gregod at, cpdaniel at, john at