Re: [boost] [proto] Looong compile times and other issues

Subject: Re: [boost] [proto] Looong compile times and other issues
From: Eric Niebler (eric_at_[hidden])
Date: 2011-09-11 15:11:46

On 9/11/2011 1:44 PM, Joel Falcou wrote:
> Le 11/09/2011 19:18, John Maddock a écrit :
>> So.... I thought I had all the expression template stuff pretty much
>> done, and then I tried some real world use cases (compiling all the
>> students_t distribution functions with my extended-precision FP type)
>> and everything fell apart:
>>
>> * VC10 wouldn't compile the code at all - more or less runs the system
>> out of swap space (takes about 10 minutes or more!), then exits with an
>> internal compiler error (if I break the code down into it's parts and
>> instantiate each part separately it does compile though - that's not a
>> solution though!).
>> * GCC-4.4.x fails to compile the code due to clashes between
>> boost::math::complement (a function) and boost::proto::complement (a
>> class). I suspect this is an old gcc bug (finding structures via ADL) -
>> I guess the solution is to not derive my number type from a proto-type
>> so ADL can't find proto:: classes? Or will I hit this from some other
>> unforeseen lookup?

This is not a gcc bug. At least, the gcc developers would say so. There
is some ambiguity in the standard about whether ADL should find classes
in addition to functions -- despite of the fact that finding classes is
never what the user wants.

John, where does complement show up unqualified in your code?

>> * GCC-4.5.0 Fails with an internal compiler error :-(
>> * GCC-4.6.0 Builds the code OK, but takes a long time - though possibly
>> just barely acceptable.
>>
>> I suspect this is a "triple template" problem:
>>
>> * proto is complex template library.
>> * my number class is a fairly large/complex template in it's own right.
>> * the code above gets instantiated from deep within Boost.Math's
>> internals.
>>
>> Other than using proto::switch in the grammar (which I'm doing already),
>> is there anything I can do to reduce the template load on the proto side
>> of things?
>>
>> Otherwise since I'm only using a tiny fraction of proto's capabilities,
>> the only other option I can see is to rip it out and replace with a
>> mini-proto designed to minimalize template instantiations within this
>> particular use case.
> 
> What kind of transform do you use ? I foudn out that colalpsing complex
> trasnform into primitive one eased the problem.

Right. The nice transform syntax that allows composition via function
types is rather expensive at compile time. It's great for knocking
together a DSEL quickly, but once you have it all working, you can
replace them one by one with the equivalent primitive transforms, like
Joel suggests.

> Next to that :
>  - improve CT through forward declaration. proto is a bit woobey on this
> side and I think it may help if we split proto files in more grain
> fashion

Joel, can you give an example of where this might help>

> - fusion is also a big hitter : lots of PP and lots of forced
> instanciation instead of lazy specialization.

I avoid fusion, sadly. Proto is not built on top of fusion for just this
reason. Proto is expensive enough all by itself. :-P

> As for the ADL, can your terminals be made PODs ? We can see if adding
> ADL barriers in proto helps

Proto already has ADL barriers. The expression types are in
boost::proto::exprns_, for instance. I would have to see the code to
know what gcc is complaining about.

> For the ICE, is there any hints in the core dump ?

There usually isn't, AFAIK.

John, if you can post your code publicly, perhaps one of the proto
cognoscenti can help.

-- 
Eric Niebler
BoostPro Computing
http://www.boostpro.com