[Boost-users] Re: long compilation time using variant and spirit

From: Joel de Guzman (joel_at_[hidden])
Date: 2005-02-24 18:27:14

Matt P. wrote:
> Hello,
> 
> I have been developing a piece of code using various boost libraries,
> but I have to pause because the compilation has become unwieldy. It's
> a 50kloc prgm, which takes about 30min to compile on a pentium 3.5GHz
> HT and 1GB of memory (gcc 3.4 on linux). The binary, using the static
> linking with 2 boost libraries (program-options, filesystem) yield a
> 380M binary. Furthermore, it doesn't compile on msvc 7.1 (c1063,
> compiler stack overflow), and I had to stop mingw after one hour....
> 
> The prgm handles the parsing, storage, and serialization of a big raw
> string. This string is organized into fragments, about 30 each
> fragment could be occuring 0 or more times. each fragment contain on
> average 30 fields. each fragment has little in common with other
> fragments.
> 
> my solution was to store the string as a vector of a type Fragment
> described by typedef variant<fragment1, fragmen2, ..., fragment30>
> Fragment. typedef vector<Fragment> string_storage.
> 
> the parsing is handled using a grammar for each fragment, and the
> grammars are split between the definition and instantiation (manual
> explicit), as suggested in the FAQ. Most fields have a unique functor
> which is used as a semantic action to store the data found in the
> fragment.
> 
> So this all scheme looked darn good when I had only a couple
> fragments, and it was working very well too! But now, my computer
> threatens to go strike whenever I type bjam on the command line.
> 
> I don't have a clear question, besides the simple 'how well does
> variant scale'?, or 'does that seem like a not so clever design?'.
> The variant page mentions an optimization opportunity with
> 'has_nothrow_copy', but I can't find a good example. Could that help?

Not sure if this is a spirit problem or a variant problem
or a combination. You do have 30 grammars, but I think that
should expand linearly in time at compile time. It would be
best to investigate more by isolating variant and timing the
compile time. If it is still slow, it is best to hide each
grammar in a cpp file whilst exposing an entry point to the
parser, in a header, in the form of a function taking in
the iterators and returning your result.

Cheers,
-- 
Joel de Guzman
http://www.boost-consulting.com
http://spirit.sf.net