Re: [boost] [forward_declare] Interest Inquiry toward Faster Compile Times

Subject: Re: [boost] [forward_declare] Interest Inquiry toward Faster Compile Times
From: Krzysztof Czainski (1czajnik_at_[hidden])
Date: 2012-08-02 02:22:58

2012/8/2 Daniel Larimer <dlarimer_at_[hidden]>

> I have been increasingly frustrated with how long it takes to compile c++
> code.  After spending some time in Java and C# I can feel the pain of
> compile times much sharper than I use to.  After seeing that Java and C#
> were able to achieve some level of generics without requiring 'everything
> in the header' and knowing that C compiles an order of magnitude faster
> than c++ I set about to find a way to increase my compile times.
>
> One of the common recommendations is to 'forward declare' everything and
> to use the PIMPL (private implementation pattern).   Unfortunately, PIMPL
> usually implies heap allocation and if you ever want to return-by-value you
> are stuck including the entire header for the types you use in your header.
>
> I started inspecting some of my code and running it through the
> pre-processor and noticed that almost every source file was in the 50K+
> lines of code post-preprocessor and that some files were north of 250K
> lines of code.   This was ridiculously inefficient, especially because I
> was already using PIMPL everywhere I could and most of these lines were stl
> + boost.
>
> After a little bit of work I was able to abstract vector,
> boost::exception, iostream, string, threading, and many of the other types
> that I had been including that resulted in 'code-bloat'.  The resultant
> library was able to reduce my post-preprocessor output to just one or two
> thousand lines of code (assuming the .cpp itself was 500+ lines).   My
> compile times dropped to sub-second per object file from 10+ seconds (some
> up to 40 seconds).
>
> There are some side effects from doing this, performance in some cases was
> slightly slower due to one level of indirection, but with with link-time
> optimization this performance loss was minimized.  And any loss of
> performance was clearly acceptable trade for the gain in development
> efficiency.   In situations were performance really is critical I can
> choose to include the raw header in individual cpp files, but in most cases
> that level of 'optimization' is overkill.
>
> One of the things I realized doing this work was that often the only
> reason you need to include a header is because the compiler needs to know
> the SIZE of the type that you wish to forward declare and nothing else.  So
>  our development is slowed entirely due to the lack of a SINGLE NUMBER.
> So I created a utility that allowed me to 'forward declare' the size of the
> variable.  This allowed me to use return by value and member variables via
> forward declaration WITHOUT having to resort to heap allocation.
>
> It occurred to me that a library that allowed developers to choose a
> little drop in runtime performance for a dramatic improvement in compile
> times would be generally useful to the C++ community.
>
> I have included an example of the 'technique' and general purpose utility
> that would allow others to quickly and easily integrate forward declaration
> of 'value types'.
>
> A full up boost submission would probably include pre-wrapped versions of
> most of the STL and Boost types.
>

Hi,

I believe the general idea has been around for years [1]. I wonder if a
utility like this gets accepted to Boost. You have my interest.

I think you can save a few more lines of code in the headers by separating
some of the member functions of fwd (like the constructors and the
destructor)  into another header file, that will only get included in .cpp
files.

Cheers,
Kris

[1] http://www.gotw.ca/gotw/028.htm