Re: [boost] fighting compilation times

From: John Torjo (john.lists_at_[hidden])
Date: 2003-10-17 10:11:23

> Yes of course,
>
> Here's another idea on the subject:
>
> Suppose you have a C++ parser on hand (using Spirit for instance) so you
> could parse the files and create a dependency (directed) graph using the
> BGL. In this graph, the files are the nodes and the edges are the
#include.
> (this problem is so important that it is one of the BGL examples).
(Doxygen
> already does that)
>
>  From there, a lot could be done using graph theory. For instance:
>
>     - by sorting the nodes by edges number, you can detect which files are
> the most often included. In boost, I suppose it would be
<boost/config.hpp>
>     - since making forward declaration enables to break some edges, could
> this algorithm be used to find which dependency could be replaced by a
> forward declaration ?
>
> Just thoughts.

Yup!

Here's what I want to do:

1. I want to see which header(s) take most time to compile.
2. I want to find out which files are most under development, and how they
affect the files that
   depend on them.

For case 1., the solution could be to pre-compile the given header, or to
include
this header from specific source files (as opposed to including a header
from within a header).

For case 2., once a header file is under development, and it affects a lot
of other files (shown
in compilation times), this file should be optimized for better
compile-times.

Also, the library should disregard well-behaved headers, which even if
included a lot of times,
take virtually no time.

The information for 1. & 2. should gather in time, as the project is under
development.
Once we know what should be optimized, another tool (which should be
developed ;) - using a C++ parser)
will take this information and modify the files.
This can be as easy as:
- aggregating pointers/references instead of values
- defining (some) methods in source files, instead of header files.
  In case the methods are templated, we can all benefit from Eric Niebler's
instantiator library.

As a side-note, the latter tool (which can allow for refactoring, etc.) I
think should already exist (anyone?).
Anyway, the tool might be your own fingertips ;) So, since you're gonna do
refactoring, might as well
know where... That's my point.

Best,
John

>
> At 11:51 17/10/2003 +0200, you wrote:
> >Dear boosters,
> >
> >Although I'm currently working on the SMART_ASSERT library, I'm willing
to
> >stop for a little, and follow this idea.
> >
> >When it hit me, I couldn't believe its simplicity, and think it would be
> >quite useful for a lot of projects.
> >
> >
> >*** Fighting compilation times ***
> >
> >This issue appears more and more, even if computers/ compilers are
getting
> >faster, mostly due to templates, but not limited to (also, see Eric
> >Niebler's latest article on CUJ).
> >
> >I'm working on a (huge) application, which contains 6 projects, each
taking
> >more than 5 mins. to compile. That wouldn't be a problem, but modifying
> >something triggers a lot of source files to be recompiled.
> >
> >What I want to know is ***where*** should I minimize dependencies (based
on
> >the old mighty: don't optimize too soon strategy).
> >
> >So, what can be done, is to have a small utility header file included in
ALL
> >compilable files from your custom project (the files that you or your
> >company own and can modify).
> >
> >This would be something similar to this:
> >
> >#define COMPILE_FILE __FILE__
> >#include "boost/compile_profiler.hpp"
> >
> >(a script to include this in every file from your project I assume is
quite
> >easy to do).
> >
> >The compile_profiler.hpp will contain amongst others, something similar
to
> >
> >namespace {
> >    some_class var( __DATE__, __TIME__, COMPILE_FILE);
> >}
> >
> >
> >This will allow for:
> >Throughout two consecutive compilations, you will know EXACTLY which
files
> >got recompiled (and eventually, dependencies). This is then used to
create a
> >database of such changes (this can also be improved, to contain the names
of
> >the classes/ functions that were modified, etc.).
> >
> >A program will then be created that will analyze this database, and in
time,
> >it can suggest to you ***where*** you should minimize dependencies.
> >I think this would be very valuable for large projects.
> >
> >So, any interest in such a library?
> >
> >Best,
> >John
> >
> >
> >(note: I'll be off from tomorrow, until Tue or Wed - I'm moving to a new
> >location).
> >
> >
> >--
> >John Torjo
> >-- "Practical C++" column writer for builder.com.com
> >
> >Freelancer, C++ consultant
> >mailto:john_at_[hidden]
> >
> >
> >_______________________________________________
> >Unsubscribe & other changes:
> >http://lists.boost.org/mailman/listinfo.cgi/boost
>
> --------------------------------------------------------------------------
---------
> Jonathan de Halleux, Research Assistant
> Center for Systems Engineering and Applied Mechanics (CESAME)
> Universite catholique de Louvain
> Batiment Euler , Av. Georges Lemaitre, 4 Tel : [phone]
> B-1348 Louvain-la-Neuve Belgium
> E-mail : dehalleux_at_[hidden]
> --------------------------------------------------------------------------
---------
>
> _______________________________________________
> Unsubscribe & other changes:
http://lists.boost.org/mailman/listinfo.cgi/boost
>
>