Re: [boost] Scalpel: a Spirit&Wave-powered C++ source code analysis library

Subject: Re: [boost] Scalpel: a Spirit&Wave-powered C++ source code analysis library
From: Doug Gregor (doug.gregor_at_[hidden])
Date: 2010-09-03 11:04:56

On Thu, Sep 2, 2010 at 6:55 PM, Florian Goujeon
<florian.goujeon_at_[hidden]> wrote:
>  Dear Boosters,
>
> I'd like to introduce to you the Scalpel library, a project on which
> I've worked during the last two years.
>
> Scalpel is a C++ library. Its name stands for source code analysis,
> libre and portable library.
> It is still under development, but is a fairly advanced work in
> progress.
>
> The purpose of this library is to produce a data structure which
> corresponds to the meaning (or semantics) of a given C++ source
> code. It reveals notions such as namespaces, classes, functions,
> variables, types, etc..
>
> Some source code analyzers, like those used by syntax coloring and
> autocomplete modules which need to be fast, perform a superficial
> analysis. Unlike them, Scalpel aims to accomplish a strict and
> exhaustive analysis so that it could even be used as a compiler
> front-end. Actually, Scalpel is a compiler front-end, since it goes
> through the phases of preprocessing, syntax analysis and semantic
> analysis, just like every C++ compiler does. Maybe one day there will
> be a Scalpel-powered C++ compiler!
>
> Besides, Scalpel's analysis depth will be adjustable in order to fit
> the needs of most programs. For example, it could be possible to
> disable the function body analysis for those who need to retrieve
> namespace and class members only.
>
> The labor of C++ source code analysis is extremely complex. This is
> why having a library wholly devoted to it is a good thing.
>
> Many programs could take advantage of such a library. Among them we
> can find modules for code editors, reverse-engineering tools, code
> audit software and many other CASE (Computer-Aided Software
> Engineering) tools that remain to be invented…

Having a good, open-source C++ parser library that could support such
tools would be wonderful. However, I am going to be a stick-in-the-mud
and propose that we already have such a library. Clang:

  http://clang.llvm.org/

Clang is an open-source C++ library developed under a Boost-compatible
BSD-like license [1]. It's written completely in C++, and performs
preprocessing, parsing, semantic analysis, and code generation for
C/C++/Objective-C/Objective-C++ (+ OpenCL, if you have a supporting
environment) on a variety of targets. It's designed as a set of
reusable libraries, so that it can form the basis of tools, and
includes support libraries for program indexing, source-to-source
transformation, code completion, and static analysis (among others!).

Clang already implements support for the entire C++98/03 language
(except exported templates), and does so well enough that it can
handle all of Boost [2]. For reference, check out the
dgregor2/clang-darwin-2.8 column on today's Boost regression-test
results:

  http://www.boost.org/development/tests/release/developer/summary.html

where you'll see that Clang is passing nearly every Boost regression
test on the release branch.

Clang is supported both by industry [3] and by an awesome open-source
community, which (as with Boost) makes for a great symbiotic
relationship: industry provides the stability and focus needed to turn
Clang into a production-quality C++ compiler, while the open-source
community provides a wealth of ideas and vision that pushes Clang into
new areas. For example, a group within the Clang community has taken
it upon themselves to start implementing Microsoft-specific extensions
to make Clang far easier to use on Windows.

Writing a C++ parser/compiler requires years of full-time technical
effort, and I strongly encourage you not to begin yet another
open-source C++ parser. There are already two good open-source C++
compilers, GCC and Clang, and I'd strongly recommend working toward
making one of those two projects better. As an added bonus, both
compilers are at a stage where you can come into the community and
work on the fun stuff (C++0x features, tools for C++ programmers,
optimizations, etc.) rather than slog through the dull parts of C++
(initialization, name lookup, access control).

  - Doug, Clang C++ technical lead

[1] Scalpel appears to be under an LGPL license, which is not Boost-compatible.

[2] Clang hit this milestone back in May:
http://blog.llvm.org/2010/05/clang-builds-boost.html

[3] Apple has already shipped Clang as a C/Objective-C compiler (about
a year ago). Apparently, several OpenCL implementations are also based
on Clang (see http://en.wikipedia.org/wiki/OpenCL), and there has been
significant interest from industry