[boost] [config][preprocessor] proposal for GCC-XML support

From: Brad King (brad.king_at_[hidden])
Date: 2006-01-07 13:06:50

Hello,

This message proposes adding GCC-XML support to the boost libraries to
enable wrapping of more code with pyste.  Most of the changes will
probably be in the config and preprocessor libraries, but there may be
others.  Here is an explanation of the motivation and what needs to be
done:

Boost's Pyste tool

   http://www.boost.org/libs/python/pyste

uses my tool GCC-XML

   http://www.gccxml.org

to parse C++ code and then generates Boost.Python wrappers.  Users
have experienced problems getting some boost code to be parsed by
GCC-XML.  See the problem description

   http://www.gccxml.org/pipermail/gccxml/2006-January/000696.html

and my explanation of what GCC-XML is doing

   http://www.gccxml.org/pipermail/gccxml/2006-January/000697.html

for details.

GCC-XML has two basic parts.  One is a patched GCC compiler that has
extra code to dump out an XML description of the interface found in a
translation unit.  The other is a front-end executable and some
support code I wrote to enable GCC-XML to parse source code in a way
that is compatible with wrapper generators.

Wrapping arbitrary C++ code requires the wrapper generator to know
about the standard library and implementation-defined types and
interfaces provided by the compiler that will be used to build the
generated code.  In order to provide this information from the source
GCC-XML simulates the preprocessing environment of a given compiler.
This means that even though the parser is GCC the preprocessor
definitions look like another compiler.

For example, if msvc71 is specified as the compiler to simulate then
the include path will be setup to use the system headers provided by
Visual Studio .NET 2003.  Preprocessor definitions such as _MSC_VER
will be set they way they are when the native preprocessor runs.
Unless a GCC compiler is being simulated, __GNUC__ is NOT defined.
The only way for code to know that it is being preprocessed by GCC-XML
is to look for the definition __GCCXML__.  This macro is defined to be
the GCC-XML version number (major*10000+minor*100+patchlevel).

In a perfect world all compilers would be standard conforming and
would all parse valid C++ code.  In the real world there are
inconsistencies and boost has many places that use compiler-specific
parser work-arounds.  Since GCC-XML uses a GCC parser but appears to
the preprocessor as the native compiler special knowledge is needed
when choosing implementation alternatives for compiler workarounds.

GCC-XML is effectively another compiler for which knowledge should be
encoded into the boost libraries.  When __GCCXML__ is defined any
*parser* work-around should use the GCC alternative, while any
*interface* work-around should use the native compiler's alternative.
For example, if a particular boost feature is not available on MSVC 6
then it should not be available if _MSC_VER=1200 even if __GCCXML__ is
defined.  On the other hand if the feature is always available but its
implementation switches among different syntax choices to support
multiple compilers then the GCC choice should be used if __GCCXML__ is
defined.  Therefore only parser work-arounds need to be updated with
knowledge of GCC-XML.  If choice of a specific GCC parser version is
needed each GCC-XML version always uses a single GCC parser version
(currently 3.3) so there is a simple mapping from the __GCCXML__ value
to the GCC parser version number.

The following are points for discussion:

1.) Is there interest in adding this support to enable both wrapping
of more boost code and of more user code that uses boost with pyste?
The motivating case mentioned above is trying to wrap serialization
code which in turn uses mpl.

2.) What is the best way to encode knowledge of GCC-XML in boost
libraries?  I suspect most of the knowledge can be put in the config
library when it defines macros encoding the availability of parser
features.

3.) Does anyone volunteer to do the work?  It can probably be done
incrementally as users request support for wrapping of more boost
code.  I'm willing to help but I cannot spend very much time on this
because it is really a boost/pyste problem and not a GCC-XML problem.

-Brad