Re: [boost] [preprocessing] Feedback requested on a C99 preprocessor written in pure universal Python

Subject: Re: [boost] [preprocessing] Feedback requested on a C99 preprocessor written in pure universal Python
From: Hartmut Kaiser (hartmut.kaiser_at_[hidden])
Date: 2017-03-09 07:28:57

> >> My main use case for writing this is to assemble my header only Boost
> >> libraries into a single "drop in and go" file. To that end it has a
> >> (still buggy) --passthru mode which passes through #define and #undef
> >> plus any #if logic which uses an undefined macro. I'm still working on
> >> pass through mode so expect showstopper bugs in that configuration, but
> >> as a straight highly standards conforming preprocessor it's ready for
> >> others to use and I welcome any feedback. There are a multitude of use
> >> cases, everything from running as a conforming preprocessor before
> >> invoking MSVC to compile right through to parsing, reflection and
> >> introspection of source code.
> >>
> >>
> >> I did something very similar to this wave quite a few years back. We
> dubbed
> >> it partial preprocessing. It's still in use in Phoenix, iirc.
> >> We developed integrations for b2 and later on integrated it into our
> cmake
> >> build process for HPX.
> > [snip]
> > The advantage here is certainly to have a fully compliant C99
> preprocessor at
> > our disposal. Your usecase might differ slightly, but I am pretty
> confident
> > that with a little change to our process, completely feasible.
> > The only reason we got rid of wave 2 years ago was that we made variadic
> > templates a prerequesite...
> 
> Interesting work of yours, and thanks for describing it. It is
> unfortunate you did not more widely publish its existence, else Facebook
> wouldn't have gone off and written Warp and I'm sure a raft of other
> people wouldn't have gone off and written their own hack solutions e.g.
> Phil Nash has a bespoke system in CATCH for example. Perhaps you should
> consider presenting at a major C++ conference on this work so a video
> search result pops up for those who come at this problem in the future?
> 
> Your description also confirmed to me something which I suspected from
> studying the Wave sources before I started pcpp, which is that Wave is
> hard to customise in the way I was wanting. The technique of generating
> long lists of macros to not expand is one approach to solving this
> problem, however I believe one can take a more programmatic approach
> based on _source annotation_. I'm still testing the feasibility of my
> idea, and if it doesn't work I'll fall back onto the same approach you
> took, but if it does work then it'll be a much less brittle approach to
> take. I also have the very big advantage that I can embed Python script
> into my C++ headers to customise pcpp processing, not that I've
> leveraged that yet.

You might have seen that wave uses special #pragma to control various
parameters of the preprocessing process. That can be used to achieve what
you want as you can easily implement your own. 

Besides, customizing wave is not more difficult than writing one of the
hooks pcpp is providing. I'd even say it's easier as wave gives you a
tokenized stream representing the input sequence instead of the raw
character stream.

The only drawback is that you have many more knobs to turn as there are
hooks for a plethora of preprocessor 'events' to weave your code into.

Regards Hartmut
---------------
http://boost-spirit.com
http://stellar.cct.lsu.edu