Subject: Re: [boost] [preprocessing] Feedback requested on a C99 preprocessor written in pure universal Python
From: Edward Diener (eldiener_at_[hidden])
Date: 2017-03-07 15:38:21
On 3/7/2017 5:55 AM, Niall Douglas via Boost wrote:
>>> Given that preprocessor checks are often used for compiler
>>> workarounds, and pcpp is not a full C++ frontend, one would have to
>>> make sure pcpp defines the same set of predefined macros the compiler
>>> does. In the particular case of MSVC that would make libraries like
>>> Boost.PP and Boost.VMD treat pcpp the same way they do MSVC, which is
>>> probably suboptimal, if at all functional. I guess, for such tandem to
>>> be workable, pcpp has to define its own predefined macros, and PP and
>>> VMD have to test it before they test other compiler-specific macros.
> You've struck exactly at the main use case for pcpp, and why I didn't
> start from Wave or an existing preprocessor but instead reinvented the
> wheel and in Python, not C. I specifically am implementing a "partially
> executing" pre-preprocessor which can be programmatically and
> dynamically instructed to transform a set of input files with
> preprocessing commands into other files with some of those preprocessor
> commands executed or expanded or replaced, and some passed through.
> pcpp by default acts as a straight preprocessor, but it can also be told
> to pass through #if logic it can't fully execute due to unknowns, or
> partially execute #if logic. It can be told to pass through #define and
> #undef but also execute them (or not) on a per-macro basis. It would be
> easy enough to tell it to not execute any preprocessing commands except
> #include and include guards for example.
> I'm sure you can see the big benefits to pregenerating canned
> preprocessed files such that #including a Boost library is much faster
> than before because most of the preprocessing work is already done.
> Right now generating those is tedious work using hacky scripts on source
> files with splicing metadata injected via comments etc which are
> brittle. pcpp will allow for a FAR more robust solution which can be
> safely left to a CI to run per commit if desired.
> pcpp is basically Facebook's Warp
> (https://github.com/facebookarchive/warp) but done much more flexibly
> and usefully (no offence intended to Warp's developers, but Warp isn't
> very useful outside a very limited use case).
> My ideal end goal is for a download page for a Boost library to provide
> a set of tick boxes and drop down menus that let a user pre-preprocess a
> Boost library into a custom "drop in and go" single file edition for
> their particular use case, just like you can with say jQuery downloads.
> Again choosing Python instead of C makes that safe and secure. I don't
> know if I'll have the time to reach that, but I'm an awful lot closer
> now than I was a month ago.
The practical problem with this is that source files with preprocessor
directives often depend on the compiler being used, with its predefined
macros, to generate the correct output.
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk