Re: [boost] [review] Formal review period for VMD library begins today, Aug 21, and ends Sat, Aug 30

Subject: Re: [boost] [review] Formal review period for VMD library begins today, Aug 21, and ends Sat, Aug 30
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2014-08-23 21:51:08

On 23 Aug 2014 at 11:23, Agustín K-ballo Bergé wrote:

> On 23/08/2014 09:21 a.m., Niall Douglas wrote:
> > C++ Modules is expected to obsolete macro programming, in fact
> > eliminating macros completely from interface files is one of its
> > chief goals. It tells you all about it in its TS.
> 
> That's a bit hard to believe, I fail to see the relation. Could you 
> point us to the proposal that states this? (as far as I know, there is 
> no Modules TS, not even a draft)

Apologies, I thought the TS out by now. I spent a whole night at C++ 
Now talking with Gaby and Chandler about Modules, and what Microsoft 
and clang have learned about them, and I had thought Richard Smith's 
significant progress would enable a timely Modules TS.

You are right that the situation is rather more fluid that my hard 
assertion above, which was based solely on the WG21 Modules papers. 
There are currently three C++ Modules proposals on the table, two of 
which have the most weight.

One of those is the most recent paper from the Modules SG which is 
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n4047.pdf and 
which talks at length about the importance of limiting macros.

Gaby's paper doesn't go as far in its anti-macro crusade as the much 
older N3347 
(http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3347.pdf) 
where how the compiler can synthesise interface files written in a 
C++-lite IDL from existing header files for you. The implication, of 
course, is that interface files are a sort of macro expanded header 
file, and you would need to supply an additional header separate from 
the interface with any macros needed (with a strong hint this is only 
for C code or legacy C++). The key feature of interface files is that 
within them macros have no effect i.e. are not expanded in interface 
files i.e. the preprocessor is not invoked before their parsing 
because they aren't written in C++, rather in a C++-like IDL.

Gaby's paper rows back on this proposal significantly. Under his 
proposal, you simply wrap up the stuff you wish to export with an 
export scope rather like C# and .NET does it. Macros are explicitly 
defined to not be able to traverse between module boundaries, but 
because the export scope is basically a specially marked namespace 
and can appear anywhere, macros can be expanded during the interface 
export. Gaby's proposal is basically an "opt in at source code level" 
proposal as in you need to go retrofit your code, whereas the earlier 
N3347 has the compiler separate out interface files into their own 
reduced subset of C++ language, with the idea that existing headers 
are simply auto-converted to equivalent module IDL files. I'm not 
much keen on this proposal, and I said so to Gaby, because it's 
basically a superior __declspec(dllexport) and whilst a definite 
advance, I think we can go a lot further and do better.

Now, the above two are the "easy" Modules proposals. The much more 
fun Modules proposal, and probably more representative of what is 
coming given it will be one standard practice, is clang's already 
somewhat working C++ Modules 
(http://clang.llvm.org/docs/Modules.html) which is based on a 
meta-header script called module maps which add the metadata above 
and beyond headers to say how they are to be combined into 
interfaces. This is an "opt in at the build level" proposal, so 
you'll be adding extra script to build to tell clang what a module 
interface is, and that sort of is a set of filters and rules for how 
to combine effectively precompiled headers into one enormous database 
of all precompiled compilands. Modules, therefore, become simply a 
database view.

clang's is a truly "full fat" Modules implementation that intends to 
solve the ODR violation problem, and where you can have, coexisting, 
multiple definitions of macros /at the same time/ and a complex set 
of rules will determine the default macro your code uses at the point 
of use (those are listed on the clang modules page as they've already 
been defined for C). If this sounds exactly like Exported Templates, 
that's because it is very similar, and the ODR violation resolution 
rules look very similar (indeed as Chandler semi-joked, clang may 
just implement exported templates anyway for the fun of it, despite 
that they have no useful use, because it's such a small bit of extra 
work after Modules so why not?).

In case you might wonder why clang bites this thorn bush so hard, it 
is because it's probably the best long term option. It certainly 
makes combining Reflection and Modules straightforward, something 
Gaby's proposal would simply have to say that Reflection cannot 
traverse a Module boundary, and the earlier EDG proposal explicitly 
says that nothing traverses a Modules partition, not symbols, macros, 
nothing.

So, to give a full and proper answer, yes the WG21 papers say no to 
macros in interfaces, but the recent trend is not in that direction 
at all [1]. Indeed, if clang's modules become C++ Modules, you get 
all the true joy and power of C macro processing in all imported 
modules at once. Let us #define return and #define int and have lots 
of fun. Yay.

[1]: For transparency, I am not in favour of this trend. I like the 
JIT nature of compiling C++ as you make possible clang compiling code 
as you type, which is cool, and skips the whole compilation step, if 
not linking. But I think interfaces as a contract rather than a 
database query are useful for ABI stable SDKs etc, and clang modules 
doesn't address that. In fairness, clang modules explicitly says it 
is not meant to in its docs, it's really there as a whole new 
generation of database driven compile-and-link from wholly 
precompiled everything, which is an even further dilution of the 
original promise of Modules :(

Niall

-- 
ned Productions Limited Consulting
http://www.nedproductions.biz/ 
http://ie.linkedin.com/in/nialldouglas/