Re: [boost] [preprocessor] Patch for stringize.hpp

From: Paul Mensonides (pmenso57_at_[hidden])
Date: 2006-02-21 07:02:19

> -----Original Message-----
> From: boost-bounces_at_[hidden] 
> [mailto:boost-bounces_at_[hidden]] On Behalf Of Steve Cornett

> Thanks for the detailed answer.  I now see more of the 
> overall purpose of the macros in stringize.hpp, but your 2nd 
> example does not in fact require the ## operator to get the 
> behavior you want with Microsoft C++.  If you delete the ## 
> out of macro B, leaving everything else in place, there is no 
> difference in the behavior; you still get p + q.  I tried 
> this with Microsoft Visual C++ 5 (v11) and Microsoft Visual 
> C++ 2005 (v14):
> 
> #define IM p, q
> #define A(im) B((im))
> #define B(par) C par    // no ## here
> #define C(x, y) x + y
> A(IM) // p + q
> 
> So despite your impressive knowledge of Microsoft's 
> preprocessor bugs, I'm not really convinced the ## is needed 
> in stringize.hpp, at least with Microsoft C++.

This happens to work here because of a different bug.  Consider:

#define EMPTY
#define A() 123

A EMPTY () // expands to A (), as it should.

#define B() A EMPTY ()

B() // expands to 123, as it shouldn't.

What is happening here is that the MS preprocessor is doing an extra scan for
expansion when it shouldn't.  The first scan for expansion yields:

A ()

But then an extra scan (that shouldn't be there) is being applied that causes
A() to expand.  The same thing is happening above: it is not expanding 'im' when
it should, but instead picks up that expansion during the first scan, resulting
in:

C (p, q)

Then, it is going back and expanding C during the extra scan (that shouldn't be
there).  So, it is working in this trivial example, for the wrong reason, and
this other bug doesn't undo the effect (completely) of the first bug in more
complicated scenarios.  What happens is that you get a build up of things that
aren't yet done that are dependent on each other.  Most of the time the
workarounds in the library pick all of these up.

The problem is that there is no way (AFAIK) to force the preprocessor to expand
things when it should.  At most, it is coaxing it do it.

> I see the similar ## usage in cat.hpp and iif.hpp, but that 
> code seems limited to Metrowerks.  I'm really only interested 
> in Microsoft C++.  In stringize.hpp was the only place ## was 
> used to produce an invalid token when I built boost with 
> Microsoft C++ ("bjam -sTOOLS=vc-8_0").  There may be others 
> places, but those wheels didn't squeak for me.

The VC++ configuration doesn't use it to the degree that I was thinking it did
(confusing the workarounds used for MS and for MW).  However, if you look at the
revision history of "stringize.hpp", it was modified to fix a specific instance
of the problem I've been referring to:

BOOST_PP_STRINGIZE( ( BOOST_PP_SEQ_ENUM((x)(y)(z)) ) )

If STRINGIZE is defined in the canonical way (the way that should work):

#define STRINGIZE(x) PRIMITIVE_STRINGIZE(x)
#define PRIMITIVE_STRINGIZE(x) #x

STRINGIZE( ( BOOST_PP_SEQ_ENUM((x)(y)(z)) ) )

...this fails--which was the reason for the workaround at this particular point.
Modifying STRINGIZE to:

#define STRINGIZE(x) STRINGIZE_I((x))
#define STRINGIZE_I(par) STRINGIZE_II par
#define STRINGIZE_II(x) #x

This works for this particular example (which--if I remember correctly--was the
motivating reason for the change).  I'll change "stringize.hpp", but if people
start having problems, I'll have to roll it back.  That may or may not happen
(the pp-lib is not 100% stable on VC++ anyway because of these exact issues) and
stringizing is not all that common in preprocessor metaprogramming.

You have to understand that these kinds of workarounds are not simply a local
application with a local effect as you seem to think.  I.e. you can't tell if it
is effective by just testing STRINGIZE in isolation.  It has to be tested in a
much more combinatorial way.  For example, these trivial scenarios make it
appear that a simple uniform application of the workaround (with or without ##)
will cause everything to work correctly.  That, however and unfortunately, is
not the case.  An example of the type of build up that occurs is (trying to keep
it simple):

#define E()

#define A() B E E E()()()()
#define B() 123

A() // B E ()()

Note that we're talking about VC++ here.  The result in this example *should*
be:

B E E()()()

...but the extra scan that VC++ is applying is causing the extra EMPTY
expansion.  However, that's just an aside, not the point.  The EMPTY's represent
build up accrues over the course of a complex macro expansion.  With some
effort, we can induce it to expand B in this particular case:

#define DO(x) DO_I((x))
#define DO_I(par) DO_II par
#define DO_II(x) x

DO( A() ) // 123

However, the amount of build up is variable--depending on how an argument is
constructed.  E.g.

#define C() B E E E E()()()()()

DO(C()) // B ()

Now the extra coaxing applied by DO is not enough.  Technically speaking, it
shouldn't be enough, but neither should the build up exist.  The point of all
this is that you can't take a trivial example and say that it proves that it
works.  It doesn't prove anything--you have to show stability with large-scale
examples.

(BTW, building Boost is not an effective test of whether the pp-lib works.  The
libraries that need building don't use even close to all of the library, so it
isn't an adequate test.  You should at least include the header-only libraries.)

Regards,
Paul Mensonides