Re: [boost] Review Request: Variadic Macro Data library

Subject: Re: [boost] Review Request: Variadic Macro Data library
From: Edward Diener (eldiener_at_[hidden])
Date: 2011-02-21 12:57:05

On 2/21/2011 3:57 AM, Paul Mensonides wrote:
> On Sun, 20 Feb 2011 13:55:47 -0500, Edward Diener wrote:
>
>> On 2/20/2011 12:19 PM, Paul Mensonides wrote:
>
>>> All uses of macros are metaprogramming.  The more programmers
>>> understand that, the less "macro problems" there will be.
>>
>> A library presents an interface where one uses a macro to do
>> "something". The programmer using that macro to accomplish some task in
>> that library. The use of that macro is not "metaprogramming" as I
>> understand it, although I think this is just an argument about a word
>> meaning than any real disagreement. Imagine a relative newbie in C++
>> using a macro and then being told he is "metaprogramming". No doubt he
>> would feel a little proud of that fact<g>, but I doubt if that would
>> prove much about his real "metaprogramming" knowledge or skills, not
>> that I think "metaprogramming" is that abstruse.
>>
>> Undoubtedly a C++ programmer should understand macros. But whether he
>> understands it at his own level and whether he understands it as you do
>> are different things.
>
> What I'm getting is actually what you mention before, mimicking function
> calls.  That even being a goal (or considered a good thing by accident)
> rather than being benign (at best) is where things go wrong.  Macros are
> code generators (or, I suppose, code "removers").  They are not even
> remotely tied to the syntax of the underlying language.  When a macro
> (that isn't a general purpose pp-lib macro) is defined, all too often the
> author designs the macro such that its invocation appears to fit into the
> syntax of the underlying language.  For example, hacks such as using do/
> while loops in order to require a semicolon outside the macro.
>
> #define REGISTER(...) \
>      do { \
>          /* whatever */ \
>      } while (false) \
>      /**/
>
> int main() {
>      REGISTER(a, b, c); //<--
> }
>
> That point of view is evil.
>
> Demanding that competent C++ programmers have a proper view of macros is
> not demanding that they know all of the tricks that I or others know to
> manipulate the preprocessor.  However, it does require them to know that
> macros are a fundamentally different thing with fundamentally different
> mechanics (even if they don't understand those mechanics to the level of
> detail that I or others do).
>
>>> So, what's the problem with SOME_MACRO((a, b, c)) instead?  I.e. a
>>> tuple rather just variadic content?
>>
>> By the same token what is wrong with SOME_MACRO(a,b,c) ?
>
> The interface is (potentially) silently broken when some other argument
> needs to be passed.  I.e. MACRO(a, b, c) =>  MACRO(id, a, b, c) is
> potentially a silent break.  MACRO((a, b, c)) =>  MACRO(id, (a, b, c)) is
> not.
>
>> The point is
>> not that their is something wrong with SOME_MACRO((a,b,c)) or even with
>> SOME_MACRO((a)(b)(c)) but that programmers using macros which a library
>> provides seem more comfortable with SOME_MACRO(a,b,c). The reason for
>> this is most probably because the latter mimics function call syntax and
>> a library implementer is looking syntactically for as much sameness as
>> possible in order to promote ease of use ( or memorability ).
>
> Another way to provide comfort is via education.  Hardcore pp-
> metaprogramming knowledge is not required for this.

Providing function-like syntax for invoking a macro with a variable 
number of parameters, as an alternative to pp-lib data syntax, is 
important to end-users and library developers if just for the sake of 
familiarity and regularity. A programmer using a "call" syntax which may 
be a macro or a function is not going to stop and say: this is a 
function so I can call it as 'somefunction(a,b,c)', this is a macro and 
therefore I must call it as 'somemacro((a,b,c))'. Instead he will ask 
that the same syntax be applied to both. You seem to feel this is wrong 
and that someone invoking a macro should realize that it is a macro ( 
and normally does because it is capital letters ) and therefore be 
prepared to use a different syntax, but I think that regularity in this 
respect is to be valued.

>
>> One may just as well ask why C++ adapted variadic macros as part of the
>> next proposed standard ? I believe it was solely for syntactic ease of
>> use with a variable amount of macro tokens. It sure wasn't because C++
>> provided much functionality to deal with the variadic data per se.
>
> Actually, it is more likely just for the sake of compatibility with C.  I
> originally proposed adding them at a committee meeting in Redmond.  That
> proposal got rolled into a general "compatibility with C preprocessor"
> proposal.
>
>>> Believe it or not, I actually use preprocessor metaprogramming (as well
>>> as template metaprogramming) quite heavily in my own code.
>>
>> Nah ! I don't believe it<g>.
>
> Shocking, but true!
>
>>>   The only time
>>> that I would provide a macro that uses the variadic data as an element
>>> sequence (that's not a general-purpose pp-library macro) as an
>>> interface is if it is absolutely certain that it will never become
>>> higher-order and that the number of arguments before the variadic data
>>> will never change.
>>
>> Fair enough. But I believe some other proposed Boost libraries, besides
>> my own TTI library, are using variadic macro syntax in their public
>> interface. Why should they not do that and take advantage of pp-lib at
>> the same time ?
>
> As I said before, I don't have a problem with these particular macros.
> I.e. DATA_TO_SEQ(__VA_ARGS__).  I only have a problem with ALGORITHM_B
> (...) =>  ALGORITHM_A(DATA_TO_SEQ(__VA_ARGS__)).  There are lots of other
> uses of DATA_TO_SEQ (et al) and element extraction (ala TUPLE_ELEM)
> besides this.  As long as the pp-lib doesn't provide definitions like
> ALGORITHM_B, I don't have a problem with it.  If users provide that
> themselves, in spite of a lack of endorsement for the library, the
> ramifications are on their own heads.
>
> The following are the chaos-pp analogs of the macros that you have in the
> sandbox right now.  If they were added to the pp-lib (where they belong
> if they are worthwhile) these are the names that I would use (not because
> I'm wedded to the names, but for symmetry with existing practice).
>
> ----
>
> BOOST_VMD_DATA_SIZE(...)
>
> Chaos has no direct analog of this ATM.  Normally the functionality is
> produced via CHAOS_PP_TUPLE_SIZE((...)).  However, if it existed it would
> be called CHAOS_PP_VARIADIC_SIZE(...).  If I (or you) added it to the pp-
> lib, I would prefer it be called BOOST_PP_VARIADIC_SIZE(...) for symmetry
> with existing practice.
>
> ----
>
> BOOST_VMD_DATA_ELEM(n, ...)
>
> The direct analog of this in Chaos is CHAOS_PP_VARIADIC_ELEM(n, ...).  If
> I (or you) added it to the pp-lib, I would prefer it be called
> BOOST_PP_VARIADIC_SIZE(...).

Did you mean BOOST_PP_VARIADIC_ELEM(n,...) ?

>
> ----
>
> BOOST_VMD_DATA_TO_PP_TUPLE(...)
>
> Chaos has no direct analog of this because (as you know), it's pointless
> (unless you're doing something internally to for compiler workarounds).

I do not think it is pointless. I am going variadics -> tuple. The 
end-user wants regularity, even if its a no-brainer to write '( 
__VA_ARGS__ )'.

I think this is where we may differ, not technically, but in our view of 
what should be presented to an end-user. I really value regularity ( and 
orthogonality ) even when something is trivial. My view is generally 
that if it costs the library developer little and makes the end-user see 
a design as "regular", it is worth implementing as long as it is part of 
the design even if it is utterly trivial. You may feel I am cosseting an 
end-user, and perhaps you are right. But as an end-user myself I 
generally want things as regular as possible even if it causes some very 
small extra compile time.

>
> -----
>
> BOOST_VMD_DATA_TO_PP_{ARRAY,LIST,SEQ}(...)
>
> Chaos has no direct analog for any of these because it doesn't consider
> variadic data to be a sequential data structure.  Chaos does have
> conversion macros to one data structure to another.  However, the basic
> functionality to do that is implemented by one generic algorithm:
>
>      CHAOS_PP_CAST(CHAOS_PP_SEQ, (CHAOS_PP_TUPLE) (a, b, c))
>          =>  (a)(b)(c)
>
>      CHAOS_PP_CAST(CHAOS_PP_LIST, (CHAOS_PP_TUPLE) (a, b, c))
>          =>  (a, (b, (c, ...)))
>
> The library normally only supplies direct non-generic versions when there
> is a reasonable efficiency gain over the generic version or when there is
> an interesting implementation mechanic that I want to preserve.  There
> are a lot of these direct conversions for CHAOS_PP_SEQ and
> CHAOS_PP_TUPLE.  The current set is:
>
>      CHAOS_PP_ARRAY_TO_LIST
>      CHAOS_PP_ARRAY_TO_SEQ
>      CHAOS_PP_ARRAY_TO_STRING
>      CHAOS_PP_ARRAY_TO_TUPLE
>
>      CHAOS_PP_SEQ_TO_ARRAY
>      CHAOS_PP_SEQ_TO_LIST
>      CHAOS_PP_SEQ_TO_STRING
>      CHAOS_PP_SEQ_TO_TUPLE
>
>      CHAOS_PP_TUPLE_TO_LIST
>      CHAOS_PP_TUPLE_TO_SEQ
>      CHAOS_PP_TUPLE_TO_STRING
>
> The primary reason that there is no direct conversion (other than
> (__VA_ARGS__)) to the other data types is that with variadic data there
> is no such thing as an empty sequence and there is no safe way to define
> a rogue value for it without artificially limiting what the data can
> contain.

I understand this.

My point of view is that although it's theoretically a user error to use 
an empty sequence for a variadic macro even if a corresponding pp-lib 
data type can be empty, in reality it should be allowed since there is 
no way to detect it. I understand your point of view that you want to do 
everything possible to eliminate user error, and I agree with it, but 
sometimes nothing one can do in a language is going to work ( as you 
have pointed out with variadics and an empty parameter ). In that case I 
do not see why functionality should not be provided anyway and if the 
user then uses it accidentally it becomes his problem. To not provide 
functionality because a user error, which can not be detected, might 
occur is not the way I usually view software design ( sorry to sound 
dogmatic, because I am normally very practical as a programmer ).

In the case of converting from variadics to pp data types, if the 
end-user passes an empty sequence, and the pp=lib data type supports an 
empty sequence, then it is fine with me.

Going the other way from a pp data type to variadics, I admit I did not 
consider the case in my current library where the pp data type is empty. 
Since this can be detected on the pp data type side, I think I can put 
out a BOOST_PP_ASSERT_MSG(cond, msg) in that case, or I can choose to 
ignore it if that is what I decide to do. But in either case it is a 
detectable problem.

>
> ----
>
> BOOST_VMD_PP_TUPLE_SIZE(tuple)
>
> The direct analog of this in Chaos is CHAOS_PP_TUPLE_SIZE(tuple).  If I
> (or you) added this to the pp-lib, it should be just BOOST_PP_TUPLE_SIZE
> (tuple).
>
> ----
>
> BOOST_VMD_PP_TUPLE_ELEM(n, tuple)
>
> The direct analog of this in Chaos is CHAOS_PP_TUPLE_ELEM(n, tuple) where
> it ignores the 'n' if variadics are enabled.

The 'n' tells which tuple element to return. How can it be ignored ?

>
>      CHAOS_PP_TUPLE_ELEM(?, 0, (a, b, c)) =>  a

OK, I see. the '?' is just a marker for the size of the tuple when there 
are variadics and must be specified when there are not.

>
> If this was added to the pp-lib, I would prefer that it did so the same
> way.  However, another way would be "overload" the macro (which as no
> direct analog in the pp-lib or your library).  Basically, that ends up
> being something like:
>
>      #if (variadics enabled)

That is an interesting technique below. Bravo ! As long as it is 
documented, since it confused me until I took a few more looks and 
realized what you are doing.

>
>      #define BOOST_PP_TUPLE_ELEM(...) \
>          BOOST_PP_CAT( \
>              BOOST_PP_TUPLE_ELEM_, \
>              BOOST_PP_VARIADIC_SIZE(__VA_ARGS__) \
>          )(__VA_ARGS__) \
>          /**/
>
>      #define BOOST_PP_TUPLE_ELEM_2(n, tuple) // ...
>      #define BOOST_PP_TUPLE_ELEM_3(size, n, tuple) \
>          BOOST_PP_TUPLE_ELEM_2(n, tuple) \
>          /**/
>
>      #else
>
>      #define BOOST_PP_TUPLE_ELEM(size, n, tuple) // old way
>
>      #endif
>
> -----
>
> BOOST_VMD_PP_TUPLE_REM_CTOR(tuple)
>
> The direct analog of this in Chaos is CHAOS_PP_TUPLE_REM_CTOR (which,
> BTW, stands for "constructed parentheses removal").  The "constructed"
> refers to when the tuple is the direction result of a macro expansion.
> CHAOS_PP_REM_CTOR(?, MACRO()).  Otherwise, if you already have a tuple
> 't', just use CHAOS_PP_REM t.  ATM, Chaos has CHAOS_PP_TUPLE_REM,
> CHAOS_PP_TUPLE_REM_CTOR, and CHAOS_PP_REM.  It does not have a
> CHAOS_PP_REM_CTOR.
>
> ----
>
> BOOST_VMD_PP_TUPLE_{REVERSE,TO_{LIST,SEQ}}(tuple)
>
> If added to the pp-lib, should be the same as BOOST_PP_TUPLE_ELEM above.
> Either ignored size argument or overload.
>
> ----
>
> BOOST_VMD_PP_TUPLE_TO_DATA(tuple)
>
> Isn't this just REM_CTOR?

Sure. But again I value regularity even when something is trivial.

>
> ----
>
> BOOST_VMD_PP_{ARRAY,LIST,SEQ}_TO_DATA(ds)
>
> These I don't see the point of these--particularly with the pp-lib
> because of compiler issues.  These already exist as BOOST_PP_LIST_ENUM
> and SEQ_ENUM.  There isn't an ARRAY_ENUM currently, but it's easy to
> implement.  The distinction between these names and the *_TO_DATA variety
> is that these are primary output macros.

That's just a name. Names can always be changed. My macros are the same 
as yours in that they are output macros which convert the pp-lib data 
types to variadic data.

>  If an attempt is made by a user
> to use the result as macro arguments, all of the issues with compilers
> (e.g. VC++) will be pulled into the user's domain.

The returned variadic data is no different from what the user will enter 
himself when invoking a variadic data macro. The only compiler issue I 
see is the empty variadic data one.

I do see the point of these macros again as a simple matter of regularity.

>
> ----
>
> To summarize what I would prefer:
>
> BOOST_VMD_DATA_SIZE(...)
>    ->  BOOST_PP_VARIADIC_SIZE(...)
>
> BOOST_VMD_DATA_ELEM(n, ...)
>    ->  BOOST_PP_VARIADIC_ELEM(n, ...)
>
> BOOST_VMD_DATA_TO_PP_TUPLE(...)
>    ->  (nothing, unless workarounds are necessary)

I know its trivial but I still think it should exist.

>
> BOOST_VMD_DATA_TO_PP_ARRAY(...)
>    ->  BOOST_PP_TUPLE_TO_ARRAY((...))
>    or BOOST_PP_TUPLE_TO_ARRAY(size, (...))
>
> BOOST_VMD_DATA_TO_PP_LIST(...)
>    ->  BOOST_PP_TUPLE_TO_LIST((...))
>    or BOOST_PP_TUPLE_TO_LIST(size, (...))
>
> BOOST_VMD_DATA_TO_PP_SEQ(...)
>    ->  BOOST_PP_TUPLE_TO_SEQ((...))
>    or BOOST_PP_TUPLE_TO_SEQ(size, (...))

For the previous three, see above discussion about using 
SOME_MACRO(a,b,c) vs. SOME_MACRO((a,b,c)). I do understand your reason 
for this as a means of getting around the empty-variadics user error. 
But I still feel that treating variadic data here as tuples is wrong 
from the end-user point of view even though it elegantly solves the 
empty variadic data problem. In my internal code I am solving the 
problem in the exact same way, but I am keeping the syntax as 
SOME_MACRO(a,b,c) as opposed to SOME_MACRO((a,b,c)).

So I would say, please consider using the SOME_MACRO(a,b,c) instead as I 
am doing.

I would even say to change my names to:

BOOST_PP_ENUM_TUPLE(...)
BOOST_PP_ENUM_ARRAY(...)
BOOST_PP_ENUM_LIST(...)
BOOST_PP_ENUM_SEQ(...)

in order to match the ENUM names you already have in pp-lib for 
converting the other way and which you have below.

But keep the SOME_MACRO(a,b,c) like syntax.

>
> BOOST_VMD_PP_TUPLE_SIZE(tuple)
>    ->  BOOST_PP_TUPLE_SIZE(tuple)
>    or BOOST_PP_TUPLE_SIZE(size, tuple)
>
> BOOST_VMD_PP_TUPLE_ELEM(n, tuple)
>    ->  BOOST_PP_TUPLE_ELEM(n, tuple)
>    or BOOST_PP_TUPLE_ELEM(size, n, tuple)
>
> BOOST_VMD_PP_TUPLE_REM_CTOR(tuple)
>    ->  BOOST_PP_TUPLE_REM_CTOR(tuple)
>    or BOOST_PP_TUPLE_REM_CTOR(size, tuple)
>
> BOOST_VMD_PP_TUPLE_REVERSE(tuple)
>    ->  BOOST_PP_TUPLE_REVERSE(tuple)
>    or BOOST_PP_TUPLE_REVERSE(size, tuple)
>
> BOOST_VMD_PP_TUPLE_TO_LIST(tuple)
>    ->  BOOST_PP_TUPLE_TO_LIST(tuple)
>    or BOOST_PP_TUPLE_TO_LIST(size, tuple)
>
> BOOST_VMD_PP_TUPLE_TO_SEQ(tuple)
>    ->  BOOST_PP_TUPLE_TO_SEQ(tuple)
>    or BOOST_PP_TUPLE_TO_SEQ(size, tuple)
>
> BOOST_VMD_PP_TUPLE_TO_DATA(tuple)
>    ->  BOOST_PP_TUPLE_REM_CTOR(tuple)
>    or BOOST_PP_TUPLE_REM_CTOR(size, tuple)

Again I value the orthogonality of the pp-data to variadic data idea in 
common names. BOOST_PP_TUPLE_REM_CTOR does not suggest that to the 
end-user. How about:

#define BOOST_PP_TUPLE_ENUM(tuple) \
BOOST_PP_TUPLE_REM_CTOR(tuple)

in order to mimic your three following names.

>
> BOOST_VMD_PP_ARRAY_TO_DATA(array)
>    ->  BOOST_PP_ARRAY_ENUM(array)
>
> BOOST_VMD_PP_LIST_TO_DATA(list)
>    ->  BOOST_PP_LIST_ENUM(list)
>
> BOOST_VMD_PP_SEQ_TO_DATA(seq)
>    ->  BOOST_PP_SEQ_ENUM(seq)
>
> also add:
>    BOOST_PP_REM, BOOST_PP_EAT

OK.

>
> The basic gist is to add the low-level variadic stuff and adapt the
> existing tuple stuff to not require the size.

I think our only real disagreements can be summed up as:

I want the end-user to view variadic data as such from a perceptual 
point of view, even with the 
empty-variadics-is-an-error-which-can-not-be-caught problem. That is why 
I supply the various conversions from variadic sequences to pp-lib types 
and back explicitly, and I want some regularity in names reflecting that 
although I do not insist on my own names.

You feel that variadics as input for conversion should in general be 
treated as a pp-lib tuple since creating a tuple from variadic macro 
data is trivial.