Re: [boost] [vmd] Library Review

Subject: Re: [boost] [vmd] Library Review
From: Edward Diener (eldiener_at_[hidden])
Date: 2014-09-14 18:12:42

On 9/14/2014 5:10 PM, pfultz2 wrote:
>> The point is, however, that you don't know that the concatenation will
>> produce a tuple when you call your APPEND_COMMA. If it does not and just
>> produces an identifier of the form IDENTIFIER_XXX then your
>> APPEND_COMMA(IDENTIFIER_XXX SQUARE) will end up as 'APPEND_COMMA_I
>> IDENTIFIER_XXX SQUARE' which you do not want.
>
> Yes, of course, you will need to check that you have an identifier first.
>
>> Actually I realized that the comma is not be needed at all in Paul's
>> IDENTIFIER macros above. There is already code in VMD that can split a
>> sequence beginning with a tuple into the beginning tuple and the rest of
>> the sequence ( BOOST_VMD_TUPLE ). So the logic of parsing for an
>> identifier at the beginning of a sequence is to start by looking for a
>> beginning tuple in the sequence, if not found test for an identifier
>> through concatenation, and if the subsequent sequence after
>> concatenation now starts with a beginning tuple, the code can separate
>> the beginning tuple from the remainder of the sequence and extract the
>> v-identifier.
>
> Yes, there is similiar logic in the `STRING_TOKEN` macro I showed, except my
> code doesn't handle custom namespaces(ie `STRING_`) nor multiple namespaces.

I am not sure what you mean by namespaces ? Do you mean the type of the 
token parsed ?

>
>> Sorry I can't follow this or udnerstand what you are doing. I don't know
>> what BOOST_PP_AUTO_REC does even when I look at its code <g>.
>
> Well the macro is auto reentrant. So it has a macro for each recursion level
> up to 16(ie `MSVC_INVOKE_1`, `MSVC_INVOKE_2`, etc). The `BOOST_PP_AUTO_REC`
> then will search for a macro that can be called(it hasn't been painted
> blue).
> It does this by calling the `DETAIL_MSVC_INVOKE_P` predicate. Then
> `DETAIL_MSVC_INVOKE_P` wil pass in nullary parenthesis into `MSVC_INVOKE`,
> and
> if that result isn't nullary then it means that recursion level got painted
> blue. So the `BOOST_PP_AUTO_REC` will do a binary search and try to find the
> first predicate that is true. It will return that recursion level, which it
> will concat with `MSVC_INVOKE_` so the correct macro will be called. Does
> that
> make sense?

No, but that's OK.

> Perhaps Paul can explain it better than me.

The BOOST_PP_AUTO_REC macro is not documented so maybe you can 
understand my being puzzled by it and what it is used for. Other 
undocumented macros in Boost PP are generally easier for me to follow.

>
>> You might find this expansion easier with VMD. With Paul's suggestion
>> and my realization that the final comma is not needed, I will be
>> reprogramming the VMD v-sequence macros so that you should be able to
>> take a v-sequence, like your 'string' of 'from(x, numbers) where(x < 3)
>> select(x * x)' and break it down into its constituent v-types without
>> the limitations current VMD incurs. In fact I intend to add a VMD macro,
>> something like BOOT_VMD_SEQUENCE(vsequence) which will return all the
>> v-types in some way ( probably a tuple, seq, or just variadic data, I
>> have yet to decide ).
>
> Of course for my case, I don't need to know what category or type it is(it
> may
> be useful for other people). So having it as a sequence like I've shown is
> very useful. Also, I was thinking it might be useful to have some kind of
> `BELONGS_TO` to query which namespace a token belong to.

The BOOST_VMD_SEQUENCE macro I plan will return for each v-type found 
not only its data by is v-type also. I am leaning toward returning the 
data as a Boost PP seq where every element of he seq is a tuple of 
(type,value) pairs. Each type will be a number identifying a distinct 
v-type (number,identifier,array,list,seq,tuple). Then you could iterate 
through that returned seq, probably with BOOST_PP_SEQ_FOR_EACH, and do 
what you want with the type/data to create your final construct. This 
will allow a v-sequence, what you call a 'string' in your example, to be 
parsed for its individual pieces of VMD v-type data.

You could actually do something similar with VMD now but you would have 
to try each v-type separately to see if it begins the v-sequence. But 
after Paul Mensonides provided a better solution to identifying 
v-identifiers and v-numbers I can improve the parsing of VMD v-sequences 
greatly.

In fact with VMD a v-sequence becomes another preprocessor "data type" 
similar to the high level Boost PP data types of array, list, seq, and 
tuple. I should really work to produce whatever operations with a 
v-sequence I can that would mimic some of those Boost PP data types. Its 
challenging to think about but it will be fun work except for the usual 
VC++isms.