Re: [boost] PP: BOOST_PP_IS_SEQ wanted

From: Paul Mensonides (pmenso57_at_[hidden])
Date: 2003-04-15 18:00:43

Aleksey Gurtovoy wrote:
>>> Why? I understand that, as the library author, you would much prefer
>>> all the facilities work properly on all the compilers (I, for
>>> instance, wish that was true for MPL as well), but _not including_ a
>>> generally useful and wanted by real-world users functionality into
>>> the library because a couple of vendors have broken compilers is
>>> exactly the strategy I wouldn't expect any boost library to take.
>>
>> And it is the strategy taken by the pp-lib as well.
>
> I suppose you meant the opposite.

Yes, what I meant was that the pp-lib makes a great effort to work on as many
compilers as possible.

>> The sheer amount of hacks contained in the pp-lib to make things work
>> on broken preprocessors is evidence of this.  It contains nearly four
>> separate implementations of the same thing--just for that purpose.
>
> I know, and your willingness to make the library work on these broken
> preprocessors is much appreciated.
>
>> However:  at a certain point, that level of 'hacking' becomes a
>> maintenance nightmare, as does fracturing portability by
>> enabling and disabling features on various compilers.
>
> I would say I don't see how saying "this feature doesn't works on such
> and such compilers" leads to a maintenance nightmare. It's definitely
> less work for the maintainer, isn't it? Say, if portability wasn't
> one of the important goals of the MPL, I would _gladly_ give up some
> of the workarounds for a simple statement "this doesn't work on broken
> compilers such as ...". It would definitely make library's maintenance
> way easier.

No, but saying that "this feature is unstable on several compilers" would.
BOOST_PP_REPEAT + BOOST_PP_SEQ_ELEM is an example of just how brittle things can
be running under VC++.  (One way or another, that is something that I _can_ make
work properly, so I will fix that when I can get to it.)  The maintenance
problems start to come up when library primitives are combined on buggy
preprocessors.  The primary problem is _encapsulation_ and avoiding a
*combinatorial* nightmare when these "next-level" primitives are used by user
code.  It isn't just a question of whether or not feature X with work on
preprocessor Y, it is a problem with whether feature X will work with feature Y
on preprocessor Z.

>> Also, in regards to IS_SEQ specifically, the macro is *unsafe*.  The
>> level of generality that it implies does not exist without variadics
>> -- and even then things can be ambiguous.  Specifically, in this
>> case,
>> it cannot accept arbitrary input:
>>
>> BOOST_PP_IS_UNARY( (a, b, c) ) // error
>
> Well, technical feasibility is a different question. What I am opposed
> to is rejection of useful and relevant functionality from the library
> because of "portability fracturing", especially since IS_SEQ is not
> the
> only primitive that was turned down under that line of reasoning.

I am not rejecting useful and relevant functionality.  I'm saying only that the
functionality is not exposed as an "official" interface.

> As for the above issue, pp-sequences themselves cannot contain that
> kind of data, can they? For instance, I am pretty sure this one will
> fail as well
>
>     BOOST_PP_SEQ_ELEM(0, (a, b, c)(a))
>
> although, strictly speaking, (a, b, c)(a) _is_ a sequence. Frankly, I
> don't think it's an important corner case either. For anyone more or
> less familiar with C++ preprocessor it's obvious that the above won't
> work, and why.

It isn't a corner case.  The long term final result of the sequence mechanism is
direct support for variadic sequences, as well as nil sequences.  This
functionality is already available in C99.  With variadics, it is easy to
manipulate arbitrary sized tuples.  However, that isn't the common case.  The
common case is something like this:

(a, b)(a, b)(a, b)

I.e. a binary sequence.  Variadics sequences support any size with only one
mechanism.  Likewise, I could have designed the sequence mechanism to support
nil sequences directly, and consequently make using sequences more difficult,
but I chose not to because that is not what it will be eventually.  Besides,
what you're doing is overloading a macro based on the "type" of its operand.
This reminds me of the conversation we had a while back regarding generic
algorithms for different pp-data types.  The different types are syntactically
too similar to be differentiated in this way--even with variadics.  Compiler
problems aside, the name BOOST_PP_IS_SEQ implies a sort of "type" detection,
which I cannot do in general.  With the IS_UNARY detection process, what your
doing is very low-level.  It is not a high-level facility that would qualify for
a name like BOOST_PP_IS_SEQ.  Which is why I prefer the more low-level name
BOOST_PP_IS_UNARY.  If you *really* think I should, I'll promote those detection
macros to interface macros, and see how stable I can make the COMPARE mechanism,
because I respect your opinion.  However:  These facilities are
*use-at-your-own-risk*.

>> This is an important concern in preprocessor metaprogramming because
>> it causes tight bindings between various library primitives and
>> user-code.  Ultimately, IS_UNARY is a low-level detection mechanism
>> that is part of a "next-level" preprocessing methodology that is only
>> minimally used even in the pp-lib itself and only in *very*
>> controlled circumstances.  The pp-lib cannot support this
>> "next-level" in any
>> general fashion. Even the use of IS_UNARY, et al, is "dicy" on
>> Metrowerks and VC++.  It will not work in contexts that are not
>> strictly controlled, because it directly relies on expansion order
>> -- something that is fundamentally flawed on those preprocessors.
>> So, in the end, what that means is that it is unsafe to use in
>> user-code without the same level of hacks that are used in the pp-lib
>> itself.  The problems with VC++ and Metrowerks are serious
>> encapsulation issues, which is why you're having the REPEAT +
>> SEQ_ELEM bug, and if I "publish" macros of this nature, this type of
>> problem will occur more and more often.
>
> Hmm, that's kind of news to me. What about the IS_SEQ implementation I
> posted? Is it "unstable" as well?

Yes.  Any time that you mess around with expansion order it becomes unstable,
which that implementation does when it uses half-open parenthesis.  The same
type of issue occurs with commas:

#define A 1, 2
#define B(x) C(x)
#define C(x, y) x y

B(A) // should expand to:  1 2

It requires things to expand at *exactly* the right time, which will work
*sometimes* and not other times on VC.

>> Ah, okay.  BOOST_PP_IS_UNARY will do it for you, _if_ the input is
>> either a unary sequence or operatic.
>
> No, it's either alphabetical date or multi-element sequence.
>
>> Nothing else with work portably. E.g. textual input will fail on
>> several compilers.
>
> Mine implementation seems to work fine at least on MSVC and
> Metrowerks.
> We don't care about the rest :).

It will work fine in many cases, as will IS_UNARY.  There are two basic
encapsulation problems with VC.  First, macro expansion does not always occur
when it should, and second, sometimes macro expansion occurs when it should not.
The net result is that these expansions get picked up at later times, many times
inside the pp-lib itself rather than user code.  In many cases, this results in
exact same result.  However, when you start to mess with parentheses and commas
in a fashion that depends on expansion order things can go bad really quick.

>> As I said:  the facility exists already but has major drawbacks.
>> It is not that I'm not trying to be helpful. Rather I think it
>> establishes a precendent that I don't want to have to deal with down
>> the road.
>
> I can understand the technical reasons for not including the
> facilities in the library - if that's where it will end up. I am not
> fond of "everything should work on every major compiler" line of
> reasoning - especially considering that many users don't care about
> some of those "major" vendors.

What I'm referring to is not a case of "everything should work on every major
compiler."  What I'm referring to is that each subsection of the library, for
instance, the sequence mechanism should be fully supported or not supported on
any given vendor.  What I don't want to see is a proliferation of individual
macros that work on preprocessor group A, work unreliably on preprocessor group
B, and don't work at all on preprocessor group C.  What I'll get is a great deal
of feedback like, "pp-lib bug: xyz" and "why doesn't this work?", and would also
seriously complicate portable usage of the library.

The detection macros, IS_NULLARY, IS_UNARY, COMPARE, etc., I can put into a
separate "detection" folder and call it a separate subsection of the library.
Then I can simply say that this subsection is not supported on Borland, IBM,
Sun, and ilk, and is only partially supported on VC++ and Metrowerks.

>> The "strict" pp-lib's primary purpose is as a concept library, a
>> teaching tool, and an idealistic reference implementation.  Whether
>> or not people actually use it in a production environment is up to
>> them--i.e. if it is worth relying on a secondary build tool.
>
> Well, that's reassuring to hear, because as much as I am excited about
> this new work, I would hate to see a drop of support and enhancement
> for the "traditional" version of one of the most useful boost
> libraries
> to date.

I'm not giving up on it, it is just that basic functionality is already there.
E.g. you can solve the "overloaded" problem in other ways, it just isn't as
"clean."  But, the types of things that you're thinking about now (such as
lambda, identifier comparison, and detection) are all much more difficult things
to "hack" into reasonable usefulness on buggy preprocessors.  I could probably
make some sort of (really ugly) lambda mechanism work on VC, but it would be
very elaborate and *very* hacked together.  I don't think that I could do it
with Metrowerks at all--which has worse problems than VC (such as magically
disappearing tokens, etc.).  The detection stuff I can get to work in "most
cases," but not all on both Metrowerks and VC.

Regards,
Paul Mensonides