Re: Understanding MPL

From: Paul Mensonides (yg-boost-users_at_[hidden])
Date: 2003-04-03 18:22:43

Edward Diener wrote:

>> What specifically would you have me do?  The pp-lib docs do not
>> attempt to explain the implementation of the pp-lib itself.
>
> I don't want an explanation of the inner workings of the pp-lib. I
> would like to see an explanation of the pp-lib which attempts to
> explain the functionality of it from the user's point of view.
> Something like a grouping of the macros in it from the point of view
> of general areas of functionality, with the specific macros grouped
> and explained within each area.

The pp-lib does not contain that type of hierarchal structure.  The structure
that does exist is fairly general.  What you propose would cause me to have to
repeat vast amounts of data that is properly reference data, and would actually
make things more complex and significantly more of a pain to use for those that
_are_ initiated.

> The topic section does not do this. Instead it jumps around through
> various ideas I find almost impossible to understand. Furthermore
> nearly everything is done at this level by showing examples rather
> than explaining what things are. I admit I hate this mode of
> explaining computer programming which gives an example with little
> explanation and an attitude of "get it yet ?".

The topics section, at least the parts that I wrote, addresses parts of the
library that are more advanced and require more of an overall picture of the
library design.  It is not intended to act like a reference, it is intended to
provide an explanation for why certain parts of the library are the way they
are.  It also has a few sections on groups of macros/facilities that must be
used in concert--which is unlike the rest of the library.

I think you have an unrealistic picture of what the pp-lib docs should be.  I
seems that you think that documentation alone can make preprocessor
metaprogramming obvious and relatively easy.  That, by documentation alone, the
learning curve can be gradual and straightforward.  That, after reading through
the documentation, you should have a relatively secure grip on how to use the
library without actually getting your hands dirty.  However, this is
_fundamentally_ not the case.  You're dealing with Cpp here, and Cpp is the
antithesis of what is considered good language design.  ;)  It has very little
support for data abstraction and encapsulation.  Systems implemented within it
are very dependent on the interactions of various parts--and how those parts are
implemented themselves.  There is no pure barrier called "interface."  This is a
fundamental issue with the preprocessor and cannot be "documented" away.  It
causes the learning curve to be steep and only experience can make you
comfortable with it.

> Documentation needs to
> explain concepts in a regularized way and present them to the end
> user slowly and logically so that people can build up an
> understanding of things to do.

This is not possible, IMO.

> Take the beginning motivation topic. I am told, as an example of the
> pp-lib, that the library can make it easier to generate repetitive
> overloaded metafunctions. Then at the bottom I am shown a series of
> Boost preprocessing macros to do this which are completely
> non-understandable to anyone coming to the pp-lib.

That is what the reference section is for, and I'm not about to explain what a
macro does every time it is referenced anywhere in the documentation.

> I skip the "known problems of the preprocessor section" and go on to
> techniques. Here I am presented with a series of examples before any
> macros have been explained. This is documentation ? I have no ideas
> what these macros are or what they do.

Then look them up.  I do not have to tools to insert inline definitions of
everything everywhere they are referred to.  I am not allowed to use scripting
and XML/XSLT does not allow for the generation of multiple documents from a
single source (even though most XSLT processors do).

> The second technique mentions
> using BOOST_PP_EMPTY and then this doesn't appear in the example at
> all.

This one is my fault, I think.  It should have the usage also.  The idea is
this:

#define macro(cv) cv()

macro( BOOST_PP_EMPTY ) // nil
macro( const BOOST_PP_EMPTY ) // const

...I'll fix this this weekend.

> I am then told to use various pp-lib macros for various general
> tasks without any explanation of what these macros are and what they
> do. Again this isn't documentation as I define it.

There is a huge amount of explanation of what each macro does.  It takes very
little effort to look them up in the reference section.

> The next section of incompatibilities mentions various macros with
> again no explanation of what these are about. Etc. etc.

Of course not, they are incompatibilities that are only relevent to people that
have existing experience and/or an existing codebase.

> Now I realize there is a reference section which lists each macro
> alphabetically. But this isn't to me an explanation of the areas of
> functionality of the pp-lib presented in any logical order.

In this you are right.  There is no explicit categorization.  There is an
implicit one by the library structure and the headers section.

> What the
> pp-lib docs need is a logical explanation of its functionality so
> that others may understand what is being done here, why it is being
> done, and how it is supposed to be used to supplement normal C++
> programming, whether that programming involves template programming
> or otherwise.

1) "...what is being done here..."

This already exists, however, a categorical index does not.  To be honest,
things can be "categorized" in only the loosest sense of the word except in a
relatively few cases.

2) "...why is it being done..."

This is simple, to provide facilities _of all kinds_ to manipulate programs at
compile-time with the preprocessor.  They are fairly low-level tools to aid in
generative metaprogramming.

3) "...how it is supposed to be used to supplement normal C++ programming..."

This one is not simple.  In fact, it is a thoroughly open-ended question.  Many
types of things can be done.  Some usages already exist, some do not.  However,
that is like asking what 'if', 'while', and 'for' can be used for in
programming.  The preprocessor library has no explicit usage pattern--it is too
low-level for that.  It is, essentially, an implementation of core language
facilities for the preprocessor, in order to make Cpp useful as a programming
environment.

> I don't mean to be harsh in my criticism of the documentation but to
> me it was created for the already "initiated" and not for the C++
> programmer who is trying to gain an idea about the functionality of
> the preprocessor library as to each of its general parts and uses.

I will write up a library overview this weekend.  However, I will not attempt to
structure the documentation in a way that makes the documentation massively
repetitive or in a way the inhibits ease of use.  The "general parts" that you
describe only exist in a few areas, and there are a large number of macros that
are not part of any given group or mechanism.

> The doc needs to be rewritten and presented in a logical manner to
> those who are coming to the pp-lib for the first time and are trying
> to understand its purpose and what problems it hopes to solve or what
> programming tasks it hopes to make easier.

These are all completely open-ended and naive questions.  There already are
examples of problems that have been solved with the pp-lib, yet you say you
don't like learning from examples.  However, these are just a few possibilities.
One of the primary uses of the pp-lib today is repetition of various kinds (the
primary use by the Python library and the MPL is in this area), but that is only
a small portion of the library.  The question of applicability is incredibly
context sensitive.  I guess my advice in this regard is simple, if it cannot be
done reasonably with template metaprogramming, examine whether it is possible to
do it with preprocessor metaprogramming--where "it" is a placeholder for
whatever you might want to do at compile-time.  There are, of course,
limitations to both template metaprogramming and preprocessor metaprogramming,
but only experience with both can tell you whether something is doable or not or
how it might be done.

> The various macros need to
> be grouped in categories which make it easier to understand what
> functionality exists and why.

These are the categories that actually exist:

- Unsigned saturation arithmetic in the range of 0 to 256.
- Manipulation of the "array" data type.
- Unsigned numeric comparisons in the range of 0 to 256.
- Control flow constructs (such as 'if', 'while', etc.)
- Vertical repetition (a.k.a. iteration)
- Manipulation of the "list" data type.
- Logical operations such as 'and' and 'or'.
- Punctuation operators for problem characters like commas and parentheses.
- Horizontal repetition constructs
- Manipulation of the "sequence" data type.
- Evaluated slots.
- Manipulation of the "tuple" data type.

...and a host of other non-groupable macros.

Paul Mensonides