Re: [boost] Boost.Preprocessor: A prototype pow(N,2) max direct iteration WHILE.

From: Vesa Karvonen (vesa_karvonen_at_[hidden])
Date: 2003-01-14 16:01:59

David Abrahams:
>FWIW, while in my opinion this is theoretically interesting work,

BTW, the same "acceleration" technique can be used with templates to 
overcome template recursion depth limitations. Probably most people know 
about it already, though I haven't explicitly checked. (Plain unrolling is 
not as effective: Theta(k*n) < Theta(pow(k,n)).)

>I can't imagine a case where the limit might cause a problem for my
>code.  In other words, the PP library works great for me, and nobody
>should feel pressure to lift its arbitrary limits on my account.

IMO, it is not a very good argument to say that more powerful tools are 
unnecessary, because a specific person (or even a set of persons) does not 
find them useful. It must have been said a million times before and probably 
every time it has been shown to be wrong (in time). A much better argument 
would be that the same effect can be had with only minimal overhead compared 
to the proposed tool with existing tools, thus making the proposed tool 
mostly unnecessary.

There are a few good practical reasons why a more powerful WHILE may not be 
very useful. One good reason is that most preprocessors are rather slow. 
This means that even if we could theoretically use the preprocessor for 
non-trivial compile time computation, it may be too slow to be practical. 
Another good reason is that preprocessor code is not necessarily very 
readable, so many interesting things may require too much programming time. 
Essentially,

"Why program by hand in 5 days what you can spend 5 years of your life 
automating?" -- Terence Parr

(, but with the opposite conclusion.)

However, I think that there are many useful things that could potentially be 
done using the preprocessor if the arbitrary limits on iteration would be 
removed and performance would be acceptable. Consider, for example, the 
state machine example of the MPL paper. A state machine generator using 
template metaprogramming techniques is essentially limited to having to 
specify functions as the actions. This can be partially circumvented by 
using a lambda library, but the solution is not perfect.

A distinct possibility would be to use the preprocessor to generate the 
transition function. What makes this interesting is that then we could pass 
in straight C++ blocks as actions. Consider the following pseudo-code 
snippet, which is a start of the core of a simplified C-style string literal 
to a string conversion function. The example below is inaccurate and 
incomplete. I don't have the time now to make a fully polished version. Just 
take it as an example of what kind of syntax could be achieved.

  std::string string_literal_to_string(std::string literal) {
    std::string result;
    state_type state=initial;

    for (int i=0; i<literal.size(); ++i)
      STATE_MACHINE(state, literal[i], { throw std::exception("Invalid 
string literal."); },
                    ( initial, (   is('"'), inside, { } ) )
                    ( inside,  (   is('"'), accept, { } )
                               (  is('\\'), escape, { } )
                               ( otherwise, inside, { result += literal[i]; 
} ) )
                    ( escape,  (   is('"'), inside, { result += '"';} )
                               (   is('n'), inside, { result += '\n';} )
                               (   is('t'), inside, { result += '\t';} )
                               /* ... */ )
                    /* ... */);

    return result;
  }

Similar things can certainly be done with various templates techniques. 
However, when it comes to passing arbitrary C++ code to the generator, 
preprocessor macros have certain advantages. Now, the above example can 
probably be done without a more powerful WHILE, but many generators / mini 
compilers may do substantially more computation. Theoretically, the 
preprocessor could be used for making a parser generator that could have an 
advantage compared to Spirit due to syntactical issues. The main practical 
problems would be the difficult of implementation and compile-time 
performance (which is exactly the same as with template techniques, but 
often worse).

Another possible direction for program generators would be to take a middle 
road: use preprocessor to define the input syntax and generate code skeleton 
(switch case statement), then use template metaprogramming or even run-time 
code to compute the transition tables. This kind of an approach would make 
it possible to get a very nice syntax for specifying many kinds of program 
generators (and in many cases the overhead of transition table generation is 
not important).

>Thanks for a great library!

Thanks for using the library! And many thanks to Paul for all the hard work 
he has put into the library!

- Vesa Karvonen

_________________________________________________________________
The new MSN 8 is here: Try it free* for 2 months 
http://join.msn.com/?page=dept/dialup