Re: [boost] Using lexertl instead of spirit

From: Hartmut Kaiser (hartmut.kaiser_at_[hidden])
Date: 2006-11-21 21:03:59

> David Abrahams wrote:
> 
> > > Yes, and Slex is the other one
> > 
> > Not to mention XPressive?
> 
> Xpressive is not really usable as a lexer, and Eric is aware 
> of that. I have a Wave lexer implemented with Xpressive here 
> on my hard disk, and it functions well, it is only 3 
> magnitudes slower as for instance the re2c based one. The 
> main reasons are:
> 
> - no optimization between different regex's used for token 
> representation (no internal NFA/DFA generation)
> - no way to tell which alternative matched if using regex's 
> containing alternatives
> 
> The first rules out using separate regex's, one for each 
> token, the second one inhibits us from using one giant regex 
> with alternatives...
> 
> Both are probably merely natural restrictions stemmed from 
> the fact Xpressive is a regex library not a lexer generator. 
> The same issues would probably occur if we were trying to use 
> Boost.Regex for this task.

Found my old timings of the different lexer types: