From: Hartmut Kaiser (hartmut.kaiser_at_[hidden])
Date: 2006-12-06 11:17:58
Ben Hanson and I were busy over the last couple of days to build a full C++
lexer based on Ben's Lexertl framework
(http://www.benhanson.net/lexertl.html) which is integrated into Wave (as
proof of concept). Now, as we have accomplished this I'ld like to take the
chance to provide some (rough) performance measurements. As a basis we took
the two existing (functionally equivalent) Wave C++ lexers (based on Re2C
and Slex) and measured the execution times to scan all possible C++ tokens
in a loop 1000 times. Here are the results:
Re2C: 1.22 [s]
Slex: 1.89 [s]
Lexertl: 1.47 [s] (1.35 [s] without minimize_dfa)
These measurements include the construction times of the lexer objects for
each of the scanned C++ tokens.
I'm personally impressed by the performance results of Lexertl. Keep in
mind, Re2C has no DFA generation overhead at runtime and the generated code
is very compiler friendly - essentially a huge switch statement, allowing
for highly optimzed code. So I didn't expect Lexertl to be as fast as Re2C.
But I didn't expect it to be so close to Re2C either.
The bottom line is: I highly encorage Ben to boostify his code since I'm
pretty sure Lexertl would make a very useful and state of the art addition
For those interested to have a peek at the code, it's in the Boost
CVS::HEAD, integrated into the Wave example 'list_includes'. The Lexertl
code itself is available from the website mentioned above.
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk