RE: [boost] Interest in a fast lexical analyser compatible

From: Hartmut Kaiser (hartmutkaiser_at_[hidden])
Date: 2004-12-27 16:16:17

Dave Handley wrote:

> I have to confess to not knowing much about Wave, but I would 
> be willing to look in more detail at using this library for 
> Wave.  Once we have our first stable version of the software 
> we will be happy to let you have a look at it in more detail. 
>  I expect this to happen within the next month or so.

I'd be willing to write the interfacing stub to plug your library into Wave.

> By static and dynamic, I am meaning compile-time and 
> run-time.  The system is designed to generate the DFA at 
> run-time.  But we are discussing a method at the moment 
> whereby the same code base could be used to generate 
> code-stubs for compile-time DFA creation.  I am keen to support both.

The two different lexers I was using in the Wave library were a re2c based
lexer (static switch based lexer, extremly fast and compact) and a SLEX
based lexer (runtime generated DFA). I haven't done serious speed
measurements, but the numbers I've got so far showed similar timings for
both with a slight advantage for re2c (as expected). I'd expect your library
to be very similar in speed as well. But just out of curiousity I'm very
interested in seeing the static DFA generation version :-)

> >We definitely should try the new upcoming Spirit-2 code base 
> as well, 
> >since it should be a lot faster then the current version. Is it 
> >possible to have a look at your test code as well? This way we could 
> >try to make a comparision as soon as the Spirit-2 codebase evolves.
> 
> I would be quite keen to see how Spirit-2 performs on similar 
> tests.  If the interest is there, then we will quite happily 
> post the code into the boost yahoo group once we have 
> completed a bit of tidying up in the New Year.  

I'm definitely interested.

> We need to put some effort into writing more detailed test cases.  
> At present, we are only directly comparing lexical analysis, and 
> have not looked at the performance of the interface in real 
> detail.  We have a desire to properly test the system with a 
> complete parse.  To do this I think there are a number of 
> useful test cases:
> 
> 1)    Flex and bison/lex and yacc.
> 2)    Spirit - without any assistance from any lexer.
> 3)    Spirit 2 once available.
> 4)    Our library (called lextl at present) with yacc/bison.
> 5)    Lextl with Spirit.
> 6)    Lextl with Spirit 2.

Sounds sensible.

Regards Hartmut