RE: [boost] Wave C++ Review Begins Today - February 7, 2005

From: Hartmut Kaiser (HartmutKaiser_at_[hidden])
Date: 2005-02-09 08:42:29

Michiel Salters wrote:

> I'm currently looking at the library, trying to get the
> (included) command-line executable to work. So far, I've 
> found two documentation minor nits:
> * Source and zip say version 1.1.2, index.html says 1.2
> * index.html says program_options must be downloaded, but it's
>   in boost 1.32 ?

Noted, will be fixed.

> one serious bug in wave\cpplexer\re2clex\cpp_re2c_lexer.hpp:
> template <typename IteratorT, typename PositionT> inline 
> lexer<IteratorT, PositionT>::lexer(IteratorT const &first, 
>         IteratorT const &last, PositionT const &pos, 
>         boost::wave::language_support language) 
> :   filename(pos.get_file()), at_eof(false)
> {
>     memset(&scanner, '\0', sizeof(Scanner));
>     scanner.fd = -1;
>     scanner.eol_offsets = aq_create();
>     scanner.first = scanner.act = (uchar *)&(*first);
>     scanner.last = (uchar *)&(*last);
> ...
> [ IteratorT = std::string::iterator in the example ]
> 
> This assumes that the range [first,last] is dereferenacable 
> and contiguous in memory. This of course is not mandated by 
> the standard.

Yes, this is a known problem (see the Changelog file), I didn't realise,
that it will assert on some systems. Seems, that I'll have to fix that soon
;-). This shouldn't be a problem on other systems, though, if you're using
iterators acting on a continuous chunk of memory (such as
std::string::iterator). 

OTOH this isn't exactly a problem in the preprocessor library, it's merely a
problem of the re2c generated lexer used in the driver. The other provided
lexer (SLEX based) won't expose this problem (i.e. the cpp_tokens example
shouldn't assert).

> I found this, because VC8.0 has iterator debugging, and it 
> asserted. Not sure if this is the exact cause, I just found 
> the code in the stacktrace.
> I tried to hack the Scanner by replacing its uchar* with 
> iterator's but that really doesn't work. The generated code 
> uses memcpy's and malloc's.
> I'll try to see if I can avoid this, perhaps try another 
> string type. Perhaps the command-line example should use 
> BOOST_WAVE_STRINGTYPE iso std::string?

I'll look into this (will have to install VC8 first).

Thanks for reporting!
Regards Hartmut

> 
> Regards,
> Michiel Salters
> 
> This e-mail and any attachment is for authorised use by the 
> intended recipient(s) only. It may contain proprietary 
> material, confidential information and/or be subject to legal 
> privilege. It should not be copied, disclosed to, retained or 
> used by, any other party. If you are not an intended 
> recipient then please promptly delete this e-mail and any 
> attachment and all copies and inform the sender. Thank you.
> _______________________________________________
> Unsubscribe & other changes: 
> http://lists.boost.org/mailman/listinfo.cgi/boost
>