Re: [boost] [GSoC] NLP Idea for GSoC 2011 project

Subject: Re: [boost] [GSoC] NLP Idea for GSoC 2011 project
From: Mathias Gaunard (mathias.gaunard_at_[hidden])
Date: 2011-03-27 07:30:42

On 26/03/2011 18:54, Sarma Tangirala wrote:

> Also, I was looking at some C++ code using Boost/tokenizer.hpp that
> tokenized some text and it looked a bit scary.
>
> Any suggestions or advice?

Look into iterators, ranges, and the various string manipulation and 
parsing libraries within Boost (Iterator, Range, StringAlgo, Regex, 
Spirit, Xpressive).
You could also want to look into my Unicode library. I could add word 
boundaries for non-thai languages if your project needs that.