RE: [Boost-users] Can boost::tokenizer tokenize 2byte character s tring?

From: Rozental, Gennadiy (gennadiy.rozental_at_[hidden])
Date: 2004-09-20 10:59:10

If it 2byte why are you using boost::char_separator<char>? 

Shouldn't you be using boost::char_separator<wchar_t>?

Gennadiy.

> -----Original Message-----
> From: boost-users-bounces_at_[hidden] 
> [mailto:boost-users-bounces_at_[hidden]] On Behalf Of 
> Lee, Joo-Young
> Sent: Saturday, September 18, 2004 12:39 AM
> To: boost-users_at_[hidden]
> Subject: [Boost-users] Can boost::tokenizer tokenize 2byte 
> character string?
> 
> 
> Hi.
> 
> I try to use 'boost::tokenizer<boost::char_separator<char> >' 
> to separate 2byte character string like Korean, Japanese, or Chinese.
> 
> But, I found that it does not works correctly.
> 
> Is there a solution?
> 
> Thanks for the help,
> 
> Lee Joo-Young
> 
> 
> 
> _______________________________________________
> Boost-users mailing list
> Boost-users_at_[hidden] 
> http://lists.boost.org/mailman/listinfo.cgi/boost-users
>