[Boost-users] Can boost::tokenizer tokenize 2byte character string?

From: Lee, Joo-Young (trowind_at_[hidden])
Date: 2004-09-17 23:39:12

Hi.

I try to use 'boost::tokenizer<boost::char_separator<char> >' to separate
2byte character string like Korean, Japanese, or Chinese.

But, I found that it does not works correctly.

Is there a solution?

Thanks for the help,

Lee Joo-Young