tokenizer for wstring

From: kalons (kalons_at_[hidden])
Date: 2002-01-17 10:50:16

Does tokenizer support wide character strings?  I'm using VC++ 6.0.  
The string sample works correctly; I tried this modification:

typedef tokenizer<char_delimiters_separator<wchar_t>, 
std::wstring::const_iterator, std::wstring> wtokenizer;
wtokenizer tok(params.w_str());
for(wtokenizer::iterator beg=tok.begin(); beg!=tok.end();++beg)
{
    ...
}

But it causes an access violation (ultimately in an isspace() call, 
which isn't the right call for wide characters).  Thanks.

Kyle Alons
http://www.kinook.com