Boost logo

Boost :

From: Gary Powell (Gary.Powell_at_[hidden])
Date: 2001-06-05 10:32:41

I was looking over my IOStreams book on the use of narrow and widen, and it
occurred to me that tokenizer should also be using the locale stuff.

Now take all of this with a large grain of salt, as by no means am I an
expert in the local/facet area of C++.

But as I read this stuff I was wondering if you could use the is(punct,c) as
the default instead of specifying the characters yourself. As in testing
ctype<Char>::is(ctype_base::punct, c); ? (same goes for whitespace.) (I
think its ctype_base::space )

And I was wondering if any locale expert out there could tell me whether
tokenizer should be congnistent of locales. (defaulting of course to some
reasonable one.) Or does this just bring in way too much code that won't be

Anyway it seems like there is great huge bit of internationalizing code that
most of us just ignore. And that a useful boost library like tokenizer could
take advantage of it.

(Also to do a character comparison, do we have to call widen?, as in

  template<class Char>


    if (*next == ctype<Char>::widen('n') ) ?


where I assume that the contents of *next is of type Char.



Boost list run by bdawes at, gregod at, cpdaniel at, john at