Re: [boost] Review Request: Boost.Locale

Subject: Re: [boost] Review Request: Boost.Locale
From: Artyom (artyomtnk_at_[hidden])
Date: 2010-05-24 04:06:21

> > So Boost.Locale reimplements standard codecvt facet to
> make this work
> > on any platform.
> 
> I didn't seen anything in the documentation about that.

http://cppcms.sourceforge.net/boost_locale/html/tutorial.html#b6d767d2f8ed5d21a44b0e5886680cb9


> >
> > But this is actually C++ standard's limitation.
> 
> The reason I ask is that I often see things requested on
> the list which I think could be better implemented as codecvt
> facets.
> Also, it seemed to me that a large part of the iostreams
> library
> could have been implemented more efficiently wiht codecvt
> facets.  Admitidly, it's somewhat unobvious how to
> make the
> best use of this - (needs another library and documentation
> of course)

Under the hood Boost.Locale uses uconv_* ICU API to do this.
So you can convert any stateless character set.

But this is actually implementation-details. From user point of view
he needs just to imbue correct locale and miracle happens.

But due to limitations of std::codecvt facet this implementation
is far from being very efficient mostly because every implementation
required to be stateless, which is quite unfortunate.

Several additional points:

1. Codecvt facets are not actually as useful as they seems to be.
   For example: you can't convert utf-8 <-> ISO-8859-8 using codecvt
   without passing wide characters.

2. std::codecvt is was really bad designed and it does not provide
   any information about std::mbstate_t and how it works which makes it
   very hard to implement anything efficiently. And does not allow 
   to store any information.

   So only standard library designed can provide something but not user
   that extends it.

3. For example all Boost code that implements UTF-8 codecvt facet
   supports only UCS-2 under windows and not UTF-16.


I don't put too much lights on it as I don't think that codecvt facets
should be widely as they very problematic.

Best,
  Artyom