Re: [boost] Re: Re: Any interest in adding unicode support to boost?

From: Rogier van Dalen (rogiervd_at_[hidden])
Date: 2004-10-22 03:57:20

On Fri, 22 Oct 2004 02:43:19 +0200, Erik Wien <wien_at_[hidden]> wrote:
> 
> "Vladimir Prus" <ghost_at_[hidden]> wrote in message news:cl7kha$tk4
> > Just to clarify: the string and wstring in the standard have a huge
> > problem:
> > you can't convert string to wstring in any way: there's just no
> > appropriate
> > converting constructor.
> 
> Correct, and that should be provided if we were to make a unicode_string
> templated in encoding. (Not that we neccesarily will.)

I think that if you don't know the encoding (and normalisation form)
at compile time, iterators will have to be really slow. I hope you
will try and find out during your research how important that is. I
guess things like regular expressions would be slow, unless you
provide algorithms with a possibility to switch to versions tailored
to the encoding at run-time, similar to the visitor pattern
boost::variant provides.

> > That would still make it easy for a user to use some different encoding
> > without good reason.
> 
> It would be possible yes, not even difficult, but I don't think that means
> people will actually do that. People usually use std::string today, not
> basic_string<whatever>, because that is "the string class". I think a
> similar thing would happen with a unicode_string typedef. Especially if
> there is no difference in the interface between the different template
> versions. Then you'd have to know what the differences between the different
> encodings are (be an andvanced user, and aware of the drawbacks), to
> actually bother using anything other that UTF-16.

I fully agree.

Rogier