Re: [boost] [string] proposal

Subject: Re: [boost] [string] proposal
From: Patrick Horgan (phorgan1_at_[hidden])
Date: 2011-01-21 20:47:14

On 01/21/2011 09:50 AM, Beman Dawes wrote:
> ... elision by patrick ....
>
> IMO, Any serious Unicode string proposal has to address UTF-8 strings,
> UTF-16 strings, UTF-32 strings, and probably UTF strings where the
> particular UTF encoding is established at runtime. Applications that
> deal with Asian languages, do a lot of random access, or would pay a
> performance or storage penalty will demand more than just UTF-8
> strings. There might be other variants, too, such as a BMP-string. If
> a Unicode string library provides a strong design framework that is
> clearly articulated, then an initial implementation would only have to
> provide the most needed types; UTF-8 and UTF-16/BMP.
>
> I really doubt any proposal will get taken very seriously is it only
> supports one of the UTF encodings.

+1 with the caveat that UTF-8 and UTF-32 is considered by many to be the 
most needed types with UTF-16 considered evil.  (Seems to be a 
Windows/non-Windows split.  I like them all;)  So all three (four if you 
want to differentiate between fixed-width UTF-16/BMP (really UCS-2) and 
the full UTF-16) would be needed to avoid people saying that it doesn't 
fill their needs so why did we bother.  The UTF string with run-time 
would carry a lot of extra code.  Wouldn't a programmer know which he 
wanted to use internally at compile time?

Patrick

p.s. Nice quick description of the differences between and history of 
UCS-2 UCS-4 utf-8 utf-16 utf-32 at 
http://en.wikipedia.org/wiki/Universal_Character_Set