Re: [boost] GSoC Unicode library: second preview

Subject: Re: [boost] GSoC Unicode library: second preview
From: Scott McMurray (me22.ca+boost_at_[hidden])
Date: 2009-06-20 15:22:55

2009/6/20 Artyom <artyomtnk_at_[hidden]>
>
> > UTF-16 ... This is the recommended encoding for dealing with
> > Unicode internally for general purposes
>
> To be honest, it is most error prone encoding to work with Unicode:
>

Amen.

Really, I don't see why people don't just use UTF-8 all over the
place.  Even UTF-32 isn't as convenient as most would like, since you
still have combining code points and other similar complications.

As a programmer what I really care about is usually some nebulous
concept of "characters", and one character can easily be 3 codepoints
or 1/3 of a codepoint.

It feels like the only way to get Unicode string handling right (at
the application level, not library or render levels) is to deal
entirely in strings and regexes.

Suppose I have "difficult" with the "ffi" ligature codepoint, and I do
a perl-style split on /i/.  I should probably be getting "d", the "ff"
ligature codepoint, and "cult".  I know if I tried to code that by
hand in every application I'd miss all kinds of evil corner cases like
that.