Re: [boost] Boost.Locale (was Re: [SQL-Connectivity] Is Boost interested in CppDB?)

Subject: Re: [boost] Boost.Locale (was Re: [SQL-Connectivity] Is Boost interested in CppDB?)
From: Matus Chochlik (chochlik_at_[hidden])
Date: 2010-12-15 12:57:54

On Wed, Dec 15, 2010 at 6:42 PM, Artyom <artyomtnk_at_[hidden]> wrote:
>> From: Matus Chochlik <chochlik_at_[hidden]>
>> On Tue, Dec 14, 2010 at 8:25 PM, Mathias Gaunard
>
> That is exactly the reason Boost.Locale does not provide iteration
> over code points...
>
> What kind of transform(c) you want to do?
>
> See... Usually code points are meaningless in context of
> natural text processing, you generally need higher units:
>
> Examples:
>
> 1. How many characters where "שָלוֹם" - there are 4 chracters and
> Â  6 code points (4 base letters+2 diacritics). Code point!= character and this
> is why you
>
> Â  do not need "indexing" over code points unless you develop
> Â  some Unicode algorithm.
>
> 2. You are rarely work (transform) stand alone code points.
> Â  You always use context, even stuff like converting case
> Â  may change the amount of code points in the string!
>
> If you want to split the text into characters, words etc, there is a break
> iterator
> that does this for you.
>
> Artyom

Ok, Thanks for the clarification.

Matus