Re: [Boost-users] string_algo and utf8 locales

From: Pavol Droba (droba_at_[hidden])
Date: 2006-08-03 07:03:27

I'm not sure if you find a std-verion, that will work with utf-8 here.
I have checked the standard and case conversion that is present in 
locales is definitely not able to perform case conversion on 
variable-size encodings. Even the batch version is defined in terms
of per-character conversion.

Unless I'm missing something, standard is quite restrictive here.

Regards,
Pavol

Maik Beckmann wrote:
> Thank you for the hint. I will google for it and try get the
> std-versions to work on utf8 or ucs4 (maybe I have to use ICU oder can
> use glib(mm) ).
> 
> Regards, Maik
> 
>    
> 
> 
>> Hi,
>>
>> To be honest, I haven't tried to use string_algo with encodings
>> that use variable number of bytes per character.
>> All algorithms work in character-wise manner, so utf enconding
>> might not work.
>>
>> Internaly std::tolower(Ch, Loc) is used to do the conversion.
>> Try to see if this function works for you. If it doesn't give
>> the required results, algorithms in the string_algo will
>> not work either.
>>
>> Regards,
>> Pavol.
>>
>> Maik Beckmann wrote:
>>> Hello list, this is my first post to you :)
>>>
>>> I've got a problem with handling locales and string_algo. 
>>> As an example I want so make this string
>>>         "hää?"
>>> uppercase (->"HÄÄ?") by using this piece of code:
>>>
>>> <code>
>>>         #include <iostream>
>>>         #include <locale>
>>>         #include <string>
>>>         #include <boost/algorithm/string.hpp>
>>>         
>>>         
>>>         using namespace std;
>>>         using namespace boost;
>>>         
>>>         int main(int argc, char** argv) {
>>>         
>>>             string loc_name = "de_DE.utf8";
>>>         
>>>             string str("hää?");
>>>             str = to_upper_copy(str,locale(loc_name.c_str()));
>>>         
>>>             // ...should give "HÄÄ?"
>>>             cout << str << endl;
>>>             // but prints "Hää?"
>>>         
>>>             return 0;
>>>         }
>>> </code>
>>>
>>> As you see I'm trying to use locales, but it doesn't work. What is my
>>> mistake??
>>>
>>> Thanks in advance, Maik
>>>
>>> PS:
>>> - I'm using gentoo-linux with gcc-4.1.1
>>> - My locales list:
>>>         LANG=de_DE.utf8
>>>         LC_CTYPE="de_DE.utf8"
>>>         LC_NUMERIC="de_DE.utf8"
>>>         LC_TIME="de_DE.utf8"
>>>         LC_COLLATE="de_DE.utf8"
>>>         LC_MONETARY="de_DE.utf8"
>>>         LC_MESSAGES="de_DE.utf8"
>>>         LC_PAPER="de_DE.utf8"
>>>         LC_NAME="de_DE.utf8"
>>>         LC_ADDRESS="de_DE.utf8"
>>>         LC_TELEPHONE="de_DE.utf8"
>>>         LC_MEASUREMENT="de_DE.utf8"
>>>         LC_IDENTIFICATION="de_DE.utf8"
>>>         LC_ALL=de_DE.utf8
>>>         
>>>
>>> _______________________________________________
>>> Boost-users mailing list
>>> Boost-users_at_[hidden]
>>> http://lists.boost.org/mailman/listinfo.cgi/boost-users
>> _______________________________________________
>> Boost-users mailing list
>> Boost-users_at_[hidden]
>> http://lists.boost.org/mailman/listinfo.cgi/boost-users
> 
> _______________________________________________
> Boost-users mailing list
> Boost-users_at_[hidden]
> http://lists.boost.org/mailman/listinfo.cgi/boost-users