Re: [boost] [General] Always treat std::strings as UTF-8

Subject: Re: [boost] [General] Always treat std::strings as UTF-8
From: Chad Nelson (chad.thecomfychair_at_[hidden])
Date: 2011-01-16 09:58:00

On Sat, 15 Jan 2011 17:04:24 +0000
Alexander Lamaison <awl03_at_[hidden]> wrote:

> On Sat, 15 Jan 2011 10:08:22 -0500, Chad Nelson wrote:
>
>> On Fri, 14 Jan 2011 10:59:09 -0500
>> Dave Abrahams <dave_at_[hidden]> wrote:
>> 
>>> Also, is there any use in trying to get the difference into the type
>>> system, e.g. by using some kind of wrapper over std::string that
>>> gives it a distinct "utf-8" type?
>> 
>> The system I'm now using for my programs might interest you.
>> 
>> I have four classes: ascii_t, utf8_t, utf16_t, and utf32_t. 
>
> ... snip
>
>> There are some parts of the code that could use polishing, but I like
>> the overall design, and I'm finding it pretty easy to work with.
>> Anyone interested in seeing the code?
>
> Yes please! 

http://www.oakcircle.com/toolkit.html

I've released it under the Boost license, so anyone may use it as they
wish.

I think one part of os.cpp uses a class from the library that I didn't
include, but it's minor and can easily be replaced with one of the
Boost.Random classes instead. Everything else should work stand-alone.

It's pretty well documented, but ask me if you have any questions.

> This sounds roughly like the solution I'd been imagining where, for
> instance, boost::filesystem::path has string and wstring contructors
> that work as they do now but also has path(utf8_string) constructors
> that must be called like this:
>
> std::string system_encoded_text = some_non_utf8_aware_library_call();
> filesystem::path
>     utf8_file_path(boost::utf8_string(system_encoded_text));
>
> The utf8_string class would do the conversion from the system
> encoding.

That's how I designed it. :-)
-- 
Chad Nelson
Oak Circle Software, Inc.
*
*
*