Re: [boost] [General] Always treat std::strings as UTF-8

Subject: Re: [boost] [General] Always treat std::strings as UTF-8
From: Alexander Lamaison (awl03_at_[hidden])
Date: 2011-01-14 11:05:55

> On Fri, 14 Jan 2011 07:27:49 -0800 (PST), Artyom wrote:
> 
> >> 
> > -1
> > 
> > I'm  opposed to this strategy simply because it differs from the way
> > existing  libraries treat narrow strings.  Not least the STL.  If you open
> > an  fstream with a narrow filename, for instance, this isn't treated as a
> > UTF-8  string.  It's treated as being in the local codepage.
> > 
> 
> First of all, neither in C++/03 nor in C++0x you can
> open a file stream with wide file name. MSVC provides
> non-standard extension but it does not exist in other compilers
> like GCC/MinGW.
> 
> So using C++ you can't open a file called: "שלום-سلام-pease-Мир.txt"
> under Microsoft Windows.
> 
> You can use OS level API like _wfopen to do this job using wide
> string. But you can't to do this in C++. Period.

The situation is abysmal, i grant you that.

> The idea is following:
> 
> 1. Provide replacement for system libraries that actually
>    use text and relate to it as text in some encoding.
> 
>    For STL and standard C library it would be filesystem API.
> 
>    So you need to provide something like boost::filesystem::fstream

+1.  Done already, I believe :)

> 2. Make all boost libraries use Wide API only and never call ANSI API.

+1

> 3. Treat narrow strings as UTF-8 and convert then to wide prior system calls.

This is the part I have problems with: interpreting it as UTF-8 _by
default_.  Unless the programmer reads the docs really well, they would
most likely expect to be able to use a narrow string as returned by other
libraries and pass it straight to boost libraries without first having to
convert it.

Boost.Filesystem v3 allows you to specify that the incoming string it UTF-8
encoded but doesn't _default_ to that.  Is this insufficient?

Alex

-- 
Easy SFTP for Windows Explorer (http://www.swish-sftp.org)