Subject: Re: [boost] [General] Always treat std::strings as UTF-8
From: Alexander Lamaison (awl03_at_[hidden])
Date: 2011-01-14 11:05:55
> On Fri, 14 Jan 2011 07:27:49 -0800 (PST), Artyom wrote:
> > -1
> > I'm opposed to this strategy simply because it differs from the way
> > existing libraries treat narrow strings. Not least the STL. If you open
> > an fstream with a narrow filename, for instance, this isn't treated as a
> > UTF-8 string. It's treated as being in the local codepage.
> First of all, neither in C++/03 nor in C++0x you can
> open a file stream with wide file name. MSVC provides
> non-standard extension but it does not exist in other compilers
> like GCC/MinGW.
> So using C++ you can't open a file called: "×©×××-Ø³ÙØ§Ù -pease-ÐÐ¸Ñ.txt"
> under Microsoft Windows.
> You can use OS level API like _wfopen to do this job using wide
> string. But you can't to do this in C++. Period.
The situation is abysmal, i grant you that.
> The idea is following:
> 1. Provide replacement for system libraries that actually
> use text and relate to it as text in some encoding.
> For STL and standard C library it would be filesystem API.
> So you need to provide something like boost::filesystem::fstream
+1. Done already, I believe :)
> 2. Make all boost libraries use Wide API only and never call ANSI API.
> 3. Treat narrow strings as UTF-8 and convert then to wide prior system calls.
This is the part I have problems with: interpreting it as UTF-8 _by
default_. Unless the programmer reads the docs really well, they would
most likely expect to be able to use a narrow string as returned by other
libraries and pass it straight to boost libraries without first having to
Boost.Filesystem v3 allows you to specify that the incoming string it UTF-8
encoded but doesn't _default_ to that. Is this insufficient?
-- Easy SFTP for Windows Explorer (http://www.swish-sftp.org)
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk