Re: [boost] [nowide] Request for interest (nowide unicode support for windows)

Subject: Re: [boost] [nowide] Request for interest (nowide unicode support for windows)
From: Alexander Lamaison (awl03_at_[hidden])
Date: 2010-06-16 10:02:29

> On Tue, 15 Jun 2010 12:38:06 -0700 (PDT), Artyom wrote:
> 
> > > > Can I write:
> > > >
> > > >  boost::filesystem::fstream
> > > >  f("ùìåí.txt",std::ios_base::out);
> > > >
> > > > When "ùìåí.txt" is UTF-8 string and Unicode file name will
> > > > be created?  If so, way to go.
> > >
> > > In v3, yes.
> > > 
> > > There are some caveats, but it should work, and there
> > > are some fairly
> > > similar test cases passing all compilers.
> > 
> > This is not what I'd understood from our previous
> > discussion.  I was under
> > the impression filesystem v3 running on Windows would take
> > this narrow path
> > string and convert it to UTF-16 using the *local code
> > page*.  This means
> > the example above would only work if the computer in
> > question were set to
> > Hebrew.  
> >
> > [snip]
> >
> > Did I misunderstand?
> 
> Yes, you did.
> 
> When I was talking about UTF-8 I mean Unicode and not subset.

Me too.

I'm saying that Filesystem v3 on Windows doesn't interpret narrow strings
as UTF-8 by default.  Berman said that it did but I beg to differ.  Here's
what the comments say:

//  For Windows, wchar_t strings do not undergo conversion. char strings
//  are converted using the "ANSI" or "OEM" code pages, as determined by
//  the AreFileApisANSI() function, or, if a conversion argument is given,
//  using a conversion object modeled on std::wstring_convert.

In other words "ùìåí.txt" would be interpreted as being in whatever
encoding the local code page is set to and would, therefore, produce a path
containing gibberish for most people.  This is standard Windows behaviour
:P

Your problem is yet another step further than this.  Assuming fs3 correctly
converted "ùìåí.txt" to the UTF-16 equivalent, how do you then open a file
using this wide-char name?  Well, MSVC has wchar_t overloads so this works
fine.  You're right about glibc++/MinGW though.  fs::fstream will fail
there.  Rather than introducing a nowide library, why don't we just try to
fix this in Boost.Filesystem?

Alex

-- 
Easy SFTP for Windows Explorer (http://www.swish-sftp.org)