Re: [boost] [nowide] Request for interest (nowide unicode support for windows)

Subject: Re: [boost] [nowide] Request for interest (nowide unicode support for windows)
From: Beman Dawes (bdawes_at_[hidden])
Date: 2010-06-16 17:23:19

On Tue, Jun 15, 2010 at 2:36 PM, Artyom <artyomtnk_at_[hidden]> wrote:
>
>> >
>> > Can I write:
>> >
>> >Â Â Â boost::filesystem::fstream
>> f("שלום.txt",std::ios_base::out);
>> >
>> > When "שלום.txt" is UTF-8 string and Unicode file
>> name will be created?
>> > If so, way to go.
>> >
>>
>> In v3, yes.
>>
>
> Are you sure about this? How the file will be open?

Depends on the standard library implementation. The Dinkumware
library, used by Microsoft and some others, has an additional
constructor/open that takes a wide character string.  The fallback is
to use the standard narrow character constructor/open.

> Can you explain what is the path the UTF-8 string passes till
> the Win32API system call or standard library call?
>
> C++ standard defines open only with "char const *"

The wide character overloads are Dinkumware / Microsoft extensions.

> Quoting latest C++ standard draft (section 27.7 std::basic_streambuf)
>
> Â  Â // 27.9.1.4 Members:
> Â  Â bool is_open() const;
> Â  Â basic_filebuf<charT,traits>* open(const char* s,
> Â  Â  Â  Â ios_base::openmode mode);
> Â  Â basic_filebuf<charT,traits>* open(const string& s,
> Â  Â  Â  Â ios_base::openmode mode);
> Â  Â basic_filebuf<charT,traits>* close();
>
> And this it is defined on GCC's libstdc++.

Yep, os if that library is in use, the fallback is to use the narrow
character open. And of course that also what is used on POSIX-like
systems.

>
> As I can see you use std::basic_filebuf for implementing
> boost::filesystem::basic_fstream.
>
> So how do you open "Wide" path or "UTF-8" path using these
> functions?
>
> - Standard library does not accept "wchar_t const *" as parameter to
> Â open (with exception of MSVC specific extension)
> - Windows API does not support UTF-8 codepage.
>
> So how do you deal with it?

Use the Microsoft UTF-codepage, 65001

HTH,

--Beman