Re: [Boost-users] Open a filename with a wstring or wchar_t *

Subject: Re: [Boost-users] Open a filename with a wstring or wchar_t *
From: Timothy Madden (terminatorul_at_[hidden])
Date: 2010-08-03 07:49:44

Bo Jensen wrote:
> On Mon, Aug 2, 2010 at 8:32 PM, Timothy Madden <terminatorul_at_[hidden]> wrote:
>> Bo Jensen wrote:
>>> On Mon, Aug 2, 2010 at 2:03 PM, Timothy Madden <terminatorul_at_[hidden]>
>>> wrote:
>>>> Bo Jensen wrote:
>>>>> I use :
>>>>>
>>>>>  typedef boost::filesystem::wpath SlmWPath;
>>>>>  typedef boost::filesystem::wfstream SlmWfstream;
>>>>>  typedef boost::filesystem::wofstream SlmWOfstream;
>>>>>  typedef boost::filesystem::wifstream SlmWIfstream;
>>>> Yes, I can create wide streams, what I want is to pass a wide string as
>>>> the
>>>> file name to be opned.
>>> This should work :
>>>
>>> boost::filesystem::wfstream test;
>>>
>>> test.open(L"somepath");
>>>
>>> The above typedefs was just to show the tools you need from booost
>>> filesystem.
>> Oh, yes, you are right !
>>
>> It must be the 'Additions to <fstream>' thing, that construct an ifstream
>> from a filesystem::path/wpath. Actually I do not even have a wchar_t * in my
>> program, I use a wpath ! :)
> 
> I don't know all the details, but on windows I think filenames is only
> utf-16. On linux you should be safe, what ever locale you use. I would
> be interested to hear how it worked out.

All Windows API functions have an ANSI version, including file system 
functions, despite NTFS having Unicode filenames. I do not know what 
happens when an ANSI function has to return some Korean/Japanize file 
name from the file system, on a computer with some latin locale, anyone 
cares to try ?

Anyway I find that I have to explicitly

#include <boost/filesystem/fstream.hpp>

and then I get the new ifstream and ofstream classes in 
boost::filesystem namespace, that work just like the std::ifstream and 
std::ofstream, except that they can be constructed from a wpath also, 
and as such and they were able to create a Korean file name on the disk, 
were my current locale is Latin-2.

I would like to point that this std::ifstream / std::ofstream issue is 
not an operating system problem, but a C++ library issue. Indeed Linux 
may allow UTF-8 as the current locale, and that makes the char * 
constructors enough to open any file name on any filesystem.

However I find the UTF-8 locale to be only a system-specific 
work-around, as I may simply not want such a locale for my application 
(think about some limited devices/systems), and C++ implementations are 
not required to implement the UTF-8 locale, so I think the iostreams C++ 
standard library simply lacks wchar_t * constructors for file streams.

Probably this stems from the misleading idea that the wchar_t * I would 
use to open a file can always be converted to a char *. The truth is 
that such a conversion limits the filename in the resulting char * 
string to characters from the narrow-character set only, so it leaves 
the problem of wchar_t filenames for ifstream/ofstream open.

I have seen this misleading conversion used also as the reason for 
main() function  only having a char *argv[] argument (and no wchar_t 
*argv[]), and for std::exception::what() returning only a 
narrow-character string, and no wchar_t * message. Again, this limits 
the actual content in the strings to characters from the 
narrow-character set only.

Timothy Madden