Re: [Boost-users] [FileSystem] Internationalization woes on windows

From: Beman Dawes (bdawes_at_[hidden])
Date: 2005-03-15 20:53:59

At 04:52 AM 3/14/2005, Mark Holloway wrote:
 >I know this has been discussed a fair bit but I'd just like to get
 >confirmation of what I believe to be the case re: boost::filesystem and
 >internationalization on windows.
 >
 >I've been using boost::filesystem::directory_iterator to (amongst other
 >things) recurse across a directory hierarchy, and on each item asking
 >boost::filesystem::is_directory.
 >
 >When a file or directory is encountered containing characters not
 >representable using the ANSI char set (alright the MS ANSI charset) an
 >exception is thrown. For example:
 >
 >"boost::filesystem::is_directory: "Scratch561\ABCDE<internationized
 >chars>MNOP": The filename, directory name, or volume label syntax is
 >incorrect."
 >
 >This is presumably because the windows ANSI API calls are being used
 >(the <API function>A variants).
 >
 >Am I missing something, or is this a pretty severe limitation for
 >windows?
 >(I appreciate unix's don't suffer in this based on their api typically
 >using UTF-8 encoding).
 >
 >Thanks in advance for setting me straight/confirming my fears.

I'd like to run a test here to be sure; could you please supply an example 
of <internationalized chars> that is causing the problem?

I'm not at all surprised you are having problems; it isn't at all clear to 
me that it is possible to do reliable processing on Windows using the A 
(narrow) variants if Unicode characters are present.

I've got the internationalized revision of the Filesystem library running 
on Windows; it seems to be handling wide characters with ease. I'm testing 
now on POSIX - I couldn't resist the $499 price tag and bought a Mac Mini. 
Slick.

One fix for your problem is to switch to the internationalized version and 
use wchar_t based paths. That will use the W Windows variants, and will 
also work well on POSIX systems with internationalized file or directory 
names.

Another possibility is that we can work on the narrow functions to make 
them function better in the face of internationalized names. But I'd need a 
lot of help on that to develop test cases and strategies to handle them.

Thanks for the report,

--Beman