Re: [Boost-users] Inconsistent unicode encoding between boost andwxonmac osx

Subject: Re: [Boost-users] Inconsistent unicode encoding between boost andwxonmac osx
From: Peter Dimov (pdimov_at_[hidden])
Date: 2010-02-13 18:41:19

Sachin Garg wrote:

> Peter, I am guessing that the problem is not the OSX locale which
> could be UTF-8 or UTF-16 or X (I am calling this 'source' encoding).

The encoding of OS X's path names is always UTF-8. It does not depend on the 
OS locale.

> Problem *seems* to be that boost converts that X into UTF16 (or UTF32)
> for storing it in wpath and that is using a normalization which is
> incompatible with normalization used by wx for its UTF16 (or 32).

I don't think that this has anything to do with normalization. By looking at 
the source, Filesystem, by default, uses the global locale to convert betwen 
narrow and wide paths, and the default locale in your case seems to perform 
no conversion. That is, it just takes every 8 bit char and converts it to a 
32 bit wchar_t. This, of course, doesn't work when the source is UTF-8, 
regardless of its normalization.