[Boost-users] Inconsistent unicode encoding between boost and wx on mac osx

Subject: [Boost-users] Inconsistent unicode encoding between boost and wx on mac osx
From: Sachin Garg (schngrg_at_[hidden])
Date: 2010-02-13 16:20:43

My project uses both boost and wxwidgets and unicode encoding by both
is different on Mac OSX. Everything works fine on windows.

Problem: Boost and WX do end up encoding the strings differently when
converting to unicode on OSX. I am detailing an example:

WX's encoding is same on both windows and osx but Boost's encoding is
different on both platforms. It is probably not a bug but I am unable
to figure out the reason and how to make them both work together.

The string:

$B9q:]E*$J0lN.$N%[!<%k%@!<(B

(I don't know if this foreign language string will show up fine in
your email client).

Hex dumps of Unicode encodings of this string

>From Boost on Mac (32-bit):

e5 0 0 0 9b 0 0 0 bd 0 0 0 e9 0 0 0 9a 0 0 0 9b 0 0 0 e7 0 0 0...

>From WX on Mac (32-bit):

fd 56 0 0 9b 96 0 0 84 76 0 0 6a 30 0 0 0 4e 0 0....

>From WX and Boost on Windows (16-bit):

fd 56 9b 96 84 76 6a 30 0 4e 41 6d 6e 30...

(To be more exact, this is how boost::filesystem encodes a directory
name it read from disk, and its the wxChar** argv value from wxApp for
a Unicode build of wx).

So wx does same thing on windows and mac, and boost does that same
thing on windows. But boost encodes it differently on mac and I have
no idea why. My wild guess is its due to locale etc settings but I
don't know how check for this or fix it.

SG