Re: [boost] [serialization] bug in wide character strings

From: Jeffrey Faust (jeff_at_[hidden])
Date: 2008-04-17 14:01:03

This test case only populates the wstring with lower case letters 
'a'-'z'.  This does not seem to be a sufficient test for serializing 
wide strings.

Jeff

Robert Ramey wrote:
> compare you application with test_simple_class.  This application saves and 
> restores
> every data type suppported by the compiler/library/platform.  This should
> include wstring.
>
> Robert Ramey
>
> Jeffrey Faust wrote:
>   
>> We've discovered an issue Boost has writing and reading wide character
>> strings (wchar_t* and std::wstrings) to non-wide character file
>> streams (std::ifstream and std::ofstream).  The issue stems from the
>> fact that
>> wide characters are written and read as a sequence of characters (in
>> text_oarchive_impl.ipp and text_iarchive_impl.ipp, respectively).  For
>> text streams, an EOF character terminates the reading of a file on
>> Windows. Some wide characters have EOF (value = 26 decimal) as one of
>> the bytes so reading that byte causes early termination of the read.
>> We
>> have worked around the issue by deriving our own input and output
>> archives from text_i|oarchive_impl<Archive> and overriding
>> load_override() and save_override for std::wstring and wchar_t*.  Our
>> implementation just sequences through the wide characters and writes
>> them 1 by 1 as wchar_t to the archive.  This isn't very elegant and is
>> even less readable in the file than the current implementation but
>> does resolve the problem.
>>
>> I looked at both Boost 1.34.1 and 1.35 and didn't see a difference in
>> the implementation here so I'm assuming 1.35 still has the issue.
>> I've
>> been working with 1.34.1.  Is this a known issue?  Does 1.35 solve it
>> in
>> some other subtle way?  Is there a better way that doesn't require us
>> to derive our own streams?  If not, is there a more elegant way of
>> implementing the reading and writing of wide characters ourselves?
>>
>> Thanks in advance.
>>
>> Jeff Faust
>>
>>