Re: [boost] [utf8_codecvt_facet]

Subject: Re: [boost] [utf8_codecvt_facet]
From: Robert Ramey (ramey_at_[hidden])
Date: 2013-01-06 01:53:31

Marshall Clow wrote:
> On Dec 11, 2012, at 9:09 AM, Robert Ramey <ramey_at_[hidden]> wrote:
>
>> I've had trouble setting up GDB for this test.  This would be my
>> normal approach.
>
> Ok - here's what I've found:
>
> Here's the crash: (mac os X, gcc library 4.2.1):
>> Breakpoint 1, 0x00007fff844e3558 in malloc_error_break ()
>> (gdb) bt
>> #0  0x00007fff844e3558 in malloc_error_break ()
>> #1  0x00007fff844ec809 in szone_error ()
>> #2  0x00007fff844ed374 in small_free_list_remove_ptr ()
>> #3  0x00007fff844eb013 in szone_free_definite_size ()
>> #4  0x00007fff844e4898 in free ()
>> #5  0x00007fff821b597f in std::basic_filebuf<wchar_t,
>> std::char_traits<wchar_t> >::_M_destroy_internal_buffer () #6
>> 0x00007fff821b54eb in std::basic_filebuf<wchar_t,
>> std::char_traits<wchar_t> >::close () #7  0x00007fff821b5440 in
>> std::basic_filebuf<wchar_t, std::char_traits<wchar_t>
>> >::~basic_filebuf () #8  0x00007fff821b753a in
>> std::basic_ifstream<wchar_t, std::char_traits<wchar_t>
>> >::~basic_ifstream () #9  0x00007fff821b74cf in
>> std::basic_ifstream<wchar_t, std::char_traits<wchar_t>
>> >::~basic_ifstream () #10 0x0000000100003552 in test_array<A> () at
>> test_array.cpp:109 #11 0x0000000100001b11 in test_main () at
>> test_array.cpp:117 #12 0x00000001000019e7 in main (argc=1,
>> argv=0x7fff5fbff720) at test_tools.hpp:203
>>
>
> You can see that it's destructing a std::basic_ifstream.
>
> If I strip a bunch of stuff out of test_array.cpp, the crash doesn't
> happen.
>
>>     {
>>         T a_array1[9];
>>         test_istream is(testfile, TEST_STREAM_FLAGS);
>>         BOOST_TRY {
>> #if 0
>>             test_iarchive ia(is, TEST_ARCHIVE_FLAGS);
>>             bool exception_invoked = false;
>>             BOOST_TRY {
>>                 ia >> boost::serialization::make_nvp("a_array",
>>             a_array1); }
>>             BOOST_CATCH (boost::archive::archive_exception ae){
>>                 BOOST_CHECK(
>>
>>
>>
>>
>>             boost::archive::archive_exception::array_size_too_short
>>             == ae.code ); exception_invoked = true; } BOOST_CATCH_END
>>             BOOST_CHECK(exception_invoked);
>> #endif
>>         }
>>         BOOST_CATCH (boost::archive::archive_exception ae){}
>>         BOOST_CATCH_END
>>     }
>>
>
> If I include the line:
>>             test_iarchive ia(is, TEST_ARCHIVE_FLAGS);
>
> then it does.
> This suggests to me that something in the test_iarchive is poking at
> the test_istream - but I don't know what.
>
>
> I tried to extract this into a simple test case (attached), but I
> can't figure out what to link against for the wide case.
>
> clang++ junk.cpp -DNARROW -I /Sources/boost/trunk -L
> $BUILD/libs/serialization/$TAIL -lboost_serialization # works
> clang++ junk.cpp -I /Sources/boost/trunk -L
> $BUILD/libs/serialization/$TAIL -lboost_wserialization # does not
>
> For the narrow case, I link against lib boost_serialization, and
> everything is good.
> For the wide case, when I link against boost_wserialization, I get
> the following errors:
>
>> Undefined symbols for architecture x86_64:
> Can you tell me how to solve this?
>

for the wide case, one would have to link with BOTH boost_serialization
as well as boost_wserialization.

As I stated before, I don't think it's a good idea to try to fix this at 
this time.

a) The approach above "might" fix the issue with that one test.  But
there are 50 other tests with the same archive which pass.  I'm suspected
that it's an issue with this particular test which interrupts the the 
process.
That is its some sort of exception safety issue.

b) Actually, I don't think it's that. I think it's a conflict between the
gcc runtime library and our codecvt facet which reimplements the
same functionaly with "almost" the same interface.

c) In other words, our failed test is not the bug - but a symptom of
a more subtle bug.  If I just make the test past, I'll be effectively hiding
the bug rather than fixing it.

I think it needs a real fix:

a) tweak the headers so that it used the built-in codecvt facets
where available.

b) Enhance the codecvt test (or add a couple more) to verify:

i) that the built in codevt facet actually works as expected.
ii) that our utf8_codecvt facet interoperates with the built in one.
That is that a file created with one can be read by the other and
vice-versa.

The change made in the trunk addresses an issue which has been
in the code for years and only produces a warning.  I would prefer
that we take our time and address it in a more definitive manner.
To try to fix it on short notice at this point isn't worth the risk
of imposing a setback on the release.

Robert Ramey

> -- Marshall
>
> Marshall Clow     Idio Software   <mailto:mclow.lists_at_[hidden]>
>
> A.D. 1517: Martin Luther nails his 95 Theses to the church door and
> is promptly moderated down to (-1, Flamebait).
>         -- Yu Suzuki
>
>
>
>
>
>>
>> _______________________________________________
>> Unsubscribe & other changes:
>> http://lists.boost.org/mailman/listinfo.cgi/boost