Re: [Boost-bugs] [Boost C++ Libraries] #2217: serialization 1.36.0 extended_type_info exit issue(s)

Subject: Re: [Boost-bugs] [Boost C++ Libraries] #2217: serialization 1.36.0 extended_type_info exit issue(s)
From: Boost C++ Libraries (noreply_at_[hidden])
Date: 2008-09-12 19:51:35

#2217: serialization 1.36.0 extended_type_info exit issue(s)
-----------------------------------+----------------------------------------
  Reporter:  timothysc_at_[hidden]   |       Owner:  ramey             
      Type:  Bugs                  |      Status:  closed            
 Milestone:  Boost 1.37.0          |   Component:  serialization     
   Version:  Boost 1.36.0          |    Severity:  Problem           
Resolution:  fixed                 |    Keywords:  extended_type_info
-----------------------------------+----------------------------------------

Comment(by Ross Lippert <ross.lippert_at_[hidden]>):

 While I believe that the iterator invalidation issues were important for
 this problem, this issue should be reopened, because the story does not
 end there.

 I believe that the 1.36 implementation of extended_type_info_typeid
 suffers from a static destructor fiasco.  In particular, the tkmap
 singleton which is local to the extended_type_info_typeid.cpp file,
 functioning as a registry of type info, can be accessed by typeid
 destructors after the tkmap has been destructed.

 I have not been able to produce a small example, and I doubt that any
 example I could produce would behave likewise across gcc versions.
 However, I did instrument my code well enough to make a solid case.

 It began with
 {{{
 *** glibc detected *** free(): invalid pointer: 0x00002b3004540dc0 ***
 }}}
 at exit.

 Running the program in valgrind produced, as the first error,
 {{{
 ==20744== Invalid read of size 8
 ==20744==    at 0x5DC69BC:
 std::_Rb_tree<boost::serialization::detail::extended_
 type_info_typeid_0 const*,
 boost::serialization::detail::extended_type_info_type
 id_0 const*,
 std::_Identity<boost::serialization::detail::extended_type_info_typ
 eid_0 const*>, boost::serialization::detail::type_compare,
 std::allocator<boost:
 :serialization::detail::extended_type_info_typeid_0 const*>
 >::lower_bound(boost
 ::serialization::detail::extended_type_info_typeid_0 const* const&)
 (stl_tree.h:
 1371)
 ==20744==    by 0x5DC673A:
 boost::serialization::detail::extended_type_info_typeid_0::type_unregister()
 (stl_multiset.h:451)
 ==20744==    by 0x40DA49: __tcf_12 (extended_type_info_typeid.hpp:85)
 ==20744==    by 0x3759830CD4: exit (in /lib64/tls/libc-2.3.4.so)
 ==20744==    by 0x375981C4C1: (below main) (in /lib64/tls/libc-2.3.4.so)
 ==20744==  Address 0x6EE09B0 is 32 bytes inside a block of size 40 free'd
 ==20744==    at 0x4905D7A: operator delete(void*)
 (vg_replace_malloc.c:244)
 ==20744==    by 0x5DC6D37:
 std::_Rb_tree<boost::serialization::detail::extended_type_info_typeid_0
 const*, boost::serialization::detail::extended_type_info_typeid_0
 const*,std::_Identity<boost::serialization::detail::extended_type_info_typeid_0const*>,
 oost::serialization::detail::type_compare,
 std::allocator<boost::serialization::detail::extended_type_info_typeid_0
 const*>>::_M_erase(std::_Rb_tree_node<boost::serialization::detail::extended_type_info_typeid_0
 const*>*) (new_allocator.h:94)
 ==20744==    by 0x5DC63E8: __tcf_0 (stl_tree.h:578)
 ==20744==    by 0x3759830FBA: __cxa_finalize (in /lib64/tls/libc-2.3.4.so)
 ==20744==    by 0x5DC6242: (within
 /d/en/lippertr-0/p4/desmond/desmond2_lippertr/base/objs/Linux/x86_64/Release/lib/libdesmond_util.so)
 ==20744==    by 0x5E46AE0: (within
 /d/en/lippertr-0/p4/desmond/desmond2_lippertr/base/objs/Linux/x86_64/Release/lib/libdesmond_util.so)
 ==20744==    by 0x3759830CD4: exit (in /lib64/tls/libc-2.3.4.so)
 ==20744==    by 0x375981C4C1: (below main) (in /lib64/tls/libc-2.3.4.so)
 }}}
 As we see, the underlying rep of the tkmap is corrupt or invalid because
 lower_bound is failing (this version of 1.36 includes the patches already
 mentioned).

 With a little bit of effort, I recompiled boost serialization to have a
 destructor for the tkmap which prints to the screen and put print messages
 in the extended_type_info_typeids.
 {{{
 in typeid destructor
 unregistering m_key= 0x5f6ef20
 N71_GLOBAL__N_desmond_src_util_timekeeper_timekeeper.cxx_00000000_435F3CA51_E
 done unregistering.
 done destructor.
 tkmap is destroyed!
 in typeid destructor
 unregistering m_key= 0x517220 N3Ark3arkE
 ==20744== Invalid read of size 8
 . . .
 }}}

 My subsequent prints show that the tkmap is being destructed in the middle
 of a series of extended_type_info_typeid destructions.  This is the static
 destructor fiasco.  The initializer fiasco is overcome by the usual
 function-scoped static trick, but destructor calls are not so easily
 serialized.  When a destructor has to talk to an instance of another
 object, you have to be sure that object hasn't destructed yet.

 To further test, I altered the tkmap implementation to set a live bit.
 {{{
 (in extended_type_info_typeid.cpp

 struct tkmap : std::multiset<
     const extended_type_info_typeid_0 *,
     type_compare
 > {
   static int live;
   tkmap() { live=1; }
   ~tkmap() { live=0; std::cerr << "tkmap is destroyed!" << std::endl; }
 };
 int tkmap::live=0;
 }}}
 and added
 {{{
     if (!tkmap::live) return;
 }}}
 to the type_unregister method.
 {{{
 in typeid destructor
 unregistering m_key= 0x5f6ef40
 N71_GLOBAL__N_desmond_src_util_timekeeper_timekeeper.cxx_00000000_435F3CA51_E
 done unregistering.
 done destructor.
 tkmap is destroyed!
 in typeid destructor
 unregistering m_key= 0x517220 N3Ark3arkE
 done destructor.
 in typeid destructor
 unregistering m_key= 0x517ec0 N7Desmond6EngineE
 done destructor.
 ==21051==
 ==21051== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 9 from 4)

 }}}
 I believe that this makes for a substantial case that the addition of
 tkmap to boost 1.36 (not in 1.35) needs to be rethought.

-- 
Ticket URL: <http://svn.boost.org/trac/boost/ticket/2217#comment:7>
Boost C++ Libraries <http://www.boost.org/>
Boost provides free peer-reviewed portable C++ source libraries.