Re: [Boost-users] serialization1.36.0 extended_type_info exit issue(s)

Subject: Re: [Boost-users] serialization1.36.0 extended_type_info exit issue(s)
From: troy d. straszheim (troy_at_[hidden])
Date: 2008-10-08 17:04:23

Robert Ramey wrote:
> 
> Just to recap.
> 
(snip)

> a) you don't have the lastest copy.(possible)

I definitely have svn trunk.

> b) the code is multi-threading and has a race condition
> after main() returns.(seems very unlikely to me)

This is all singlethreaded, debug build.

> c) my implemenation is flawed(possible - but that's why you testers
> get the big bucks!)

Would that it were so.

I'll go back and look at all this again with one eye on your recap (thanks).
In case this knocks anything loose, here's the smallest test case I can
produce at the moment.

Classes Base and Derived1 live in a shared library, and a Main.o links to it:

---------------------  Objects.hpp --------------------------

#ifndef OBJECTS_HPP_INCLUDED
#define OBJECTS_HPP_INCLUDED

struct Base
{
   int b;
   virtual ~Base();
   Base(int);
   template <typename Archive> void serialize(Archive& ar, unsigned);
};

struct Derived1 : Base
{
   int d1;
   ~Derived1();
   Derived1(int, int);
   template <typename Archive> void serialize(Archive& ar, unsigned);
};

#endif

---------------------- Objects.cpp -----------------------------

#include "Objects.hpp"
#include <boost/serialization/export.hpp>
#include <boost/serialization/nvp.hpp>
#include <boost/serialization/base_object.hpp>
#include <boost/archive/xml_oarchive.hpp>

namespace bs = boost::serialization;

Base::Base(int i) : b(i) { }
Base::~Base() { }

template <typename Archive>
void
Base::serialize(Archive& ar, unsigned)
{
   ar & bs::make_nvp("b", b);
}
template void Base::serialize(boost::archive::xml_oarchive&, unsigned);

Derived1::Derived1(int i, int j) : d1(i) , Base(j) { }
Derived1::~Derived1() { }

template <typename Archive>
void
Derived1::serialize(Archive& ar, unsigned)
{
   ar & bs::make_nvp("base", bs::base_object<Base>(*this));
   ar & bs::make_nvp("d1", d1);
}
template void Derived1::serialize(boost::archive::xml_oarchive&, unsigned);

BOOST_CLASS_EXPORT(Derived1)

------------------------ (end) -------------------------

Here is the Main.cpp, which is compiled to a .o and then linked
against the .so:

------------------------- Main.cpp -------------------------
#include <boost/shared_ptr.hpp>
#include <boost/serialization/shared_ptr.hpp>
#include <boost/serialization/nvp.hpp>

#include <boost/archive/xml_oarchive.hpp>

#include "Objects.hpp"

int main(int argc, char** argv)
{
   boost::archive::xml_oarchive xoa(std::cout);
   Derived1 d1(9999,8888);
   xoa & boost::serialization::make_nvp("d1", d1);
}

-------------------------------------------------------------

Now, this doesn't crash unless you run it under valgrind
and have valgrind paint free'd memory with 0xAAAAAAA:

% make
rm -f *.o *.so
gcc -g -I. -I/home/troy/Projects/boost/trunk -c Objects.cpp
gcc -shared -o libshlib.so Objects.o -L/home/troy/Projects/boost/trunk/lib -lboost_serialization-d -lstdc++ -Wl,-rpath,/home/troy/Projects/boost/trunk/lib
gcc -g -I. -I/home/troy/Projects/boost/trunk -c Main.cpp
gcc -o buggy Main.o -L/home/troy/Projects/boost/trunk/lib -lboost_serialization-d -lstdc++ -Wl,-rpath,/home/troy/Projects/boost/trunk/lib -L. -lshlib -Wl,-rpath,`pwd`
valgrind --tool=memcheck --free-fill=AA --malloc-fill=BB ./buggy
==11538== Memcheck, a memory error detector.
==11538== Copyright (C) 2002-2007, and GNU GPL'd, by Julian Seward et al.
==11538== Using LibVEX rev 1804, a library for dynamic binary translation.
==11538== Copyright (C) 2004-2007, and GNU GPL'd, by OpenWorks LLP.
==11538== Using valgrind-3.3.0-Debian, a dynamic binary instrumentation framework.
==11538== Copyright (C) 2000-2007, and GNU GPL'd, by Julian Seward et al.
==11538== For more details, rerun with: -v
==11538==
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE boost_serialization>
<boost_serialization signature="serialization::archive" version="5">
<d1 class_id="0" tracking_level="1" version="0" object_id="_0">
         <base class_id="1" tracking_level="0" version="0">
                 <b>8888</b>
         </base>
         <d1>9999</d1>
</d1>
</boost_serialization>

==11538== Invalid read of size 4
==11538==    at 0x410E21B: std::_Rb_tree<boost::serialization::extended_type_info const*, boost::serialization::extended_type_info const*, std::_Identity<boost::serialization::extended_type_info const*>, boost::serialization::detail::key_compare, std::allocator<boost::serialization::extended_type_info const*> >::lower_bound(boost::serialization::extended_type_info const* const&) (stl_tree.h:1430)
==11538==    by 0x410E2AE: std::multiset<boost::serialization::extended_type_info const*, boost::serialization::detail::key_compare, std::allocator<boost::serialization::extended_type_info const*> >::lower_bound(boost::serialization::extended_type_info const* const&) (stl_multiset.h:436)
==11538==    by 0x410D3CC: boost::serialization::extended_type_info::key_unregister() (extended_type_info.cpp:90)
==11538==    by 0x410D582: boost::serialization::extended_type_info::~extended_type_info() (extended_type_info.cpp:132)
==11538==    by 0x410ECB6: boost::serialization::detail::extended_type_info_typeid_0::~extended_type_info_typeid_0() (extended_type_info_typeid.cpp:74)
==11538==    by 0x804DA4F: boost::serialization::extended_type_info_typeid<Derived1>::~extended_type_info_typeid() (extended_type_info_typeid.hpp:83)
==11538==    by 0x804CFED: __tcf_1 (singleton.hpp:105)
==11538==    by 0x42C5083: exit (in /lib/tls/i686/cmov/libc-2.7.so)
==11538==    by 0x42AD457: (below main) (in /lib/tls/i686/cmov/libc-2.7.so)
==11538==  Address 0x440d080 is 16 bytes inside a block of size 20 free'd
==11538==    at 0x40222EC: operator delete(void*) (vg_replace_malloc.c:342)
==11538==    by 0x410DE38: __gnu_cxx::new_allocator<std::_Rb_tree_node<boost::serialization::extended_type_info const*> >::deallocate(std::_Rb_tree_node<boost::serialization::extended_type_info const*>*, unsigned) (new_allocator.h:97)
==11538==    by 0x410DE6D: std::_Rb_tree<boost::serialization::extended_type_info const*, boost::serialization::extended_type_info const*, std::_Identity<boost::serialization::extended_type_info const*>, boost::serialization::detail::key_compare, std::allocator<boost::serialization::extended_type_info const*> >::_M_put_node(std::_Rb_tree_node<boost::serialization::extended_type_info const*>*) (stl_tree.h:371)
==11538==    by 0x410DECD: std::_Rb_tree<boost::serialization::extended_type_info const*, boost::serialization::extended_type_info const*, std::_Identity<boost::serialization::extended_type_info const*>, boost::serialization::detail::key_compare, std::allocator<boost::serialization::extended_type_info const*> >::_M_destroy_node(std::_Rb_tree_node<boost::serialization::extended_type_info const*>*) (stl_tree.h:401)
==11538==    by 0x410DF23: std::_Rb_tree<boost::serialization::extended_type_info const*, boost::serialization::extended_type_info const*, std::_Identity<boost::serialization::extended_type_info const*>, boost::serialization::detail::key_compare, std::allocator<boost::serialization::extended_type_info const*> >::_M_erase(std::_Rb_tree_node<boost::serialization::extended_type_info const*>*) (stl_tree.h:1325)
==11538==    by 0x410DF62: std::_Rb_tree<boost::serialization::extended_type_info const*, boost::serialization::extended_type_info const*, std::_Identity<boost::serialization::extended_type_info const*>, boost::serialization::detail::key_compare, std::allocator<boost::serialization::extended_type_info const*> >::~_Rb_tree() (stl_tree.h:592)
==11538==    by 0x410DFBE: std::multiset<boost::serialization::extended_type_info const*, boost::serialization::detail::key_compare, std::allocator<boost::serialization::extended_type_info const*> >::~multiset() (stl_multiset.h:91)
==11538==    by 0x410D349: __tcf_0 (singleton.hpp:105)
==11538==    by 0x42C53B0: __cxa_finalize (in /lib/tls/i686/cmov/libc-2.7.so)
==11538==    by 0x40F7CA2: (within /home/troy/Projects/boost/trunk/lib/libboost_serialization-d.so)
==11538==    by 0x413ED0B: (within /home/troy/Projects/boost/trunk/lib/libboost_serialization-d.so)
==11538==    by 0x400DFDE: (within /lib/ld-2.7.so)
==11538==
==11538== Invalid read of size 4
==11538==    at 0x40FCD08: boost::serialization::extended_type_info::get_key() const (extended_type_info.hpp:69)
==11538==    by 0x410DFE2: boost::serialization::detail::key_compare::operator()(boost::serialization::extended_type_info const*, boost::serialization::extended_type_info const*) const (extended_type_info.cpp:46)
==11538==    by 0x410E22F: std::_Rb_tree<boost::serialization::extended_type_info const*, boost::serialization::extended_type_info const*, std::_Identity<boost::serialization::extended_type_info const*>, boost::serialization::detail::key_compare, std::allocator<boost::serialization::extended_type_info const*> >::lower_bound(boost::serialization::extended_type_info const* const&) (stl_tree.h:1430)
==11538==    by 0x410E2AE: std::multiset<boost::serialization::extended_type_info const*, boost::serialization::detail::key_compare, std::allocator<boost::serialization::extended_type_info const*> >::lower_bound(boost::serialization::extended_type_info const* const&) (stl_multiset.h:436)
==11538==    by 0x410D3CC: boost::serialization::extended_type_info::key_unregister() (extended_type_info.cpp:90)
==11538==    by 0x410D582: boost::serialization::extended_type_info::~extended_type_info() (extended_type_info.cpp:132)
==11538==    by 0x410ECB6: boost::serialization::detail::extended_type_info_typeid_0::~extended_type_info_typeid_0() (extended_type_info_typeid.cpp:74)
==11538==    by 0x804DA4F: boost::serialization::extended_type_info_typeid<Derived1>::~extended_type_info_typeid() (extended_type_info_typeid.hpp:83)
==11538==    by 0x804CFED: __tcf_1 (singleton.hpp:105)
==11538==    by 0x42C5083: exit (in /lib/tls/i686/cmov/libc-2.7.so)
==11538==    by 0x42AD457: (below main) (in /lib/tls/i686/cmov/libc-2.7.so)
==11538==  Address 0xaaaaaab2 is not stack'd, malloc'd or (recently) free'd
==11538==
==11538== Process terminating with default action of signal 11 (SIGSEGV)
==11538==  Access not within mapped region at address 0xAAAAAAB2
==11538==    at 0x40FCD08: boost::serialization::extended_type_info::get_key() const (extended_type_info.hpp:69)
==11538==    by 0x410DFE2: boost::serialization::detail::key_compare::operator()(boost::serialization::extended_type_info const*, boost::serialization::extended_type_info const*) const (extended_type_info.cpp:46)
==11538==    by 0x410E22F: std::_Rb_tree<boost::serialization::extended_type_info const*, boost::serialization::extended_type_info const*, std::_Identity<boost::serialization::extended_type_info const*>, boost::serialization::detail::key_compare, std::allocator<boost::serialization::extended_type_info const*> >::lower_bound(boost::serialization::extended_type_info const* const&) (stl_tree.h:1430)
==11538==    by 0x410E2AE: std::multiset<boost::serialization::extended_type_info const*, boost::serialization::detail::key_compare, std::allocator<boost::serialization::extended_type_info const*> >::lower_bound(boost::serialization::extended_type_info const* const&) (stl_multiset.h:436)
==11538==    by 0x410D3CC: boost::serialization::extended_type_info::key_unregister() (extended_type_info.cpp:90)
==11538==    by 0x410D582: boost::serialization::extended_type_info::~extended_type_info() (extended_type_info.cpp:132)
==11538==    by 0x410ECB6: boost::serialization::detail::extended_type_info_typeid_0::~extended_type_info_typeid_0() (extended_type_info_typeid.cpp:74)
==11538==    by 0x804DA4F: boost::serialization::extended_type_info_typeid<Derived1>::~extended_type_info_typeid() (extended_type_info_typeid.hpp:83)
==11538==    by 0x804CFED: __tcf_1 (singleton.hpp:105)
==11538==    by 0x42C5083: exit (in /lib/tls/i686/cmov/libc-2.7.so)
==11538==    by 0x42AD457: (below main) (in /lib/tls/i686/cmov/libc-2.7.so)
==11538==
==11538== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 23 from 1)
==11538== malloc/free: in use at exit: 0 bytes in 0 blocks.
==11538== malloc/free: 25 allocs, 25 frees, 1,265 bytes allocated.
==11538== For counts of detected errors, rerun with: -v
==11538== All heap blocks were freed -- no leaks are possible.
make: *** [buggy] Segmentation fault

What is really odd is that if you change the Main to serialize
through shared-pointer-to-base, things look fine:

------------------ working Main.cpp --------------------------

#include <boost/shared_ptr.hpp>
#include <boost/serialization/shared_ptr.hpp>
#include <boost/serialization/nvp.hpp>
#include <boost/archive/xml_oarchive.hpp>

#include "Objects.hpp"

int main(int argc, char** argv)
{
   boost::archive::xml_oarchive xoa(std::cout);

   boost::shared_ptr<Base> bp(new Derived1(777,666));
   xoa & boost::serialization::make_nvp("ptr", bp);
}

--------------------- output of the run -----------------------

% make
rm -f *.o *.so
gcc -g -I. -I/home/troy/Projects/boost/trunk -c Objects.cpp
gcc -shared -o libshlib.so Objects.o -L/home/troy/Projects/boost/trunk/lib -lboost_serialization-d -lstdc++ -Wl,-rpath,/home/troy/Projects/boost/trunk/lib
gcc -g -I. -I/home/troy/Projects/boost/trunk -c Main.cpp
gcc -o buggy Main.o -L/home/troy/Projects/boost/trunk/lib -lboost_serialization-d -lstdc++ -Wl,-rpath,/home/troy/Projects/boost/trunk/lib -L. -lshlib -Wl,-rpath,`pwd`
valgrind --tool=memcheck --free-fill=AA --malloc-fill=BB ./buggy
==11556== Memcheck, a memory error detector.
==11556== Copyright (C) 2002-2007, and GNU GPL'd, by Julian Seward et al.
==11556== Using LibVEX rev 1804, a library for dynamic binary translation.
==11556== Copyright (C) 2004-2007, and GNU GPL'd, by OpenWorks LLP.
==11556== Using valgrind-3.3.0-Debian, a dynamic binary instrumentation framework.
==11556== Copyright (C) 2000-2007, and GNU GPL'd, by Julian Seward et al.
==11556== For more details, rerun with: -v
==11556==
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE boost_serialization>
<boost_serialization signature="serialization::archive" version="5">
<ptr class_id="0" tracking_level="0" version="1">
         <pxvoid boost::archive::detail::basic_oarchive::register_basic_serializer(const boost::archive::detail::basic_oserializer&)
  class_id="2" class_name="Derived1" tracking_level="1" version="0" object_id="_0">
                 <base class_id="1" tracking_level="1" version="0" object_id="_1">
                         <b>666</b>
                 </base>
                 <d1>777</d1>
         </px>
</ptr>
</boost_serialization>

==11556==
==11556== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 23 from 1)
==11556== malloc/free: in use at exit: 0 bytes in 0 blocks.
==11556== malloc/free: 32 allocs, 32 frees, 1,409 bytes allocated.
==11556== For counts of detected errors, rerun with: -v
==11556== All heap blocks were freed -- no leaks are possible.

----------------------------------------------------------------

That's all I've got for now...

-t