[Boost-users] [serialization] bad cast for shared library compiled by g++ with visibility=hidden

Subject: [Boost-users] [serialization] bad cast for shared library compiled by g++ with visibility=hidden
From: Pavol Juhas (pavol.juhas_at_[hidden])
Date: 2011-08-15 01:33:56

Hello,

I am working on a C++ shared library (DLL) that should have
a hierarchy of serializable classes using fixed archive format.
The serialize template methods of the classes are all instantiated
in the DLL, the idea is the library clients should be able to
load/save derived objects using shared_ptr<Base>, without an
explicit knowledge of the Derived classes.

Everything seems to work fine if I build the shared library with g++
using the default visibility option (i.e., every function in the DLL
is exported).  However, if I compile the code with "-fvisibility=hidden"
the test program crashes with bad_cast exception, whenever it tries to
serialize a derived object stored as a shared_ptr<Base>.

Strangely, if I build the library with NDEBUG defined, the code works
fine.  The difference seems to be in serialization/smart_cast.hpp
which uses different cast code with the NDEBUG macro defined.

Any idea why is this happening?  Is serialization supported for
the shared libraries build with g++ -fvisibility=hidden option?
I am using g++ 4.4.3 and the Boost libraries 1.40.

Attached is a minimum example where the testlib.hpp and testlib.cpp
are the shared library and testprg.cpp is the test program.  The
code can be built with the attached Makefile as follows:

(a) make clean runtest
    - default g++ visibility, test program works
    
(b) make clean runtest VISIBILITY=1
    - builds the library with -fvisibility=hidden,
      test program fails with the bad_cast exception

(c) make clean runtest VISIBILITY=1 NDEBUG=1
    - builds the library with -fvisibility=hidden and NDEBUG defined.
      test program works.
      
Thank you for any pointers,

Pavol