[Boost-users] [serialization] Potential problems with void_caster

Subject: [Boost-users] [serialization] Potential problems with void_caster
From: Bogdan (bogdan.indy_at_[hidden])
Date: 2010-03-26 19:12:15

I am trying this again, because if it needs a fix it would be nice to make
it into the next boost release. I get an access violation after explicitly
unloading a DLL that contains serialization code for a series of classes.

I believe there might be a bug in void_caster::recursive_unregister(): that
function member is called in the destructors of the void caster template
classes. It searches the set inside void_caster_registry singleton for all
the entries that are descendants of the current one (i.e., the void_caster
being destroyed) and performs the following operations:
- delete the shortcuts. 
- remove the corresponding entries from the set.
However, void_caster::recursive_unregister() doesn't remove from the
registrar the entry corresponding to the current void_caster(). This creates
the problem below:

Consider a scenario where a dll containing some serialization functionality
is loaded explicitly at runtime (i.e., using LoadLibrary() on Windows or
dlopen on Unix). There are no parent-child relationships among the
serialized classes registered by that DLL: all serialized classes are
derived from one abstract base. While unloading the library, its void_caster
singletons are destroyed but there are no shortcuts to remove from the
global registrar since there are no parent-child relationships. All entries
corresponding to those are still in the global registrar and point to the
void_caster  singletons ALREADY DESTROYED AND DISCARDED.

When the main process exits, the registrar repeats the traversal for
whatever executable-bound void_caster singletons are about to be destroyed.
During these subsequent traversals, void_caster::recursive_unregister()
visits the entries left after the previous explicit dll download and tries
to identify the parent-child relationship. In doing so, it accesses the
dangling vc pointers generating access violations:

void_cast.cpp(Line 286):        const void_caster * vc = *it;
void_cast.cpp(Line 287):        if(vc->m_parent == this){
//<--------------------- here is the access violation

I can reproduce the crash systematically, but I am not 100% sure about the
reason or whether a simple unregistration is enough.

I believe this is (again) a question for Robert Ramey, but I thank for any
feedback I might get.

Bogdan

PS
On a different tack I would like to confirm I tested Robert fix (@60273) for
multiple registrations and it looks good in my runs. Thank you!