Re: [boost] [Serialization] (Commented) assertion when using types in multiple DLLs on Windows (Ticket #3934 and #4394)

Subject: Re: [boost] [Serialization] (Commented) assertion when using types in multiple DLLs on Windows (Ticket #3934 and #4394)
From: Robert Ramey (ramey_at_[hidden])
Date: 2011-12-13 12:22:21

Martin B. wrote:
> [Serialization] (Commented) assertion when using types in multiple

> Robert writes in his comment:
>
>> The problem is that the way the code is structured,
>> you'll have multiple instances of some functions.
>> The best way would be eliminate the source of the
>> problem by structuring code like this:
>> ...
>>
>
> and he proposes to move the `template<class Archive> void serialize`
> function of the classes from the header file into the cpp file.
>
> However, *our* code already does this, and we get this assertion
> regardless. The reason I suspect is that both the main module code as
> well as the DLL module code are calling into the recursive_register
> function to register the type, and since the code for the class is
> (identically) duplicated in the DLL as well as in the executable,
> obviously it is registered twice.

You might want to investigate this a little more.  The "registration"
occurs before main(..) is called.  The static object which does this
is created as a side-effect of invoking serialization.  So if registrations
are occuring in multiple modules, I would say it's because invokations
are occuring in multiple modules.  It has been my believe that
structuring the code as above would address this issue.

As I said that is my belief - I feel that I could be proved wrong
and perhaps you've done just that.  or perhaps not.  So I would
be interested in seeing what you find when you get to the bottom
of it.  Here a couple of ideas you might try.

Using the debugger, trap the code at the registration point and
verify where each module is invoking the registration from.  Also
you can track the registration lookup to verify that the code
isn't being called from some unexpected place.

In a large program, it's my custom to create a local library
of all my modules and link this code into my executable.
This speeds up application compilation/link time and helps
keep me from coupling modules in unintended ways.  I suspect
that many do this for the same reason.  The problem is
that when one does this, you might be linking the code
from the library into both the DLL and the main module
without realizing it.  If you application is gigantic - which
it might well be, this would be hard to find.  So you might
consider putting the serialization modules in a separate library
so that this library ONLY linked when creating the DLL.

As I said, I believe this assertion can be trapped only if
the serialization code is included in multiple modules.

> ## Question: Am I correct?

lol - as far as anyone else is.  We're sort of in uncharted
territory.

> So, the first question obviously is whether I'm correct in my
> analysis.
> I.e., the `assert(result.second);`that is currently commented out can
> *never* work properly for types that live in two modules (DLLs) in the
> same process and Boost:serialization is used as DLL.

I"m not sure what "never work" means.  I would phrase this as:
the assert will trap whenever serialization for the same type is include
in more than one module.

> ## Question: Any harm done?
>
> Secondly, is there any adverse effect of a type being registered twice
> with the serialization DLL? What will happen on unload? Is it possible
> the type is unloaded too early because of this setup?

Care has been taken to be sure that this will work as one would expect.
That is, when a module is unloaded, the registry entry dropped is the
same one that was created when the module was loaded.  That is
the intention at least.  I don't know that it's every been explictly
verified or tested.

> Note: What about accidentally identical type keys for actually
> different types in different DLLs?

lol - I would expect that that would be disasterous.  the exported
type name string has to uniquely identify the type.  The whole
house of cards depends upon this.

A more interesting question is:  What if serialization code for the same
is instantiated in multiple modules?  I have taken care that that
should create no problem.  Again, I haven't explicitly tested this.
On the other hand, it doesn't seem that anyone has reportd a problem
since I commented out the assertion. for whatever that's worth.
My motivation for including the assertion was the possibility
that one might have different versions of the serialization
code in different DLLS.  That seemed to me to be a situation
which would/could errors which would be almost impossible
to debug.  Hence the assertion and my advice to structure
the code to avoid this kind of problem.  Unfortunately,
structuring the code in this way proved to be very burdensome
to many developers.  I would argue that not being able to
structure one's code in this way is an indication that the
design has some issue such as circular dependency or
undesirable coupling between modules.  But, I'm not here
to argue - just to get the work done so here we are.

> ## Question: Solutions?
>
> What is a proper solution? Should we use dllexport types, so that the
> type is only registered once per process? What else?

I would like to see you invest some more effort to see
exactly where the extra registration comes from.  I realize
that this seems like "extra work" when all you (or your
management) just want's a fix.  But I really think this effort
will be rewarded in tracking down an unanticipated
dependcy or something like that.  If that doesn't turn
out to be the case, then we (I) will have a counter example
to my belief that structuring can elminate this and other
potential problems.  You've made the effort to structure
the code this way.  You're 99% of the way to perfection.
Don't give up now!

Robert Ramey

>
>
> Robert, I hope you can chime in on this. I'll try to add a link to
> this mail to the trac issue #4394.
>
> cheers,
> Martin
>
> [1] : https://svn.boost.org/trac/boost/ticket/4394
> [2] : https://svn.boost.org/trac/boost/ticket/3934
>
>
> _______________________________________________
> Unsubscribe & other changes:
> http://lists.boost.org/mailman/listinfo.cgi/boost