Re:[boost] [serialization] serializable objects implemented in

From: Robert Ramey (ramey_at_[hidden])
Date: 2004-03-12 13:02:43

David Tonge wrote:

> Anyway, I've resolved that by making a more complicated macro for 
> exporting the guids, which doesn't use any typedefs.

Although the shared_ptr serialization does work in the demos and tests, I've
conceded that its probably going to need another pass to address such
situations such as mult-threading, exception safety, polymorphic pointers
and now perhaps instanciation issues.  I think this will require access to
the class header which I don't have (or want).   I think for now it's OK to
handle situations on an ad-hoc basis as you have done.  In the long run I
hope someone gets around to making the shared_ptr serialization complete and
bullet-proof.

> I think that part of the problem was that my solution for exporting the 
> guids for shared_ptrs was like this...

>typedef boost::detail::sp_counted_base_impl<T*, ....> T_shared_ptr;
>BOOST_CLASS_EXPORT(T_shared_ptr);

In general, the macros don't really handle template arguments. Maybe they
should. (Is there a pre-processor macro IS_TEMPLATE(x) ? ) 

> Now to my other problem.  I wanted to investigate the serialization
> tools in the context of a larger project.  My habit is to define my
> class interfaces in header files, like base_class.h, and provide their
> implementations in source files like base_class.cpp.  I like my headers
> to be lightweight and include the minimum number of other header files
> in order that compilation times don't get out of hand.

> I had been including the BOOST_CLASS_EXPORT(T) commands in the headers
> but that means I have to include all of the archive headers, and
> export.hpp in every header, and that makes for slow compilations.  My
> hope is that I can include all of those only in the implementation .cpp
> and put the BOOST_CLASS_EXPORT(T) there.  I was also worried that if I
> included the BOOST_CLASS_EXPORT(T) in the header then every other .cpp
> which included that header would also try to register the guid for T -
> surely an unnecessary replication of effort.

Having done all of this I found that my program would work fine if
everything existed in a single EXE.  However, if I implement them as a
DLL containing my base_class and derived_class implementations and an
EXE, which uses these classes, then things fail.  

The export/registration mechanism is used to relate a key/class_id stored in
the archive with the appropriate de-serialization function.  With the case
of polymorphic pointers, the issue of instantiation of de-serialization code
also has to be addressed.  As things get more ambitious, the situation has
to be explicitly considerd.

I think the key example here is demo_pimpl.  I couldn't get this to link
properly with VC 6.0 and VC 7.0.  With gcc it always worked.  With VC 7.1 it
started to work for Microsoft compilers. To make it work I had to explicitly
instantiate the templates used for exported classes.  I believe that you're
situation will also require a little extra help - I'm not sure yet, but I'll
look into it.

I've stepped through the code for exe_using_dll and I can see that the
failure occurs when it tries to get extended type information for the
derived_class (at line 297 of oserializer.hpp).  However, I know that
the class has been registered by the guid_initializer (at line 130 of
export.hpp) because I put a breakpoint there, and saw initialization
happen for base_class, derived_class and the shared_ptr versions of
both.

> My suspicion is that the problem is caused by the serialization library
> using a static library, rather than a DLL.  If the BOOST_CLASS_EXPORT
> commands updated a list contained in the static library then there would
> be one instance of this list statically linked into my DLL, and another
> in my EXE.  Thus, the registration occurring in the DLL wouldn't affect
> the list used in the EXE.  

As Vladimir pointed out (I think) , a DLL version of the library is easily
generated by a simple enhancement to the jamfile in the build directory.  I
haven't done this nor have I tested it.  I see no reason why it shouldn't
work without problem. (hmm - I don't know if this automatically generates an
export lib).  Adding auto_lib functionality to the serialization library
would cleanly handle all issues related to build and usage of DLL versions.
I've not done so as I think it's premature given that the library has not
been accepted into boost. 
  
I suspect your suspicion is more or less on the right track. The library
uses a little bit of trickery to be sure that extended_type_info information
is generated and stored in a global table for each exported class at
pre-compile time.  When called from a DLL, most likely it's a separate table
in the DLL so it wouldn't be found when doing the lookup from the main exe.
I suspect this is addressable though I haven't yet considered it.

> If this is the case then it's going to be quite inconvenient.  It would
> mean one would have to BOOST_CLASS_EXPORT every class in every DLL where
> an instance of it (or a shared_pointer which might contain it) might be
> saved.  If that were the case then I'd advocate making
> boost_serialization into a DLL, rather than a static library.

> It would mean one would have to BOOST_CLASS_EXPORT every class in every
> DLL where an instance of it (or a shared_pointer which might contain it) 
> might be saved.

I think a better solution would be to build you implementation classes as
static libraries.  (see the demo_pimpl).  This addresses the issue of long
compile times nicely.  I believe will get you what you want.  You won't have
a separate DLL which may or may not be an issue depending upon your taste
and situation.  

Also, I believe that the DLL situation will probably work now if ALL the
code that invokes serialization for a particular class is found in the same
DLL.  An interesting experiment would be to see what would happen if the
demo_pimpl is reformlated to use a (1) static library and (2) a DLL.

In the longer run we'll look at this more carefully.  Its conceivable that
just the extended_type_instantion might be in a static lib while the rest is
in a DLL or maybe the export process can be tweaked to be sure that DLLS,
use the same table.  At this point I really can't say.

Another option (also mentioned by Vladimir and also in the documentation) is
the creation of a "virtual_archive" interface.  This is similar to the way
the package that was reviewed over a year ago handled the issue.  I have
looked into this a little and have concluded that it will be quite easy to
implement - though I've deferred it for now.
  
> If that were the case then I'd advocate making
> boost_serialization into a DLL, rather than a static library.

I don't think that would help

> I'd be grateful if you could take a look at my example, and let me know
> if there is something I can do to resolve my problems.

I'll look at it.

Robert Ramey