[boost] Re: [boost::serialization] - serialization + shared_ptrs

From: Robert Ramey (ramey_at_[hidden])
Date: 2005-04-08 00:13:35

Jarl Lindrud wrote:
> Robert Ramey <ramey <at> rrsd.com> writes:
>
> Requiring extra registration macros for shared_ptr<T>, beyond the
> ones needed to serialize T*, to me indicates a deeper flaw with how
> Boost.Serialization handles shared_ptr's.
>

Hmmm - how do you know it doesn't indicate a flaw in TR1::shared_pointer
that doesn't specify an interface rich enough to support serialization as
all the other standard library components do?

>>> As was discussed before, a good way to
>>> achieve this is to use a pointer-to-shared_ptr map in the input
>>> archive when loading archives that contain shared_ptrs. This is
>>> actually the approach that we've hacked into the version of
>>> boost::serialization here at work. We've been using it for a couple
>>> of months
>>> now like this and have not experienced any problems with it.

Of course, you'll have to do this again if you want to use the next version
of the serialization library - but that's your choice.

If I had been faced with this problem and needed an immediate solution I
would have made a derivation of the included archives along the lines of the
following

class text_iarchive_with_special_shared_ptr_handling :
    public text_iarchive
{
    std::map<void *, ?>  shared_ptr_map;
};

Now you have a map when you need it and everything is about done.  Not need
to meddle in the serialization library implemenation.

> Why can't it happen?

a) The author of boost shared pointer doesn't want to include serialization
implementation in his implementation of boost::shared_ptr

b) I don't want to add code to the implementation of serialization to handle
one particular data type.  This strikes me as fragile, open ended and
contrary to the whole idea of separating the serialization of the
serialization library from the types it is meant to serialize.  After
implementing serialization for all the standard library types without major
problem, we now want to make a special case for one particular type?  Where
does that lead us?  What about the other types in TR1?  Has anyone
considered whether or not they expose enough of their state to permit
serialization independent of their implemenation?

The problem I'm concerned about is adding stuff into the serialization
library just to handle one particular type.  That creates a sort of open
ended maintainence issue.  I would hope someone might suggest some elegant
edition to the interface that would be guarenteed to resolve

>
> There was a thread on this a while back, presenting the same solution
> as above. Is there a particular reason why that approach wasn't
> taken? Was it ever considered?
>

This was in response to my request for access to the implementation of
smart_ptr.  My request wasn't really even considered.  But the real answer
is that although I understand the problem, I havn't figured out a general
solution and I haven't heard one from anyone either.

> I've implemented it myself, both in my own serialization framework
> (http: //tinyurl.com/3nhmc), and on top of boost's, and it's pretty
> simple. 100% external serialization, and it would work with just
> about any reference-counted smart pointer. No extra registration
> macros needed either.

Congratulations - maybe you want to upload to the  file section

> The way it's done now just seems very heavy handed to me,

no argument there

> and doing
> it that way for every smart pointer that might come along, eg in
> std::tr1, is not even feasible. Since its possible to serialize a
> shared_ptr, or any reasonable smart pointer for that matter, from its
> corresponding public interface

I really can't know that one solution is going to work for every
"reasonable" smart pointer. and even so it would require adding every new
type that came along into the serialization library.  This would defeat the
original purpose of factoring out this code into a "library"

> shared_ptr's are my only gripe :)

And since you've got your own copy fixed to taste, you must be a very happy
man.

For what its worth, my thoughts along these lines are something like:

calss helper_base {
    virtual ~helper_base() = 0;
}

class text_iarchive ...
    std::list<helpbase *> helper_list;
    text_iarchive(){
        for(it = help_list.begin();it != helper_list.end();++it)
            delete *it;
        }
};

template<class T>
shared_ptr_helper :
    public helper_base
{
    std::map<void *> ptr_map;
};

template<class Archive, class T>
void serialize(Archive &ar, boost::shared_ptr<T> &t){
    help_base * hbp = ar.find_helper(shared_ptr_helper<T>)
    if(NULL == hbp){
        hbp = new share_ptr_helper<T>;
        ar.add(hbp);
    }
    // we now have the address of a map which will be deleted
    // when archive is deleted.
    // use map as necessary
    hbp->ptr_map ...
    ...
}

This would handle situatations like shared_ptr.  I don't know if it would be
general enough to handle them all.  Its easy to design a class that is not
serializable even in principle.  I don't know about the other boost types
addressed in tr1.  Anyway, its only a suggestion.  I'm still open to
suggestions that address all the conflicting concerns raised so far.

It seems that relatively few boost types have had serialization added.  I
presume there is a lack of usage of the combination of serializaton and
these types.  We did get boost::varient implemented.  Ironically, this is
implemented through only its public interface - at the cost of an extra copy
even though its not part of any standard.  Personally, I would have
preferred to elminate the copy have have serialization an implementation
feature of boost::varient but I didn't need any more aggravation.

The latest updates to shared_pointer checked into CVS about 10 days ago
break current serialzation of shared pointer.  In no way will I have time to
fix this before 15 April.  So I would expect we'll have to deprecate shared
pointer serialization for 1.33 .  Current users of this combination will
have to wait until Peter Dimov's "proper" implementation in 1.34.

Robert Ramey