[Boost-users] Design for boost::serialization of STL iterators.

From: Carlo Wood (carlo_at_[hidden])
Date: 2006-03-12 13:41:09

On Sat, Mar 11, 2006 at 11:45:36PM +0200, Peter Dimov wrote:
> Carlo Wood wrote:
> 
> > 1) WHY does boost::serialization not support serialization
> >   of iterators?
> 
> It's very hard, if not impossible, in general.

It is either impossible or possible. When it is possible,
it can't be hard :p

I think that at the moment of serialization, the actual container
needs to be known. So, only concentrating on the syntax of the code
that serializes an iterator, the following cannot work:

  ar & iter;

However, the container will be known at that point (or at least,
we might as well assume that). Therefore, I could imagine a syntax
like:

  ar & foobar(container, iter);

where 'foobar' is some template function (that will be part of
boost::serialization in the future).

Lets start with the last one (have to start somewhere).
In order to deserialize the iterator (and taken into account that
we already have the container) -- which makes me think that it might
be better to pass a functor to operator& above. In other words,
(at least for writing), use:

template<class Container>
struct Foobar {
  Foobar(Container const&, typename Container::const_iterator& iter) : M_container(container), M_iter(iter);
private:
  Container const& M_container;
  Container::const_iterator& M_iter;
};

and then assign to M_iter, whatever we find in M_container.

> If you have access to the container to which the iterator i refers:
> 
> a) if the container is a map, you can serialize i->first, then on 
> deserialization look up the key in the map using .find();
> 
> b) if the container is a sequence, and it has the same contents on 
> deserialization as it does on serialization, you can serialize the index of 
> the element - distance( c.begin(), i ). 

I don't think that it is a requirement that the whole container
has to be read: either it's a map or set, in which case later
insertions will not invalidate the container, or it is a vector,
in which case we use an index, from which we can construct
the end iterators before the whole container is reconstructed.

Therefore, the following syntax can work imho:

  ar & FooBar(container, iter);

Would you agree?

There are a zillion questions and implementation details
open - but I'd like to get anyones feedback about this
type of interface for iterators before continuing.

Current questions: What is a good name for Foobar?
Am I right when I think that Foobar needs to be declared
in boost::serialization?

Now the hard part (for me) is that part that involves
the internals of the boost library.

At the moment one tries to serialize an iterator, it is
possible (though not necessarily) that the object that
the iterator points to is already serialized. In the case
that we are dealing with an associative container (map or
set etc), we don't want to store the contents of the
object (or key part, in the case of a map) again: I'd
like to just store that once, and store a reference in
the archive at the other place. How can I make the library
store an object only the first time and the store a
reference the second time? This would be for an object
of arbitrary type thus. I think that it is only feasible
to do this when it's actually the same object (same
memory address), which is the case; but while storing
the object element of the container, the address of it
isn't stored as well, is it? Is there a way to make:

std::set<SomeData> s;
[...]
  ar << s;

Store the address of s (&s) along with the contents of s?
And would that then automatically cause the library to
not store that same object again when I do:

  std::set<SomeData>::iterator iter = s.find(...);
[...]
  ar << *iter;

Because then, &*iter would again be the same memory
address as what was stored before.

-- 
Carlo Wood <carlo_at_[hidden]>