Re: [Boost-users] [serialization] custom allocation/deletion

Subject: Re: [Boost-users] [serialization] custom allocation/deletion
From: Brian Budge (brian.budge_at_[hidden])
Date: 2012-06-07 12:22:24

On Thu, Jun 7, 2012 at 9:45 AM, Robert Ramey <ramey_at_[hidden]> wrote:
> Brian Budge wrote:
>> On Jun 6, 2012 9:43 PM, "Robert Ramey" <ramey_at_[hidden]> wrote:
>
>> The case I am specifically thinking about is a pool allocator where
>> when the shared_ptr destructor is called, it invokes the deleter on
>> the pointer, returning the object to th pool.  In the MPI case, a
>> unique pool exists on each node.
>
> right.  I don't see any problem with the custom deleter..
> If a custom deleter is specified for a shared pointer,  I would
> expect it works as it always does.

Well, shared_ptr is not templatized on the custom deleter.  It stores
it in a magical way under the hood.  The deleter is accessed through
this standalone function:

template<class D, class T>
D * get_deleter(shared_ptr<T> const & p);

I don't see how serialization of the custom deleter could work.
Perhaps it could work if serialization was performed and then
deserialization was performed in the same process, but this wouldn't
be the case between processes (saving to a file for a later time, or
with MPI).  It seems that in this case, the default deserialization of
shared_ptr cannot work with custom deleters.  On the other hand, it
seems that if "custom" allocation were used, then the default
deserialization into a shared_ptr already would not work, and the
container of the shared_ptr would have to manually serialize and
deserialize the contents of the actual pointer, and then build a
shared_ptr anyway, so perhaps its a larger issue than the custom
deleter.

> More of a problem is the allocator.  Allocation is currently
> a detail in the serialization library.  As such, I'm not sure
> how easy to override it would be.  On the other hand, since
> the STL collections have an allocator parameter, it might
> be that this is already addressed automatically by the
> serialization library.  This would require more research
> into how the library allocates new object.  I realize I wrote
> this code, but it was a number of years ago and I don't
> remember how it works.
>

Okay, fair enough :)  What I want might actually be even simpler than
what I said.  I essentially want to be able to pass contextual data
along the entire deserialization process.  With

struct Bar {
    Foo *foo;
    int    numFoo;
};

This could enable changing something like

template <class Archive>
void load(Archive &ar, Bar &b, const unsigned int version) {
    ar >> b.numFoo;
    b.foo = new Foo[b.numFoo];
    ar.load_binary(b.foo, b.numFoo * sizeof(Foo));
}

to

template <class Archive, typename Context>
void load(Archive &ar, Bar &b, const unsigned int version, Context &ctx) {
    ar >> b.numFoo;
    b.foo = ctx.get_buffer<Foo>(b.numFoo);
    ar.load_binary(b.foo, b.numFoo * sizeof(Foo));
}

In this case, the Context might be a buffer object pool or a set of
pools for different types or really anything contextual.  Of course
you could do this with global objects, but if we want to keep these in
logical allocation groups so all memory can be flushed once operation
on a context is done, that won't work.

Thanks again for your comments.
  Brian