[Boost-users] [serialization] hacking serialization : handles to pointers

From: Sean Cavanaugh (worksonmymachine_at_[hidden])
Date: 2007-07-08 05:20:19

Ok I've spent a good chunk of my day making a set of custom archive class,
based on the portable_binary_iarchive and portable_binary_oarchive.

I have a non-virtual class hierarchy named Asset, in memory this is a cyclic
graph, but my requirements are that this structure cannot be serialized all
at once, it has to be broken up at boundaries.   So the archiver serializes
the first Asset* (which is always the root object passed into operator&).
This object serializes as normally as per the boost serialization code,
except except that when subsequent Asset*'s are encountered I send the
pointer into an AssetManager class and translate the Asset* into a
ResourceID, and serialize the ResourceID instead of the object.  This is
done by directly calling operator& inside save_override.

So far all of this works:

The reverse case of loading, by hooking load_override almost works.  Almost,
because I have to apply a hack of skipping 4 bytes inside load_override,
before trying to read the ResourceID structure with operator&.   I can't
figure out how to make it 'proper', without simply hard coding the 4 byte
read.  As far as I can tell before save_override some bookkeeping
information is written, but the reverse operation in load_override has not
yet stripped this bookkeeping information from the stream, so these
functions aren't quite mirror operations.   As best as I can tell the
function guilty of reading the bookkeeping information is load_pointer.
What I need is another function in the boost archive API that pulls the
bookkeeping information out of the stream safely, that I can call from
within load_override, since I have no intention of calling load_pointer
through any means.

Aside from the hack, it technically works, but even then I still have a few
aching problems:

I also have to hard-code the full list of derived Asset types and manually
provide specializations for all of them in save_override and load_override.
If I use the base class, I end up slicing my class down to its base, and
cannot serialize it.  I can't make serialize virtual, since the intrusive
serialize methods are templates, but it certainly would solve the problem if
it were possible.  In addition the bodies of all of my overrides are
completely identical except for the classname (AssetModel, AssetTexture,
etc).  Which means I'll be wrapping the bodies of a generic save_override
and load_override in a macro, and have to manually add all Asset derived
classes to a list of classes inside my archive class.  Which means that my
archiver cannot be generic, even though I have managed to make it a template
in the sense that the passed in asset manager and base asset types are
template parameters.

the type ResourceID  is a std::string

saving...
class asset_oarchive : ...
    void save_override(const AssetModel& t, int i)
    {
        if (!RootAssetSaved)
        {
            RootAssetSaved = true;
            binary_oarchive_impl::save_override(t, i);
        }
        else
        {
            AssetManagerType::ResourceID ResourceId =
TheAssetManager->ResourceIdForAsset(&t);
            operator&(ResourceId);
        }
    }

loading ...
class asset_iarchive : ...
    void load_override(AssetModel*& t, int i)
    {
        if (!RootAssetLoaded)
        {
            RootAssetLoaded = true;
            binary_iarchive_impl::load_override(t, i);
        }
        else
        {
            unsigned char c[10];
            for (INT x=0; x<10; ++x)
            {
                c[x] = 0xFF;
            }
            for (INT x=0; x<4; ++x)
            {
                operator&(c[x]);   //hackhack
            }
            AssetManagerType::ResourceID ResourceId;
            operator&(ResourceId);
            AssetType* a = TheAssetManager->FindAsset(ResourceId);
            if (!a)
            {
                a = TheAssetManager->LoadAsset(ResourceId);
            }
            // TODO: throw on missing resources, potentially query asset
manager for substitutes
            t = static_cast<AssetModel*>(a);
        }
    }

So I have a working implementation, how do I make it better?