Re: [boost] Re: persistence

From: Scott Woods (scottw_at_[hidden])
Date: 2003-05-25 22:03:43

> * use an XML schema, but some schema information, perhaps in another
>    file, support C++ specific issues (e.g. does the data go into a list
>    or a vector?)

Like the XML idea. Have had to field that one already and couldnt come
up with any reason not to tackle it. Maybe the processing overhead at
load-time relative to processing of a binary representation that contains
no schema. Also type-safety? Type-checking is currently exhaustive.

Where the variant contains a something "with internal structure" (e.g. an
array, vector or record) it is currently using a std::vector. Give the user
(author of schema) control over underlying container? If this is what you
meant then its interesting. Toyed with allowing declaration of map inside
a schema but more general mechanism whereby user declares underlying
container? schema_traits<>?

>
> * support both XML and binary data formats
>
> * support both native-only (useful for IPC) and cross-platform formats

Not sure of benefit of supporting both? Better performance of native-only
mode?
Currently doing a network order type conversion on transfers so it is
"cross-
platform". Generally it might be fair to say that the processing associated
with
this conversion is low with respect to all the machine cycles consumed by
a transfer (remembering that there is a least 1 physical I/O operation in
there :-)
So, potential benefits seem small vs added complexity.

Conversion was originally included as generated data files may move between
platforms. Since then the "persistence mechanism" has actually morphed
to also cover IPC - it's actually being used to shunt arbitrarily complex
data around a few distributed components already. Kinda stumbled on this
one but of course the work being done during the load and save operations
is just another form of (automated!) marshalling.

>
> * support for "integrated header files" -- that is, rather than
>    generating a C++ header, it should be possible to use an existing one
>    (allowing you to customize it).

Considered. Possibly hiding from the nasty of parsing a C++ header ;-)

>
> * support for complex data structures, including recursive structures.
>    It should be possible to re-establish pointers, in other words.

Currently supports arbitraily complex structures;

    sensor_signal log[][ 2 ]

log could be described as a "vector of pairs of sensor_signals". Think I put
my "toe-in-the-water" as far as pointers goes - another "nasty" if I follow
your suggestion properly. Generally this is moving into an area that is
covered by OODBs and relational DBs. My simplification of what (I think)
you want would be some kind of "persistent object id"? Worked with
CORBA?

So far I have been pleased with the value provided by the current
schema technique which does handle arbitrarily complex, but non-
relational data.

> To me, that would be the holy grail.  It should probably be developed
> like the thread library, though -- start with core capabilities, then
> build on it over time.

Sounds good to me.

>
> I've love to take a gander at your code...
>

Don't have a real problem with this. Currently tangled up in a beta so
kinda busy. Code will probably be a better quality in a couple of
months :-)

Thanks for feedback,
Scott