Re: [Boost-users] [serialization] skipping data

From: Joel de Guzman (joel_at_[hidden])
Date: 2006-07-10 10:58:30

Peter Dimov wrote:
> Joel de Guzman wrote:
>> Peter Dimov wrote:
>>> Joel de Guzman wrote:
>>>> Hi,
>>>>
>>>> Here's a use case that I can't seem to find a way to
>>>> do using boost.serialization.
>>>>
>>>> Say with version 1, I want to serialize X Y and Z. These
>>>> are hierarchical and expensive data structures. Now, with
>>>> version 2, I do not need to create Y objects anymore. Yet,
>>>> I still have to open version 1 files for backward compatibility.
>>>> So, when loading version 1 files, I want to skip reading Y
>>>> objects. How do I do that without having to create a temporary
>>>> Y object that will be discarded later?
>>> You need to define a proxy Y object that can deserialize itself into
>>> thin air.
>> I thought about that. But that would mean that blank_proxy<Y>
>> knows all about Y -- its composition down to the leaves -- and
>> wrap them all as blank_proxy<T>s. A hairy proposition, IMO.
> 
> This is the only way that works reliably with any archive. :-) (And, since 
> the serialization library does not document its external format, the only 
> way that works reliably with it.)
> 
> The option of actually reading a temporary Y and destroying it afterwards is 
> also feasible. You don't necessarily have to optimize the new version so 
> that it reads v1 files faster than v1 itself. :-) 

There might be a possibility that Y is already obsolete at
version 2. Sure you can "emulate" it through the proxy thing,
but then again that's less than ideal. I simply want to skip
some data and I can't do it with Boost.Serialization. I can
do it with simple streams, for example, by prepending the data
with a length (in bytes). I can also do it with XML by ignoring
everything in between the current tag and its matching end tag.
Rene (Rivera) noted that some archive types also allow it. For
example, IFF (http://en.wikipedia.org/wiki/Interchange_File_Format)
allows it through chunking. "Because the spec includes explicit
lengths for each chunk, it is possible for a parser to skip over
chunks which it either can't or doesn't care to process."
IMO, this is a valid use case that Boost.Serialization can address,
at least, for archive forms that allow it. Perhaps a SkippableArchive
concept? IMO, the ability to skip is a prerequisite for transparent
versioning.

Regards,
-- 
Joel de Guzman
http://www.boost-consulting.com
http://spirit.sf.net