Re: [Boost-users] Usingboost::serializationinreal-timewithoutallocating memory

Subject: Re: [Boost-users] Usingboost::serializationinreal-timewithoutallocating memory
From: Robert Ramey (ramey_at_[hidden])
Date: 2009-09-22 12:52:24

Peter Soetens wrote:
> On Tue, Sep 22, 2009 at 00:11, Peter Soetens
> <peter.soetens_at_[hidden]> wrote:
>> On Mon, Sep 21, 2009 at 23:44, Peter Soetens
>> <peter.soetens_at_[hidden]> wrote:
>>> On Mon, Sep 21, 2009 at 21:25, Robert Ramey <ramey_at_[hidden]> wrote:
>>>> Stefan Strasser wrote:
>>>>> Am Monday 21 September 2009 20:27:59 schrieb Robert Ramey:
>>>>>> Stefan Strasser wrote:
>>>>>>> I think to support this an other use cases like it
>>>>>>> common_archive would have to be moved outside of the linked
>>>>>>> library into a header and be made more generic.
>>>>>>
>>>>>> common_archive is a template - it's not in the linked library.
>>>>>
>>>>> ok, I must have mixed up some types. the point was that the
>>>>> ultimate base class, that handles the type registration and
>>>>> object tracking etc., is
>>>>> linked .cpp and not very configurable.
>>>>> the optimum case imho would be that you could pass a traits class
>>>>> to that archive base class, that gets called whenever there is an
>>>>> object or type to be registered and is queried for existing
>>>>> objects and types later on.
>>>>> that way you could disable registration, save type registration
>>>>> outside of the archive itself, let type registration span multiple
>>>>> archives etc, make sure type registration doesn't allocate etc.
>>>>
>>>> as I alluded to before - what I would like to know is why I derived
>>>> trivial_archive from common_archive. I don't remember what
>>>> made me do this. It seems that if this isn't done then
>>>> save/load_object is necessary. So the question is - why is this?
>>>> It is an oversight somewhere? or does the archive concept have to
>>>> be changed.
>>>>
>>>> Personally, I generally don't like the "traits class" as an
>>>> argument. It sort of implies that that the template has a lot of
>>>> "if -" code which navigates the traits. A "policy class" is better
>>>> but can
>>>> still make things complex. Personally I prefer composition
>>>> of funtionality through inheritance. This can be seen in
>>>> the class diagram for the serialization library. And your example
>>>> makes me more convinced than ever that this is basically
>>>> right. Now the only loose end it investigate why save/load
>>>> object is required.
>>>
>>> I found a lead to that. When the archive calls archive::save, a
>>> chain of 'invoke' calls is done to select the right serialization
>>> method for type T. When I pass std::vector<double> as T, it
>>> encounters a branch in oserializer.hpp:245, in struct
>>> 'save_conditional::invoke' .
>>>
>>> <code>
>>> // adds class information to the archive. This includes
>>> // serialization level and class version
>>> struct save_conditional {
>>> static void invoke(Archive &ar, const T &t){
>>> //if(0 == (ar.get_flags() & no_tracking))
>>> save_standard::invoke(ar, t);
>>> //else
>>> // save_only::invoke(ar, t);
>>> }
>>> };
>>> </code>
>>>
>>> The alternative branch is commented out, but let that be the path
>>> that needs to be taken by my code. Current code calls save_standard,
>>> which calls directly ' ar.save_object'. save_object in turn depends
>>> on basic_oarchive to do its job. If save_only was taken, the
>>> 'non-basic_oarchive' serialization code path would have been taken.
>>>
>>> For some reason (I've spent an hour staring at it), my unit test did
>>> not discover this flaw in my code and does as if serialisation of
>>> the std::vector was fine. I'm overlooking something stupid
>>> clearly...
>>>
>>> Anyway, as it seems now, the code path taken by archive::save/load
>>> mandates derivation of basic_?archive. Maybe I should have called
>>> another 'selection' function ? like archive::save_only directly ?
>>
>> I tried using this instead of archive::save() :
>> <code>
>> boost::archive::detail::load_non_pointer_type<binary_data_iarchive,T>::load_only::invoke(*this,t);
>> </code>
>>
>> But the compiler (gcc 4.3.3-5ubuntu4) refuses to compile this when
>> the
>> nvt types are in use. Basically, this is not fine:
>> <code>
>> int count;
>> ar >> BOOST_SERIALIZATION_NVP(count);
>> </code>
>> While this is:
>> <code>
>> int count;
>> nvp<int> n = BOOST_SERIALIZATION_NVP(count);
>> ar >> n
>> </code>
>>
>> You can try this out in plain user code. Beats me. The errors I get
>> for the first case are in the line of:
>>
>> usr/include/boost/serialization/access.hpp: In static member function
>> ‘static void boost::serialization::access::serialize(Archive&, T&,
>> unsigned int) [with Archive = RTT::marsh::binary_data_iarchive, T =
>> const boost::serialization::nvp<int>]’:
>> /usr/include/boost/serialization/serialization.hpp:74: instantiated
>> from ‘void boost::serialization::serialize(Archive&, T&, unsigned
>> int) [with Archive = RTT::marsh::binary_data_iarchive, T = const
>> boost::serialization::nvp<int>]’
>> /usr/include/boost/serialization/serialization.hpp:133: instantiated
>> from ‘void boost::serialization::serialize_adl(Archive&, T&, unsigned
>> int) [with Archive = RTT::marsh::binary_data_iarchive, T = const
>> boost::serialization::nvp<int>]’
>> /usr/include/boost/archive/detail/iserializer.hpp:316: instantiated
>> from ‘static void
>> boost::archive::detail::load_non_pointer_type<Archive,
>>>>> load_only::invoke(Archive&, T&) [with Archive =
>> RTT::marsh::binary_data_iarchive, T = const
>> boost::serialization::nvp<int>]’
>> /home/kaltan/src/git/orocos-rtt/src/marsh/binary_data_archive.hpp:198:
>> instantiated from ‘RTT::marsh::binary_data_iarchive&
>> RTT::marsh::binary_data_iarchive::load_a_type(T&, mpl_::false_) [with
>> T = const boost::serialization::nvp<int>]’
>> /home/kaltan/src/git/orocos-rtt/src/marsh/binary_data_archive.hpp:131:
>> instantiated from ‘RTT::marsh::binary_data_iarchive&
>> RTT::marsh::binary_data_iarchive::operator>>(T&) [with T = const
>> boost::serialization::nvp<int>]’
>> /home/kaltan/src/git/orocos-rtt/tests/mqueue_archive_test.cpp:71:
>> instantiated from here
>> /usr/include/boost/serialization/access.hpp:109: error: passing
>> ‘const boost::serialization::nvp<int>’ as ‘this’ argument of ‘void
>> boost::serialization::nvp<T>::serialize(Archive&, unsigned int) [with
>> Archive = RTT::marsh::binary_data_iarchive, T = int]’ discards
>> qualifiers
>>
>> Passing to operator>> the nvt struct as a return value of make_nvt()
>> makes gcc add a const qualifier, which gets propagated all the
>> way...I
>> have no idea why the other archive implementations don't suffer this.
>
> I solved it, and came to the conclusion that things got worse. It
> seems that, even when only calling
> <code>
> 
> boost::archive::detail::load_non_pointer_type<binary_data_iarchive,T>::load_only::invoke(*this,t);
> </code>
>
> I need to implement more function overloads (but not the load_object
> one), which are present in the binary archive case, but not in my
> class. I needed to add 3 load_override overloads in the iarchive:
> <code>
>             template<class T>
>             void load_override(T & t, BOOST_PFTO int){
>                 load_a_type(t,
> boost::mpl::bool_<boost::serialization::implementation_level<T>::value
> == boost::serialization::primitive_type>() );
>                 //archive::load(* this->This(), t);
>             }
>
>             void load_override(const
> boost::serialization::nvp<boost::serialization::collection_size_type>
> & t, int){
>                  size_t x=0;
>                  * this >> x;
>                  t.value() =
> boost::serialization::collection_size_type(x);
>              }
>
>             template<class T>
>             void load_override(const boost::serialization::nvp<T> &
> t, int){
>                  T x=0;
>                  * this >> x;
>                  t.value() = x;
>              }
>
>             template<class T>
>             Archive & operator>>(T & t){
>                 this->load_override(t, 0);
>                 return * this;
>             }
> </code>
>
> If you try to delete any of these three, serializing an std::vector
> will no longer work. In some way, the basic_iarchive accomodates for
> this case. So I traded one overload for load_object with three
> overloads for load_override. Because my load_object function was
> empty, it's very likely that the load_overrides were not required
> because the marshalling was not done at all.
>
> Someone with strong knowledge of all this needs to take my example +
> the existing codebase to something that does what is promised by the
> concept docs.
>
> The attached file is how the class lives in my current project, in the
> RTT::marsh namespace. I fixed my unit tests, and the class in
> attachment now works 100% for:
> * marshalling primitive types (non pointer)
> * marshalling structs/classes similar to std::vector, but which
> contain non-pointer data.
>
> It's not looking how I had imagined/hoped for, but it does the trick
> for me now. I'm happy to stick on this thread, but I'm not going to be
> able to solve this or make this an example that meets your
> expectations. It also looks that this thread no longer belongs on
> boost-users but on the boost list.
>
> To end with a positive note: I'm very happy how the separation between
> serialization and archiving has been (or has intended to be) done. It
> allows me to tap in a real-time archiver and re-using all
> serialization code. That's so cool. I checked the now working
> implementation again (stepped through the debugger as well) and it
> truely works for my test set.
>
> Peter
>
>
>
>
>> _______________________________________________
>> Boost-users mailing list
>> Boost-users_at_[hidden]
>> http://lists.boost.org/mailman/listinfo.cgi/boost-users