Re: [Boost-users] Boost MPI + Serialization

Subject: Re: [Boost-users] Boost MPI + Serialization
From: Raymond Wan (rwan_at_[hidden])
Date: 2008-11-11 03:16:49

Hi John,

John Phillips wrote:
> Raymond Wan wrote:
>>
>> Hi all,
>>
>> I am using Boost MPI and would like to send user-defined objects 
>> between processes.  As the Boost MPI tutorial / documentation 
>> suggests 
>> (http://www.boost.org/doc/libs/1_37_0/doc/html/mpi/tutorial.html), I 
>> should use serialization and in this document, they show an example 
>> class gps_position which is a friend with boost::serialization::access.
>>
>> So far, I understand what's going on, but I don't know what is the 
>> next step.  The example in the documentation stops there and  says 
>> that the example continues in the Serialization document.  It also 
>> just mentions that one could store it in many forms, such as XML on 
>> disk, in addition to sending it to processes.  If all I want to do is 
>> send it between processes, which one of the formats listed in 
>> boost/archive/*.hpp should I use?  In the Serialization document, it 
>> continues by creating a boost::archive::text_oarchive object.
>>
>> But, if I am sending things like vectors of unsigned integers and 
>> strings, a binary format should be more compact?  Or does this not 
>> matter and sticking with the Serialization example is fine?
>>
>
>   Ray,
>
>   The reason that the tutorial doesn't talk about archive selection is 
> because it is not a concern. The design of the Serialization library 
> separates the archive description away from the rules for serializing 
> the class, so the class doesn't ever need to know what format the 
> archive is using. The same serialization rules work, no matter what 
> format the archive is in.
>
>   The MPI library defines the "archive" formats it uses to transmit 
> your class in the detail folder of the library, and you need not worry 
> about them. The library makes the right choice of archive based on 
> what you put in your class definition and use. The reason the 
> documentation points out other archive formats is to let you know that 
> the ability to use these formats to save your data in other ways comes 
> for free once you have added serialization functionality to your class.
>
>   A cruise through the examples for the library will show you a use of 
> serializing a custom class, and show that you need never specify the 
> archive type. There is an efficiency choice you need to consider. That 
> is, do your types have a fixed layout in memory. As the tutorial 
> points out, this can provide significant savings. (Note, there is also 
> a useful savings if you can assure a homogeneous cluster. There is a 
> macro definition for this one.)
>
>   I hope that helps. And, of course, if I got anything wrong or the 
> authors have a better way to talk about it, I'm sure a correction will 
> show up, soon.

Yes, this was very helpful; thank you!  I definitely misunderstood 
something.  I didn't know that all I had to do was to define a friend 
function and then MPI will do the rest for me.  I thought I had to 
specify an archive format, for example, since the Serialization example 
used serialization and archiving; I didn't realize archiving was just 
meant to demonstrate how serialization can be used.  Also, I didn't  
realize the mention of XML was meant to mean that "if you plan to do 
something with it like save it to disk, you already have it ready".  I 
guess I was lost in the clouds...

So, I had been trying to select an archive format and getting my code to 
compile...

Got what you mean and wrote the serialize function and sent the object 
directly...never thought of trying that before.  But indeed, that worked 
immediately.  A bit surprised it was "that easy" [knock on wood ;-) ].

Thank you very much!

Ray