Re: [Boost-users] [Boost.MPI] question about serializingcustomobject

Subject: Re: [Boost-users] [Boost.MPI] question about serializingcustomobject
From: Robert Ramey (ramey_at_[hidden])
Date: 2010-08-27 13:44:39

David Abrahams wrote:
> At Thu, 26 Aug 2010 16:05:41 -0800,
> Robert Ramey wrote:
>>
>> Dave Abrahams wrote:
>>> BoostPro Computing, http://boostpro.com
>>> Sent from coveted but awkward mobile device
>>>
>>>> If one were using heterogenious machines, I could understand the
>>>> usage of MPI types. But as I understand it, the MPI serialization
>>>> presumes that the machines are binary compatible.
>>>
>>> You're mistaken.
>>>
>>>>   So I'm just not seeing this.
>>>
>>> Start reading here:
>>> http://www.boost.org/doc/libs/1_39_0/doc/html/mpi/tutorial.html#mpi.skeleton_and_content
>>> and all will be revealed
>>
>> lol - I've read that several times.
>
> I always wonder, when you write that, whether you're physically
> laughing out loud.  That's OK; don't spoil the mystique ;-)

I actually do think I snicker a little bit.  Sort of one "snick".

This above is exactly the type of thing which provokes this reaction.
Reading your comment, one has to conclude that

a) you think I haven't read it but am expressing an opinion anyway
b) that you think that the documentation is clear and complete
c) that I'm somehow responsable for not finding/reading/understanding it.

You make this kind of presumption all the time.  It doesn't bother me
but it always provokes at least one "snick"

>> I just never found it to be very revealing.  The word skeleton
>> seemed pretty suggestive. It's still not clear to me how such a
>> think can work between heterogeneous machines.  For example, if I
>> have an array of 2 byte integers and they each need to get
>> transformed one by one into a 4 byte integer because that's closest
>> MPI data type,

> I think you don't understand what MPI datatypes do.

This is true.  I suppose that's one reason why the documentation
made no sense to me. They just looked like special types
to make identify primitives accross differing architectures.

<snip>
Interesting information which one should consider adding to the
MPI documentation.
</snip>

> And since  Boost.MPI and Boost.Serialization are so closely related, I 
> think it's
> especially important that *you* underestand.

I disagree.  Boost MPI depends upon Boost.Serialization
but not the other way around.  I shouldn't have to understand Boost MPI just
as I can't be expected to understand all the usages to which boost 
serialization
might be applied.  Trying to do this, aside from the time involved, might 
well
be counter productive in that it can trick one into coupling things which
would otherwise not be.  For all these reasons I've refrained from investing
a lot of time in understanding MPI as it relates to serialization.  I'm 
happy
with Mathias efforts and commitment to supporting his library and
don't want to muck up the works.

I only really have few observations/suggestions at this point.

a) It would be  helpful if there were a way to test the serialization
of the archive classes in boost MPI without having MPI installed.
If this is not possible, it would seem to me that the serialization is
intertwined with the data transport - rather than be separated
as it is in the stream/streambuf io/stream design.  This would look
like a design flaw to me.

b) user experience seems to show that archive construction/destruction
is a significant performance issue when a new archive is made for each
data transmission.  On the otherhand, one has to do this since the
current archive implemenation track addresses of serialized obects
so the same archive can't e use send the same structure (maybe
with changed data) multiple times.  Given that MPI has a focus
on performance, I wonder if this has been considered. I looked
a the documentation, code and examples and it wasn't obvious
to me how this is question was addressed - if at all.

c) I think the above information regarding how MPI and serialization
fit together in boost MPI would be a worth addition go the MPI
documentation.  AND it's already written !

You should know that all your efforts to educate me are not wasted.

a) Sometimes I pass your advice along to other people.

b) Motivated by our recent discussions on the subject as well as some
other issues has clarified my ideas on how the archive concept
should be refined to be more helpful to those creating their own
archives.  I think there will be improvements in this area in part
due to your observations.

Robert Ramey