Re: [Boost-users] [Boost.MPI] question about serializingcustomobject

Subject: Re: [Boost-users] [Boost.MPI] question about serializingcustomobject
From: Robert Ramey (ramey_at_[hidden])
Date: 2010-08-27 16:26:03

David Abrahams wrote:
>>> And since Boost.MPI and Boost.Serialization are so closely
>>> related, I think it's especially important that *you* underestand.
>>
>> I disagree.  Boost MPI depends upon Boost.Serialization
>> but not the other way around.
>
> I didn't say they were interdependent, just that they were
> closely-related.  But anyway, I think you're being shortsighted: The
> success of tools built on top of Boost.Serialization is an important
> indicator of the correctness and genericity of its design.

> Boost.Serialization *is* dependent on Boost.MPI for a portion of its
> userbase,

<off topic> what portion of the user base? (This is a rhetorical question
please don't spend any time answering it.)  It's always bothered me
that we have no per library data/feedback on the interest/usage
on a library by library basis. I've spoken about this before and
I know you agree with me about it.  But until this get's addressed
I think we just have to set these questions aside and recognize
that the answers are anyone's guess.

<snip>

> You can't be a generic serialization library without reference to real
> applications.  Maybe you still haven't really decided whether serving
> Boost.MPI and its users is something you want to do.

 I think that
> might account for a good deal of the recurring friction we experience.
> It would be a good idea to settle on an answer to that question, along
> with the question of what other applications you're willing to
> support.

I am acutely aware of this aspect of "generic library design".  Every
post on this list reminds me of it.

>> I only really have few observations/suggestions at this point.
>>
>> a) It would be  helpful if there were a way to test the serialization
>> of the archive classes in boost MPI without having MPI installed.
>> If this is not possible, it would seem to me that the serialization
>> is intertwined with the data transport - rather than be separated
>> as it is in the stream/streambuf io/stream design.  This would look
>> like a design flaw to me.
>
> Helpful to whom?  I agree this is all a good idea, but I'm not sure
> how this relates to anything else we're discussing.

It would permit me to use the MPI serialization as a use case
to verify that I hadn't broken anything. I know what your going
to say - concepts - but that is going to be a while before that
happens.

>> b) user experience seems to show that archive
>> construction/destruction is a significant performance issue when a
>> new archive is made for each data transmission.  On the other hand,
>> one has to do this since the current archive implemenation track
>> addresses of serialized obects so the same archive can't be use send
>> the same structure (maybe with changed data) multiple times.

> I can't understand how you reach that conclusion.  In my long
> explanation I thought I made it clear that Boost.MPI and its database
> get a great deal of its performance advantage from exactly that:
> sending the same structure multiple times with the same MPI type map.

hmm - what about tracked objects? I suggested turning off tracking
and Matthias told me that it was needed to send pointers.  But for this
to work an archive has to be reconstructed everytime to re-initialize
the tracking lists.  I don't want to spend a lot of time on this as that
means I have to delve into the MPI serialization and I don't want
to do this.

>> Given that MPI has a focus on performance, I wonder if this has been
>> considered. I looked a the documentation, code and examples and it
>> wasn't obvious to me how this is question was addressed - if at all.
>
> We've obviously misunderstood one another somewhere along the way.  It
> would be good if we could get that cleared up.

I think it would take too much time for me to understand.  And if I did
come to understand it, I might have something to say about it.  If I
said it, then it might start a thread which would cost even more time
which I don't have right now.

Robert Ramey