Re: [boost] [1.44] Beta progress?

Subject: Re: [boost] [1.44] Beta progress?
From: Matthias Troyer (troyer_at_[hidden])
Date: 2010-07-24 00:10:07

> 
> Now, taking a look at  the mpi usage of serializaiton.
> 
> I realy haven't looked at it enough to really understand it
> so I maybe wrong about this - these are only casual
> observations.
> 
> a) it seems that the "skeleton" idea seems to depend on the
> idea that the size of the data stored in the bniary archive be the same
> as the size of the underlying data type.  Up until now that
> has been true even though there was never any explicity
> guarantee to that effect.  I had to change the behavior in
> order to extract myself from some other fiasco and this
> "feature" was no longer true.  I think this is where the
> problem started.  It's no one's fault.

Not at all. We don't use binary archives at all.
> 
> b) the MPI file sends the class versions over the wire.
> It doesn't need to do this.  If you look at some of the
> archives there is class_optional_id which is trapped
> by the archvie classes and suppressed both on input
> and output because that particular archive class
> doesn't need it. But it's there if someone want's
> to hook it (like an editing archive). I think MPI
> might want to do the same thing with version_type.

That's an idea but it does not solve the issue we have.

> 
> c) I'm not sure how MPI uses portable binary archive
> (if at all).  Seems like that might be interesting.

It does not and it would make it VERY inefficient if it did. Please keep in mind that on modern networks memory bandwith is sometimes comparable to network bandwidth.

> 
> d) what is really needed to send data "over the
> wire" is to be able to supress tracking at the
> archive level.  

No, we might want to send pointers.

> The would permit the same data
> to be sent over and over and wouldn't presume
> that the data constant. So you wouldn't have to create
> a new archive for each transaction.  I've puzzled about how
> to do this without breaking the archive concept. Turns
> out it's a little tricky.  And there doesn't seem to be
> much demand for it - but maybe there would be if
> I did it.
> 
> e) this bit of code is what created the the issue
> with the Sun compiler.
> 
>> The problem comes from this line in boost/mpi/datatype_fwd.hpp:
>> 
>> template<typename T> MPI_Datatype get_mpi_datatype(const T& x = T());
>> 
> 
> Frankly, it's just plain wrong and should be fixed.  You might
> say that you know it's wrong but it works around this
> or that template or compiler quirk and it's too hard to fix.
> I could accept that.  But if it's fixable, it should be fixed.
> 
> I did make the constructors of version_type public.  I had
> made them private to trap errors in code where they
> were constructed but not initialized.  Now error like this
> arn't trapped.  So I think you should fix this.

So far ALL primitive types had default constructors and this was perfectly legal. Yes, I agree that you never specified what those types were and what there properties were. As I mentioned before, what you say now summarizes as "use any part of the library at your own risk. If you want to write your own archives the only safe way is to reimplement Boost.Serialization from scratch."

> f) I believe that MPI uses binary_archive_base? as a basis.
> you could have used a higher level class as a basis.  I don't
> know that that woudl have made things easier or harder
> but it's worth looking into.  The binary_archive is actually
> very small - only a few hundred lines of code.  This could
> have been cloned and edited. This might or might not
> have made things more/less intertwined with the other
> archive classes.  This isn't a suggestion - just an observation
> that it might be worth looking into.
> 
No, again that is not the issue. The issue were changes in the primitive types that any archive has to support.

Matthias