[boost] serialization - fast wire

From: Hurd, Matthew (hurdm_at_[hidden])
Date: 2003-12-16 08:55:07

Robert, or other interested party,

Any thoughts about including a so called "fast wire" approach in
serialisation.  That is, using a block of data that is the same as the
internal representation, including alignment padding, and being able to
blurt that out, perhaps with an optional native data representation
(NDR) suffix or prefix that describes the block.  I read an interesting
report, "Efficient Wire Formats for High Performance Computing",
Bustamante, Eisenhauer, Schwan, Widener (gatech.edu), on this speeding
network comms up considerably.  The principle being that sending and
reconstruction can be quicker as well as there existing zero-copy
potential...  The paper suggests 3 orders of magnitude speed up for send
and one order for receiving to give a 45% reduction in round trip time
for their network case (relative to MPI).  

The authors' Portable Binary I/O approach seems a little clumsy for a
C++ app compared to the neat potential to do the same with the
serialization library.  There is a bit more meat in their sandwich as
they can generate custom translators on the fly on the receiver side to
translate the NDR incoming format into a consumable morsel. 

I guess such a beast would only work with POD compatible stuff, but
could be extended to having some fast pointer replacement scheme on
reception with an appropriate customer allocator thrown in...  

I understand that you're concentrating on accuracy and "beauty" rather
than performance, as you quite rightly should, but there might be some
interesting thoughts here for you and boost::serialization, so I thought
I'd pass this on...

Regards,

Matt Hurd

PS:  The paper is available at the always too busy citeseer, but if
anyone would like it, send me a mail and I'll forward it on.
_______________________

Susquehanna Pacific P/L
hurdm_at_[hidden]
+61.2.8226.5029
_______________________

IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments.  Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited.  Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument.  Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.