Re: [boost] [boost::endian] Request for comments/interest

Subject: Re: [boost] [boost::endian] Request for comments/interest
From: Jonathan Franklin (franklin.jonathan_at_[hidden])
Date: 2010-06-01 17:41:19

Sorry for my late replies.  I have been offline for a bit.

On Fri, May 28, 2010 at 2:27 PM, Terry Golubiewski <tjgolubi_at_[hidden]> wrote:
> Since IP packets cannot be 10GB, I submit that you're going to have to break
> your 10GB array down into messages.  Then you're going to copy portions of
> the 10GB array into those messages and send them.   In the type-base
> approach the message may indeed contain an array.

As Tomas already pointed out, explicitly breaking a blob of data into
chunks for network transfer is superfluous in streaming protocols such
as TCP.  The protocol will take care of this more efficiently at the
network and/or transport layer than I can at the application layer.
Further, if I break my data blob into k segments and insert my own
message header as a preamble to each segment, then I unnecessarily add
k*sizeof(my_header) to the stream.

> In the message-based interfaces that I am used to, one always must copy some
> data structures into a message before you send it.

As Tomas also pointed out, this is not a marshaling library.  However,
a marshaling library could make good use of an "endian" library such
as this.

In addition to my network needs, I also have at least two image file
formats that have the option of storing the arbitrarily sized data on
disk in big- or little-endian format.  A library that allows me to
slurp up the image data into memory and swap in place if needed is
terribly useful to me.

Jon