Re: [boost] [boost::endian] Request for comments/interest

Subject: Re: [boost] [boost::endian] Request for comments/interest
From: Terry Golubiewski (tjgolubi_at_[hidden])
Date: 2010-06-02 00:59:18

----- Original Message ----- 
From: "Jonathan Franklin" <franklin.jonathan_at_[hidden]>
Newsgroups: gmane.comp.lib.boost.devel
To: <boost_at_[hidden]>
Sent: Tuesday, June 01, 2010 4:41 PM
Subject: Re: [boost::endian] Request for comments/interest

Sorry for my late replies.  I have been offline for a bit.

On Fri, May 28, 2010 at 2:27 PM, Terry Golubiewski <tjgolubi_at_[hidden]> 
wrote:
> Since IP packets cannot be 10GB, I submit that you're going to have to 
> break
> your 10GB array down into messages. Then you're going to copy portions of
> the 10GB array into those messages and send them. In the type-base
> approach the message may indeed contain an array.

As Tomas already pointed out, explicitly breaking a blob of data into
chunks for network transfer is superfluous in streaming protocols such
as TCP.  The protocol will take care of this more efficiently at the
network and/or transport layer than I can at the application layer.
Further, if I break my data blob into k segments and insert my own
message header as a preamble to each segment, then I unnecessarily add
k*sizeof(my_header) to the stream.

> In the message-based interfaces that I am used to, one always must copy 
> some
> data structures into a message before you send it.

As Tomas also pointed out, this is not a marshaling library.  However,
a marshaling library could make good use of an "endian" library such
as this.

In addition to my network needs, I also have at least two image file
formats that have the option of storing the arbitrarily sized data on
disk in big- or little-endian format.  A library that allows me to
slurp up the image data into memory and swap in place if needed is
terribly useful to me.

Jon

In another post, I finally understood this use-case.
Again (being the non-swap advocate) to address this, I would suggest...

template<endian_t E1, class T, endian_t E2>
endian<E2, T>* copy(const endian<E1, T>* first, const endian<E1, T>* last, 
endian<E2, T>* dest) {
  if (E1 == E2 && first == dest)
    return dest + distance(first, last);
  while (first != last)
    *dest++ = *first++;
  return dest;
}

Assuming you just read a TCP stream of big-endian T's into memory at a 
location 'char* buf', then you could endian-convert the T's, if necessary, 
like this...

const endian<big, T>* src = reinterpret_cast<const endian<big, T>*>(buf);
endian<native, T>* dst = reinterpret_cast<endian<native, T>*>(buf);

(void) copy(src, src+numTs, dst);

Would this do what you want?

terry