Re: [boost] [boost::endian] Request for comments/interest

Subject: Re: [boost] [boost::endian] Request for comments/interest
From: Stewart, Robert (Robert.Stewart_at_[hidden])
Date: 2010-06-03 09:55:19

Terry Golubiewski wrote:
> Robert Stewart wrote...
> >
> > Reading the data into memory shouldn't be part of the timed
> > code.  If you apply the swap-in-place logic an odd number of
> > times, the result will be host order.
>
> The test application is reading a big-endian (not-native)
> file into memory from a disk file.  The reading disk into
> memory code is exactly the same for both approaches.

It is the same in both cases, but there could be plenty of variation in that larger body of code you actually timed that swamp what you wanted to measure.

> The application might be a video decompression library where
> the data was compressed and stored on a big-endian machine.
> The receiving machine is little-endian though (in my case)
> and should decode the large "coefficient" file into memory in
> native format.

I'm not arguing against reading data from disk, just against timing that part of the code.

> Because once in memory, each coefficient will be accessed
> multiple times.  This is the "killer-app" that favors
> in-place-swapping.

Sure, but you didn't read the data multiple times.

> Reading data from a file may also help to avoid cache issues.

That may be true, yet hardly useful when trying to compare the performance of the two approaches.

> I wasn't trying to measure how fast each approach is, just
> how to measure their relative speed using a specific
> application where swapping was expected to have an advantage.

To do that, you must measure the speed of each approach.

> >>       disk_array& src = reinterpret_cast<disk_array&>(array2);
> >>       interface::copy(src.begin(), src.end(), array2.begin());
> >
> > Two lines instead of one.  I find this less desirable.  I
> > imagine you could make your copy function more helpful.
>
> I could make it more helpful; but then it would be called
> swap_in_place and would be implemented just like Tomas'!

Yours would still be in terms of the endian types.

> >> --- typed based ---
> >>   for (int trial=0; trial != 1000; ++trial) {
> >>     {
> >>       ifstream input("array.dat", ios::binary);
> >>       input.read(reinterpret_cast<char*>(&tmp_array),
> >> sizeof(tmp_array));
> >>       interface::copy(tmp_array.begin(), tmp_array.end(),
> >> array2.begin());
> >
> > Does this actually swap anything?  Doesn't this just copy
> > the data to unswapped objects that *would* swap on access?
> > That's hardly a fair comparison.
>
> array2 is just a plain array<uint32_t, SIZE>.  tmp_array is an
> array<endian<big, uint32_t>, SIZE>.  So the conversion/copy
> happens in interface::copy.

I don't think that's right.  The copy is done in interface::copy(), but the conversion (swap) doesn't occur until you read from the endian objects, right?  My point is that you need to walk the array summing the values, or something, to read each value at least once.  Otherwise, your code merely copies the data as read from disk to the endian objects but doesn't swap any bytes, and yet you compare that to function-based code that does swap.

(Don't forget to write the sum -- or whatever result you compute -- to stdout, for example, to avoid the logic's being optimized out of your test.)

_____
Rob Stewart                           robert.stewart_at_[hidden]
Software Engineer, Core Software      using std::disclaimer;
Susquehanna International Group, LLP  http://www.sig.com

IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.