Re: [boost] [boost::endian] Request for comments/interest

Subject: Re: [boost] [boost::endian] Request for comments/interest
From: Terry Golubiewski (tjgolubi_at_[hidden])
Date: 2010-06-03 09:45:03

Robert Stewart wrote...
>
> I have some problems with your tests.

Me too, I didn't get the results I expected.  :o)

> Reading the data into memory shouldn't be part of the timed code.  If you 
> apply the swap-in-place logic an odd number of times, the result will be 
> host order.

The test application is reading a big-endian (not-native) file into memory 
from a disk file.  The reading disk into memory code is exactly the same for 
both approaches.
The application might be a video decompression library where the data was 
compressed and stored on a big-endian machine.  The receiving machine is 
little-endian though (in my case) and
should decode the large "coefficient" file into memory in native format. 
Because once in memory, each coefficient will be accessed multiple times. 
This is the "killer-app" that favors in-place-swapping.

The disk file remains in big-endian format, and each time is converted to 
little-endian.  I just do it a lot of times so that I get a long measurement 
interval, so interference from other threads cancels out. The deviation of 
the results was around 0.1 seconds.

Reading data from a file may also help to avoid cache issues.  I wasn't 
trying to measure how fast each approach is, just how to measure their 
relative speed using a specific application where swapping was expected to 
have an advantage.

>>       disk_array& src = reinterpret_cast<disk_array&>(array2);
>>       interface::copy(src.begin(), src.end(), array2.begin());
>
> Two lines instead of one.  I find this less desirable.  I imagine you 
> could make your copy function more helpful.

I could make it more helpful; but then it would be called swap_in_place and 
would be implemented just like Tomas'!

>>       for ( ; src != end; ++src, ++dst)
>>         *dst = swap<little_to_machine>(*src);
>
> s/little_to_machine/big_to_machine/?

I did the tests in both big-endian --> little-endian and little->endian to 
little->endian.  Unfortunately, I forgot to change the swap<arg> back to 
big_to_machine when I cut&pasted.
Sorry about the confusion.

>> --- typed based ---
>>   for (int trial=0; trial != 1000; ++trial) {
>>     {
>>       ifstream input("array.dat", ios::binary);
>>       input.read(reinterpret_cast<char*>(&tmp_array),
>> sizeof(tmp_array));
>>       interface::copy(tmp_array.begin(), tmp_array.end(),
>> array2.begin());
>
> Does this actually swap anything?  Doesn't this just copy the data to 
> unswapped objects that *would* swap on access?  That's hardly a fair 
> comparison.

array2 is just a plain array<uint32_t, SIZE>.  tmp_array is an 
array<endian<big, uint32_t>, SIZE>.  So the conversion/copy happens in 
interface::copy.

> I don't see any code reading the resulting values.  That unfairly taints 
> the tests in favor of the object-based approach.  If the underlying data 
> is big endian, then the object-based approach implies swapping on every 
> access to the data.  Reading each value once would be the optimal use case 
> for the object-based approach.  Reading multiple times would clearly favor 
> the function-based approach.

Both tests convert the disk-file into native machine format, so there is no 
reason to re-read the data once it been converted, other than to verify its 
correctness.

terry