Re: [boost] CRC: endian madness

From: Darin Adler (darin_at_[hidden])
Date: 2001-04-19 12:42:29

on 4/17/01 9:24 AM, Daryle Walker at darylew_at_[hidden] wrote:

> Several people have had problems with the augmented CRC tests, while I
> haven't.  I realized that the CRC algorithms assume that they get numbers
> most-significant bit first.  For most raw data, you don't care about endian
> issues, but the augmented CRC tests read their own CRC after the raw data,
> causing a problem if the data is not in big-endian order.  I suspect that
> the users reporting problems are using little-endian computers, while I have
> a big-endian computer.  What can I do to solve this problem?

The key to almost all endian problems is to not convert between different
sizes by type casting. The mistake made in the CRC test program (in
augmented_test) is that it generates 32-bit random numbers, and then
converts them to bytes to make a CRC for them. One way to fix this is to use
8-bit random numbers instead.

This is the danger of providing a CRC algorithm that takes a "const void *"
as the parameter type. It's very easy to inadvertently create an
endian-dependent CRC. I would have instead made the parameter be "const char
*", "const unsigned char *", and "const signed char *" overloaded. The
alternate version that takes "const void *" could be included, but needs
some kind of name that makes it clear it's potentially machine-specific
(Sorry, I haven't yet weighed in on the CRC review because I have such
limited time to contribute to boost, but this was something I had planned to
mention.)

If you want to make a 32-bit CRC, and then put it into an array of bytes,
you can do that by extracting the various 8-bit quantities using shifting.
There are classes to do this in my endian.hpp header:

    http://groups.yahoo.com/group/boost/files/endian/endian.hpp

You would do a boost::store_big_endian to put the 32-bit CRC into 4
consecutive bytes in your array of bytes.

Another approach to this class of problem is to use endian-specific classes
like the ones in my endian.hpp header. If ran_data was declared as an array
of boost::big_endian<uint32_t> instead of an array of uint32_t, the problem
would presumably just "go away" with no additional code changes.

    -- Darin