Re: [boost] Access to unaligned (packed) structs

From: Phil Endecott (spam_from_boost_dev_at_[hidden])
Date: 2007-10-11 09:16:21

Hi Roland,

Roland Schwarz wrote:
> I am searching for a library / component / pattern that will help me 
> with the following problem:
>
> I have a memory structure in raw memory (received from a measurement 
> instrument), that is unaligned in its data members.
>
> Now I want to have an access interface to this data that comes as close 
> as possible to access of a struct.

I have had to do this for USB descriptors.  In my case, the hardware 
(ARM) does not support unaligned accesses.  There is also a question of 
endianness to worry about.

Do you need read-only or read-write access?

> (This should work compiler/platform independent of course.)

It's not difficult to write NON compiler-independent code that does the 
right thing, e.g. for gcc:

struct foo {
      char a;
      int b;
} __attribute__ ((packed))

This will do the right thing.  In particular, when you read from foo.b 
on my ARM system the compiler will generate a series of four byte read 
instructions and combine the results.

However, you have to be very careful what you do with such a field.  In 
particular, you mustn't take the address of foo.b:

void f(int* p) { ... }
f(&foo.b);

Here, the code inside f will assume that p is aligned; it will NOT 
generate the byte reads and if you're lucky you'll get a bus error; if 
you're unlucky you'll just get the wrong answer.  (I spent a long time 
debugging something like this.... largely because gcc does not give any 
warning when you take the address of an unaligned field.  On x86, 
however, it probably will work since the hardware supports unaligned 
accesses - though I believe that the compiler is still allowed to 
assume that pointers are aligned, even on x86.

I believe that other compilers have ways to declare a struct as packed 
with very similar semantics.  So you could probably write a library 
with some per-compiler macros to make this multi-compiler, but not 
compiler-independent.  If you really want standards-compliant 
compiler-independent code:

Quoting from a later message:
> template<class T>
> struct packed
> {
>      unsigned char data[sizeof(T)];
>      operator T&() const
>      { return static_cast<T>(*data); }
> }

I think you mean something like

operator T() const { return *(reinterpret_cast<T>(*data)); }

don't you?  If that's what you mean, the answer is that No, it won't 
work (but will "probably" work on x86).  You need to memcpy into an 
aligned T:

operator T() const {
   T t;
   memcpy(&t,data,sizeof(T));
   return t;
}

If you need it to be writeable you can add an operator= and 
copy-constructor that memcpy the other way.

My experience is that g++ will optimise-away the memcpy in this sort of 
code and you'll end up with something close-to-optimal.  I don't know 
about other compilers (and would love to know).

But I think the problem is that when you build your struct containing 
these packed<T> objects:

struct foobar
{
      packed<char> a;
      packed<int> b;
}

I don't think (but am not sure) that you can assume that there is no 
padding between a and b.  Can someone confirm?  I have a vague 
recollection that the compiler has some freedom in this case because it 
changed at some point for gcc on ARM, which resulted in binary incompatibility.

I hope that is of some use.  Good luck!

Phil.