Re: [boost] [integer] Promotion of endian library code from vault

From: Phil Endecott (spam_from_boost_dev_at_[hidden])
Date: 2008-05-23 12:32:54

dizzy wrote:
> On Friday 23 May 2008 14:05:56 Roland Schwarz wrote:
>>    The standard gives no provisions for struct layout. So e.g. for
>>
>>    struct foo {
>>       big8_t a;
>>       big32_t b;
>>    };
>>
>>    one cannot predict the alignment of the members. (Or am I wrong
>>    in this respect?)
>
> Correct, which is why protocol binary structures are never mapped directly in 
> memory (you can with some compiler extensions but you won't gain anything 
> since I/O is the bottleneck in such cases and not memory copy). Instead a 
> serialization aproach should solve such issues.

Never say never :-)

I find that mmap()ing binary files into memory has the following 
significant advantage compared to read()ing them in: if the file is 
large and memory is tight, then read-in data must be swapped out i.e. 
written to disk.  In contrast, read-only mmap()ed pages can simply be 
discarded from RAM.  Even if the data is never actually swapped out, 
unless the OS over-commits swap space, disk will be reserved for this 
data.  So mmap() has a performance benefit on all memory-constrained 
systems and also a disk (i.e. flash) space benefit on embedded systems.

So while I'm generally quite pedantic about standards-compliance 
issues, struct layout is an area where I'm prepared to assume that the 
compiler does the "obvious" thing.  In this sort of code I tend to use 
integers with fixed sizes (i.e. int32_t rather than int) so that the 
data files are more likely to be portable between 32- and 64-bit 
systems.  Being able to declare their endianness using something like 
this library would be useful too.  But in practice, I'm nearly always 
doing this in an environment where the file is always going to be read 
by the exact same application binary.

Phil.