Re: [Boost-users] Writing binary data in a specific byte ordering (i.e. network order)

From: dizzy (dizzy_at_[hidden])
Date: 2008-01-18 05:03:16

Hi

On Friday 18 January 2008 00:30:00 Richard wrote:
> [Please do not mail me a copy of your followup]
>
> Does boost have a facility for this?
>
> I have a bunch of code that did binary I/O on x86.  I want to make the
> code endian-neutral.  This is similar to writing binary data in
> "network order" as is done with TCP and other internet protocols.
>
> I was thinking that you could now determine completely at compile-time
> whether or not bytes need to be swapped using a template class.

Can you please detail on this? I am currently working on such a thing, a 
portable C++ low level serializer (for integrals and strings that IMO are the 
building blocks of almost any protocol or format specification) that does 
almost 100% compile time decisions and code generation but I was unable to 
find a solution to determine at compile time the host endianess (other than 
checking for various ifdefs as boost/config/endianess.hpp does if I remember 
well including having a configure program for your build that sets such 
defines by running a test program to detect host endianess).

> My thought was that you would have a class for writing "LittleEndian"
> bytes that took a boolean template parameter.  The parameter is
> determined at compile-time based on the endian-ness of your
> architecture.  Then a specializations for the template determines
> whether or not you just use stream.write() to write the binary bytes
> directly from the POD datum or whether or not you manually extract the
> individual bytes for writing in the proper order.

Yeah, something similar I have in my code (although more generic, in the form 
of convert_endianess<HostEndianess, ExternEndianess> where HostEndianss and 
ExternEndianss can be "little_endian" or "big_endian" (note that there is 
also "middle endianess" or that found on PDP systems which is also a valid 
endianess).

Another issue to worry about is byte size on your system, it may be different 
than for the network data you need to send (ie std::numeric_limits<unsigned 
char>::digits != 8) which also complicates things (but have specializations 
for when the host byte size matches extern byte size).

Other issues to worry are signess (if you will need to serialize signed types 
there are at least 2 types of representing negative integral values on C/C++ 
platforms, that is 1's complement and 2's complement) and bit order (this 
however I am not sure how to deal with and if it is an issue at all since you 
cant really determine how your hardware orders bits, you can just access them 
by masks of values which means accessing them by significance order and not 
by hardware order).

So my integral types encoding are structs of byte size, type size (in bytes), 
byte endianess and signess.

> I hope that's clear... it hasn't gelled in my mind enough to have a
> sample implementation.  I'm hoping that there's a facility in boost
> for this already?

No idea if it is, even if it is I have to reimplement this myself because it 
is for a univesity project but would be interesting to see how others 
(especially with high quality as boost) does it.

-- 
Mihai RUSU					Email: dizzy_at_[hidden]
			"Linux is obsolete" -- AST