Re: [boost] [endain_ext] Beman's Integer.Endian extensions to work with endian unaware data

Subject: Re: [boost] [endain_ext] Beman's Integer.Endian extensions to work with endian unaware data
From: Stewart, Robert (Robert.Stewart_at_[hidden])
Date: 2010-06-14 07:42:19

vicente.botet wrote:
> Rob Stewart wrote:
> > vicente.botet wrote:
> >>
> >> // definition of a point of view (domain)
> >> struct network {};
> >>
> >> // mapping of native types to an endian tree  for the network
> >> point of view (domain)
> >> namespace boost { namespace endian {
> >>     template <>
> >>     struct domain_map <network, X::big_c> {
> >>         typedef mpl::vector<big,big> type;
> >>     };
> >>     template <>
> >>     struct domain_map <network, X::little_c> {
> >>         typedef mpl::vector<little,little> type;
> >>     };
> >>
> >> }}
> >
> > I have no idea what that means.  (Having now studied the
> > rest of the example, I understand the purpose for the above,
> > but that doesn't mean I really understand it.)
>
> I think that the use of the newtwork works was not
> adequented. Just replace "netwrok" by "ifA".
> This means that relative to the interface "ifA" the structure
>  X::big_c has its two fields in big endian format, and that
> the structure X::little_c has its two fields in little endian
> format. Note that the integer used when not endian aware.

The MPL vectors are, IIUC, a sequential list of the endiannesses of each field in the domain_map's second template parameter, right?

I don't quite get the "relative to the interface" part.  It would seem more reasonable to declare how to convert each field's endianness to the first template parameter's endianness or else to simply declare each field's endianness (more below).

> >> // view of the structures as fusion sequences
> >> BOOST_FUSION_ADAPT_STRUCT(X::big_c,
> >>     (uint32_t, a)
> >>     (uint16_t, b)
> >> )
> >> BOOST_FUSION_ADAPT_STRUCT(X::little_c,
> >>     (int32_t, a)
> >>     (int16_t, b)
> >> )
> >> BOOST_FUSION_ADAPT_STRUCT(X::mixed_c,
> >>     (X::big_c, a)
> >>     (X::little_c, b)
> >> )
> >
> > The type duplication is unfortunate, but that was necessary
> > in Beman's approach, too.
>
> You mean Tom's approach?

No, I did mean Beman's approach, which necessitates declaring a duplicate structure of endian types when you need or are given a native type structure (such as an OS or RTL structure) for an API.  (struct tm, for example, would require a duplicate, endian-type-based structure in Beman's approach.)

> The difference is that Tom used a different macro, so an
> application needing to see the structure has  fussion
> sequence and to define the Tom swap  function needed to declare both.

I've totally lost track of how Tomas' worked, hence my wishing to see the same example in both approaches.

> >>     X::mixed_c m;
> >>     // fill with data from the 'network' domain (emulating
> >> the reception from the domain 'network')
> >>     m.a.a=0x01020304;
> >>     m.a.b=0x0A0B;
> >>     m.b.a=0x04030201;
> >>     m.b.b=0x0B0A;
> >
> > This is just putting raw data, of assumed endianness, into
> > the structures, right?
>
> Yes. Sorry if the code comment was not clear.

I was just trying to confirm what I understood you to write and the meaning of that part of the code, so there was no confusion as we progressed through the example.

> >>     // convert from the network to the native domain
> >>     endian::convert_from<network>(m);
> >
> > Here you have declared that the current data in m is in
> > network order and requested that it be converted to host
> > order, right?
>
> Just a precission. Convert from *the point of view given by
> the domain map 'network or "ifA"* to the native endian. I
> repeat the use of the "network" point of view let some of you
> think that it was big-endian,but it wsa not big-endian in all
> the cases.

So, this code, combined with the domain_map specialization above says that, m.a is big endian relative to "network" (or "ifA" if you had written endian::convert_from<ifA>(m)) and that "network" somehow implies a mapping to host order.  I don't quite get the point of that relativity.

If your domain_map were, instead, an endianness_map that merely noted the endianness of each field, then the foregoing would make more sense.  I mean something like this:

   template <>
   struct endianness_map<X::big_c>
   {
      typedef mpl::vector<big,big> type;
   };

That would enable the following invocation, unless I'm mistaken:

   endian::convert_from(m);

Here, "convert_from" implies from the endianness of "m" which is described by the endianness_map<X::big_c> specialization, to host order (implicit in the name).  At that point, it seems that "to_host" would be clearer:

   endian::to_host(m);

Isn't that possible and more straightforward?

It would be nice if the BOOST_FUSION_ADAPT_STRUCT functionality could be reused to get the following syntax:

   BOOST_ENDIAN_ADAPT_STRUCT(X::big_c,
      (uint32_t, big, a)
      (uint16_t, big, b)
   )

> > That's no different, other than implementation details,
> > than what Tomas originally provided, IIUC.
>
> Tom's swap_in_place force to interpret all the fields of the
> structure with the same endianness.

Ah, right.

> Here it is the code generated (gcc-4.4 -O3) to do the
> conversion from big to little + the initialization of the variable.
>
>  movl $67305985, %eax
>  movb %al, __ZN12_GLOBAL__N_11mE+3
>  movl %eax, %edx
>  shrl $8, %edx
>  movb %dl, __ZN12_GLOBAL__N_11mE+2
>  movl %eax, %edx
>  shrl $16, %edx
>  movb %dl, __ZN12_GLOBAL__N_11mE+1
>  shrl $24, %eax
>  movb %al, __ZN12_GLOBAL__N_11mE
>  movl $2826, %eax
>  movb %al, __ZN12_GLOBAL__N_11mE+5
>  shrl $8, %eax
>  movb %al, __ZN12_GLOBAL__N_11mE+4
>
> instead of without
>
>  movl $67305985, __ZN12_GLOBAL__N_11mE
>  movw $2826, __ZN12_GLOBAL__N_11mE+4
>
> I don't understand this code, but IMO the code has been
> completly inlined.

I'm not certain whether that is optimal, but I don't know what the assembly should be for the necessary operations in this case.  Perhaps generating the same conversion using an intrinsic and comparing the generated assembly would help.

_____
Rob Stewart                           robert.stewart_at_[hidden]
Software Engineer, Core Software      using std::disclaimer;
Susquehanna International Group, LLP  http://www.sig.com

IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.