Re: [boost] Is there interest in a library for object (especially STL object) marshalling?

Subject: Re: [boost] Is there interest in a library for object (especially STL object) marshalling?
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2013-10-11 13:00:50

On 11 Oct 2013 at 11:30, Gavin Lambert wrote:

> On 10/10/2013 3:07 PM, Quoth Roger Sanders:
> > The only compiler requirement here is that each assembly uses a compatible
> > C++ ABI, or at least, as much as is required to allow calling a virtual
> > member function on an object that was constructed within another assembly.
> 
> That could be a steep requirement, since AFAIK vtable location and 
> layout isn't standardised, especially once you start getting into 
> multiple and virtual inheritance.  And name mangling can get in the way 
> of finding the things to be called in the first place.

We had an internal prototype at a former employer of mine which 
interoped between the Itanium ABI (GCC) and MSVC as a thought 
exercise of just how bad things could get. Single inheritance vtables 
are actually identical between the two, but virtual inheritance is 
not so we didn't allow that, and multiple inheritance can be worked 
around with some difficulty (amongst the issues MSVC doesn't do empty 
base class optimisation at times, and persuading GCC to respect that 
is non-obvious). An interesting caveat is that x86 MSVC uses stdcall 
calling convention for member function calls, but x64 MSVC uses 
cdecl. Therefore on x86 the interop thunk has to flip the parameter 
stack and do callee cleanup. You also have to avoid all returns 
completely, because MSVC and GCC don't do the same thing, so 
returning via a pointer write is the only portable way.

Mangling was very straightforward: MSVC's is far more rich up to the 
point of needing a non-trivial tokenising parser, and everything the 
Itanium ABI specifies is easily representable in MSVC's, albeit 
sometimes not obviously (MSVC inverts what an array type means in 
mangling at times depending on context). For the other way round 
generating a link error where it can't interop is straightforward.

> And memory management is always entertaining given that all sorts of 
> weird and different allocators can be in use even before you introduce a 
> different compiler into the mix.  (Though this is where shared_ptr and 
> unique_ptr's pointer+deleter concept can get you out of a jam, although 
> it's more common to use opaque handles and explicit destroy functions.)

C libraries are actually somewhat binary interchangeable, at least 
more so than STLs. Local C library malloc was always used in our 
prototype.

> Having said that: if you can make it work, it would be awesome.

Getting the above system to work reliably needs a graph database to 
mark out what needs to be done with what, and what does work versus 
the majority which does not work. Such a graph database needs to work 
at the binary load layer, and therefore needs to be the filing system 
and little extra. For such a graph database to have any chance of 
performance, it needs async batch file i/o.

And hence me writing Boost.AFIO, with hopefully that thin graph 
database coming next. It'll take at least a year :)

Niall

-- 
Currently unemployed and looking for work.
Work Portfolio: http://careers.stackoverflow.com/nialldouglas/