Re: [boost] Marshalling library

From: Hans Larsen (hans_at_[hidden])
Date: 2007-04-17 01:42:23

On 17-Apr-07, at 1:07 AM, Stjepan Rajko wrote:

> Hi Scott,
>
> On 4/16/07, Scott Woods <scottw_at_[hidden]> wrote:
>>
>> Just a bullet-point review of your example of marshalling in case
>> I'm missing something;
>> * a register of return type+parameter list, keyed on small, unique  
>> integer,
>> * generate portable image (string) of a call, i.e. "marshall",
>> * call the registered function using one of several available  
>> conventions.
>
> Yep, you got it.   That's it so far :-)  The key is templated so
> theoretically it could be something other than an int (I used int
> because that was in the original example by Hans).

I like the idea of having a type as an identity mean.  We'll have to  
make it type-safe during serialization though, but it's a good idea.

>
>>>
>>> In any case, I'd be interested in getting some marshalling into  
>>> boost
>>> because that allows for distributed signal networks that can not  
>>> only
>>> pass signals from one computer to another but also execute RPCs.
>>
>> That would be inherent, i.e. the ability to execute RPCs is a freebie
>> once distributed signaling is working? I dont mean literally, more  
>> that
>> an RPC would be achieved through an exchange of a particular set
>> of signals.
>
> Yeah, what I have right now in the signal network library prototype is
> a little pair of objects which serve as a trans-socket signal.  One of
> them receives a signal / function call locally, serializes the
> arguments via Boost.Serialization into a string, and sends the string
> through an asio socket.  On the other end, the other object receives
> the packet, de-serializes it, and generates an identical
> signal/function call on that side.

That's exactly what I had in mind.

>
> Marshalling would work the same way except the packet would also carry
> the function id/key, and on the other end a function would be called
> from a function registry.
>
> Neither of these mechanisms handle returning of the data from the
> function call right now, and even if they did I'm sure there's a lot
> of other marshalling/RPC issues to be considered...  not an expert.

Basically, I see marshalling in current implementations as a four  
steps process (although it's more like 2 step mirrored...):
1. The call and all its parameter is made into a structured sequence  
of symbols (serializing into binary or XML) so that it can be  
reconstructed (basically, a function [F: Any -> Something] which can  
be reversed).  This may imply a way to identify the function to call  
(in a type-safe manner[1]).
2. The sequence is sent over a stream of communication.[3]
3. The call is made, and the return parameters (function return  
considered as a parameter of type reference in this case) are  
serialized again. [2]
4. The return parameters are sent back over the stream of  
communication and back to the callee.  It may really be interesting  
if objects were deleted during the calls.  Also, thread safety has a  
funny eerie aura around those things...

[1] The problem is much more complex than it seems, and may be left  
to the user: how can we be sure that function X will be called if we  
ask to call function X.  Registering a function does not ensure type- 
safety over two different codes.  Maybe we'll need to identify the  
types further, but how can we put an id on type "CEmployee"...?  BTW,  
basically nothing prevents a COM object from being mis-called if  
interface and GUID doesn't concur, meaning you may register a GUID  
but use it with an I... different, and I think CORBA has same  
problem.  We may want to say it's up to user to register functions  
correctly.
[2] We'll have to find a way to identify in, out and in-out  
parameters (return parameter included).  I'm working on a way right now.
[3] Network security in mind.  I've seen enough SAMBA and WinDCE buff  
overflow vulnerability to make me "vomit in my own mouth".

What I offer to solve in a short term is only point 1.  Point 2  
require at least a way to communication over a network (using  
Signals) or inter-processes (I read once about Boost.Interprocess...  
what's the status?).  Once a serialization is made, sending it over a  
stream is not really difficult.  Waiting the return value is not  
necessarly a problem.  Basically, if we can save the marshall and  
load it then call it from another executable, then the networking or  
interprocessing is of little challenge.

Example of the kind of nasty case we're facing:

char* func0( int& i, int n )
{
        i = i + n;
        return "Hello World!";
}

int main() {
        marshall::registry<int> reg;
        reg.set< char*(int&, int) >(0, func0);

        int k = 8;
        // Putting the call id aside from param list may be a good way to  
improve readability.
        // !!! {a}
        marshall::call c1 = reg[0]( k, 1 );
        // This should print "Hello World!9\n"...
        cout << c1() << k << endl;
        // !!! {b}

        return 0;
}

Basically this is a two step process (the call is not actually made  
when we create the variable, but only when we call it).  Final  
solution may be more along the lines of:
        // !!! {a}
        marshall::call c1 = reg[0];
        // This should print "Hello World!9\n"...
        cout << c1(k,1) << k << endl;
        // !!! {b}
Then c1(...) will then send over network, wait for answer, et cetera.

I will put together some code this week, so we can have a prototype  
started.
Hans

PS: No use reinventing the wheel, I will check around CORBA for the  
network communication part.  I was more thinking along the "serialize  
params and returns" as a simple step toward the final solution than  
the whole yard all at once.  We might use their (CORBA) protocol if  
it's perfect for our needs.  We might even want to template the  
network protocol so that we can call Web Services instead...  I'm  
still dreaming :)

> Stjepan
> _______________________________________________
> Unsubscribe & other changes: http://lists.boost.org/mailman/ 
> listinfo.cgi/boost