Re: [boost] Interest in Remote Procedure Call Library?

Subject: Re: [boost] Interest in Remote Procedure Call Library?
From: Daniel Larimer (dlarimer_at_[hidden])
Date: 2010-02-07 20:26:27

Overmind,
        Thanks for offering your code.  I can appreciate the desire not to use macros, but I believe that carefully used they can make a world of difference to the end user and eliminate many kinds of "typo bugs" that don't show up until runtime. 

Since I am new to this list, I will try not to foist to much of my "world view" on how an API should be developed, but I firmly believe that syntax matters and that the end user (average C++ college grads who know little of templates, member function pointers, etc) can use the API.  At my company I am the resident expert in C++ and working with a bunch of mechanical engineers whom I have been teaching C++ "on the job".  So I prefer the macros.  If there is some amazing way to achieve the same without macros then I will support it.  But nothing keeps the reflection synchronized with method names better than a macro. 

Certainty the macro's should just build upon the "real API" and thus there should be some considerable effort made to make the "real api" as friendly as possible.  Unfortunately, I am not as skilled with the pre-processor as I would like to be and so often my limited ability to manipulate tokens with the pre-processor is compensated for by code structure.

Perhaps the discussion on what should be in an RPC library should begin with requirements.

1) Minimize code/syntax required to expose a class to the "transport"
2) Ability to expose any number of methods (overloaded or not)
3) Fully support multiple/virtual inheritance of interfaces
4) Support Boost.Signals
5) Support Boost.Serialization
        - on this note, I have some concern over performance and "predictability" for interfacing with other code not using Boost.Serialization.
6) Ability to implement any "protocol" (TCP/UDP/SharedMemory/SOAP/etc)
7) Support for exceptions
        - this requires a factory that is able to dynamically create the exception type from serialized data.  Normal return values do not need this because it is known at compile time what the serialization of the return type is; however, with exceptions it could be "anything".  For this I have used boost::any combined with a 32 bit hash of the class name.  
8) Some kind of minimum performance metric? In my opinion the library should perform on par with the best RPC libraries available only provide a simple, cross-platofrm, c++ native solution and not depend upon code generation.

As an added bonus, the same system should work to expose any object to any scripting engine because all of the required meta-information would be available.  

If anyone else on this list has ideas on what kind of requirements / API (macros/no macros) is best for RPC please chime in.

In reality there are two libraries here.  One for RPC and one for "reflection" and creation of
proxy objects.  The reflection library could use some of the features of Boost.Mirror instead of its own mechanism for getting the name of a type:

TYPESTRING(type)   // register the type in a header
TypeString<type>::str()  // get the const char* string "type" 
TypeString<type*>::str()  // get the const char* string "type*" etc....

Clearly the idea of a reflection library in boost is not a new one.  What are the main drivers that have prevented any of the proposed reflection techniques from being adopted?   Is there anything I am doing / proposing that is clearly taboo for boost libraries?

Dan 

On Feb 7, 2010, at 6:52 PM, OvermindDL1 wrote:

> On Sun, Feb 7, 2010 at 1:08 PM, vicente.botet <vicente.botet_at_[hidden]> wrote:
>> /* snip RPC stuff */
> 
> For note, I created an RPC system a while ago (of which I donated some
> of the code to RakNet, but my version is still more complete).  It
> uses no macros, it handles overloads fine, and it was to be plugged
> into any system as well (I had a networking interface, as well as an
> interface into a little custom scripting language).  It was used like
> this:
> 
> 
> // Some functions and member function to link in
> void _myFunc(someClass *m1, anotherClass &m2, yetAnotherClass m3, int
> i, float f, RPCInfo *rpc, std::string s)
> {
>    /* do stuff */
> }
> 
> class myClass
> {
>    public:
>    void _myMemberFunc(int i, float f);
>    static RPCHandler::type_of_callback<myClass::_myMemberFunc>::type
> myMemberFunc;
> }
> 
> // You "register" it to the RPC system like this:
> RPCHandler global_rpc;
> 
> auto myFunc = global_rpc.register(_myfunc, 0); // Either globally
> 
> void someFunctionSomewhere(RPCHandler &rpc)
> {
>    auto myLocalFunc = global_rpc.register(_myfunc1); // or locally
> 
>    // Registing a member function is the same way
>    myClass::myMemberFunc =
> global_rpc.register(myClass::_myMemberFunc, "theMemberFunc") // and
> yes, the second parameter, the id, can be an int or string, whichever
> is easier for your situation
> }
> 
> To use it you can set flags in the RPC class as to how to route that
> callback, whether it is local only, remote only, both, etc...
> And to use it, you just call the returned object from the register
> function as just any other function:
> 
> myFunc(m1,m2,m3,42,3.14,"Hello World");
> If that need to be called on a remote system, then it will be
> serialized up, sent out, and called remotely.  If it is supposed to be
> called locally, then it will just call it locally right then and
> there.
> 
> myClass m;
> myClass::myMemberFunc(m, 42, 3.14); // You would probably make a
> wrapped around this though...
> 
> 
> RPC calls can be registered at any time, and can be unregistered, and
> with bind they can be bound to specific objects and so forth as well.
> But the register call returns a specialized callback struct (that you
> can stuff into a boost::function if you do not care about overloading,
> or can be used as-is by the type_of_callback thing or auto).
> 
> I had talked about this on boost before, hinted at making it a
> library, but no interest was expressed.  I still have my code, it is
> well tested and in the wild (through RakNet), so if anyone is
> interested then I can clean it up and write documentation and submit
> it.
> 
> The way it works though is just very heavy use of Boost.Fusion and
> function traits and a few other things.  In the RakNet version I
> serialized things out to its bitstream, otherwise I use
> Boost.Serialization (which can be overridden to use yet another thing
> if you want, like I do for my scripting language version).  But if a
> parameter is by value, it just packages it up according to
> Boost.Serialization (or whatever you use internally).  If it is a
> reference or pointer it still follows the same serialization pattern,
> unless a few conditions are met.  If it is a pointer or reference and
> it is to an RPCInfo type, then it passes in information about the
> call, like if it remote, local, and a few other things.  In the RakNet
> version I added another overload so if a class was passed by
> pointer/reference and it was a subclass of NetworkIDObject, then
> instead of serializing the object it just sends its network ID number
> and looks up that same object on the remote system (perfect for *this
> and any other passed in parameters).  It is easy to add new overrides
> too, and it threw a compile error in a specific spot if the thing
> could not be serialized and/or did not match other things and various
> other conditions as well, all compile-time.
> 
> But yes, if anyone wants the code, I can easily give it, just no
> documentation and bit a messy, but it works quite well, no macros
> needed, no needing to register types, etc...
> 
> And as you may notice, mine will silently drop return values, it is
> asynchronous only, but functionality like futures or synchronized
> calls could easily be added (although I would prefer async futures or
> async callbacks personally).
> 
> And let me say, Boost.Fusion is so awesome!
> _______________________________________________
> Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost