Re: [Boost-users] New library for RPC-calls using only boost and C++0x

Subject: Re: [Boost-users] New library for RPC-calls using only boost and C++0x
From: OvermindDL1 (overminddl1_at_[hidden])
Date: 2009-06-19 23:02:30

On Fri, Jun 19, 2009 at 12:36 AM, Siegfried
Kettlitz<siegfried.kettlitz_at_[hidden]> wrote:
>> On the open source (but not free in cost for commercial app)
>
> Not being free to use it in a commercial application is kind of a no-go for
> many areas like in boost i think. Your library therefore needs to be
> independent of RakNet.

Only in that I gave him that code to use, I still have my own code.  I
altered it from a scripting language binding I created, it
automatically built the scripting links necessary just by passing the
function (kind of like Boost::Python, but a little more powerful).
The code itself to do the calls, if you ignore all the RakNet stuff
and the extra overloaded operators to handle things like the
NetworkIDObjects and so forth is actually very small, might fit on a
printed page actually.

So yes, I can give you code that is independent of RakNet, the code is
very simple though, but I can walk you through it if you wish, just
makes use of Boost.Fusion::invoke.

On Fri, Jun 19, 2009 at 12:36 AM, Siegfried
Kettlitz<siegfried.kettlitz_at_[hidden]> wrote:
>> networking library RakNet, I created a templated RPC interface for it.
>>  It is very efficient, type-safe on both side, and the external
>> interface is more simple then what you have above.  Quite literally
>> you just define you function, class member, whatever, then just
>> register it.
>
> I looked at the example in RakNet but didn't look at the source yet.
>
> How is this call made type-safe?
>
> rpc3Inst.CallCPP("&C::ClassMemberFunc", GetNetworkID(),
> a1,a2,c1,c2,RakNet::_RPC3::Deref(d1),d2,bs1,bs2,rpcFromNetwork);
>
> How do you avoid naming collisions with other namespaces?

When you register the call, if you have identically named functions
then just register it directly instead of using the macro, the only
thing the macro does is basically this (Rak'kar added the
'cpp/c/etc...' names, my system actually supports anything directly
with no difference, even functors, yes you can register functors):
#define REGISTER_RPC_CALL(func, rpc) rpc->RegisterCall(func, #func, 0);
Just make that call yourself and put a namespace specific name for the
#func param.   The flags I initially added to handle a vastly more
powerful system, which he never ended up using... >.<

The RegisterCall (not its actual name, but I do not have the code
available right now, at work) is just a templated function for the
func param (the #func is an std::string&) of boost::function, then
through use of boost::function_types and boost::fusion and an external
struct it verifies it returns void, it recurses through all the types
of that function parameters and creates a templated struct for each
one (that is optimized out in release mode, I checked the machine
code).  The templated structs can be overridden for detailed behavior
of specific types (like for pointers and references and such), but the
general case just operator>>/<<() the data with the Raknet::Bitstream
(or whatever you will use), which lets the user override how it
serializes the data as well.

If is very simple code though, as I recall, one of the examples in
boost::fusion for invoke is very close to what I did, might look at
that.

On Fri, Jun 19, 2009 at 12:36 AM, Siegfried
Kettlitz<siegfried.kettlitz_at_[hidden]> wrote:
>>  There is a helper thing to that it returns that you call
>> instead.  The tutorial video is at
>> http://www.jenkinssoftware.com/raknet/manual/RPC3Video.htm and if you
>> download it, it is RPC3.  I wrote that and donated it to his project.
>> I mostly ripped it out of an old script registration system (that
>> build up the variable conversion operations and called the native
>> C/C++ function as appropriate).  It makes heavy use of Boost.Fusion
>> (this was right when Fusion was added to Boost).  It made for a
>> wonderful and easy to use system.  The above tutorial mostly focuses
>> on features that his old assembly driven RPC system was not capable
>> of, but mine had more features that he never seems to use.  For
>> example, when you register a function you can store it in a
>> boost/tr1::function<> object, then if you call that instead it saves a
>> map lookup for the name, and based on the setup you set for it (only
>> call on client systems, only call on server, only call on owner,
>> etc...) it called everything appropriately, very powerful (I really
>> wish he used that feature, I do not like the fact he shows them how to
>> do it dynamically, hence the hashmap lookup...
>
> It looks like there's one object registration for the whole network in RPC3,
> while in my library every node has its own registrations because nodes can
> join and leave the network at every point in time. There isn't even "one
> network" because there are only real and virtual peer-to-peer-connections
> between nodes.
There is not one registration in RakNet.  He wanted the macros
introduced for ease-of-use, but if you do not use them then you can
even dynamically register and unregister things over the lifetime of
the system, and each node can have different registrations, but for
one node to send to another it needs to have the same function
signature (not necessarily the same function) registered at the same
name for serialization to work.  There can also be multiple RPC
instances, although not sure what that purpose would be...

On Fri, Jun 19, 2009 at 12:36 AM, Siegfried
Kettlitz<siegfried.kettlitz_at_[hidden]> wrote:
>> But yea, look at the tutorial video for the basic feature rundown (I
>> did not make that video, Rak'kar did).  Another feature mine has that
>> he does not display, if your function has a parameter of Raknet::RPC3*
>> then it auto-fills that in with the RPC3 registration object, letting
>> you get detailed information about your call like if it is a remote
>> call, local call, etc...
>
> This is analogous to the optional "link_callee&" reference in my library and
> it's really useful for functions aware of the remote calls.
Yes, provides a noticable speed boost over an unordered_map lookup, I
really wish he advertised that feature of it, but the video is more
geared toward new programmers rather then real programmers.

On Fri, Jun 19, 2009 at 12:36 AM, Siegfried
Kettlitz<siegfried.kettlitz_at_[hidden]> wrote:
>>  It also check for various class hierarchies,
>> such as if a NetworkIDObject is a subclass of any class pointer passed
>> in, it will use the corresponding registered NetworkIDObject on the
>> remote system and so forth.  Raknet::Bitstream is a bitstream as you
>> would expect (I really want a bitstream like Raknet::Bitstream in
>> boost, does anyone know if one already exists?  the only ones I have
>> found do not have the capabilities), and I had him add the operator
>> >>/<< functions so streaming can be overloaded.
>
> That's one advantage over my library, because so far, nothing other is done
> with the parameters but serializing/deserializing them to call the
> function. It's not aware of networking capable objects. And i haven't
> checked, how it would handle pointers to objects.
That is just one of the struct overloads (like with pointers and
such), it uses enable_if to enable and disable various specs depending
on if the class inherits from NetworkIDObject, can of course be
expanded.  NetworkIDObject is just a 64-bit integer that matches one
on another system for matching serialization calls between the objects
in RakNet.

On Fri, Jun 19, 2009 at 12:36 AM, Siegfried
Kettlitz<siegfried.kettlitz_at_[hidden]> wrote:
>> Either way, he changed the code slightly, added a few more default
>> overloads, etc... since I gave it to him a long while back (back in
>> Boost 1.35's time period, 1.35 was new), but take a look at the RPC3
>> code and see how it does it, actually very simple, very powerful,
>> type-safe, overloadability, etc...  I would be quite happy if my code
>> ended up in Boost.
>
> Is your library independent of RakNet? Can you put your code under the
> boost-license?
>
> When reading the mailing list, the process of putting a library into boost
> is rather time-consuming and can take years until everything works on all
> required compilers is well documented and in in harmony with the other boost
> libraries. From that point on, it requires constant attention to keep up
> with new compilers and changes in other boost libraries.

The code it came from depends on raknet, but the code itself only
requires changes on maybe 2 or 3 lines to support the serialization on
a different system type, and the code it came from, my script system,
is free for whatever use (just not released yet, but I would be quite
happy to generate code for your system).  What network archetecture
would I build it around, Boost::ASIO or something, or perhaps it would
be best to build it based on concepts so it is more generic?  I doubt
it is complete enough to ever be considered a standalone library in
Boost, maybe an example for Boost.ASIO or Boost.Fusion or something.

On Fri, Jun 19, 2009 at 12:36 AM, Siegfried
Kettlitz<siegfried.kettlitz_at_[hidden]> wrote:
>> But, thanks to fusion and such, does not require variadic templated
>> (although tr1::function is the main restriction, if it uses variadic
>> templates then there is no real limit to my system then).
>
> Not being forced to require variadic templates indeed is a big plus for the
> next 3 years until all common complilers understand them correctly. Until
> then i'll be mostly restricted to GCC.

Yea, such things did not exist a couple years ago when I made this, so
it definitely works without it.  And for note, my main compiler is
Visual Studio 2005, so I could not use yours either.  :)