Re: [Boost-users] New library for RPC-calls using only boost and C++0x

Subject: Re: [Boost-users] New library for RPC-calls using only boost and C++0x
From: Eric Woodruff (eric.woodruff_at_[hidden])
Date: 2009-10-13 18:35:43

Seigfried,

Around the same time your message was posted, I implemented an RPC
library on top of C++98 using boost serialization that is communications
medium neutral (data can be sent/received with a simple subclass) and
supports:

* callback and blocking calls
* deferred completion
* exceptions

It also requires no macros by the user. RPC function
signatures/prototypes are defined by a simple instance of a template.
boost::bind is used to implement an RPC.

There would have to be some interest in it before HP would let me
release it.

Thanks,

Eric

Siegfried Kettlitz wrote:
> Hello boost users and developers,
> 
> i'm currently writing a library for making remote procedure calls to
> C++ objects. The library is based on the variadic templates extension
> coming with C++0x (already present in GCC 4.3) in conjunction with the
> boost library (threads, serialization, asio). The aim is to have a
> library, that is easy to integrate and doesn't require additional
> tools. Writing it, so far, has thaught me a lot about C++, templates,
> variadic templates, threading, networking etc..
> 
> I post this message, because i think that this library is quite handy
> in situations, where a developer already uses boost libraries and
> wants to add some networking functionality to his/her project and the
> usage of variadic templates instead of additional tools is quite new.
> One example would be remote monitoring an application by adding a
> "std::string get_status()"-function to some object. Implemented as
> example is a chat-system with a server and clients interacting by
> remote calls. This example is tested with a x64-linux and win32 using
> the portable boost-text-serialization. Multiple clients can connect to
> the server via the TCP/IP-acceptor (and disconnect without crashing
> the system). Implemented is a console-client for text input and
> display and a morse client, that interacts via a digital output device
> (relais card), which is connected via a serial port. The serial port
> and the digital output driver are both also connected via the
> rpc-system and can be physically located on separate computers.
> 
> At this point, i have the basic bits together and a working
> rpc-library, that needs to be cleaned, refined and extended to match
> the practical needs. This means, that some programs and libraries have
> to be written, which use the rpc-library and show its bugs and
> limitations. The downside of it is, that the interface of the
> rpc-library may change one more time until it can be considered
> stable. Therefore it would not be good to test it widely in libraries
> posing similar requirements, but to test it with a variety of possible
> applications and provide the implementations as examples and
> (regression-/unit-)tests.
> 
> - One library for Measurement/Automation is already in development.
> - The chat-server/client exists.
> - A transaction based game server is under consideration.
> - Some distributed parallel computing application with dynamic
> joining/leaving of clients.
> 
> Can you give me some more ideas of different applications, which are
> easy to implement but yet useful?
> What would you like to have implemented and accessible by an rpc-library?
> 
> One speed measurement using local (shared memory) channels resulted in
> 20000 calls/s and 60000 messages/s on a 1.8GHz Opteron. Do you know
> how this compares to other implementations on similar hardware?
> 
> The TCP/IP-channel implementation using ASIO resulted in only about
> 100 synchronous calls/second regardless of the payload and almost no
> CPU load, which suggests that ASIO somewhere "sleeps" on linux systems
> for a few milliseconds. This would be fixed in some future
> implementation. Using asynchronous calls (1000 open calls) again leads
> to about 10000 calls/s.
> 
> I'll have a closer look on what can be merged with "channels" from the
> boost vault[1], but from the first look it seems that "channels"
> targets a much more general approach. After the first tries, i
> realized, that my implemention has to be very specific (e.g. templates
> only where necessary). So, the design approaches might be too
> different.
> 
> After the refinement, cleanup, some testing and documenting, i'd like
> to publish the rpc-library (and libraries based on it) under GPL- or
> boost-license. Then you'll hear from me again. In future i'll probably
> work 2-8 hours per week on the library and use it in some projects,
> but there's plenty of work left to do in various regions. This will
> leave space for other developers who might want to use and modify it
> to fit their needs. Some of the harder parts that would need
> implementation are redundant parallel connections and advanced
> routing, authentication of nodes, signing of messages, encryption of
> payload, access restriction to objects.
> 
> Please let me know if
> * you are interested in this library in some way
> * have comments
> * know very similar libraries or
> * are interested in serialization using variadic function templates.
> 
> Regards,
> Siegfried Kettlitz
> 
> ---
> 
> A short introduction on the library:
> 
> * "nodes" are the central parts. They contain a factory for objects,
> store objects and forward "messages" to objects or other nodes in the
> network and contain a "scheduler" for executing function calls. The
> nodes are connected by "channels", which forward messages between the
> two connected nodes. Currently there is a TCP/IPv4 implementation
> using ASIO and direct forwarding.
> 
> * Nodes connected by channels form a network, create and monitor
> virtual connections between each other. The network map is used to
> route messages between connected nodes.
> 
> * An object has to supply a template for a class name (only if it can
> be constructed by the factory) and a template for registering the
> callable functions (also for abstract base classes which can't be
> constructed).
> 
> * A function call is executed asynchronously (wrapper for synchronous
> call exists). The call to node.call(...) returns a
> "link_caller"-object.
> The function to be executed itself can optionally have "link_callee"
> as the first argument (which is invisible for the caller). The
> "link_caller"/"link_callee" is used for synchronization (wait for call
> to begin/finish execution), message forwarding (push/pop strings at
> caller/callee) and transfer/extraction of the return value.
> 
> * Objects of class hierarchies with abstract base classes are possible
> and handled transparently.
> 
> * Objects can be constructed by the factory of a node or an existing
> (externally constructed) object can be registered by the node.
> 
> * Objects don't necessarily need to know (be modified, inherit
> something etc.) that they are part of the RPC-system and some of their
> functions are called remotely.
> 
> * Objects can be aware that they are constructed by the factory and
> can, on construction be informed about their identity and get a
> reference to the node to make remote calls themselves.
> 
> * An object is identified by a "reference" consisting of the "node_id"
> and the "object_id".
> 
> * Parameters are passed (serialized, deserialized) by value. All
> function parameters must have the possibility to be serialized.
> Modifying referenced parameters won't affect the parameter at the
> caller. Pointers obviously won't work between nodes not sharing the
> same memory-address-space.
> 
> * Called functions can use "link_callee" to make remote calls
> themselves, regardless of the object.
> 
> * Priorities are assigned to calls and affect message forwarding and scheduling.
> 
> * Calls to objects can be transformed into a boost::function using
> node.bind(...).
> 
> * If a called function throws, the exception is forwarded as
> rpc::exception or a more specialized exception class. The exception is
> thrown at the caller, when waiting for interaction with the callee
> (e.g. wait for return value).
> 
> * Template functions as targets are possible when specifying the
> template parameters of the function pointer when registering the
> function.
> 
> * There is no need to use macros at all, although macros could be used
> to simplify some things.
> 
> 
> ----------
> 
> // Example for creating and calling an object.
> 
>     // Create a service.
>     boost::asio::io_service node_io_service;
> 
>     // Create a node.
>     rpc::node node_s(node_io_service, config);
> 
>     // Register the builder in order to build the server object later.
>     node_s.register_builder<devicelib::chat_server_impl>();
> 
>     // Register the interface of the client to be able to make calls to it.
>     node_s.register_interface<devicelib::chat_client>();
> 
>     // Create the server object.
>     rpc::reference cs( node_s.get_id(),
> node_s.allocate_object_template<devicelib::chat_server_impl>() );
> 
>     // Store the server in name server.
>     // rpc::reference( node_s.get_id(),
> rpc::oid_simple_object_name_server ) -> A reference to the
> object_name_server.
>     // rpc::sched_default() -> Default scheduling parameters.
>     // &rpc::simple_object_name_server::store, cs -> function to call.
>     // cs, "chat_server_object" -> function parameters.
>     node_s.wrap_call( rpc::reference( node_s.get_id(),
> rpc::oid_simple_object_name_server ), rpc::sched_default(),
> &rpc::simple_object_name_server::store, cs, "chat_server_object" );
> 
> 
> ----------
> 
> // Example for abstract base class.
> namespace devicelib {
> class chat_server : public chat_types
> {
> public:
> 
> 
>     virtual void receive( std::string xml_message ) = 0;
> 
>     virtual user_id_t connect_user( rpc::reference a_client,
> std::string xml_user_info ) = 0;
>     virtual std::string query_user( user_id_t id ) = 0;
>     virtual void disconnect_user( user_id_t user ) = 0;
> 
>     virtual void join_room( user_id_t user, room_id_t room ) = 0;
> 
> 
>     virtual std::list<user_id_t> list_users( ) = 0;
>     virtual std::list<room_id_t> list_rooms( ) = 0;
> 
> };
> }
> 
> ---------------
> 
> // Example for registering the interface of the server object.
> 
> // The template is declared in namespace rpc, therefore we need to
> define it there.
> namespace rpc {
> 
> // For the base object.
>     template<>
>     void register_proxy_functions<devicelib::chat_server>(
> function_call_proxy<devicelib::chat_server>& ar_proxy )
>     {
>         // Register the functions of the abstract base object using
> unique strings.
>         ar_proxy.register_interface(
> &devicelib::chat_server::connect_user, "&chat_server::connect_user" );
>         ar_proxy.register_interface(
> &devicelib::chat_server::query_user, "&chat_server::query_user" );
>         ar_proxy.register_interface(
> &devicelib::chat_server::disconnect_user,
> "&chat_server::disconnect_user" );
>         ar_proxy.register_interface(
> &devicelib::chat_server::join_room, "&chat_server::join_room" );
>         ar_proxy.register_interface(
> &devicelib::chat_server::list_users, "&chat_server::list_users" );
>         ar_proxy.register_interface(
> &devicelib::chat_server::list_rooms, "&chat_server::list_rooms" );
>     }
> 
> // For each implementation.
>     template<>
>     void register_proxy_functions<devicelib::chat_server_impl>(
> function_call_proxy<devicelib::chat_server_impl>& ar_proxy )
>     {
>         // Register the functions of the base object.
>         register_proxy_functions<devicelib::chat_server>(
> (function_call_proxy<devicelib::chat_server>&) ar_proxy );
>         // Make the functions of this object available.
>         ar_proxy.register_interface(
> &devicelib::chat_server_impl::connect_user,
> &devicelib::chat_server::connect_user );
>         ar_proxy.register_interface(
> &devicelib::chat_server_impl::disconnect_user,
> &devicelib::chat_server::disconnect_user );
>         ar_proxy.register_interface(
> &devicelib::chat_server_impl::join_room,
> &devicelib::chat_server::join_room );
>         ar_proxy.register_interface(
> &devicelib::chat_server_impl::list_users,
> &devicelib::chat_server::list_users );
>         ar_proxy.register_interface(
> &devicelib::chat_server_impl::list_rooms,
> &devicelib::chat_server::list_rooms );
>     }
> 
> // The implementation can be built, the base object not.
>     template<>
>     std::string class_builder_name<devicelib::chat_server_impl>() {
>         // Give it an unique name here.
>         return "chat_server_impl";
>     }
> }
> 
> -----------------
> 
> 
> 1: http://www.boostpro.com/vault/index.php?&direction=0&order=&directory=Distributed%20Computing