[Boost-users] New library for RPC-calls using only boost and C++0x

Subject: [Boost-users] New library for RPC-calls using only boost and C++0x
From: Siegfried Kettlitz (siegfried.kettlitz_at_[hidden])
Date: 2009-06-11 14:27:21

Hello boost users and developers,

i'm currently writing a library for making remote procedure calls to
C++ objects. The library is based on the variadic templates extension
coming with C++0x (already present in GCC 4.3) in conjunction with the
boost library (threads, serialization, asio). The aim is to have a
library, that is easy to integrate and doesn't require additional
tools. Writing it, so far, has thaught me a lot about C++, templates,
variadic templates, threading, networking etc..

I post this message, because i think that this library is quite handy
in situations, where a developer already uses boost libraries and
wants to add some networking functionality to his/her project and the
usage of variadic templates instead of additional tools is quite new.
One example would be remote monitoring an application by adding a
"std::string get_status()"-function to some object. Implemented as
example is a chat-system with a server and clients interacting by
remote calls. This example is tested with a x64-linux and win32 using
the portable boost-text-serialization. Multiple clients can connect to
the server via the TCP/IP-acceptor (and disconnect without crashing
the system). Implemented is a console-client for text input and
display and a morse client, that interacts via a digital output device
(relais card), which is connected via a serial port. The serial port
and the digital output driver are both also connected via the
rpc-system and can be physically located on separate computers.

At this point, i have the basic bits together and a working
rpc-library, that needs to be cleaned, refined and extended to match
the practical needs. This means, that some programs and libraries have
to be written, which use the rpc-library and show its bugs and
limitations. The downside of it is, that the interface of the
rpc-library may change one more time until it can be considered
stable. Therefore it would not be good to test it widely in libraries
posing similar requirements, but to test it with a variety of possible
applications and provide the implementations as examples and
(regression-/unit-)tests.

- One library for Measurement/Automation is already in development.
- The chat-server/client exists.
- A transaction based game server is under consideration.
- Some distributed parallel computing application with dynamic
joining/leaving of clients.

Can you give me some more ideas of different applications, which are
easy to implement but yet useful?
What would you like to have implemented and accessible by an rpc-library?

One speed measurement using local (shared memory) channels resulted in
20000 calls/s and 60000 messages/s on a 1.8GHz Opteron. Do you know
how this compares to other implementations on similar hardware?

The TCP/IP-channel implementation using ASIO resulted in only about
100 synchronous calls/second regardless of the payload and almost no
CPU load, which suggests that ASIO somewhere "sleeps" on linux systems
for a few milliseconds. This would be fixed in some future
implementation. Using asynchronous calls (1000 open calls) again leads
to about 10000 calls/s.

I'll have a closer look on what can be merged with "channels" from the
boost vault[1], but from the first look it seems that "channels"
targets a much more general approach. After the first tries, i
realized, that my implemention has to be very specific (e.g. templates
only where necessary). So, the design approaches might be too
different.

After the refinement, cleanup, some testing and documenting, i'd like
to publish the rpc-library (and libraries based on it) under GPL- or
boost-license. Then you'll hear from me again. In future i'll probably
work 2-8 hours per week on the library and use it in some projects,
but there's plenty of work left to do in various regions. This will
leave space for other developers who might want to use and modify it
to fit their needs. Some of the harder parts that would need
implementation are redundant parallel connections and advanced
routing, authentication of nodes, signing of messages, encryption of
payload, access restriction to objects.

Please let me know if
* you are interested in this library in some way
* have comments
* know very similar libraries or
* are interested in serialization using variadic function templates.

Regards,
Siegfried Kettlitz

---
A short introduction on the library:
* "nodes" are the central parts. They contain a factory for objects,
store objects and forward "messages" to objects or other nodes in the
network and contain a "scheduler" for executing function calls. The
nodes are connected by "channels", which forward messages between the
two connected nodes. Currently there is a TCP/IPv4 implementation
using ASIO and direct forwarding.
* Nodes connected by channels form a network, create and monitor
virtual connections between each other. The network map is used to
route messages between connected nodes.
* An object has to supply a template for a class name (only if it can
be constructed by the factory) and a template for registering the
callable functions (also for abstract base classes which can't be
constructed).
* A function call is executed asynchronously (wrapper for synchronous
call exists). The call to node.call(...) returns a
"link_caller"-object.
The function to be executed itself can optionally have "link_callee"
as the first argument (which is invisible for the caller). The
"link_caller"/"link_callee" is used for synchronization (wait for call
to begin/finish execution), message forwarding (push/pop strings at
caller/callee) and transfer/extraction of the return value.
* Objects of class hierarchies with abstract base classes are possible
and handled transparently.
* Objects can be constructed by the factory of a node or an existing
(externally constructed) object can be registered by the node.
* Objects don't necessarily need to know (be modified, inherit
something etc.) that they are part of the RPC-system and some of their
functions are called remotely.
* Objects can be aware that they are constructed by the factory and
can, on construction be informed about their identity and get a
reference to the node to make remote calls themselves.
* An object is identified by a "reference" consisting of the "node_id"
and the "object_id".
* Parameters are passed (serialized, deserialized) by value. All
function parameters must have the possibility to be serialized.
Modifying referenced parameters won't affect the parameter at the
caller. Pointers obviously won't work between nodes not sharing the
same memory-address-space.
* Called functions can use "link_callee" to make remote calls
themselves, regardless of the object.
* Priorities are assigned to calls and affect message forwarding and scheduling.
* Calls to objects can be transformed into a boost::function using
node.bind(...).
* If a called function throws, the exception is forwarded as
rpc::exception or a more specialized exception class. The exception is
thrown at the caller, when waiting for interaction with the callee
(e.g. wait for return value).
* Template functions as targets are possible when specifying the
template parameters of the function pointer when registering the
function.
* There is no need to use macros at all, although macros could be used
to simplify some things.
----------
// Example for creating and calling an object.
    // Create a service.
    boost::asio::io_service node_io_service;
    // Create a node.
    rpc::node node_s(node_io_service, config);
    // Register the builder in order to build the server object later.
    node_s.register_builder<devicelib::chat_server_impl>();
    // Register the interface of the client to be able to make calls to it.
    node_s.register_interface<devicelib::chat_client>();
    // Create the server object.
    rpc::reference cs( node_s.get_id(),
node_s.allocate_object_template<devicelib::chat_server_impl>() );
    // Store the server in name server.
    // rpc::reference( node_s.get_id(),
rpc::oid_simple_object_name_server ) -> A reference to the
object_name_server.
    // rpc::sched_default() -> Default scheduling parameters.
    // &rpc::simple_object_name_server::store, cs -> function to call.
    // cs, "chat_server_object" -> function parameters.
    node_s.wrap_call( rpc::reference( node_s.get_id(),
rpc::oid_simple_object_name_server ), rpc::sched_default(),
&rpc::simple_object_name_server::store, cs, "chat_server_object" );
----------
// Example for abstract base class.
namespace devicelib {
class chat_server : public chat_types
{
public:
    virtual void receive( std::string xml_message ) = 0;
    virtual user_id_t connect_user( rpc::reference a_client,
std::string xml_user_info ) = 0;
    virtual std::string query_user( user_id_t id ) = 0;
    virtual void disconnect_user( user_id_t user ) = 0;
    virtual void join_room( user_id_t user, room_id_t room ) = 0;
    virtual std::list<user_id_t> list_users( ) = 0;
    virtual std::list<room_id_t> list_rooms( ) = 0;
};
}
---------------
// Example for registering the interface of the server object.
// The template is declared in namespace rpc, therefore we need to
define it there.
namespace rpc {
// For the base object.
    template<>
    void register_proxy_functions<devicelib::chat_server>(
function_call_proxy<devicelib::chat_server>& ar_proxy )
    {
        // Register the functions of the abstract base object using
unique strings.
        ar_proxy.register_interface(
&devicelib::chat_server::connect_user, "&chat_server::connect_user" );
        ar_proxy.register_interface(
&devicelib::chat_server::query_user, "&chat_server::query_user" );
        ar_proxy.register_interface(
&devicelib::chat_server::disconnect_user,
"&chat_server::disconnect_user" );
        ar_proxy.register_interface(
&devicelib::chat_server::join_room, "&chat_server::join_room" );
        ar_proxy.register_interface(
&devicelib::chat_server::list_users, "&chat_server::list_users" );
        ar_proxy.register_interface(
&devicelib::chat_server::list_rooms, "&chat_server::list_rooms" );
    }
// For each implementation.
    template<>
    void register_proxy_functions<devicelib::chat_server_impl>(
function_call_proxy<devicelib::chat_server_impl>& ar_proxy )
    {
        // Register the functions of the base object.
        register_proxy_functions<devicelib::chat_server>(
(function_call_proxy<devicelib::chat_server>&) ar_proxy );
        // Make the functions of this object available.
        ar_proxy.register_interface(
&devicelib::chat_server_impl::connect_user,
&devicelib::chat_server::connect_user );
        ar_proxy.register_interface(
&devicelib::chat_server_impl::disconnect_user,
&devicelib::chat_server::disconnect_user );
        ar_proxy.register_interface(
&devicelib::chat_server_impl::join_room,
&devicelib::chat_server::join_room );
        ar_proxy.register_interface(
&devicelib::chat_server_impl::list_users,
&devicelib::chat_server::list_users );
        ar_proxy.register_interface(
&devicelib::chat_server_impl::list_rooms,
&devicelib::chat_server::list_rooms );
    }
// The implementation can be built, the base object not.
    template<>
    std::string class_builder_name<devicelib::chat_server_impl>() {
        // Give it an unique name here.
        return "chat_server_impl";
    }
}
-----------------
1: http://www.boostpro.com/vault/index.php?&direction=0&order=&directory=Distributed%20Computing