Re: [boost] [RPC] Compile-time reflection for RPC and mocking

Subject: Re: [boost] [RPC] Compile-time reflection for RPC and mocking
From: Hartmut Kaiser (hartmut.kaiser_at_[hidden])
Date: 2014-12-10 15:36:09

> I've searched through the Boost mailing list history to find related
> emails
> and the last I found was from January where Oleg Labutin and Hartmut
> Kaiser
> discussed compile-time reflection in the context of RPC mechanisms.
> 
> Based on Hartmut's speech last Saturday introducing HPX and explaining a
> bit about async/future/promise I tried to implement a simple RPC mechanism
> that uses then to decouple work and the result I have so far is very
> promising. I have a simple interface with a call that returns a future,
> and
> the caller does not distinguish between calling the actual object and
> calling on a proxy, as both return a future<T>. It does what the
> discussion
> mentions - serialize the name of the interface and function, as well as
> the
> arguments (basically the mangled name of the interface function) to ensure
> there are no mismatches.
> 
> I would like to take this oppurtunity to propose two things:
> 
> - I can work on this idea to work it out into a full RPC mechanism, to be
> proposed for Boost.RPC, if there is enough interest. Currently it uses
> macros to generate and wrap functions, and a simple serializer that
> serializes in the same way that Protobuf and Thrift also serialize.
> - Given a compile-time reflection method, the entire proxy class and
> dispatch class can be auto-generated from the interface class - the full
> code for registering an interface with the RPC mechanism would collapse to
> RPC::Register<T>(); To do so, it would need two language additions. It
> needs access to a compile-time iterator over functions and traits about
> the
> argument types, return type, name and whether it is a virtual function.
> For
> the prototype this can be manually generated for an interface to show it
> works. Additionally, it would need to be able to use such a definition to
> instantiate a virtual function from a base class - basically, hooking up
> an
> std::function<X> to a virtual function X directly. I think this is
> well-implementable (probably reusing most code from lambdas for the link
> code), but I don't know whether that's true.
> 
> On the second note, I also proposed a Mock library a few years ago. This
> basically fell flat on the code that I suggested being very unsafe and
> using many hackish methods to mock functions. Using the compile-time
> reflection proposed for the second bullet I would be able to replace the
> entire hackishness from that with 100% compliant code, with only one minor
> regression in functionality. That should make it Boost-OK at least from a
> language standards perspective.
> 
> Any ideas? Reactions on RPC or Mocking?

In our experience, there is no need for having yet another special
reflection solution if you use Boost.Serialization (or similar). However,
those serialization libraries use some kind of reflection internally. 

Since you mentioned HPX, here is how we implemented the generic RPC solution
we have. In essence, in order to marshal a function to another machine
(which can't be done directly, we let the user create a special unique type
representing the function. This type is then serialized (together with the
function arguments). This allows to invoke the function after
de-serialization on the receiving end. Here is a sketch of what's going on:

    template <typename F, F* f>
    struct action;

    template <typename R, typename F, typename... Args>
    struct action <R F(Args...), F* f>
    {
        tuple<Args...> arguments;

        // construction, serialization, etc.

        // this is called on the receiving end
        R invoke() const
        {
            return invoke_fused(f, arguments);
        }
    };

    int foo(double) { return 42; }

    typedef action<decltype(&foo), &foo> foo_action;

Now, 'foo_action' can be used to invoke 'foo' on a destination machine:

    future<int> f = async(foo_action(), target_address, 42.0);

where async() is a special overload implementing the actual serialization of
foo_action and the arguments, and 'target_address' represents the node where
foo() should be invoked on. Note that the return value is sent back using a
similar mechanism targeting the promise instance which originally generated
the future<int> we got back from our async.

The benefit of this is that you can wrap any existing function without
changing it by just adding the typedef defining the needed action. In HPX
this is done using a macro:

HPX_ACTION(foo); // defines 'foo_action'

HTH
Regards Hartmut
---------------
http://boost-spirit.com
http://stellar.cct.lsu.edu