Subject: Re: [boost] [RPC] Compile-time reflection for RPC and mocking
From: Thomas Heller (thom.heller_at_[hidden])
Date: 2014-12-09 12:18:48
On Monday, December 08, 2014 15:27:33 Peter Bindels wrote:
> Hi all,
> I've searched through the Boost mailing list history to find related emails
> and the last I found was from January where Oleg Labutin and Hartmut Kaiser
> discussed compile-time reflection in the context of RPC mechanisms.
> Based on Hartmut's speech last Saturday introducing HPX and explaining a
> bit about async/future/promise I tried to implement a simple RPC mechanism
> that uses then to decouple work and the result I have so far is very
> promising. I have a simple interface with a call that returns a future, and
> the caller does not distinguish between calling the actual object and
> calling on a proxy, as both return a future<T>.
(This is one of the HPX developers speaking)
Please bear with me, I didn't attend meetingcpp and don't know which questions
have been answered already or not. I just would like to add my two cents here.
The biggest question I currently have here is: How do you distinguish if it is
remote or local?
The HPX answer to that problem is a distributed database which is responsible
for mapping Global Identifiers (GIDs) to actual objects. We call that the
Active Global Address Service (AGAS). The GID therefor acts as your proxy
which is, in HPX, kinda like a void pointer to a possibly remote object where
you can invoke so called actions on. Actions our mechanisms to do RPC, more on
By meaning "return a future<T>", do you refer to std::future<T> or your own
class behaving like a future<T>?
> It does what the discussion
> mentions - serialize the name of the interface and function, as well as the
> arguments (basically the mangled name of the interface function) to ensure
> there are no mismatches.
You say "basically the mangled name", what does that mean exactly? How do you
get unique names?
> I would like to take this oppurtunity to propose two things:
> - I can work on this idea to work it out into a full RPC mechanism, to be
> proposed for Boost.RPC, if there is enough interest. Currently it uses
> macros to generate and wrap functions, and a simple serializer that
> serializes in the same way that Protobuf and Thrift also serialize.
As has been noted in a different mail in that thread, boost already has a
solution for that: Boost.Serialization. We currently use it but were not very
happy with its performance (and still aren't), but that's a different issue ...
I think the interface that Boost.Serialization exports is great to implement
compile time reflection without compiler support. It has its caveats, but it is
a decent, and accepted way of accomplishing this task.
> - Given a compile-time reflection method, the entire proxy class and
> dispatch class can be auto-generated from the interface class - the full
> code for registering an interface with the RPC mechanism would collapse to
> RPC::Register<T>(); To do so, it would need two language additions. It
> needs access to a compile-time iterator over functions and traits about the
> argument types, return type, name and whether it is a virtual function. For
> the prototype this can be manually generated for an interface to show it
> works. Additionally, it would need to be able to use such a definition to
> instantiate a virtual function from a base class - basically, hooking up an
> std::function<X> to a virtual function X directly. I think this is
> well-implementable (probably reusing most code from lambdas for the link
> code), but I don't know whether that's true.
Mechanisms for that are already there: Boost.Serialization. Nevertheless I'd
be very interested to see the code of your prototype.
Here is more on actions: In order to turn a regular function into a "RPC"able
action, you essentially need a way for the communicating partners which
function should get executed on the other hand. The simplest way to do that is
by it's function pointer and the arguments that are supposed to be passed to
that function. The function pointer, and it's type is the hard part here. We
currently solve this by having a template (instantiated on all communicating
partners) which is serialized over a polymorphic base pointer. As such you
only need to register this function as a action with your serialization
library and call it a day. Side note: We haven't found a truly portable way to
automate that, in addition, with this techniques, we are not able to send
lambdas over the wire, and the committee had come to the conclusion that it
isn't possible with the current standard, IIUC.
> Any ideas? Reactions on RPC or Mocking?
I like the idea of having a generic RPC library lying around somewhere.
The requirements, i can think of right now, for us in order to use it would
1) Extension mechanism for the actual network communication
2) Extension mechanism for the actual execution on the remote end
3) Extension mechanism for the future which gets returned
4) High performance of the serialization with zero copy support (preferably
API compatible with Boost.Serialization)
With that being said, HPX is a little more than just a library for doing RPC,
there is a whole bunch of stuff we implemented to make it convenient and fast
to use (and we still aren't really there yet ...). It would be great though if
we could out source some of our infrastructure to 3rd parties to ease our
maintenance burden ;)
> Best regards,
> Peter Bindels
> Unsubscribe & other changes:
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk