Re: [boost] Boost.Python sandox or multiple interpreters

Subject: Re: [boost] Boost.Python sandox or multiple interpreters
From: David Bellot (david.bellot_at_[hidden])
Date: 2017-02-17 10:03:35

in fact, I found a much better solution, much more element by using the
wrapper<T> class and having the boost::python::object instanciated in C++
rather than in the interpreter (if my interpretation of the interpreted is
correctly interpreted in this context).

So each plugin is a python object and thanks to wrapper<>, the Python user
can now derive from a base Plugin class and implement whatever callbacks
they need, instead of doing the strange file gibberish I did before.

For those wondering how it works:
1- Make a Base class with pure virtual methods that you want the python
user to implement

class Plugin
{
public:
    Plugin(Server&); // Server is whatever C++ code control the plugin in
fact
    virtual ~Plugin() = default;

    // evaluates the plugin
    virtual bool eval(int x) = 0;

    // use some Server function
    void a_function(const std::string& val);

private:
    Server& server;
};

2- Make an intermediate class which inherits from Base and
boost::python::wrapper<Base> in which the user's function are implemented
like in the following example:

class PluginWrapper final : public Plugin, public wrapper<Plugin>
{
    using Plugin::Plugin;

    void eval(int x) override
    {
        return get_override("eval")(x); // This will call the user's Python
code
    }
};

3- Expose your API in Python as usual:

BOOST_PYTHON_MODULE(Foobar)
{
    class_<Server>("Server");

    class_<PluginWrapper, boost::noncopyable>("PyPlugin", init<Server&>())
        .def("eval", &Plugin::eval);
}

4- When you want to initialize your plugins, you can load the user's Python
code as follow, where module is the Python module name (can be foobar.py or
a foobar/ directory with __init__.py inside) and path is the ... path to it
!

    dict locals;
    locals["module_name"] = module;
    locals["path"]        = path;

    exec("import imp\n"
             "new_module = imp.load_module(module_name, open(path), path,
('py', 'U', imp.PY_SOURCE))\n",
             globals,
             locals);

in the end  locals["new_module"] contains the module and that's what you
want to use to instantiate an "object" in your C++ code.

5- Start your C++ code with something like this:
  Py_Initialize();

for Python 2
  PyImport_AppendInittab("Foobar", initFoobar);

or for Python 3:
  PyImport_AppendInittab("Foobar", PyInit_Foobar);

    Server server;

    // import the __main__ module and obtain the globals dict
    object main     = import("__main__");
    object globals  = main.attr("__dict__");

    // *** HERE PUT CODE FROM STEP 4 ***

    object module = locals["new_module"] // This contains the Python module
    object PluginClass = module.attr("MyUserPlugin"); // This contains the
plugin class as defined by the user
    object plugin = PluginClass(server); // This contains the running
plugin itself

// And you can instantiate as many object plugin as you want. They all will
be different even if instantiated from the same class because they live in
the C++ space. It's what solved my problem in the most elegant way. I know
it looks soooo obvious now :-D but I just didn't know about the wrapper<>
class before !

    // eval the plugin
    bool result = plugin.attr("eval")(32); // Yep, as simple as that !!!!!

6- And finally write your plugins:
from Foobar import *

class MyUserPlugin(PyPlugin):
    def __init__(self, server):
        PyPlugin.__init__(self, server)

    def eval(x):
        print "hello, world from the plugin. The value is ", x
        return True

7- one last note: the Server is used for the Python code to call C++ code
or for example send data to the C++ part of your sofware. So you can add
Python functions in you BOOS_PYTHON_MODULE to do that too.

And that's it. Look so simple now :-D

Best regards,
David

ref: https://skebanga.github.io/embedded-python/

On Fri, Jan 13, 2017 at 6:02 PM, Stefan Seefeld <stefan_at_[hidden]> wrote:

> On 13.01.2017 12:46, David Bellot wrote:
> >>> - or encapsulate each plugin in an independent "environment" ?
> >> Hard to tell without knowing these plugins. Right now it sounds like
> >> that's a question only you can answer. :-)
> >>
> > ​The plugins are quite simple in fact:
> >
> > The Python code define a few functions like
> >
> > def on_value_update(​x):
> >    blah blah blah
> >    return foobar
> >
> > and they can have global variables (if it's a problem I can forbid user's
> > to use those global variables in their plugins).
>
> For avoidance of doubt: "global variables" aren't variables in
> module-scope (they are still "local" to the module, and don't cause any
> problem.)
>
> > Each plugin is defined in a .py file and at runtime I load all those
> files,
> > one after each other.
>
> That sounds all good. Each plugin corresponds to a Python module (which
> is an object), and you can call module-level functions (such as
> 'on_value_update') on that object as if it was a method. So if you keep
> a list of module objects in your C++ runtime, you can access their
> methods without any global functions colliding. (See
> http://boostorg.github.io/python/doc/html/tutorial/tutorial/
> embedding.html#tutorial.embedding.using_the_interpreter
> for how to embed Python code into your C++ app.)
>
> In case that doesn't help, can you explain where in your current setup
> the plugin methods overwrite each other as they are loaded ?
>
>         Stefan
>
> --
>
>       ...ich hab' noch einen Koffer in Berlin...
>
>
> _______________________________________________
> Unsubscribe & other changes: http://lists.boost.org/mailman
> /listinfo.cgi/boost
>