Re: [boost] yomm2 - open methods

Subject: Re: [boost] yomm2 - open methods
From: Jean-Louis Leroy (jl_at_[hidden])
Date: 2018-01-24 14:21:43

Sorry for all the typos and mistakes in my previous post.

>> I am still hesitating about the syntax for defining a method
>> (overridern). I could easily support:
>>
>> // no return type here
>> // ---------V
>> YOMM2_BEGIN(kick, (Dog& dog)) {
>>   return "bark";
>> } YOMM2_END;
>>
>> It breaks the symmetry with YOMM2_DECLARE though.
>>
>
> I think I prefer having the return type present.
> (Does auto work?)

It does now! That's a good compromise.

> Can't you just rearrange it to:
> ...
> struct _YOMM2_SPEC {
>   static RETURN_T body ARGS;
> };
> ... register_spec<> init();
> }}
> inline RETURN_T _YOMM2_NS::_YOMM2_SPEC::body ARGS

This wouldn't work as is, because _YOMM2_NS generates a new namespace name each
time it is called (using __COUNTER__) but I found out about BOOST_PP_SUB and
changed _YOMM2_NS so it can re-generate a previous namespace. So now YOMM2_END
is gone.

> There's nothing here that specifically prevents inlining.
> Whether the compiler actually inlines it is another
> question but that's already very compiler-specific
> and not guaranteed.

Not only does clang inline, but LLVM is smart: since the data needed for
dispatch comes from three locations (the hash table, the method and the class),
and the first two can be acquired independently, LLVM orders the instructions
so they can be executed in parallel. That's probably what explains the
(pleasantly) surprising speed of a 1-method call - within 15% of the equivalent
compiler-generated virtual member function call.

> - "Each name that ... begins with an underscore followed by
>   a capital letter is reserved to the implementation for any
>   use" [global.names]

I'm working on it.

> - update_methods looks like it's totally thread-unsafe.
>   You can probably get away with this if you only call
>   it at the beginning of main, but it seems quite dangerous
>   if you load or unload shared libraries.

That's a complex question.

For starters, is dlopen thread safe? GNU dlopen is explicitly documented as
"MT-Safe", but this SunOS page
https://docs.oracle.com/cd/E26502_01/html/E26507/chapter3-7.html does not say
anything on the subject. And then there are bug reports circulating about
dlopen in multi-threadec context.

And what of dlclose? Better make sure that a thread does not call dlclose while
another is still executing the library's code. Or that each thread that uses
the library calls dlopen itself (and increments the library's ref count).

In the light of this, I have so far left it to "the application" to manage its
calls to dlopen, dlclose and update_methods.

But this is just the beginning. Adding a mutex to serialize calls to
update_method (and the static ctors and dtors that are generated by the macros)
is not enough, because a thread may be executing the method dispatch code while
update_methods is running. I would need a read/write mutex, with the dispatch
code acquiring a read lock until it has fetched the pointer to the appropriate
function. But that would be too penalizing.

I wonder if Pirkelbauer, Solodkyy and Stroustrup addressed this problem when
they worked on open methods. Their paper mentions dynamic loading but it
doesn't say much except that it's important to support it. I'll ask Solodkyy -
we exchanged emails in the past.