Re: [OpenMethod] review starts on 28th of April

From: Steven Watanabe (watanabesj_at_[hidden])
Date: 2025-05-11 03:01:44

AMDG

On 5/10/25 6:33 PM, Jean-Louis Leroy via Boost wrote:
> <snip>
> I experimented with passing a Policy* as the first argument to the function
> after Ruben's remarks. It's not hard to make it work, but it requires
> boost_openmethod_vptr() to either specify the exact policy it expects, or be a
> template if it wants to ignore it.
> 

I think specifying the exact policy is going to be the right thing most 
of the time. Do we ever want a vptr to be able to be used with any 
policy? virtual_ptr has a Policy template param and checks compatibility.

> 
>> I think virtual_ptr is doing too much. What would it take to make it
>> possible to implement virtual_ptr without any special support in the
>> core library?
>> - boost_openmethod_vptr provides a way to get the vtable
>> - The rtti policy and/or static_cast should allow downcasting
>>     to the overrider parameter type.
>> - The only thing that is clearly missing is the ability to
>>     indicate that virtual_ptr is a virtual parameter.
> 
> In YOMM2, virtual_ptr is an afterthought. While preparing OpenMethod for review,
> I struggled with explaining clearly and simply that `virtual_` should be used
> only in the base-method, and *not* in the overriders. YOMM2 beginners often
> struggled with that.

Can we strip off the virtual_?

struct NAME ## _overrider<...> {
   template<typename T>
   using virtual_ = T;
   ...
};

Actually, what if we make virtual a keyword handled by the macro:

BOOST_OPENMETHOD(foo, (virtual T&), void)

It would look something like
#define HAS_VIRTUAL_TESTvirtual ~,~
#define HAS_VIRTUAL_I(a, b, r, ...) r
#define HAS_VIRTUAL(arg) HAS_VIRTUAL_I(HAS_VIRTUAL_TEST ## arg, 1, 0, ~)
#define REMOVE_VIRTUALvirtual
#define REMOVE_VIRTUAL(arg) REMOVE_VIRTUAL ## arg

This won't quite work as is. It needs some work to adjust the order of 
macro expansion. There's also the problem that the preprocessor's view 
of the arguments is not the same as the compiler's. So we can insert a 
dummy argument into the signature, which can be stripped out by 
metaprogramming:
void(next_parameter_is_virtual, T&)

> Then I though, on top of that, virtual_ptr gets us dispatch
> in three instructions instead of nine (for 1-methods). Shouldn't it be the
> "golden path"? And virtual_ be demoted to an entry point into advanced stuff?
> 

To get the performance benefit, virtual_ptr has to be integrated into 
the surrounding code. If we're doing that, then it should act more like 
a normal pointer, and the conversion from a reference (which makes some 
sense when it's being used as a method parameter type) is not such a 
good idea.

>> I tried using shared_ptr as a parameter type like this:
>>
>> <snip>
> 
> I am surprised. I have a unit test for that (cast_args_shared_ptr_by_value). You
> didn't attach the example, here is what I tried:

Maybe I forgot to make Derived inherit from Base. I remember doing that 
at least once.

> <snip>
> 
> By the way, do you have an opinion on allowing smart pointers as virtual
> parameters?
> 

I think it should be allowed. Can I bring my own smart pointer?

>> <snip>
>> throw_error_handler directly throws the error type as an exception, but
>> openmethod_error does not inherit from std::exception. Even though this
>> is perfectly legal, dealing with such exception can be rather
>> painful.
> 
> I am not exception-phobic. Quite the opposite. But doing this would mean that
> the error classes would be different whether or not exceptions are enable. I
> tried to avoid ifdefs as much as I could...
> 

That's not necessarily true. The actual type thrown can be an internal 
type that inherits from both std::exception and the error type.

> <snip>
>> The simplest way to solve this is to allow a policy to customize
>> is_base_of and is_abstact.
> 
> This is an interesting research path...abstract sub-categorization away from
> inheritance...I think that Clojure does something like that.
> 
> A few times people requested value-based dispatch for YOMM2. Like in CLOS. I
> have a design for this. But then the latest requester wnats dispatch on *ranges*
> of values. I wonder if there is a general way of allowing such extensions, that
> is also practical.
> 

Value-based dispatch can sort of be done now by registering a type for 
each distinct group of values and adjusting dynamic_type to distinguish 
them. This won't work well with virtual_ptr or any other type that 
caches the vptr, though.

The main issue with ranges is that they can overlap in various ways. 
You'd need to look at all the ranges that might be used for a given 
parameter, and find all the subranges where the ranges intersect with 
each other.

For a parameter type T, and a range specification R, the user would need 
to provide a function that partitions the set of values of type T into 
subsets.

template<typename T>
struct partition {
   // To determine the best match we need to know
   // whether an input set is a subset of another input set.
   // To determine whether an overrider matches we need to
   // know whether a particular subset is part of an input set.
   bitmatrix subset_info;
   // When dispatching we need to quickly find the right subset,
   // which can then be used to look up the
   std::function<std::size_t(T)> select_subset;
};

partition user_defined_make_partition(std::vector<R> input_sets);

If the sets are ranges, we can store the boundaries and do a lower_bound 
search.

> <snip>
>> It can also generate results that are obviously silly (at least to a human)
>> pretty easily.
> 
> Do you mean silly and incorrect, or just silly?
> 

Just silly. I worked through what it would generate for several 
hierarchies and was a little surprised.

>> <snip>
> 
>> I believe that in the absence of virtual inheritance, this is guaranteed to
>> assign slots contiguously without leaving any holes.
> 
> I am puzzled by this remark, because YOMM2 and OpenMethod are a bit myopic
> regarding virtual and repeated inheritance.
> 

I'm puzzled by it too. The problem is cycles in the undirected 
inheritance graph, and I forgot that virtual inheritance isn't the only 
way to get a cycle.

> At the bottom of it, the library suffers from being able to have only one
> v-table per object. That is why repeated inheritance is not supported. And that
> is the reason for the contortions about slot allocation.
> 
>> compiler.hpp:1147:  *cls.static_vptr = gv_iter - cls.first_slot;
>> This is quite scary. It's undefined behavior if it goes before the
>> beginning of the vector. <snip>
>> 
> 
> Oh, right. I doubt any real program will crash on this, but you are right, this
> is UB, and it can be fixed, so let's fix it. It is probably just a matter of
> ordering the v-tables from lowest first_slot up, and perhaps from highest size
> up if first_slot is the same.
> 

I think just ordering by first_slot is sufficient, as every vtable that 
has a slot before first_slot will come before it and every slot should 
be used in at least one vtable.

What if the slots are reserved by an abstract base that has no concrete 
derived classes? Does that situation get filtered out somewhere earlier?

> I guess that you will frown at the idea of using the same trick to avoid storing
> entries in the perfect hash table that come before the minimum value in the hash
> function's image.
> 

Yeah, but it's less obvious how to fix it.

In Christ,
Steven Watanabe