Re: [boost] Re: New New Callbacks Uploaded

From: Karl Nelson (kenelson_at_[hidden])
Date: 2000-12-05 17:50:08

> > I should point out that shared pointer to a cloned object is not
> > the same as a a pure refcounted system.   Rather that trying
> > to explain it a picture is worth a thousand words...
> 
> The slot class holds data that describes a function call (be it a 
> true function or an object/method pair) and manages the lifetime for 
> this data.  Why, specifically, can't this data be replaced by a 
> callback concept that uses cloning?

Let me explain my UML diagrams better.
  _____      _____
 |  X  |--->|  Y  |
 |_____|    |_____|

this means there are two objects (possibly on the heap) and X
contains a pointer to Y.
  _____
 |  X  |
 | ___ |
 || Y ||
 ||___||

X is a class which contains a Y.  This may mean

  class X
    {
      Y y;
    };
 
or 
  class X : public Y
    {};

[....]
> > The object pointed to by the handle is composite of all the objects.
> 
> Makes sense.

In the above you have one vtable in the compose object.
  
> > shared pointer to cloned object..
> >    _____
> >         |------------> counter  
> >         |      ____________________
> >  smart  |     |                    |
> >   ptr   |---->|  adaptor 2         |
> >    _____|     | __________________ |
> >               || adaptor 1        ||
> >               || ________________ ||
> >               ||| generic_functor|||
> >               ||| ______________ |||
> >               |||| user functor ||||
> >               ||||              ||||
> >               |____________________|

Here we have one vtable and one counter.  (the counter may or
may not be composite with the callback.  If it is there is one
heap item, else there is 2 and the size of smart_ptr is 2 pointers) 
 
> This isn't what is meant at all.  The smart pointer would be an 
> implementation detail within generic_functor in your picture above.

If the cloned system constructs things with composites and than 
you add smart pointers I can only see two possible systems.  The
one above or....
 
               _______       ____________________
             |        |     |                    |
 smt_ptr |*->| handle |---->|  adaptor 2         |
             |________|     | __________________ |
              (counter)     || adaptor 1        ||
                            || ________________ ||
                            ||| generic_functor|||
                            ||| ______________ |||
                            |||| user functor ||||
                            ||||              ||||
                            |____________________|

You can't change the composite object because it was already 
built be the other system.
  

> > Pure counter system
> >   _______     _______     _______     _________________
> >          |   |       |   |       |   |                 |    
> >   handle |*->| adp 1 |-->| adp 2 |-->| generic_functor |
> >   _______|   |(count)|   |(count)|   |  (counter)      |
> >              |_______|   |_______|   | _______________ |
> >                                      || user functor  ||
> >                                      |_________________|
> 
> I'm not sure why this design got "flattened".  It's the same as the 
> previous design, just with the counter as a part of generic_functor 
> instead of seperated.

No, totally different.  There are 3 items on the heap.  All of the heap
items are derived from a common base class.  Adp1 and Adp2 have pointers
to the next object in the chain.

We have 3 vtables, 2 pointers for indirection, and 3 counters.
Further, the handle can have multiplicity in this diagram and
could potentially be pointing to adp2, adp1 or generic_functor.
And of the handles could potentially start the chain which emits
the user function.  

There is no possible way the above system could be arrived at
from the previous cloned system diagram.  It is a fundamentally 
different design.

[...]  I'm missing something.
Did you find it?  ;-)

 
> > The best you can do is build a good self contained system
> > to solve a select set of problems.  Then build a second system
> > to solve different set of problems and describe a reasonable
> > set of interaction between the two.
> 
> I also don't follow what you mean here.

I am merely suggesting solve the problem of making a good cloning
system.  Trying to make a system which some how builds up to 
lifetime and multicast is too much work over starting the
multicast system from scratch.  Trying to meld all these ideas
into one system is too much to chew.  

Build your system to be fast and efficient without trying to
add the other baggage or somehow plan for some sore of massive
expansion later.  When done, look at the problem set which is not
covered, build the best system for needed to cover those regions.  
Then define how those two systems should interact.  They will be
very stringent but likely workable.

Sigc++ covers multicast, lifetime and even event queuing
and trivially could do callbacks.  It can be expanded to 
multithread but the cost goes up.  Using with generic functors
is just dangerous.

The boost system is targeted at multithread, generic functors,
and speedy callbacks.  Expanding it to multicast, lifetime, or
event queuing likely will give poor result.

On the surface these look like loads of overlap with both having
adaptors, creation, and providing a way to call a single callback.
But the points of the design are indeed vastly different.   
  
[...]
> > If you were to allow it you could do this....
> > 
> >    functor<void,int> f= functor_from_method( object, 
> &Object::method);
> >    slot<void,int> sl=f; 
> > 
> > Unless you pierce the f to get the object from which the functor
> > was made you can't add the dependency for the slot to track.  
> 
> The slot can still be created only through the "approved" factory 
> functions (which would preclude creation from generic function 
> objects).  Internally the generated function object could be based on 
> the basic cloning callback, with extra plumbing to hook into 
> destruction of objects associated with member functions.  Trivial to 
> implement, just not necessarily the best implementation.

That was tried in an early version of sigc++ and found too
memory consuming.  But then again I had a target heap size I had
to meet with no more the 64 bytes of heap to describe the entire
connection from signal to object and back the other way.
My requirements are indicative of a GUI set which uses many hundreds
of different callback types.  (I didn't chose it, the underlaying C kit
I wrap did that.)

That is why sigc++ abandon the most generic structures like 
supporting a stack based non-tracking callback in favor of a
more suited multicast structure.

<aside> 
for reference the sigc++ 0.5 generic callback looked like
  
  template<....>
  class Callback# 
     {
       virtual R operator()(....);
       virtual ~Callback#();
       Object* obj(); // gives access to target so we can add dependency
       private:
         Node* data;  // list of heap data for adaptors 
         void* object;
         union {
            void (*fuct)();
            void (*O::method)();
         };
    };

This allowed it to support function functors and object pointer
functors without going to the heap.  Only generic functors and 
and adaptors pushed the data out onto the heap.  This gave a stack
size of 5*sizeof(void*) for most cases with no heap size.  I used
these to build the dependency tree by peering into callback through
object.  But building this to a dependency system did not meet
my size goal as the slot ended up with much the same stuff from
the generic callback.  Sigc++ 0.7 dropped all but the callback in
name which is still in sigc++ 1.0 just for the convert adaptor.

Then this is optimized for function and methods because
I didn't really care about generic functors.  Likely yours
won't use functions and method often enough that the extra
stack size is worth it. 
</aside>

> I'm not 100% convinced of the need to track the object's lifetime.  I 
> think it's simpler, even as the end user, to leave this management up 
> to the programmer.  

Yet, for some cases this is virtually impossible.  Adding tracking to 
objects that lack it and were created from inside a library is no
fun at all.  I do appreciate that lifetime tracking is not something
every programmer would consider a necessity.  But sometimes having
it makes for much simpler user designs.
  
> However, I fully understand that this may be an 
> opinion that is very much in the minority, and understand that 
> there's significant users of sigc++ that would not stand for this.  
> This is another reason why the concepts should be seperate.  Whether 
> they share any implementation is, frankly, a non-issue to me.  It's 
> very possible for them to do so, but there may be compelling reasons 
> not to, such as efficiency.

> What's important to me is that a ref-counted variant *can* be 
> implemented on top of the cloning variant, which could be very 
> important for several concepts that we've not thought of yet.

It is certainly possible to build a ref-counted cloned system
which is a good compromise for the above debate.  I just 
wanted it clear the result of that system would be very different
from an entirely ref-counted system to begin with.
I get the feeling that you had not considered the segmented
shared design which sigc++ is using.  Shared composite object and 
unshared composite objects are basically the same devices.  A
shared uncomposited system is an entirely different beast.

It is the vast differences in the heap layouts of the sigc++ design
which require what appeared to you as largely replicated code.
(there are other many hybred designs like the one sigc++ 1.0 uses, but
those have additional problems we have yet to discuss.)

--Karl