Re: [boost] Re: Callbacks Again

From: Karl Nelson (kenelson_at_[hidden])
Date: 2000-11-19 12:22:52

> > 
> > Okay 3 issues here...
> > 
> > 1) Should a library have a callback and later build slots?
> > I have had that arguement a number of times and what always gets 
> pointed
> > out is that building just callbacks causes some many problems
> > it isn't worth it.  Libsigc++ provided the class Callback# which
> > was exactly that.  It was a union of pointers, function pointers,
> > methods with objects etc.  However, as I discovered you
> > would have to build your own set of factories for those callbacks
> > in effect doubling the size of the library.
> 
> That's what libraries like Lambda are for.

Well, the only difference between a Callback and a slot is
the Slot can track both forward and reverse dependencies.

Forward means when the sender doesn't need the callback the
callback gets destroyed, if chaining is allowed then adaptors
are possible.  SigC supports cahining which is I can build
a slot and then make another slot point to it and so forth.

Reverse dependencies includes the ablity for the target to
force the clean up.  That is if the target falls out of scope 
it takes out the chain. 

Example.

   Slot0<void> s;
     {
       Foo f;
       s=slot(f,&Foo::some_function);
       s(); // calls f.some_function();
     }
   s();  // calls nothing

This is the only functionality provided by a slot beyond
a callback.  

> >  Second, you can't
> > really build adaptors with the callbacks because they would
> > require chaining of resources.   In other words, slots
> > in the clean up capablity allow cleaning up the objects which 
> > allow adaptors.  So Callbacks are of limited use.
> 
> I *really* don't follow this.  Care to post some specific examples of 
> what you mean?

SigC provides for adaptors which in turn form cascades...

  int foo(int, float);
  Slot0<void> s;
  s=slot(rettype<void>(bind(slot(&foo),1,1.5)));
  s();  // calls foo(1,1.5);

Also note the chaining adds some complexity to the 
cleanup.  In other words if you get both a chain and 
a target removal it can get quite messy.  

> 
> >  Further,
> > they aren't all that useful to build the Slots themselves as
> > the slot just ends up carrying arround the bulk of
> > a general callback where it could be trimmed down.
> 
> I'm not suggesting that Slots have to be built off of a general 
> callback, just that it would be worth considering once done.  As 
> for "bulk"... we're talking a virtual table, a pointer and the size 
> of the functor.  A Slot must carry at least this any way.  I don't 
> see problems with bulk.

Okay, we are thinking in different terms here after looking at the 
Callback library someone submitted.  SigC does not do Functors
like that it does something more like Closures.  SigC does not
carry the Functor with it only the address of a target who will
receive the call.  Thus it targets pure functions and methods 
not function objects.

(Side note, SigC was designed not to carry virtual tables in
templatized types.  Many compilers at the time in which 
SigC developed generate a huge bulk of vtables if each templatization
was a virtual.  Thus it has no virtuals in those classes but rather
a function pointer.  Early testing found that using vtable like
that expanded the gtk-- library by several megs in pointless vtables 
typeinfo and exception info largely because the linkers didn't
combine the info when building libraries.  Essentially, every copy
of a template with virtuals would implement the vtable in every
object file and thus make a huge library with many repeated 
symbols.  Compiler tech has improved since and this may not be 
an issue since no one is using the compiler sigc originally
targeted gcc 2.7.2.3)

> > 2) Can a Slot be a Callback?  
> > The answer to this is yes.  That is that if I make a Slot like
> > concept with the ablity to disconnect and clean up the 
> > traces between objects, I don't have to use that path.  
> > Thus it is a very common misunderstanding the to use SigC
> > you must use SigC::Object.  You don't, you simply must use a 
> different
> > factory to produce pure callbacks rather than slots.  Thus
> > nothing in SigC requires SigC::Object other than SigC internals.
> 
> But Signals/Slots are multi-casting concepts, which is *not* the same 
> as a callback.  Or are you suggesting that the Slot by itself is a 
> pure callback?  If so, the documentation in libsigc++ doesn't address 
> this at all, which is a huge disservice.  Callbacks are frequently 
> needed in areas in which a multi-casting Signal/Slot is either 
> overkill or even problematic.

Signals are the multicast.  A Slot is just a generic callback with
the target tracking.  Escentially, if you make a good callback library
all you need is a class called Signal which contains a list of
callbacks and one specialized method which unifies the calling
of the list.   

Someone released a SigC companion library which added Event.  So
certainly Slot is more general that multicast.

  
> > 3) Can a Signal/Slot be used in Threads?
> > There are issues but they aren't as bad as they first appear.  
> > Ther real problem is that only one thread can be destroying or
> > adding things to a list at one time.  Some of this 
> > can be done properly with make before break and atomic flags on
> > the lists.  Further, the reference counters have to be protected.
> > Really advanced adaptors and signals are out unless the signal
> > queue is protected internally.  But for the most part it
> > is doable. 
> 
> I never suggested it wasn't ;).  The Boost thread library (which is 
> far from ready for prime time) could be quite useful for insuring 
> thread safety here.

Well, sigc is going to be switching away from providing the 
thread capablity.  That is I intend to make the library compile
itself twice, once with locks and once without.  Most people won't
need locking so there is no point in placing if except in the 
multithread environment.

 
> > Okay so you quickly think why can't I just add a list in the 
> > the object the callback is pointed to to kill the callback and
> > make a slot?  But that creates its own problem, who cleans up
> > the list if the callback goes away?  Basically you end up with
> > dependency loops, which require a unified method to handle the 
> > internal cleanups.  Adaptors to callbacks also cause this
> > dependency loop because the data in the adaptor can become stale.
> > Thus for efficency building a really good Signal/Slot library
> > which has the ablitity to do the event and threading is 
> > the right approach.
> 
> I still have problems with the multi-cast issue.  If you can address 
> that to my satisfaction I'll have a better idea of where you're 
> coming from.

Okay here are the issues which SigC was seeking to address.
(NOte, that you will find much of this
in the documentation, but it has been so abstracted in the 
code to save memory it is practically indeciperable.)
Source is the Slot which points to adaptors and callbacks.

1) Target clean up of a Callback

   Source  -> Callback -> Target

If  Target goes away Callback should go away and free any
extra resouces it was holding.  Source should revert to
uninitialized.  If Source goes away Callback must go. 

2) Chaining

   Source -> Adaptor0 -> Adaptor1 -> Callback -> Target

If Target goes way, Adaptor# and Callback should be
destroyed.  Source should return to uninitialized.
If Source goes away, same.

3) Multicast

A Signal is a container of Slots.  It connects to a Slot
with a list element.

  Signal *-> SignalNode -> Callback -> Target

This has the same issue as before.  If the target goes
away we must remove everything up to the signal node. 

When emitted the signal must trigger all the Slots
in the callback list.  

4) Multicast return marshalling

Given a list of Slots how do we resolve the return values.
A class called Marshal handles this.  The Marshal can do
anything with the returns from stacking them up in lists 
to ignoring them completely.

peusdo C++
  
  template <R,P...,Marshal>
  R Signal::emit(P...)
    {
      if (empty()) 
        return Marshal::default_value;
      Marshal m;
      for (iterator i=begin();i!=end();++i)
        m.marshal(i->call(p));
      return m.value();
    }

5) Generate readable errors.  

Okay this was the tall order.  Gtk-- is not supposed to
be hyper advanced C++.  It was meant for people constructing 
GUIs who used Qt.  They may be experts but more likely 
they are a bit green.  In the design of the library, I
wanted to isolate the errors.  Thus rather the spitting out...

  on line  (From boost/Callback)
     s=mul_to_int();  // where mul_to_int doesn't take right arguments

boost/callback.hpp: In method `int boost::detail::functor_callback_impl<2>::base<mul_to_int,int,float,float,boost::detail::Unused>::operator ()(float, float)':
boost/callback.hpp:461:   instantiated from here
boost/callback.hpp:256: no match for call to `(mul_to_int) (float &, float &)'
callback1.cpp:13: candidates are: int mul_to_int::operator ()(float, float, bool

SigC spits out 

  on line
    s=slot(&mul_to_int);

test.cc:8: no match for `SigC::Slot2<int,float,float> & = SigC::Slot3<int,float,float,bool>'
/usr/local/include/sigc++/slot.h:489: candidates are: class SigC::Slot2<int,float,float> & SigC::Slot2<int,float,float>::operator =(const SigC::Slot2<int,float,float> &)

Likely this last one isn't an issue to you.  But to my users it
really is important. 

  
> > Well, signal/slots are always going to require their own generators
> > as to get the unregistering capablity you must form both the 
> > bound pointer and make the dtor mechanism to clean up.
> 
> But such a factory could be simplified by using Lambda or other 
> binding libraries.  If the factory needed only a simple functor to 
> generate the "slot" things would be greatly simplified, no?

I don't believe so.  The Slot can not extract to target from the
Functor, thus I am not sure how you would construct to reverse 
tree give a functor which holds a method pointer to another 
object.  I think we lack some level of generality, because 
I don't see how to expand SigC to use functors as the boost/Callback
does and at the same time I don't see how boost/Callback could
be expended to work like Slots.

> > 
> > I won't look at libsigc++ TOO closely, because there was a lot
> > of crap there which I quite frankly and throwing out.  
> > 
> > Things that are going...  Handles, Scopes, Callback#.  These
> > were attempts to make a unified type ie a Slot was a Object
> > etc.  These bad attempts are going away.  The reduced library is
> > a fraction of the size.  Further I am paring out the 
> > use of some memory elements.  Thus the cost of an object slot
> > is going from 64 bytes to 28 and so on.  
> > 
> > With luck the cost of a good slot system will be almost the 
> > same as the cost of a good callback system.
> 
> Any work in progress we can look at?  I realize you're time is 
> limited, but the need for callbacks in Boost is great enough to 
> generate some urgency on my part.  If you plan to submit the slot 
> system any way I'd really like to not step on your toes by pushing 
> for a (possibly) competing callback system.

I can send you the code I have.  It is missing multicast and
thread locks but has most of the rest there.  Can you send me a 
email address privately?  (The new code is well outside the level I 
want to release for boost.)

As for whether SigC is approprate for boost or could serve in
that function, that is fairly questionable.  THere is a fairly wide
gap between your functors and sigc slots, though both are
clearly attempts at the same general class of problem.

--Karl