Re: [boost] Re: Callbacks Again

From: Karl Nelson (kenelson_at_[hidden])
Date: 2000-11-20 13:03:04

[...]
> 
> Valuable implementation details, though it makes for some interesting 
> overhead for uses like that needed for thread callbacks (in case you 
> don't understand what I mean here, as *might* have been indicated by 
> something you said in another post, the thread callback is what is 
> invoked as the "work" of the thread when it's created).  I'd need to 
> evaluate how much overhead exists before really commenting here.

SigC is definitely way overkill for launching a thread.  SigC
is more appropriate for the following situation.

You have an event queue.  Slots are being produced and placed
on the queue.  However, at some later point the target of 
operations in that queue becomes removed.  The queue then cleans
itself up. 

> > Reverse dependencies includes the ablity for the target to
> > force the clean up.  That is if the target falls out of scope 
> > it takes out the chain. 
> > 
> > Example.
> > 
> >    Slot0<void> s;
> >      {
> >        Foo f;
> >        s=slot(f,&Foo::some_function);
> >        s(); // calls f.some_function();
> >      }
> >    s();  // calls nothing
> > 
> > This is the only functionality provided by a slot beyond
> > a callback.  
> 
> I'm not sure how you achieve this magic.  Unless Foo derives from 
> some base class (not acceptable for generalized callbacks) I'm not 
> sure how you'd know when it goes out of scope.  I definately need to 
> go through libsigc++ with a fine tooth comb.  I'm in danger here of 
> arguing with you about which concept is more appropriate with out 
> having a clear idea about the particulars.

You got it.  This is only possible with target cooperation.  
For this to work the target must derive or contain an SigC::Object.
If it isn't derived the second call will segfault on access to 'this'. 
It is just the property of slots that they can handle this reverse case.
 

[...]
> > SigC provides for adaptors which in turn form cascades...
> > 
> >   int foo(int, float);
> >   Slot0<void> s;
> >   s=slot(rettype<void>(bind(slot(&foo),1,1.5)));
> >   s();  // calls foo(1,1.5);
> > 
> > Also note the chaining adds some complexity to the 
> > cleanup.  In other words if you get both a chain and 
> > a target removal it can get quite messy.  
> 
> The Lambda library already allows for the "cascades" you show in the 
> example above.  The only thing I'm unsure of is the references you 
> make to "cleanup".

How is the cascade achieved?  When developing SigC I
tried two approaches.  One was to copy the old type into
the new and the other was to separate by some neutral abstraction
(without a vtable.)  

If it is done by copying the 
old functor type to the new type and so on then you will get a serious
size problem as you end up with classes like...

  rettype_adaptor0<void,int,
    bind_adaptor2<int,int,float,
      callback2<int,int,float> > >

SigC achieved this through template separation to give

  rettype_adaptor0<void,int>
  bind_adaptor2<int,int,float>
  slot2<int,int,float>

Thus allowing strong reuse of the classes.  

Copying the classes also proving nasty as if you copied a object
slot to get the reverse dependency working you would need to copy 
the list members again.

[...]
> > Okay, we are thinking in different terms here after looking at the 
> > Callback library someone submitted.  SigC does not do Functors
> > like that it does something more like Closures.  SigC does not
> > carry the Functor with it only the address of a target who will
> > receive the call.  Thus it targets pure functions and methods 
> > not function objects.
> 
> Not precisely, since it does what I know as "closures", which are 
> functors that bind an object instance and a member function.  Such 
> closures are "functors" in every sense of the word.
> 
> No, what I think you really mean is that "slots" are the 
> only "functors" and must be built using only function pointers 
> and "closures".  *If* I'm guessing correctly (still need to really 
> look at the code closely) this makes it easier for you to control 
> the "cleanup" magic, but it sacrifices a lot of functionality, 
> especially with Lambda Library and other expression template 
> libraries.  Sounds like the tradeoffs need to be carefully weighed.

Yes.  Likely this is the distinction.  The slots are
composable from function and object/member function pairs only.
You can build your own slot types and thus make functors
but that not commonly done.  (The new version will allow more tricks
like this

  Slot0<void> sl;
    {
      Signal0<void> sig;
      sig.connect(slot(&foo));
      sig.connect(slot(f,&Bar::bar));
      sl=sig;
    }
  sl(); // calls both foo() and f.bar()

which further increases the power of the multicast.)

[size tricks]
> I assume you used techniques similar to those in the Hickey paper on 
> callbacks?  I'm a little concerned about whether or not this "tricks" 
> are standards compliant, even if they work on known compilers.

I don't think I am.  I was using some fairly general template factorization.
Here is an example from SigC 1.1 (more readable)

// Abstract slot class (vtable, reference count, list node, and proxy)
class Slot_: public Node_
  {
    public:
      typedef void* (*Callback)(void*);
      virtual ~Slot_()=0;
      ...
      Slot_(Callback proxy);
      Callback proxy_;
//            ^^^^^^^^
//    This will be our vtable for callbacks.
  };

// Implement a class of Slots for use
struct FuncSlot : public Slot_
  {
    Callback func_;
    ...
    FuncSlot(Callback proxy,Callback func);
    virtual ~FuncSlot();
  };

// Now we templatize it.
template <class R,class P1>
struct FuncSlot1
  {
    typedef R (*Callback)(P1);
    static R proxy(P1 p1,void * s)
      { return ((Callback)(((FuncSlot*)s)->func_))(p1); }

    static FuncSlot* create(Callback func)
      {
        return new FuncSlot((Slot_::Callback)(&proxy),
                            (Slot_::Callback)(func));
      }
  };

Since the template versions of the Slot are not derived from
a base class with a virtual but simply provide one static 
function,  I have sacraficed some speed and runtime overhead
(1 function pointer) for a reasonably large ammount of 
binrary size.  The second thing this factorization did was
allow most of my functionality to move from a header to the
libary implementation.  Thus I get one 32k sharable chunck of
code which can be linked in with a shared library, where 
a standard template implementation would have had no
implementation section whatsoever.  (I am not explaining
this particularly well.)

Is that clear?

[...]
> > Someone released a SigC companion library which added Event.  So
> > certainly Slot is more general that multicast.
> 
> This makes it sound like we should have a Callback, which is similar 
> to a Signal but with out multicast.  You connect a Slot to a Callback 
> the same as you do to a Signal.

Yes.  I can certainly make my slots hook up to the boost::callback,
it is just the other way around which is fun because the slots
don't have a way of communicating target loss back.

Ie.  this code works....

#include <boost/callback.hpp>
#include <sigc++/signal_system.h>

...

main()
  {
    F f;
    boost::callback<void,int> cb = SigC::slot(f,F::foo());
    cb();  // calls f.foo();
  }

[...]
> > Well, sigc is going to be switching away from providing the 
> > thread capablity.  That is I intend to make the library compile
> > itself twice, once with locks and once without.  Most people won't
> > need locking so there is no point in placing if except in the 
> > multithread environment.
> 
> It may make sense to templatize on a "mutex type" with a null_mutex 
> that gets optimized away at compile time.  This allows you to mix and 
> match the locking requirements within a single program, which can be 
> very beneficial.
> 
> In any event, I was only suggesting the Boost Threads library as a 
> means of implementing the locks in a portable way.

Okay, SigC is way overkill for that task.

[...]
> > 
> > 1) Target clean up of a Callback
> > 
> >    Source  -> Callback -> Target
> > 
> > If  Target goes away Callback should go away and free any
> > extra resouces it was holding.  Source should revert to
> > uninitialized.  If Source goes away Callback must go. 
> 
> Removal if Target goes away sounds like a noble goal.  I *really* 
> have to figure out how you do this, and if it can be applied to 
> generalized functors.  As for removal when Source goes away... that 
> seems a non-issue to me.  In the versions of callbacks we've been 
> discussing a callback is a first class object.  Its lifetime is 
> controlled the same as any other first class object.  In other words, 
> when it's no longer being used (Source goes away) it's destroyed.  
> The only thing that would prevent this would be if the callback 
> object were placed on the heap and not managed by Source, but I don't 
> think you can, or should, control this behavior.

Right, SigC requires target to be from Object to achieve this.
All SigC slots reside in the heap to meet the lifetime requirements.

> > 2) Chaining
[2...]
> 
> This already happens with Lambda Library style expressions.
>  
> > 3) Multicast
> > 
> > A Signal is a container of Slots.  It connects to a Slot
> > with a list element.
> > 
> >   Signal *-> SignalNode -> Callback -> Target
> > 
> > This has the same issue as before.  If the target goes
> > away we must remove everything up to the signal node. 
> > 
> > When emitted the signal must trigger all the Slots
> > in the callback list.  
> 
> Very important functionality for Signal, but we need a Callback in 
> that case.  Multicasting would be bad for the use I have.

Agreed.

> > 4) Multicast return marshalling
[4...]
> 
> Also a good idea.
>  
> > 5) Generate readable errors.  
[5...]

> > 
> > I can send you the code I have.  It is missing multicast and
> > thread locks but has most of the rest there.  Can you send me a 
> > email address privately?  (The new code is well outside the level I 
> > want to release for boost.)
> 
> My e-mail address should be available through the eGroups 
> postings :).  Just to make it clear here, it's sirwillard_at_my-deja.com.

(Just checking some people hide their address.)
  
> > As for whether SigC is approprate for boost or could serve in
> > that function, that is fairly questionable.  THere is a fairly wide
> > gap between your functors and sigc slots, though both are
> > clearly attempts at the same general class of problem.
> 
> General class only.  I'm beginning to see very different goals 
> between the two beyond the obvious (multicast and non-multicast).  
> The obvious goals of multicast and non-multicast I think are both 
> appropriate for Boost, and would go so far as to say that both are 
> needed.  The trick now becomes addressing the subtler differences.

Well, if the callback concept can be expanded to handle the 
target cooperation, then you should likely just add multicaste
and you would have signal/slot implementation.  If not, then
they should be separate concepts which accept some interaction.

--Karl