Re: Callbacks Again

From: William Kempf (sirwillard_at_[hidden])
Date: 2000-11-20 14:43:23

--- In boost_at_[hidden], Karl Nelson <kenelson_at_e...> wrote:
> [...]
> > I'm not sure how you achieve this magic.  Unless Foo derives from 
> > some base class (not acceptable for generalized callbacks) I'm 
not 
> > sure how you'd know when it goes out of scope.  I definately need 
to 
> > go through libsigc++ with a fine tooth comb.  I'm in danger here 
of 
> > arguing with you about which concept is more appropriate with out 
> > having a clear idea about the particulars.
> 
> You got it.  This is only possible with target cooperation.  
> For this to work the target must derive or contain an SigC::Object.
> If it isn't derived the second call will segfault on access 
to 'this'. 
> It is just the property of slots that they can handle this reverse 
case.

Then doesn't it eliminate Slots from being used with generalized 
functors, such as those created by the standard library or libraries 
such as Lambda?
  
> > > SigC provides for adaptors which in turn form cascades...
> > > 
> > >   int foo(int, float);
> > >   Slot0<void> s;
> > >   s=slot(rettype<void>(bind(slot(&foo),1,1.5)));
> > >   s();  // calls foo(1,1.5);
> > > 
> > > Also note the chaining adds some complexity to the 
> > > cleanup.  In other words if you get both a chain and 
> > > a target removal it can get quite messy.  
> > 
> > The Lambda library already allows for the "cascades" you show in 
the 
> > example above.  The only thing I'm unsure of is the references 
you 
> > make to "cleanup".
> 
> How is the cascade achieved?  When developing SigC I
> tried two approaches.  One was to copy the old type into
> the new and the other was to separate by some neutral abstraction
> (without a vtable.)  

I believe it copies, but I could be way wrong here.  You should look 
at the Lambda library or talk to it's creators.
 
> If it is done by copying the 
> old functor type to the new type and so on then you will get a 
serious
> size problem as you end up with classes like...
> 
>   rettype_adaptor0<void,int,
>     bind_adaptor2<int,int,float,
>       callback2<int,int,float> > >
> 
> SigC achieved this through template separation to give
> 
>   rettype_adaptor0<void,int>
>   bind_adaptor2<int,int,float>
>   slot2<int,int,float>
> 
> Thus allowing strong reuse of the classes.  
> 
> Copying the classes also proving nasty as if you copied a object
> slot to get the reverse dependency working you would need to copy 
> the list members again.

Much of this overhead could be reduced, I would think, through ref-
counted pointers, much as we've done in the various callback 
implementations posted so far.  Regardless, you're getting into 
implemenation details (even if important) that are not really 
appropriate at this level of discussion.
 
> > > Okay, we are thinking in different terms here after looking at 
the 
> > > Callback library someone submitted.  SigC does not do Functors
> > > like that it does something more like Closures.  SigC does not
> > > carry the Functor with it only the address of a target who will
> > > receive the call.  Thus it targets pure functions and methods 
> > > not function objects.
> > 
> > Not precisely, since it does what I know as "closures", which are 
> > functors that bind an object instance and a member function.  
Such 
> > closures are "functors" in every sense of the word.
> > 
> > No, what I think you really mean is that "slots" are the 
> > only "functors" and must be built using only function pointers 
> > and "closures".  *If* I'm guessing correctly (still need to 
really 
> > look at the code closely) this makes it easier for you to control 
> > the "cleanup" magic, but it sacrifices a lot of functionality, 
> > especially with Lambda Library and other expression template 
> > libraries.  Sounds like the tradeoffs need to be carefully 
weighed.
> 
> Yes.  Likely this is the distinction.  The slots are
> composable from function and object/member function pairs only.
> You can build your own slot types and thus make functors
> but that not commonly done.  (The new version will allow more tricks
> like this
> 
>   Slot0<void> sl;
>     {
>       Signal0<void> sig;
>       sig.connect(slot(&foo));
>       sig.connect(slot(f,&Bar::bar));
>       sl=sig;
>     }
>   sl(); // calls both foo() and f.bar()
> 
> which further increases the power of the multicast.)

Nice "trick", especially if the overhead is managed by the 
implementation.
 
> > I assume you used techniques similar to those in the Hickey paper 
on 
> > callbacks?  I'm a little concerned about whether or not 
this "tricks" 
> > are standards compliant, even if they work on known compilers.
> 
> I don't think I am.  I was using some fairly general template 
factorization.
> Here is an example from SigC 1.1 (more readable)
> 
> // Abstract slot class (vtable, reference count, list node, and 
proxy)
> class Slot_: public Node_
>   {
>     public:
>       typedef void* (*Callback)(void*);
>       virtual ~Slot_()=0;
>       ...
>       Slot_(Callback proxy);
>       Callback proxy_;
> //            ^^^^^^^^
> //    This will be our vtable for callbacks.
>   };
> 
> // Implement a class of Slots for use
> struct FuncSlot : public Slot_
>   {
>     Callback func_;
>     ...
>     FuncSlot(Callback proxy,Callback func);
>     virtual ~FuncSlot();
>   };
> 
> // Now we templatize it.
> template <class R,class P1>
> struct FuncSlot1
>   {
>     typedef R (*Callback)(P1);
>     static R proxy(P1 p1,void * s)
>       { return ((Callback)(((FuncSlot*)s)->func_))(p1); }
> 
>     static FuncSlot* create(Callback func)
>       {
>         return new FuncSlot((Slot_::Callback)(&proxy),
>                             (Slot_::Callback)(func));
>       }
>   };
> 
> Since the template versions of the Slot are not derived from
> a base class with a virtual but simply provide one static 
> function,  I have sacraficed some speed and runtime overhead
> (1 function pointer) for a reasonably large ammount of 
> binrary size.  The second thing this factorization did was
> allow most of my functionality to move from a header to the
> libary implementation.  Thus I get one 32k sharable chunck of
> code which can be linked in with a shared library, where 
> a standard template implementation would have had no
> implementation section whatsoever.  (I am not explaining
> this particularly well.)
> 
> Is that clear?

Not just yet.  I see some of the magic here but I'll have to think on 
it for a bit before I really comment.
 
> > > Someone released a SigC companion library which added Event.  So
> > > certainly Slot is more general that multicast.
> > 
> > This makes it sound like we should have a Callback, which is 
similar 
> > to a Signal but with out multicast.  You connect a Slot to a 
Callback 
> > the same as you do to a Signal.
> 
> Yes.  I can certainly make my slots hook up to the boost::callback,
> it is just the other way around which is fun because the slots
> don't have a way of communicating target loss back.
> 
> Ie.  this code works....
> 
> #include <boost/callback.hpp>
> #include <sigc++/signal_system.h>
> 
> ...
> 
> main()
>   {
>     F f;
>     boost::callback<void,int> cb = SigC::slot(f,F::foo());
>     cb();  // calls f.foo();
>   }

I meant that boost::callback could be implemented to function exactly 
the same as a SigC::signal but with out the multicast functionality.  
I.e. it could handle the communication of target/source loss.  This 
sounds reasonable, and like the best approach, _except_ that 
SigC::slot can't handle generalized functors.  That seems to be the 
rub here, and is the big difference between the two concepts.
 
> > > Well, sigc is going to be switching away from providing the 
> > > thread capablity.  That is I intend to make the library compile
> > > itself twice, once with locks and once without.  Most people 
won't
> > > need locking so there is no point in placing if except in the 
> > > multithread environment.
> > 
> > It may make sense to templatize on a "mutex type" with a 
null_mutex 
> > that gets optimized away at compile time.  This allows you to mix 
and 
> > match the locking requirements within a single program, which can 
be 
> > very beneficial.
> > 
> > In any event, I was only suggesting the Boost Threads library as 
a 
> > means of implementing the locks in a portable way.
> 
> Okay, SigC is way overkill for that task.

I think we still have a disconnect.  Let me rephase yet again, being 
careful to spell it out:  I'm suggeting that Boost Threads, with 
their portable mutex classes, could be used in the implementation of 
SigC to insure SigC's thread safety.
 
> > > 1) Target clean up of a Callback
> > > 
> > >    Source  -> Callback -> Target
> > > 
> > > If  Target goes away Callback should go away and free any
> > > extra resouces it was holding.  Source should revert to
> > > uninitialized.  If Source goes away Callback must go. 
> > 
> > Removal if Target goes away sounds like a noble goal.  I *really* 
> > have to figure out how you do this, and if it can be applied to 
> > generalized functors.  As for removal when Source goes away... 
that 
> > seems a non-issue to me.  In the versions of callbacks we've been 
> > discussing a callback is a first class object.  Its lifetime is 
> > controlled the same as any other first class object.  In other 
words, 
> > when it's no longer being used (Source goes away) it's 
destroyed.  
> > The only thing that would prevent this would be if the callback 
> > object were placed on the heap and not managed by Source, but I 
don't 
> > think you can, or should, control this behavior.
> 
> Right, SigC requires target to be from Object to achieve this.
> All SigC slots reside in the heap to meet the lifetime requirements.

This is a tradeoff issue.  The other implementations posted for 
callbacks take a different approach.  The functor is copied once and 
then managed through ref-counting.  The only issue is 
with "closures", where the object portion of the closure may be 
destroyed leaving the closure in an "invalid" state.  I see the 
danger here, and knowing what SigC was supposed to address I can see 
the frequency with which this could occur.  What I'm wondering, 
though, is if it might not be appropriate to require the programmer 
to manage such issues?  I realize how difficult this could be, which 
is why you didn't leave it this way I'm sure, but at the same time it 
seems less intrusive.  It's the intrusive nature of how you handle 
this that prevents generalized functors from being used.  Considering 
the power that generalized functors provide, is the intrusive nature 
worth it?  I ask, because this is an issue for the current Callback 
postings, as well as for any "closure" implementation, so the 
concepts go beyond the applicability to SigC.
 
> > > As for whether SigC is approprate for boost or could serve in
> > > that function, that is fairly questionable.  THere is a fairly 
wide
> > > gap between your functors and sigc slots, though both are
> > > clearly attempts at the same general class of problem.
> > 
> > General class only.  I'm beginning to see very different goals 
> > between the two beyond the obvious (multicast and non-
multicast).  
> > The obvious goals of multicast and non-multicast I think are both 
> > appropriate for Boost, and would go so far as to say that both 
are 
> > needed.  The trick now becomes addressing the subtler differences.
> 
> Well, if the callback concept can be expanded to handle the 
> target cooperation, then you should likely just add multicaste
> and you would have signal/slot implementation.  If not, then
> they should be separate concepts which accept some interaction.

Yes.  I think my question about the appropriateness of the management 
is the crux of the whole thing here.  Since it applies beyond 
SigC/boost::callback I think it's something worthy of extensive 
discussion on here.

Bill Kempf