Re: [boost] [signal network] SoC draft of Signal Network proposal posted (based on recent "signal flow" inquiry)

From: Stjepan Rajko (stipe_at_[hidden])
Date: 2007-03-19 21:36:34

Hi Tobias,
> So basically we'll want to chain routines, that encapsulate tight loops
> that work on buffers based on control input.
>
> The framework does not have to provide (at least not as a core
> component) or know about buffers, but rather implement memory management
> for arbitrary objects. From the framework's perspective there should be
> no distinction between buffers and control signals, but different
> properties of the individual /pins/ (I'm using this term for the
> sources&sinks here) and the /links/ between them, that control how the
> framework handles the associated data entities.
>
> Connecting two processing nodes would simply set source and sink data
> pointer to the same memory location. Eventually (see below) we also have
> to set up a notify mechanism. Boost.Signal might be the way to go, but
> I'm not at all sure at this point.

With the library as it stands, it is possible to send references /
pointers to objects through the signals, which could be used to work
on shared buffers (although I see that this is slightly different from
what you are proposing).  But, it would be up to the objects to figure
out how to share - introducing functionality that would help with
shared buffers would definitelly help with signal processing apps that
need to be fast.

As another performance boost, it might be possible to offer "fast"
versions of the provided components, which would use a boost function
instead of a boost signal to handle the output.  The benefit is
increased speed, and the drawback is losing all of the benefits of
Boost.Signals (including connecting multiple sinks to the same source
directly).

> So we get three basic pin types:
>
>    - in         // processing node reads data from it
>    - out        // processing node writes data to it
>    - in_out     // processing node changes data in it

With the Boost.Signals base (or even Boost.Function), the "in" pin can
be any function, and receiving data is analogous to the function call.
 "out" is a boost signal, and basically calling the "out" pin results
in all of the connected "in" pins (functions) to be called.  In_out
functionality can be achieved by passing a reference or pointer.

If I understand you correctly, you are envisioning something that
makes "the wire" an object in itself - a buffer which can be placed in
between in and out pins, or something manipulated by an in_out pin.
Making connections would then, instead of setting up future function
calls which pass the signal data in arguments, be connecting the
components to the buffers (through a one-time notification like
"here's your buffer").  Then, executing the network would consist of
executing each component in the appropriate order.  Is this correct?

>
> I could go on like this for a while and outline my image of how such a
> framework should look like (which is pretty clear by now - I've been
> working on a design for such a thing for several years, every once in a
> while) but I'll stop here, for now. Let's first see whether our plans
> are compatible until here. Maybe you're up to something totally else...

I think we're interested in very similar things.

>
> Further, I think we don't need an overloaded operators interface. Don't
> get me wrong, I'm not the "oh no - not so much tricky template stuff"
> kind of guy (in fact, I like template hacking a lot), but I believe the
> power of this thing lies in the runtime world.

I like the operator interface because you can connect objects (or
change / append the network) using something that almost looks like a
diagram.  But that's just my preference :-)

> Serialization and a careful implementation that allows to extend things
> at runtime by loading shared object files is much more important, than
> fancy compile time setup, IMO. So is multi-threading - ideally the
> framework is thread-aware and self-optimizing.

What do you mean by serialization?  Yes, thread-aware is very
important.  Self-optimizing sounds great but I'm not sure what all
exactly it can mean and in what all ways it could be tackled.  I think
as we extend the library towards multi-threading I'll be more aware of
what the possible optimizations are.

> Also note that I'm not only talking about a signal processing framework
> but something more general (signal processing gives an interesting use
> case, though). One could also use it to wire up the data flow in a batch
> system, resolve GUI dependencies, create vector graphics or use it as a
> general purpose programming system.
>
> As you can see, I'm potentially interested in collaborating on this
> project, if I haven't scared you off already ;-).

I do think we are definitelly looking at the same sort of beast :-)
Also, I'd be keen to collaborate.  I'm going to try to sumbit this as
a SoC proposal, and if it gets accepted I think I'll have to be
careful to delineate what constitutes the project that I am to
complete from things to collaborate on.

Have you given the current version of the library a try?  If you find
that there is a way in which it lets you do the things you are
interested in (or something similar), we can use it as a starting
point.  Otherwise, we can try to rethink the design.  If you have any
of the code you've been working on available, I'd be happy to try it
out and do the same.

Cheers,

Stjepan