Re: [boost] Flow-based programming library for Boost?

Subject: Re: [boost] Flow-based programming library for Boost?
From: Vicente J. Botet Escriba (vicente.botet_at_[hidden])
Date: 2013-01-04 04:46:58

Le 04/01/13 08:51, Marcus Tomlinson a écrit :
>>> * You use a bus of input/output untyped signals. Why the signals are not
>>> typed?
> 1. Flexibility - inputs and outputs can dynamically accept changing signal types. E.g. Varying sample size in an audio stream.
I don't know why this couldn't be modeled with a vector signal 
containing the samples. Maybe you have other examples needing untyped 
iunterfaces.
>
> 2. Less coding, more expandability - the component designer need not specify the types required at each input / output when configuring IO, only how many. Consider the situation where you would like to update a component to accept int instead of bool for one of its inputs. This would (in your implementation) require you to edit multiple sections of code. With DSPatch, you need only edit / add to the process method.
Well, my example is just a draft that can be polished. In particular the 
library could provide a macro that generates all this stuff from a free 
function.

bool And(bool a, bool b) { return a && b);

DSP_GENERATE_COMPONENT(And, DspAnd);
>
> 3. Allow for generic virtual base process method (explained below).
>
>>> * Why do you need that the process function be virtual?
> This is due to the way the circuit works in DSPatch. When a component is required to process, the circuit needs a way of triggering it without having to know anything about the specifics (input / output count, types etc.) of that particular component. A generic inherited virtual method guarantees this kind of catch-all interface.
Ok I think I understand the need.
>
>>> // 1. Derive component class from DspComponent
>>> // ===========================================
>>> class DspAnd : public DspComponent<DspAnd, InPort<bool>,InPort<bool>,
>>> OutPort<bool>>
>>> {
>>> typedef DspComponent<DspAnd, InPort<bool>,InPort<bool>, OutPort<bool>>
>>> base_type;
>>> public:
>>> // 2. Configure component IO buses
>>> // ===============================
>>> DspAnd(DspComponentBase& parent, Signal<bool>& x, Signal<bool>& y,
>>> Signal<bool>& o)
>>> : base_type(parent, x, y, o)
>>> {
>>> }
> These requirements for component construction are quite overwhelming. There is a lot to remember (and type) every time you need to create / edit a component. Besides, as mentioned before, flexibility is paramount with DSPatch. This kind of static IO configuration goes against DSPatch's design (more on this later).
See the macro approach.
>
>>> protected:
>>> // 3. Implement  a non-virtual operator() method
>>> // ======================================
>>> bool operator()(bool x, bool y)
>>> {
>>>    return x && y;
>>> }
>>> };
> Having to keep track of every component input and output (count, types, etc) in multiple places (constructor, operator(), and in external use) seems overly complex and difficult to expand on / maintain. And again, static IO configuration is quite limiting.
See the macro approach.
>
>>> void main()
>>> {
>>> // 1. Create a DspCircuit where we can route our components
>>> // ========================================================
>>> DspCircuit circuit;
>>> // 2. Create the internal signals
>>> DspSignal<bool> i1, i2, o;
>>> // ========================================================
>>> // 3. Create instances of the components needed for our circuit and
>>> connect the signals and the ports
>>> // ============================================================
>>> DspRandBool randBoolGen1(circuit, i1);
>>> DspRandBool randBoolGen2(circuit, i2);
>>> DspAnd logicAnd(circuit, i1,i2,o);
>>> DspPrintBool boolPrinter(circuit, o);
> Firstly, I like how the signals are referenced externally and are common between all components. This is something I need to improve on in the circuit class. ATM there is a fair amount of data copying going on.
Well, maybe more that if the connection were done to the producing 
component, but this let the possibility that two components can produce 
the same signal (of course only one at a time). On the domain I'm 
working this is a real need. Maybe your dynamic reconfiguration can take 
it in account.
>
> I assume that in your idea of this type of implementation, there is still a means of disconnecting and reconnecting signals at a later point? Wiring and rewiring at runtime is a large emphasis of the DSPatch design. I'm not sure wiring at construction is all that important though -I especially don't think it should be mandatory.
No I don't take the possibility to disconnect/reconnect signals/ports 
dynamically in account. I need to think a bit on that. Maybe an 
alternative is for a component to ignore a signal at runtime.
>
> This looks like the kind of code you'd find in other dataflow libraries. However, this is exactly why I designed DSPatch, and why I designed it the way I did. In my opinion, the code is convoluted (doesn't read well). But alas, I fear that most people actually prefer this kind of syntax over mine.
Syntax preference goes after covering all the needed functionalities. So 
I think that you will need to describe the needs you try to cover so 
that people like me don't request why you are doing things like you do ;-)
>
>>> Components with several outputs should define a operator() returning the
>>> tuple of outputs.
> Much like your implementation of input configuration, tuples have to be defined at compile time and hence can't be adjusted at a later point. By design, DSPatch avoids static IO, in fact it tries to avoid anything that is static. Not only does DSPatch allow you to adjust IO types and runtime, you can add / remove IO dynamically too. E.g. An AI minimax tree where each node dynamically creates a new output, and a feedback input for each possible player move from current position.
>
> It may be worth mentioning (again) that DSPatch was originally designed for audio DSP applications (hence the name). Therefore runtime features like branch synchronization, dynamic routing, and adaptive IO signal types are all part of the core design requirements. These features then subsequently cover a very large range of applications, allowing for small, simple static circuits, to large, complex dynamic circuits.
>
> I hope this helps you understand why DSPatch is designed the way it is.
>
Yes it helps a lot. I will try to see how disconnection of signals could 
be covered by a static design.

It would be great if the documentation describes this design rationale 
and show examples where this dynamic needs are evident and why it is 
impossible if not difficult to do it with a static model.

Best,
Vicente