Re: [boost] Flow-based programming library for Boost?

Subject: Re: [boost] Flow-based programming library for Boost?
From: Marcus Tomlinson (themarcustomlinson_at_[hidden])
Date: 2012-12-10 15:48:01

Thanks again for taking the time to look at my code Julian, it's a big help :)

> Suggestion: describe for each member function what it does. And give a 
> shorter, more conceptual overview at the front page of how the library 
> works and why.
> Note that you find the need to expand upon your documentation while 
> it's already quite beefy. Some of your classes have lots of public 
> methods. These are clear signals that your library is not simple 
> enough.

I've added a sort overview to the introduction section, I think this should help. In terms of the method descriptions / number of public methods, I'm gonna have to tackle that over a weekend I think, both valid points!

>> From that page I gather that there may be threads specific to a
> component as well as "general" (circuit) threads that take up work 
> from each component in a circuit in turn. What is the motivation for 
> this design? How do you avoid having two threads execute the same 
> component's job on the same inputs, especially if one is a component 
> thread and one is a circuit thread?

> Am I right to assume that the circuit object 
> is not necessary to enable multithreading?

Component threads and circuit threads don't interact, they perform completely different tasks. Component threads even act on physically different memory than the circuit threads do (so access violations are not a concern). The motivation for this design is actually the answer to your second question -you have the option of multi threading via a circuit object, or directly via the components themselves. However, as I have just rediscovered, the interconnected, free-running components approach is currently incomplete. I will get onto finishing this next. Good observation! Thanks :)

> IIUC you have to "extract" the wire from the source component and manually assign it an index or name. Why not just take a wire object as the sole argument 
> and give it an index automatically? In addition you can connect only 
> one wire at a time. Why not offer a ConnectInputs method that takes a 
> collection of wires?

I'm not sure I understand what your suggesting. Should you be able to instantiate a wire object externally and pass it in a an argument to an AddWire() method in the component? I think I know where you're going with this (and the multiple wire thing), the problem is that I don't assume anything wrt input and output signals. It is not assumed that one component's outputs all connect 1:1 as a bus to another component's inputs, it's also not assumed that the signal name of an output corresponds to the equivalent signal name of an input. I see this as flexibility, but perhaps it could be perceived and overly complex.

> Actually I find this interface really worrying because it seems to 
> suggest that a component needs to know about other components in order 
> to receive input from wires. On the specification page I read that the 
> /wires/ need to know about the components as well. "Separation of 
> concerns" is not being fully applied here. I would strongly prefer a 
> more generic interface where wires can operate regardless of whether 
> they're connected to components and vice versa.

Good point, I will look into upgrading the DspWire structure to a smarter class that isolates dependency and exposes safe methods for signal retrieval. On conception I did feel that this was going to have to change at some point.

> 1) why can an input accept only a single wire at a time and
> 2) why can't wires transport multiple values at a time?

What would be returned from a call to the input's GetValue() method in a case where there are multiple wires connected to it, or where there are multiple values assigned? Remember, you can transfer arrays over wire link -still regarded as single values. Otherwise, do you suggest there be the option to continually call GetValue() on an input until all wires / values are processed?

>> From the tutorial it was quite clear to me that this would be
> possible, but it appears that the user would have to implement this by 
> themself every time. There is no ready facility to abstract and 
> automate it. Besides, given that components can extract only a single 
> value from each input at a time, the possibility to output multiple 
> values at once is only marginally useful.

When i say an output can be distributed to many inputs, I mean that one outgoing signal can be received by multiple components. An output simply provides its current produced value to any component that wishes to use it. An example of this could be a global sample rate provider to many audio processing components. I think I know where the confusion is coming from. In DSPatch, the terms input/output, and signal are interchangeable. Input/output bus = signal bus.

> You can do polymorphism in safer, more or less "static" ways. Consider 
> Boost.Variant.

Will do.

> OOD is a useful tool with some uncontroversial use cases, such as game 
> engines. However it should never be a goal in itself.

I think a good example of where OO dataflow has been extremely successful is in Steinberg's VST framework.