Subject: Re: [boost] Flow-based programming library for Boost?
From: Julian Gonggrijp (j.gonggrijp_at_[hidden])
Date: 2012-12-09 09:18:42
Marcus, could you please write inline replies and zap all parts that
you don't reply to? Thanks in advance.
Marcus Tomlinson wrote:
> Hi Julian, thanks for the post. I see a pattern forming :) I think I need to make my documentation clearer in a number of places.
Suggestion: describe for each member function what it does. And give a
shorter, more conceptual overview at the front page of how the library
works and why.
Note that you find the need to expand upon your documentation while
it's already quite beefy. Some of your classes have lots of public
methods. These are clear signals that your library is not simple
> Ok, in DSPatch, circuits provide 2 main purposes: encapsulation of a component network, and parallel processing control (see more on how parallel processing is done see: http://www.adaptaudio.com/DSPatch/spec_page.html).
From that page I gather that there may be threads specific to a
component as well as "general" (circuit) threads that take up work
from each component in a circuit in turn. What is the motivation for
this design? How do you avoid having two threads execute the same
component's job on the same inputs, especially if one is a component
thread and one is a circuit thread?
> Components don't have to be added to a circuit to be routed together, this is just recommended due to the advantages (mentioned above) gained in this approach.
Well that is good news. Am I right to assume that the circuit object
is not necessary to enable multithreading?
> Components have ConnectInput / DisconnectInout methods that can be called directly in order to create component networks outside of a circuit.
I looked up those methods in the class reference. IIUC you have to
"extract" the wire from the source component and manually assign it an
index or name. Why not just take a wire object as the sole argument
and give it an index automatically? In addition you can connect only
one wire at a time. Why not offer a ConnectInputs method that takes a
collection of wires?
Actually I find this interface really worrying because it seems to
suggest that a component needs to know about other components in order
to receive input from wires. On the specification page I read that the
/wires/ need to know about the components as well. "Separation of
concerns" is not being fully applied here. I would strongly prefer a
more generic interface where wires can operate regardless of whether
they're connected to components and vice versa.
> You are correct, there isn't a way to extract multiple values from a single input. This is because each component input can only accept one wire at a time. When another wire is connected to an input that already has a connected wire, that wire is replaced with the new one.
1) why can an input accept only a single wire at a time, and
2) why can't wires transport multiple values at a time?
> This is not the case with outputs though. One output can be distributed to multiple inputs. This is definitely something that should be explained explicitly (I'll add it to the list).
From the tutorial it was quite clear to me that this would be
possible, but it appears that the user would have to implement this by
themself every time. There is no ready facility to abstract and
automate it. Besides, given that components can extract only a single
value from each input at a time, the possibility to output multiple
values at once is only marginally useful.
> I don't see dynamic typing as a limitation, I see it as quite useful. Lets say your processing an input stream of audio samples. With dynamic typing, you can adjust the audio's sample size at run-time and have the component input adapt dynamically without breaking flow.
You can do polymorphism in safer, more or less "static" ways. Consider
> I also have to disagree with you that object-oriented design has nothing to offer to dataflow programming, but perhaps this is just one man's opinion against other.
Of course it's partly a matter of taste. However, object-oriented
design does incur overhead. If dynamic binding or superfluous pointer
dereferencing is involved in the chain that transports a value from
one component to another (output-signal-wire-signal-input?), it may
well be significantly slower than it could be, especially since every
value has to be transported in a separate invocation of the chain.
Apart from the overhead, OOD does not easily permit the same powerful
modularity that generic programming offers. I think this explains most
of the limitations in your library. I also think it explains why OOD
is largely out of fashion in Boost libraries, in favour of generic
OOD is a useful tool with some uncontroversial use cases, such as game
engines. However it should never be a goal in itself. Some areas
benefit more from a generic design, such as containers and algorithms.
As far as I'm concerned dataflow programming is more like the latter.