Re: [boost] Flow-based programming library for Boost?

Subject: Re: [boost] Flow-based programming library for Boost?
From: Julian Gonggrijp (j.gonggrijp_at_[hidden])
Date: 2012-12-10 18:12:22

Marcus Tomlinson wrote:

> Thanks again for taking the time to look at my code Julian, it's a big help :)

May I compliment you for being so patient with my comments. :-)

> I've added a sort overview to the introduction section, I think this should help.

It looks like a good start. I'm still wondering about how the thread 
scheduling is actually accomplished (how do threads obtain work?). 
Would it be possible to provide some high-level understanding of that 
matter in a single paragraph?

> [...] -you have the option of multi threading via a circuit object, or directly via the components themselves. However, as I have just rediscovered, the interconnected, free-running components approach is currently incomplete. I will get onto finishing this next. Good observation! Thanks :)

I was not really aware of the missing feature, but I'm glad my 
questions helped you find it.

>> IIUC you have to "extract" the wire from the source component and manually assign it an index or name. Why not just take a wire object as the sole argument 
>> and give it an index automatically? In addition you can connect only 
>> one wire at a time. Why not offer a ConnectInputs method that takes a 
>> collection of wires?
> 
> I'm not sure I understand what your suggesting. Should you be able to instantiate a wire object externally and pass it in a an argument to an AddWire() method in the component? I think I know where you're going with this (and the multiple wire thing), the problem is that I don't assume anything wrt input and output signals. It is not assumed that one component's outputs all connect 1:1 as a bus to another component's inputs, it's also not assumed that the signal name of an output corresponds to the equivalent signal name of an input. I see this as flexibility, but perhaps it could be perceived and overly complex.

I'm afraid we're simply not on the same page with relation to this 
matter. I don't really understand what you're saying here, either. 
Perhaps we can return to this point later, or perhaps somebody else 
reading our discussion is better able to comprehend what's going on 
here and willing to help us.

>> 1) why can an input accept only a single wire at a time and
>> 2) why can't wires transport multiple values at a time?
> 
> What would be returned from a call to the input's GetValue() method in a case where there are multiple wires connected to it, or where there are multiple values assigned?

A tuple? To be honest I think my first question was misguided because 
the fact that input pins are connected only to a single wire is not 
preventing components from receiving multiple values at a time. Still, 
a tuple seems a sensible way of bundling several simultaneous inputs 
together.

> Remember, you can transfer arrays over wire link -still regarded as single values. Otherwise, do you suggest there be the option to continually call GetValue() on an input until all wires / values are processed?

Well yes, I think something like that would be very useful in a 
dataflow library. On the condition that there's also an option to 
continually push values to an output pin, or course.

> When i say an output can be distributed to many inputs, I mean that one outgoing signal can be received by multiple components. An output simply provides its current produced value to any component that wishes to use it. An example of this could be a global sample rate provider to many audio processing components.

Accepted. What I meant is sending output values to different receivers 
in turn, ideally depending on which one is ready to process it. I 
believe both interpretations (i.e. yours and mine) of "distributing 
output" should be provided by a dataflow library.

> I think I know where the confusion is coming from. In DSPatch, the terms input/output, and signal are interchangeable. Input/output bus = signal bus.

Perhaps this is also something to clear up in your documentation. It 
mentions wires, signals, pins and buses. It would be good to eliminate 
all alternative names and make explicit what exactly consists 
of/connects to/transports what. It's especially unclear to me whether 
a signal is a value wrapper transported by a wire, or some kind of 
intermediate interfacing object between a wire and a component, or 
still something else.

> I think a good example of where OO dataflow has been extremely successful is in Steinberg's VST framework.

I'm not knowledgeable about VST nor did I find a quick way to become 
sufficiently knowledgeable about it, but my web search does seem to 
indicate that VST is successful. That might be thanks to OOD, in spite 
of it or anything in between, but I'll take your word for the first 
being the case.

Regardless, I believe one could also write a dataflow library without 
OOD, specifically by a generic design, and that such a library could 
be very elegant and efficient. I would prefer such a library over an 
OOD one. Admittedly what I'm suggesting now is vapourware until I 
describe what I have before my inner eye. I will do so as soon as I 
can.

-Julian