Re: [boost] Stacking iterators vs. dataflow

Subject: Re: [boost] Stacking iterators vs. dataflow
From: Stjepan Rajko (stipe_at_[hidden])
Date: 2008-09-03 15:48:48

On Wed, Sep 3, 2008 at 11:13 AM, Phil Endecott
<spam_from_boost_dev_at_[hidden]> wrote:
> Stjepan Rajko wrote:
>>
>> The direction indicated by >>= aligns with the direction of the signal
>> (function call), but the data can flow in either way (either sent
>> forward in the function call argument, or sent back through through
>> the return value).  So, you could do
>>
>> rng >>= funcA >>= funcB
>>
>>  or
>>
>> funcB >>= funcA >>= rng
>>
>> depending on how the func and rng components are implemented.
>
> That seems a bit odd; this is a DATA-flow library, so I would expect the
> direction of the symbol to describe the direction of the data flow.  And
> isn't the data passing method a detail of the implementation (in this case
> Signals) that your Generic layer should be hiding?
>

The data flow with signals is bidirectional in the general case.

For example,

float fn(int x, int &y, int &z)
{
  y = x + y;
  z = x + 1;
  return y*2;
}

... has the following flow of data:
 * data flows in through x and y (and z, but there it is ignored)
 * data flows out through y, z and the return value.

Since Dataflow.Signals uses Boost.Signals which uses function calls,
Dataflow.Signals is inherently a bidirectional framework.  To make
things simpler, most of the Dataflow.Signals documentation (and
indeed, many of the provided components) is geared towards a
simplified push-based approach where the data flows left to right
(with left and right as in connect(a,b) or a >>= b).  But, as
indicated above, data could just as easily flow right to left, or in
both directions, in which case a >>= b becomes misleading.  I could
provide <<= as well, so the user could use a <<= b to indicate data
flowing right to left.  Or, the user could just always use
connect(a,b).

I think that when the flow of data becomes complicated (as in the fn
function above), the dataflow programming paradigm has a good chance
of turning into a headache and should probably not be used.  That is
why I tend to limit the discussion in the documentation to purely
push-based or purely pull-based networks in the case of
Dataflow.Signals.

> But I'm not sure I understand you; here rng is a data source and (surely) it
> supplies values via its return value; how can it be implemented to supply
> values via a parameter?
>

I'm sorry, I have a tendency to throw up syntax and neglect to explain
the semantics :-(.  I was assuming that rng in rng >>= funcA >>= funcB
was a Boost.Dataflow component, as the input component in this
example:
http://www.dancinghacker.com/code/dataflow/dataflow/introduction/dataflow.html

...or the generator (used in combination with timer) here:
http://www.dancinghacker.com/code/dataflow/dataflow/signals/introduction/examples/gil.html

>> As far as the dataflow library goes, some sort of a "automatic task
>> division" library would indeed be great in conjunction with dataflow,
>> but I see this as orthogonal to dataflow.
>
> It doesn't have to be _automatic_ (i.e. runtime) task division; just some
> way of running some components in their own threads.  The
> thread-safe-signals work that you linked to before may be sufficient for
> this, though I don't know enough about Boost.Signals to fully understand it.
>

I will try to expand that example and add some documentation /
comments to make it clearer.  It would be a good thing to discuss as a
part of the review.

Kind regards,

Stjepan