Re: [boost] [rfc] getting the dataflow library ready for submission

From: Roland Schwarz (roland.schwarz_at_[hidden])
Date: 2007-11-12 06:00:22

Stjepan Rajko wrote:
> I am getting close to having the Dataflow library ready to be
> submitted for review, and would like to respark some interest.  The
> current version of the library offers a generic layer for dataflow
> programming that can be applied to various data transport mechanisms,
> and a Dataflow.Signals support layer for Boost.Signals that provides a
> number of general-purpose components and easy connectability.

I just skimmed over the docs, and I have some questions:

1) How would I specify the control flow? I.e. How would I choose between
    "source driven" and "sink driven"?

2) Is there a mechanism that will give support for block oriented
    processing, or will a function be called on every data item?
    To clarify: I do not mean a couple of primitive data items blocked
    together into a larger container which is itself a data item on
    its own, but something like the fread() function is able to do:
    *) request a number of items
    *) return as much as currently available (may be less than wanted)
    *) block process what has been received
    *) wait for more data

You are using VTK as an example and so I suspect you are using the
demand driven processing in your library. Are you also providing
a caching and timestamping mechanism to hold intermediary results
that need no updating?

Which special concepts for memory management are you suggesting (if
at all)? Memory management is one of the crucial parts of data flow
programming. Will you provide a single copy mechanism?

I always found that in data flow paradigm these are the hard parts.
It is not the hard part to come up with a "nice syntax".

Roland aka speedsnail