Re: [boost] [review] Dataflow Review starts today, September 1st

From: Phil Endecott (spam_from_boost_dev_at_[hidden])
Date: 2008-09-02 17:53:33

> The review of Stjepan Rajko's Dataflow library starts today, September 1st, 
> and will run until September 10th.

[snip]

> While the Dataflow library contains other layers, only the
> Dataflow.Signals layer is ready for review.  Reviewers are welcome to
> provide feedback for any part of the library, but please be aware that
> the documentation and implementation for the other layers may be
> lacking (for example, there is the generic support layer, which
> provides concepts applicable to different dataflow frameworks, and can
> be used to develop generic dataflow code, as well as the
> Dataflow.Blueprint layer which provides run-time reflection and
> modeling of dataflow networks in a Boost Graph Library graph for any
> dataflow framework with implemented Dataflow library support).  For
> the time being, please consider these other layers as implementation
> details or proof-of-concept examples, as appropriate.

[snip]

> - Do you think the library should be accepted as a Boost library? 

Is the work proposed for inclusion in Boost in its current form?  (In 
which case, we should take into account its scope and reject it if we 
consider it too limited.)  Or are we to review this as a first part of 
something, with subsequent parts to be reviewed separately before 
anything is finally accepted?  For example, am I right in thinking that 
the Signals component is not even usable without the Generic component, 
which is not being reviewed?  Perhaps the review manager could comment.

A couple of initial thoughts, to provoke discussion:

- I hope everyone has seen the 2-dimensional "ASCII Art" dataflow 
graphs using operator overloading:

     // ---Connect the dataflow network -----------------------------------------
     //
     //                  ,---------.
     //                  | control | -----------------------+
     //                  `---------'                        |
     //                       |                             |
     //                       v                             v
     //   ,-------.     ,-----------.                   ,-------.
     //   | timer | --> | generator | -+--------------> 0       |    ,---------.
     //   `-------'     `-----------'  |                |  mux  | -> | 
display |
     //                                |  ,--------.    |       |    `---------'
     //                                +->| filter |--> 1       |
     //                                   `--------'    `-------'
     //
     // -------------------------------------------------------------------------
     timer >>= generator | mux.slot<0>()
                         | (filter >>= mux.slot<1>());

     mux >>= display;

     control.value_signal >>= generator;
     control.select_signal >>= mux.select_slot();

My comment: if the operator overloading were good enough, you wouldn't 
need the picture in the comment block.  Since it isn't good enough, why 
not stick to connect():

     connect(timer,generator);
     connect(generator,mux.slot<0>);
     connect(generator,filter);
     connect(filter,mux.slot<1>);
     connect(mux,display);
     connect(control.value_signal,generator);
     connect(control.select_signal,mux.select_slot);

- The hard aspects of this sort of thing are buffering, distributing 
work over threads, and so on.  I don't think you have any of that; the 
components always process one datum at a time.  As it happens I've 
recently written some code that grabs an image from a web cam, 
annotates it with some text and shows it on the screen - very much like 
your example.  This was using the video4linux2 API.  To get a 
respectable framerate you need to queue up a number of frame requests 
in the kernel, but if you queue too many you'll get too much latency.  
I honestly can't say that I feel this library would have made this code 
any easier to write, or given any better results.  I wonder if there 
are more complex motivating examples that would better illustrate it?

Regards,  Phil.