[boost] Stacking iterators vs. dataflow

Subject: [boost] Stacking iterators vs. dataflow
From: Phil Endecott (spam_from_boost_dev_at_[hidden])
Date: 2008-09-03 08:36:23

I just noticed this in the "lifetime of ranges vs. iterators" thread 
(which I've not really been following):

Arno Sch?dl wrote:
> rng | filtered( funcA ) 
>     | filtered( funcB ) 
>     | filtered( funcC ) 
>     | filtered( funcD ) 
>     | filtered( funcE )

I thought it worth pointing out the similarity, and also the 
difference, between this and the proposed dataflow notation.  Here, 
operator| is being used like a shell pipe operator.  In dataflow, 
operator| has a quite different meaning: it's a vertical line, 
distributing the output of "rng" to the inputs of the funcs in 
parallel.  Confusing, perhaps?  Anyway you could presumably write 
something like

   rng >>= funcA >>= funcB ....

and I would be interested to hear how the two implementations compare.  
Is it true to say that stacked iterators implement a "data pull" style, 
while dataflow implements "data push"?

I also note that Arno wants to use stacked iterators because this alternative:

  result = fn1( fn2( fn3( fn4( huge_document ) ) ) );

creates large intermediates and requires dynamic allocation.  Again, a 
framework that allowed buffering of "sensible size" chunks and 
potentially distributed the work between threads could be a good solution.

Phil.