Re: [boost] Boost.Pipeline -- scheduling of segments

Subject: Re: [boost] Boost.Pipeline -- scheduling of segments
From: christophe.j.henry_at_[hidden]
Date: 2014-07-09 15:17:47

>Well, if it's possible to use as many threads as there are segments
>(transformation), then I think it's a fairly easier problem. Making it
>possible for a thread to alternate between segments is something
>interesting. (Please correct me if I misunderstood your solution)

The simple solution is using 1 thread per segment, though it can be 1..n.
I'm unsure what you mean by migrating. In this first solution, a thread 
takes only care of one segment, so no thread handles tasks from another 
segment. Work stealing would do it.

>Regarding work stealing: I don't know how does it work, but is the order of
>the input stable across segments?

No. If you have more than one thread per segment, or use work stealing 
(where a thread from another segment which has nothing to do at some point 
will steal work from another segment according to the order where schedulers 
are defined), or a pool of threads, then it is possible that input data gets 
processed in a different order than read.
This is not specific to work stealing but likely also applies to each 
solution where more than one thread handles a segment.
It might or might not be a problem but the performance is likely to be 
higher, even if you need to reorder data afterwards to stay stable.

HTH,
Christophe