Subject: Re: [boost] GSoC: Boost.Pipeline -- scheduling of segments
From: Benedek Thaler (thalerbenedek_at_[hidden])
Date: 2014-07-04 05:52:58
On Thu, Jul 3, 2014 at 2:27 AM, Adam Berkan <adam.berkan_at_[hidden]> wrote:
> I was suggesting that the thread would grab a single piece of data from a
> queue, do a single transformation, and then return where it would make a
> scheduling decision about where to go next. Then it would once again grab
> one piece of data and do one transformation before returning to scheduling.
In case of 1-1 transformations, I think this is done in the `development`
branch, altough the really interesting issues are not taken care of... ->
> I'm not suggesting that we would interrupt a thread in the middle of a
> stage... That would require writing most of an operating system kernel :)
> The tricky part is knowing where there's work to be done. The "smart
> queues" would have to update some stats in the scheduler on every push, and
> potentially wake up threads. It's certainly a non-trivial task.
-> like this. Now I think we are on the same page regarding such queues or
> There's also the issue of the multi-in (N:1 & N:M) transformations, i.e.
> that take input queues. They would either have to take a whole thread,
> provide some kind of yield, or something else. I'd probably ignore this
> for now (just consume a thread) and focus on the simple 1:1 and 1:N cases
> as they naturally return after each transformation.
I think only 1:1 is an easy case. Even 1:N has an issue: consider a
transformation which duplicates its input. What if the second element
cannot be pushed to the downstream queue because it's full? The
transformation should yield (and not spin).
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk