Re: [boost] Boost.Pipeline -- scheduling of segments

Subject: Re: [boost] Boost.Pipeline -- scheduling of segments
From: Benedek Thaler (thalerbenedek_at_[hidden])
Date: 2014-07-10 13:17:00

On Thu, Jul 10, 2014 at 5:42 PM, Hartmut Kaiser <hartmut.kaiser_at_[hidden]>
wrote:

>
> > > I've put together some slides to show what I'm thinking of and make
> sure
> > > we
> > > are on the same page:
> > > http://www.slideshare.net/erenon/boostpipeline-scheduling-of-segments-
> > > 36832455
> > >
> > > To point here, assuming a pipeline, one can use extra information to
> > make
> > > some educated guesses regarding scheduling: Yield to offending queue
> > > (upstream if input is empty, downstream if output is full), choose
> > > specific
> > > segment to optimize latency.
> > >
> > > Probably this can be achieved using fibers, or even with coroutines.
> > >
> > > Due to the complexity of HPX I'm not sure how does it compare.
> >
> > Huh? Complexity? It's 100% API compatible to C++11 and various proposals
> > to C++14/17.
> > I'd like to know more about why you think HPX is complex.
>
> I added a much simplified implementation of what you described in your
> slides. Most of the complexity lies in the operator&, not it the HPX code
> itself.
> [snip]
>
>
> Thanks for showing this sample, it's definitely very clean! By complexity,
I meant internal details I couldn't cover yet, not something silly in the
API.

Three things come to my mind regarding this example:
 - It **seems** every segment creates a future instead of pushing the
result into a bounded queue. Possibly the later would be faster.
 - The segments never yield. Is there a complete scheduler underneath?
 - By launching tasks using hpx::async, it seems the scheduler can't have
any notion of the pipeline, which makes efficient scheduling harder.

Sorry for my ignorance, I can't answer these questions just by glancing at
the docs.

Thanks,
Benedek