Re: [Boost-users] [dataflow.signals][thread_safe_signals] How to alter the Calling of chained signals

From: Stjepan Rajko (stipe_at_[hidden])
Date: 2008-08-29 00:10:30

On Thu, Aug 28, 2008 at 1:19 PM, Manuel Jung <gzahl_at_[hidden]> wrote:
>
> I checked out the whole dataflow from sandbox. Example compiles and runs
> fine.
>

Great!

>> [snip]
>>
>> // our network
>> //  increase >>= increase2 will be in its own thread
>> //  increase3 will be in its own thread
>> //  increase4 will be in its own thread
>> source
>>     | (make_delay(pool, increase) >>= increase2)
>>     | (make_delay(pool, increase3) >>= make_delay(pool, increase4));
>>
>> // submit the first task
>> submit(pool, source);
>>
>
> This seems very usable. I also like that you are able to add filters to
> different pools.
>

Yeah, that turned out to be a nice advantage.

>>
>> Like you said, delay might not be the best name, but works for now.  A
>> couple of comments:
>>
>> * The example is hard-wired to a void(int) signature.  It wouldn't be
>> hard to generalize, if there is interest.
>
> I need to have it generalize. Would be great if you could do it, otherwise,
> i'll try at the weekend.
>

It just need slight tweaks in things that are already there.  I might
not find time until Sunday, but once I get to it it shouldn't take me
long.

>
> Hmm. thread_safe_signals are needed too. There will be new signal
> connections after the threading.
>

Frank Mori Hess has integrated a change that I believe is the only
change necessary to make thread_safe_signals work with
Dataflow.Signals.  When I work on the generalization, I will also test
it with thread_safe_signals.

>>
>> * I don't use chained_submit - instead, the delay object will do a
>> regular submit when it gets its input.  This was more in line with the
>> push-oriented design of Boost.Signals.  To use chained_submits and
>> futures would require either using a pull-based network (each
>> component taking in the input as the argument and returning the
>> result), or propagating the final result back through return values.
>>
>
> That is ok with me. I would prefer all the tasks to be added at the
> beginning (because of an easier priority handling of the queue), but i can
> this way too, i think.
>

To do the chaining at the beginning, we would need have access to the
network graph.  We can have that using the Dataflow.Blueprint layer,
so maybe it's something we can explore in the future.

>
> Seems very great so far. This way you can do late evaluated dataflow
> networks :-). Just create a pool and make a dataflow network (even a
> cyclic! :-)) and you could evaluate it step for step, depending by the
> threadpools queueing pollicy.
>

Yeah, this was a really good (and concrete) suggestion, thanks for
bringing it up.  I'm excited about the cyclic possibilities too :-)

> Thanks for what you did so far :-). I like the dataflow lib. Can i help
> writing some code, the generalization for all data types or something else?
> I just don't know the inside of the dataflow library that good, it would
> take much longer, if i try to do it... but im of course willing to help.
> Maybe an other example?
>

I really appreciate your offer to help.  The Dataflow.Signals layer is
coming up for review on September 1st, so it would be great if we
could continue developing and discussing this example through the
review period.  As someone that has used / is using the lib (or at
least looking at using it), your feedback would be very much
appreciated and helpful during the review!

If you're interested in joining the development, I have recently
started a rewrite of the Dataflow library (the rewrite is to address
some things that I already know I want to redesign as well as any
feedback received during the review).  I am doing the rewrite in a
very step-by-step fashion, starting with the library foundations and
only moving on when things are properly designed and documented.  I
can keep you informed of the progress if you'd like, and you would be
more than welcome to join in at any point.

Best,

Stjepan