Proto for Boost mailing list page: Re: [proto] Streamulus v0.1 is out: An EDSL for Event Stream Processing with C++

Subject: Re: [proto] Streamulus v0.1 is out: An EDSL for Event Stream Processing with C++
From: Irit Katriel (iritkatriel_at_[hidden])
Date: 2012-06-24 11:50:29

On 24 Jun 2012, at 03:47, Dave Abrahams wrote:
> 
> Well, I think the hello world example is too simple to illustrate what
> this does, and the blog posting is TL;DR, but I skimmed it, and still
> didn't really have a clue.  Have you looked at
> http://www.boost.org/doc/libs/1_49_0/doc/html/accumulators.html?  It
> seems to have some overlap with the problems you're solving.
> 

I was not aware of accumulators, and I agree that there is some overlap. As far as I can tell, the main differences are as follows. Please correct me if I misunderstood anything.

In the accumulators library, all the accumulators are invoked for every update to the input. This is why the visitation order can be determined at compile time.  I am building a dependency graph that is used at runtime to determine which nodes need to be activated, so that inputs only propagate through the part of the expression that may be affected. Obviously I am imagining large expressions on many inputs, so that this is worthwhile doing.

In addition, I am trying to achieve a more programming-like syntax for complex expressions, by making the expression (rather than the accumulator type) encode the dependencies between nodes. 

Example: you want to compute f(g(x),h(x)) over a stream. With streamulus you define each of the functions f,g,h, and when you subscribe the expression f(g(x),h(x)), and only then, f learns that its inputs are the outputs of g and h. With accumulators you would need to define the accumulator f_of_g_and_h(), make it depend on g and h, give it a tag, and then add it to the accumulators list. 

Similarly, with streamulus you can define your function f, and then use it multiple times within the same expression:  sqrt(sqrt(x)). This will build a graph with three nodes : (x--> sqrt-->sqrt) where x is an input node and the two sqrt's don't need to know or care about each other because a different instance was constructed for each node.   

I'm sorry about the too-long blog post.. It mostly deals with the bad things that can happen if you DIY your stream computations. You can skip most of it and just look at the definition of the sample application in the beginning, and then the streamulus solution in the end. 

Irit