Re: [boost] Flow-based programming library for Boost?

Subject: Re: [boost] Flow-based programming library for Boost?
From: Marcus Tomlinson (themarcustomlinson_at_[hidden])
Date: 2012-12-18 01:23:35

Hi Topher, thanks for the post :)

> 1) /Simplicity is a simplistic goal/.  Generally good design means that doing simple things is simple and doing more complex things is more complex proportionately (and no more than proportionately) to the degree of additional complexity.  This may mean that there are many "components" (methods, parameters, libraries, classes, or whatever) and options but that at any given time most can be ignored.  Part of the design process is to be clear what can be ignored when, and part of the implementation is documentation (whatever form that that takes) that makes it easy to focus on what one needs for a task and to be barely aware that there is more available.

I hope that I am at least close to achieving this. DSPatch was designed with just this in mind. Even the documentation was structured as to describe only the bare essentials to using DSPatch up front, while providing the possibility for additional functionality based on those essentials.

> One of the tools for accomplishing this are careful selection early on of an explicit set of use cases.  Careful use of defaults, especially of defaults that interact intelligently with explicits (this can lead to a system that from a usage viewpoint that "just does what is expected" but which can be a bear to implement and formally describe, with lots of "unlesses" and "with this combination of factors this, and with this combination that"), as well as things like policies and pre-specified frameworks that specify lots of things all at once.

With a framework that is (or at least tries to be) generic, it is hard to describe or list use cases. There are so many uses for DSPatch that I am not aware of. I understand what you're saying though, and once again, I hope that with my tutorials and example application I am achieving this. If not, please let me know what's lacking.

> I would suggest that a library of common ways of handling multiple inputs ("tuplers", "ands", "ors", "averagers", etc. as well as the current "only one allowed"), and that these be "declared" (or, given the run-time reconfigurability "redeclared") along with the input itself.  Any wire addressed to the input actually gets attached to the combiner, and redeclaration would automatically pass the existing inputs to the new combiner.

Indeed a valid point. I have come across this very issue in my personal work where operations such as AND. OR, and AVERAGE that could be behaviors adapted by inputs themselves have had to operate as separate components. This is definitely something I will have to look into.

> I really should not be able to attach an RGB wire to an input meant to process an aerial heading just because they both use a 3-tuple of numbers for representation. Its conceivable that one could generate tons of nonsense data without detection this way.

I guess I could argue that in a real-world situation, connecting 2 terminals that source and sink data packets of the same structure (header, CRC etc.), regardless of the payload (nonsense to the receiver or not) these packets would be accepted. However...

> Note that this is a run time type -- whether or not it is implemented via run-time type labels or as a reflection (no pun intended) of the C++ type system is not the issue.  Performance wouldn't be an issue since the checking would occur when a wire is attached and wouldn't have any overhead during "ticks".  If it is purely run-time, however, there wouldn't be any obvious way to enforce that the type placed on an output has any relation to the type label.  On the other hand, dynamically modifying the output of a component would not be possible.  Maybe a mix would be the right thing, though I'm not sure exactly how that would be done.
> 
> Of course, I would think that a "not type specified" type, providing the present behavior, would be appropriate, and is a reasonable default.  However, specifying a different default, within some scope, would be  valuable as well -- if you are simulating logic circuits then either 2-state or 3-state logic signals would be a good default, even if parts of the system, dealing with edge-triggered circuitry or A/D/A, or analog sensors, need other types.

This is something I have thought of implementing right from the start. Selecting an expected input type would go hand-in-hand with selecting an input's behavior towards multiple inputs (mentioned above), in the IO configuration phase of component design.

> 4) /Compile-time vs run-time configuration? /(a.k.a., declaration vs execution?)  There seems to be the potential for large gains in fixed compile-time configuration and in some cases, in reliability.  On the other hand, its clear that the primary intended use cases require more flexibility.  It would seem that, once again, a hybrid system would useful.  What occurs to me is that one should be able to create components by combining "primitive" components (classes derived from DspComponent) at compile time using the same concepts (wires, inputs, outputs) as the run-time system.  Prototypes could be turned into fixed sets of components when fully understood and debugged, and also compiled units could be replaced by the same set of primitive components wired together dynamically (in fact, it might be possible for the class representing the hard-wired version to automatically include a static method that could be called to create a soft wired version of itself, an instance method instead of or in addition to the static one would allow compiled components to be hot-swapped out for a run-time configurable version of itself).

This is what I have in mind too.

> 5) /Object-Oriented vs Generic Interface?/ -- I'm not going to take sides here, but it seems unlikely that the small overhead of run-time-bound calls would make much of a difference except in the limited case of a large network of simple components (e.g., a large, low-level, logic gate system) with high sequentiality and very little I/O or logging.  In any other circumstances, I would say that the time for the indirection would be completely swamped by the component internals, by other kinds of system overhead and by I/O.  That doesn't mean that generic programming isn't preferable but only that the performance overhead of virtual calls isn't an argument for it unless one can show that the non-monitored, large logic-gate type system requiring high-performance is an important design case.

> 

I'm always looking for ways to optimize DSPatch, I've come a long way but I'm in no way disillusioned about how far I can still go.

Thanks again for your interest in DSPatch!