From: Jeff Garland (jeff_at_[hidden])
Date: 2004-05-22 09:45:15
On Sat, 22 May 2004 20:20:07 +1000, Arash Partow wrote
> Hi all,
> I was wondering if there would be any interest in a template
> library based around producers and consumers.
> I've developed something already and have a series of working
> examples, I propose a library that consist of 3 basic entities, a
> producer a consumer and a hybrid entity called an interchange.
> the entities themselves will be threaded and will be linked together
Making each element of the processing chain a thread is rather constraining.
What if I want to use a leaders / followers approach to threading my chain of
transformers? So when I get a callback on my socket I chain thru all the
transformations in the callback thread thus avoiding several thread context
switches just to process the data.
> via a linking mechanism that is itself a base template but which can
> be extended to provide data transfer between same system processes
> and threads and also extended versions which provide data transfer over
> pipes, sockets, serial lines etc..
> the idea of the producer-consumer is very simple the producer
> produces a type of data ie: string, a class, a struct etc.. and a
> consumer that is instantiated as a consumer of the type will consume
> the data provided by the producer and do something with it.
How does this differ from the signals and slots library?
Couldn't we just create a slot endpoint for the various types? Can I connect
2 or more consumers to a particular interchange? For this to be useful, I
should be able to.
> The link mechanism is a basically a type that supports the common
> type between a producer and its consumer, it is a one way flow of
> data, the consumer is signaled when data is ready for it to collect
Again, seems like signals and slots to me...
> A link has a basic get and add functionality but depending what
> entity is being implemented they will only see it via an interface
> and hence only be able to invoke 1 kind of method, a producer can
> only invoke "add" where-as a consumer can only invoke a get, an
> interchange can invoke both because it is instantiated with both a
> consumer and producer link interface, but the links are separate for
> an interchange, because it requires 2 links, one link for the
> incoming data and one link for the out going data. both link maybes
> of different types, ie:
> Producer <string> ----> Interchange <string, AClass> ----> Consumer <AClass>
> I need some opinions, about whether or not people will find this
> library useful and hence have it as part of Boost, if so could people
> give me their opinions about my design, its just a simple first trial
> run at designing it, i intended for the producers and interchange
> entities to be policy driven as far as production of data is
Well, I think it could be useful and wouldn't mind having it in boost, but it
seems like we need other basic infrastructure like a reactor, callback timers,
and sockets before there will be much advantage.
> I think this library can be used in distributed computing, in network
> packet assembly, and also for interfacing and passing data up and
> down threaded software layers.
Yes, it is a useful design in 'data flow' systems. Distributed
publish-subscribe type systems which have various data transformers use this
type of design (or they should).
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk