[boost] behond IOStreams

From: Carlo Wood (carlo_at_[hidden])
Date: 2004-09-12 06:06:17

Hiya all.

I already briefly discussed this with Jonathan Turkan
and I understood he is interested to continue on
a new project when IOStreams is accepted.  I'd like
to work together with him in this new project.

In the past week I've done some "research" (between
quotes because I still very new to this, can't call
myself knowledgable yet).  I'd start an early kick-off
with this post ;).

IOStreams can be seen as a library that allows one
to define the following:

   Source --> filter --> filter --> Sink

The relationship between the types involved can be
delayed till runtime, but each individual type above
has to be coded before that.  It makes most sense
to look at the above as instances instead of classes
therefore (cause I did draw in the relationships
between the different types already):

   Source_instance --> filter_instance --> filter_instance --> Sink_instance

But for brevity I will not type the '_instance' part
anymore.

Obviously, we can have multiple Sources and Sinks at
the same time:

   Source --> Sink
   Source --> filter --> filter --> Sink
   Source --> filter --> filter --> filter --> Sink
   Source --> filter --> Sink
   ...

Each of these Sources will need to be handled by the
Operating System; for example if they are sockets.
In order to make an application work one will need
"event demultiplexing" like forinstance is provided
by select() or poll().

In order to write a framework that can "connect" the
Sources and Sinks to this event demultiplexor in a
flexible, portable and efficient way - a lot of design
decisions need to be taken.

Below I have made a list of a few topics that we can
discuss, one by one - in order to make progress with
these decisions.

I am aware that there has been made a beginning with
a design for such a framework already with boost in mind
(see http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl?BoostSocket)
This design mainly considers sockets, but I think that
a much wider approach is necessary (as we can discuss
by following the points below). But of course we will
use anything of this project that is useful.

Another important source of experience is libACE.
Although libACE provides everything we might need, it
doesn't 'connect' well to boost; it has a lot of interfaces
that are already done in boost but in another way.
Also, I think that libACE is doing more than we need
and not everything that we might want.  If it were
possible to write a more 'native' event demultiplexing
library for boost then that will definitely add useful
and additional functionality for the open source community.
Apart from that, there is the license issue.  I believe
that the boost libraries are more 'free' then ACE, so
in some cases ACE might not even be an option.

==========================================================
Follows: Topics that can be discussed.
I'd like to urge to stick to one Topic per thread.

1. Is there a need for this library?  Should be a new effort
   like proposed or do we have to revive 'boost Sockets'
   and only build upon that without throwing things away?

If "yes - there is a need for it and we might as well start
with a clean plate":

For the below I have chosen to phrase the topics as statements
that I believe I can give sufficient argumentation for (if needed).

2. It is unavoidable that this library uses threads.

3. User programs compiled for platform A, and linking with
   a shared versions of this library - do not need to be
   able to run on platform B (with same architecture)
   without recompilation.

4. It is unacceptable that we would depend on other libraries
   (libevent, libACE) for this, except standard libraries
   (libc, libstdc++, socket, ws2_32, etc).

5. We should heavily lean on the expertise that is condensed
   in libACE for things like the following:
   There must be a concept "Event Handler" (the 'call back').
   There must be a concept "Acceptor", "Connector" and
   "Handle" types.

6. The ideas about I/O filtering as discussed for Apache
   (see http://www.serverwatch.com/tutorials/article.php/1129721)
   need to be analyzed and probably implemented in IOStreams
   before we can really design the demultiplexor library.

I will start a thread for each of these topics.  Please reply
to the first subject that you do not agree with (yet) only:
it makes no sense to participate in discussions with a higher
number if you disagree with the statement of one with a lower
number :)

Regards,

-- 
Carlo Wood <carlo_at_[hidden]>