Re: [boost] asio formal review - portability

From: Darryl Green (darryl.green_at_[hidden])
Date: 2005-12-26 00:19:25

Hi Chris,
I think the main issue I have with the library implementation and 
interface really comes down to a different perspective on portability 
and extensibility. Perfect portability would be great (well, maybe not, 
if nothing else it would be boring :-) but in the absence of that, 
interoperability and ease of porting are better. Also porting of 
libraries, porting of applications and porting of entire systems are 
different things, as I'm sure you are aware. The trade offs involved in 
any given "port" may be quite different - some service providing app 
that needs to provide services over some form of channel *should* make 
relatively few demands on the channel but the actual channel(s) it uses 
on a particular platform, occupying a particular place in a particular 
larger system may be quite different from those it uses in a different 
deployment environment. Ideally the set of channels supported should be 
able to be easily selected/extended at compile and/or runtime (ie. in 
some cases tuning/compiling for a very specific environment is necessary 
and in some cases very broad interoperability at runtime is necessary).

With that perspective in mind maybe I'll make slightly more sense?

Christopher Kohlhoff wrote:

> This is not consistent with the goal of portability. As I
> mentioned in the "reactor versus proactor" thread, the proactor
> is a more portable pattern than the reactor. In the longer term
> I'm not averse to allowing a reactor abstraction to become part
> of some secondary public interface, where the restricted
> portability is explicitly noted.

I see no reason why a reactive interface can't be implemented on top of 
an async completion notification based system - in fact this is rather 
common within the OS itself, or as an optional layer/lib in an embedded 
kernel/rtos. All you need is a receive buffer managed by the lib that 
notifies when it goes non-empty (edge) or while non-empty (level) - 
obviously there are a variety of trade offs about buffer size/number of 
pending buffers etc to consider. Transmit is similar - claim to be ready 
for transmit while there are less than N lib managed requests not yet 
completed.

> There's no reason why, in principle, policy-driven services
> could not be added to allow further customisation. But these
> would be inherently non-portable, and so would only be part of
> some secondary public interface.

I disagree that all of these would be non-portable. I'm not sure what 
you mean by "secondary public interface" but if it is a public interface 
I'm happy - I just don't want all policy hidden in detail or worse yet 
rejected from the lib altogether because of a lack of portability. This 
is because I need *portability* but not in the same sense/at the same 
level we are talking about here. Perhaps "adaptability" would be a 
better term for what I am after.

>>I would like to see the aspects of a socket that have nothing
>>to do with reading/writing better separated, as this would
>>make for a more consistent interface for other forms of I/O
>>such as pipes, message queues etc as well as files.

> For stream-oriented I/O, this separation is presented in the
> form of the Stream concept. This concept is implemented by
> stream_socket, ssl::stream, buffered_read_stream and so on. It
> would be implemented by a hypothetical pipe class.

Yes - I didn't describe my reasoning very well. What I was trying to 
suggest was that you already have implementations of:

Pure notifiers (timers) - that implement the Async_Object concept

Streams (stream_socket) that implement the Async_Object and
Async_Read/Write_Stream concepts.

Messages (datagram_socket) - there isn't really a conceptual model for 
these above the Async_Object level - maybe there should be. While the 
send/receive interfaces look similar to the socket stream interface 
parameter-wise, the semantics of messaging are of course different (even 
without considering the send_to/receive_from bundling of message and 
endpoint).

I would like to see:

1) Driver services that bind a concept (or a minimal set of concepts) to 
a particular underlying, possibly platform dependent facility. These 
clearly won't in general be portable.

2) Policy-based composed services bringing together (composition may be 
reflected at runtime ie. there would be multiple "service references" 
per active object or at compile time ie. an actual, useful service is 
built from policies) "drivers" from (1) to produce useful services. I 
would expect the service to be at least conceptually portable, even 
though there might be some platforms for which a particular service 
cannot be composed from the supplied drivers.

3) Default/standard/portable services using/implemented in terms of (1) 
and (2) (ie. standardizing the policies used).

Concrete examples of 1:

A driver for Async_Read_Stream on posix-esq systems that uses a reactive 
model and any file descriptor.

A protocol (actually, I'm not sure if that is too much of a stretch for 
the existing concept - it crosses over into ACE acceptor/connector 
concepts, though I'd be inclined not to follow those exactly either) for 
  posix named pipes or anon pipe - read end or unix domain sockets.

Example of 2:

generic async readable stream - this needs not just the policy for 
Async_read_Stream but also a compatible protocol, etc) to allow the 
channel to be created, connected etc. By selecting polices can produce 
stream-socket or char device I/O for example.

Examples of 3:

The current stream_socket_service, a local IPC messaging service using 
unix domain sockets on some platforms, windows named pipes on others, 
posix message queues on others - default selection based on service 
availability.

The effects of this are:

On posix-esq platforms, just about anything/everything is a "file 
descriptor" that a select/poll/epoll call can be used on (interesting 
omission is actually conventional disk files) and most of these are 
streams, so it is possible to provide async streams that will work with 
pipes/char devices at least, once you figure out how to c'truct/open 
them, presumably through providing a protocol for opening things that 
works on (char devices), a protocol for opening named pipes (fifo's) and 
a protocol for dealing with anon pipes/inherited fds etc in general.

On windows overlapped I/O is similarly widely applicable, with analogous 
issues re handling creating and connecting to particular "special" files.

I would expect to see some abstract local IPC channel policies that are 
portable (but in general only if both ends of the channel use asio and 
compatible policies) that simply promise to offer eg. a duplex message 
oriented channel or a simplex stream (pipe-like). While these can be 
sensibly defaulted on particular platforms for interoperability reasons 
it should be possible to eg. use a very portable shared memory IPC 
message passing impl or (more likely to interoperate with processes not 
using asio, or more stable across asio version updates or...) an 
appropriate platform facility such as posix message queues.

I would like to see fine-grained composition from policies similar to 
the concepts just to make it easier to maintain and extend the lib (or 
maybe more the other way around ;-).

None of the above implies any major change of design, rather it takes 
advantage of the design to offer adaptability and portability without 
limiting either. I think it would be unfortunate if forms of IO that are 
not 100% portable were excluded from "first class" status in the 
library. Encouraging the utilization of the design's strengths by making 
it easy to re-use and extend the library can only encourage its 
acceptance and wider use and development. This would perhaps require 
some refactoring of the services and documenting/exposing some of the 
detail, but not much else.

What do you think?

Regards
Darryl Green.

-- 
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.1.371 / Virus Database: 267.14.7/214 - Release Date: 23/12/2005