[boost] Re: [network] An RFC - updated

From: Michel André (michel.andre_at_[hidden])
Date: 2005-04-24 13:49:07

Don G wrote:
>>>>So basically layer 0 should support this portable
>>>>subset.
>>>
>>>Well, I would mostly agree, but I don't think
>>>everyone will. One of the goals several people
>>>have expressed is the ability to use level 0
>>>wrappers in non-portable ways.
>>
>>Isn't that just saying the handle should be
>>accessible. And you should be able to attach and
>>release an underlying handle from the objects at 
>>this level.
> 
> 
> I believe so, at level 0 - the socket wrapper classes. As you
> probably noticed, the interfaces I propose do not expose this handle
> and indeed they cannot as it may/probably won't be what is expected
> if it even exists.

Are you agreeing that we should have a level 0 close to the socket api 
supporting a portable subset or are you giving in to pressure ;).

> I guess I am still focused on the user creating network objects and
> using them directly since that is how I use this where it was born. I
> will have to think further on this because, while this textual
> address centric approach sounds reasonable, I am concerned by so much
> happening without direct control. In some environments (from my
> experiences with plug-ins and ActiveX mostly), it is important to
> know when things can be started and at what point they must end. The
> lazy init is good for starting, but the stopping is more troublesome.
> I cannot tell you how frustrating it was to debug threading issues
> during global destruction! :(

I can see the troubles you are afraid of. But having weak_refernces in 
the registry would get the objects destroyed as soon as there is no 
strong references to it, so if you have destroyed all objects associated 
with a network you should be home free. But I guess some nasty circual 
references or whatever could keep the object alive. Btw, arent you using 
shared ptr for the network object so essentially you have the same 
problem today?

> Also, the mapping from user entered form into some of the cryptic
> forms we have discussed would be more tedious to me than just using
> the right object. ;) Even in config files this seems a bit hard to
> understand and document. As a user of a product, I would not react
> well to this stuff. Imagine configuring your proxy server setting:
> 
> proxy=tcp:/proxy.boost.com:80
> 
> I would expect it to be:
> 
> proxy=http://proxy.boost.com
> 

It would be quite easy to write a function that maps uris to our address 
  notation. And usually you provide some kind of gui which handles this.

> The contexts are usually different methods invoked by the GUI
> framework based on the current page the user is working with.
> 
> We try to avoid having the user enter arcane syntax addresses, so "Do
> Foo" is where we establish the scheme/protocol as well.

ok.

> The map would be important in this kind of app if the user entered
> such addresses directly. It would also come up if certain things were
> in config files (they haven't in my experience). Other kinds of apps
> may run into this if they need to expose multiple networks at that
> level.

I think probalbly what you want to do, is to use the same protocol 
implementation over several different transports ie tcp/named 
pipes/serial or whatever and it's here the address and factory comes to 
play. I think as I have expressed before if you actually know the type 
you should be able to use concrete classes directly such as 
tcp_stream/accpetor/connector since they could expose more functionality 
than the generic net_stream.

> 
> Yes. The "generic async library" I keep mentioning is where this is
> handled w/o the user A) being called in the wrong thread or B) the
> user manually doing any posting.
> 
> In pseudo code:
> 
>    class foo
>    {
>     public:
>       foo (net::stream_ptr s)
>        : strm_(s)
>       {
>          ch_.open();
>          strm_->async_connect(ch_.bind_async_call(&on_connect));
>       }
> 
>       // implicit: ~foo () { ch_.close(); }
> 
>     private:
>       net::stream_ptr strm_;
>       channel  ch_;
> 
>       void on_connect () // called in same thread as ctor
>       {
>          strm_->async_write("Hello world!", ...,
>                             ch_.bind_async_call(&on_written));
>       }
> 
>       void on_written () // also called in ctor's thread
>       { ... }
>    };
> 
> There is a lot that could be explained about channel. I posted this
> some weeks ago as Asynchronicity or something like that, but I
> haven't focused much on it.
> 
> The current thread (app main in this case) must cooperate for the
> channel to open; the thread must declare its intent to deliver queued
> messages (boost::function<> objects) by creating a different object
> (I called it a nexus in my post, but that was just a first crack at
> renaming the concept from the code at work).
> 
> The channel class exists to allow ~foo (or some other impulse) to
> cancel delivery of any messages that might be in the queue. Imagine
> if "delete foo" happened with the call to on_connect already in the
> queue. Yuk. So, channel solves this problem.

Ok I think I need some more explaining in these concepts and some better 
names than nexus and channel. But basically you have an event_sink 
(channel) and a event_source(nexus) that posts completions to the sink. 
And if the event_sink goes out of scope notifications is'nt delivered.

I have implemented this with having synchrounous/blocking close and 
deletion that would deque and completion whit a error stating the 
operation has been cancelled. But agreed it sometimes can be tricky to 
get this right and don't get spurious completions after deletion.

> The channel approach does complicate usage in this case, and it could
> be improved by exposing the concept from the network layer:
> 
>    class foo
>    {
>     public:
>       foo (net::stream_ptr s)
>        : strm_(s)
>       {
>          strm_->set_async_context(); // to this thread
>          strm_->async_connect(&on_connect);
>       }
> 
>     private:
>       net::stream_ptr strm_;
> 
>       void on_connect () // same thread as set_async_context
>       {
>          strm_->async_write("Hello world!", ...,
>                             &on_written);
>       }
> 
>       void on_written () // also by set_async_context thread
>       { ... }
>    };
> 
> This just adds some (useful<g>) coupling between "net" and the
> general async library.

And also might have consequences on switching/overhead and performance I 
think.

> Without threads the network would have a capacity limit of
> FD_SETSIZE, but even with threads the system has a limit. 

Unfortunately computers are discrete and bounded machines with limits on 
the resources at all levels, but we do our best to hide these limits 
from the programmer and user so he can sail away and believe there is 
infinite memory and sockets available, dont' we ;).

/Michel