[boost] Re: [network] An RFC - updated

From: Michel André (michel.andre_at_[hidden])
Date: 2005-04-23 20:13:44

Hi Don!

Don G wrote:
>>I would prefer for a fixed set of networks (mainly
>>ip/ipv6 to be available from start maybe lazily
>>created on first reference). And registering a
>>network creation function to the registry would
>>let the registry use weak references to networks
>>and all objects spawned from a network holds strong
>>references in that way the network is destroyed
>>when the last object spawned from it goes away. The
>>network creation function could be passed along
>>with an address so an address could be used across
>>dll boundaries where each dll has its own registry.
> 
> 
> I have to admit that I avoid DLL/.so's whenever possible, so this
> approach is not one I had considered. I don't see any problem with
> the application registering network objects inside main or near
> there<g>, that way only the networks one needs to support are linked.

I still think that at least ip/ipv6 should be on by default (and 
implemented in such a way that the lazilly initialize themselves so the 
cost is low of having them active at all times). I have no problem with 
user registering the some odd net providers or their own user defined.
One should be aware that several dlls could load the libary 
simoltaneously and the library should preferably cater for that or at 
least not break down. Maybe just set some rules.

>>Wouldn't you need to store the settings and then
>>start the applications from the stored settings
>>and isn't the map needed then?
> 
> 
> At most, the network settings we store are some port overrides. So we
> just read them as we create the appropriate network object. Following
> that, we have "mTcp", "mSerial", etc. and which is needed is known by
> context. This is just my experience. A map would come into play with
> the address-tells-all approach. More on this in my reply to Peter. :)

So you have a switch somewhere like
switch(transport) {
   case serial: return mSerial;
   case tcp; return mTcp;
}

Or something to that matter that corresponds to the map? Or maybe you 
don't provide the same services/protocols over two networks, in that 
case the context would be clear from my_fancy_serial_protocl_handler use 
mSerial and from my_cool_tcp_handler us mTcp?

>>I see no reason whatso ever to have a windows gui
>>program using the library to be single threaded. It
>>would be easy enough to spawn a single io thread
>>polling and handling the notifications and posting
>>results to be processed by the gui. I have only
>>gotten into trouble trying to integrate this kind
>>of polling in the eventloop of a gui application
>>and always moved to an simple thread model posting
>>notifications to the gui thread.
> 
> 
> The desire to eliminate threads from code is not restricted to
> servers<g>. There is a long history of using the windows message
> queue (via PostMessage) for handling networking that pre-dates
> threads just as select and signal-driven I/O pre-date threads for
> Unix. [FWIW: the "right" way is PostMessage-to-hidden-window, not
> splicing up the GetMessage/DispatchMessgae loop, and no polling is
> required]

That was the method i was referring to when saying post notifiactions to 
the gui thread. And of course eliminating threads isn't important for 
regular apps as well. But one additional io thread and a hidden window 
and some post message calls and all your code is serialized and a very 
simple model without chopping up the event loop.

> I see this as a perfectly reasonable approach, especially when the
> needs are modest. Even a complex GUI app (at least for X, Windows and
> Mac) is typically single-threaded and asynchronous. 

Yes there hasn't been much sense in threading gui apps that heavily and 
keep the model simple.

> I hadn't noticed that tid bit in select's docs before, but you are
> correct: it can be set higher, and it probably could cause issues
> with some drivers. On a side note: the product I work on a the job
> has run into issues related to specific network drivers when using
> DuplicateHandle on a socket as prescribed in MSDN, so this kind of
> issue is real. :(

I don't envy you having go through that I have luckily been able to 
avoid these kinds of problems.

> I am presupposing that one network impl will make heavy use of worker
> threads, and deliver callbacks as I described elsewhere. The same
> impl cannot also be fully single threaded. :) Of course, what I have
> been saying as well is that one can use the general async facility to
> get callbacks queued to a specific (single) thread, thereby hiding
> the worker threads. The issue is that some don't find this detail
> insignificant: they want to eliminate the worker threads altogether.

I dont think threads can't be avoided as I see it at least not when 
hitting an internal limit of the notfication mechanism choosen eg 64 for 
windows and 1024 for Linux FC3. Otherwise polling would have to be done 
in some roundrobin way over the available handle sets and that would 
introduce latency.

> I'm just curious to see if someone has some simple ideas on automatic
> buffering that doesn't require massive overhead when unused. It
> wasn't that all if-checks had to be eliminated, or that some extra
> code might get linked w/o being used. What I want to avoid is large
> chunks coming in via iostream (for example) that in many uses will be
> dead code.

Agreed. I think eg that the stream_buf interface is to complex and 
arcane to fit at this level even if eg Jeff Garland proposes it.

/Michel