Re: [boost] [asio] RFC on new reliable UDP library

Subject: Re: [boost] [asio] RFC on new reliable UDP library
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2014-09-12 06:59:29

On 11 Sep 2014 at 9:50, adzm wrote:

> asio support for UDT could easily stand alone. I have considered embarking
> upon this project myself, but the UDT library does not easily fit into
> asio's patterns. However, there is also the possiblity of implementing the
> UDT protocol rather than wrapping the UDT library, but that is obviously
> much more work and more surface area for bugs. But UDT fulfills most needs
> for mixing reliable with unreliable transport.

You hit the nail on the head exactly. There will have to be _some_ 
breakage to ASIO norms to implement this. The question is how much 
and in which ways? If you are going to break, why not use 11/14 
idiomatic C++ instead of 03 C++? Should we bother to do absolute 
zero-copy from the NIC given the additional design complexity? Just 
how much do you change before it starts to look not that much like 
ASIO at all?

During the last few days I persuaded proposed 
experimental::expected<T, E> to compile and run on VS 14 CTP 3, and I 
believe a usable port to VS2013 ought to be straightforward. That 
opens the door to very low latency future-promise on all recent 
compilers.

> Are there other popular user-mode protocols built over UDP that provide
> similar functionality? Elsewhere in this thread uTP was mentioned, but I am
> unfamiliar. SCTP is also mentioned, but has the unfortunate and significant
> drawback of being both painful and rare on Windows.

Our primary purpose is actually to provide an early RUDP 
implementation by Chris. His original needs rewriting. UDT is only 
really in there because we believe we can reuse implementation and 
save effort.

> I really think supporting UDT, either via wrapping the library or
> implementing the protocol, would be a great first step. It would be
> immediately usable by many users, and I am certain the process of
> implementation will reveal where the limitations of asio hurt the most.
> 
> Beyond that, I think there is a definite need for this, but the biggest
> obstacle is that most applicable standards are either very complex or don't
> exist in the first place. And boost is probably not the best place for
> implementing a custom protocol, although would be a great home for the
> building blocks.

If the standard C++ networking library provides some standard 
reliable messaging implementation with performance far in excess of 
TCP for some use cases, you betcha it becomes the standard (if it 
isn't braindead designed).

In GSoC 2014 a new Boost HTTP library was designed and written. It 
could look very interesting to combine that with reliable UDP for a 
whole new viewpoint on HTTP 2.0, especially as Google QUIC seems to 
have ground to a halt.

> For example, over the years I've coded several UDP hole punch
> implementations, and have always wanted to submit them to boost, but they
> ended up depending on some non-standard functionality for things like
> authentication, match-making, IP address detection, etc. The alternative
> leads to something like ICE, but the complexity level rises quickly after
> that.

Well, we won't be going beyond what the client absolutely needs - the 
reason for considering the future-promise-on-expected design is due 
to wider needs to offload incoming and outgoing messages to OpenCL 
for processing, and therefore some form of low latency continuations 
framework is needed, and it might as well be the potential C++ 17 
one. Chris's RUDP can do NAT hole punching, and we'll be duplicating 
his implementation which looks very similar to UDT's. We are 
certainly not looking to reinvent anything here, just aim for a very 
conservative set of minimum goals which are to reimplement Chris' 
RUDP in a more scalable solution.

Niall

-- 
ned Productions Limited Consulting
http://www.nedproductions.biz/ 
http://ie.linkedin.com/in/nialldouglas/