ASIO in the Standard (was Re: C++ committee meeting report)
Cross-posting to networking@isocpp.org and c++std-lib-ext. On Wed, Jun 25, 2014 at 12:47 AM, Beman Dawes <bdawes@acm.org> wrote:
* The Library Evolution Working Group (LEWG) voted to base the Networking TS on Boost.ASIO. That was a major surprise, as a Networking Study Group had been trying to build up a Networking TS from many small proposals. But in LEWG polls, support for ASIO was overwhelming, with no one at all voting against. This was a major vote of confidence for Boost.ASIO and Chris Kohlhoff. And several ASIO supporters were not even in the room at the time because Filesystem issues were being worked on elsewhere.
FWIW, I was in the room when the Boost.Asio proposal was being discussed way back in Kona. I haven't seen the most recent update to the proposal, but if my original concerns were addressed, mainly: - The io_service type was trying to be too many things at the same time (executor, ran in thread-pools, implementing IO-specific hooks, etc.) and was too prominent in the interfaces of primitive types (i.e. sockets, etc.). - There were too many leaking abstractions crossing boundaries -- being able to reach in to get native handles, buffer lifetimes crossing in-and-out, etc. - The fact that it seems the only implementation of the networking parts (and inter-operating bits implied by the interface) would be through a proactor -- how the API was only proactive, no support for just purely reactive applications. - There's no means for doing "zero-copy" I/O which is a platform-specific but modern way of doing data transfer through existing interfaces. ... then this I think is *good news*! Having worked with quite a few network applications now in the past few years, I'm happy with what Boost.Asio provides. However I think there are other systems/libraries we can learn from to inform the design here (I tried getting a comprehensive survey paper in, but that proved a little harder than I originally thought and ended up with just doing a poor job for HTTP). A few that come to mind for potential approaches here are: - Transport-level abstractions. Considering whether looking at policy-driven transport protocol abstractions (say, an http connection, a raw TCP connection, an SCTP connection with multiple streams, etc.) would be more suitable higher-level abstractions than sockets and write/read buffers. - Agent-based models. More precisely explicitly having an agent of sorts, or an abstraction of a client/server where work could be delegated, composed with executors and schedulers. I admit I haven't exactly been following through on attending the Networking SG meetings and sending in papers myself. I sincerely hope though that our collective love for Boost.Asio doesn't preclude us from thinking at higher levels of abstractions and in broadening our view/understanding of the network-programming landscape. I say this with all due respect -- while network programming sounds like it's all about sockets, buffers, event loops, and such there are the "boring" bits like addressing, network byte ordering, encoding/decoding algorithms (for strings and blobs of data), framing algorithms, queueing controls, and even back-off algorithms, congestion control, traffic shaping. There's more things to think about too like data structures for efficiently representing frames, headers, network buffers, routing tables, read-write buffer halves, ip address tries, network topologies, protocol encodings (ASN.1, MIME and friends), and a whole host of network-related concepts we're not even touching in the networking discussions. Having said this, I'm happy that we're willing to start with Boost.Asio. :)
The next C++ committee meeting is November 3rd to 8th in Urbana-Champaign, IL, USA. As always, Boosters welcome.
Would really love to go, or at least be able to send comments to papers in time for that one. Thanks for the report Beman!
On 3 Jul 2014 at 23:16, Dean Michael Berris wrote:
FWIW, I was in the room when the Boost.Asio proposal was being discussed way back in Kona. I haven't seen the most recent update to the proposal, but if my original concerns were addressed, mainly:
- The io_service type was trying to be too many things at the same time (executor, ran in thread-pools, implementing IO-specific hooks, etc.) and was too prominent in the interfaces of primitive types (i.e. sockets, etc.).
Some would call that the low-level interfacing necessary to achieve bare metal performance.
- There were too many leaking abstractions crossing boundaries -- being able to reach in to get native handles, buffer lifetimes crossing in-and-out, etc.
Ditto.
- The fact that it seems the only implementation of the networking parts (and inter-operating bits implied by the interface) would be through a proactor -- how the API was only proactive, no support for just purely reactive applications.
Kinda hard to do true async without.
- There's no means for doing "zero-copy" I/O which is a platform-specific but modern way of doing data transfer through existing interfaces.
I don't know what you refer to here. ASIO doesn't copy data being sent or received. It passes through literally what it receives from the OS.
A few that come to mind for potential approaches here are:
- Transport-level abstractions. Considering whether looking at policy-driven transport protocol abstractions (say, an http connection, a raw TCP connection, an SCTP connection with multiple streams, etc.) would be more suitable higher-level abstractions than sockets and write/read buffers.
- Agent-based models. More precisely explicitly having an agent of sorts, or an abstraction of a client/server where work could be delegated, composed with executors and schedulers.
These are all outside the remit of a core C++ networking library.
I say this with all due respect -- while network programming sounds like it's all about sockets, buffers, event loops, and such there are the "boring" bits like addressing, network byte ordering, encoding/decoding algorithms (for strings and blobs of data), framing algorithms, queueing controls, and even back-off algorithms, congestion control, traffic shaping. There's more things to think about too like data structures for efficiently representing frames, headers, network buffers, routing tables, read-write buffer halves, ip address tries, network topologies, protocol encodings (ASN.1, MIME and friends), and a whole host of network-related concepts we're not even touching in the networking discussions.
I'm personally not unsympathetic to this sentiment. However, it would surely need POSIX to move on it first before C++ could. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
On Fri, Jul 4, 2014 at 9:18 PM, Niall Douglas <s_sourceforge@nedprod.com> wrote:
On 3 Jul 2014 at 23:16, Dean Michael Berris wrote:
FWIW, I was in the room when the Boost.Asio proposal was being discussed way back in Kona. I haven't seen the most recent update to the proposal, but if my original concerns were addressed, mainly:
- The io_service type was trying to be too many things at the same time (executor, ran in thread-pools, implementing IO-specific hooks, etc.) and was too prominent in the interfaces of primitive types (i.e. sockets, etc.).
Some would call that the low-level interfacing necessary to achieve bare metal performance.
But, there's no reason you couldn't decompose this and not make it part of the invariants of the objects associated with them. For example, when you create a socket object, can you not do that by acquiring it from some function that internally would determine how it's wired? auto connection = establish_connection(remote_address, my_executor);
- There were too many leaking abstractions crossing boundaries -- being able to reach in to get native handles, buffer lifetimes crossing in-and-out, etc.
Ditto.
- The fact that it seems the only implementation of the networking parts (and inter-operating bits implied by the interface) would be through a proactor -- how the API was only proactive, no support for just purely reactive applications.
Kinda hard to do true async without.
Can you expand this a little?
- There's no means for doing "zero-copy" I/O which is a platform-specific but modern way of doing data transfer through existing interfaces.
I don't know what you refer to here. ASIO doesn't copy data being sent or received. It passes through literally what it receives from the OS.
Right. Does it support direct-memory I/O -- where I write to some memory location instead of scheduling a write? See RDMA, and on Linux vmsplice(...). Can I "donate" the memory I have from user-space to the kernel, which just remaps the memory onto a device's memory buffer? How do I achieve "raw" buffer performance if I have to provide a user-space allocated buffer for data to be written into from the kernel, and then write out data if the data was in a user-space buffer that gets passed to the kernel? We've moved on to better interfaces in the lower levels of the networking stack, and it would be a shame if we precluded this from being adopted by a standard library specification.
A few that come to mind for potential approaches here are:
- Transport-level abstractions. Considering whether looking at policy-driven transport protocol abstractions (say, an http connection, a raw TCP connection, an SCTP connection with multiple streams, etc.) would be more suitable higher-level abstractions than sockets and write/read buffers.
- Agent-based models. More precisely explicitly having an agent of sorts, or an abstraction of a client/server where work could be delegated, composed with executors and schedulers.
These are all outside the remit of a core C++ networking library.
Why?
I say this with all due respect -- while network programming sounds like it's all about sockets, buffers, event loops, and such there are the "boring" bits like addressing, network byte ordering, encoding/decoding algorithms (for strings and blobs of data), framing algorithms, queueing controls, and even back-off algorithms, congestion control, traffic shaping. There's more things to think about too like data structures for efficiently representing frames, headers, network buffers, routing tables, read-write buffer halves, ip address tries, network topologies, protocol encodings (ASN.1, MIME and friends), and a whole host of network-related concepts we're not even touching in the networking discussions.
I'm personally not unsympathetic to this sentiment. However, it would surely need POSIX to move on it first before C++ could.
Why?
On 4 Jul 2014 at 21:59, Dean Michael Berris wrote:
Some would call that the low-level interfacing necessary to achieve bare metal performance.
But, there's no reason you couldn't decompose this and not make it part of the invariants of the objects associated with them. For example, when you create a socket object, can you not do that by acquiring it from some function that internally would determine how it's wired?
I agree this is what should be done in a ground up refactor. But that isn't what's on the table. What is proposed is to standardise a subset of the common practice.
- The fact that it seems the only implementation of the networking parts (and inter-operating bits implied by the interface) would be through a proactor -- how the API was only proactive, no support for just purely reactive applications.
Kinda hard to do true async without.
Can you expand this a little?
http://www.boost.org/doc/libs/1_55_0/doc/html/boost_asio/overview/core/async... http://www.artima.com/articles/io_design_patterns2.html You may of course not meant reactor when you wrote "reactive". For me personally, how QNX (and I assume Hurd) do async i/o is the gold standard. The NT kernel does a very faithful emulation of true async. BSD at least provides a usable aio_* POSIX API. The rest really aren't great on async.
- There's no means for doing "zero-copy" I/O which is a platform-specific but modern way of doing data transfer through existing interfaces.
I don't know what you refer to here. ASIO doesn't copy data being sent or received. It passes through literally what it receives from the OS.
Right.
Does it support direct-memory I/O -- where I write to some memory location instead of scheduling a write? See RDMA, and on Linux vmsplice(...). Can I "donate" the memory I have from user-space to the kernel, which just remaps the memory onto a device's memory buffer? How do I achieve "raw" buffer performance if I have to provide a user-space allocated buffer for data to be written into from the kernel, and then write out data if the data was in a user-space buffer that gets passed to the kernel?
We've moved on to better interfaces in the lower levels of the networking stack, and it would be a shame if we precluded this from being adopted by a standard library specification.
Yeah ... well, actually no, I wouldn't call vmsplice() or any of its ilk as anything deserving the label "better". That whole way of making DMA possible is a hack forced by Linux's networking stack being incapable of using DMA automatically. After all, BSD and Windows manage DMA with socket i/o automatically and transparently. No magic syscalls needed. Linux should up its game here instead of syscall hacks. Even if splice() et all were a good idea to encourage, they aren't standardised by POSIX and therefore are out of scope for standardisation by C++. ISO standards are there to standardise established common practice, not try to design through central committee.
A few that come to mind for potential approaches here are:
- Transport-level abstractions. Considering whether looking at policy-driven transport protocol abstractions (say, an http connection, a raw TCP connection, an SCTP connection with multiple streams, etc.) would be more suitable higher-level abstractions than sockets and write/read buffers.
- Agent-based models. More precisely explicitly having an agent of sorts, or an abstraction of a client/server where work could be delegated, composed with executors and schedulers.
These are all outside the remit of a core C++ networking library.
Why?
ISO standards are there to standardise established common practice, not try to design through central committee. Also, think in terms of baby steps. Start with a good solid low level async networking library which is tightly integrated into threading, executors and the rest of the STL async facilities. That already will be hideously hard to do. For the next major C++ standard build on that with better abstractions.
I say this with all due respect -- while network programming sounds like it's all about sockets, buffers, event loops, and such there are the "boring" bits like addressing, network byte ordering, encoding/decoding algorithms (for strings and blobs of data), framing algorithms, queueing controls, and even back-off algorithms, congestion control, traffic shaping. There's more things to think about too like data structures for efficiently representing frames, headers, network buffers, routing tables, read-write buffer halves, ip address tries, network topologies, protocol encodings (ASN.1, MIME and friends), and a whole host of network-related concepts we're not even touching in the networking discussions.
I'm personally not unsympathetic to this sentiment. However, it would surely need POSIX to move on it first before C++ could.
Why?
Once again: ISO standards are there to standardise established common practice, not try to design through central committee. If all the platforms did something like managing routing interfaces identically, we could argue in favour of it on the case of merit. But they don't, so we can't. One would also be extremely hesitant to standardise anything which hasn't been given full and proper deliberation by the ISO working group responsible for it. I feel little love for how the AWG see the world personally (I find interacting with them deeply frustrating), but they have their role in ISO and they haven't done too bad a job of things in the wider picture. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
On 6 Jul 2014 at 16:20, Niklas Angare wrote:
"Niall Douglas" wrote:
For me personally, how QNX (and I assume Hurd) do async i/o is the gold standard.
As a QNX user I'm curious about what you're referring to here.
All micro kernel OSs act as if all syscalls are non-blocking, even async. I cannot speak for the Hurd, but for QNX the central scheduler works by dispatching coroutines which when they block due to a message send they are saved and the next unblocked coroutine is continued. For example, if you try writing a byte to a fd, under the bonnet in the write() implementation that turns into a MsgSend which sends the byte to the channel corresponding to that fd which will appear in the resource manager's channel for that fd. The sending coroutine is then marked as blocked on message send, is suspended and the next coroutine executed for the next message arriving or in the queue. What this turns into is that you can pretty much skip threads entirely in most QNX programs - you use threads solely for having work executed on multiple CPU cores concurrently, not for concurrency in kernel space as you get for free concurrency in "kernel space" all the time (there is almost no kernel space in QNX, device drivers or handling page faults or interrupt handling are just another message dispatch and handle). As all syscalls can be scheduled and completed later on the same thread via coroutines, that means you can issue a read gather across multiple fds and as the results complete you'll receive messages with the results which appear to your code as the API returning (note you can bypass this or override it to your heart's content, QNX lets you hack the message dispatch very easily). Obviously such a facility would be extremely useful to something like ASIO because one can dispense entirely with the select()/epoll() machinery at the heart of the io_service which dispatches completions to the next available thread worker. In QNX an ASIO equivalent is already there at the heart of the OS - in fact, in QNX, the message passing infrastructure is pretty much the only thing there is, as everything from handling page faults in memory mapped files through to device drivers is implemented with the same core message passing infrastructure. QNX is a neat system. Once you've written a device driver for it you'll never enjoy writing one for Linux ever again. Linux looks quite stone age and backwards in comparison, though in fairness QNX struggles to scale as Android can. In particular, QNX copies a *tremendous* amount of memory around and will eliminate any advantages of L1/L2 caches very quickly indeed. BB10 runs at main memory speed a lot of the time as the L2 cache might as well not be there under message passing load. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
participants (3)
-
Dean Michael Berris -
Niall Douglas -
Niklas Angare