Re: [boost] [afio][v1.30 "C++ Now edition" released]

Subject: Re: [boost] [afio][v1.30 "C++ Now edition" released]
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2015-03-19 15:37:43

On 19 Mar 2015 at 19:58, Mathias Gaunard wrote:

> Hi, this is my first time actually looking at the documentation of the 
> library.
> 
> I think it would be interesting to develop a couple more things in the 
> design or FAQ sections.

Suggested topics are welcome.

> Why use a specific asynchronous model with its own when_all etc.? There 
> is a standard way of defining asynchronous tasks, which is getting 
> extended by compiler support using await/yield_value, so why not do 
> something using that?

It's not too obvious from first glance, but an async_io_op is a 
struct with a shared_future inside it which is publicly accessible 
called 'h'. So it uses the standard STL11 asynchronous model, just 
slightly hidden. The when_all etc is purely due to legacy reasons.

Indeed the reason I couldn't use shared_future directly is purely 
legacy, back in the day I had to use shared_ptr<shared_future<T>> due 
to STL11 bugs, and it was easier to wrap that into an async_io_op, 
plus back in 2013 there was no fully functioning when_any/when_all 
implementation so I had to write my own and I needed type sugar to 
stop my when_any/all conflicting with the ones in Thread. I think the 
v1.2 engine replaced the shared_ptr<shared_future> with 
shared_future, so that was the major refactor for v1.2.

Hence, especially as the Concurrent TS continuations API has finally 
firmed up, async_io_op can go away next release now, and an 
afio::future<T> with .then() etc replaces it. I've tried to bridge 
the APIs with a get() and a few other future like member functions to 
make porting the code easier, but it's going to be a major break, not 
least that the current central hash table is going away too in favour 
of a lock free linked list.

So basically AFIO currently looks the way it does mostly due to 
legacy baggage and problems with C++ 11 back in 2013. Each release I 
refactor some of the baggage out. v1.3 was the first to drop < VS2013 
support for example, it was a ton of work purging the VS2010 custom 
code paths. v1.4 will purge async_io_op + when_any/all and replace 
those with a proper C++ 17 compliant custom continuable future type 
which understands resumable functions. A big part of why that didn't 
enter v1.3 is because VS2015 and its resumable functions support I 
can test against isn't released yet, plus I was waiting for the 
Concurrency TS to firm up, it's only really stabilised these past six 
months.

> What about the relationship with ASIO? ASIO has its own event-driven 
> main-loop system for asynchronous tasks, can AFIO use that and can ASIO 
> use AFIO's future method for other types of I/O too?

AFIO uses ASIO's event loop. It doesn't use the async_result idiom of 
ASIO because for file i/o you want strong graph ordering, and that's 
a pain to do with async_result. A more functional idiom, for which I 
chose future-promise, makes strong ordering much easier to program. 
Once I get Fibers/resumable function support in there next release, 
writing strongly graph ordered filing system algorithms ought to 
become optimally easy, and then I can get down to writing race free 
tree visitors etc which right now are very messy to write and debug.

The AFIO-ASIO integration isn't deep right now, though of course 
ASIO's async_result can work with any arbitrary sync object, and that 
includes an async_io_op. Writing code (mostly completion lambdas) 
which arbitrates between the two is straightforward, if verbose and 
tedious. What is needed is zero copy kernel side support, and AFIO's 
design intentionally allows that (i.e. async write this gather list 
from this file to this socket all in the kernel using DMA). I'll be 
honest it's not high on my priority list given no one is using AFIO 
anyway, but potential deep AFIO-ASIO integration is very high on the 
radar. I will say though I'll need changes to ASIO to make that work, 
so it's not a trivial feature add.

Niall

-- 
ned Productions Limited Consulting
http://www.nedproductions.biz/ 
http://ie.linkedin.com/in/nialldouglas/