Re: [boost] [afio] Formal review of Boost.AFIO

Subject: Re: [boost] [afio] Formal review of Boost.AFIO
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2015-08-31 10:45:40

On 31 Aug 2015 at 2:33, Andreas Schäfer wrote:

> > 1. Should Boost.AFIO be accepted into Boost? Please state all 
> >    conditions for acceptance explicity. 
> 
> I vote no. For acceptance I'd expect a library that outperforms
> "naive" OpenMP code.

I will place a wager with you for €100 right now that nobody will 
EVER implement an asynchronous file i/o library on Linux < 4.0 which 
beats OpenMP.

Linux currently does not provide the facilities to do better.

> Also, the design should be greatly simplified.

AFIO is a simple design which keeps close to the bare metal in order 
to expose the underlying systems file system concurrency guarantees.

If you want even more simple, ASIO provides a simple asynchronous 
file i/o library. I think you'll find it delivers you no useful 
benefit.

> > 3. What is your evaluation of the implementation?
> 
> I tried to assess the library's performance as IMHO (and the library's
> documentation) performance is the major motivation for using AFIO. The
> code failed to compile with Intel's C++ compiler icpc[2]. With g++
> 4.9.3 the code compiled, but segfaulted when traversing a Linux 4.1.6
> source tree[3].

The library's documentation clearly specifies that the compilers 
tested are GCC >= 4.8, clang >= 3.3 and VS2015. Any other compiler is 
not supported.

As to why you saw a segfault when traversing a Linux source tree (was 
it with find regex in files?), I am sorry this occurred. I would 
suggest to use the find regex in files binary compiled using 
Boost.Build rather than any custom build and see if it happens there 
too. If it does, open an issue on github please.

> The code (and the naive OpenMP implementation provided in the
> documentation) completed successfully on a Boost.AFIO source tree, but
> the OpenMP code (~600 MB/s) was significantly faster than the AFIO
> code (~270 MB/s), see [4]. Test system: Intel Core i7-3610QM, Samsung
> EVO 1TB SSD, Linux 4.1.5. (I can provide further details if
> necessary.) All of this does not positively impress me.

You have highly unrealistic expectations. On Linux there is no useful 
async backend, so AFIO emulates async using a thread pool and the 
normal synchronous APIs. It goes without saying that will always be 
slower than OpenMP using the same APIs. It can never be faster.

What AFIO provides is a *portable* API for async file system and file 
i/o. It provides the least benefit on Linux of any of the platforms, 
but that is not AFIO's fault. When Linux does provide a decent async 
file i/o implementation, I'll add a backend and you'll see a 
corresponding benefit.

Niall

-- 
ned Productions Limited Consulting
http://www.nedproductions.biz/ 
http://ie.linkedin.com/in/nialldouglas/