Boost logo

Boost :

Subject: Re: [boost] [afio] Formal review of Boost.AFIO
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2015-08-31 10:45:40


On 31 Aug 2015 at 2:33, Andreas Schäfer wrote:

> > 1. Should Boost.AFIO be accepted into Boost? Please state all
> > conditions for acceptance explicity.
>
> I vote no. For acceptance I'd expect a library that outperforms
> "naive" OpenMP code.

I will place a wager with you for €100 right now that nobody will
EVER implement an asynchronous file i/o library on Linux < 4.0 which
beats OpenMP.

Linux currently does not provide the facilities to do better.

> Also, the design should be greatly simplified.

AFIO is a simple design which keeps close to the bare metal in order
to expose the underlying systems file system concurrency guarantees.

If you want even more simple, ASIO provides a simple asynchronous
file i/o library. I think you'll find it delivers you no useful
benefit.

> > 3. What is your evaluation of the implementation?
>
> I tried to assess the library's performance as IMHO (and the library's
> documentation) performance is the major motivation for using AFIO. The
> code failed to compile with Intel's C++ compiler icpc[2]. With g++
> 4.9.3 the code compiled, but segfaulted when traversing a Linux 4.1.6
> source tree[3].

The library's documentation clearly specifies that the compilers
tested are GCC >= 4.8, clang >= 3.3 and VS2015. Any other compiler is
not supported.

As to why you saw a segfault when traversing a Linux source tree (was
it with find regex in files?), I am sorry this occurred. I would
suggest to use the find regex in files binary compiled using
Boost.Build rather than any custom build and see if it happens there
too. If it does, open an issue on github please.

> The code (and the naive OpenMP implementation provided in the
> documentation) completed successfully on a Boost.AFIO source tree, but
> the OpenMP code (~600 MB/s) was significantly faster than the AFIO
> code (~270 MB/s), see [4]. Test system: Intel Core i7-3610QM, Samsung
> EVO 1TB SSD, Linux 4.1.5. (I can provide further details if
> necessary.) All of this does not positively impress me.

You have highly unrealistic expectations. On Linux there is no useful
async backend, so AFIO emulates async using a thread pool and the
normal synchronous APIs. It goes without saying that will always be
slower than OpenMP using the same APIs. It can never be faster.

What AFIO provides is a *portable* API for async file system and file
i/o. It provides the least benefit on Linux of any of the platforms,
but that is not AFIO's fault. When Linux does provide a decent async
file i/o implementation, I'll add a backend and you'll see a
corresponding benefit.

Niall

-- 
ned Productions Limited Consulting
http://www.nedproductions.biz/ 
http://ie.linkedin.com/in/nialldouglas/



Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk