Re: [boost] [afio] Formal review of Boost.AFIO

Subject: Re: [boost] [afio] Formal review of Boost.AFIO
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2015-08-31 13:19:42

On 31 Aug 2015 at 17:20, Andreas Schäfer wrote:

> - The naive OpenMP example is using a thread pool, AFIO is using these
>   elaborate, efficient futures, continuations etc. I'd have expected
>   this to be more efficient, especially given the nature of the OpenMP
>   example (which does a sequential directory walk before spawning
>   threads).

More code executed usually means slower performance.

> - Even if we assume that AFIO can't be faster than an OpenMP code in
>   this setting: why is the overhead so high? What is making it >2x
>   slower?

I have been as transparent and honest about the inefficiencies in 
AFIO as I can be. I have listed copious benchmarks and graphs in the 
AFIO documentation which show AFIO in as poor a light as I can make 
it. I have identified another round of low hanging fruit which I feel 
lightweight futures will help me solve, and I am about to go 
implement. What more do you want here?

In the end AFIO does a lot more than a naïve OpenMP solution, and I 
never expect it to be quicker without a true kernel async backend to 
offset the extra work performed. It executes more syscalls and does 
more processing. In exchange, you get much superior guarantees and 
reliability.

If you don't want those extra guarantees, then don't use AFIO. As I 
have suggested on multiple occasions now, ASIO has a perfectly fine 
async file i/o library.

> > What AFIO provides is a *portable* API for async file system and file
> > i/o. It provides the least benefit on Linux of any of the platforms,
> > but that is not AFIO's fault.
> 
> Sorry, but that's not true. It remains AFIO's fault as long as you
> can't explain where this slowdown comes from. Why don't you, say,
> default to a naive, simple OpenMP implementation on Linux?

I have repeatedly explained where I believe slowdowns come from. You 
keep choosing to not listen, twist and cherry pick, and harrass me 
again and again over the same items. I could guess at your motives 
given you are also in German HPC, but I will leave it at that.

I have now had enough of this harrassment and repeatedly explaining 
the same thing over and over. You have your answers and you have made 
your vote. Please now be quiet and let others speak.

Niall

-- 
ned Productions Limited Consulting
http://www.nedproductions.biz/ 
http://ie.linkedin.com/in/nialldouglas/