Re: [boost] [AFIO] Formal review

Subject: Re: [boost] [AFIO] Formal review
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2015-09-02 15:18:48

On 2 Sep 2015 at 14:46, Andrey Semashev wrote:

> > You can disable the race free semantics, which are slow, using
> > afio::file_flags::no_race_protection.
> 
> Will this avoid obtaining any unnecessary data? I.e. will opening a file 
> then be equivalent to a single syscall, which is to open the file? If 
> so, I think this should be the default and any additional data to obtain 
> should be an opt-in.

It definitely breaks race protection, and you lose an undefined 
amount of that i.e. there is still some race protection, but an 
undefined quantity.

As for less work, it gives no guarantees. Probably yes it will 
involve many fewer syscalls.

The reason it will never be defaulted on is because it introduces 
substantial behaviour differences between platforms. Even FreeBSD and 
Linux will no longer behave the same. That's whole point of the 
emulation and workaround code to fix up platform specific differences 
so you don't need to think about that when writing code against AFIO. 
You can program against a standardised, universal filesystem model 
and not worry about platform specific differences (mostly).

If you do disable race protection - and you can turn it off per 
operation, per handle or per dispatcher - you are explicitly 
exchanging predictability for an unspecified performance increase.

> I think you target two very different goals - provide an async API for 
> file system operations and provide a race-free API to file system. One 
> of the key benefits of making things async is to make it more 
> responsive, if not plain faster.

I will repeat myself once again because you are not listening to me: 
making things async without a fundamental rearchitecture makes them 
*slower* not faster. That even goes for socket i/o - feel free to 
test 1000 threads using a synchronous socket against single threaded 
ASIO with 1000 sockets.

Here are some empirical benchmarks by Google: 
https://www.mailinator.com/tymaPaulMultithreaded.pdf.

Async i/o is *always* slower because you are doing more work. It 
didn't used to be once upon a time, but in any OS of the past decade 
it is - as that Google presentation shows.

The ONLY good reason to go async is CONTROL. You can achieve better 
worst case performance with async. You can achieve more easily better 
design with async. That goes for sockets as well as file i/o.

> If that is not achieved then there is 
> no point in being async as it just complicates the code. I think many 
> reviewers who took a look at Boost.AFIO were looking for this kind of 
> benefit and seeing you saying that Boost.AFIO should not be used for 
> that is surprising and discouraging. (This is why I questioned the name 
> AFIO above.)

Many reviewers have inaccurate starting preconceptions. They believe 
async automatically correlates with faster. That might have been true 
in the 1990s and early 2000s. It is no longer true, without a 
fundamental rearchitecture i.e. throw out all your design 
preconceptions.

> Race-free API may be interesting on its own, but apparently being 
> race-free hampers performance, which is essential for the first goal. 

Again, I repeat the point about control and worst case behaviour.

AFIO can and does produce superb file system performance an order of 
magnitude or more in excess of anything achievable with the STL or 
anything in Boost. But you'll need to completely rewrite your 
algorithms according to file system theory first.

If I don't reply to any reply you make, I will have gone on my 
holidays away from Boost. Thanks though for the feedback Andrey.

Niall

-- 
ned Productions Limited Consulting
http://www.nedproductions.biz/ 
http://ie.linkedin.com/in/nialldouglas/