Boost logo

Boost :

Subject: Re: [boost] [AFIO] Formal review
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2015-09-02 15:18:48

On 2 Sep 2015 at 14:46, Andrey Semashev wrote:

> > You can disable the race free semantics, which are slow, using
> > afio::file_flags::no_race_protection.
> Will this avoid obtaining any unnecessary data? I.e. will opening a file
> then be equivalent to a single syscall, which is to open the file? If
> so, I think this should be the default and any additional data to obtain
> should be an opt-in.

It definitely breaks race protection, and you lose an undefined
amount of that i.e. there is still some race protection, but an
undefined quantity.

As for less work, it gives no guarantees. Probably yes it will
involve many fewer syscalls.

The reason it will never be defaulted on is because it introduces
substantial behaviour differences between platforms. Even FreeBSD and
Linux will no longer behave the same. That's whole point of the
emulation and workaround code to fix up platform specific differences
so you don't need to think about that when writing code against AFIO.
You can program against a standardised, universal filesystem model
and not worry about platform specific differences (mostly).

If you do disable race protection - and you can turn it off per
operation, per handle or per dispatcher - you are explicitly
exchanging predictability for an unspecified performance increase.

> I think you target two very different goals - provide an async API for
> file system operations and provide a race-free API to file system. One
> of the key benefits of making things async is to make it more
> responsive, if not plain faster.

I will repeat myself once again because you are not listening to me:
making things async without a fundamental rearchitecture makes them
*slower* not faster. That even goes for socket i/o - feel free to
test 1000 threads using a synchronous socket against single threaded
ASIO with 1000 sockets.

Here are some empirical benchmarks by Google:

Async i/o is *always* slower because you are doing more work. It
didn't used to be once upon a time, but in any OS of the past decade
it is - as that Google presentation shows.

The ONLY good reason to go async is CONTROL. You can achieve better
worst case performance with async. You can achieve more easily better
design with async. That goes for sockets as well as file i/o.

> If that is not achieved then there is
> no point in being async as it just complicates the code. I think many
> reviewers who took a look at Boost.AFIO were looking for this kind of
> benefit and seeing you saying that Boost.AFIO should not be used for
> that is surprising and discouraging. (This is why I questioned the name
> AFIO above.)

Many reviewers have inaccurate starting preconceptions. They believe
async automatically correlates with faster. That might have been true
in the 1990s and early 2000s. It is no longer true, without a
fundamental rearchitecture i.e. throw out all your design

> Race-free API may be interesting on its own, but apparently being
> race-free hampers performance, which is essential for the first goal.

Again, I repeat the point about control and worst case behaviour.

AFIO can and does produce superb file system performance an order of
magnitude or more in excess of anything achievable with the STL or
anything in Boost. But you'll need to completely rewrite your
algorithms according to file system theory first.

If I don't reply to any reply you make, I will have gone on my
holidays away from Boost. Thanks though for the feedback Andrey.


ned Productions Limited Consulting

Boost list run by bdawes at, gregod at, cpdaniel at, john at