Subject: Re: [boost] [AFIO] Formal review
From: Andrey Semashev (andrey.semashev_at_[hidden])
Date: 2015-09-02 18:53:15
On 02.09.2015 22:18, Niall Douglas wrote:
> On 2 Sep 2015 at 14:46, Andrey Semashev wrote:
> The reason it will never be defaulted on is because it introduces
> substantial behaviour differences between platforms. Even FreeBSD and
> Linux will no longer behave the same. That's whole point of the
> emulation and workaround code to fix up platform specific differences
> so you don't need to think about that when writing code against AFIO.
> You can program against a standardised, universal filesystem model
> and not worry about platform specific differences (mostly).
> If you do disable race protection - and you can turn it off per
> operation, per handle or per dispatcher - you are explicitly
> exchanging predictability for an unspecified performance increase.
Ok, so it's not about just being race-free, it's about providing a
portable API. The cost on this I can accept, as long as the portable API
is not requiring too much emulation on every platform. I'm not sure this
precondition is fulfilled though, based on your comments. I mean, what
portable API would require obtaining file metadata and actual path on
opening a file?
>> I think you target two very different goals - provide an async API for
>> file system operations and provide a race-free API to file system. One
>> of the key benefits of making things async is to make it more
>> responsive, if not plain faster.
> I will repeat myself once again because you are not listening to me:
> making things async without a fundamental rearchitecture makes them
> *slower* not faster. That even goes for socket i/o - feel free to
> test 1000 threads using a synchronous socket against single threaded
> ASIO with 1000 sockets.
I wasn't comparing a _single_ threaded async solution to a 1000-thread
sync one - that would be silly of me. I would expect AFIO to perform
async operations in parallel, involving a thread pool with optimal
number of threads, if necessary. I would expect it to be about as fast,
and possibly use less memory too. Much depends on the async reactor
>> If that is not achieved then there is
>> no point in being async as it just complicates the code. I think many
>> reviewers who took a look at Boost.AFIO were looking for this kind of
>> benefit and seeing you saying that Boost.AFIO should not be used for
>> that is surprising and discouraging. (This is why I questioned the name
>> AFIO above.)
> Many reviewers have inaccurate starting preconceptions. They believe
> async automatically correlates with faster. That might have been true
> in the 1990s and early 2000s. It is no longer true, without a
> fundamental rearchitecture i.e. throw out all your design
Noone's arguing about having to design your code properly to make use of
Async programming is a form of parallel programming, just like
multithreaded programming is. Either can be slower or faster than the
other for different reasons, but both are arguably faster than an
equivalent synchronous single threaded program. That's where the speedup
expectation is coming from, and I think it's a fair expectation.
Async code is notably harder to write than multithreaded, so you have to
give something in return if you design your library the async way. If
not performance or resource utilization then what? You mention control
and worst case performance, but I can't see why an async program would
do any better in this regard than an equivalent multithreaded sync one.
Can you give a more concrete example when it does?
>> Race-free API may be interesting on its own, but apparently being
>> race-free hampers performance, which is essential for the first goal.
> Again, I repeat the point about control and worst case behaviour.
> AFIO can and does produce superb file system performance an order of
> magnitude or more in excess of anything achievable with the STL or
> anything in Boost. But you'll need to completely rewrite your
> algorithms according to file system theory first.
I'm sorry, you lost me here. You just said that AFIO, being an async
library, will always be slower than STL, and now you say its performance
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk