[Boost-users] [afio][v1.30 "C++ Now edition" released]

Subject: [Boost-users] [afio][v1.30 "C++ Now edition" released]
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2015-03-18 00:47:42

Dear Boost,

I am very pleased to announce the v1.30 release of proposed Boost 
AFIO, a Boost library implementing asynchronous file input/output 
which has been in the review queue since 2013. This is the version of 
AFIO which will be overviewed in my talk at C++ Now 2015. It also 
demonstrates the validity of the clang AST driven BindLib standalone 
Boost library conversion technique which will also be overviewed at 
C++ Now.

Source distro: 
https://github.com/BoostGSoC13/boost.afio/archive/v1.30_Boost_v1.57.zi
p

Docs and CI dashboard: https://boostgsoc13.github.io/boost.afio/

Major new features in the v1.30 release:

* Performance was improved by removing the transactional memory 
support in the v1.2 engine which is now compile time enabled only. 
Throughput now reaches up to 85% of raw ASIO and average latencies 
were reduced by 48%, yielding a 15 microsecond average latency with 
+/- 0.15 microsecond at a 95% confidence interval. This should enable 
AFIO to handle the latest generation PCIe SSDs with their 80 
microsecond latencies reasonably well.

* AFIO is now capable of complete standalone usage no Boost 
dependencies whatsoever, including for its unit testing via a 
Boost.Test emulation.

* Due to its use of libclang AST generated template aliasing to allow 
user swapping of STL 11 implementation, minimum compilers required 
are now VS2013, GCC 4.7 and clang 3.2. All the VS2010 workarounds 
baggage has been eradicated.

* All things related to async_data_op_req are now fully 
metaprogrammed with automatic generation by the compiler of suitable 
ASIO scatter-gather buffers from input types and STL containers with 
correct scatter-gather buffer coalescing where safe to do so. This 
automatic generation is user customisable via free function 
specialisation.

* Support for relative paths from an open file descriptor was added. 
This lets you specify a file descriptor as a base from which a path 
fragment
operates with the path resolution atomically performed in the kernel, 
and
therefore the path of the base file descriptor may rapidly change 
safely.

* AFIO is now optimally filing system race condition safe on all 
supported platforms, with per-API documentation of how much race 
safety is guaranteed per platform. A soak unit test verifies per 
commit that perfect race safety under severe and rapid filing system 
change is maintained.

* A new afio::path type which very thinly wraps filesystem::path was 
added. The new afio specific path type makes the management of NT 
kernel path to Win32 path transition and conversion explicit, and 
therefore much harder to accidentally get wrong. A new 
normalise_path() function can convert a NT kernel path to a 
traditional Win32 path, complete with drive letter.

* New platforms supported: Apple OS X and Android 5.0.

* New filing system features support:

All platforms: sparse file support, storage extent enumeration, 
extent deallocation ("hole punching"), volume metadata fetching 
(statfs), open file handle path discovery, hard link support, atomic 
relink (rename) support.

Windows only: file symlinking, NT lock file performance flags (lock 
file performance is now not far from POSIX), workaround semantics for 
all the ways Windows will surprise you on the filing system such as 
files and directories which won't delete. AFIO now does a fair job of 
replicating POSIX filing system semantics on Windows (and better than 
Mingw/Cygwin presently).

Linux: symlinks are now opened as file descriptors on kernels 
supporting that feature.

* Doubled the size of the tutorial, with all the new material being 
non-toy examples. There is a particularly nifty "File locking via 
only atomic appending and hole punching" algorithm implementation in 
there, and it holds up surprisingly well compared to traditional file 
locks except on ZFS. Plus, it's completely portable.

* Some utility routines in preparation for AFIO v1.4: a 
cryptographically strong random filename generator, a hugepage TLB 
STL allocator useful for efficient file copy buffers, a very highly 
optimised C++ 14 constexpr constructing arbitrary Hamming code SECDED 
template class (an amazing 22 cycles per byte for the 32784,32768 
code), and secure to-hex and from-hex validating converters.

* No shortage of bug fixes, helped hugely by additional unit testing 
for correct error handling.

Detailed changelog can be found at 
https://boostgsoc13.github.io/boost.afio/doc/html/afio/release_notes.h
tml. I'll 
tag a Boost 1.58 release on github once Boost 1.58 is out.

This release took far longer than I originally intended mainly due to 
the 
extensive debugging of weird filing system races, and for six weeks 
now I've 
been working till 5am every night after my day job. I cannot tell you 
how glad I am to see this release out! My enormous thanks to my 
family who have put up with a lot for this release, and to my 
employer for a fairly dopey employee recently. Bed, here I come!

Niall

-- 
ned Productions Limited Consulting
http://www.nedproductions.biz/ 
http://ie.linkedin.com/in/nialldouglas/