Re: [boost] Questions about upcoming review of Boost.AFIO

Subject: Re: [boost] Questions about upcoming review of Boost.AFIO
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2015-05-20 21:08:00

Many replies, thank you everyone, replied to each in a single email 
to avoid repeating myself:

On 20 May 2015 at 10:55, Robert Ramey wrote:

> On 5/20/15 9:56 AM, Niall Douglas wrote:
> > Question: Given the API will so radically change, is now the time 
for
> > a community review? Do note that APIBind allows me to support the
> > existing API and the new API concurrently without issue.
> 
> Given the above, I would very strongly recommend that you postpone 
the 
> review until you feel that the submission reflects what you want 
the 
> library to look like.  How would you feel if it was rejected - 
wouldn't 
> you just kick yourself for not delaying for the submission of the 
> presumably best version you can submit.

I feel the current API *is* the best version I can submit given the 
constraints (VS2013, ABI stability) imposed.

I also have done *three* full release iterations now to get AFIO 
ready for peer review. They take weeks, each one, with dozens of 
hours of unit and stability soak testing done each time. I am kicking 
myself for having effectively prepared a new Boost library for review 
three times now, with arguably three times the work of others in the 
review queue. I'd really like to see the ball start rolling. I have 
been waiting in the queue two years now. I'd like to see a review so 
I know what I am doing wrong, and what I should be doing instead.

>  As far as I know in the whole 
> history of Boost there is only one time that a library was reject 
only 
> to be accepted after a subsequent submission.  Save yourself the 
agony
> of doing this more than once.

I would expect a conditional acceptance. And I'll tell you why: I am 
unaware of anyone using AFIO in production code. Until at least one 
person is, I myself would not recommend unconditional acceptance.

The problem here really is chicken and egg, plus asynchronous file 
and filesystem is a very, very niche use case. I would wonder if any 
Boost library is needed by so few users in fact.

However for that tiny number of users, async file and filesystem is a 
god send. I have some benchmarks in the tutorial which show just how 
much control async can give the programmer - find in files regex is 
quite literally warm cache double the speed of a naïve multithreaded 
algorithm with almost no cold cache penalty over single threaded, 
unlike the -50% cold cache penalty of a naive multithreaded 
algorithm.

The race freedom guarantees on filesystem are also a god send for 
that tiny minority who care.

On 20 May 2015 at 22:33, Andrey Semashev wrote:

> > Question: Given the API will so radically change, is now the time 
for
> > a community review? Do note that APIBind allows me to support the
> > existing API and the new API concurrently without issue.
> 
> IMO, the library should be in a more or less finalized state to be
> reviewed. After all, the review verdict shows acceptance or rejection of
> the library in its form during the review, not something entirely
> different that it will become in the future. 

Be aware that even with the next release after the next release new 
API variants are not finished, as I intend to add a Microsoft COM API 
with C API wrappers in a still further away release. That will mean 
no less than four separate APIs with which to work with AFIO:

(i) ABI stable (the current one, will remain more or less in the 
future)
(ii) Monadic programming (next release)
(iii) Fiber programming (via proposed Boost.Fiber) which also doubles 
as the synchronous API WG21 want (next release).
(iv) Microsoft COM API with C bindings (release after next release).

I would also see any review being that of the library in its present 
form. Any future changes are, well, in the future and not in the 
scope of any review now.

I would certainly hope that any review now might find problems very 
useful to know about now before any API refactor begins.

On 20 May 2015 at 21:35, Asbjørn wrote:

> The OSS project I worked on had enough issues with Boost.Filesystem v2
> to v3 transition for it to be somewhat of a pain, and something to be
> avoided if possible. 
>
> If AFIO was included into Boost with an already obsolete API I'd stay
> away anyway until the new API was released. 

Firstly, the current API isn't obsolete. It's fast, ABI stable, and 
very portable being deliberately designed to be usable from C. Future 
editions of AFIO will retain a mostly API compatible level with the 
only breaking change being std::future turning into afio::future and 
that error_code is now returnable from the afio::future in addition 
to exceptions. The monadic and fiber APIs almost certainly will wrap 
the current API, and therefore will be slightly slower.

Boost libraries already undergo substantial API evolution. Look at 
Boost.Thread - it provides four API versions, all with breaking 
changes. I am unaware that any of those underwent a formal review, 
rather a post was made here informing people of any change in the 
default version provided (which indeed was rolled back last time).

Also AFIO in its present form is 100% compatible with any future AFIO 
implementation even in the same translation unit. I can actually 
*guarantee* you that your code will not break in future Boost 
releases, unlike almost any maintainer here because I can ship N 
separate versions of AFIO and none will conflict.

Do also remember AFIO is completely independent of Boost. You can use 
AFIO, including any past version, at any time including in 
combination with any future version with no ODR violation. This makes 
it very unlike Filesystem, or any other library here.

On 21 May 2015 at 1:30, Peter Dimov wrote:

> > The internal engine is the same very well tested code, just the API is 
> > very different from before. The current API was forced by VS2010's 
> > limitations and the lack of constexpr in MSVC - now VS2015 RTM has C++ 11 
> > constexpr, I can adopt the custom future based API I always originally 
> > wanted. This new API will require C++ 14 in the compiler.
> 
> What is your contingency plan if people loudly demand C++11 support (or even 
> C++03 support)?

Most of the upcoming C++ 11 mandatory libraries require C++ 14. One 
already conditionally approved (Fiber) requires C++ 14.

I think people also think in terms of compilers supported rather than 
C++ editions used.

> Would it be possible to for the new API to degrade more or less gracefully 
> to C++11, or is it C++14 or nothing?
>
> The answers to these questions may prove useful for your final decision. 

In the refactor I'm aiming for the subset of C++ 14 provided by 
VS2015 RTM. That's about clang 3.4 levels minus variable templates 
and relaxed constexpr.

This new API refactor won't happen till 2016 BTW. No time between now 
and then to go any faster. Plus I need Expression SFINAE, and VS2015 
SP1 may have that so no point jumping before MSVC is ready.

Right now AFIO v1.3 demands >= clang 3.2, >= GCC 4.7, >= VS2013. The 
-std=c++11 is sufficient, though -std=c++14 will produce faster code.

Niall

-- 
ned Productions Limited Consulting
http://www.nedproductions.biz/ 
http://ie.linkedin.com/in/nialldouglas/