[boost] [asio][coroutine] forbid_suspend blocks

Subject: [boost] [asio][coroutine] forbid_suspend blocks
From: Vinícius dos Santos Oliveira (vini.ipsmaker_at_[hidden])
Date: 2018-08-15 12:06:07

I've implemented a dumb linter using Python + clang API that allows you to
mark certain blocks as “atomic in respect to coroutines” using C++11
attributes.

Usage example follows:

void f() {
    int yield;
    g(yield);
    [[bt::forbid_suspend]] {
        // Comment this call to make linter accept the code
        g(yield);
        t();
    }
}

You can change bt::forbid_suspend to another identifier by tuning settings
on the beginning of the script. The prefix forbid was inspired by Rust's
linters[1] as you cannot disallow the linter from inside the forbid_suspend
block.

Some limitations follow:

   - A suspend point is identified by looking references to any id equals
   to yield in the last argument passed to function calls. It means that
   yield[ignored_ec] and alike will be correctly identified, but it needs to
   be named yield and it needs to be the last argument to the function call.
   - I haven't found a way to extract attributes related to COMPOUND_STMT
   nodes with clang API, so I manually look for token gaps and do a simple
   regex. It means you cannot use macros to [[bt::forbid_suspend]] as the
   macro won't be expanded.
   - You better call the script with '-x c++' and also '
   -I/usr/lib/clang/5.0.1/include'.

Like I've said in the beginning of the email, it's a *dumb* linter, which
means it doesn't try to handle all cases. Therefore, I don't hope to do
many changes to it or maintain it to work against newer releases of clang
and I'm only uploading it to gist (not fullblow Github): <
https://gist.github.com/vinipsmaker2/d930fbe5b7597432b021effe618da171>. It
should also be easy to hack as it is very small. I'm releasing the linter
under public domain. Enjoy.

I have been using fibers in a daily basis on the last months of my job to
solve networking problems on a gateway project. Some of the technologies I
use are:

   - Boost.Asio and its spawn function.
   - A few custom synchronization primitives for the fibers (e.g. mutex,
   semaphore).
   - Eventually I'll have to write a custom spawn function that returns a
   joinable handle so I can kill the usage of a fiber::barrier in a few places.

I've been using the linter for about a week already and looks fine. My use
case are:

   - To prevent iterator invalidation in regions accessing shared variables.
   - To guarantee events order in algorithms that handle races (cannot
   yield while task YYY hasn't finished).

The project I use it in is not an open source project, so I cannot talk
much more about it.

Also, I don't claim my solution is original. I wouldn't be surprised if the
very same solution exists in the wild (please let me know if you know of a
similar approach). This is not a new problem. It is know for some time. For
instance, in P0171, Gor Nishanov noted this same problem:

“In coroutines the suspend point is clearly marked with await, which tells
the reader that something unusual happens in this function and allows the
reader, for example, to confirm whether the lifetimes of the objects of
interest align with the lifetime of the coroutine or not, whether some
locks need to be acquired to protect some concurrently accessed data, and
whether some locks need to be released before the execution reaches the
suspend point.”
  — <http://open-std.org/jtc1/sc22/wg21/docs/papers/2015/p0171r0.html>, 2015

However, the solution he presented was to manually examine whole blocks of
code without help from the compiler. I argue that the solution he presented
is as error-prone as manually looking for exit paths that don't free
acquired resources in languages lacking RAII. And I've been spoiled too
much by RAII. Furthermore, I take this comment of his with highly
suspicious look:

“Since we are on the subject of maintenance nightmares, we would like to
offer a conjecture that the absence of the await in P0114 is a likely
source of many maintenance nightmares. Without a syntactic marker to signal
to the person reading the code that something funny is going on, it is
impossible to tell whether the following code is correct or not”
  — http://open-std.org/jtc1/sc22/wg21/docs/papers/2015/p0171r0.html

If I had to give a vote to the proposal he was defending or Chris'
proposal, I'd probably have chosen Chris'.

[1] https://doc.rust-lang.org/reference/attributes.html#
lint-check-attributes

-- 
Vinícius dos Santos Oliveira
https://vinipsmaker.github.io/