[Boost-users] boost::asio blocking socket read with timeout with multiple threads

Subject: [Boost-users] boost::asio blocking socket read with timeout with multiple threads
From: tom_at_[hidden]
Date: 2018-03-15 19:04:19

I've got a boost::asio based server, and generally it's working well. The server runs multiple threads, each simply running the io_service::run method on the same io_service object. It accepts multiple connections and processes multiple concurrent sockets. It uses a strand per connection to ensure that the data for each connection is processed strictly sequentially.

For various reasons it uses a mixture of async and sync reads. The trouble I'm having is implementing a timeout on the synchronous (blocking) reads. I want to protect the server in the case of a client sending an erroneous or malicious packet for example, and make sure a read doesn't block forever.

There was a post back in 2007 that gave a technique (https://lists.boost.org/Archives/boost/2007/04/120339.php). There's a stack overflow question that references that (https://stackoverflow.com/questions/13126776/asioread-with-timeout). There are some examples in the docs that perform a similar kind of technique. 

In the examples, the server is all asynchronous. The client is synchronous, but only uses one thread. The example from 2007 also only uses one thread. 

The examples all revolve around the technique of starting an *async* read, then performing a nested loop of io_service::run_one. However, so far I've been unable to find a form of code that works reliably in a multithreaded environment.

The basis for my experimentation is here: https://gist.github.com/anonymous/1160c11f8ed9c29b9184325191a3a63b
It starts a server thread, then starts a client that makes a connection and then writes nothing, to simulate a "bad" client and to provoke a "timeout" condition. 

What I'm finding is this:
With one server thread I can make it work OK. The nested loop of "run_one()" operates correctly. 

With multiple server threads though, I can't make it work reliably.

The basis of the loop is:
                bool timedOut = false;
                bool readComplete = false;

                timer.async_wait(strand_.wrap(boost::bind(&handleReadTimeout, &timedOut, &socket_, _1)));
                async_read(socket_, buffer(b),
                        strand_.wrap(boost::bind(handleReadComplete, &ec, &bytesRead, &readComplete, &timer, _1, _2)));

                while (socket_.get_io_service().run_one()) {
                        if (timedOut && readComplete) {
                                break;
                        }
                }

handleReadTimeout sets the timedOut flag to true and calls "cancel" on the socket. The handleReadComplete callback sets readComplete  to true and calls cancel on the timer.

With multiple threads, the handleReadTimeout/handleReadComplete callbacks are run on other threads. So the while loop here just blocks, as there is never anything to run. That's my surmise of what's going on anyway. I've experimented with strands to try and force it all onto the same thread, but so far failed (if the above code is called in the context of the same strand, it just seems to block the handleReadTimeout and handleReadComplete callbacks from being called).

Note that if I expend my example to include other clients talking to a simple "echo" service to simulate other traffic, then the while loop above *does* wake, as that other traffic gives it something to do. But that's fragile. 

The alternative formulation, the one I started with actually, is to do an async wait on the timer, and a normal synchronous read on the socket. The timer callback performs a cancel (or close -- I tried both) on the socket hoping to cause the socket read to error. This is the kind of thing you'd do if you were programming raw sockets. That works fine on Windows, but won't work on Linux. The documentation for cancel does say it cancels any outstanding *async* calls, but I'm surprised calling close doesn't work and cause the read to wake. 

So I'm stuck. I was hoping someone here could point me in the right direction, as it seems an obvious thing (to me) to want try and do. Is there a form of code that allows me to perform a synchronous read from a socket and time it out, which will work in a multi threaded/single io service environment? 

This is all with boost 1.64 by the way.

Thanks for any help in advance.