Re: [boost] Asio - is it possible not to remove descriptor from epoll?

Subject: Re: [boost] Asio - is it possible not to remove descriptor from epoll?
From: Gruenke, Matt (mgruenke_at_[hidden])
Date: 2011-03-23 18:56:04

I can't help you with the way that ASIO uses epoll, but perhaps you can write your own io_service?
 
You could take the other approach and work around this part of ASIO, all together.  Get the socket's native_type and use it directly with epoll.  Your requirements seem so demanding that perhaps you'll just have to give up some of what ASIO provides.  At least this way, you can still get some of its benefits.
 
http://www.boost.org/doc/libs/release/doc/html/boost_asio/reference/basic_raw_socket/native.html <http://www.boost.org/doc/libs/release/doc/html/boost_asio/reference/basic_raw_socket/native.html> 
 
 
What I mentioned before is that in order not to block on non-async reads, you can put the socket in non-blocking mode.  See:
 
http://www.boost.org/doc/libs/release/doc/html/boost_asio/reference/basic_raw_socket/non_blocking_io.html <http://www.boost.org/doc/libs/release/doc/html/boost_asio/reference/basic_raw_socket/non_blocking_io.html> 
 
 
This should map to O_NONBLOCK, on Linux.
 
 
Matt

________________________________

From: Vlad Lazarenko [mailto:vlad_at_lazarenko.me]
Sent: Wed 3/23/2011 5:27 PM
To: boost_at_[hidden]
Cc: Gruenke, Matt
Subject: Re: [boost] Asio - is it possible not to remove descriptor from epoll?

Asio does add-and-remove on every operation which is a bit of a problem here. Doing reads util the data is exhausted won't help. I think that it can even hurt because blocking system call will put thread to sleep if there is no data and also block other consumers running of the same "io_service" (run () is called by a single thread). In case of multicast, I always get complete packet (or nothing). 

The background is that I have a client application for server(s) pushing tremendous amount of data over multicast. Latency is a huge concern (we are talking sub*micro*seconds). So locking on epoll to add/remove events (or doing blocking calls) hurts. Basically, I optimized everything - from not calling "bind" every time, not copying function internals, and re-using the same pre-allocated memory to store state associated with read. Re-registering event in epoll is the last thing. I hope to find a way to stick this feature into Asio or somehow get epoll internals and work with this socket myself.

Thanks,
Vlad

On Wed, Mar 23, 2011 at 4:58 PM, Gruenke, Matt <mgruenke_at_[hidden]> wrote:

        Can't you simply put the descriptor in non-blocking mode and, in your async_read handler, perform further non-async reads until the waiting data is exhausted?  I don't know how ASIO uses epoll, but if it's doing an add-and-remove for every async operation, you'd at least get rid of the overhead of extra adds & removes while data is still wating.
        
        
        Matt