Problem with asych_write() when sending more than 65536 bytes data on Solaris
We are observing one weird problem when sending message of size more than 65536 bytes using boost asynch_write on Solaris (though the problem is not observed on Linux). We are using boost version 1.47 with studio 12 CC compiler (on Solaris) and g++ 4.4.3 (on Linux). When a message of size more than 65536 (sometimes we are observing the issue with around 50K bytes size also but with 65536 bytes the issue is always reproducible) is sent, the message is delivered correctly to the receiver. But once the message is sent, the io service thread on the sender side goes into busy-wait loop. "truss" on the sender pid shows the following calls continuously. /6: recvmsg(57, 0xFFFFFD7FEB9FDDF0, 0) Err#11 EAGAIN /6: write(13, " 9\0\0\019\0\0\0", 8) = 8 /6: ioctl(13, DP_POLL, 0xFFFFFD7FEB9FE460) = 1 /6: recvmsg(57, 0xFFFFFD7FEB9FDDF0, 0) Err#11 EAGAIN /6: write(13, " 9\0\0\019\0\0\0", 8) = 8 /6: ioctl(13, DP_POLL, 0xFFFFFD7FEB9FE460) = 1 /6: recvmsg(57, 0xFFFFFD7FEB9FDDF0, 0) Err#11 EAGAIN /6: write(13, " 9\0\0\019\0\0\0", 8) = 8 /6: ioctl(13, DP_POLL, 0xFFFFFD7FEB9FE460) = 1 /6: recvmsg(57, 0xFFFFFD7FEB9FDDF0, 0) Err#11 EAGAIN /6: write(13, " 9\0\0\019\0\0\0", 8) = 8 /6: ioctl(13, DP_POLL, 0xFFFFFD7FEB9FE460) = 1 /6: recvmsg(57, 0xFFFFFD7FEB9FDDF0, 0) Err#11 EAGAIN /6: write(13, " 9\0\0\019\0\0\0", 8) = 8 /6: ioctl(13, DP_POLL, 0xFFFFFD7FEB9FE460) = 1 "lsof" shows that the socket descriptor 57 above is the socket descriptor corrected to the receiver. Thread 6 here is the ioservice thread whose stack trace is - thread t@6 -- ???????? thread_proxy boost::detail::thread_data<>::run des::tunnel::ServiceScheduler::processServiceWork boost::asio::io_service::run boost::asio::detail::task_io_service::run boost::asio::detail::task_io_service::do_one boost::asio::detail::dev_poll_reactor::run The busy-wait loop continues until we stop the receiver. Also, as mentioned, the issue is always reproducible if the size of the message more than 65536 (2^16) bytes and sender and receiver are running on 2 different Solaris hosts. If both sender and receiver are running on same Solaris host then the issue is always reproducible with size 131067 (2^17). Can someone throw some light on why the ioservice thread here is continuously trying to read from the receiver though it has successfully sent the message and receiver is not actually writing anything on the socket. Please let me know if anyone came across this problem or suggest any solution. Thanks Ranadheer -- View this message in context: http://boost.2283326.n4.nabble.com/Problem-with-asych-write-when-sending-mor... Sent from the Boost - Users mailing list archive at Nabble.com.
Here is the code to reproduce the issue (I've just changed boost asynchronous TCP daytime server sample to return the specified number of bytes message instead of daytime and put the client connection in read mode) #include <ctime> #include <iostream> #include <cstdlib> #include <string> #include <boost/bind.hpp> #include <boost/shared_ptr.hpp> #include <boost/enable_shared_from_this.hpp> #include <boost/lexical_cast.hpp> #include <boost/asio.hpp> #include <boost/scoped_array.hpp> using boost::asio::ip::tcp; std::string make_message(unsigned int message_size) { using namespace std; std::string data(message_size, 'A'); return data; } class tcp_connection : public boost::enable_shared_from_this<tcp_connection> { public: typedef boost::shared_ptr<tcp_connection> pointer; static pointer create(boost::asio::io_service& io_service, unsigned int message_size) { return pointer(new tcp_connection(io_service, message_size)); } tcp::socket& socket() { return socket_; } void handleMessage(const boost::system::error_code& message_error) { if (message_error) { std::cout<<"Error while reading the message from client"<<std::endl; } else { std::cout<<"Read the message from client"<<std::endl; } } void start() { // Perform async_read on client connection to read data from the client. _header.reset(new char[6]); _header[5] = '\0'; boost::asio::async_read(socket_, boost::asio::buffer(_header.get(), 5), boost::bind(&tcp_connection::handleMessage, shared_from_this(), boost::asio::placeholders::error)); message_ = make_message(message_size_); boost::asio::async_write(socket_, boost::asio::buffer(message_), boost::bind(&tcp_connection::handle_write, shared_from_this(), boost::asio::placeholders::error, boost::asio::placeholders::bytes_transferred)); } private: tcp_connection(boost::asio::io_service& io_service, int message_size) : socket_(io_service), message_size_(message_size) { } void handle_write(const boost::system::error_code& /*error*/, size_t bytes_transferred) { std::cout<<"Bytes written: "<< bytes_transferred << std::endl; } tcp::socket socket_; std::string message_; boost::scoped_array<char> _header; unsigned int message_size_; }; class tcp_server { public: tcp_server(boost::asio::io_service& io_service, unsigned int port, unsigned int message_size) : acceptor_(io_service, tcp::endpoint(tcp::v4(), port)), message_size_(message_size) { start_accept(); } private: void start_accept() { tcp_connection::pointer new_connection = tcp_connection::create(acceptor_.get_io_service(), message_size_); acceptor_.async_accept(new_connection->socket(), boost::bind(&tcp_server::handle_accept, this, new_connection, boost::asio::placeholders::error)); } void handle_accept(tcp_connection::pointer new_connection, const boost::system::error_code& error) { if (!error) { new_connection->start(); start_accept(); } } tcp::acceptor acceptor_; unsigned int message_size_; }; int main(int argc, char* argv[]) { if (argc != 3) { std::cerr << "Usage: server port message_size" << std::endl; return 1; } unsigned int port = boost::lexical_cast<unsigned int>(argv[1]); unsigned int message_size = boost::lexical_cast<unsigned int>(argv[2]); try { boost::asio::io_service io_service; tcp_server server(io_service, port, message_size); io_service.run(); } catch (std::exception& e) { std::cerr << e.what() << std::endl; } return 0; } For the client we can use boost synchronous TCP daytime client sample (changed to accept port as an argument) . #include <iostream> #include <boost/array.hpp> #include <boost/asio.hpp> using boost::asio::ip::tcp; int main(int argc, char* argv[]) { try { if (argc != 3) { std::cerr << "Usage: client <host> <port>" << std::endl; return 1; } boost::asio::io_service io_service; tcp::resolver resolver(io_service); tcp::resolver::query query(argv[1], argv[2]); tcp::resolver::iterator endpoint_iterator = resolver.resolve(query); tcp::resolver::iterator end; tcp::socket socket(io_service); boost::system::error_code error = boost::asio::error::host_not_found; while (error && endpoint_iterator != end) { socket.close(); socket.connect(*endpoint_iterator++, error); } if (error) throw boost::system::system_error(error); for (;;) { boost::array<char, 128> buf; boost::system::error_code error; size_t len = socket.read_some(boost::asio::buffer(buf), error); if (error == boost::asio::error::eof) break; // Connection closed cleanly by peer. else if (error) throw boost::system::system_error(error); // Some other error. std::cout.write(buf.data(), len); } } catch (std::exception& e) { std::cerr << e.what() << std::endl; } return 0; } Here is how I ran the server and client on two different solaris hosts - server 9081 200000 client <server_host> 9081 After running the server and client, do "truss" on server which will be showing tight polling loop as mentioned above. -- View this message in context: http://boost.2283326.n4.nabble.com/Problem-with-asych-write-when-sending-mor... Sent from the Boost - Users mailing list archive at Nabble.com.
participants (1)
-
Ranadheer