[Boost-users] [asio] continuous reads into streambuf

Subject: [Boost-users] [asio] continuous reads into streambuf
From: Stephan Menzel (stephan.menzel_at_[hidden])
Date: 2018-05-24 14:59:40

Hello all,

once again I find myself struggling with asio streambufs but wanted to ask
here before chicken out to char buffers again.
My problem is that I cannot read more than one protocol loop successfully.

I got a TCP connection here that continuously receives messages on a simple
protocol like that:

header_size CRLF
header             (contains payload size)
payload

<repeat>

I thought this is a prime use case for reading into a streambuf as I can
easily deal with additional data being in there and the client pushing as
fast as it can.

Here is what I do (simplified quite a bit, sorry for copy and paste and
simplify errors):

(m_socket and m_streambuf are only used by one io_context, one thread, no
races as far as I can tell)

void read_header_size() {

  asio::async_read_until(m_socket, m_streambuf, "\r\n",
    [self](const boost::system::error_code, const std::size_t) {
        // < handle error skipped >
        std::string hsizestr;
        hsizestr.reserve(32);
        {
            std::istream is(&self->m_streambuf);
            std::getline(is, hsizestr);
        }
        std::size_t header_size = parse(hsizestr);
        self->read_header(header_size);
  });
}

void read_header(const std::size_t n_header_size) {

  asio::async_read(m_socket, m_streambuf,
boost::asio::transfer_exactly(n_header_size),
    [self, packet, n_header_size](const boost::system::error_code, const
std::size_t) {

        // < handle error skipped >
        // Tried two versions of parsing the header (manual consume and
istreams of various sorts)
        // manual consume:
        size_t payload_size = parse_header(n_streambuf.data().data(),
n_header_size)) {
n_streambuf.consume(n_header_size);

        self->read_payload(payload_size);
    });
}

void read_payload(const std::size_t n_payload_size) {
  asio::async_read(m_socket, m_streambuf,
boost::asio::transfer_exactly(payload_size),
    [self, n_packet, packet_size](const boost::system::error_code, const
std::size_t) {
         // < handle error skipped >
         {
    std::istream is(&self->m_streambuf, std::ios::binary);
            is.read(target_buffer, target_buffer_size);                //
target buffer is elsewhere and well guarded
         }

         // read next message
         self->read_header_size();
    });
}

When I run this and make sure the client pushes at least 2 packets I can
safely receive and process the first but after that the next asnc_read() in
read_header never returns. As in never calls its handler. From debugging
and looking at the streambuf I can reasonably assume the buffer contains
all the data after the first async_read_until because the packages are very
small (only about 60 bytes total).
This leads me to believe the transfer_exactly don't return unless something
is actually really transferred. But if such is the case, why does the first
loop succeed?

I am quite lost here. If anybody has some suggestion it is much appreciated.

Thanks a bunch!
Stephan