Boost logo

Boost Users :

Subject: Re: [Boost-users] [iostreams] Devices and WOULD_BLOCK
From: David Hawkins (dwh_at_[hidden])
Date: 2015-01-22 00:40:12

Hi Gavin,

>> In theory the filtering_stream 'filter' and 'device' layers should
>> pass characters, EOF, and WOULD_BLOCK. Unfortunately the code snippet
>> I posted suppresses the propagation of the WOULD_BLOCK return value
>> from the 'device' layer to the 'filter' layer.
> Generally I would recommend using only blocking or async code.
> Non-blocking is a holdover from when async didn't really exist yet, and
> with ASIO you shouldn't have that excuse.

Right, I'd be happy using async code, but this particular feature is
new with filtering_streams - per the documentation

Since async support is new, it might not be fully supported (or at
least fully debugged). Now that I am getting more familiar with
the boost code, I need to dig around a little more in the
Boost.Iostream example and test folders.

>> The Boost chat client/server has similar features to my problem,
>> i.e., it involves encoding/decoding messages. The encoding/decoding
>> is different than my case, since the chat protocol adds a
>> header with the message length. This simplifies the socket read
>> code, since you can read the fixed-length header, then read the
>> now known) message length, i.e., each call to read has a fixed
>> length parameter. In my case, the message length is unknown, and
>> depends on the content of the message (since data can be escaped).
>> The data stream is generated by a field-programmable gate array
>> (FPGA), and adding a buffer to determine the message length before
>> sending the response would use too many resources, so the encoding
>> protocol cannot easily be changed.
> My point is that theoretically this shouldn't be a problem, as long as
> the lowest level (actual device reading) behaves as described earlier.
> When you read(buffer, 512) it would block until the device provides some
> data; perhaps it sends a 28 byte packet, of which the driver eagerly
> grabs the first 8 bytes and so read() returns 8; you look at that data,
> decide it's not a complete message yet, so you stash it into a separate
> buffer and call read(buffer, 512) again. (There are other approaches if
> you want zero-copy.) This time the driver already had the remaining 20
> bytes queued up and so read() immediately returns 20. You tack those
> bytes onto the end of your prior stash, parse the whole thing, and now
> you've got a message you can return to a higher layer, and then go back
> to reading. Async works similarly, you just break the code up a little
> more so that it can return while it's waiting.

Yes, this is what I'm in the process of getting working with my
modified version of the chat client/server. The async_read_until
callbacks do nicely sequence through my simplified packet protocol.

> (Just remember that it might also return a few bytes from the start of
> the *next* message, so you need to strip exactly one message out of the
> stash and keep accumulating bytes for the next message.)

Yep, I ran a few tests where I ensured that two complete messages
were received into a boost::asio::streambuf and confirmed that
read_until '[' returned the SOP first index, read_until ']' returned
the first EOP index, I then consume()'d those characters and
repeated the call to read_until and confirmed it returned immediately
based on the streambuf contents. So this all works as described by
the documentation.

What is the policy of this list with regards to posting code inline
in messages?
Once I finish my variation on the chat client/server, I'd be happy
to post the code. At a minimum it would provide code for people on
the list to review/comment on, and any final version of the code
would benefit anyone interested in reading streams containing
a different style of packet than that used in the boost example
chat client/server.
I'll then go back to looking at what I did wrong with filtering_streams
'filter' and 'devices', or see if I can get the async stuff working.
Thanks again for the helpful discussion.

Boost-users list run by williamkempf at, kalb at, bjorn.karlsson at, gregod at, wekempf at