Re: [Boost-users] [iostreams] Devices and WOULD_BLOCK

Subject: Re: [Boost-users] [iostreams] Devices and WOULD_BLOCK
From: David Hawkins (dwh_at_[hidden])
Date: 2015-01-22 00:40:12

Hi Gavin,

>> In theory the filtering_stream 'filter' and 'device' layers should
>> pass characters, EOF, and WOULD_BLOCK. Unfortunately the code snippet
>> I posted suppresses the propagation of the WOULD_BLOCK return value
>> from the 'device' layer to the 'filter' layer.
>
> Generally I would recommend using only blocking or async code.
> Non-blocking is a holdover from when async didn't really exist yet, and
> with ASIO you shouldn't have that excuse.

Right, I'd be happy using async code, but this particular feature is
new with filtering_streams - per the documentation

http://www.boost.org/doc/libs/1_57_0/libs/iostreams/doc/guide/asynchronous.html

Since async support is new, it might not be fully supported (or at
least fully debugged). Now that I am getting more familiar with
the boost code, I need to dig around a little more in the
Boost.Iostream example and test folders.

>> The Boost chat client/server has similar features to my problem,
>> i.e., it involves encoding/decoding messages. The encoding/decoding
>> is different than my case, since the chat protocol adds a
>> header with the message length. This simplifies the socket read
>> code, since you can read the fixed-length header, then read the
>> now known) message length, i.e., each call to read has a fixed
>> length parameter. In my case, the message length is unknown, and
>> depends on the content of the message (since data can be escaped).
>> The data stream is generated by a field-programmable gate array
>> (FPGA), and adding a buffer to determine the message length before
>> sending the response would use too many resources, so the encoding
>> protocol cannot easily be changed.
>
> My point is that theoretically this shouldn't be a problem, as long as
> the lowest level (actual device reading) behaves as described earlier.
>
> When you read(buffer, 512) it would block until the device provides some
> data; perhaps it sends a 28 byte packet, of which the driver eagerly
> grabs the first 8 bytes and so read() returns 8; you look at that data,
> decide it's not a complete message yet, so you stash it into a separate
> buffer and call read(buffer, 512) again.  (There are other approaches if
> you want zero-copy.)  This time the driver already had the remaining 20
> bytes queued up and so read() immediately returns 20.  You tack those
> bytes onto the end of your prior stash, parse the whole thing, and now
> you've got a message you can return to a higher layer, and then go back
> to reading.  Async works similarly, you just break the code up a little
> more so that it can return while it's waiting.

Yes, this is what I'm in the process of getting working with my
modified version of the chat client/server. The async_read_until
callbacks do nicely sequence through my simplified packet protocol.

> (Just remember that it might also return a few bytes from the start of
> the *next* message, so you need to strip exactly one message out of the
> stash and keep accumulating bytes for the next message.)

Yep, I ran a few tests where I ensured that two complete messages
were received into a boost::asio::streambuf and confirmed that
read_until '[' returned the SOP first index, read_until ']' returned
the first EOP index, I then consume()'d those characters and
repeated the call to read_until and confirmed it returned immediately
based on the streambuf contents. So this all works as described by
the documentation.

---
What is the policy of this list with regards to posting code inline
in messages?
Once I finish my variation on the chat client/server, I'd be happy
to post the code. At a minimum it would provide code for people on
the list to review/comment on, and any final version of the code
would benefit anyone interested in reading streams containing
a different style of packet than that used in the boost example
chat client/server.
I'll then go back to looking at what I did wrong with filtering_streams
'filter' and 'devices', or see if I can get the async stuff working.
Thanks again for the helpful discussion.
Cheers,
Dave