Re: [Boost-users] [iostreams] Devices and WOULD_BLOCK

Subject: Re: [Boost-users] [iostreams] Devices and WOULD_BLOCK
From: David Hawkins (dwh_at_[hidden])
Date: 2015-01-21 18:56:55

Hi Gavin,

>> The "problem" with the protocol I need to decode (which I have
>> no choice to change) is that the data stream possibly contains
>> escaped characters, so there is no way to know the read data
>> length at the socket layer - its up to the filtering_stream layers
>> to request data from the device layer until a complete packet is
>> decoded.
>>
>> This all sounds good in theory, but in practice the filter layers
>> attempt to read in blocks, and the read size is often larger than
>> the read data that the device layer will supply. This lead to
>> higher-level layers blocking in read(). I figured (perhaps
>> incorrectly) that I could deal with this using non-blocking
>> socket reads.
>
> I can't really answer your specific questions about the Boost
> implementations

No problem, I appreciate you taking time to respond.

> but in general sockets (both blocking and non-blocking)
> and code dealing with them are expecting that read() will only block (or
> return WOULD_BLOCK) if no data can be read -- if at least one byte of
> data is available then that is what it will return, regardless of the
> amount actually requested.  (The read size acts only as a maximum.)

Yep, understood.

In theory the filtering_stream 'filter' and 'device' layers should
pass characters, EOF, and WOULD_BLOCK. Unfortunately the code
snippet I posted suppresses the propagation of the WOULD_BLOCK return
value from the 'device' layer to the 'filter' layer.

> Serial ports in particular sometimes operate this way and sometimes
> don't.  Under Windows, the SetCommTimeouts API function selects (via
> ReadIntervalTimeout and ReadTotalTimeoutConstant) whether normal serial
> ports will return "early" as above or whether it will wait longer to see
> if more data is received, and whether there's an overall timeout or if
> it will block forever.  There may be a similar API call you need to make
> to the FTDI library.

Ok, this is good to know, thanks.

I consider solving this particular "problem" a good way to
learn Boost. Without a "problem" to solve, reading code gets
old quickly :)

The Boost chat client/server has similar features to my problem,
i.e., it involves encoding/decoding messages. The encoding/decoding
is different than my case, since the chat protocol adds a
header with the message length. This simplifies the socket read
code, since you can read the fixed-length header, then read the
now known) message length, i.e., each call to read has a fixed
length parameter. In my case, the message length is unknown, and
depends on the content of the message (since data can be escaped).
The data stream is generated by a field-programmable gate array
(FPGA), and adding a buffer to determine the message length before
sending the response would use too many resources, so the encoding
protocol cannot easily be changed.

I'm in the process of modifying the chat example to encode
the messages with a start-of-packet [, end-of-packet ], and
escape \ code (so that a [ or ] or \ in the message is encoded
as \[ or \] or \\), i.e.,

   Hello!    ->(encodes to)->  [Hello!]
   [Hello!]  ->(encodes to)->  [\[Hello!\]]

The modified code will use async_read_until to parse the encoded
data streams ... I'll probably have to use a regex or match
procedure to deal with the fact that \] is not the end-of-packet.

Once I get the packet parsing working, I'll see if I can still use
the filtering_stream components I came up with for my application,
but instead of having them operate on a socket or serial_port device,
I'll just have them operate directly on the contents of the
streambuf that I filled using async_read_until. Its possible that
the async_read_until match condition could use the filtering_stream,
i.e., successful decoding of a message results in a match.

Cheers,
Dave