Re: [boost] [beast] Chunking example

Subject: Re: [boost] [beast] Chunking example
From: Vinnie Falco (vinnie.falco_at_[hidden])
Date: 2017-07-06 20:14:05

On Thu, Jul 6, 2017 at 12:41 PM, Bjorn Reese via Boost
<boost_at_[hidden]> wrote:
> When the user passes a buffer to Beast in chunked-mode, then
> serialization can convert it directly into a chunk. That ought to be
> very easy to do.

Something like that could be made to work if the caller is using
`buffer_body`. That type of body represents caller supplied buffers,
there could be a guarantee that each caller provided buffer will be
output as a single chunk:

<https://github.com/vinniefalco/Beast/blob/89c416cde64de3265299ced216a6eef9428c2a51/include/beast/http/buffer_body.hpp#L22>

>> you would convince Beast to keep a message in "perpetually sending"
>> mode since that is a use-case not anticipated by rfc7230.
>
> Are you implying that Beast has an upper limit for persistent
> connections? If not, what are you trying to tell me?

There's no upper limit, but I have not given consideration to a mode
where a BodyReader never runs out of buffers. Perhaps it already works
without change. But there are no tests for it and I have not tried it,
so I cannot make a claim that it works.

> ...and likewise pass a chunk to the application as it is parsed.
> This preserves the chunk boundaries without any special facilities.

Beast does not buffer the entire chunk in memory so the application
would only see pieces of it. The callback-based solution I provided,
is the closest to what you're asking ("pass a chunk to the
application").

>> I think if users demand control over outgoing chunks, its reasonable
>> to ask that they manually output the chunked message body; Beast will
>> take care of the header.
> ...
> Does this mean that the user has to manually insert chunk-size etc?

In the worst case, yes, but if there is a need for an easier way for
users to manually produce chunked output, a simple adapter can be
provided. Beast had one which I removed after I made significant
improvements to the serialization interfaces, it can be brought back.

This function takes a caller provided ConstBufferSequence and lazily
transforms it into a new ConstBufferSequence (non-allocating) which
performs a chunk-encoding:
<https://github.com/vinniefalco/Beast/blob/1e3543f63ed46bb1b2e698939f3cf0b1324dc180/include/beast/http/detail/chunk_encode.hpp#L88>

Thanks