IIUC, you're trying to mix and match calls to
async_read_until (with a streambuf) and calls to async_read (with a buffer
parameter), but having problems with "extra data past the length data". If you
keep the same design, you will need to first extract the "extra data" out of the
streambuf, place it in to the buffer you want to use, then call async_read or
async_read_at with the partially filled buffer. While I haven't done anything
similar, I can't think of a reason this won't work.
However, I think it would be easier to use
async_read_until calls only, and keep all data in the streambuf. Create a
match_condition function object, track the state of the HTTP transfer in the
function object, and then return a true condition when the full buffer has been
received. This encapsulates all of your "buffer filling logic" into one small
place and I think is the simplest code of everything I'm writing in this
e-mail.
If you cannot use a streambuf for the full transfer
for whatever reason, then I would not use async_read_until and instead use
async_read or async_read_at with a single buffer for everything. It
shouldn't be hard to manage the "two states" of the transfer
("need length header", "have length header, getting rest of
data"). Your handler will be called multiple times (most likely) while in
the second state (and it's possible it will get called multiple times while
in the first state, depending on how congested
the TCP incoming buffers are). I think your design of trying
to mix the two models is more complicated than it needs to be.
Cliff