Re: [boost] [beast] Supporting ICY 200 OK

Subject: Re: [boost] [beast] Supporting ICY 200 OK
From: Vinícius dos Santos Oliveira (vini.ipsmaker_at_[hidden])
Date: 2017-10-04 16:37:05

2017-10-04 10:37 GMT-03:00 Vinnie Falco via Boost <boost_at_[hidden]>:

> On Tue, Oct 3, 2017 at 6:06 PM, Vinícius dos Santos Oliveira
> <vini.ipsmaker_at_[hidden]> wrote:
> > You could also try to standardize HTTP/1.1 and HTTP/2.0 under the same
> > interface.
>
> That is incorrect. All I'm talking about is a simple wrapper which
> meets the requirements of SyncReadStream and AsyncReadStream which
> checks the first incoming 3 bytes of data on the stream against the
> string "ICY" and translates it into "HTTP/1.1" to allow the existing
> beast parser to recognize a *slightly* non-standard response format.
> Note the emphasis on the term *slightly*. I have designed a reasonable
> solution to enable this niche use-case to work without incurring
> significant technical debt for something that is clearly outside the
> scope of rfc7230.
>
> And you are suggesting that the same technique should be used to make
> HTTP/2 appear like HTTP/1.1? Or am I not understanding your statement?
>

There's a big misunderstanding between us here.

>From the code you showed in the opening message of this thread:

    asio::ip::tcp::socket sock{ios};
    icy_stream<asio::ip::tcp::socket&> stream{sock};
    beast::http::response<beast::http::empty_body> res;
    beast::multi_buffer buffer;
    beast::http::read(stream, buffer, res);

I thought you were trying to generalize `beast::http::read`. Now it's clear
that you're focusing on the parser. It was a misunderstanding of your
intent on my behalf.

> The version attribute in the message model[1] might not make sense to all
> > HTTP backends.
>
> I'm not sure what an "HTTP backend" means, but for practical purposes
> there are three versions of HTTP:
>
>     HTTP/1.0
>     HTTP/1.1
>     HTTP/2
>

Add ZeroMQ to the list. The point is making applications that answer HTTP
requests (if you're writing a server). For some, this meant writing plugins
for existing HTTP servers. From my point of view, there is no need restrict
to this set of HTTP versions. You should abstract the meaning/behaviour
beyond that. Just like I wrote above:

This discussion reminds me of the difference between inductive reasoning
> and deductive reasoning. Only the immediate problem is pursued and no
> attention is given trying to solve the general problem at hand.
>

The complaint remains valid. A lack of abstract thinking.

While the content of the messages is the same (start line, zero or
> more fields, optional body) the semantics of the status-line and the
> fields differ between versions. For example, the absence of the
> Connection field in HTTP/1.1 implies keep-alive, while the absence of
> the Connection field in HTTP/1.0 implies closure. While in HTTP/2 the
> Connection field is illegal and may not appear at all.
>

I'm aware of connection differences.

The beast message container was designed from the ground up to work
> with the three versions of HTTP that we have today. In order to
> capture all of the information available in a message, the container
> must reflect the HTTP version that is implicitly or explicitly stated
> in the message. In order to empower authors to write libraries that
> work with any message (a goal of Beast), the version field must be
> present.
>

The “In order to [...] in a message, the container must reflect the HTTP
version” sentence is very interesting. It means I need to expose exactly
the same message received from the wire to the user... like you need
different API to support HTTP/2.0. Is my understanding of your sentence
correct?

Because the container doesn't need to reflect the HTTP version. In my view,
there are capabilities, like `native_stream`. Even the HTTP/2.0
multiplexing behaviour can be supported under the same simple API with no
additional complexities in the message container.

> This discussion reminds me of the difference between inductive reasoning
> and
> > deductive reasoning. Only the immediate problem is pursued and no
> attention
> > is given trying to solve the general problem at hand.
>
> That is incorrect. The beast message container is designed to be
> HTTP/2-ready.
>

“that is incorrect” + “message container is designed to be HTTP/2-ready”
seems to me that you didn't understand my complaint.

> Also, if beast::http::read is what I'll call, then this implies I need to
> > throw my abstractions in `beast::http` namespace
>
> That is incorrect. Beast HTTP stream operations use abstractions for
> the following objects:
>
>     stateful stream data uses the DynamicBuffer concept
>      <http://www.boost.org/doc/libs/master/libs/beast/doc/
> html/beast/concepts/DynamicBuffer.html>
>
>     message payload uses the Body concept
>     <http://www.boost.org/doc/libs/master/libs/beast/doc/
> html/beast/concepts/Body.html>
>
>     message headers use the Fields concept
>     <http://www.boost.org/doc/libs/master/libs/beast/doc/
> html/beast/concepts/Fields.html>
>
>     the stream uses the SyncReadStream, SyncWriteStream,
>     AsyncReadStream, and AsyncWriteStream concepts from Asio:
>     <http://www.boost.org/doc/libs/1_65_1/doc/html/boost_
> asio/reference/SyncReadStream.html>
>     <http://www.boost.org/doc/libs/1_65_1/doc/html/boost_asio/reference/
> SyncWriteStream.html>
>     <http://www.boost.org/doc/libs/1_65_1/doc/html/boost_asio/reference/
> AsyncReadStream.html>
>     <http://www.boost.org/doc/libs/1_65_1/doc/html/boost_asio/reference/
> AsyncWriteStream.html>
>
> Beast provides several general implementations for most of these
> concepts, in the beast namespace, but objects may be passed from types
> in any namespace. For example, beast stream operations work with
> streams of type `boost::asio::ip::tcp::socket`, which is clearly not
> in the `beast::http` namespace. This obvious counterexample disproves
> your claim.
>

Thanks for the clarification.

> However, a problem is the fact that this translation in the stream needs
> to
> > happen to actually use the Beast message framework.
>
> That is incorrect. The translation for converting the three character
> sequence "ICY" at the beginning of a read stream, into "HTTP/1.1" is
> not necessary to use the beast message container. Rather, it is
> necessary in order to use the HTTP/1 parsers which come with beast.
> The parser implementation strictly adheres to rfc7230. As the text
> "ICY 200 OK\r\n" is not a valid status-line according to rfc7230, I'm
> sure that you can understand why a strict parser would reject it.
>

I got the wrong impression. I thought your first email was more about the
message model than the parser.

Talking about parsers, what is your opinion on the design of parsers like
showed in this talk: https://vimeo.com/channels/ndcoslo2016/171704565

My apologies but I am unfamiliar with these new terms "virtual HTTP
> stream" and "virtual field" that you have introduced. Could you
> perhaps define them so that I can discuss them on equal footing?
>

Sure thing.

>From fortune (don't worry, it is **NOT** a joke):

(definitions)
%
transparent, adj.:
        Being or pertaining to an existing, nontangible object.
        "It's there, but you can't see it"
                -- IBM System/360 announcement, 1964.

virtual, adj.:
        Being or pertaining to a tangible, nonexistent object.
        "I can see it, but it's not there."
                -- Lady Macbeth.

Virtual field is not to be confused with HTTP field. There is the objective
reality and the virtual reality, a model you create from it, in your brain.
There is the _field_ — not to be confused with the word/semantic “field”
from HTTP messages — of real things and virtual things.

A good example to understand this separation, closer to the examples you're
used to is the `-o loop` option from the `mount` UNIX command. Your
application thinks it's a real file, but it's not, it's “emulated”/virtual.

Getting back to the thread, “ICY 200 OK\r\n” is real, “HTTP/1.1 200 OK\r\n”
is fake/virtual. “HTTP/1.1 200 OK\r\n” is what the application thinks it
has happened.

> Why does it need to go to memory?
>
> My apologies but I do not understand this question. What is the "it"
> that you think "needs" to "go to memory?"
>
> > Why isn't just limited to the API?
>
> My apologies but I do not understand this question at all. Perhaps you
> could provide a small code example which illustrates your question or
> counter-example?
>

To be limited to the virtual field, it'd mean that no
memory/RAM/array<char> translation from “ICY” to “HTTP” would happen. The
“HTTP interaction” of what happened would be limited to the function
signatures/contracts. No “HTTP” char-array in memory would need to exist
(i.e. it'd be virtual, not real).

However, like I stated in the beginning of the email, I thought you were
talking about the Beast message model. However, you're talking about the
parser. In this case (reusing the parser), I'll limit myself to Bjørn
words: “Sounds like a good solution.”

But I'm still curious about what you think about this talk:
https://vimeo.com/channels/ndcoslo2016/171704565

> Boost.Http managed to expose this amount of feature without exposing a
> > parser at all.
>
> Does Boost.Http expose a feature which allows it to interpret "ICY 200
> OK\r\n" as a valid HTTP status-line according to rfc7230?
>

Once again: I thought you were talking about Beast message model, not the
parser.

I think the solution for Boost.Http message model would be the same within
Boost.Beast message model. And the message model is not the solution
mentioned in the beginning of this thread.

As per the parser, I'm writing lots of documentations, so better translate
whatever answer I'd write here in the documentation of the parser.

-- 
Vinícius dos Santos Oliveira
https://vinipsmaker.github.io/