Re: [boost] [beast] Request for Discussion

Subject: Re: [boost] [beast] Request for Discussion
From: Vinnie Falco (vinnie.falco_at_[hidden])
Date: 2016-09-29 09:34:47

On Thu, Sep 29, 2016 at 5:23 AM, Christof Donat <cd_at_[hidden]> wrote:
> I think frameworks for HTTP Servers should be separated in various parts:
>
> 1. Low Level HTTP stuff - that is what beast is perfect for
> 2. A Request Router
> 3. A Database Access Layer - most Web applications are good off to use a
> database
> 4. A way to create the HTML output without mixing it with the code

I'm proposing that there is a server layer that is even lower than the
items listed above. Specifically a generic server with these features:

1. An object modeling the entire server
2. An object to model an acceptor (listening port)
3. Server methods to add and remove acceptors dynamically
4. For acceptors supporting TLS, control over certificates
5. Acceptor supports plain, TLS, or auto-detected plain/TLS
6. An object to model a session (active connection)
7. Customization of the server with caller-provided behavior and state
8. Customization of the acceptor with caller-provided behavior and state
9. Customization of the session with caller-provided behavior and state
10. Control over the associated io_service and its threads

Explanation of each of these items:

1. A type that represents the server, its acceptors, its active
sessions, and caller provided state. This allows the caller to declare
one or more variables of server type.

2. Each listening port is represented by an object with its own
settings such as the address to bind to, port number, and whether it
supports plain, TLS, or auto-detect. By auto-detect, I mean that the
acceptor can detect whether the remote is attempting to perform a TLS
handshake or not, and allow either type of connection on the active
socket.

3. Allow callers to modify the listening ports on the fly. For
example, to stop allowing new connections, remove the port, or add a
new port.

4. For a port that supports TLS, an interface giving control of
certificates. It should also be possible to require client
certificates thus permitting development of a middleware server.

6. The active connection is represented by an object in memory with
suitable member functions.

7. The caller can associate the server with state. One example, is the
root directory from which to serve files. This could be done using
CRTP, or a caller supplied template argument. Acceptor and session
objects need access to this state so they can make decisions. For
example, the server could have an IP whitelist that each acceptor can
look at.

8. The caller can associate the acceptor with state. For example, the
domain specific permissions to give sessions associated with that
acceptor.

9. The caller can associate the session with state, access the state
of the associated acceptor from which the connection was established,
and access the state of the owning server.

10. Handling of the io_service should not constrain users' choice of
design patterns.

> For 3 and 4 I'd propose Roland Bocks sqlpp11
> (https://github.com/rbock/sqlpp11) and kiss-templates
> (https://github.com/rbock/kiss-templates)
>
> So some weeks ago I started a request router as hobby project based on beast
> v1. It does not even remotely work yet and of course the interface is not at
> all fixed now. As usuall hobby projects might get a lot of love, but never
> enough time, so the advance is very slow at the moment.

Well, I think that is great!

> The handlers might look like this:
>
> namespace app {
>     class foo: public base_http_handler<foo> {
> ...

What you have done here, and what you should be congratulated for, is
to become the first stakeholder to provide an example of code (I
think). Code samples are a universally understood way to get your
point across regarding design decisions.

The interface to the router is a bit high level for me. I would be
interested to see your ideas about how your class base_http_handler is
instantiated. Is it created when the socket is created? How does the
handler gain access to session specific information, such as the
remote IP address? Or caller provided state such as the filesystem
root or permissions? Did you start with Beast's http_async_server and
modify it? Or do you have your own implementation that manages
connections?

> template<typename Request>
> beast::http::response_v1<beast::http::string_body>
> operator ()(const std::smatch& match, const Request& req);
> ...
> What do you think about such an interface?

I love that you have provided a function signature for everyone to
consider, I wish more people did this.

With respect to your signature, I think there a problem with using the
return type in that fashion. What if different routers want to use
different types for the HTTP body? A key feature of Beast is that
callers can get control over the algorithms used to parse and
serialize the body. For a server, this ability is critical - its the
intention that streaming from a coroutine, progressive
download/upload, handling of the Content-Encoding, are all done using
user-defined body customizations. C++ only allows for one return type
so this would complicate things.

A robust server will almost never use beast::http::string_body or
beast::http::streambuf_body except for error messages like 404 Not
Found. More likely, someone will develop a rich Body type that
supports a wide variety of features such as the encoding as I
mentioned above, multi-part/mime extensions, and the like.

Another approach, instead of using the return type, is to give the
handler an object which when invoked with a message, queues or sends
the message. This allows the caller to use any choices of body type
(or headers type) depending on what the router selects. A modified
version of your handler might look like this:

struct http_handler
{
  /** Provides the HTTP response given a request.

      SendFunction will have this equivalent signature:

      @code

        template<class Body, Class Headers>
        f(response_v1<Body, Headers>&& res)

      @endcode

      If res is movable, the send function will acquire ownership. Otherwise, if
      res is copyable then a copy will be made.

      SendFunction Requirements:

        res must be MoveConstructible or CopyConstructible.
  */
  template<class SendFunction, class Body, class Headers>
  void
  operator()(request_v1<Body, Headers> const& req, SendFunction&& f);
};

Thank you for sharing your ideas, I think this is going to really help
the community make forward progress on HTTP.

Regards