Re: [boost] [beast] Request for Discussion

Subject: Re: [boost] [beast] Request for Discussion
From: Christof Donat (cd_at_[hidden])
Date: 2016-09-30 10:34:33

Hi,

Am 29.09.2016 15:34, schrieb Vinnie Falco:
> On Thu, Sep 29, 2016 at 5:23 AM, Christof Donat <cd_at_[hidden]> wrote:
>> I think frameworks for HTTP Servers should be separated in various 
>> parts:
>> 
>> 1. Low Level HTTP stuff - that is what beast is perfect for
>> 2. A Request Router
>> 3. A Database Access Layer - most Web applications are good off to use 
>> a
>> database
>> 4. A way to create the HTML output without mixing it with the code
> 
> I'm proposing that there is a server layer that is even lower than the
> items listed above. Specifically a generic server with these features:
> [...]

So that would be between the low level HTTP stuff and the request 
router. Looks like a good idea and probably will make the request router 
simpler.

> 6. The active connection is represented by an object in memory with
> suitable member functions.

We could try define a kind of concept for the session store and the 
session objects, so that the session could also be persistent on disk, 
or in e.g. memcached. The session handling code then should be a 
template that is instantiated with these types. You'll probably like to 
be able to define the session objects class anyway.

One important thing, I think is, that we should also be able to have no 
session tracking at all without having to pay for that, e.g. we don't 
want a session cookie then. So maybe we define a session object class, 
that triggers the corresponding meta program to cut out all the session 
management code all together.

> 7. The caller can associate the server with state. One example, is the
> root directory from which to serve files.

When you look back at my examples, my approach to serve static files was 
to use a special handler: 
static_file_handler{boost::filesystem::current_path(), "$1"s} That will 
instantiate a handler that works on the directory, the application has 
been started in (why-ever you might want that). It will take the first 
sub-expression of the match as path to find the file to serve in that 
directory.

I don't think, that serving static files should be implemented on a 
lower level. You'd have to twiddle around with the routing system on a 
lower level then in order to know, which URLs are served by the 
application and which should be served by files.

>> The handlers might look like this:
>> 
>> namespace app {
>>     class foo: public base_http_handler<foo> {
>> ...
> 
> What you have done here, and what you should be congratulated for, is
> to become the first stakeholder to provide an example of code (I
> think). Code samples are a universally understood way to get your
> point across regarding design decisions.

:-) that was the idea.

> The interface to the router is a bit high level for me. I would be
> interested to see your ideas about how your class base_http_handler is
> instantiated.

Base_http_handler is a service class for other http handlers. Handlers 
don't have to be derived from base_http_handler, it just provides some 
stuff, that many handlers will probably need.

> Is it created when the socket is created?

The handlers are only instantiated, when the route is defined. So in a 
typical application, there will be only one instance of each handler. 
The constructor parameters are global to all requests to that handler. 
E.g. url_match_cast<int>{"$1"s} will simply provide a type, that can be 
used to extract the first parameter of any regular expression match as 
int. The default implementation uses boost::lexical_cast<>, but of 
course you can create specializations for your own types.

When the request is processed, I only call the call operator of that 
handler. That is, why it can also be a lambda.

> How does the
> handler gain access to session specific information, such as the
> remote IP address?

The current idea is, to get that from the request object in future. 
There might be better options, of course.

> Or caller provided state such as the filesystem
> root or permissions?

Those should be passed in as constructor parameters, like I did with the 
base path for the static_file_handler.

> Did you start with Beast's http_async_server and
> modify it? Or do you have your own implementation that manages
> connections?

I tried using the code in http_async_server, but at the moment I'd be 
happy to get the unit tests running, before I even hope to soon have a 
working example server.

> With respect to your signature, I think there a problem with using the
> return type in that fashion. What if different routers want to use
> different types for the HTTP body? A key feature of Beast is that
> callers can get control over the algorithms used to parse and
> serialize the body. For a server, this ability is critical - its the
> intention that streaming from a coroutine, progressive
> download/upload, handling of the Content-Encoding, are all done using
> user-defined body customizations. C++ only allows for one return type
> so this would complicate things.
> 
> A robust server will almost never use beast::http::string_body or
> beast::http::streambuf_body except for error messages like 404 Not
> Found. More likely, someone will develop a rich Body type that
> supports a wide variety of features such as the encoding as I
> mentioned above, multi-part/mime extensions, and the like.

Actually I obviously misunderstood that part of beast then. I guess, 
that is the reason, why I have searched for a reasonable body object and 
then resorted to beast::http::string_body. I thought, having different 
body types would be for some rare cornercases, that we usually will not 
have in a high level framework.

So am I correct to assume, that the idea is to have a more or less 
arbitrary body object. In the end that will have to serialized to a 
string of bytes. Did you intend to have that implemented in the body 
object? If so, that would be a great place to plug in e.g. kiss 
templates. My current idea was to call the template inside the handler 
and write the result to the body.

I think, the router can be made to handle different return types, as 
long as they implement a yet to define concept. I also have some 
thoughts to make it more flexible on the parameters. For some handlers 
you'll be happy to just get the URL match, others will maybe need a 
stream for websocket interaction. I think, that can be achieved using 
boost call traits, std::result_of, std::is_callable, or similar things, 
and a little bit of not too difficult meta programming magic. I hope to 
get some time this weekend to look into that.

> Another approach, instead of using the return type, is to give the
> handler an object which when invoked with a message, queues or sends
> the message.

I also thought about that. For the time being, I chose to return the 
response object for semantic reasons. For me that is the result of the 
operation. With the approach of passing in a kind of response object, 
the operation would, as a side effect, respond to the request, and 
return something else, like an error code, or even void.

> Thank you for sharing your ideas, I think this is going to really help
> the community make forward progress on HTTP.

I hope so :-)

Christof