Re: [boost] [beast] Request for Discussion

Subject: Re: [boost] [beast] Request for Discussion
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2016-09-24 11:22:53

On 23 Sep 2016 at 13:22, Vinnie Falco wrote:

> ]> Almost everyone here and in the wider C++ community wants somebody to
> > take the HTTP client API from the Python Requests module
> > (http://docs.python-requests.org/en/master/) and very closely
> > replicate it in C++
> [snip]
> Unfortunately, even a reduced subset of these features goes far beyond
> my resources to implement. I also do not feel confident in my
> knowledge of the domain or my abilities to provide a robust C++
> interface for all of this.

As I mentioned to you yesterday, wrapping the Python implementation 
in Boost.Python and exporting it as C++ solves the cost of 
implementation problem. I'd even consider a more exciting method of 
transducing implementation say via static conversion of Python to C 
or C++ via adapting one of the Python compilers.

Most Boost library authors want to build cathedrals their own way, 
and that's fine, it gets you up in the morning to write code you're 
not getting paid for. However from an end user's perspective, they 
*really do not care* how a library is implemented so long as it 
works, has reasonable documentation and has reasonable performance. 
Naval gazing or "purity" by library authors is fairly unimportant to 
almost all of them. Given the very high popularity of Python 
Requests, I do not doubt a very similar C++ library would be 
immensely popular in the C++ world, even if its implementation is 
actually the Python code itself.

> I think that what Niall has done, probably inadvertently, is to
> demonstrate just how broken the Boost review process has become. We
> have Beast, which provides a great implementation of the low level
> HTTP operations (read and write a message, provide a message model). I
> am sure that someone or some group with expert knowledge in creating
> robust HTTP clients could come along and build the C++ equivalent of
> Python Requests on top of Beast. It should not be controversial when I
> say that Beast offers useful functionality today.

All libraries, even terrible ones, offer useful functionality by 
definition.

I also don't think it controversial that a HTTP utilities library 
would be a valuable addition to Boost. What I did say to you, and 
others strongly agreed with me at CppCon if you remember, is that 
it's very hard to get something like a HTTP low level library 
usefully past a Boost peer review. There are umpteen ways of 
implementing it, everyone has their own opinion, and a lot of people 
otherwise intelligent will argue and comment from a profoundly 
ignorant base of understanding. It'll be hard to decide between the 
really valuable feedback from those who really understand this niche 
domain from the rest who really don't know what they're talking 
about.

I don't think this means the Boost peer review is broken, just that 
it fits badly to really niche libraries, and a well designed low 
level HTTP client library is a very niche domain very few will ever 
use. We've already seen some reviews having only three or four 
reviewers because the topic is such a niche domain that most of us 
here can't usefully comment. For the heavy maths libraries, most here 
realise their opinions are close to worthless and they stay away. For 
low level multi purpose libraries like HTTP ones, that tends to not 
happen because people think themselves competent which makes the 
review process not as valuable nor work as well as one would wish 
for.

> The modern consensus is that C++ libraries need to become more focused
> and smaller, performing a single task and doing it really well. And
> that is exactly the design principle of Beast - model the HTTP
> message, serialize and deserialize HTTP messages synchronously or
> asynchronously. This might not satisfy the majority of use cases but
> it gives interested parties something they can work with. Who are the
> interested parties? Anyone who wants to write a generic web server. Or
> a full featured HTTP client. You want those features right, and you
> want them in Boost? Then why would we reject a library that offers the
> primitives for other people to create such high level implementations?

As I mentioned above, I've no problem with people building their own 
cathedrals. It's what most of us are here to do.

However you have to be realistic. If it seems to take as long for an 
end user to learn off your low level HTTP framework as to quickly 
hack something bespoke together, they are going to choose the latter. 
That outcome is *extremely* likely for any low level HTTP framework. 
End users are simply not going to invest the effort to master the 
framework you've invested years of your life into unless they are 
highly and immediately convinced it will save them a lot of time and 
effort.

A high level HTTP client library has a very visible and clear cost 
benefit ratio. Any semi decent implementation is going to see rapid 
and very widespread adoption because most people just want to go 
fetch the bloody HTTP page already.

A low level HTTP client, or server library is always going to have to 
fight the not-invented-here syndrome and the hard reality that the 
majority of end users will roll their own half baked solution rather 
than take the time to master your inevitably complex and brain 
hurting framework.

Again, that's fine. Plenty of Boost libraries have been usefully 
inspirational rather than actually used. The biggest contribution 
Boost made to the C++ community past the standardised facilities was 
a source code implementation study reference manual. Your cathedral 
won't necessarily be used much, but a lot of people will derive 
benefit from it once it's into Boost by studying it when writing 
their own code.

In the end you'll need to decide, as all of us have done, whether you 
want to write a library end users really want to use and will adopt 
in droves and will really solve a big pressing problem for the C++ 
community, or whether you want to write a library which is a 
beautiful and pure cathedral stunning and inspirational to all who 
behold it, but rarely enter nor use it. That's absolutely a personal 
decision. I'll support you whichever you choose to the best of my 
capabilities.

Niall

-- 
ned Productions Limited Consulting
http://www.nedproductions.biz/ 
http://ie.linkedin.com/in/nialldouglas/