Re: [boost] [beast] Request for Discussion

Subject: Re: [boost] [beast] Request for Discussion
From: Edward Diener (eldiener_at_[hidden])
Date: 2016-09-24 12:21:18

On 9/24/2016 11:22 AM, Niall Douglas wrote:
> On 23 Sep 2016 at 13:22, Vinnie Falco wrote:
>
>> ]> Almost everyone here and in the wider C++ community wants somebody to
>>> take the HTTP client API from the Python Requests module
>>> (http://docs.python-requests.org/en/master/) and very closely
>>> replicate it in C++
>> [snip]
>> Unfortunately, even a reduced subset of these features goes far beyond
>> my resources to implement. I also do not feel confident in my
>> knowledge of the domain or my abilities to provide a robust C++
>> interface for all of this.
>
> As I mentioned to you yesterday, wrapping the Python implementation
> in Boost.Python and exporting it as C++ solves the cost of
> implementation problem. I'd even consider a more exciting method of
> transducing implementation say via static conversion of Python to C
> or C++ via adapting one of the Python compilers.
>
> Most Boost library authors want to build cathedrals their own way,
> and that's fine, it gets you up in the morning to write code you're
> not getting paid for. However from an end user's perspective, they
> *really do not care* how a library is implemented so long as it
> works, has reasonable documentation and has reasonable performance.

I agree with this but there is nothing which prevents a low-level 
library from fulfilling another programmer's needs. In other words it is 
just as important that a low-level library provides a good design for 
others to build upon as it is for a higher-level library to provide a 
good design for end-user's to use in their applications.

> Naval gazing or "purity" by library authors is fairly unimportant to
> almost all of them. Given the very high popularity of Python
> Requests, I do not doubt a very similar C++ library would be
> immensely popular in the C++ world, even if its implementation is
> actually the Python code itself.
>
>> I think that what Niall has done, probably inadvertently, is to
>> demonstrate just how broken the Boost review process has become. We
>> have Beast, which provides a great implementation of the low level
>> HTTP operations (read and write a message, provide a message model). I
>> am sure that someone or some group with expert knowledge in creating
>> robust HTTP clients could come along and build the C++ equivalent of
>> Python Requests on top of Beast. It should not be controversial when I
>> say that Beast offers useful functionality today.
>
> All libraries, even terrible ones, offer useful functionality by
> definition.
>
> I also don't think it controversial that a HTTP utilities library
> would be a valuable addition to Boost. What I did say to you, and
> others strongly agreed with me at CppCon if you remember, is that
> it's very hard to get something like a HTTP low level library
> usefully past a Boost peer review. There are umpteen ways of
> implementing it, everyone has their own opinion, and a lot of people
> otherwise intelligent will argue and comment from a profoundly
> ignorant base of understanding. It'll be hard to decide between the
> really valuable feedback from those who really understand this niche
> domain from the rest who really don't know what they're talking
> about.
>
> I don't think this means the Boost peer review is broken, just that
> it fits badly to really niche libraries, and a well designed low
> level HTTP client library is a very niche domain very few will ever
> use. We've already seen some reviews having only three or four
> reviewers because the topic is such a niche domain that most of us
> here can't usefully comment. For the heavy maths libraries, most here
> realise their opinions are close to worthless and they stay away. For
> low level multi purpose libraries like HTTP ones, that tends to not
> happen because people think themselves competent which makes the
> review process not as valuable nor work as well as one would wish
> for.
>
>> The modern consensus is that C++ libraries need to become more focused
>> and smaller, performing a single task and doing it really well. And
>> that is exactly the design principle of Beast - model the HTTP
>> message, serialize and deserialize HTTP messages synchronously or
>> asynchronously. This might not satisfy the majority of use cases but
>> it gives interested parties something they can work with. Who are the
>> interested parties? Anyone who wants to write a generic web server. Or
>> a full featured HTTP client. You want those features right, and you
>> want them in Boost? Then why would we reject a library that offers the
>> primitives for other people to create such high level implementations?
>
> As I mentioned above, I've no problem with people building their own
> cathedrals. It's what most of us are here to do.
>
> However you have to be realistic. If it seems to take as long for an
> end user to learn off your low level HTTP framework as to quickly
> hack something bespoke together, they are going to choose the latter.
> That outcome is *extremely* likely for any low level HTTP framework.
> End users are simply not going to invest the effort to master the
> framework you've invested years of your life into unless they are
> highly and immediately convinced it will save them a lot of time and
> effort.
>
> A high level HTTP client library has a very visible and clear cost
> benefit ratio. Any semi decent implementation is going to see rapid
> and very widespread adoption because most people just want to go
> fetch the bloody HTTP page already.
>
> A low level HTTP client, or server library is always going to have to
> fight the not-invented-here syndrome and the hard reality that the
> majority of end users will roll their own half baked solution rather
> than take the time to master your inevitably complex and brain
> hurting framework.
>
> Again, that's fine. Plenty of Boost libraries have been usefully
> inspirational rather than actually used. The biggest contribution
> Boost made to the C++ community past the standardised facilities was
> a source code implementation study reference manual. Your cathedral
> won't necessarily be used much, but a lot of people will derive
> benefit from it once it's into Boost by studying it when writing
> their own code.

I think you are forgetting that some of those "cathedrals" have also 
been immensely used by other Boost developers. MPL is the most obvious 
candidate for that, but many developers use other low-level Boost 
libraries in their own library.

>
> In the end you'll need to decide, as all of us have done, whether you
> want to write a library end users really want to use and will adopt
> in droves and will really solve a big pressing problem for the C++
> community, or whether you want to write a library which is a
> beautiful and pure cathedral stunning and inspirational to all who
> behold it, but rarely enter nor use it. That's absolutely a personal
> decision. I'll support you whichever you choose to the best of my
> capabilities.

Your cathdral metaphor is one-sided. Mere beauty in a low-level library 
may be enough for that library to interest others but if the low-level 
library is not seen as practically useful as something other developers 
can build upon it will probably not become part of Boost.