July 2017 - Boost - lists.boost.org

[beast] Platform proxy settings, SSL verification
by Phil Endecott 05 Jul '17

05 Jul '17

Here are just a few quick comments re Beast: 1. It's a terrible name, but I believe others have already mentioned that. 2. I don't believe there is any integration with platform proxy settings. I'll explain: I have my own basic HTTP client and server code that I've used in a handful of projects. There are a few features that it lacks that have caused me problems. In particular, when I tried to use my HTTP client code in an iOS app, it worked on WiFi networks but not on some cellular networks. After much remote debugging I realised that some cellular carriers in some countries require that HTTP traffic goes via a (non-transparent) proxy. If you use the iOS http client then this is handled for you, but my code knows nothing about it. For me to test a fix for this I would have had to fly to another continent.... This is the sort of thing that is particularly valuable in a Boost library: I try to write portable code, but I don't know the details of every platform that it will run on or even have test systems available to me. Having a library that knows about the platform-specific details, and which has been tested on a wide variety of those platforms, is very valuable. (And a library that instead just does the "easy parts" is something that I could have written, or already have written, myself.) 3. Finally a comment about the SSL example at https://github.com/vinniefalco/Beast/blob/master/example/http-client-ssl/ht… This says stream.set_verify_mode(ssl::verify_none); Please correct me if I'm wrong, but I think that means that it does not do any sort of certificate verification. As a matter of principle, I think that the *first* SSL example should show how to establish a properly-secure connection. See for example http://www.cs.utexas.edu/~shmat/shmat_ccs12.pdf (""The Most Dangerous Code in the World: Validating SSL Certificates in Non-Browser Software") and e.g. https://lwn.net/Articles/522111/ for discussion. A quick look at http://www.boost.org/doc/libs/1_47_0/doc/html/boost_asio/overview/ssl.html suggests that this might add as few as two more lines to the code. Regards, Phil.

2 4

[release] Release notes for 1.65.0 beta
by Daniel James 05 Jul '17

05 Jul '17

Hi, If you haven't done so yet, please write your release notes for 1.65.0 beta. You can create a pull request at: https://github.com/boostorg/website/blob/master/feed/history/boost_1_65_0.q… Or if you'd rather you can send them to me, and I'll add them for you. thanks, Daniel

1 0

[build] header-only lib need dir 'build'?
by Oliver Kowalke 05 Jul '17

05 Jul '17

Hi, does a header-only library require sub-directory 'build' (especially if the lib depends on another, non-header-only library)? In other words; is it safe to remove sub-directory 'build' from boost.coroutine2? thx, Oliver

2 2

[beast] write_some EWOULDBLOCK/EAGAIN error recovery non-blocking operations
by Artyom Beilis 04 Jul '17

04 Jul '17

Have been Beast tested with non blocking sockets? I hadn't found a single reference to asio's non_blocking_io? One of the common use cases is when I want to push a small message to stream (chunk). In 99% of cases the operation will not block so I want to handle this as an optimization for not returning to even loop and creating a callback. According to the reference I didn't find how write_some will behave? I expect to get EWOULDBLOCK kind of error and start asynchronous operation. However I didn't find any reference to the code to non_blocking_io so I wonder if it is supported at all. On same note - how do I run unit tests using b2 - Beast ins't fully Boostified which BTW is also bothering me. Artyom

4 6

[xpressive] Merge PR for fixing MSVC 14.1
by Andrey Semashev 04 Jul '17

04 Jul '17

Hi, There is this PR from Daniela Engert to fix Boost.Xpressive compilation failure with MSVC 14.1: https://github.com/boostorg/xpressive/pull/4 It replaces the removed std::unary_function with the equivalent typedefs. The change looks rather harmless. Could this PR be merged before 1.65 is out? Thanks.

1 1

[iterator][range] BoostIteratorTraversalConcepts-aware boost::advance/distance
by Michel Morin 04 Jul '17

04 Jul '17

Hi, The Boost iterator traversal concepts have not been adopted by the C++ Standard. Because of this, some of RandomAccessTraversalIterators (Boost's concepts) are treated as InputIterators (Standard's concepts) by the stdlib. IMHO, Boost.Iterator should provide BoostIteratorTraversalConcepts-aware `boost::advance` and `boost::distance` to avoid the inefficiencies. However, neither of them are implemented. (Boost.Range has `boost::distance` for ranges, but it just calls `std::distance`.) I'm attaching files that implement `boost::advance` and `boost::distance`. Would these functions be useful additions? Regards, Michel

4 13

[beast] Review
by Peter Dimov 04 Jul '17

04 Jul '17

This is my review of the Beast library. I. DESIGN --------- The design of the library is very good and appears mature. The concepts are well specified, well documented, well thought out. The use of concept checks is a nice touch and a mark of professionalism. I think that basing the library on ASIO (resp. the Networking TS) is a fine decision. This does mean that if one finds an ASIO/NetTS concept less than ideal, one has to live with it, but that's an acceptable tradeoff. Severing the ties with ASIO based on the yet-hypothetical "structured blobs in, structured blobs out" model may sound good in theory, but in practice, a supermajority of the potential users of Beast need to produce a working HTTP or Websocket server, and the library as it stands addresses this need. The design is not perfect; at times the library makes it much too easy for asymptomatic mistakes to be introduced by the omission of a required member function call. For example, if one forgets to set the Content-Length, everything will still appear to work; or if one forgets to initialize the .result of the response, it remains uninitialized. (Note that these, and subsequent, observations appertain to the library as submitted for review; they may no longer be relevant since the author is very quick with the fixes.) There is, at times, unnecessary verbosity, even in the two storefront examples in http://vinniefalco.github.io/stage/beast/review/beast/quick_start.html that are supposed to show the library in the best possible light. For instance, // Set up an HTTP GET request message http::request<http::string_body> req; req.method(http::verb::get); req.target("/"); req.version = 11; Every request will need a method and a target, so having to call members to set them feels unnecessary. A constructor ought to have been provided. Similarly, in the next example, // WebSocket says that to close a connection you have // to keep reading messages until you receive a close frame. // Beast delivers the close frame as an error from read. // beast::drain_buffer drain; // Throws everything away efficiently for(;;) { // Keep reading messages... ws.read(drain, ec); // ...until we get the special error code if(ec == websocket::error::closed) break; // Some other error occurred, report it and exit. if(ec) return fail("close", ec); } the drain loop is basically a requirement for every correct program, so it should have been encapsulated into a helper function. (Even though the example as written does illustrate the use of a drain_buffer.) I'm not particularly fond of the design decision to declare parts that in practice every user of the library will need as "out of scope". People should not be made to reinvent these wheels. I understand the aim of producing a standard library proposal, and that some of these wheels would not be suitable for standardization, but there's nothing easier than just including the suitable wheels in the formal proposal and omitting the unsuitable ones. In practice, these necessary parts end up as examples, so one has to #include "../example/wheel.hpp" instead of "beast/wheel.hpp". It would be better, in my opinion, if one includes, say, "beast/ext/wheel.hpp", with it being known that ext/ is where "out of scope" things go. The ASIO coupling did leave me with one question, whether it would be possible for the library to accommodate OS-specific file to socket transfers such as the one provided by TransmitFile: https://msdn.microsoft.com/en-us/library/windows/desktop/ms740565(v=vs.85).… The idea here is that the OS kernel can dump the file to the socket without involving user mode, which has the potential of much increased performance. For the median user, this would not make much of a difference, but it's an interesting design question, and perhaps an argument that the example file_body class ought to be part of the library proper. ("Is in scope", to use the preferred terminology.) II. IMPLEMENTATION ------------------ I did not pay close attention to the implementation. A random sampling reveals that it's professionally written, and it appears to work, which is good enough for me. The tests run when `b2` is invoked in `test`, have good coverage, and pass with MSVC 14.0 and 14.1. Travis/Appveyor are put to good use. The library provides a doc/Jamfile, which is a plus, but this Jamfile fails, which is a minus. The file reference.qbk has its own separate build script, and this would need to be fixed one way or another when the library is integrated into the Boost documentation build. III. DOCUMENTATION ------------------ The documentation is good. The non-reference parts are very good, the examples are informative. The reference is very good at times, and not so much at others. This is caused by the fact that to get to the good parts one needs to drill down a level or two. For instance, looking at http://vinniefalco.github.io/stage/beast/review/beast/quickref.html I noticed the "async_teardown" function in the WebSocket section, so I clicked on it to see if I need to call that to tear down a WebSocket connection. This lead me to http://vinniefalco.github.io/stage/beast/review/beast/ref/beast__websocket_… which only says "Start tearing down a boost::asio::ssl::stream/connection/boost::asio::ip::tcp::socket." Googling told me that I'm not the only one left not entirely satisfied with these descriptions: https://github.com/vinniefalco/Beast/issues/274 The confusion here stems from the fact that would I, and the issue submitter, have clicked on one of the async_teardowns in that page, we would have reached f.ex. http://vinniefalco.github.io/stage/beast/review/beast/ref/beast__websocket_… which is indeed fine, and mentions that the implementation calls this function, not the user. This is a general pattern with the reference. The leaves are perfectly fine, but the upper levels are often uninformative. IV. PRACTICAL USE ----------------- As part of conducting this review, I ported two existing servers of mine to Beast, one HTTP, one WebSocket. The HTTP server can be seen at https://github.com/pdimov/beast_tpc_server and the WebSocket one is at https://github.com/pdimov/beast_ws_async_server The experience was pleasant and things went more or less as expected, with virtually no nasty surprises. The author was very helpful. The library proper worked exactly as advertised, and the servers were both easier to write than the originals, and became more robust. I only wish that more out of scope things (target parser, basic authorization handling) become in-scope, because their absence is being felt. V. VERDICT ---------- Beast should be ACCEPTED. It's useful, it works, it serves a legitimate need, and I see no design or implementation problems that would preclude acceptance.

2 1

[gil::io] Compilation issue
by Christian Henning 04 Jul '17

04 Jul '17

Hi all, my metaprogramming zen is failing me and I like to ask for some help. Using gil's updated develop branch the following code creates a compilation error with Visual Studio 2015 Update 3. #include <boost/gil/extension/io/tiff_all.hpp> using namespace std; using namespace boost; using namespace gil; int main() { gray1_image_t img; write_view( "", view( img ), tiff_tag() ); return 0; } The compiler output is here: https://pastebin.com/raw/d1AB9QMz I think the error is in https://github.com/boostorg/gil/blob/develop/include/boost/gil/premultiply.… but I'm not sure what to do. Any help would be very helpful! Thanks, Christian

1 1

[Beast] Review
by Artyom Beilis 03 Jul '17

03 Jul '17

Hello, This is my formal review of Boost Beast. > What is your evaluation of the design? Also the general design is solid provides generic API that does the job it defined. The most problematic issue with Beast is it total reliance on static polymorphism. This comes to sometimes very painful issues like: Having the object code duplicated for different variants HTTP/HTTPS code Having all the implementation in headers increasing compilation time Adding complexity to end user since he needs to propagate the templates to various places. Virtually all concepts could be done with classic interfaces and virtual functions. Using of static polymorphism does not provide there performance advantage but reduces code maintainability and adds lots of headache to end user. Note: it does not contradict of using ASIO but lots of stuff could be pimpled or hidden having only few forward declarations of the relevant classes. > What is your evaluation of the implementation? >From what I seen didn't find may flows besides thous mentioned before (content-length - that was fixed) and non-blocking sockets aren't covered by tests. However I didn't look too much since the code is quite hard to read being too template heavy. Second major issue is that it puts all the heavy work on end user providing tiny parsing messaging layer that to be honest is the simple part of HTTP handing. It significantly limits its usability to virtually void. > What is your evaluation of the documentation? Documentation is OK - similar to ASIO - I think more examples should be needed and tutorials regarding how to do common tasks. > What is your evaluation of the potential usefulness of the library? VERY limited. The library does too little to be useful for real development. I don't really see a real need for a library that could potentially be used to create CURL or could potentially be used to write web framework that users will work over them. As a user of this library you need to deal with: - User has to have deep knowledge of HTTP protocol itself and be highly aware of various security related problems on WWW. - Protocol itself and know it up to the small details, timeouts and much more. - Being able to parse all parameters passed from getting query string, parsing multipart/form-data (a.k.a. file upload format) parse percent encoding, cookies. I suggest to drawing the line at some point when library becomes useful for somebody who for examples needs to implement RESTful service or send some post data to a form. The separation line that was drawn does not suite the needs of too many of potential users. I understand it is huge task to do but if first lines in your FAQ and many other talk about what the library doesn't do than probably something goes wrong. And please note I don't talk about sessions, url mapping, but much more basic stuff like multipart parsing, url-encoding. URL parsing, > Did you try to use the library? With which compiler(s)? Did you have any problems? Tried to compile several examples no problem. > How much effort did you put into your evaluation? A glance? A quick > reading? In-depth study? At least 10-12 hours of reading documentation, code, running some examples. Notifying potential problems. > Are you knowledgeable about the problem domain? Yes, I'm myself an author of CppCMS C++ Web Framework - fully featured web development platform for C++ that works in both synchronous and asynchronous modes, I implemented HTTP and FastCGI protocols, networking engines myself. Designed high performance services and more. > Should the library be included to Boost? Unfortunately NO. Two major reasons: - Abuse of static polymorphism where classing object oriented design with virtual functions can do better job. - The scope of the library is way too limited that it brings the question of its usability for real users. I fully acknowledge the huge high quality work the author did - something that kept me reviewing the library and providing the comments, but the library as it is today IMHO doesn't suit Boost Best Regards and Good Luck, Artyom Beilis

1 0

[Beast] HTTP/1.0 Protocol compatibility
by Artyom Beilis 03 Jul '17

03 Jul '17

I can't find there an (sami)automatic way to manage HTTP/1.0 vs HTTP/1.1 compatibility? Example I have a client that supports HTTP/1.0 only and I need to transfer an unknown amount of data. Now as one who writes the server I need to check the clients protocol and choose chucked transfer encoding with possibility to keep connection keep alive or close one after data transfer using socket shutdown. Shouldn't this kind of stuff should be handled automatically or sami-automatically? Keep alive semantics between HTTP/1.0 and HTTP/1.1 is tricky and it is very easy to make a mistake. Also note I tested some of your examples and found that they responding with HTTP/1.1 on HTTP/1.0 requests. Artyom

2 1