- Boost - lists.boost.org

[review][beast] Five-days Remaining
by Michael Caisse 10 Jul '17

10 Jul '17

Hi all - We are half-way through the review and have some great involvement and exchanges and a few reviews already. I hope if you haven't written a review yet that you still will. Please provide whatever effort you can afford toward the review process. Boost depends on community involvement. You might have noticed that I have solicited input from you regardless of your experience with the domain. I personally think it is a mistake to have input only from domain experts. True innovation often comes from "outsiders". If you have no experience with Asio or with HTTP, how approachable is this library? Were you able to understand the documents? Could you use the examples? Were you able to build something of your own? Domain experts -- you know the dark corners of the domain. The hard parts that have caused you hours of lost sleep and the need to hack around an existing library or to write your own. How does Beast address your concerns and the problems you have experienced in the past? Is it flexible enough to be put into your tool chest? So far we have reviews from: * Marcel Ebmer * Peter Dimov * Artyom Beilis * Zach Lain * Nik Bougalis * Klemens Morgenstern * Glen Fernandes If I missed a review from you, please let me know! Happy Coding! michael -- Michael Caisse Ciere Consulting ciere.com

6 13

[beast] Review
by Jens Weller 09 Jul '17

09 Jul '17

This is my review of the beast libary. First, I find the name quite fitting, but also we clearly need a "cage" library to make the internals of beast more useful for the endusers. I spend a few days on and off reviewing examples, and documentation. I used the library as an example for a fuzzing experiment on the weekend. What is your evaluation of the design? The design is very boost like, the library fits well into boost. One thing the library shares with asio is the buffer concept, which I never really liked. Buffer feels to me to abstract, to far away from the usual type vocabulary one is used to from standard C++. But once you're used to the concept, its ok. So no big deal. The dependence on Asio was already mentioned during the review and I am not sure about this. Surely a nice goal to have, but also not an easy one. But replacing asio with an abstraction that allows for pluggin in other socket implementations would be very powerful. Its nice that the library is header only, but over asio you get the dependency to boost::system for error_code. I'd love to be able to exchange boost::error_code dependency to the std::error_code one. As the beauty of header only really shines when you don't need to link to other non header only libraries... What is your evaluation of the implementation? Fuzzing. I spend this weekend some time to fuzz beast with libFuzzer. The basic_parser and the websocket::stream were fuzzed. A bug (buffer overflow) in basic_parser was found, and is already fixed. What is your evaluation of the documentation? Boost typical. Its good, helps with getting the library to know. What is your evaluation of the potential usefulness of the library? This library could be the foundation of many things to come. The internet is the basic building block for so much today, that beast is a nice improvement upon asio in providing the vocabulary to use boost in the internet age. As I already mentioned, I think that other libraries are needed in order to make this library really useful for most end users. But the scope the author aims at, beast fits very well. Did you try to use the library? With what compiler? Did you have any problems? Fuzzing was based on clang 5.0, but I also used the library with GCC 5.4 to generate example data for the fuzzing. I plan to use the library to write a small http server and clients for my own usage. How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? My time for this review is limited, but as the author mentioned on this list, that the library hasn't been fuzzed, I aimed for doing this instead of implementing an http client/server. I still had to write code in order to reach the parts in beast to be fuzzed, this gave a more detailed tour of beast, then one would need for simple example exploring. Are you knowledgeable about the problem domain? A little, I am mostly a high level user of various HTTP libraries, but not at the low level beast covers. I found it interesting to get a glance at the details. And finally, every review should answer this question: Do you think the library should be accepted as a Boost library? Be sure to say this explicitly so that your other comments don't obscure your overall opinion. I think that this library should be ACCEPTED into boost. It is an important improvement on asio, in order to offer a more detailed networking stack in boost. The library has the exact right scope to do this, widening the scope into providing client/server implementations would not be good. I see this in another library. thanks, Jens Weller

3 4

Boost 1.65.0 Beta 1 Release Candidate 1
by Marshall Clow 09 Jul '17

09 Jul '17

The release candidates for the 1.61.0 release are now available at: https://dl.bintray.com/boostorg/beta/boost_1_65_0_beta1_rc1.7z https://dl.bintray.com/boostorg/beta/boost_1_65_0_beta1_rc1.tar.bz2 https://dl.bintray.com/boostorg/beta/boost_1_65_0_beta1_rc1.tar.gz https://dl.bintray.com/boostorg/beta/boost_1_65_0_beta1_rc1.zip The SHA256 checksums are as follows: cd7429340bb4aae38e0ad2e9f59c39e66335302c41577956123e55fc1e2a28ee boost_1_65_0_beta1_rc1.7z fda076f7b3c994524d9cd41926c72f4a9b7c017bef310a055956fdd0bed7c358 boost_1_65_0_beta1_rc1.tar.bz2 0d1473041f7a9fa9ce1e256c870cc52af6f98abf723cf4f1ac61c2b45f45d38a boost_1_65_0_beta1_rc1.tar.gz f45bbb75b84cdc0c66f8a571d77c518cfbac076d7a6fc7937c4812f0f242a792 boost_1_65_0_beta1_rc1.zip As always, the release managers would appreciate it if you download the candidate of your choice and give building it a try. Please report both success and failure, and anything else that is noteworthy. Thanks! -- The release managers

4 5

[Safe Numerics] Review
by John Maddock 09 Jul '17

09 Jul '17

OK, here's my review of Robert's library. By way of something different, I've tried to not read the docs (though I ended up scanning them in the end) and just tried to use the library. Here's what I've found: 1) Using: safe<int>(-1) & 1 I get a hard assert failure at safe_base_operations.hpp:1373. I'm assuming that this is a mistake, and the usual error handlers should be called? Bitwise or has the same issue. More tests required ;) 2) I'm not sure if this is an msvc issue but in: constexpr safe_base operator~() The static assert is triggered for *unsigned types* and not for signed. Even if this is a compiler issue, it indicates a missing test case or two. 3) If I assign a float to an integer, then I get the error: "conversion of integer to float loses precision" which isn't quite what you meant to say. More particularly, for float conversions I would expect: * Conversion from a float holding an integer value that's within the range of the target type to always succeed. * Conversion from a float holding a non-integral value to conditionally succeed (with trunction) depending on the policy in effect. * Conversion *to* a float may also fail if the integer is outside the range of the float (no checking may be required if the number of bits in the integer is less than the number of bits in the float). 4) Is constexpr arithmetic supposed to be supported? I tried: constexpr boost::numeric::safe<int> j = 0; constexpr boost::numeric::safe<int> k = 3; constexpr boost::numeric::safe<int> l = j + k; Which generates an internal compiler error for msvc, and for gcc-5.3 I get: ../../../include/safe_base_operations.hpp:332:35: error: call to non-constexpr function ‘void boost::numeric::dispatch(const boost::numeric::checked_result<R>&) [with EP = boost::numeric::throw_exception; R = int]’ dispatch<exception_policy>(r); Similarly if I try to add a literal to a safe<int>. Again as the tests all pass, so I'm assuming they're missing constexpr tests? 5) What is the purpose of class safe_literal? constepxr initialization seems to work just fine without it? 6) In the docs, the example of using error handler dispatch is nonesense (top of page 41)? 7) Related to the above, I see the need to be able to define your own primitives which behave just like yours. The example I have is for bit-scan-left/right which has as precondition that the argument is non-zero, but there are other bit-fiddling operations (bit count etc) which some folks may require. A "cookbook" example would be good. 8) In the tests, you have an undocumented function "log" in the library. Not sure if this is intended to be part of the API or not, but IMO it's misnamed. ilog2 seems to a semi-standard name for this. 9) By way of testing the libraries performance I hooked it into the Millar-Rabin primality test in the multiprecision lib (test attached, it counts how many probable primes there are below 30000000), here are the times for msvc: Testing type unsigned: time = 17.6503 seconds count = 1857858 Testing type safe<unsigned>: time = 83.5055 seconds count = 1857858 Testing type 32-bit cpp_int: time = 36.9026 seconds count = 1857858 and for GCC-MINGW-5.3.0: Testing type unsigned: time = 17.1037 seconds count = 1857858 Testing type unsigned: time = 18.4377 seconds count = 1857858 Testing type unsigned: time = 21.0592 seconds count = 1857858 I'm surprised by the difference in times between msvc and gcc, also deeply impressed that there is effectively no performance penalty with GCC (I was expecting there should be some!) 10) I don't find the scope of the library over-broad, in fact I probably find it about right wrt the policies etc. 11) There were a few traits and such that it would be nice to have, the problem as ever, is what to call them and where to put them: * A trait for extracting the underlying type of a safe<> - use case for example where you enable_if a function on is_safe so don't have the template argument list to safe<>. * Do we need a function for extracting the underlying type of a safe<>? Or is casting to the underlying type optimal? Actually, maybe having a function would negate the need for the above trait, as we could just write 'auto x = mysafe.value();' ? * Do we need traits equivalent to is_signed/is_unsigned/make_signed/make_unsigned? My gut feeling is yes we do, the issue is what to call them, and where to put them - we're not allowed to specialize the type_traits versions (either boost or std), but we would need a central "one true trait" version that can be specialized by other libraries as well (Multiprecision for example). We've backed ourselves into a corner over this I fear! 12) When calling boost::integer::gcd/lcm the sub-optimal(?) non-binary versions are called for safe<> - you might want to look into seeing what traits need to be defined to ensure that the same version as the underlying type is called. Those are all the issues I can see for now... in conclusion, as for whether the library should be accepted, yes I believe it should be, modulo the issues above. Regards, John. --- This email has been checked for viruses by AVG. http://www.avg.com

4 14

Re: [boost] [review][beast] Review of Beast starts today : July 1 - July 10
by Vinnie Falco 09 Jul '17

09 Jul '17

On Mon, Jul 3, 2017 at 9:54 AM, Emil Dotchevski via Boost <boost(a)lists.boost.org> wrote: > ... > The question is must Beast be coupled to Asio, not can it be used without > something like Asio. And I don't mean just in the case of someone using > serializer/basic_parser and nothing else. > > In my opinion, arguing that coupling is necessary should begin with > defining an interface which Beast can use, which can be trivially > implemented in terms of Asio. In designing this interface the only concern > should be avoiding the coupling; specifically, performance considerations > should be completely ignored. Only then we can have a reasonable discussion > is it worth it. Regardless, in my experience this kind of exercise always > improves the design of a library. The question is, can Beast be written against generic concepts which define synchronous and asynchronous operations over buffer oriented streams, instead of being tied to Boost.Asio? First lets imagine what such generic concepts might look like, independent of Beast. We will start with the synchronous case which is the easiest. In the code which follows: * `Stream` is a type representing a buffer oriented I/O stream * `Buffers` is type representing an ordered sequence of zero or more non-owning references to contiguous octet buffers * `Error` is type representing an error code The following synchronous stream operations are defined: /** Read data from a stream synchronously `stream` The stream to read from `buffers` The buffers to read into `error` Set to the error if any occurred. returns: The number of bytes transferred, or 0 on error. */ std::size read( Stream& stream, Buffers const& buffers, Error& error); /** Write data to a stream synchronously `stream The stream to write to `buffers The buffers to write from `error` Set to the error if any occurred. returns: The number of bytes transferred, or 0 on error. */ std::size_t write( Stream& stream, Buffers const& buffers, Error& error); This looks pretty generic and at this point users familiar with Boost.Asio might raise their hands to tell me that the interfaces look identical. This is just a coincidence because, lets face it - synchronous interfaces don't come in that many varieties. I don't think synchronous interfaces are particularly interesting and Beast could certainly be made to use something like this but even with simple interfaces some issues come up: 1. What is the concrete type of Error? Or is Error a template parameter that meets certain requirements? 2. HTTP stream read algorithms need to know when the stream, is closed, what error code represents End-of-File? 3. What are the requirements of Buffers? Beast could be written against generic concepts, but in order to do so the questions above need to be answered (points 1 through 3). Otherwise there is no specification and thus, no way to write an algorithm. We will come back to these questions, but now on to the next topic. We want Beast to also provide HTTP operations on asynchronous streams, so we need to have a generic concept against which to work. There are many models of asynchronous computation. For example there are futures, coroutines, callbacks, and queues (I can't really think of any others at the moment). Lets imagine what callback-based generic asynchronous stream read and write operations might look like: In the code which follows: * `Stream` is a type representing a buffer oriented I/O stream * `Buffers` is type representing an ordered sequence of zero or more non-owning references to contiguous octet buffers * `Error` is type representing an error code * `Callback` is a type which is Callable with the signature `void(Error, std::size_t)` The following asynchronous stream operations are defined: /** Begin an operation to read data from a stream asynchronously `stream` The stream to read from `buffers` The buffers to read into `callback` A callback to invoke when the operation completes. The callback will receive the error if any, and the number of bytes transferred. returns: The number of bytes transferred, or 0 on error. */ void async_read( Stream& stream, Buffers const& buffers, Callback const& callback); /** Begin an operation to write data to a stream synchronously `stream` The stream to write to `buffers` The buffers to write from `callback` A callback to invoke when the operation completes. The callback will receive the error if any, and the number of bytes transferred. returns: The number of bytes transferred, or 0 on error. */ void async_write( Stream& stream, Buffers const& buffers, Callback const& callback); Now anyone who knows Asio is going to stop me and say, hey! This looks a lot like Asio! But that's a side issue because this generic interface I have provided has some defects. Or rather, you can say that it is under-specified. We have the same questions about the requirements for the types Error and Buffers, and the value representing End-of-File. But now there are even bigger questions with significant ramifications: 1. Upon which thread is the callback invoked? 2. Who invokes the callback and how is it invoked? 3. How is access to the `stream` synchronized? 4. What about futures and coroutines? The simple generic free function signatures alone which I provided above are not sufficient to describe a complete model for asynchronous operations on streams. We also need provisions to answer the questions above. Does the implementation provide the threads, or are they provided by the user? Does synchronization use mutexes? Lock-free programming? Queues? Back to this in a moment... Some folks who either dislike callbacks or just happen to love futures may raise an objection that the callback-based generic model I have described does not capture their use case. Actually, Christopher Kohlhoff (author of Boost.Asio) demonstrates how the callback-based model is a proper superset of all other asynchronous computation models. And he shows that callback-based models provide the greatest opportunity for performance and efficiency by doing away with unnecessary synchronization points. This is explained in his paper N3896 "Library Foundations for Asynchronous Operations" and shows how initiating functions like `async_read` and `async_write` above may be trivially modified to support not just callbacks but also futures, coroutines, and user-defined types. Beast fully supports this system and even comes with tutorials and helper classes which let you implement the model yourself in your own initiating functions. "Library Foundations for Asynchronous Operations" (N3896) <http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3896.pdf> "Writing Composed Operations" (Beast) <http://vinniefalco.github.io/stage/beast/review/beast/using_networking/writ…> Thankfully this solves one piece of our puzzle. N3896 allows us to say with certainty that our hypothetical generic network model should use callbacks as the foundational notification mechanism for the completion of asynchronous operations. Perhaps we can look towards the author of N3896 for more solutions? Drumroll... Working Draft, C++ extensions for Networking (N4588) a.k.a. "Networking-TS" <http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/n4588.pdf> What is this you ask? Well... this is a working draft which describes a proposed set of C++ extensions for Networking. It has TCP/IP sockets and UDP streams, serial ports I think (?) but lets not worry about that. Most importantly, the Networking-TS defines a generic model for synchronous and asynchronous stream-oriented I/O operations! This means that: Synchronous and asynchronous algorithms written against Networking-TS concepts will work with any stream types that meet the requirements ** Yes, Networking-TS comes with generic stream concepts. Lets restate the original question: "Can Beast be written against generic concepts which define synchronous and asynchronous operations over buffer oriented streams?" The answer is: Yes. Beast is already written against generic concepts which define synchronous and asynchronous operations over buffer oriented streams. These are provided by Boost.Asio. Specifically, these concepts: SyncReadStream <https://timsong-cpp.github.io/cppwp/networking-ts/buffer.stream.reqmts.sync…> SyncWriteStream <https://timsong-cpp.github.io/cppwp/networking-ts/buffer.stream.reqmts.sync…> AsyncReadStream <https://timsong-cpp.github.io/cppwp/networking-ts/buffer.async.read> AsyncWriteStream <https://timsong-cpp.github.io/cppwp/networking-ts/buffer.async.write> Its not the responsibility or goal of Beast to invent a new generic model of synchronous or asynchronous networking. That's already been taken care of. Beast follows it. It is the job of authors to make sure their network implementations meet the requirements of the various Networking-TS stream concepts. Once they do that, then any algorithms written against the concepts can now work with these implementations. Again folks might raise their hand and say "but Beast depends on Boost.Asio!" and they would be right. The reality is that Networking-TS has not entered the standard and is not uniformly available across compiler vendors. Boost.Asio is the closest thing to the standardized version of Networking-TS, so Beast is written against that. Its not perfect, but it is the best we can do and the runner-up is not even in the race (are there any other generic stream designs?). I believe that the burden of proof rests with those who claim that Beast should be written against a different generic stream design. Please show me one such design that is mature, well-specified, and in-use, other than Boost.Asio. tl;dr; Boost.Asio's models of synchronous and asynchronous buffer-oriented streams are already generic, and Beast is written against Boost.Asio. Thanks

2 1

Re: [boost] Boost 1.65.0 Beta 1 Release Candidate 1
by berne＠notadragon.com 09 Jul '17

09 Jul '17

Hi, I've been playing aournd with boost coroutines2 and maybe I'm just confused about where this project is, but with 1.65 coroutines2 doesn't seem to be buildable out of the box any longer: I've tried looking through mailing list archives for anything about the status of coroutine2 to no avail, and coroutine still spits out warnings to use coroutine2 when used. I'm not sure where in the build system things hook in to fix this. Here's the output on my system: berne@Bugg:~/tmp$ tar xzf boost_1_65_0_beta1_rc1.tar.gz berne@Bugg:~/tmp$ cd boost_1_65_0/ berne@Bugg:~/tmp/boost_1_65_0$ ./bootstrap.sh --with-toolset=clang Building Boost.Build engine with toolset clang... tools/build/src/engine/bin.linuxx86_64/b2 Detecting Python version... 2.7 Detecting Python root... /usr Unicode/ICU support for Boost.Regex?... not found. Generating Boost.Build configuration in project-config.jam... Bootstrapping is done. To build, run: ./b2 To adjust configuration, edit 'project-config.jam'. Further information: - Command line help: ./b2 --help - Getting started guide: http://www.boost.org/more/getting_started/unix-variants.html - Boost.Build documentation: http://www.boost.org/build/doc/html/index.html berne@Bugg:~/tmp/boost_1_65_0$ ./b2 --with-coroutine2 error: wrong library name 'coroutine2' in the --with-<library> option. Thanks. - Josh

2 2

[beast] Review
by Gyorgy Szekely 09 Jul '17

09 Jul '17

Hi, Some background: this is my first Boost library review. I followed the discussion on Beast and I'm probably not as experienced as others on this topic. I started learning Beast along with Asio and it was a really good experience. After understanding ASIO concepts, I could get an async multi-threaded http server running in a few hours. I leave it up to the review manager whether to take this a formal review or just informal feedback (impressions of newbie :)) *** What is your evaluation of the design? The library in my opinion is very well designed. I doesn't try to offer too much. I understand the author's position: HTTP is vast and Beast is a low-level library. Still, while using it I felt with a few additional bits it would be more complete. What I was missing is parsers/serializers for uri and cookies. I think Beast should provide these (even in a separate namespace), as they are not higher level functionality than parsing/serializing http headers. (I implemented them in my application, and I'm sure my code is neither correct regarding corner cases, nor standards compliant. I'm also sure that I'm not alone with this, so there's a place for this in either Beast, or as the author suggests a higher level lib.) Though the library is heavily templated, as a user I'm not forced to write templated code, implementing the most common use-cases is simple and straightforward. That's a big plus. I really like the symmetric client/server design. My application is http/websocket server, being able to use the the same classes and functions in unit tests made reasoning about the code and tests much easier. Finally there's was a lot of debate whether being ASIO and stream concept based is a good idea or not. For me (as a newcomer) seamless integration with ASIO helped getting started really fast, though I didn't do awfully complex stuff. *** What is your evaluation of the implementation? I did not review all the implementation. While using the library, I looked into some of the files, and my impression was that the code is high quality. The test coverage is also high, and issue fixes are accompanied with rigorous tests. The library is mostly composed of templates, which makes a bit harder to understand its internals, but I didn't mind it at all given the benefits (composablity, customization points). *** What is your evaluation of the documentation? The documentation is well structured and plenty. Sections from the Intro up to the Reference are a good read by themselves. Design decisions and library scope are clearly stated. There're lots of examples in form of code snipets highlighting a particular use-case, as well as fully functional programs demonstrating these in action. I liked that the documentation is similar to ASIO's and I could use them together. *** What is your evaluation of the potential usefulness of the library? It's super useful. Even with its (current) dependency on ASIO and some other Boost libs Beast is small while providing both client and server side foundations for Websocket and HTTP. NodeJS and Java makes it so simple to create REST services, this should be hard in C++ either with standards based (NetTS) solutions. Beast is a step in the right direction. *** Did you try to use the library? With which compiler(s)? Did you have any problems? I'm actively using the library in the Http/Websocket <-> ZeroMQ gateway since Oct 2016 on Ubuntu 16.04 with GCC5, C++14. We've run extensive performance and stability tests on the HTTP interface and we're quite satisfied with the results. I migrated our codebase to newer Beast versions multiple times, and during this I reported a few minor issues or requested new features. The library author was really responsive and fixed/implemented them quickly. *** How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? A few hours this time, but I've read the documentation and the public API reference multiple times in the last months. I didn't go through the implementation, but peeked into it here and there to understand some of the details. *** Are you knowledgeable about the problem domain? Not really. I've implemented some NodeJS and Java Spring REST services, but that's all. My conclusion: Beast should be ACCEPTED. Regards, Gyorgy Szekely

1 0

[sort] pdqsort mini-review
by Steven Ross 08 Jul '17

08 Jul '17

The (mini) review for pdqsort will start tomorrow 6/21, and run through 6/30. I will manage the review; Orson Peters is the author. The proposed boost source is here: https://gist.github.com/orlp/24f27aada1ed724d1809b372c33eb92e Documentation is here: https://github.com/orlp/pdqsort#pdqsort And a draft paper is here: https://drive.google.com/open?id=0B1-vl-dPgKm_T0Fxeno1a0lGT0E If you're interested, please answer these questions: 1. Are there any specific concerns you have about pdqsort vs. std::sort? 2. Does it compile for you when you try to use it as a replacement for std::sort? 3. How does the performance compare on your system vs. std::sort? 4. Do you think we should include pdqsort in Boost.Sort? 5. Do you have any concerns about switching the default/fallback sort in Boost libraries (including Boost.Sort) from std::sort to pdqsort?

7 10

[review] Mp11 review starts in one week
by Bjorn Reese 08 Jul '17

08 Jul '17

The formal review of Peter Dimov's Mp11 library is scheduled for July 15 - July 24, 2017 [1]. Mp11 is a C++11 metaprogramming library for compile-time manipulation of data structures that contain types. It’s based on template aliases and variadic templates and implements the approach outlined in the article "Simple C++ metaprogramming" [2] and its sequel [3]. These articles are useful background information for the review. * Mp11 aims to make simple usage simple, and to support complex usage without complicating the simple use cases. * Mp11 works on any type-list, whether its own type-list mp_list, or standard type-lists such as std::tuple and std::variant, or user-defined type-lists. * Mp11 works with any meta-function, such as C++11 or Boost type-traits, or user-defined type-traits. Mp11 can be found here: * Documentation: https://rawgit.com/pdimov/mp11/master/doc/html/mp11.html * Source code: https://github.com/pdimov/mp11/tree/master [1] http://www.boost.org/community/review_schedule.html [2] http://pdimov.com/cpp2/simple_cxx11_metaprogramming.html [3] http://pdimov.com/cpp2/simple_cxx11_metaprogramming_2.html

1 0

[beast] Missing documentation link
by Bjorn Reese 08 Jul '17

08 Jul '17

The link to example/common/file_body.hpp in the Examples section has gone missing. Please fix this ASAP. All links in the Examples section point the the master branch instead of the review branch.

2 2