March 2017 - Boost - lists.boost.org

[Beast] Redesigned HTTP algorithms and parser!
by Vinnie Falco 31 Mar '17

31 Mar '17

Background: Websocket connections start with an HTTP UPGRADE request/response sequence. Often people want to customize this handshake so I added code in Beast to read and write HTTP messages. Upon reviewing the HTTP RFC (rfc2616 and rfc7230) it seemed to be logical that the HTTP message could be modeled as a single class. And there could be free functions implementing the algorithms to serialize and deserialize these messages in th HTTP/1 format. I took this as far as I could and I was pretty happy with the result. You have functions like read, async_read, write, and async_write which take beast::http::message objects as parameters. Example: http://vinniefalco.github.io/beast/beast/ref/http__read/overload3.html The innovation here is the "Body" template type which controls not only the type of container used to represent the body but also the algorithms used to transfer bytes from the message body on the wire into and out of that container. The Body concept, and its nested Reader and Writer concepts, are documented here: http://vinniefalco.github.io/beast/beast/ref/Body.html http://vinniefalco.github.io/beast/beast/ref/Reader.html http://vinniefalco.github.io/beast/beast/ref/Writer.html Previous efforts and HTTP implementations focused heavily on the representation and parsing of the headers. To my knowledge Beast is the first implementation to provide caller flexibility in the body representation. Now the Beast websocket implementation has a reasonable set of primitives it could use to perform the handshake, and users could get control of that process. The Problem: Unfortunately, while treating the message as an object that can be read/written atomically works great for example programs and trivial use-cases, it fails in the real world. These are issues with it: * The caller must commit to a Body representation before receiving the HTTP Header * No way to properly read and respond to Expect: 100-continue * An efficient HTTP relay function (one that does not buffer the entire body up front) cannot be implemented by the caller * The caller cannot reasonably set a timeout, for example what if the body is 2 gigabytes received over a slow connection? * The parser interface forces algorithms to perform an unnecessary buffer copy for body containers that are entirely stored in memory (e.g std::string or boost::asio::streambuf). These issues sparked quite a bit of discussion on GitHub: https://github.com/vinniefalco/Beast/issues/154 https://github.com/vinniefalco/Beast/issues/265 In October of 2016 these problems became apparent. I tried a lot of different ideas to improve the design to solve it, but none of them worked. Some of my co-workers looked at it but we didn't really find anything great. One user in particular, on GitHub, had strong ideas involving the concept of "completion conditions" to augment the read/parse algorithms but it didn't feel right to me. The Solution: After suffering from writer's block for 5 months inspiration hit and I tried a combination of interface changes that got the design to where it needs to be to solve these problems. * The Reader concept is refined into two types: "indirect" readers, and "direct" readers. This is described here: http://vinniefalco.github.io/stage/beast/beast/ref/Reader.html * The parser exposes a small number of internal states, informing callers of where it is at and giving the caller control over if and when the parser advances to the next state: https://github.com/vinniefalco/Beast/blob/http/include/beast/http/basic_par… * The parser is rewritten to work only on linear buffers (allowing it to be optimized to become 20 times faster if SSE2 instructions are allowed). To facilitate this requirement, the flat_streambuf class is provided which works just like a basic_streambuf except it guarantees a single buffer. If the parser is given multiple buffers, it allocates memory to flatten it: https://github.com/vinniefalco/Beast/blob/http/include/beast/core/flat_stre… * New functions parse_some and async_parse_some allow the caller to perform incremental parsing. This allows for the implementation of relay functions, or reading just the header (for Expect: 100-continue): https://github.com/vinniefalco/Beast/blob/http/include/beast/http/parse.hpp… With these changes callers can get control of the inner loop that reads the body off the socket or stream, so they can do whatever custom steps are necessary if they want. They can set timeouts, and write their own loop that calls parse_some repeatedly. I think it solves all of these issues. I've written some test code which demonstrates how direct and indirect body readers are implemented, and what calling code looks like. I've also written a prototype "relay" function (read a message from one socket and efficiently write it to another socket). And there's an example of how Expect: 100-continue might be implemented. These prototypes are located here: https://github.com/vinniefalco/Beast/blob/http/test/http/design.cpp#L30 How to Help? These are some controversial changes and I think they could use some polish and vetting before I merge it to the master branch. Maybe there is something I haven't thought of, or a use-case it doesn't address. Perhaps there is a better way, or perhaps there are tweaks to make it even more usable and understandable. If some kind soul(s) would look through it and offer some feedback, positive or negative, that would be enormously helpful. The formal review for Beast is now scheduled for the first week of July and like an expectant father I am anxious and would like everything to go just right! My only objective is to polish up the library and its documentation so it is in the best possible shape for review. Thanks for listening!

1 1

[release] Boost 1.64.0 Beta 2
by Rene Rivera 31 Mar '17

31 Mar '17

Boost release 1.64.0 beta 2 is now available at: <https://dl.bintray.com/boostorg/beta/1.64.0.beta.2/source/> The SHA256 checksums are as follows: 49c9339b9d8e7b2610ef84c80cd10b2252c119ebcc3222450beb0e94c1da1522 boost_1_64_0_b2.7z 92ec7cbf1e77c5979e381dd8104f9e96c0471c276e566272751a981d806db94b boost_1_64_0_b2.tar.bz2 da5670892b7b572808491b13a7b9421c3c18e6f395d6a5ec822466db466eda01 boost_1_64_0_b2.tar.gz 61d6e29e8a1d7503fe13a9d916842f0ec2e625f733b98b29ce563f8cbdb482bf boost_1_64_0_b2.zip The notable difference from Beta 1 is the inclusion of the missing Boost Process library. For details of what's in the release, see < http://www.boost.org/users/history/version_1_64_0.html>. Note that the links to files on this web page are for the final release - use the link above to get the beta files. There are also pre-built libraries provided by Tom Kent here: <https://dl.bintray.com/boostorg/beta/1.64.0.beta.2/binaries/> Please download the beta, give it a try, and report any problems you encounter. Thanks, -- The Boost Release Team -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail

6 5

[gsoc17]Proposal draft shared of Boost.Geometry Project 1
by Ruoyun Jing 31 Mar '17

31 Mar '17

Hi Adam, Vissarion, boost all: I have finished my proposal of Boost.Geometry Project 1, I am very pleasure you to give me more advice of my proposal! :) Any aspect of the proposal is OK! I would like to get feed back from you! Looking forward from you! :) Thanks all and Best Regards. Google docs: https://docs.google.com/document/d/1wj9xPSmdehh9MzE-3MGHMu7cwMhDPASUEvU9pJb… Ruoyun Jing -- Northwest University of China Software Engineering jingry0321(a)gmail.com <jingry0321(a)163.com>

1 0

Boost.SIMD review
by serge guelton 30 Mar '17

30 Mar '17

Hi Boost Dev, I'm the core maintainer of Pythran[0], a python-to-native-code compiler focused on numeric computations. For more than 3 years, we've been using Boost.SIMD [1] as a generic backend for SIMD code generation, without much worries. Basically our compiler generates C++ code that includes Boost.simd calls. The big plus to us are: + Performances. The generated code runs at the same speed as the one written with intrinsics; + Target abstraction, esp. concerning the vector size and the various arithmetic operations; + Complete libm API, vectorized, which is a big plus when porting existing code (or when matching an existing API as we do); + Header only, which actually makes it easier for us to ship; + Low requirements (it does not require us to ship a large part of boost).? The compilation time is sometimes a worry, but it has decreased over the year. We don't use complex data movement like shuffle or pack, so I don't have any feedback on that. All in one, if that ever matters, I would be happy to have it integrated in Boost! ++ Serge [0] https://pythonhosted.org/pythran/ [1] https://github.com/NumScale/boost.simd

1 0

NuDB: A fast key/value insert-only database for SSD drives in C++11
by Vinnie Falco 30 Mar '17

30 Mar '17

I just released version 1.0.0 of my header-only C++11 library NuDB. Its quite a mature library, haven't had to make any changes in a while. Its been running on production servers for 2 years or so with no problems, managing ever-growing databases of 2TB. I'm wondering what the level of interest, if any, there would be for making this a part of Boost. You can check it out here: https://github.com/vinniefalco/NuDB

11 91

Problem creating mapping from string to vector_expression
by SHIVAM MITTAL 30 Mar '17

30 Mar '17

Hi, I have been trying to create a mapping from string to ublas::vector_expression. template <class T> boost::unordered_map<std::string, vector_expression <T> > c; vector<int> v (10); for (unsigned i = 0; i < v.size (); ++ i) v (i) = i; c <int>["first"] = v; I have been getting this error : error: no match for ‘operator=’ (operand types are ‘boost::unordered::unordered_map<std::__cxx11::basic_string<char>, boost::numeric::ublas::vector_expression<int>, boost::hash<std::__cxx11::basic_string<char> >, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<const std::__cxx11::basic_string<char>, boost::numeric::ublas::vector_expression<int> > > >::mapped_type {aka boost::numeric::ublas::vector_expression<int>}’ and ‘boost::numeric::ublas::vector<int>’) columns <int>["first"] = v; Please help.

1 0

[build][geometry][regression] Geometry develop regression tests are blank
by Adam Wulkiewicz 30 Mar '17

30 Mar '17

Hi, Boost.Geometry develop tests (http://www.boost.org/development/tests/develop/developer/geometry.html) stopped showing around 22.03. The first blank run (teeks99-03-dc3.9-1z-Docker-64on64) is from Wed, 22 Mar 2017 01:20:13. Do you have an idea what may be the cause? I see that many changes in Build and Regression was done around 22.03. Geometry was also modified that day but the first commit is from 12:40+1 so later than the first blank run. I checked the XML of one of them (teeks99-09-n-win2012R2-64on64) and it seems that the revision attribute of <test-log> tag is not set for the most of the tests. I'm currently running the regression.py myself in hope of finding the cause. I see that in bjam.log the only tests that are dumped are: boost-test(RUN_PYD) "geometry/__boost_check_library__" : "status/boost_check_library.py" boost-test(RUN_PYD) "geometry/extensions/__boost_check_library__" : "status/boost_check_library.py" boost-test(RUN) "geometry/index/varray_old" : "libs/geometry/index/test/varray_old.cpp" boost-test(RUN) "geometry/index/varray" : "libs/geometry/index/test/varray.cpp" boost-test(RUN_PYD) "geometry/index/__boost_check_library__" : "status/boost_check_library.py" varray tests are the only ones that are run at the most top level of Geometry Jamfiles (of sublib geometry/index in this case) with test-suite: https://github.com/boostorg/geometry/blob/develop/index/test/Jamfile.v2#L24 The rest of the tests is run with build-project, also for the main part of the library: https://github.com/boostorg/geometry/blob/develop/test/Jamfile.v2#L26 The varray tests results are shown on the regression page and can be found in bjam.log and in XML file with revision set. The rest of the tests is neither dumped nor has the revision set, however they are run during regression testing so can be found in bjam.log and also in XML file but with empty revision. Adam

2 5

Broken link to conversion library
by Antony Polukhin 29 Mar '17

29 Mar '17

Hello, There is a broken link to Conversion library on the changelog page: http://www.boost.org/users/history/version_1_64_0.html Can not figure out what's the problem. Could someone take a look? -- Best regards, Antony Polukhin

3 3

[Boost.Test] Custom DataSet compiler error with no match for »operator^
by Olaf Peter 29 Mar '17

29 Mar '17

Hello, I run into a compile problem with [Boost.Test] DataSet. I can reduce it by re-using the example provided by the docs: http://www.boost.org/doc/libs/1_62_0/libs/test/doc/html/boost_test/tests_or… #define BOOST_TEST_MODULE dataset_example68 #include <boost/test/included/unit_test.hpp> #include <boost/test/data/test_case.hpp> #include <boost/test/data/monomorphic.hpp> #include <sstream> namespace bdata = boost::unit_test::data; // Dataset generating a Fibonacci sequence class fibonacci_dataset { public: // Samples type is int using sample=int; enum { arity = 1 }; struct iterator { iterator() : a(1), b(1) {} int operator*() const { return b; } void operator++() { a = a + b; std::swap(a, b); } private: int a; int b; // b is the output }; fibonacci_dataset() {} // size is infinite bdata::size_t size() const { return bdata::BOOST_TEST_DS_INFINITE_SIZE; } // iterator iterator begin() const { return iterator(); } }; namespace boost { namespace unit_test { namespace data { namespace monomorphic { // registering fibonacci_dataset as a proper dataset template <> struct is_dataset<fibonacci_dataset> : boost::mpl::true_ {}; }}}} // Creating a test-driven dataset BOOST_DATA_TEST_CASE( test1, fibonacci_dataset() ^ fibonacci_dataset(), fib_sample, exp) { BOOST_TEST(fib_sample == exp); } error: no match for 'operator^' (operand types are 'boost::unit_test::data::result_of::make<fibonacci_dataset>::type {aka fibonacci_dataset}' and 'fibonacci_dataset') fibonacci_dataset() ^ fibonacci_dataset(), My use case has to load input and expect files provides by two datasets. How to fix it? Thanks, Olaf

3 3

Re: [boost] [Boost-users] Going insane trying to build boost for android
by Robert Dailey 29 Mar '17

29 Mar '17

On Mon, Mar 27, 2017 at 11:54 AM, John Maddock <jz.maddock(a)googlemail.com> wrote: > >> That won't do me any good because I'm already using this toolchain >> perfectly fine via CMake and ndk-build. bjam isn't working, maybe >> because I haven't set up the config.jam file properly. I'm not sure at >> this point. Boost is the only thing not working here. > > > Then if you compare the compilation command line generated when building > under b2 and when building under cmake the error should be obvious? I took your advice and compared a real build that worked using CMake (using a simple test cpp file) and what I got from b2. Here is the command line used to build the sample app using CMake. Note that the compilation succeeded: C:\android\ndk\toolchains\arm-linux-androideabi-4.9\prebuilt\windows-x86_64\bin\arm-linux-androideabi-g++.exe --sysroot=C:/android/ndk/platforms/android-15/arch-arm -Dndk_test_EXPORTS -isystem C:/android/ndk/sources/cxx-stl/llvm-libc++/include -isystem C:/android/ndk/sources/android/support/include -isystem C:/android/ndk/sources/cxx-stl/llvm-libc++abi/include -isystem C:/android/ndk/platforms/android-15/arch-arm/usr/include -march=armv7-a -mthumb -mfpu=vfpv3-d16 -mfloat-abi=softfp -funwind-tables -no-canonical-prefixes -fexceptions -frtti -g -fPIC -std=gnu++14 -MD -MT CMakeFiles/ndk_test.dir/main.cpp.o -MF CMakeFiles\ndk_test.dir\main.cpp.o.d -o CMakeFiles/ndk_test.dir/main.cpp.o -c ../main.cpp And here is the command line from b2: "C:/android/ndk/toolchains/llvm/prebuilt/windows-x86_64/bin/clang++" -c -x c++ -O3 -Wno-inline -Wall --sysroot=C:/android/ndk/platforms/android-15/arch-arm -isystem C:/android/ndk/sources/cxx-stl/llvm-libc++/include -isystem C:/android/ndk/sources/android/support/include -isystem C:/android/ndk/sources/cxx-stl/llvm-libc++abi/include -isystem C:/android/ndk/platforms/android-15/arch-arm/usr/include -std=c++14 -fno-strict-aliasing -D__GLIBC__ -DANDROID -D__ANDROID__ -D_GLIBCXX__PTHREADS -D__arm__ -D_REENTRANT -O2 -DNDEBUG -H -DBOOST_ALL_NO_LIB=1 -DBOOST_ATOMIC_SOURCE -DBOOST_ATOMIC_STATIC_LINK=1 -DNDEBUG -I"." -o "bin.v2\libs\atomic\build\clang-linux-android\release\link-static\target-os-android\threadapi-pthread\threading-multi\lockpool.o" "libs\atomic\src\lockpool.cpp" The build for boost still fails. Note I also passed the -H option, so I could see the include searches performed by clang as it was building boost. The output is long, but I've included it below. Can anyone help me identify why boost still won't build? clang-linux.compile.c++.without-pth bin.v2\libs\atomic\build\clang-linux-android\release\link-static\target-os-android\threadapi-pthread\threading-multi\lockpool.o "C:/android/ndk/toolchains/llvm/prebuilt/windows-x86_64/bin/clang++" -c -x c++ -O3 -Wno-inline -Wall --sysroot=C:/android/ndk/platforms/android-15/arch-arm -isystem C:/android/ndk/sources/cxx-stl/llvm-libc++/include -isystem C:/android/ndk/sources/android/support/include -isystem C:/android/ndk/sources/cxx-stl/llvm-libc++abi/include -isystem C:/android/ndk/platforms/android-15/arch-arm/usr/include -std=c++14 -fno-strict-aliasing -D__GLIBC__ -DANDROID -D__ANDROID__ -D_GLIBCXX__PTHREADS -D__arm__ -D_REENTRANT -O2 -DNDEBUG -H -DBOOST_ALL_NO_LIB=1 -DBOOST_ATOMIC_SOURCE -DBOOST_ATOMIC_STATIC_LINK=1 -DNDEBUG -I"." -o "bin.v2\libs\atomic\build\clang-linux-android\release\link-static\target-os-android\threadapi-pthread\threading-multi\lockpool.o" "libs\atomic\src\lockpool.cpp" . C:/android/ndk/sources/cxx-stl/llvm-libc++/include\\cstddef .. C:/android/ndk/sources/cxx-stl/llvm-libc++/include\\__config .. C:\\android\\ndk\\toolchains\\llvm\\prebuilt\\windows-x86_64\\bin\\..\\lib64\\clang\\3.8.275480\\include\\stddef.h ... C:\\android\\ndk\\toolchains\\llvm\\prebuilt\\windows-x86_64\\bin\\..\\lib64\\clang\\3.8.275480\\include/__stddef_max_align_t.h .. C:/android/ndk/sources/cxx-stl/llvm-libc++/include\\__nullptr . .\\boost/config.hpp .. .\\boost/config/user.hpp .. .\\boost/config/select_compiler_config.hpp .. .\\boost/config/compiler/clang.hpp .. .\\boost/config/select_stdlib_config.hpp .. .\\boost/config/stdlib/libcpp.hpp .. .\\boost/config/select_platform_config.hpp .. .\\boost/config/platform/linux.hpp ... C:/android/ndk/sources/cxx-stl/llvm-libc++/include\\cstdlib .... C:/android/ndk/sources/cxx-stl/llvm-libc++/include\\stdlib.h ..... C:/android/ndk/sources/android/support/include\\stdlib.h ...... C:/android/ndk/platforms/android-15/arch-arm/usr/include\\stdlib.h ....... C:/android/ndk/platforms/android-15/arch-arm/usr/include\\sys/cdefs.h ........ C:/android/ndk/platforms/android-15/arch-arm/usr/include\\sys/cdefs_elf.h ........ C:/android/ndk/platforms/android-15/arch-arm/usr/include\\android/api-level.h ....... C:/android/ndk/sources/cxx-stl/llvm-libc++/include\\stddef.h ........ C:\\android\\ndk\\toolchains\\llvm\\prebuilt\\windows-x86_64\\bin\\..\\lib64\\clang\\3.8.275480\\include\\stddef.h ....... C:/android/ndk/sources/cxx-stl/llvm-libc++/include\\stddef.h ........ C:\\android\\ndk\\toolchains\\llvm\\prebuilt\\windows-x86_64\\bin\\..\\lib64\\clang\\3.8.275480\\include\\stddef.h ....... C:/android/ndk/sources/cxx-stl/llvm-libc++/include\\string.h ........ C:/android/ndk/sources/android/support/include\\string.h ......... C:/android/ndk/platforms/android-15/arch-arm/usr/include\\string.h .......... C:/android/ndk/sources/cxx-stl/llvm-libc++/include\\stddef.h .......... C:/android/ndk/platforms/android-15/arch-arm/usr/include\\malloc.h ........... C:/android/ndk/sources/cxx-stl/llvm-libc++/include\\stddef.h ....... C:/android/ndk/platforms/android-15/arch-arm/usr/include\\alloca.h ....... C:/android/ndk/platforms/android-15/arch-arm/usr/include\\strings.h ........ C:/android/ndk/platforms/android-15/arch-arm/usr/include\\sys/types.h ......... C:/android/ndk/sources/cxx-stl/llvm-libc++/include\\stddef.h .......... C:\\android\\ndk\\toolchains\\llvm\\prebuilt\\windows-x86_64\\bin\\..\\lib64\\clang\\3.8.275480\\include\\stddef.h ......... C:/android/ndk/sources/android/support/include\\stdint.h .......... C:/android/ndk/platforms/android-15/arch-arm/usr/include\\stdint.h ........... C:/android/ndk/sources/cxx-stl/llvm-libc++/include\\stddef.h ........... C:/android/ndk/platforms/android-15/arch-arm/usr/include\\sys/_types.h ............ C:/android/ndk/platforms/android-15/arch-arm/usr/include\\machine/_types.h ......... C:/android/ndk/platforms/android-15/arch-arm/usr/include\\linux/posix_types.h .......... C:/android/ndk/platforms/android-15/arch-arm/usr/include\\linux/stddef.h ........... C:/android/ndk/platforms/android-15/arch-arm/usr/include\\linux/compiler.h .......... C:/android/ndk/platforms/android-15/arch-arm/usr/include\\asm/posix_types.h ......... C:/android/ndk/platforms/android-15/arch-arm/usr/include\\asm/types.h ......... C:/android/ndk/platforms/android-15/arch-arm/usr/include\\linux/types.h ......... C:/android/ndk/platforms/android-15/arch-arm/usr/include\\machine/kernel.h ......... C:/android/ndk/platforms/android-15/arch-arm/usr/include\\sys/sysmacros.h ....... C:/android/ndk/platforms/android-15/arch-arm/usr/include\\memory.h In file included from libs\atomic\src\lockpool.cpp:16: In file included from .\boost/config.hpp:57: In file included from .\boost/config/platform/linux.hpp:15: C:/android/ndk/sources/cxx-stl/llvm-libc++/include\cstdlib:108:9: error: no member named 'strtold' in the global namespace using ::strtold; ~~^ C:/android/ndk/sources/cxx-stl/llvm-libc++/include\cstdlib:126:9: error: no member named '_Exit' in the global namespace using ::_Exit; ~~^ C:/android/ndk/sources/cxx-stl/llvm-libc++/include\cstdlib:149:9: error: no member named 'at_quick_exit' in the global namespace using ::at_quick_exit; ~~^ C:/android/ndk/sources/cxx-stl/llvm-libc++/include\cstdlib:150:9: error: no member named 'quick_exit' in the global namespace using ::quick_exit; ~~^

2 1