- Boost - lists.boost.org

[beast] Formal review
by Bjorn Reese 14 Jul '17

14 Jul '17

I am going to focus exclusively on the HTTP API in the review below because that needs most attention. Furthermore, to keep this review within reasonable bounds, I am omitting issues already reported by others. I. DESIGN --------- Beast offers a well-designed WebSocket API along with a minimal and deficient HTTP API. Beast claims to offer low-level building-blocks that can be used to produce higher-level HTTP server and client libraries. The low-level approach makes Beast minimal. I do not have a problem with a minimal API, other than it diminishes the usefulness of the library. Although this can be solved by adding more functionality over time, if I was to evaluate Beast with its current feature-level, I would only choose it if I needed WebSocket. This, however, is not a precondition for library acceptance. But the low-level approach also means the abstractions must be right. The abstractions that Beast currently offers are deficient, which is best shown by the poor support for HTTP chunking. Chunking can and should be represented more naturally by a HTTP API. HTTP chunking splits a message into several parts, where each part can contain metadata (chunk extensions) and data (chunk.) The metadata is a set of key-value pairs, just like the HTTP headers. During the review period I have described several use cases for HTTP chunking that does require metadata (e.g. radio metadata,) as well as the preservation of chunk boundaries (e.g. event-based publish-subscribe channels.) Conceptually, a normal HTTP message has the following structure: message = start-line metadata* body? A chunked HTTP message can be described in a similar manner as: message = start-line metadata* ( metadata* chunk? )* The Beast message container is designed for the structure of the normal HTTP message, and assumes that chunks are appended the body. The chunk metadata does not fit into this model, and therefore Beast discards them by default. The repetitive structure of chunked messages is not accommodated by the Beast message container. This means that we have to make our own chunk decorator to generate chunk metadata, and our own custom parser to receive chunk metadata. The chunk decorator inserts metadata via a callback mechanism. Its design is poor for two reasons. First, the user must keep track of context to ensure that the correct metadata is applied to the correct chunks. Second, the user must write the metadata in correct HTTP syntax. The latter may sound trivial, but does anybody here know if " ;key=value\r\n" is correctly formatted? (notice the initial whitespace.) The answer is that it is correct according to RFC 2616, incorrect according to RFC 7230, and correct according to RFC 7230 errata. The best approach for handling this situation is to never insert whitespaces, but to always accept whitespaces from others. Users should not have to be bothered with such detail. Receiving chunk metadata rather than having them discarded by default requires a custom parser. The Beast message parser is a push parser (SAX) where callbacks are invoked for each parsed message part. This means that the user must keep track of which metadata belongs to which chunk. The user must also parse the metadata manually. Beast does not even check if the syntax is correct. The custom parser is also necessary if the user needs to receive individual chunks rather than having them appended to the body. I would have expected that a low-level API returns the next message part as soon as it is identified. The current collect-entire-message design is a higher-level design. In summary, 1. Beast should not discard anything by default, but should faithfully forward all information received. 2. Beast should handle all low-level syntax and not leave parts of it to the user. 3. Beast should not break the end-to-end principle unless allowed by the user. As a way forward I am going to sketch another simple message model that does encompass HTTP chunking. Conceptually it looks like this (I am going to ignore that, say, metadata may require different syntax depending on context) message = start-line section* section = metadata* body-part? Building an API around this model should offer read/write functions that handles partial bodies (whether chunked or progressively downloaded.) The current collect-entire-message API could be build on top of that. With this approach there is no need to mess around with custom parsers. This design does require state information to be able to select the proper syntax for both metadata and data. On another topic, the HTTP status codes can be roughly categorized as instructions (the 1xx and 2xx series) and errors (the 4xx and 5xx series.) I recommend that you create two error categories and pass status codes as error_code. Notice that I have no comment about whether or not they should be stored in the message container. What I really want to achieve is to enable the user to easily propagate HTTP errors in the same way that non-HTTP errors (including beast::http::error) are propagated. II. IMPLEMENTATION ------------------ I read the implementation to understand the design, so I did not look for QA issues. My overall impression is that much of the code is well-written, but there are parts that are convoluted and difficult to verify. There are too many goto statements for my taste; I do not mind the goto pattern used in the parser, but the state-machines in read_some_op and serializer looks more like micro-optimization. Furthermore, "case do_body + 1:" looks like an awful hack. The dynamic buffers are useful even by themselves. The code contains several foreign files with different licensing conditions: * include/beast/core/detail/base64.hpp * include/beast/zlib/zlib.hpp * include/beast/websocket/detail/utf8_checker.hpp * include/beast/http/detail/basic_parser.hpp All these include both the BSL and the original license. This should be checked by the Boost legal council. III. DOCUMENTATION ------------------ The documentation is generally well-written (and it looks like the DynamicBuffer section got a little help from Networking TS.) The Reference is missing some concepts, e.g. SyncReadStream. IV. NAMING ---------- The library name must change. "Boost.Beast" sounds like a bad joke. I do not have strong opinions about the new name. V. VERDICT ---------- I recommend to REJECT the library because the HTTP API provides the wrong low-level abstractions as explained in the design section above.

12 46

Follow-up to the follow-up of the CMake follow-up
by Peter Dimov 13 Jul '17

13 Jul '17

In branch feature/install-cmake-config-2 of the superproject, you can find an improved version of the award-winning work presented in my previous follow-up. This one integrates into the top-level "b2 install" and "b2 stage" and installs (or, respectively, stages) the CMake configuration files necessary for find_package to work. The instructions for trying it out are the same as last time: git clone --recursive --depth 64 -b feature/install-cmake-config-2 https://githib.com/boostorg/boost bootstrap b2 headers b2 install then add the installation prefix to CMAKE_PREFIX_PATH. An example CMakeLists.txt could look like this: cmake_minimum_required(VERSION 3.5) project(cmake_demo CXX) list(APPEND CMAKE_PREFIX_PATH C:/Boost) find_package(boost_filesystem 1.63.0 REQUIRED) find_package(boost_program_options 1.63.0 REQUIRED) # disable autolink add_definitions(-DBOOST_ALL_NO_LIB=1) # sample program add_executable(cmake_demo cmake_demo.cpp) target_link_libraries(cmake_demo boost::filesystem boost::program_options)

4 8

[review][beast] Review of Beast starts today : July 1 - July 10
by Michael Caisse 13 Jul '17

13 Jul '17

The formal review of Vinnie Falco’s Beast library will take place from July 1 through July 10, 2017. Please consider participating in this review. The success of Boost is partially a result of the quality review process which is conducted by the community. You are part of the Boost community. I will be grateful to receive a review based on whatever level of effort or time you can devote. Beast is a C++ header-only library serving as a foundation for writing interoperable networking libraries by providing low-level HTTP/1, WebSocket, and networking protocol vocabulary types and algorithms using the consistent asynchronous model of Boost.Asio. Beast is designed for: * Symmetry: Algorithms are role-agnostic; build clients, servers, or both. * Ease of Use: Boost.Asio users will immediately understand Beast. * Flexibility: Users make the important decisions such as buffer or thread management. * Performance: Build applications handling thousands of connections or more. * Basis for Further Abstraction: Components are well-suited for building upon. A branch has been made for the review. You can find Beast for review at these links: * Documentation : <http://vinniefalco.github.io/stage/beast/review/> * Source code : <https://github.com/vinniefalco/Beast/tree/review> * The FAQ answers real questions that come up : <http://vinniefalco.github.io/stage/beast/review/beast/design_choices/faq.ht…> A lightning talk from CppCon 2016 introducing Beast can be found here: <https://www.youtube.com/watch?v=uJZgRcvPFwI> Please provide in your review whatever information you think is valuable to understand your final choice of ACCEPT or REJECT including Beast as a Boost library. Please be explicit about your decision. Some other questions you might want to consider answering: - What is your evaluation of the design? - What is your evaluation of the implementation? - What is your evaluation of the documentation? - What is your evaluation of the potential usefulness of the library? - Did you try to use the library? With which compiler(s)? Did you have any problems? - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? - Are you knowledgeable about the problem domain? More information about the Boost Formal Review Process can be found here: <http://www.boost.org/community/reviews.html> Thank you for your effort in the Boost community. Happy coding - michael -- Michael Caisse Ciere Consulting ciere.com

24 107

Master branch now open for merging
by Marshall Clow 13 Jul '17

13 Jul '17

I will be uploading 1.65.0 beta1 this morning. Hint: It's the same as rc2 ;-) -- Marshall

2 1

[fiber] PR: Replace #include "Windows.h" by "windows.h"
by Frédéric Bron 13 Jul '17

13 Jul '17

Hi, A #include "Windows.h" causes troubles when cross-compiling on Linux in C++14 mode with MinGW-W64 with Posix threads. Can we change it to "windows.h"? Here is a PR: https://github.com/boostorg/fiber/pull/134 Thanks, Frédéric

2 2

[Boost.Range] Documentation
by degski 12 Jul '17

12 Jul '17

I had a need to try and understand Boost.Range. I don't know what to say, did this documentation pass a review? Introduction first "example": Example - Iterate over the values in a map using namespace boost; using namespace boost::adaptors; for_each( my_map | map_values, fn ); 3 lines, out of which 2 'using' lines, that make this "example" even less informative than it might have been without them, in wich case I would at least have known in which namespace(s) these things are living. Ah, wait, there's also an example section, so I'll copy the first example <http://www.boost.org/doc/libs/1_64_0/libs/range/test/string.cpp> in here: // Boost.Range library // // Copyright Thorsten Ottosen 2003-2004. Use, modification and // distribution is subject to the Boost Software License, Version // 1.0. (See accompanying file LICENSE_1_0.txt or copy at // http://www.boost.org/LICENSE_1_0.txt) // // For more information, see http://www.boost.org/libs/range/ <http://www.boost.org/doc/libs/1_64_0/libs/range/> // //#define _MSL_USING_NAMESPACE 1 #include <boost/detail/workaround.hpp <http://www.boost.org/doc/libs/1_64_0/boost/detail/workaround.hpp>> #if BOOST_WORKAROUND(__BORLANDC__, BOOST_TESTED_AT(0x564)) # pragma warn -8091 // suppress warning in Boost.Test # pragma warn -8057 // unused argument argc/argv in Boost.Test #endif #include <boost/array.hpp <http://www.boost.org/doc/libs/1_64_0/boost/array.hpp>> #include <boost/range/as_array.hpp <http://www.boost.org/doc/libs/1_64_0/boost/range/as_array.hpp>> #include <boost/range/as_literal.hpp <http://www.boost.org/doc/libs/1_64_0/boost/range/as_literal.hpp>> #include <boost/range/functions.hpp <http://www.boost.org/doc/libs/1_64_0/boost/range/functions.hpp>> #include <boost/range/metafunctions.hpp <http://www.boost.org/doc/libs/1_64_0/boost/range/metafunctions.hpp>> #include <boost/static_assert.hpp <http://www.boost.org/doc/libs/1_64_0/boost/static_assert.hpp>> #include <boost/type_traits.hpp <http://www.boost.org/doc/libs/1_64_0/boost/type_traits.hpp>> #include <boost/test/test_tools.hpp <http://www.boost.org/doc/libs/1_64_0/boost/test/test_tools.hpp>> #include <boost/config.hpp <http://www.boost.org/doc/libs/1_64_0/boost/config.hpp>> #include <vector> #include <fstream> #include <algorithm> namespace { template< class CharT, std::size_t Length > class test_string { public: typedef CharT value_type; typedef value_type* pointer; typedef const value_type* const_pointer; typedef std::size_t size_type; typedef value_type array_t[Length]; typedef const value_type const_array_t[Length]; explicit test_string(const CharT literal_sz[]) { std::copy(literal_sz, literal_sz + Length, m_buffer.data()); m_buffer[Length] = value_type(); } const_pointer const_sz() const { return m_buffer.data(); } pointer mutable_sz() { return m_buffer.data(); } private: typedef boost::array<value_type, Length + 1> buffer_t; buffer_t m_buffer; }; } template< class T > inline BOOST_DEDUCED_TYPENAME boost::range_iterator<T>::type str_begin( T& r ) { return boost::begin( boost::as_literal(r) ); } template< class T > inline BOOST_DEDUCED_TYPENAME boost::range_iterator<T>::type str_end( T& r ) { return boost::end( boost::as_literal(r) ); } template< class T > inline BOOST_DEDUCED_TYPENAME boost::range_difference<T>::type str_size( const T& r ) { return boost::size( boost::as_literal(r) ); } template< class T > inline bool str_empty( T& r ) { return boost::empty( boost::as_literal(r) ); } template< typename Container, typename T > BOOST_DEDUCED_TYPENAME boost::range_iterator<Container>::type find( Container& c, T value ) { return std::find( str_begin(c), str_end(c), value ); } template< typename Container, typename T > BOOST_DEDUCED_TYPENAME boost::range_iterator<const Container>::type find( const Container& c, T value ) { return std::find( str_begin(c), str_end(c), value ); } template< typename Container, typename T > BOOST_DEDUCED_TYPENAME boost::range_iterator<Container>::type find_mutable( Container& c, T value ) { str_size( c ); return std::find( str_begin(c), str_end(c), value ); } template< typename Container, typename T > BOOST_DEDUCED_TYPENAME boost::range_iterator<const Container>::type find_const( const Container& c, T value ) { str_size( c ); return std::find( str_begin(c), str_end(c), value ); } std::vector<char> check_rvalue_return() { return std::vector<char>( 10, 'm' ); } using namespace boost; void check_char() { typedef boost::range_difference<std::string>::type diff_t; typedef char* char_iterator_t; typedef char char_array_t[10]; test_string<char, 8> a_string("a string"); test_string<char, 14> another_string("another string"); const char* char_s = a_string.const_sz(); char my_string[] = "another_string"; const char my_const_string[] = "another string"; const diff_t my_string_length = 14; char* char_s2 = a_string.mutable_sz(); BOOST_STATIC_ASSERT(( is_same< range_value<char_iterator_t>::type, detail::iterator_traits<char_iterator_t>::value_type>::value )); BOOST_STATIC_ASSERT(( is_same< range_iterator<char_iterator_t>::type, char_iterator_t >::value )); BOOST_STATIC_ASSERT(( is_same< range_difference<char_iterator_t>::type, ::std::ptrdiff_t >::value )); BOOST_STATIC_ASSERT(( is_same< range_size<char_iterator_t>::type, std::size_t >::value )); BOOST_STATIC_ASSERT(( is_same< range_iterator<char_iterator_t>::type, char_iterator_t >::value )); BOOST_STATIC_ASSERT(( is_same< range_iterator<const char*>::type, const char* >::value )); BOOST_STATIC_ASSERT(( is_same< range_value<char_array_t>::type, char>::value )); BOOST_STATIC_ASSERT(( is_same< range_iterator<char_array_t>::type, char* >::value )); BOOST_STATIC_ASSERT(( is_same< range_iterator<const char_array_t>::type, const char* >::value )); BOOST_STATIC_ASSERT(( is_same< range_difference<char_array_t>::type, ::std::ptrdiff_t >::value )); BOOST_STATIC_ASSERT(( is_same< range_size<char_array_t>::type, std::size_t >::value )); BOOST_STATIC_ASSERT(( is_same< range_iterator<char_array_t>::type, char* >::value )); BOOST_STATIC_ASSERT(( is_same< range_iterator<const char_array_t>::type, const char* >::value )); BOOST_CHECK_EQUAL( str_begin( char_s ), char_s ); const diff_t sz = str_size(char_s); const char* str_end1 = str_begin( char_s ) + sz; BOOST_CHECK_EQUAL( str_end( char_s ), str_end1 ); BOOST_CHECK_EQUAL( str_empty( char_s ), (char_s == 0 || char_s[0] == char()) ); BOOST_CHECK_EQUAL( sz, static_cast<diff_t>(std::char_traits<char>::length(char_s)) ); BOOST_CHECK_EQUAL( str_begin( my_string ), my_string ); range_iterator<char_array_t>::type str_end2 = str_begin( my_string ) + str_size(my_string); range_iterator<char_array_t>::type str_end3 = str_end(my_string); BOOST_CHECK_EQUAL( str_end3, str_end2 ); BOOST_CHECK_EQUAL( str_empty( my_string ), (my_string == 0 || my_string[0] == char()) ); BOOST_CHECK_EQUAL( str_size( my_string ), my_string_length ); BOOST_CHECK_EQUAL( str_size( my_string ), static_cast<diff_t>(std::char_traits<char>::length(my_string)) ); char to_search = 'n'; BOOST_CHECK( find_mutable( char_s, to_search ) != str_end( char_s ) ); BOOST_CHECK( find_const( char_s, to_search ) != str_end(char_s) ); BOOST_CHECK( find_mutable( my_string, to_search ) != str_end(my_string) ); BOOST_CHECK( find_const( my_string, to_search ) != str_end(my_string) ); BOOST_CHECK( find_mutable( char_s2, to_search ) != str_end(char_s) ); BOOST_CHECK( find_const( char_s2, to_search ) != str_end(char_s2) ); BOOST_CHECK( find_const( as_array( my_string ), to_search ) != str_end(my_string) ); BOOST_CHECK( find_const( as_array( my_const_string ), to_search ) != str_end(my_string) ); // // Test that as_literal() always scan for null terminator // char an_array[] = "foo\0bar"; BOOST_CHECK_EQUAL( str_begin( an_array ), an_array ); BOOST_CHECK_EQUAL( str_end( an_array ), an_array + 3 ); BOOST_CHECK_EQUAL( str_size( an_array ), 3 ); const char a_const_array[] = "foobar\0doh"; BOOST_CHECK_EQUAL( str_begin( a_const_array ), a_const_array ); BOOST_CHECK_EQUAL( str_end( a_const_array ), a_const_array + 6 ); BOOST_CHECK_EQUAL( str_size( a_const_array ), 6 ); } void check_string() { check_char(); #ifndef BOOST_NO_STD_WSTRING typedef wchar_t* wchar_iterator_t; test_string<wchar_t, 13> a_wide_string(L"a wide string"); test_string<wchar_t, 19> another_wide_string(L"another wide string"); const wchar_t* char_ws = a_wide_string.const_sz(); wchar_t my_wstring[] = L"another wide string"; wchar_t* char_ws2 = a_wide_string.mutable_sz(); BOOST_STATIC_ASSERT(( is_same< range_value<wchar_iterator_t>::type, detail::iterator_traits<wchar_iterator_t>::value_type>::value )); BOOST_STATIC_ASSERT(( is_same< range_iterator<wchar_iterator_t>::type, wchar_iterator_t >::value )); BOOST_STATIC_ASSERT(( is_same< range_iterator<const wchar_t*>::type, const wchar_t* >::value )); BOOST_STATIC_ASSERT(( is_same< range_difference<wchar_iterator_t>::type, detail::iterator_traits<wchar_iterator_t>::difference_type >::value )); BOOST_STATIC_ASSERT(( is_same< range_size<wchar_iterator_t>::type, std::size_t >::value )); BOOST_STATIC_ASSERT(( is_same< range_iterator<wchar_iterator_t>::type, wchar_iterator_t >::value )); BOOST_STATIC_ASSERT(( is_same< range_iterator<const wchar_t*>::type, const wchar_t* >::value )); typedef boost::range_difference<std::wstring>::type diff_t; const diff_t sz = str_size( char_ws ); BOOST_CHECK_EQUAL( str_begin( char_ws ), char_ws ); BOOST_CHECK_EQUAL( str_end(char_ws), (str_begin( char_ws ) + sz) ); BOOST_CHECK_EQUAL( str_empty( char_ws ), (char_ws == 0 || char_ws[0] == wchar_t()) ); BOOST_CHECK_EQUAL( sz, static_cast<diff_t>(std::char_traits<wchar_t>::length(char_ws)) ); wchar_t to_search = L'n'; BOOST_CHECK( find( char_ws, to_search ) != str_end(char_ws) ); BOOST_CHECK( find( char_ws2, to_search ) != str_end(char_ws2) ); #if BOOST_WORKAROUND(_MSC_VER, BOOST_TESTED_AT(1300)) BOOST_CHECK( find( my_wstring, to_search ) != str_end(my_wstring) ); #else boost::ignore_unused_variable_warning( my_wstring ); #endif #endif find( check_rvalue_return(), 'n' ); } #include <boost/test/unit_test.hpp <http://www.boost.org/doc/libs/1_64_0/boost/test/unit_test.hpp>> using boost::unit_test::test_suite; test_suite* init_unit_test_suite( int argc, char* argv[] ) { test_suite* test = BOOST_TEST_SUITE( "Range Test Suite" ); test->add( BOOST_TEST_CASE( &check_string ) ); return test; } Well, that obviously got me up and running, thanks. Is there any other documentation/examples to have a look at? degski -- "*Ihre sogenannte Religion wirkt bloß wie ein Opiat reizend, betäubend, Schmerzen aus Schwäche stillend.*" - Novalis 1798

2 1

[thread] Trying to cross build 1.64.0 boost.thread with mingw posix threads
by Frédéric Bron 12 Jul '17

12 Jul '17

Hi, I built a w64 cross-compiler to make windows binaries on linux. The cross-compiler has Posix threads, not win32 threads. I would like to build boost.thread with it but I have some difficulties: 1. I do not need to link with any library to build applications with C++11 threads but boost build asks for PTW32_INCLUDE and PTW32_LIB otherwise it sets <build>no so that the library is not built. This is easily changed with the following patch: diff -r -u boost_1_64_0-0/libs/thread/build/Jamfile.v2 boost_1_64_0-1/libs/thread/build/Jamfile.v2 --- boost_1_64_0-0/libs/thread/build/Jamfile.v2 2017-04-17 04:22:25.000000000 +0200 +++ boost_1_64_0-1/libs/thread/build/Jamfile.v2 2017-07-12 14:24:03.960979687 +0200 @@ -271,10 +271,6 @@ { result += $(paths) ; } - else - { - result = <build>no ; - } } result += <define>BOOST_THREAD_DONT_USE_CHRONO ; if ! [ configure.builds has_atomic_flag_lockfree 2. Once this patch is applied, the library is built but there are errors when compiling libs/thread/src/pthread/thread.cpp. The command that is run by b2 is given below. Here are the errors: libs/thread/src/pthread/thread.cpp:541:2: error: #error #error ^~~~~ In file included from ./boost/thread/thread_only.hpp:17:0, from libs/thread/src/pthread/thread.cpp:11: ./boost/thread/pthread/thread_data.hpp: In member function ‘void boost::thread_attributes::set_stack_size(std::size_t)’: ./boost/thread/pthread/thread_data.hpp:53:35: error: ‘getpagesize’ was not declared in this scope std::size_t page_size = getpagesize(); The first error is because apparently neither BOOST_HAS_SCHED_YIELD nor BOOST_HAS_PTHREAD_YIELD is defined. I understand that the 2nd error comes from the fact that getpagesize() is not in unistd.h provided by mingw-w64. There is a strange thing about the command line: it contains -mthreads although I never provided it. I tried to remove it or replace it with -pthread or -lptread but I still get the same error message. "i686-w64-mingw32-g++-7.1.0" -ftemplate-depth-128 -std=c++14 -O2 -DNDEBUG -O3 -finline-functions -Wno-inline -Wall -pedantic -march=i686 -mthreads -m32 -Wextra -Wno-long-long -Wno-unused-parameter -Wunused-function -pedantic -DBOOST_ALL_NO_LIB=1 -DBOOST_LOG_WITHOUT_EVENT_LOG -DB OOST_SYSTEM_STATIC_LINK=1 -DBOOST_THREAD_BUILD_LIB=1 -DBOOST_THREAD_DONT_USE_CHRONO -DBOOST_THREAD_POSIX -DBOOST_USE_WINDOWS_H -DNDEBUG -DWIN32_LEAN_AND_MEAN -I"." -c -o "bin.v2/libs/thread/build/gcc-mingw-7.1.0/release/boost.locale.iconv-on/boost.locale.icu-off/boost.locale.std-o ff/link-static/target-os-windows/threading-multi/pthread/thread.o" "libs/thread/src/pthread/thread.cpp"

1 0

[root_ptr] Deterministic Javascript-like memory manager
by Phil Bouchard 12 Jul '17

12 Jul '17

Hello Boost / Qt, After some experience I have gained in Javascript over the last year, I decided to integrate root_ptr in a quest to replace their garbage collector. So I did and I already have an example. If we mimic the Javascript function scope in C++ then we'll have the following: https://github.com/philippeb8/root_ptr/blob/develop/example/javascript_exam… The application was compiled with: g++ -fpermissive -DBOOST_DISABLE_THREADS -std=c++11 -I../include -I/usr/local/include -L/usr/local/lib -lboost_thread -lboost_system javascript_example1.cpp And it outputs: Scope 0: BEGIN Scope 1: BEGIN A::A(const boost::node_proxy&) A::A(const boost::node_proxy&) A::~A() Scope 1: END A::~A() Scope 0: END Which is exactly what we want. Note that I didn't stress tested it yet and in Brave New World we would be able to integrate it to other languages relatively easily given Javascript is a worse-case scenario. What do you think? Regards, -Phil http://www.fornux.com http://www.finitetheory.com https://www.actatabula.com https://github.com/philippeb8/root_ptr/

3 10

[beast] Chunked Serialization, feedback requested! (Or: Bjorn, will you come out to play?)
by Vinnie Falco 12 Jul '17

12 Jul '17

During the review it was brought up that Beast could do a better job giving users control over the serialization and parsing of message payloads that have the chunked transfer coding applied. I collaborated with some helpful folks on the #boost slack channel and I have put together something solid for serializing chunks. I'm looking for feedback from anyone (especially Bjorn) before I merge this. The documentation is quite extensive and explains step by step how to achieve all of the goals stated in the review related to serialization of chunk encoded message payloads: <http://vinniefalco.github.io/stage/beast/bjorn/beast/using_http/chunked_enc…> I'm interested in hearing from anyone who cares to weigh in. Is the documentation sufficient? Are there any obvious defects? Do the interfaces meet the requirements for obtaining full control over the emitted chunked message payload? If anyone wants to play with or inspect the code, it is in the "bjorn" branch: <https://github.com/vinniefalco/Beast/commits/bjorn> In the meanwhile, I will start putting something together to provide better control during parsing of chunks. Thanks!

1 0

My Beast Review
by Phil Endecott 12 Jul '17

12 Jul '17

Dear All, This review is not very thorough - I've only looked at the docs and some small parts of the code, and read the discussion on the list. I've not tried to use the library. But I think I've seen enough to express an opinion. I've never used ASIO. I've studied it in the past, and concluded that it offers portability to platforms that I don't use but doesn't work well with other platforms that I do use. It also seems more complicated than I would have liked, perhaps in order to achieve some optimised level of performance that I don't need (or perhaps for no good reason). I have however quite often used HTTP (hasn't everyone?) and I think it's an embarrassment that std::c++ can't download something from the 'net with one or two lines of code in the way that almost every other language can. So, my first reaction is that it's a disappointment to see that a proposed HTTP library is tied to ASIO. On further investigation, however, I think this is no great loss to me as the functionality that this library provides is so limited in scope. Parsing HTTP headers, for example, is not much work. Even my own hacked-together-in-an-afternoon HTTP classes do more than this - to mention just one thing, parsing and formatting of dates in the correct format. Everyone who uses this library will have to implement this sort of thing themselves on top of it. (And many of them will implement it wrongly.) Vinnie says that he hopes that more functionality will be layered on top of this work. The danger is that the person who comes along wanting to implement the other 75% will not like Vinnie's starting point, and re-implement it all in their own style. (I know I would.) Unless, of course, Vinnie is going to offer this "other 75%" himself eventually. *That would be great*. If that is the plan, my review is "this is a fine start but please remove the ASIO dependency and come back when it's finished". But I think Vinnie has said that the library is to be reviewed "as is". In that case, my review is "just add this to ASIO and call it boost.asio.http and boost.asio.websocket, no need for a review" (assuming that the ASIO developer is happy with that). Regarding what I have seen of the implementation: - Security must be one of the most important properties of an HTTP library, and I'd really hope to find that such a library had been implemented by someone with significant experience in this area. I never cease to be amazed by the ingenuity of attackers. - I fear that there may be much premature optimisation in the parser code that I've looked at. For example: static bool is_digit(char c) { return static_cast<unsigned char>(c-'0') < 10; } Is that right? Say c='!' (ASCII 33). '0' is ASCII 48. So c-'0' is -15. But you cast that to an unsigned char, so -15 = 241, and 241 > 10, so the function returns false. So it is correct - but (1) it needs a comment to explain it, (2) it's the sort of thing that makes me worry from a security viewpoint, and (3) is it actually an improvement on the more obvious c >= '0' && c <= '9'? Well I've done a quick test and I get the same assembler for both - so why bother? (Except on close inspection actually I don't get exactly the same assembler. To get the same assembler I have to replace the constant 10 in your code with 10U, which changes the final comparison from signed to unsigned. I don't think that matters, but it again makes me wonder whether this security-critical code is ideal.) But hey, why even do c >= '0' && c <= '9' when we have std::isdigit()? But it would be much slower, right? Wrong, it's actually one instruction shorter; somehow it manages to avoid one of the zero-extension instructions. So this is not a premature optimisation, it's a premature pesimisation. (My test code is at the end of this message.) Now, is_digit is just about the simplest thing in the parser; Vinnie has also implemented his own code for parsing numbers, and then we come to parse_field. Please have a look at this (around line 666 of basic_parser.hpp). Note that it calls find_fast() near the top of that file, which uses some Intel SIMD instruction to, I think, skip forward over the header field name, and then over the field value to the CRLF. Not that there are any comments in find_fast to explain what it does, for those of us who don't know what _mm_loadu_si128 and _mm_cmpestri do. Has this code actually been benchmarked against a much simpler and more obviously-correct version, and found to have a worthwhile performance benefit? (I believe that some of the discussion has used this "optimised parser" as a justification for not making the code more generic, i.e. to work with any iterator pairs. I believe that to be flawed, since (a) I don't think this optimisation is useful, and (b) even if it were, the code could use it for any contiguous iterators, and fall back to "unoptimised" code in other cases. I think that having the "unoptimised" code present would be useful for documenting what the parser is trying to do, in any case.) Could it be that the reason why the scope of the library seems so limited is that Vinnie has spent his time fiddling with this optimisation, rather than adding useful features? My opinion: having looked at the implementation, reject. Test for is_digit: #include <cctype> bool is_digit_1(char c) { return static_cast<unsigned char>(c-'0') < 10U; } bool is_digit_2(char c) { return c >= '0' && c <= '9'; } bool is_digit_3(char c) { return std::isdigit(c); } Aarch64 assembler output from g++ 6.3.0 -O3, with the uninteresting bits removed: _Z10is_digit_1c: uxtb w0, w0 sub w0, w0, #48 uxtb w0, w0 cmp w0, 9 cset w0, ls ret _Z10is_digit_2c: uxtb w0, w0 sub w0, w0, #48 uxtb w0, w0 cmp w0, 9 cset w0, ls ret _Z10is_digit_3c: uxtb w0, w0 sub w0, w0, #48 cmp w0, 9 cset w0, ls ret Regards, Phil.

6 7