- Boost - lists.boost.org

Designing a multi-threaded file parser
by Aaron Boxer 30 Apr '16

30 Apr '16

Hello, I hope this is not off-topic, but I am interested in hearing about boost classes that will help me design the following: 1) I am designing a multi-threaded file parser for C++11 threads. The file format I am interested in parsing is a tagged format, with some tags requiring further processing, and others not. My working model right now is to have one thread through the file, and if it hits a tag that requires further processing, to push the file offset of this tag into a queue, to be processed by a thread pool. The read thread will spend a lot of time waiting for file IO to complete. Would it be faster to us the ASIO classes to do the reading? 2) I would like to design the following thread scheduling library to help process the tags: The library would have the following features: 1. smallest unit of work for a thread is a job 2. jobs have dependencies - job FOO cannot complete until job BAR has completed 3. threads are divided into groups 4. groups can be divided into bundles (a bundle can share the same processor affinity, for example) 5. groups are assigned to domains - domains contain jobs with similar characteristics, for example slow I/O activity 6. domains contain queues of jobs 7. threads in a group assigned to a domain can execute any jobs in any of the domain's queues Does boost support this type of thread scheduling framework? Can boost help me manage dependencies between jobs? Sorry for these broad design questions - just hoping for some insight into best design here. Many Thanks, Aaron

2 5

[RFC] CallableTraits: 47 traits and metafunctions for "callable" types
by Barrett Adair 29 Apr '16

29 Apr '16

Hello all, I'm working on a project called CallableTraits [1]. Is there any interest in adding this to Boost? I think it would work well as an extension to Boost.TypeTraits. >From the introduction section: CallableTraits is a C++11/14/17 header-only library for the inspection, synthesis, and decomposition of callable types. Whether it's calling conventions, const volatile &&, or container-like manipulation of parameter lists, CallableTraits provides all the tools you need to completely rid your codebase of function type specializations. This project aims to be the "complete type manipulation facility for function types" mentioned in the last section of C++17 proposal p0172 [2]. CallableTraits currently supports GCC 4.8 and later, Clang 3.5 and later, AppleClang from XCode 6.3 and later, and MSVC 19. Much of the remaining documentation work lies in the reference sections. Many examples are provided, but many more are planned. Test coverage is good, but insufficient. Daily improvements are being made to address these issues. The headers amount to approximately 4200 lines of code, while the tests and examples tally around 6000 lines (excluding comments and blank lines). An "issues" list is maintained on GitHub [3]. Barrett [1]: http://badair.github.io/callable_traits/doc/html/index.html [2]: http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2015/p0172r0.html [3]: https://github.com/badair/callable_traits/issues

2 2

Re: [boost] New Lib "Beast", HTTP + WebSocket protocols
by Vinnie Falco 29 Apr '16

29 Apr '16

On Fri, Apr 22, 2016 at 8:13 PM, David Stone <david(a)doublewise.net> wrote: > How does this compare to www.zaphoyd.com/websocketpp We have experience with websocketpp as it is the solution we have been using in our open source peer to peer software (http://github.com/ripple/rippled) for the last five years. It is that experience that drove the development of this new library as its replacement. Since the WebSocket protocol starts with a HTTP handshake, we also developed Beast.HTTP as an added bonus - think of it as a double feature at the drive-in. Beast.WebSocket has these things going for it: * Construct a stream from a socket or ssl::stream that's already connected or accepted: boost::asio::ip::tcp::socket sock(ios); websocket::stream<boost::asio::ip::tcp::socket> ws(std::move(sock)); * Accept a WebSocket handshake request with bytes that have already been read from the socket: template<class ConstBufferSequence> void websocket::stream::accept(ConstBufferSequence const& buffers); * Accept a WebSocket HTTP request handshake that was already parsed (e.g. from Beast.HTTP): template<class Body, class Headers> void websocket::stream::accept(http::request<Body, Headers> const& request); * Wraps any object meeting these type requirements, use your own type if you want: websocket::stream<YourTypeHere> ws(...); The type requirements (sync, async, or both): http://www.boost.org/doc/libs/1_60_0/doc/html/boost_asio/reference/SyncRead… http://www.boost.org/doc/libs/1_60_0/doc/html/boost_asio/reference/SyncWrit… http://www.boost.org/doc/libs/1_60_0/doc/html/boost_asio/reference/AsyncRea… http://www.boost.org/doc/libs/1_60_0/doc/html/boost_asio/reference/AsyncWri… * Receive functions accept any object meeting the requirements of Streambuf, permitting custom memory management strategies including the use of statically sized buffers. For example games using messages of fixed length. Streambuf type requirements: http://vinniefalco.github.io/beast/beast/types/Streambuf.html * In the Asio doc, Christopher Kohlhoff alludes to alternate implementation strategies for boost::asio::streambuf using multiple discontiguous buffers of varying sizes. We fulfill this idea with an optimized stream buffer that follows the strategy he described, providing a useful building block for sending and receiving variable size data. See: https://github.com/vinniefalco/Beast/blob/master/include/beast/basic_stream… * Interfaces are designed to closely resemble Asio in every possible way. This includes support for the extensible asynchronous model - support for std::future and stackful/stackless coroutines right out of the box. The library is designed to eliminate the learning curve for those familiar with Asio: extern void handle_read(boost::system::error_code ec); websocket::stream<boost::asio::ip::tcp::socket> ws(ios); ... websocket::opcode op; beast::streambuf sb; ws.async_read(op, sb, std::bind(&handle_read, std::placeholders::_1)); * All functions are template driven so everything is transparent to the compiler, including completion handlers passed to asynchronous initiation functions and objects used as buffers. By design, Beast.WebSocket avoids doing these things: * Managing the io_service * Managing a listening socket * Managing threads or a thread pool * Requiring the use of exceptions in certain places * Imposing a particular message buffering strategy * Using std::function for asynchronous callbacks * Requiring callers to know the size of the message in advance. The last point above is especially important for applications that produce data incrementally. For example, a huge JSON reply that comes from a database query. Planned improvements for Beast.WebSocket include: * Explicit proxy support with basic authentication * Tools for parsing subprotocols and message headers * permessage compression: https://tools.ietf.org/html/draft-ietf-hybi-permessage-compression-28 Years of experience working with Asio and and WebSockets have gone into the development of this library. It tries to be a natural extension of Asio, in a manner that is as narrow and lean as possible to provide maximum flexibility and performance.

2 3

New Lib "Beast", HTTP + WebSocket protocols
by Vinnie Falco 28 Apr '16

28 Apr '16

"Beast" This is a new library that implements the HTTP and WebSocket protocols using Boost.Asio and a few other Boost tidbits. Here's a quick synopsis of what's in it: http::message: - class template modeling the HTTP message with customization points to satisfy even the most tough-minded reviewer. http::read http::write http::async_read http::async_write - Free functions to easily send and receive HTTP messages. Modeled to work exactly like boost::asio free functions. Role-agnostic, build clients or servers! websocket::stream - class template wrapping any asio sync or async stream. Implements the WebSocket protocol including the initial handshake. Examples: - websocket echo servers - HTTP servers for simple websites (no cgi) "I've heard enough, show me example code!" http://vinniefalco.github.io/beast/beast/intro/example.html Main page, with links to GitHub repository, HTML documentation and benchmarks: http://vinniefalco.github.io/ Feedback welcome, the author checks email and Issues on the GitHub repository. This project has been submitted to the Boost incubator.

2 2

CMake - one more time
by Robert Ramey 28 Apr '16

28 Apr '16

Inspired by the recent discussion regarding CMake, I've spend some more time looking at this. As a result, I've done the following. a) I experimented with using CMake in the more customary manner in my canonical project Safe Numerics - (see Boost Library Incubator). The current version does distribute CMakeLists.txt around the subdirectories of the library root. My original objections to doing this were assuaged when I realized that boost build does the same thing. That is I typically have Jamfiles in side of test, example and sometimes other directories. b) One of the features that I much like about Boost Build is the possibility of arbitrary nesting of Jamfiles. This permits one to build/test a portion of one's library (e.g. test, example) as well as the whole library. For a large library this is indispensable. In spite of what many people think, this is not easily supported with CMake. However, I have found that with some extra effort, it is possible to support this to the extent we need it. So in this project ONE CAN BUILD (actually create an IDE project or makefile that builds) ANY OF THE "SUBPROJECTS" (TEST, EXAMPLE) AS WELL AS THE LIBRARY "SUPERPROJECT" c) CMake is indispensable to me: * it creates IDE projects for "any?" platform. I use IDE alot. * everyone else uses it - so it makes it easier to promote my work to others. * it's easier to make work - still a pain though * There is lots of information, around the net about how to use CMake, how easy it is etc. Although they help when you're looking for an answer (which is all the time) - they really betray how complex, arbitrary and complex the system is. * It has an almost of idiot proof GUI version which I use a lot. I really like this. * CMake is well maintained and supported by it's promoters. d) Boost Build * either just works (great!) or doesn't. If it doesn't it's almost impossible fix without getting help * I've never run across anyone outside of boost who uses it. It makes it harder to promote my work. * It's natural to compose projects into "super projects" * it's almost impossible to integrate with and IDE. At one time, I had things setup so I could debug executables created with boost build with the Visual Studio IDE and debugger. But it was just too fragile and time consuming to keep everything in sync. * it has a lot of "automatic" behavior which can be really, really confusion. A small example: you've got multiple compilers on your system. When it discovers this, it just picks the "best" one and you don't know which one you got until the project builds (or not). I'm sure this was implemented this way to make usage of boost build "simple" but it has the opposite effect. Much better would be fail immediately with a message "multiple compilers found:... use toolset=<compiler name> to select desired one." Some Conclusions - I'm trying to not make this a rant a) The ideal, platform independent build system does not yet exist. I guessing it never will. I'm sure it won't happen in my life time - but then I'm 68 - maybe you'll get lucky. b) Both systems are much more fragile, complicated and opaque than their promoters try to make you believe. It's not that they're lying, they truely believe in their own stuff. There is much re-invention of the wheel - The each created their own (half-assed) little language for goodness sake!!! c) Neither has really resolved the issue of nested projects in a clear way. Boost Build probably does or can do this. CMake has a system of "packages" and a whole 'nuther layer about "finding" them. Assuming it can be made to work - given the amount of time I've invested in CMake, I should know how to do this by now. d) I think it's time for Boost to be a little more open about tolerating/encouraging alternative build systems. I think our documentation approach is a model. Yeah it's a hodgepodge. But the various ways of doing pretty much work and generally don't stop working and we don't have to constantly spend effort bringing things up to the latest greatest system (which we couldn't agree upon anyway). We have libraries which have been going strong 15 years - and people can still read the docs. e) We should find some way to recognize those who have made the system work as well it has. Doug Gregor (boost book), Eric Niebler, Joel Guzman (quickbook). Vladimir Prus, Rene Rivera, Steve Watanabe. I know there others but these come to mind immediately. Note that I have only addressed the issue of library development which is my main interest. I'm really not seeing this issues related to users of libraries. In particular, CMake has the whole "find" thing which I'm still not even seeing the need for. If I want to use a library, I can build the library and move it to a common place with the headers, specify the include directory and I'm on my way. I'm sure someone will step up to enlighten me on this. Robert Ramey

15 59

[1.61.0] Master branch is closed
by Vladimir Prus 27 Apr '16

27 Apr '16

Boosters, The master branch is now closed for all changes, except by permission from a release manager. Per schedule, the branch will be totally frozen on Saturday. Thanks, The Release Managers

11 51

Test report generation stalled? Re: [1.61.0] Master branch is closed
by Rene Rivera 26 Apr '16

26 Apr '16

On Tue, Apr 26, 2016 at 2:36 PM, Thomas Trummer <th.trummer(a)gmail.com> wrote: > > > On 26 Apr 2016, at 19:47, Rene Rivera <grafikrobot(a)gmail.com> wrote: > > > > Thanks for the info.. But given that we don't have *any* test results > with > > MPI passing (i.e. nothing on develop yet).. I'm afraid it's going to stay > > broken this release. > > Is the test matrix actually working? There should be more recent results > than April 21… > Grrr.. Looks like report generation stopped on the 21st :-( -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail

2 3

modular-boost doc ?
by alainm 26 Apr '16

26 Apr '16

Hi, Would it be possible to have a pointer to the documentation related with the modular boost git mess under the "development" item of the boost main page ? it seems at least as important as some stuff like "Coding Guidelines for Integral Constant Expressions". Thanks Alain

5 6

Re: [boost] New Lib "Beast", HTTP + WebSocket protocols
by Bjorn Reese 24 Apr '16

24 Apr '16

On 04/22/2016 07:01 PM, Vinnie Falco wrote: > Main page, with links to GitHub repository, HTML documentation and benchmarks: > http://vinniefalco.github.io/ > > Feedback welcome, the author checks email and Issues on the GitHub > repository. This project has been submitted to the Boost incubator. This looks like a nice library. You have clearly encountered the same pain points as I while working with other HTTP-based libraries, and you seem to have come up with solutions similar to those found in the Boost.Http project [1]. It may be a good idea if we joined the two projects together. Would you consider such an option? Last year, the Boost.Http project went through a formal Boost review. The summary [2] shows what the Boost community expects from a HTTP library. In order to address one of these expectations, we have a Boost Summer of Code project this year for a HTTP parser. [1] https://github.com/BoostGSoC14/boost.http [2] http://lists.boost.org/boost-announce/2015/08/0452.php

3 4

[boot] using the boost name on unofficial libraries
by Sam Kellett 23 Apr '16

23 Apr '16

i see a *lot* of rejected / unfinished libraries that use the boost name with the intentions of one day becoming a boost library. doesn't this affect the brand strength associated with the boost name? i would have thought that it wouldn't be ok to name your library a boost library before it is officially accepted, but that doesn't seem to be the case? or is it the case but it isn't / can't be enforced?

6 7