- Boost - lists.boost.org

[contract] differentiate regressions with libc++
by Lorenzo Caminiti 31 Oct '17

31 Oct '17

Hello all, Another question on the regression tests. How am I supposed to differentiate GCC 6.0 between CrystaX-apilevel-19-armeabi-v7a-gnu-libstdc++ and CrystaX-apilevel-19-armeabi-v7a-hard-llvm-libc++? http://www.boost.org/development/tests/develop/developer/contract.html They both use the same toolset name "gcc-gnu-6", but when libc++ is used instead of libstdc++ Boost.Contract tests give these linker errors: ../boost/smart_ptr/detail/sp_counted_impl.hpp:146: error: undefined reference to 'operator delete(void*, unsigned int)' /home/cislave/.crystax/ci-boost/regression/android-19-armeabi-v7a-hard-llvm-develop/tmp/crystax-ndk-10.3.2/sources/cxx-stl/llvm-libc++/3.6/libcxx/include/sstream:189: error: undefined reference to 'operator delete(void*, unsigned int)' ../boost/iostreams/detail/streambuf/indirect_streambuf.hpp:50: error: undefined reference to 'operator delete(void*, unsigned int)' /home/cislave/.crystax/ci-boost/regression/android-19-armeabi-v7a-hard-llvm-develop/tmp/crystax-ndk-10.3.2/sources/cxx-stl/llvm-libc++/3.6/libcxx/include/sstream:740: error: undefined reference to 'operator delete(void*, unsigned int)' collect2: error: ld returned 1 exit status I wanted to mark the link errors for libc++ as expected failures while leaving libstdc++ unmarked (because there are no link errors for libstdc++). However, I don't know how to do that because the toolset name is the same "gcc-gnu-6" for both runs with libstdc++ (column 6) and with libc++ (column 10). Should the toolset names be unique instead? For example, should names like gcc-gnu-6~libstdc++ and gcc-gnu-6~libc++ have been used in this case instead of using just gcc-gnu-6 for both runs? Thank you. --Lorenzo

1 1

fuzzing boost at OSS-Fuzz
by Kostya Serebryany 31 Oct '17

31 Oct '17

Hi, I would like to invite boost developers to use OSS-Fuzz, a continuous automated fuzzing service. https://github.com/google/oss-fuzz https://opensource.googleblog.com/2017/05/oss-fuzz-five-months-later-and.ht… I've made the initial set up that fuzzes boost::regex and it found 8 bugs there, see https://svn.boost.org/trac10/ticket/12818#comment:7 3460 boost: Integer-overflow in boost::re_detail_NUMBER::basic_regex_parser... 3464 boost: Integer-overflow in boost::re_detail_NUMBER::perl_matcher... 3469 boost: ASSERT: jmp->type == syntax_element_jump 3471 boost: Stack-overflow in boost::re_detail_NUMBER::basic_regex_parser... 3472 boost: Stack-overflow in boost::re_detail_NUMBER::perl_matcher… 3478 boost: Stack-buffer-overflow in boost::re_detail_NUMBER::perl_matcher... 3479 boost: Null-dereference READ in boost::re_detail_NUMBER::basic_regex... Vinnie Falco pointed me to Jens Weller's blog post about fuzzing beast: https://www.meetingcpp.com/blog/items/fuzzing-beast.html . Jens used libFuzzer, which is one of the two fuzzing engines used by OSS-Fuzz. Adding a boost library to OSS-Fuzz will look very similar to this blog post. For those of you who are at CppCon this week: we can discuss this face to face. --kcc

1 1

property_map/compose_property_map get() return type
by Brook Milligan 31 Oct '17

31 Oct '17

I am trying to understand the logic behind the get() function for the Boost compose_property_map, which is defined in boost/property_map/compose_property_map.hpp. The document states the following: This property map is an adaptor that composes two property map. The new property map will of the same model as FPMap. I interpret these two sentences to mean that the category, value_type, and reference types of the composition will be the same as those for FPMap, while the key_type will be the same as for GPMap. Thus, I expect that get(composition,gkey) should return the same type that get(fpmap,fkey) would. However, the implement does not follow this expectation, which leaves me puzzled. The following is the relevant portion of the code: template <typename FPMap, typename GPMap> class compose_property_map { public: typedef typename boost::property_traits<FPMap>::reference reference; // return type of get(): // if (reference is not a ref) // value_type // else if (reference is const) // reference // else // const value_type& inline friend typename boost::mpl::if_< boost::mpl::not_< boost::is_reference<reference> >, value_type, typename boost::mpl::if_< boost::is_const<reference>, reference, const value_type& >::type >::type get(const compose_property_map &m, const key_type &k) { return get(m.f, get(m.g, k)); } Notice the comment and the selection of the return type. The result is always a value_type or _const_ reference, irrespective of the reference type of FPMap. Consequently, the information advertised as the reference type, which is the same as that of FPMap, is incorrect, and get() does not return a simple composition of the two property maps as described in the documentation. What is the rationale for this behavior? It seems that there must be a reason to go to the great lengths to second guess what FPMap is doing, rather than simply doing what the documentation says should occur, but I fail to see it. I look forward to any clarification. Thanks a lot. Cheers, Brook

1 0

[review] The review of Boost.DoubleEnded starts today: September 21 - September 30
by Thorsten Ottosen 31 Oct '17

31 Oct '17

Dear users and members of Boost, I'm proud to announce that the formal review of Benedek Thaler's Boost.DoubleEnded library starts today, September 21, and runs through September 30. This library is the result of Benedek's participation in Google Summer of Code 2015. The library may be downloaded from https://github.com/erenon/double_ended and the documentation may be found here: http://erenon.hu/double_ended/ Anyone is welcome to post a review and/or take part in subsequent discussions (see below for review guidelines). Introduction ------------ This is a container library that provides two containers: A. boost::devector B. boost::batch_deque Both containers provides efficient push_front and push_back operations, hence the name DoubleEnded. The boost::devector class can be seen as an extension of std::vector with efficient support for push_front (so boost::devector provides contiguous storage). In addition, it provides a customizable small buffer optimization like boost::small_vector(*). It furthermore allows for customization of the growth factor such that the user can decide if a reallocation should allocate 1.5 or 2 or whatever more memory. And finally it provides new functionality that does not exist in normal vector implementations -- this functionality allow the class to excel in performance in certain situations like serialization or transfer of data from sources that require a preallocated buffer. The boost::batch_deque is similar to std::deque, but with two important twists. Like std::deque, it provides reference stability when calling push_front/push_back. Reference stability is crucial for programs that uses intrusive containers e.g. using Boost.Intrusive (**). boost::batch_deque has the following main advantages: 1. It allows the user to specify the segment size 2. It allows for external iteration over the segments Taken together, they provide a very efficient and flexible basis for intrusive containers. Access to the segments also increases performance for tasks such as serialization. Finally, boost::batch_deque may be seen as an efficient alternative to std::vector if you want to replace a percent-wise memory overhead with a constant memory overhead and have no need for contiguous storage. Review guidelines ----------------- Please provide in your review whatever information you think is valuable to understand your final choice of ACCEPT or REJECT including Fit as a Boost library. Please be explicit about your decision. Some other questions you might want to consider answering: - What is your evaluation of the design? - What is your evaluation of the implementation? - What is your evaluation of the documentation? - What is your evaluation of the potential usefulness of the library? - Did you try to use the library? With which compiler(s)? Did you have any problems? - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? - Are you knowledgeable about the problem domain? More information about the Boost Formal Review Process can be found here: http://www.boost.org/community/reviews.html Kind regards Thorsten, Review manager (*) boost::small_vector: https://tinyurl.com/ycslp4ot (**) http://www.boost.org/doc/libs/1_64_0/doc/html/intrusive.html

24 159

[boostbook] Building documentation with generate.consistent.ids
by Daniel James 30 Oct '17

30 Oct '17

Hi, This was originally sent to the boost-docs list, but it doesn't seem to be working, so I'm sending it here. Sorry if you receive it twice. A while ago Gregor Jasny suggested building with the generate.consistent.ids option to reduce the changes between snapshots (as many elements in the documentation had random id numbers). I've tried turning it on for the develop documentation build, and tweaked boostbook a little to take advantage of it as well. With this setting generated ids can change when the documentation is edited, but not on every build. If you have standalone documentation and you want to use this feature, you need to use '<xsl:param>generate.consistent.ids=1' in the boostbook rule in your jamfile. This mainly affects boostbook reference documentation, such as: http://www.boost.org/doc/libs/1_65_1/doc/html/boost/accumulators/as_feature… This page gets a new location in every build. With consistent ids it has the consistent, although admittedly uglier, URL: http://www.boost.org/doc/libs/develop/doc/html/boost/accumulators/as_featu_… The advantage is that this will only change when the accumulators documentation is changed (btw. the text part of that url is probably truncated more than it needs to be, I might try to fix that). Other documentation won't be affected so much, as sections usually have explicit ids (quickbook always generates them). An example where a section doesn't is: http://www.boost.org/doc/libs/1_65_1/doc/html/idp22435328-bb.html I suggest you should probably add an explicit id to something like that (I've done so in that case). Daniel

1 0

[review][DoubleEnded] Review Results
by Thorsten Ottosen 30 Oct '17

30 Oct '17

Dear users and members of Boost, I'm happy to to announce that Benedek Thaler's Boost.DoubleEnded is /conditionally/ accepted into Boost provided that it can be integrated smoothly with Boost.Container. There are several other conditions, some major and some minor which I will outline below. For now, I would like to thank everyone that participated with reviews and comments. We didn't have that many reviews, but the reviewers did spend a lot of time both on review and the following discussions. Special thanks also goes to Benedek for his hard work. It is not too often that a GSOC code ends with something of this quality. Two of the world's expert C++ container developers used phrases like "solid" and "everything looks clean and beautiful" about the implementation, "tests look very good, corner cases and all", "The reference section is very good". Congratulations, Benedek. ============== A path forward ============== Ion has most kindly accepted the idea of merging the library into Boost.Container. This consists of A. taking selected elements from batch_deque and adding them to boost::container::deque. B. putting devector into boost.container after changes have been applied to code and documentation C. back-porting the devector code to C++03 (C) does not need to happen immediately, that is, we can have a release where devector is available in a C++11 version. Ion has also most kindly agreed to help with this process. I will help write a new introduction that focuses on the main principles of devector. We also have a number of benchmarks which we can gather and use for a small section of on that topic. I'll leave it to you and Ion to work out the details. ========================== Detailed comments devector ========================== Apart from the merging with Boost.Container, the most heard complaint was about the "fat", the non-essential parts if the interface, that could be removed. This includes -- Remove the direct support for small buffer optimization form devector. When the time comes, provide small_devector similar to the way we have vector/small_vector (there is no requirement that small_devector is provided). -- remove all of the following: - all constructors that uses reserve_only_tag, - all constructors that uses unsafe_uninitialized_tag, - void unsafe_uninitialized_resize_front(size_type); - void unsafe_uninitialized_resize_back(size_type); - unsafe_push_front(...); - unsafe_push_back(...); -- add instead - constructors that uses default_init_t like boost.container - resize versions that uses default_init_t - T& unchecked_emplace_back(...) noexcept(...) - T& unchecked_emplace_front(...) noexcept(...) At some point, I hope we can upgrade default_init_t to mean no construction for all trivially copyable types. There was a wish for more symmetry in devector and so functions that break symmetry or can appear ambiguous should be removed. This means we should remove: -- capacity() There is no need for front_capacity() and back_capacity(). We could remove reserve(), but then we can't use devector with a flat_set. Reserve() should not be an alias for reserve_back() though, but controlled by the policy. We could remove resize(), but even deque has it where it means resize_back(). I totally agree it would be ideal to remove all of reserve(),resize(),capacity(),clear(), but I see no easy way we can use devector with flat containers if we remove the first and the last. Unless we start deprecating stuff in boost::deque/boost::vector and provide new consistent alternatives, we will never get a 100% satisfactory solution. At the end of the day, we need to be pragmatic, so if deprecation is not on the table, this seems the best we can do. Since we don't want two allocations implied by calling reserve_back() and reserve_front(), you can add a mechanism for doing that. That is, you can provide -- reserve( new_capacity ); -- reserve_front( new_front_capacity ) -- reserve_back( new_back_capacity ) -- reserve( new_front_capacity, new_back_capacity ) or, since reserve() is such a weird beast, simply provide -- reserve( new_capacity ); // for flat containers -- anticipate_front( new_front_elements ) -- anticipate_back( new_back_elements ) -- anticipate( new_front_elements, new_back_elements ) to get rid of the funky -- container.reserve_front( container.size() + new_front_elements ) dance. There were suggestions for shrink_back/front_to_fit, but since we are trying to remove fat, I suggest that we wait for a use-case. There were also suggestions that clear() should be amended. As the growth (or resizing policy) mandates what clear does, we need a way to escape. The minimal change that allows that is to provide -- clear( front_free_capacity ) which gives full power to the user. You may provide clear_back() as clear(0) and clear_front() as clear(c.size()+c.front_free_capacity()+c.back_free_capacity()). Ideally, clear() would have been removed entirely like capacity()/resize(), but like reserve(), it just seems like impossible when we start using it with flat containers. Other minor comments: - drop strong exception safety guarantees and do it like boost.container - prefer to make iterator a type (should be easy with reuse from boost::container::vector - allocator should not be inherted from - make clear O(1) amortized gurantees depend on growth policy - consider calling growth policy for resizing policy - make sure SFINAE works properly with constructors - make the default growth factor 2 (as we heard from Sean Parent: no factor below 2 is empirically good) - consider renaming front_free_capacity to free_front_capacity. Rationale: "front_capacity" is a concept, noun, and "free" is the adjective. Ditto for back_free_capacity. - remove size_type from growth policy and take it from the allocator - make the allocator template argument the 2nd argument - fix remaning TODO in code - documentation should not sell devector as drop-in replacement for vector, but rather stress it is not - the allocator supports needs to be improved. I guess doing it the Boost.Container way ensures that (ask Daniel James for help) - exception-safety issues pointed out by Ion - postconditions of insert/erase needs to specify behavior to front_free_capacity/back_free_capcity - use of local containers should be avoided - rotate should be avoided - delegate range move operations to those defined in Boost.Container - make sure all functions excepting non-overlapping range uses memcpy (not just memmove). This includes: - range constructor - copy-constructor - copy-assignment - range-assignment - range insert - destructor should used std::has_trivial_destructor to avoid loop in that case to make sure all compilers optimize it away (similar for erase). There is one final puzzle and that is the growth policy. I hope we can finalize that by testing Joaquin's and Ion's suggestions over the coming weeks. ============================= Detailed comments batch_deque ============================= For deque, the comments were similar, although there was far less to discuss than for devector. All agreed that one should be able to control the segment size. - stable_insert was not needed. - segment iteration was good, but should be usable with range-based for loop. The segment iterator should return a segment type if possible, and the segment should have begin(),end(),size() members. - since boost::deque already has a resize with default_init_t, that should be sufficient to provide fast serialization load for trivial types. No need for - resize_front - resize_back - unsafe_uninitialized_resize_front - unsafe_uninitialized_resize_back - reserve - shrink_to_fit() - capacity() The following are still needed for optimal code (or code that can be known not to reallocate, or for better locality in deserialization, avoiding double initialization) - free_front_capacity; - free_back_capacity; - reserve_front - reserve_back - unchecked_emplace_front - unchecked_emplace_back Boost.Container's unit-tests should be extended with those from batch_deque where it makes sense. Since segmented iteration was somewhat faster, it should be use internally where it makes sense. In particular - copy-constructor - copy-assignment - destructor As for devector, all constructor/functions that iterate over non-overlapping ranges should use memcpy. destructor should use is_trivially_destructible to ensure it is optimized away (similar for erase). ====================== Serialization comments ====================== I suggest that the default versions just do what Boost.Serialization does, perhaps just replacing resize with resize with default_init_t. That ensures it works correctly for all archive types. It also needs to use make_nvp() so the output is identical to std::vector. We will then talk to Robert about how we can take advantage of the binary archive in a correct fashion. ====================== kind regards Thorsten, review manager

8 12

PSA: Travis OS X bottleneck, <cxxstd> new Boost.Build feature
by Peter Dimov 30 Oct '17

30 Oct '17

Travis's OS X resources for open source projects seem to be insufficient, so OS X jobs are very slow and seem to be falling further and further behind. Therefore, library authors are encouraged to keep the OS X jobs to the minimum necessary. Spawning many OS X jobs takes hours (~25 minutes waiting time per job at a quick estimate). On a not entirely unrelated note, Rene Rivera has added a new feature <cxxstd> to Boost.Build that controls the C++ standard in use. So for instance, instead of the old b2 libs/mylib/test toolset=gcc cxxflags=-std=c++11 one can now use b2 libs/mylib/test toolset=gcc cxxstd=11 In addition to being more convenient, this also allows several invocations to be combined into one: b2 libs/mylib/test toolset=clang cxxstd=03,11,14,1z which can be leveraged to cut down on the number of jobs. An example of using cxxstd in .travis.yml can be seen here: https://github.com/boostorg/system/blob/develop/.travis.yml

7 13

Adapting boost::grid_graph vertex descriptors to Boost Geometry
by Brook Milligan 30 Oct '17

30 Oct '17

Generally Boost Graph vertex descriptors are opaque types with limited means of constructing them; they may be default constructed or constructed with the null_vertx() function, by dereferencing vertex iterators, calling source()/target() on an edge descriptor, or by calling the add_vertex() function. It is never expected that one will construct a vertex descriptor directly with a non-default constructor. The Boost grid_graph is similar, with the exception that vertex descriptors may additionally be constructed from a vertex index. However, a vertex descriptor for an n-dimensional grid_graph is quite naturally an n-dimensional point. Consequently, it should be possible, without breaking the abstraction, to construct one (and perhaps use one) as a point. Given that this use is not documented in any way, doing so breaks the abstraction. At least three solutions are possible: - #include <boost/geometry/geometries/adapted/boost_array.hpp> in grid_graph.hpp: boost::array<> is the implementation of the grid graph vertex descriptor, so this would immediately make vertex descriptors model Boost Geometry points. - Add a boost/graph/adapt_grid_graph_to_geometry.hpp (better name needed) file that includes the same file so that the selection is optional. - Add a boost/geometry/geometries/adapted/grid_graph.hpp file that includes the same file, again making the selection optional. Since the contents of boost/geometry/geometries/adapted generally refer to external libraries (e.g., Boost Array), the last option seems the most appropriate, although it suffers in case the vertex descriptor is changed but not the adaptor. That however applies to all the adaptors equally. Any of these solutions differs from using boost/geometry/geometries/adapted/boost_array.hpp directly in user code, because that requires unpacking the nature of the vertex descriptor and leaves open the potential for breaking changes in user code if the vertex descriptor changes. Because a vertex descriptor is opaque, breaking the abstraction seems inappropriate, even here. Thus, I suggest the following: - add boost/geometry/geometries/adapted/grid_graph.hpp which includes boost/geometry/geometries/adapted/boost_array.hpp. - add the new adaptor to the Boost Geometry documentation - add a note to the Boost Graph grid_graph documentation indicating that it is possible to adapt the vertex descriptor and use it as a point. Thoughts? Cheers, Brook

2 2

[contract] help marking regression test expected failures
by Lorenzo Caminiti 27 Oct '17

27 Oct '17

Hello all, I need some help with marking some Boost.Contract regression tests as expected failures. I don't understand why my explicit-failures-markup.xml: https://github.com/boostorg/boost/blob/develop/status/explicit-failures-mar… Doesn't math the expected failures I see on the regression report: http://www.boost.org/development/tests/develop/developer/contract.html For example, the following from explicit-failures-markup.xml (around line 887) should marked old-if_copyable test as an expected failure on msvc-11.0: ... <mark-expected-failures> <test name="old-if_copyable"/> <test name="old-if_copyable_macro"/> <toolset name="gcc-gnu-4.7.2"/> <toolset name="gcc-gnu-4.7~c++11"/> <toolset name="gcc-mngw-gnu-4.6c+"/> <toolset name="gcc-mngw-gnu-4.7c+"/> <toolset name="msvc-10.0"/> <toolset name="msvc-11.0"/> <note author="Lorenzo Caminiti"> Test fails because this complier version does not properly implement SFINAE causing incorrect errors on substituion failures for private and protected members (this seems to be fixed in GCC 4.8 and MSVC 12.0). </note> </mark-expected-failures> ... But the old-if_copyable test still shows up as a failure on the regression test report: http://www.boost.org/development/tests/develop/developer/output/teeks99-08-… Any idea what I am doing wrong? Thank you. --Lorenzo

5 19

path length too long for 64 bits builds
by Vicente J. Botet Escriba 27 Oct '17

27 Oct '17

Hi, we have an issue since we have added threadapi= . On 64 bits windows machines we are seen Austin Beer said in https://github.com/boostorg/thread/pull/188#issuecomment-339829423 "FYI, the 64-bit Windows builds are failing due the unit tests exceeding the 260 character filepath limit on Windows. Every filepath includes |\msvc-14.1\debug\address-model-64\threadapi-win32\threading-multi|. Do you know if there is an easy way to shorten this? On the 32-bit builds the filepaths don't have |\address-model-64| which is just enough to keep them under the 260 character limit so they don't fail." Is there a workaround to this issue without changing the specific part of the test files folder structure? Best, Vicente

3 3