May 2017 - Boost - lists.boost.org

[poly_collection] User-defined polymorphism model
by Ion Gaztañaga 14 May '17

14 May '17

Hi Joaquín and all reviewers, Before we reach the end of the review period, I wanted to comment an (IMHO) interesting feature used in the internal design of PolyCollection. The library offers three of polymorphism models: base-derived, boost::any and signature-sharing callable types. Under the hood, the library abstracts those polymorphic models into a generic class in the detail namespace: template<typename Model,typename Allocator> class poly_collection; and each concrete container is just implemented in terms of poly_collection: template<typename Base,typename Allocator> class base_collection : public detail::poly_collection<detail::base_model<Base>,Allocator> The first template parameter (Model) is the abstraction ("concept") that implements the non-generic part of each polymorphism model. A model implements the following types and operations (comments on each attribute are my interpretation, they can be completely wrong): template<class ModelArg> class Model { // //Value-related attributes // //The type we want to expose as value_type of the container using value_type= ...; //Is T a subtype of the general type (acceptable as value)? template<class T> using is_subtype= ... //Is T the "most refined type"? template<class T> using is_terminal=... //Id of a value template<class T> static const std::type_info& subtypeid(const T& x); //Obtain a type-erased pointer to the value template<class T> static void* subaddress(T& x); template<typename T> static const void* subaddress(const T& x); // //Segment-related // //Segment iterators that point to type_erased value_types using base_iterator=... using const_base_iterator=... //const<->non-const conversions static base_iterator nonconst_iterator(const_base_iterator it); template<typename T> static iterator<T> nonconst_iterator(const_iterator<T> it); //Concrete iterators for Derived template<typename Derived> using iterator=...; template<typename Derived> using const_iterator=... //Generic type, comparable with base_iterator. //base_iterator == the last returned base_sentinel //means that base_iterator points one-past the last //element of a segment. using base_sentinel=...; using const_base_sentinel=... //Construct a segment for type T using the given allocator and //return it type-erased. template<typename T,typename Allocator> static detail::segment_backend<Model>* make(const Allocator& al); }; I have several questions on this polymorphism model. 1) Wouldn't be a good idea to make this model-based polymorphism (or a refined one) public? Maybe the current model needs more work to support additional polymorphism models. Of course requirements for each operation in a model should be detailed. A correct abstraction to define user-provided polymorphism models is hard but this seems really a great feature that makes this library more valuable for users. 2) If understand this model abstraction correctly, each Model chooses the segment type to store values (it could be any sequence type, I guess). The segment_backend abstraction does not only abstract ConcreteTypes from value_types (using void pointers that will be casted to real types in the implementation of the virtual function), but also the segment type (the sequence holding the concrete values). Current implementation uses two segment types: split_segment (stores Concrete types and generic value_type's in two separate vectors) and packed_segment (stores ConcreteTypes which already contain value_type's as a subobject). function_collection and any_collection use the first segment, base_collection uses the second. Why are different segment types needed? Can't we store value_types and ConcreteTypes in a struct and use only one type of segment? If this is possible, I guess iteration performance would be improved for function_collection and any_collection. Model definition could be simplified. On the other hand, if the sequence type can be left out of the model definition, the underlying sequence type (change vector with deque, small vector or any other) could be made configurable. Would this be useful for users? 3) I understand that caching a base_sentinel in each segment is an optimization to avoid a virtual function call when iterating with base_iterators. Is this assumption correct? Why is a different base_sentinel type needed to mark the end of each segment? Wouldn't a "base_iterator" pointing to that on-past-the end position suffice? Is an optimization (maybe base_sentinel is cheaper) or a must have? Thanks, Ion PD: I have more design questions related to other topics but I will post a different thread for each one.

2 2

Re: [boost] [review] Review of PolyCollection starts today (May 3rd)
by Glen Fernandes 13 May '17

13 May '17

Gavin Lambert wrote: > On that note, do you know of a more-comprehensible-than-standardese guide > on how allocator_traits is supposed to be used for node-based custom > container types? I've tried to figure it out a couple times but I'm sure I'm > missing important points. What do you want to know? 1. The storage for the node should be allocated with the Allocator's 'allocate()', for an allocator instance 'a' whose value_type is the containers node type (i.e. 'a' can be achieved by rebinding an existing allocator instance in the normal way). 2. The node object itself would be constructed in the normal way - e.g. placement new to construct the node object. 3. The containers' value_type object within the node must be constructed by allocator_traits<U>::construct(u, p, args...) where u of type U is a rebound copy of the allocator such that the allocator's value_type is the container's value_type. 4. Similar to 3, for destruction of the container's value_type object should be destroyed by allocator_traits<U>::destroy(u, p) 5. Similar to 2, the node object is destroyed in the normal way - e.g. invoking the destructor of the node type. 6. Similar to 1, the storage for the node is deallocated with the Allocator's deallocate() (for an allocator instance whose value_type is the container's node type). Glen

1 0

[poly_collection] Andrzej's review
by Andrzej Krzemienski 12 May '17

12 May '17

Hi Everyone, Some comments from me about PolyCollection library, First, Joaquín, thank you for writing and sharing this library. I am not an an expert in the domain. In my programs I never needed this optimization. But I have seen it described in a number of publications, and I am convinced it deserves a place in Boost libraries, so that people are not forced to reinvent it. [1] Now, because I have no practical experience with the problem, what I say in this paragraph might be incorrect. But I have a problem here. You will use this library for improved performance, maybe for a game implementation, yet because a game needs super-performance, can a programmer afford to use dynamic dispatch, or RTTI or exceptions? I always imagined taht such library, rather than resembling `ptr_vector<I>` would resemble `vector<variant<A, B, C>>`, that is: I decide and fix the container on a set of types I will be storing inside. This set of tyes is embedded in the type, so I can detect many mis-usages at compile time. The implementation can be faster, because the number of segments is fixed (no segment management), and no registration checks need to be performed. Then, iterate over such collection with a static_visitor (from Boost.Variant). Of course, this would be constraining: I need to know all the types I will be using ahead of time. But maybe in practice this is often the case? If you disagree, maybe it is wort discussing it in the rationale. [2] I am not comfortable with per-segment functions having the same name as container-wide functions, and being only overloaded based on function or template parameters, like `cc.size()` and `cc.size(index)`. These two functions do different things, so they deserve different names. Maybe `cc.segment_size(index)`, or `cc.serment(index).size()`? [3] I downloaded it and tried toy examples with GCC 6.3 on Windows with -std=c++11, and clang 3.8.1 on Fedora with -std=c++11. It compiIes fine. I observed that the following small program crashes (assertion fails): ``` #include <boost/poly_collection/base_collection.hpp> struct Iface { Iface() = default; virtual ~Iface() = 0; }; inline Iface::~Iface() {} struct Type1 : Iface { Type1() = default; Type1(Type1&&) = delete; }; struct Type2 : Iface { Type2() = default; Type2(Type2&&) {} // throwing move // no move assignment }; int main () try { boost::base_collection<Iface> c; c.insert(Type1{}); // fires an assert c.insert(Type2{}); // fires an assert } catch (std::exception const& e) { } ``` Admittedly, `Type1` and `Type2` are not "acceptable", but according to the documentation this should throw an exception upon insertion rather than firing assertions. [4] The documentation is good. As to implementation, I did not look at it in detail. Regards, &rzej;

2 2

[poly_collection] Small review
by Pete Bartlett 11 May '17

11 May '17

> > Dear Ion, Joaquin and boost mailing list, > > I hope this brief review is of some value to you. I don’t expect to be a recognised name on the mailing list but have contributed to it intermittently over the years. I have also used the author’s MultiIndex library in production for about a decade. > > I would recommend that PolyCollection be accepted into Boost. > > I think the design pattern is common enough to be Boost-ified. I have done something very similar to boost::base_collection myself in the past and have achieved useful speed ups over a solution like boost::ptr_vector in a production system. My speed-ups were not quite as good as Joaquin documents, but I can’t tell if this was due to the ineptitude of my own solution or due to lab conditions of the Boost.PolyCollection tests. > > I spent a couple of hours reading the docs and running basic examples on my default compiler (VS2015 x64) and did not experience any significant problems. > > It may be worth remarking in the docs (sorry if I missed it) what facilities are used for the type registries that are presumably behind the scenes. From the reference section I think RTTI is required. Is this a hard limit or could Boost.TypeIndex be used? > > NB the reference section has a note “REVIEW THIS DESIGN DECISION” in it. > > Personally I would attach value to the fact that the author has been a great servant to Boost over the years (as have many others) so it is likely that this would be a well-maintained library. E.g. if algorithms were added to <algorithm> in the future, they would likely end up in <boost/poly_collection/algorithm.hpp> too. > > Regards, > Pete >

3 4

[review][poly_collection] Review period EXTENDED until May 17th
by Ion Gaztañaga 11 May '17

11 May '17

Dear boosters, Some have requested a bit of time to extend the review period in order to have time to submit reviews or issue technical comments. With the approval of Review Wizards, the review of PolyCollection is extended until May 17th. Please feel free to submit review and comments. Best, Ion

1 0

[poly_collection] Two remarks
by Dominique Devienne 10 May '17

10 May '17

Just as Pete Bartlett mentioned, I'm also a fan of Joaquín's Multi-Index, which I'm used in production for many years as well. My only complaint about his proposed Poly-Collection is the fact I can't leverage and thus benefit from the performance of it, in a multi-index scenario, but as Joaquín already answered a few months ago, that's off-topic and by design, I know :) Many of my BMIs have a by-type index, and the type-partitioned storage of those in contiguous memory for by-type scans would be (greatly?) beneficial. I'd be willing to give up some iterator guarantees BMI provides, as a node-based container, for intelligent contiguous storage of elements, without myself having to maintain by-value homogeneous containers with a manually maintained BMI of polymorphic pointers to those containers' elements. My second remark is about parallelism. I suspect the new poly_collection could be used for more efficient parallel (and/or concurrent) processing of elements, and would be interested in examples and even a ::parallel_for_each() algorithm that leverages the collection's internal storage. And perhaps even on the write side, rather than having to make the whole container thread-safe, but external mutexing for example, provide ways for a more granular thread-safety by type or segment perhaps. Again, probably off topic, but hardly something you can "add from the outside" I think, and why I mention it here and now. As merely a user of Boost, I don't feel qualified as a reviewer, beyond the two above remarks. Thanks, --DD

5 4

[build] please review the long standing pull request
by Konstantin Ivlev 10 May '17

10 May '17

Hello, There's a 2 month old pull request https://github.com/boostorg/build/pull/169. Please review and let me know any feedback.

1 0

[Thread] What is the meaning of BOOST_THREAD_USE_LIB?
by Frédéric Bron 09 May '17

09 May '17

Hi, I always use boost as static libraries. For a long time, I have been used to define macro BOOST_THREAD_USE_LIB on windows when cross-compiling from linux to windows with mingw. I have now made a new cross-compiler conformant with pthread (the previous one had only win32 api) but then boost does not build (with threadapi=posix). It seems that not defining BOOST_THREAD_USE_LIB helps. I do not remember why I used to define BOOST_THREAD_USE_LIB a long time ago and cannot find any help in the doc (http://www.boost.org/doc/libs/1_64_0/doc/html/thread/build.html) What is the meaning of BOOST_THREAD_USE_LIB? Frédéric

2 1

gsoc17 and boost.ublas
by Rishabh Arora 09 May '17

09 May '17

Hi Everyone, My name is Rishabh Arora and I have been selected to work with the BoostC++ libraries on the the project, "dataframes in ublas" as a part of GSoC '17. It will be great if someone could provide me with suggestions on how to get started with it. Thank you in advance Best Regards, Rishabh Arora

2 1

[GSOC'17] Project: Optimising Multiplications in uBLAS
by mohammad sharique 09 May '17

09 May '17

Hello Boost Community, My name is Mohd Sharique. I have been selected to work with Boost C++ Libraries under Google Summer of Code '17. My project is "Optimising Multiplications in uBLAS". It would be great if I could get some starting point for the same. Thank you, Your's sincerely, Mohd Sharique

2 1