April 2019 - Boost - lists.boost.org

[variant2] Andrzej's review -- design
by Andrzej Krzemienski 05 Apr '19

05 Apr '19

Hi Everyone, First of all, I wanted to thank Peter for writing this non-trivial library and sharing it with the community. This is not a full review yet. I just wanted to discuss the design first, especially the never-empty guarantee. And I am going to rehash the old argument. The core of variant2 is its never-empty guarantee and the mechanism to achieve it: a number of different algorithms chosen depending on what the potentially stored types permit. In fact this mechanism is not much different than the one present in boost::variant. The issue to solve is what happens in copy/move assignment of variant<X, Y> when we have to change the stored type from X to Y. X has to be destroyed and Y initialized. But if the initialization of Y throws we would be left without either X (already destroyed) or Y (not successfully constructed). (The same problem applies to member function emplace()). Except for the last step boost::variant and variant2 try to apply the same sequence: 1. If all the subtypes have noexcept initialization, no safety measures need to be taken. 2. If sub-types have noexcept move constructor, a temporary local storage is used and the backup value is moved to and from it. 3. If one of the subtypes is a special type boost::blank or std::monostate respectively, in case of a throw this type is default constructed in the variant. 4. If there is any nothrow-default-constructible subtype in the variant, it gets default-constructed upon exception. If neither of the above can be applied, the two libraries apply different approaches. variant2 is big enough to fit two unions of X and Y within its storage. This way it does not have to destroy the X when initializing the Y. boost::variant, for the additional storage uses heap allocated memory. Interestingly, the double storage as in variant2 has been considered in the initial design of boost::variant but rejected by the community, because it added spacial overhead even for use cases that never invoked variant's assignment. This is described in the Rationale section of boost::variant documentation: https://www.boost.org/doc/libs/1_69_0/doc/html/variant/design.html#variant.… In boost.variant, only these programs pay for the overhead that really invoke the assignment operator. Neither of the solutions seems strictly superior than the other. The double storage could be thought as better because it seems to be able to cover use cases where heap allocations are banned altogether. On the other hand, in these applications it is difficult to imagine that X and Y would be throwing exceptions in move constructors or during initialization. One inferiority of boost::variant is that it can fail for its own reasons unrelated to failures of X and Y. Even if no operations on X and Y throw during the assignment of variant, the variant can still throw because it allocates memory. Some people believe that an application cannot recover from bad_alloc anyway, but this is not a universally held view. To me, the important question is, do we really need the never-empty guarantee. It was an unquestionable requirement for the initial design of boost::variant. Now, variant2 is advertised as an allegedly superior solution to std::variant, because the latter does not provide the never-empty guarantee, even though the cost of providing it is significant. But I myself fail to see the *practical* value of the never-empty guarantee. The argument I hear is that, we want to make it an invariant that either X or Y is stored in variant. I agree with this goal in general, but the costs and contortions required to satisfy it outweigh the benefit. Now, I hear that if the invariant is weakened everyone will have to check everywhere if any variant is in a valueless_by_exception state. This seems to be confirmed by the semantics of std::variant::visit(), which requires that valueless_by_exception is checked and an exception thrown on valueless. This last part is indeed strange in std::variant, but I do not agree that the fix is to provide a never-empty guarantee at the cost of compromising run-time performance and complicating the design. I can think of the following alternative design. The invariant of a variant (pun intended) allows for a valueless_by_exception state. There is no way to get to this state other than by throwing form the initialization of Y in variant's assignment or .emplace() operation. Calling .visit() on a valueless variant is UB. The destructor and reset functions such as assignment and .emplace() can handle the valueless state. This is almost what std::variant does except for UB in .visit(). Why I find this superior to variant2. 1. It is simpler, faster and smaller. 2. Observing the valueless state can only happen when somebody is handling exceptions incorrectly. And in that case this is exception handling strategy that needs to be fixed: not the variant's guarantees. It should be noted that variant2 DOES NOT PROVIDE A STRONG EXCEPTION SAFETY GUARANTEE. The never-empty guarantee has a certain negative effect of confusing people so that they are led to believe that they have strong exception safety. But they don't. even expert programmers in this forum expressed their surprise that changing from X to Y in variant<X, Y, Z> can get them a variant storing Z. Sometimes it is even trickier: X may be a container that currently stores 10 elements. You assign Y, and after the exception you get an X that stores zero elements. Because of these problems, the only reasonable thing to do is to follow the advice from Dave Abraham's article: https://www.boost.org/community/exception_safety.html To quote the relevant part, "If a component has many valid states, after an exception we have no idea what state the component is in; only that the state is valid. The options for recovery in this case are limited: either destruction or resetting the component to some known state before further use." If this principle is followed no-one will ever observe the valueless_by_exception state. This also applies to variant2. If this principle from Dave Abrahams is followed, no-one will ever benefit from the never-empty guarantee (but people will still pay the costs of providing it). You cannot do anything meaningful with variant that threw other than destroy or reset it. (But it is probably best to destroy it.) If objects are allowed to survive the stack unwinding, they should either offer a strong exception safety guarantee, or you have to have some very detailed knowledge of a given particular type how its state gets modified under exceptions, but this is usually very difficult and best left to experts. I have started a thread in this list a while ago, requesting for an example of code that *correctly handles exceptions* (does not stop stack unwinding at random places), where the programmer would make use of the never-empty guarantee , and chose something else than destroying or resetting the variant. And although I received some generic statements, referring to the purity of the design, strength of the invariants, and the easiness of thinking or "correctness", none of the proponents of the never-empty guarantee gave such an example. Which might be an indication that the problem we are solving may be nonexistent. I am not sure of this claim, but I have not seen any convincing example either. I believe that we are facing one of this problems where the strong invariants of the type need to be weakened due to practical considerations. I can see no correctness issues with valueless state other than in programs that are already exception-unsafe. Also, I have heard claims that if someone is fine with valueless state they should put std::monostate as one of the types and the same effect without double buffering is achieved. But this is not true. Variant with monostate can get the degenerate state in a correct program; often: in the variant constructor. Now I really have to special-case it whenever I inspect the variant object, which is a nightmare: this is really a weak invariant in an allegedly strong-invariant variant. In contrast, for std::variant you only get the valueless state in the program that incorrectly handles exceptions: no need to special-case the valueless_state when doing visitation.. variant2 is not a replacement for std::variant, as the choice to support never-empty guarantee at the cost of more expensive operations and increased object size is not necessarily superior. Even if my view is not generally held, it has to be acknowledged that the issue is controversial. Variant2 cannot replace boost::variant either. This is not only about supporting pre c++11, but also for different trade-offs in never-empty guarantee, which were deemed inferior in boost::variant design. variant2 is ismply a third point in the multi-dimensional design space of different trade offs for variant-like libraries. On a different subject, one thing that is missing from both: boost::variant, and std::variant, is the narrow-contract observers for the specific stored type, useful when I know by other means what is currently stored in the variant object. This is something the third variant implementation has a chance to change, however it chose to follow its predecessors. This seems somewhat counter to the spirit of C++. We requested a similar thing of Niall's Outcome library, and the narrow-contract observers were added. Below is the comparison between boost::variant, std::variant and variant2. I do not know how to make a table in the email discussion interface, so I will have to make it with a list. 1. Works in pre-c++11 dialects * boost::variant: yes * std::variant: no * variant2: no 2. Size of variant<X, Y> (when X and Y are unfriendly) * boost::variant: sizeof(void*) + SXY (SXY is the size of union{X x; Y y;}) * boost::variant: sizeof(int) + SXY * variant2::variant: sizeof(int) + 2 * SXY 3. Uses heap allocations * boost::variant: yes * std::variant: no * variant2: no 4. Throws its own exceptions in assignment * boost::variant: yes * std::variant: no * variant2: no 5. Fast (narrow contract) access to X alternative. * boost::variant: no * std::variant: no * variant2: no 7. Strong exception-safety guarantee * boost::variant: no * std::variant: no * variant2: no 8. Never-empty guarantee * boost::variant: yes * std::variant: no * variant2: yes 9. Throws its own exceptions in visitation * boost::variant: no * std::variant: yes * variant2: no Regards, &rzej;

5 19

[variant2] Andrzej's review -- design
by Mike 05 Apr '19

05 Apr '19

> Every time you touch a variant object, you must consider the possibility of it being valueless.Â No I don't. It just works and I explained why. Could you please point out specifically where I made a mistake in my reasoning in the mail before where I discussed the behavior of all member functions? And again: I'm not arguing that one is better than the other. I'm saying it doesn't matter, but I'd much perfer a design that is in line with std::variant instead of having yet another slightly incompatible version of variant that doesn't offer any significant benefit (would be a different story if it would always offer the strong exception guarantee).

2 1

Regarding Boost.Astronomy GSoC19
by sarthak2007 04 Apr '19

04 Apr '19

Hello, For the competency test I had used quantity<length> of boost::units in place of double to store the values of coordinates with their units. But in project we have to deal with boost::geometry. One way it can be done is that whenever we use functions for dot product, cross product, etc. directly from boost::geometry then we just pass the values of coordinates in them instead of units. There will be no problem considering units in calculations as boost::units convert the values stored into SI units so all the calculations will be involving coordinates with SI units. But then we will have to change the way of storage of coordinates as boost::geometry can't be used to store coordinates(with units) as it does not support units. Is it the right approach or there is some nicer way to implement this? Thank you, Sarthak Singhal -- Sent from: http://boost.2283326.n4.nabble.com/Boost-Dev-f2600599.html

2 1

[gsoc 19]proposal to project 1 Boost.GIL
by raghuchandra kanakamedala 04 Apr '19

04 Apr '19

Hi my proposal link: https://docs.google.com/document/d/1-BZO0VuIcebe1ZcdyxHGjOEkQL3eeaXwGtKCWyQ… .Please suggest me any changes if required. Thanking you K RaghuChandra

2 2

variant - off topic
by Robert Ramey 04 Apr '19

04 Apr '19

I've been and observer on the discussions regarding variant, expected, outcome, optional, outcome and ?? for sometime. I'm pretty amazed at the amount of controversy each and everyone of these in every forum - here, C++ committee, etc. I'm beginning to wonder this isn't telling us something. It seems natural that when making a new C++ data type, we should want to implement the assignment operator. Why - well every variable needs to be assigned to from time to time. Why is this so? Well ... Suppose we break this assumption and consider the idea that assignment doesn't make sense for every type. A variant is a "meta-type" or "sum type" which "holds" one of a set of other types. Such a variant can take on the character of any one of type types it is holding. When does it make conceptual sense for a variable to change it's type one it has been initially assigned. I've been trying to think of one and haven't been successful. Here are a few of the scenarios I've considered. a) Returning a sum type - the most common scenario for a variant. Doesn't require a assignment. Copy/move may still be required of course. b) looping through a vector of variants as in vector variant<...> v; for( variant<...> v : [](){...} fix with change to vector variant<...> v; for( variant<...> & v : [](){...} c) set according to condition variant<...> r; if(i < 0) r = v<t1>(...) else r = v<t2>(...) fix with change to variant <...> r = (i < 0) ? v<t1>(...) : v<t2>(...); d) insert your own use case here. That is, look at your own usages of assignment to these types and see if there is not an easy way to re-factor your code to avoid assignment. I'll concede that, influenced by functional programming style, with a little experimentation with F# and propaganda from the Haskell community, I've become accustomed to code in the above way. Also I've become afraid to assign anything to these types. The discussion has made me aware of a raft of unintended consequences of doing what on it's surface seems a trivial operation. I've gotten to the point that assignment to a variant type, or "changing the type" of a value at runtime is not code smell to me. Maybe all this could be just eliminated if this became the accepted position. Of course I realize that by engaging in the practice I do, the whole issue becomes irrelevant to me personally. And I can always create variant types as "const". It's not about that. It's about avoiding the creating of vocabulary types which lead to problems. So - simple proposal. Require that sum types have operator= deleted. Robert Ramey

3 5

[variant2] never-empty guarantee, again
by Andrzej Krzemienski 04 Apr '19

04 Apr '19

wt., 2 kwi 2019 o 22:59 Peter Dimov via Boost <boost(a)lists.boost.org> napisał(a): > Andrzej Krzemienski wrote: > > > Can you show a destructor like that, does a visitatoin on a variant, and > > that you wouldn't be ashamed to put in your program? > > The destructor doesn't have to do visitation. It merely needs to call a > function f1, which calls a function f2, which calls a function f3, which > calls a function f4, which does visitation. > > What we've been telling you from the start is that this forces you to > partition your functions into two classes, one allowed to do visitation, > the > other not, then keep track of which is which, never calling the wrong > class > in a destructor, or from a catch clause. > The fact that I have to partition all functions into two categories isn't new or surprising. When defining an operation that needs to provide a strong exception-safety guarantee I need to some use operations that offer no-fail guarantee. So I need to be aware which operations offer no-fail guarantee and use only those. I do the partitioning, and I need to be cautious. When I define the destructor, again I do not have the liberty to use any function I can use in other parts of the code. In destructor I can only use functions that do not signal failure through exceptions if they fail. And not signalling failures is ugly on its own, so I am inclined to also just use functions that cannot fail. I have to do the partitioning again. When I try to go back and recall how the destructors I write look like, they fall into three categories: 1. Some types just aggregate other types. The implicitly defined destructor only calls the destructors of subobjects. 2. Sometimes I need to provide a RAII interface for a system resource or a C component. In that case my implementation uses only ints and pointers and system calls. When defining the destructor, I have confidence that there are no unexpected types hidden by templates, or functions with unexpected types with unpredictable guarantees. 3. When I define a generic component, such as optional<T>, in my destructors the only operation on the unknown type T is to call its destructor. Other operations in the destructor are performed on concrete types with no-fail guarantee such as `bool`. The situation is no different for std containters: for T, they only call destructors, other operations are on concrete types: pointers, memory deallocation. No way a variant or other unexpected type could get there, even indirectly. I know people are using destructors for other things, like committing the DB transaction, but this is what I call bad exception handling. Such implementations are usually incorrect: either you do not report failures in such case, or you are reporting failures when there is none. Your example of a destructor, to me, also falls into this category: template<class T> class monitor > { > char const* file_; > int line_; > T const& v_; > T old_; > > public: > explicit monitor( char const* file, int line, T const& v ): > file_( file ), line_( line ), v_( v ), old_( v ) {} > > ~monitor() > { > if( v_ != old_ ) { > std::clog << file_ << ":" << line_ > << ": monitored value changed from " > << old_ << " to " << v_ << std::endl; > } > } > }; It calls operations on an unknown T that could fail and throw: comparison, streaming. Streams themselves could throw. Such implementation has other problems before we even consider a variant. And, of course, this has nothing to do with visitation, specifically. The > exception safety guarantee is a global thing. Once you lose basic, you lose > it everywhere, not just for visit. > Why are you mentioning "basic" (guarantee) here? In the vision that I propose variant offers basic guarantee. It is just that the invariant is weaker. This is analogous to a (potentially smart) pointer: normally you can call `*p`, but sometimes (on nullptr) it is UB, so you may need to check what state it is in. Regards, &rzej;

2 2

Boost commercial support policy
by SOCIPROJECT 04 Apr '19

04 Apr '19

Hi,Good day.  This is Shuvransu Banerjee, development manager at Ushamartin Technologies. We build software products on Telecom Billing.We are at the planning level and thinking about using Boost in our legacy code. We have the following queries: Can we use Boost commercially? Is there any support available? If yes, then please let us know the various options for support.Thanks & Regards,Shuvransu Banerjee

2 1

B2, transition to C++
by Rene Rivera 04 Apr '19

04 Apr '19

After six months of, slowly, working towards this I can now say that the C++ compiled version of the B2 engine is ready for use. As such I plan, if there are no objections from Steven Watanabe and Vladimir Prus, to merge the result into the develop branch shortly after the 1.70 release. The history note accompanying this "new" engine version goes like this: === Version 4.0.0 After even more years of development the landscape of build systems has changed considerably, and so has the landscape of compilers. This version marks the start of B2 transitioning to a C++ implementation. Initially this means that the engine will be compiled as C++ source but that source is still the base C implementation. Over time it will transform to a C++ code base in both the engine and build system. Some changes in this start: * Requires C++ 11 to build engine. * Simplified build scripts to make it easier to maintain. * Building with C++ optimizations gives an immediate performance improvement. === There will obviously be bumps on the road when that merge goes in. As the C++ 11 requirement might require some changes for testers and users. But hopefully they will be minor. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net

2 1

Re: [boost] [variant2] Review of Variant2 started today : April 1 - April 10
by Peter Dimov 03 Apr '19

03 Apr '19

> Andrey Semashev wrote: > > Also, IMHO, it's better to have libraries more focused and fine grained. > > Why not have `expected` as a separate library? > > expected<T, E...> is basically a variant, with a slightly different > interface. To expand on that a bit... to implement expected<T, E...> you need an underlying variant. You can either implement your own ad-hoc one, which is a duplication of work; or you could use an existing variant, but then you inherit its properties and have to reflect those in the specification of `expected`. For instance, if the variant can be valueless, you need to figure out what to do in `expected` when it gets into that state. For those reasons, it makes most sense to develop `variant` and `expected` in parallel, as parts of the same library.

4 4

Master branch for 1.70.0 closes today!
by Marshall Clow 03 Apr '19

03 Apr '19

We're still in the post-beta phase, so please refer to < https://github.com/boostorg/boost/wiki/Releases%3A-Beta-Merge-Policy> -- Marshall

2 2