Emailed privately by Pavel Vozenilek:

On 7/1/06, Kevin Spinar <spinarkm@gmail.com> wrote:

...

Is there interest in a clone_ptr class to supplement the current smart_ptr library? A clone_ptr is a smart pointer which performs a deep copy of the pointed-to object whenever the clone_ptr is copied. Also, the polymorphic type is maintained, eg:

clone_ptr<Base> p(new Derived); clone_ptr<Base> p2; p2 = p; /* p and p2 now point to different Derived objects */

What is use case for such behaviour?

The boost.ptr_container library probably does address most of the possible use cases for clone_ptr. In fact, a ptr_container containing only one object could be used instead of a clone_ptr. However, a "container of one object" would be confusing in production code. Maybe clone_ptr wouldn't be used enough to warrant including it in Boost. Perhaps Matt Calabrese, since he showed such enthusiasm, could provide a real-life usage for clone_ptr.

A clone_on_write_ptr is something recognisable but why to clone on copy? This should be addressed.

To me it seems clone-on-copy vs clone-on-write is an optimization issue and not a semantics issue. Considering it isn't possible to determine if someone is merely accessing the object instead of modifying it (foo = *myptr; and *myptr = foo; both use the same operator* and the same operator-> is used regardless whether the member function is const) I don't believe clone-on-write would significantly reduce the number of copies that are made. On 7/2/06, Matt Calabrese <rivorus@gmail.com> wrote:

As a suggestion, if you weren't going to do so already, use the pointer containers' clonable concept to get the cloning functionality required for copies (

http://boost.org/libs/ptr_container/doc/reference.html#the-clonable-concept ).

Yes, that is something I'll look into.

On 7/3/06, Thorsten Ottosen <thorsten.ottosen@dezide.com> wrote:

As long as C++ does not have move-semantics, the semantics of clone_ptr seems to be far too expensive to be useful: std::vector<clone_ptr<T>> will have to re-clone all objects on buffer expansion, whereas boost::ptr_vector<T> will not.

Firstly, while it is true that it may be expensive to copy such objects for some users, it certainly isn't "far too expensive" for all users, as many programmers likely are simply not bound by the copy operations of the types they may be using with it -- an assumption either way made at the library level, in my opinion, is a mistake in design. Let the user decide what's too expensive and what isn't, as at the library level you simply don't know every user's requirements. As well, keep in mind that we are talking about a trade-off and a change in semantic meaning, not an optimization. If value semantics are desired for a given type, a programmer shouldn't have to jump through hoops using a type with reference semantics and explicit cloning in order to get the functionality they want -- this is refering to the types in general, not just with respect to them being stored inside of containers. But the big point here is that, apart from all of this, even without move semantics in the language, efficient containment can be accomplished very easily in the form of compliant STL containers, and I'd argue that it can be done in a simpler and more intuitive manner than what is provided by the current pointer containers and even be exactly as efficient! This can be done by just creating the clone_ptr template and then partially specializing the standard STL containers rather than working with ptr_container containers, as is explicitly allowable per 17.4.3.1 paragraph 1 of the standard. Specializations would be fully compliant to the standard, while also providing many of the ptr_container optimized operations on top of the standard operations. The benefit of this is that the user of the library is really only introduced to one new template, the clone_ptr template, and they can use that type with any STL container they wish, even with ::std::vector, and the implementation may avoid unecessary copies just like ptr_vector. The user doesn't even have to be aware of the additional member functions unless they need such functionality. This also has several other benefits over the current ptr_containers, particularly with respect to writing generic code, as instantiating a vector of clone_ptrs can be done exactly as you would a vector of other types, making the optimization entirely transparent to the writer of the generic code. For instance, imagine a type template whose instantiations encapsulate an ::std::vector of a type which is specified through an argument of the encapsulating template. As an example: template< typename Type > struct some_type { ::std::vector< Type > container; }; Now, imagine that someone instantiating the template wishes to have the internal container contain "animals," where animal is an abstract base class and the user would be storing dynamic instances of cats, dogs, etc. which are derived from the animal type. This could be desired for some of the same reasons you may wish to have a container of variants, but with the restriction of used types being those which are any possible children of a specified type rather than types which must be individually explicitly specified when instantiating a variant. With the clone_ptr concept and the associated specializations, the person would merely instantiate the encapsulating some_type template with clone_ptr< animal >, and internally the optimized vector implementation would be used automatically without any explicit intervention on the part of the person using the template nor the person writing the template. Without a clone pointer type of concept, it is much more difficult to make the code both generic and efficient. Specifically, if the above functionality were desired, the user would likely have to instantiate some_type with shared_ptrs and then work with reference semantics in order to get a simple and exception-safe solution, or the creator of the some_type template would have to make some way for users instantiating the template to specify the semantics they desire and then internally use a ptr_vector in cases where it would be acceptable to do so. Even then, since ptr_containers have subtle differences from their respective STL containers, additional steps would generally have to be taken in order to make the code consistent regardless of arguments passed to some_type. So with clone_ptr, the optimizations stay just as optimizations in that they do not affect the way generic code is written unless even further optimizations are required. Finally, since clone_ptrs are copy constructable and assignable, they can be used in any containers developed by other programmers right out of the box. If optimizations are needed, specializations can be made for the containers rather than entirely new ptr_containers, and all of the benefits from above once again hold true -- optimizations remain as implementation details rather than separate containers and any code already written does not have to be changed in order to take advantage of the optimizations. -- -Matt Calabrese

On 7/3/06, Thorsten Ottosen <thorsten.ottosen@dezide.com> wrote:

Well, we usually prefer "as efficient as possible" for libraries.

Again, this really is "as efficient as possible" for the given concept being modeled, and with specializations it provides just as efficient storage in containers as ptr_containers, only with the benefit of being true STL containers which are optimized and automatically used when working with a clone_ptr instantiation. On 7/3/06, Thorsten Ottosen <thorsten.ottosen@dezide.com> wrote:

If value semantics are desired, then why not provide them directly? Having polymorphic types with value semantics is fairly rare.

I'd say fairly rare simply because the tools to make it simple and efficient were never there. Think of a clone_ptr much more like a "smart object" rather than a "smart pointer," and its uses more similar to a nullable variant (note that nullability could even be toggled on and off via a template argument), only whose possible types are restricted to children of a specified type rather than by a finite number of explicitly specified types. This concepts makes working with polymorphic types much more consistent with the rest of the language. If you write a generic piece of code which takes in an instance of a type and stores a copy of it, it is simple to write generically in a way which may work intuitively for scalar types and most user-defined types -- all you do is take a reference to a const instance of the type and then use the copy constructor or assignment operator, nothing strange there. Now, let's say you have an abstract base class "shape" and you want to have that generic piece of code store instances of squares, circles, triangles, or any number of different types derived from shape. Again, this is not a horribly strange or uncommon concept. With clone_ptr, all you do is instantiate the type in question with clone_ptr< shape > and you get the value semantics you want. That's it -- no extra manually created types, and you don't have to make a special case in the generic code. In short, as described earlier, this is because clone_ptrs provide a simple way to have polymorphic types be used more consistently in generic code when compared to any other types in the language. Copying constructing a clone_ptr< shape > intuitively implies copying a shape, regardless of whether it is a square, circle, triangle, etc., just like copying an int means copying an integer. You get value semantics by default so that you may use such types in the same manner that you would any other types when working with generic code that is able to work with non-dynamically allocated objects with value semantics. I know I keep repeating myself, but I'm not sure exactly what it is that you don't understand. It is a lot more work to write an efficient/correct standard container

library, than to write the pointer container library on top of the standard library. So I don't agree with "simpler" here.

Simpler for users, not necessarily for the programmer of the library. Anyone who knows the STL knows how to use a given type in an STL container. Using a clone_ptr in an STL container can be done with the same code one would write with any other type, though with an optimized internal implementation entirely transparent to the user. All a programmer needs to learn is the interface of a clone_ptr and they get container optimizations without any extra work. Then, if they really need to optimize further, they can look at additional operations provided by the specializations.

Without a clone pointer type of concept, it is much more difficult to make

...

the code both generic and efficient.

Depends on your definition of efficient.

Alright, then by what definition of efficient are you refering to? It's just as efficient as using a ptr_container concerning speed and memory usage and it is easier to use in generic code when such semantics are required. Keep in mind that this is not intended as a replacement for ptr_containers, I just offer it as a suggested replacement for a subset of usage of them which can potentially make working with generic code much easier for both library writers and users.

optimizations remain as implementation details

...

rather than separate containers and any code already written does not have to be changed in order to take advantage of the optimizations.

If you think you doing programmers a favor by muddling the distinction between value semantics and polymorphic semantics, I think you're wrong.

I appreciate your criticisms, but I still disagree entirely. I believe you are missing the intended purpose of such a template and associated specializations. -- -Matt Calabrese

On 7/4/06, Thorsten Ottosen <thorsten.ottosen@dezide.com> wrote:

I think the idea of specializing the entire container library is very bad, to say the least. For example, std::vector< std::vector< clone_ptr<T> > > will still behave like a super slow beast without move-semantics.

Again, calling such an operation "super slow" and "a beast" is entirely premature and is up to the user to decide. Copying vectors or lists is always potentially a costly operation, regardless of whether or not it is a container of dynamically allocated elements -- this is not some special case. Further more, if a user decides that a vector of vectors of clone_ptrs is too costly for their given situation, they can do similar to what they would do with ptr_containers, only instead of making a ptr_container of ptr_containers, they make it a container of clone_ptrs of containers of clone_ptrs. Again, with this you can get the same performance as a ptr_container. Aside from all of this, the example you are giving -- resizing beyond the capacity of the vector, is generally a non-issue anyway as people who are concerned about speed here most-likely wouldn't be optimizing a resize-beyond-capacity operation, but rather they would not resize beyond the capacity, just like they would avoid it with any other type in times where they are concerned about speed. In cases where having a constant capacity is not feasible, they would use a deque instead, again, just like they would with other types. Short of all of that, they can go further and have a container of clone_ptrs of containers and get the same overall functionality that they would with a ptr_container of ptr_containers. No new problems, no new performance issues, no new containers. Optimization is fine, and again, all of this allows for just as efficient code as ptr_containers, but the difference here is that the syntax for doing so is consistent with the rest of the language and is much easier to use in order to produce efficient generic code, and it doesn't require the user to learn additional container types. To me it seems win-win for almost all uses, having very few down-sides. -- -Matt Calabrese

Thorsten Ottosen wrote:

If you not willing to accept that value-based programming and OO programming are different disciplines, we will never agree on this one. Period.

Entity/identity-based and value-based are different, no question. What is controversial is your claim that the mere presence of a virtual function by necessity implies that the identity of the object is important (as opposed to its value). To give an example, in the texbook OO hierarchy Shape -> Circle, Square, ... the identities of the objects play no role, they are pure values. You can copy a collection of shapes and you'll have a perfect replica of the original.

On 7/5/06, Thorsten Ottosen <thorsten.ottosen@dezide.com> wrote:

I think it's ok to say boost::function is a *not* a very common type. It's one of a kind.

What makes you say that? I've used such behavior before and Dave Abrahams claims he's used it several times. Perhaps in your particular work you don't need such behavior very often, but there is nothing logically wrong with it. In OO programming, it is common to deep-clone a little here and there.

It's not common to do it all the time.

No one is saying you should do it all the time. If you want reference semantics you may use a shared_ptr or similar solution, and if you need value semantics you use a clone_ptr or similar solution. On 7/5/06, Thorsten Ottosen <thorsten.ottosen@dezide.com> wrote:

I can't imagine what kind of generic code you're talking about.

The type of code generic code that I already posted an example of. For instance, you have a template parameterized by type which internally uses value-semantics with said type, perhaps even having a container of a parameterized type in the implementation. Then, in a given situation you wish to use that template in order to work with varying children of a specific polymorphic type. With a clone pointer type, all you have to do is instantiate the template with a clone_ptr of the desired type and everything works fine. Not only that, but it is also efficient at runtime, since even if, for example, an STL container of them is used in the implementation, the respecitve specialization is also automatically used. Without a clone pointer, you pretty much hit a wall, at least in terms of simple solutions. One option is to manually create a type which works with the polymorphic type internally and has value semantics (which is exactly what clone_ptr would do automatically for you), with the down-side being that you don't get optimized container implementations unless you also make those yourself. Another solution may be to add a template parameter which may be used as a way to specify that you wish to work with dynamically allocated instances of the type and have value semantics, which forces an unecessary burden on the coder of the template and is more complex than simply instantiating the template with a clone_ptr type. Finally, even if the user is able to alter the template in order to give it such functionality as was just described, in order to have optimized code you would have to use explicit template metaprogramming in order to take advantage of ptr_containers -- using them only when such dynamic value semantics are required. Even then, due to the differing interface of ptr_containers from regular containers, you need to do quite a bit of specialized internal coding to be sure that everything works consistently between instantiations which use basic containers and those which use ptr_containers. Keep in mind that you'd have to do all of this just to get the same functionality and efficient code that you would get by using a clone_ptr. If you not willing to accept that value-based programming and OO

programming are different disciplines, we will never agree on this one. Period.

Until you understand that object oriented programming is an entirely orthogonal issue from programming with value semantics or reference semantics, we will never agree on this one. There is absolutely nothing about object oriented programming that logically restricts it to reference-based programming, nor has there ever been such a restriction. I'm not sure how you are coming to such a conclusion. -- -Matt Calabrese

On 7/6/06, Thorsten Ottosen <thorsten.ottosen@dezide.com> wrote:

The problem I have with that is that such "generic" code cannot assume anything about the stores type. It must be manipulation of the container itself.

Even for such tasks I can easily imagine problems depending on how you specify clone_ptr. (For example, does < compare the pointee=).

No, it doesn't compare the pointee. You still work with the object through indirection. Performance-wise, even doing something like

container::value_type v = cont.back(); cont.push_back( v );

is going to be very efficient for vector<int>, but very costly for T=clone_ptr<U>. So the amount of generic algorithms you intend to write at the container level is certainly not easy.

And? You have the same "problem" with any complex user-defined type. This is not a special case for clone_ptrs. Lot's of types internally use dynamic memory allocation or have otherwise complex copy operations -- this is nothing new. So what happens with those complex user-defined types? Such code still compiles, as it should, it works correctly, and it remains up to the user to decide if it is fast enough for his given situation. clone_ptr has all of the same characteristics as any relatively complex object used in generic code, only it also has the benefit of easily allowing for specializations of generic code which use instantiations of it, not to mention the fact that it allows one to easily adapt polymorphic types to be able to be used in such generic code to begin with. And consider sort() on a collection of clone_ptr<T>. This will be

immensely expensive without move-semantics whereas ptr_vector<T>::sort() stays efficient.

Like your other statements, "immensely expensive" is entirely relative to the situation and you should not be assuming at the library level that no one should ever do it. It logically makes sense, it's not slow for all situations, and it makes generic code much easier to write. Not only that, but once again, sorting a range of clone_ptrs is no more inneficient than sorting a range of any relatively complex type. What's more is, it's also entirely possible for the clone_ptr library to specialize std::sort for standard container iterators of clone_ptrs, therefore once again giving you just as efficient code as ptr_containers, but with entirely generic code. And I'm still convinced that specializing the entire container library

from std:: is a very bad idea, maintenance wise and header-wise. Your header for clone_ptr would be *huge*, because if you don't include all container specializations, one risk that the specialization is not picked up.

At least we agree on this. I have worked with C++, Java and C# for years, also on big projects, and

object cloning have never been a major part of the code, although the code was OO all the way. Never. Once in a while you need to copy the entire colelction of shapes or what-ever. But it's never the main application.

Right, it's not often that it is a major part of a given project, though the fact that you say "major" does show that it does happen and has even happened to you. Sure it's not needed for every single project, but such is the case for most libraries. I've only found myself using variants in a few projects, I've never used boost::any, I personally don't have a need for uBLAS, and I don't use all of the algorithms in the standard library. Just because a library or feature of a library isn't a central part to many applications does not mean that it is not useful and it certainly doesn't mean that it is not logically correct nor that its functionality should be excluded. -- -Matt Calabrese

On 7/6/06, David Abrahams <dave@boost-consulting.com> wrote:

Second, you're not allowed to introduce overloads to std::. Third, the overloads might not match due to non-deduced contexts.

True, my mistake. -- -Matt Calabrese

Thorsten Ottosen <thorsten.ottosen@dezide.com> writes:

...

and it doesn't require the user to learn additional container types. To me it seems win-win for almost all uses, having very few down-sides.

If you not willing to accept that value-based programming and OO programming are different disciplines, we will never agree on this one. Period.

An absolutist perspective can be useful for establishing programming discipline when we're still sharpening our basic skills, but it rarely serves to make better programmers in the long run. Yes, OO programming and value-based programming have differences, but they are not totally distinct: they both overlap and synergize, and the intersection is commonly used, interesting and valuable. Stuff like http://www.gotw.ca/gotw/062.htm doesn't appear out of thin air; people get interested in these patterns because they are useful in solving real problems. -- Dave Abrahams Boost Consulting www.boost-consulting.com

David Abrahams wrote:

Thorsten Ottosen <thorsten.ottosen@dezide.com> writes:

...

Matt Calabrese wrote:

...

On 7/3/06, Thorsten Ottosen <thorsten.ottosen@dezide.com> wrote:

...

As long as C++ does not have move-semantics, the semantics of clone_ptr seems to be far too expensive to be useful: std::vector<clone_ptr<T>> will have to re-clone all objects on buffer expansion, whereas boost::ptr_vector<T> will not.

Firstly, while it is true that it may be expensive to copy such objects for some users, it certainly isn't "far too expensive" for all users, as many programmers likely are simply not bound by the copy operations of the types they may be using with it -- an assumption either way made at the library level, in my opinion, is a mistake in design.

Well, we usually prefer "as efficient as possible" for libraries.

I don't know who "we" is supposed to refer to. Your philosophy seems to be the same as the one you used to forbid copying of ptr_containers: if you think the user _might_ be unhappy with the cost, don't provide the capability through the usual interface. This approach has no precedent in Boost or the STL.

Right, because neither Boost nor the STL has ever dealt with anything but value-semantics.

...

...

If value semantics are desired for a given type, a programmer shouldn't have to jump through hoops using a type with reference semantics and explicit cloning in order to get the functionality they want -- this is refering to the types in general, not just with respect to them being stored inside of containers.

If value semantics are desired, then why not provide them directly? Having polymorphic types with value semantics is fairly rare.

Only because it's more work to implement than it should be. I've had to do it several times. And, just to point at one familiar example, see boost::function.

I fail to see the relevance of boost::function is this context. It's not exactly exposing any virtual functions in its interface. -Thorsten

Matt Calabrese wrote:

On 7/4/06, Thorsten Ottosen <thorsten.ottosen@dezide.com> wrote:

...

I fail to see the relevance of boost::function is this context. It's not exactly exposing any virtual functions in its interface.

The relevance is that it is an example of a type that has value semantics and that logically encapsulates different dynamic types that all share a common interface. This is the exact concept that you refer to as "extremely rare"

I think it's ok to say boost::function is a *not* a very common type. It's one of a kind. In OO programming, it is common to deep-clone a little here and there. It's not common to do it all the time. -Thorsten

David Abrahams wrote:

Thorsten Ottosen <thorsten.ottosen@dezide.com> writes:

...

...

Only because it's more work to implement than it should be. I've had to do it several times. And, just to point at one familiar example, see boost::function.

I fail to see the relevance of boost::function is this context. It's not exactly exposing any virtual functions in its interface.

Its operator() might as well be a virtual function. The fact that it's implemented differently is merely a matter of special-purpose optimization that wouldn't be worth the trouble for the vast majority of similar classes. I know that won't make an impression on you, because you don't seem to think there are many similar classes... but there are.

Where? -Thorsten

David Abrahams wrote:

Thorsten Ottosen <thorsten.ottosen@dezide.com> writes:

...

...

Its operator() might as well be a virtual function. The fact that it's implemented differently is merely a matter of special-purpose optimization that wouldn't be worth the trouble for the vast majority of similar classes. I know that won't make an impression on you, because you don't seem to think there are many similar classes... but there are.

Where?

In the literature at the very least. Read the GoF book or look up the pimpl pattern.

Which pattern in the GoF book are you referring to? As for pimpl's, I still don't see the conenction. Where does virtual functions even play a role in pimpls? -Thorsten