[boost] Re: policy-based smart pointers are EVIL!

From: Fernando Cacciola (fcacciola_at_[hidden])
Date: 2002-04-19 11:18:14

Just my 2 cents...

I almost entirely agree with Dietmar. In order to contribute to his
suggestion, I'll expose my own analysis of the matter.

First, I think that it is important to realize that the problem Dietmar is
reffering to is not really about policy-based designs -as an engineering
tool-, but about the particular instrumentation of a policy introduced as an
additional template argument. That is, the 'strategy-oriented design' (a
term I just coined in to generalize the idea of policies) is really powerful
and we are not against this.

I started to realize this when I presented my optional<T> class, and
Gennaidy pointed out that one of the features of the class: its ability to
bypass the default constructor for T (since it's unnecessary) involves a
small overhead, but that this shouldn't be used for types with a trivial or
implicit def ctor, such as a fundamental type. He suggested a policy class
to solve this design choice. I refused, and the reasons why I did that, are,
IMO, the very same reasons why Dietmar is refusing to use a policy-based
approach to a smart-ptr.

The first thing I noticed that would happened if I used a policy *as an
additional template parameter*, that is,  an "intrusive policy", is that

optional<A,Policy1> and optional<B,Policy2> become distinct types.

In the case of optional<>, it is clear to me that this is a mistake, because
these Policies describe an implementation detail of optional<> (whether it
should /shouldn't bypass A's def ctor), so they souldn't be part of the
static-type.

In the case of smart_ptr<> however, it could be argued that a given policy
must be accounted as part of the static-type of the smart pointer.

In any event, whether the fact that some_class<A,Policy1> and
some_class<A,Policy2> are different types is correct or not for the concept
being modeled with some_class<>; this difference introduces some problems
that must be carefully considered.

Dietmar pointed out that smart_ptrs with different policies can not be easly
converted, but this is just the simplest of the problems introduced by
'intrusive' policies (policies that become part of the static type of the
strategy-oriented class).

The really big problems appears when you realize that you need complex
machinery or too much knowldege of the objects you are dealing with in order
to do otherwise simple things. Consider:

Scenario 1:  I want to write a class which does something with a given
source object. This class needs to keep the object alive, but it is quite
possible that the caller disposes the object even before this class
completes its task.  This scenario calls for a common pattern:

class Processor
{
  Processor ( smart_ptr<A> const& obj ) : m_obj(obj) {}
  ...
  smart_ptr<A> m_obj ;
} ;

The important thing in this example is that all I really know of is A, and
that I need it to be alive as long as Proccesor needs it.
If smart_ptr<> is intrusively-policy-based, I -the user of A- must make
decisions about which of those policies would fit here. Some of those
decisions might be hard to take on this side of the system, but even if can
take them, there is still this problem:
Suppose I do:

  Processor ( smart_ptr<A,Policy1,Policy2> const& obj ) : m_obj(obj) {}

what happens with the caller?

Likely, there is a smart_ptr<> holding A objects on the caller side already.
If the caller has a different (in terms of policies) smart_ptr, he could
probably -given the appropriate machinery- do a conversion, but this is
unlikely to scale well, so, most likely, Processor will eventually end up
being parametrized:

template<class APtr>
class Processor
{
  Processor ( APtr const& obj ) : m_obj(obj) {}
  ...
  APtr m_obj ;
} ;

These, however, intoduces a much higher engineering complexity to the
system, because we have changed a non-template class to a template class
only because the smart_ptr is overparametrized (from the pov of the user of
A who just wants to keep it alive).

Scenario 2:

It is quite common to specialize certain things -such as predicates- on a
smart_ptr<>. This is in order to allow certain expressions to be applied
indistinctely to bare and smart pointers.
For example, the boost shared-ptr declares:

// get_pointer() enables boost::mem_fn to recognize shared_ptr
template<typename T> inline T * get_pointer(shared_ptr<T> const & p) ;

This is used -as the comment says- inside mem_fn which allows a very
powerful usage of mem_fn.

If shared_ptr were intrusively policy-based, this wouldn't be feasible
becuase mem_fn couldn't know about the smart pointer policies.

So....

Just as is the case with optional<>, a smart pointer is a library fature so
general that the ability to express a smart pointer as a single-parameter
template class: smart_ptr<T> is so powerful that directly competes with the
power of the flexibility given by a policy-based class.

Intrusive vs external policies:

Since the real problem is not the usage of policies as compile-time
strategic-drivers for the smart ptr, but instrumenting the approach using
intrusive policies, it appears to me that the obvious solution is to use
external policies, as optional<> does.

An external policy is just like any other policy template class, except that
is is identified by a fixed template-id.
A good example of it is std::iterator_traits<>. This is an external policy
class, with a fixed template-id,  so you don't need to pass it as an
argument.

Of course, external policies are a lot less flexible than their intrusive
counterparts.

IMO, whenever a policy is directly associated with a type, that is, when the
choice is determined uniquely by the type, an external policy should be
used.

Dietmar already shown -and I totally agree- that most of the smart ptr
policies are usually type dependent, so they perfectly call for external
policies. (BTW, the sketch made by Doug seems very reasonable).

The problematic case is the threading policy which would certainly loose too
much of its flexibility if it is specified externally in a global basis, as
Andrei warned.

In summary...

IMHO, Dietmar has raised very strong points.

I believe that most of the important policy choices could be made
externally, based only on the type. This will cut-down a lot of the
combinatorial possibilities, but, IMO, will still give a significant degree
of flexibility while allowing the system to retain the smart_ptr<T> simple
signature which I've shown (I hope) to be fundamental.
I don't know, however, what to do with the policies that will loose
flexibility if they are external.
But I do know -or stronly believe, perhaps- that the 'unrolled' syntax of a
smart-ptr (that the user just wraps T with a template-id) is one of the most
important features of any smart pointer.

BTW, I am on middle-ground about having a unique shared_ptr<> as Dietmar
suggest or both a shared_ptr<> and a intrusive_ptr<> as Peter holds.
FWIW, I currently use deaily both boost::shared_ptr and my own intrusive prt
with zero problems.

--
Fernando Cacciola
Sierra s.r.l.
fcacciola_at_[hidden]
www.gosierra.com