[boost] Re: policy-based smart pointers are EVIL!

From: DKl_at_[hidden]
Date: 2002-04-19 07:14:54

> Andrei Alexandrescu wrote:
> "Dietmar Kuehl" <dietmar_kuehl_at_[hidden]> wrote
> >The problem with policy-based designs is that similar entities with a
> similar intention become incompatible. For example, you cannot assign
> a
> smart pointer using an intrusive counter to a smart pointer using an
> external counter (well, if you are really careful this may be even
> workable but in general it is definitely not). That is, for the sake
> of
> interfaces you want to use just one style for an entity.<
> 
> You don't want to assign a smart pointer using one ownership policy to
> a smart pointer using a different, incompatible ownership policy.

Yes, this is also true although from an interface point of view it is
irrelevant whether you want to or not. It is more important that you
can't do this in the first place: If you mix components from different
sources it is pretty likely that different policies are used. As a
result, you cannot [easily and efficiently] use objects from one
component with services of the other component.

>From an interface point of view, you don't want any form of choice!
Everybody passing a certain kind of entity as an argument to a service
shall use exectly the same type. An example where this has gone wrong
in the standard library is the argument to the file stream's 'open()'
function: The obvious choice is to use a 'std::string' rather than a
'char const*' because strings in C++ are expressed as 'std::string'.

The same principle applies to basically everything which is
configurable an may appear in interfaces: If a polymorphic object is
created by some service, it is probably returned using some form of
smart pointer. Of course, if the implementer knows that it is small a
special allocator for small object may be used because it improves
performance. However, the person at the other end may be supposed to
pass this pointer to a function taking a smart pointer configured using
the default allocation procedure but in turn an intrusive counter. And
so on.

For internal implementation of services choice is a good thing such
that it is possible to tune the strategy to the specific needs. When
it comes to interfaces choice is evil!

> Policy-based design allows you to allow exactly the converting
> assignments/construction that you want, and disable all others.

The crucial part is "disable all others": If you have a choice which
configuration is to be used in the interface of some component, you
will end up with combinations which prevent interoperation between
different components! You want seamless interoperability between
different components and hence you don't want a choice - in the context
of interfaces. What happens behind the scenes is the matter of the
component and have different choices there is often necessary.

> > Of course, this is just one side of the coin. The other side is
> > that you definitely want to use different approaches to deal with
> > special situations. For example, you can't use an intrusive smart
> > pointer to maintain a Boolean object but you need an intrusive
> > smart pointer if you want to return a pointer to 'this'. That is,
> > neither approach is workable in both situations.<

> I cannot understand what you say here, so I cannot comment.

Well, I tried to express that it is impossible to have just one smart
pointer configuration which is suitable for all situations. The aspect
under consideration is the counter of a reference counted smart
pointer:

- Obviously, for a 'bool' object you cannot have an intrusive
  reference count because with a total of two values which are possible
  to store there is simply no place for an intrusive count (if you want
  to object that a shared pointer to a Boolean object is non-sensical
  in the first place, consider a Boolean flag shared amoung multi
  objects, eg. a level of detail in a GUI system).

- On the other hand, if a method of class needs to return itself
  (see eg. the XML DOM specification: there are several methods doing
  just this and it is in general not such a ridiculous requirement)
  an external count is no viable option: The object needs to construct
  a smart pointer from itself. This is a situation where an intrusive
  counter is just the obvious choice.

That is, it is impossible to have just one smart pointer configuration
applicable across all kinds of objects: For some an external counter is
the right thing, for other an internal counter. However, with respect
to the counter I would claim that this aspect depends entirely on the
type: If one object of this type can use an intrusive counter, all
other will be able to do so, too. The external counter is need for a
complementary set of types. Of course, local needs may dicate different
choices but these shall not leak to the component's interface.

That is, different policies for smart pointer for different types are
needed and something like the current 'boost::shared_ptr<T>' is not the
right approach (because it tries to make the choice how the counter is
maintained for all types). However, a highly parameterizable smart
pointer is neither because it will cuase different users to make
different choices although the resulting objects are supposed to
interoperate seamlessly.

> > My general feeling is that there needs to be something like
> > 'boost::smart_ptr<T>' which is appropriately configured for the
> > specific needs of type 'T' (eg. intrusive vs. external counters are
> > probably type specific), to application wide preferences (eg.
> > multi-threading support is likely to be configured application wide
> > - at least for most cases), and otherwise useful defaults.
> 
> The problem here is that you don't want to specialize smart_ptr<T> for
> all types that, for example, use COM's intrusive reference counting.
> There are many such types in the application and you don't really want
> to configure smart_ptr separately for each.

Admittedly, I don't want to do this work but I want to get exactly this
effect: I want a type dependent configuration of a smart pointer which
can then be used to interface with other components. Just because the
obvious solution involves lots of work it does not mean that the
requirement does not exist nor that there is not an appropriate
solution (no, I'm not claiming that I have a solution right away).

My point is basically, that for each type there should be a central
configuration (probably involving a reasonable default like something
along the lines of Boost's shared pointer) which is the obvious choice
for a smart pointer as part of an interface. Without this central
configuration we'll end up with lots of mutally incompatible components
(of course, a central configuration for smart pointers is just one
aspect of the overall problem...).

> The threading policy is not application-wide; there are places when
> you want, and places when you don't want, to share smart pointers
> between threads.

... which I have acknowledge in various places in my previous message.
Just to make this crystal clear: There are places for policy-based
smart pointers. Interfaces are just not a place for this! For interface
choices are bad.

> > This would be the type used for passing around pointers to 'T'
> > objects. For specific needs like package-internal maintainance,
> > processing local to functions, or special performance requirements
> > differently configured smart pointers may be used.
> 
> How do you configure those?

Internal to components, a smart pointer like the one in Loki is
probably an appropriate choice and I don't mind policy-based smart
pointers for this.

> > I haven't been following the smart pointer discussions recently so
> > please excuse if this has been pointed out before (well, I guess it
> > probably was pointed out but I think it is worth keeping this
> > mind).
    
> I'm not sure I exactly understood what "this" was. What was the
> point?

Well, basically my point is that neither 'boost::shared_ptr<T>' nor a
policy-based smart pointer nor a combination of both is a complete
solution to the subject of smart pointers: These approaches are all
appropriate for internal processing with a component but they are of no
help when it comes to interfaces which need to pass around smart
pointers (which in turn just means to pass around polymorphic objects,
largs objects, resources, etc. by reference with automated life-cycle
control). The current 'boost::shared_ptr<T>' is inappropriate because
it does not allow the needed configuration and policy-based smart
pointers are inappropriate because they allow too much configuration.

I'm basically in agreement with the statement by Greg Colvin that only
one smart pointer shall be used (as far as I understood he has a
similar motivation as I do, namely to make use of smart pointers in
interfaces viable). However, I disagree with Greg in the approach to
the solution: I don't think that runtime-only configuration is the
right approach. Instead, I'm envisioning more or less a combination of
compile-time and runtime-time configuration:

- Fundamental aspects which are reflected in the type or would prevent
  a configuration at all should compile-time configuration which are
  made central at one place for a type. Examples for this are intrusive
  vs. external counter (if the type has an internal counter, there is
  no point in using an external counter; if it has not, there is no
  choice other than using an external counter) and multi-thread
  protection (for runtime-configuration it would be necessary to
  protect the configuration indicator; well, with some restrictions
  this may be avoided).

- Certain aspects may be configured at run-time like use of allocation
  strategies. Whether this dynamic configuration is supported may be
  a static configuration, however.

- The static configuration is made at a central point for a type.
  This may involve [partial] specialization for the actual
  configuration and possibly some traits to derive a default for
  cases where there is no explicit configuration. The static
  configuration may very well just choose a specific combination of
  policies of a policy-based smart pointer.

I think that something like this can be achieved in a form which
requires configuration of [root] base classes only (conflicts between
different policies inherited from multiple bases are likely to cause a
compile-time error but this is a reasonable approach because the class
would have to choose the system of classes with which it should
interoperate easist). In any case, there shall be *THE* smart pointer
type which is appropriate to be used in interfaces which can be
configured for each type to use a set of appropriate policies to this
type. In addition, there may be other smart pointers for internal use,
too, of course.
--
<mailto:dietmar_kuehl_at_[hidden]> <http://www.dietmar-kuehl.de/>
Phaidros eaSE - Easy Software Engineering: <http://www.phaidros.com/>