[boost] Traits-based design and ODR

From: Andrei Alexandrescu (andrewalex_at_[hidden])
Date: 2002-04-29 18:14:29

This is a long message that assumes a fair amount of context (namely, the
smart pointer discussion that's been around for a while). If you do not have
a particular interest in that discussion, you might be better off skipping
this post.

In a previous post of mine, I briefly mentioned that Dietmar's advice
(against policy-based design in the case of smart pointers) is wrong.
Dietmar mentioned that a traits-based design is much preferrable, especially
when it comes about sharing pointers between libraries written in
separation.

My opinion, after giving the issue some more thought, is that the advice in
favor of traits-based design is wrong, and that policies are preferrable. I
won't repeat the arguments I've already mentioned, but recently I realized a
couple of new ones, which I'd like to point out. I hasten to mention that I
haven't really been looking for "pro-policies" or "anti-traits" arguments,
but simply compare the two techniques. My interimary conclusion is that
policies are not only more flexible and source-code-savers, but also that
they are inherently safer than traits. The latter point stems from the One
Definition Rule, and I'll detail that point below.

But first, please allow me to make another remark. It has been said that
having a unique customizable smart pointer is ineffective for code
interfacing, mainly for a simple psychological reason: when people have many
options, they tend to use them. Hence, given that a traits-based smart
pointer would offer many useful pre-canned behaviors, library writers would
tend to take advantage of them. Potentially, some would even define their
own policies. Allegedly, this "too much flexibility" would inexorably lead
towards a Babel tower of miscommunication between various pieces of code.
The solution, the argument goes, would be a one-parameter smart pointer that
would be the lingua franca for all the builders in Babel. The nice part is
that that smart pointer could still be configured via traits. So we get the
best of all worlds.

But this solution doesn't solve the problem; it merely transports it in
another dimension. People do need smart pointers with different
capabilities; that's a fact. What policy-based smart pointers achieve with
policies, other smart pointers achieve by simply creating new types.
Consequently, today there are a plethora of smart pointers in boost, and
proposing new ones is a favorite, and ever-recurring, theme of discussion.
(After all, it's so easy to create new smart pointers - you copy a lot of
code from an existing smart pointer and you tweak some aspect.) If you could
please allow a bit of bragging, there's no salient variation that can't be
accomodated by yours truly's smart pointer design (modulo bugs), design that
is two years old.

So, given so many smart pointer types, why wouldn't library writers choose
whichever they find fit for their taste, thus again falling in the same
tarpit that we tried to avoid by dropping policies? Then, the anti-policy
argument continues, this is a documentation issue. The smart pointers
documentation will clearly states that of all pointers, shared_ptr is the
only one recommended for interfaces; the rest are better suited for
implementation. But then, a pro-policy proponent could say, you didn't solve
the problem; you took it in the documentation space. There, a policy-based
design would not only do just as good, it actually does /much better/. The
documentation for a policy-based smart pointer could simply say that the
recommended type for interfaces is the smart_ptr with all defaulted
policies. This documentation is nicely supported by the code itself, which
is not the case for the design without policies.

In essence, in the above I tried to prove that the argument against policies
mentioning the library miscommunication issue is moot.

But there's actually more to it. As I have explained in my post "Re:
Policy-based vs. traits-based design", policies offer a combination of
efficiency and flexibility when it comes about copying smart pointers
featuring different policies. This comes from the way objects are copied
(policy-by-policy copying). Combined with converting constructors and
template constructors, this copying strategy offers ultimate efficiency and
flexibility in copying policy-based objects.

On to the actual body of this post, which will seemingly be shorter than the
introduction. It has been mentioned that traits are an effective technique
for dealing with customization of smart pointers in the absence of policies.
There's a problem associated with such an approach, and it's akin to the One
Definition Rule.

In essence, the One Definition Rule states that the same type name should
have the exact same definition throughout an application, otherwise the
effects are undefined.

Now consider we have an application composed of three modules written
independently. One defines a class Widget:

// File widget.h
class Widget { ... };

The second defines a function:

// File produce.h
#include "widget.h"
#include "smart_ptr.h"
shared_ptr<Widget> ProduceWidget();

and the third defines, again, a function:

// File manipulate.h
#include "widget.h"
#include "smart_ptr.h"
void ManipulateWidget(smart_ptr<Widget>);

So far, so good. Now, let's say we follow the advice of customizing some
aspect of smart_ptr for Widgets by using traits. (This, of course, assumes
certain design hooks in smart_ptr.) Say smart_ptr uses a trait called
ownership_trait<T>, which says in which way a type T is owned. Then, that
trait would be specialized for Widget:

template <>
struct ownership_trait<Widget>
{
    ... Widget-specific refcounting primitives ...
};

The question is, where to plant the policy? The fundamental problem is that
the code that doesn't see the specialized version of ownership_trait will
still compile, is likely to link, and sometimes might even run. This is
because in the absence of the explicit specialization, the non-specialized
template kicks in, likely implementing a generic refcounting scheme that
works for Widget as well. Consequently, if not all the code in an
application sees the same definition of ownership_trait<Widget>, the ODR is
violated.

The following approaches to solving the problem all fall short:

1. Best place is to put the trait where the type is defined, that is, in
widget.h. This way, the risk of code using the wrong trait with Widget is
diminished, but not eliminated. The technique still leaves the possibility
to use the wrong policy with an incomplete type:

// File neutral.cpp
#include "smart_ptr.h"
class Widget;
void Target(SmartPtr<Widget> sp);
void Forward(SmartPtr<Widget> sp)
{
    Target(sp); // undefined behaviour
}

But the biggest problem with approach (1) is that often widget.h is not
accessible in writing.

2. Put the specialized trait in either producer.h and manipulate.h. This is
clearly a dangerous approach because the two modules might not include each
other.

3. Put the specialized trait in a separate file, and ask everyone who wants
to use Widget to include that file. This is a discipline-based approach and
therefore is error-prone.

No option is good because it makes it very easy to introduce very subtle
errors. The errors are invisible to a classic file-by-file compiler.

By contrast, in a policy-based design this problem doesn't exist. Policies
don't rely on explicit specializations, instead they rely on typed template
arguments for choosing the desired behavior. The largest danger is that two
library implementers choose or define incompatible policies, case in which
the error is signaled at compile time. This issue is the same as two library
implementers choosing two different smart pointers, so by no means
policy-based smart pointers are at a disadvantage.

In my previous posts, I mentioned that I agree that policy-based smart
pointers are better used in implementation rather than interfaces. In light
of all of the above, I am now tempted to say that policy-based smart
pointers are also the best vehicle for interfacing templated code, and that
the alternatives are suboptimal, dangerous, or both. That is, unless I've
made some terrible mistake, case in which I'd be glad to stand corrected.

Andrei