Re: [boost] Interdependence of Boost libraries

From: Ullrich Koethe (u.koethe_at_[hidden])
Date: 2001-11-13 14:57:57

Hi all,

I'd like to make some general remarks about interdependence of
libraries. First, I think that boost has grown to a size where
dependency management really becomes an issue. We can no longer consider
boost a single module, but should rather think of it as a collection of
modules. How much interdependency between these modules can we tolerate?
How should these dependencies be realized technically? These questions
must be answered very systematically for boost to remain usable and easy
to learn. 

Of course, it is well known that the dependency graph must be acyclic.
Otherwise, separate testing of modules with cyclic dependencies becomes
impossible. But this is not sufficient. In a large system, dependencies
must also be _flexible_. I'll explain what this means, and why it is
needed, by an example:

Suppose a programmer implements component A which depends on component
B. B resides in module MB and, in turn, depends on component C. Then,
the programmer of A must always ensure that a suitable C is present so
that B can do its work. Experience shows that this can be very difficult
if the dependency B->C is hard-wired (i.e. inflexible). 'Hard-wired'
here means that

* to replace C with another server C*, the source code of B must be
changed, and
* there is no precise formal specification which capabilities of C are
needed by B.

In contrast, if a dependency is flexible

* it is explicitly stated which capabilities the server must have.
* servers can be exchanged without changing the dependent's source code.

If the latter two statements are true, I say that 'B has a Required
Interface w.r.t. C'. A Required Interface leads to flexible dependencies
because it ensures that a server can easily be replaced. (In contrast,
the usual notion of interface is what I call an 'Offered Interface' - it
formally describes the capabilities a specific provides.)

A typical example for a Required Interface is an iterator category: by
defining an algorithm as a template which accepts _any_ forward iterator
(say), the STL makes the dependency 'algorithm->data structure'
flexible. In a traditional design, the algorithm would specifically
require an 'int *' (say), leading to a hard-wired dependency. Another
example for an RI is a hook method like 'atexit()'.

Dependencies can be grouped into the following cases:

1: The dependency B->C is hard-wired. Then a specific C must be present
for B to work.

1.1: C is located in the same module MB as B. Thus, whenever B is
present, a suitable C is also present. In this case, the dependency can
completely be hidden from the programmer of A.

1.2: C is located in another module MC. Then, the dependency cannot be
hidden because the programmer of A must at least install MC. There are 2
subcases here:

1.2.1: MC is standardised. Then again, a suitable C is usually
available. (But this does not yet apply to boost ;-)

1.2.2: MC is not standardised. Then, MB and MC have independent release
cycles, which may cause incompatibilities.

2: The dependency B->C is flexible. Then, a different server C* can be
used whenever the default C doesn't fit the bill.

Here are some problems of hard-wired dependencies:

a) (occurs mainly in case 1.2.2): Suppose, B is described as 'needs
version 1.2 of C', but I only have v1.3. Without a required interface
specification, I cannot decide whether B will still work with C v1.3.

b) (occurs mainly in case 1.2.2): Suppose, A additionally depends on D,
which in turn depends on C too. But B depends on C v1.2, and D on C
v1.3. Often, one cannot use two different versions of C in the same
programm. Even if its possible, it would be a mess.

c) (occurs in all variations of hard-wired dependencies): Suppose, C
does not quite do what it should from A's point of view. For example,
for debugging I might want C to log B's requests. Or C is called in a
loop, and I want to add a progress bar. Or I want to realize design
patterns like 'Proxy', and 'Decorator'. All this is impossible when
dependencies are hard-wired.

As the dependency graph becomes larger, dependency problems will become
ever harder to spot, let alone fix. 

Now contrast this with flexible dependencies: Problem a) could be solved
by looking at B's Required Interface and then the either use C v1.3
directly, or use it as the basis for an adapter that emulates the
required behavior. Problem b) could be solved similarly, by linking
against v1.3 only, and passing an adapter to B that emulates v1.2.
Proxies and decorators as in problem c) are just variations on this
theme.

In conclusion, I think it will be necessary to
* specify which modules boost consists of
* identify which interdependencies exist (or should be introduced)
* make the dependency graph acyclic
* decide which dependencies should be flexible, and which can/must be
hard-wired (flexible dependencies may be overkill, slow, complicated, or
impossible in some cases)

Regards
Ulli

-- 
 ________________________________________________________________
|                                                                |
| Ullrich Koethe  Universität Hamburg / University of Hamburg    |
|                 FB Informatik / Dept. of Computer Science      |
|                 AB Kognitive Systeme / Cognitive Systems Group |
|                                                                |
| Phone: [phone]                Vogt-Koelln-Str. 30 |
| Fax:   [phone]                D - 22527 Hamburg   |
| Email: u.koethe_at_[hidden]               Germany             |
|        koethe_at_[hidden]                        |
| WWW:   http://kogs-www.informatik.uni-hamburg.de/~koethe/      |
|________________________________________________________________|