Re: [boost] [tools][bcp] using boost internally in other libraries

From: David Abrahams (dave_at_[hidden])
Date: 2005-09-23 18:09:13

Bjørn Roald <bjorn_at_[hidden]> writes:

> I have been working for some time to get acceptance for using Boost
> libraries as elements in a higher level portable platform for library
> development.  It has surprised me how hard it is to make compelling
> and attractive set of arguments for using Boost (or other libraries)
> without attempting to ignore some pretty serious concerns.  These
> concerns are:
>
> - commercial support for our use of Boost

I know of a certain company offering commercial support for boost :)

> - quality assurance of Boost itself

Specifically?

> - stability of Boost API

Yeah, there are no rules about it.  However, we generally avoid
breaking source compatibility.

> - maintainability of Boost source code

I actually find that most of the time it's pretty easy to dig into a
Boost library and fix things, although I seldom have to because the
authors tend to be responsive to bug reports.  However, there are no
uniform standards for certain things that make maintenance easier for
some people, like code formatting.  But you probably had something
else in mind?

> - deployability of our library into environments using different
>   versions of Boost or libraries used in boost

Hmm, that's a tough one, especially if your library traffics in boost
types in its public API, or if you use compilers that don't support
symbol visibility control.

> - configuration management issues in general for us and our customers

Such as?

> - platform support in Boost ( or if the platform support Boost if you
> like ;-)

Sorry; what do you mean?

> All of these concerns can be declared acceptable, given that you are
> willing to accept trade offs and risk.  The issue I try to raze is if
> it is possible to reduce the magnitude of these risks and trade offs.

Good question.

> One of the key issues is that you have to decide on some strategy of
> how to maintain your boost version.  The most obvious strategy may be
> to stay with the latest Boost release at any time, this will however
> add risk as you have to adopt new changes as they are introduced into
> Boost and not as it fits your own release plans.  The obvious
> alternative is to stay with a Boost release longer, possibly until you
> are forced to switch. This strategy will quickly force you into
> maintaining your own branch as Boost developers can not be expected to
> support old releases.  

A company offering commercial Boost support can support old releases
by integrating fixes into older release branches.  Whether or not
those branches can ever become "official Boost releases" is not clear,
but even if they don't, at least you can share a common set of fixes
with everyone else who is depending on that old release.

> Between these there are many shades of gray which may fit more or
> less well depending on your project and your customers.  If your
> customers themselves use boost you are likely to meet a added
> configuration management, testing and release related cost and risk.
>  
>
> As library developers have a very different type of customer
> relationship than what is the case for end application developers
> there are a number of concerns that make adaptation of Boost, or any
> other 3rd party library for that sake, more problematic for library
> developers.  The main issue is to what degree a library depends on
> other libraries and hence introduces indirect or direct dependencies
> to its users.  These dependencies tend to be troublesome for the
> library developer to support, as well as they may be troublesome for
> users to accept.  In fact users may easily be in situations where such
> dependencies are completely unacceptable.  To help to understand the
> issue I like to define categories of libraries which affect the degree
> they are affected by these concerns.
>
>  
>
> Cat1. open source, source distributions, allowing encapsulation
>
> Cat2. open source, source distributions in general
>
> Cat3. open source, binary distributions
>
> Cat4. closed source
>
>  
>
> Cat1 and Cat2 are build-able source code distributions.  

I believe there are closed source libraries that are also delivered
int the form of buildable source... but that's probably beside the
point.

> This give the user the ability to build the library and ensure that
> it is built correctly with other library it depends on, on the
> target system.  Cat1 add to this the ability to hide the use of it
> inside other libraries, i.e. it can live concurrently in an end
> system with other versions of the same Cat1 library.  

E.g. via symbol isolation or namespace renaming?

> Boost is a Cat2 library.  Even if the extensive use of headers only
> libraries help some of the issues addressed in this post, it is not
> a general solution that makes Boost or part of Boost a Cat1 library.
>
> Now an example showing some dependencies:
>
>  
>
> MyCustomersApplication
>
>   |---- Boost 1.33
>
>   |---- MyLib
>
>           |---- Boost 1.32
>
>  
>
> If MyLib is Cat1 or Cat2 the chances are good that we may be ok.  Most
> likely MyLib will soon adopt the migration to new Boost releases by
> the actions of the maintainer or by patches from end users.  If
> however MyLib is Cat 3 or 4 we have a problem.  This problem is real
> even if the use of Boost in MyLib is strictly internal.  In my view
> this is an especially bad problem.  Even if I do not intend to use
> Boost in any way in the API of MyLib I still have this issue of
> incompatibility.  

Not if you maintain symbol isolation.  Of course, that may not be
possible for you.

> Ok, back to the example.  If MyLib have no intention of using Boost in
> the API, is there a way?  

There are several ways to maintain symbol isolation.  On Windows, it's
fairly easy: simply use a shared library and don't dlexport anything
involving boost.  On *nix, recent GCCs can give you control over
symbol visibility, so you can play the same kind of game.  Otherwise
-- and this is inconvenient at best -- you can do it by using dlopen
to load a shared library containing the boost symbols, and speak to
that library through one or more known entry points.  That approach
can be followed on Windows as well (using LoadLibrary) if symmetry
with *nix turns out to be more appropriate than using the platform
facilities.

> Side Note:
>
> I am aware of symbol encapsulation abilities in windows DLLs and
> similar new features supported by GCC, however these are not generally
> supported on all platforms, so I have selected to ignore them. They
> may however be helpful to many.

Okay, then you're stuck with the "plug-in" approach using dlopen.
>
> The general idea is that there are no references to symbols in the
> boost namespace between any MyLib libraries, nor in any header files
> in or referenced from MyLib API.  How can this be achieved?  I have
> concluded that accessing boost trough namespace wrapping header files
> are unpractical at best.  This needs to be designed into all of boost
> for this to work, and I do not know exactly how.  The main challenge
> is #include <boost/xxxx.hpp> directives which does not refer to the
> wrapping headers.

Other libraries that allow that sort of thing have all of their
headers #include both a prefix AND a suffix header.  It would be a big
change.

> So to wrap up:
>
> I do not believe anything prevents me from going along this path if I
> decide to do so.  I am however interested in the boost developers
> attitude toward this approach, and in particular if this is something
> you will encourage and/or support.  I am willing to work on the tool
> support for this in boost or to maintain a modified bcp tool if that
> is more appropriate.

I have mixed feelings about this.  One of the things some people
complain about when they look at the boost code is that the many
workarounds get in the way of understanding.  In fact, maintainability
was one of the issues you listed at the beginning.  So if we put in
facilities for namespace renaming, our code would get uglier and less
transparent.  Maybe that's not a big deal, as we've already stepped
off the "workaround cliff," but it is a concern for me.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com