Re: [boost] 5 Observations - My experience with the boost libraries

Subject: Re: [boost] 5 Observations - My experience with the boost libraries
From: John Phillips (phillips_at_[hidden])
Date: 2010-03-24 00:25:37

Tom Brinkman wrote:
> 
> What is the point of having the source code, when it take an experienced
> developer many weeks to understand what the library is doing underneath?
> 
> What is the practical value of a library that can only be maintained
> by the original developer(s)?
> 
> This issue has surprised many, but has comes up often.
> 
> If it takes more than a day to examine a libraries source code and
> have some idea what its doing, its very dangerous to use.  This issue
> will cause many to avoid complex templated macro-style boost libraries.
> 
> 
> 
> Tom Brinkman

   Tom,

   I disagree with the underlying principle of this argument, and I want 
to expand on why.

   For some reason, people believe that because Boost comes with source 
code, they can only use it if they can both read the source and maintain 
it. This strikes me as contrary to the whole idea of a library.

   To my understanding, the reason to use a library is that the 
underlying problem either is too cumbersome to allow for a new solution 
to be built in a reasonable time, or because the solution contains some 
set of subtle pitfalls that will lead a new development to make errors 
that will not be obvious to diagnose or fix. In such a case, you off 
load the work and specialized understanding needed to someone else, and 
use what they produce.

   When using a library of any sort, there should be a limit to the 
blind trust applied. You should combine whatever tests are provided with 
the library along with tests created in house to assess the suitability 
of the library for the intended use. If the library passes all the tests 
you come up with that address your needs, and it passes the tests that 
the developers came up with to address their vision of t=what it should 
do, then you are assured that it is at least as good as anything you 
would develop in house. So, you use it.

   Only if it fails at providing some identified need do you need to 
worry about how to modify it and maintain it. (Almost all software does 
fail, sooner or later, but until it does there is nothing to fix.) If it 
is failing before you even try to use it, you contact the developer and 
try to work toward improvement.

   If it happens to be simple, you can try to modify it yourself, but 
this is always a dangerous course of action. After you create in house 
modifications that your program depends on, you have cut yourself off 
from all of the future development of the library without a potentially 
large later development and merge effort. So, it is almost always better 
to work with the providers of the library to get your requested fixes to 
be a part of the future releases instead of making them independently 
and hoping they will be compatible with future releases from a provider 
who knows nothing of your changes. In general, this is a case of relying 
on knowing the internal implementation of the library, instead of 
depending on the provided interface. Almost no software promises 
anything about the details of the internal implementation from one 
release to the next, and it is almost always foolish to assume you know 
what it will be.

   That means, good development practice implies that you never plan to 
provide maintenance for a third party library wholly in house. So, any 
argument that starts from wanting to provide it in house is already 
starting from bad practice. From that foundation, it has no persuasive 
power.

   The truth is, third party libraries are tools provided by someone 
else. When I use tools provided by someone else, I have a responsibility 
to test them to generate reasonable assurance that they fulfill my 
needs, and to work with a source that I think will be responsible in 
those cases where new flaws are exposed. This is no different from using 
closed source third party tools such as some of the Intel numerical 
libraries, or the MS compilers. I might be able to hack the binaries to 
do what I want (I work for a company that provides tools for analyzing 
binaries and for modifying them, so this is possible. It is just 
unlikely to work right.), but I would be making a mistake to try. If I 
don't think Intel or MS are responsible enough to work with me when 
flaws are discovered, then I should find ways not to use their tools. If 
I use their tools, then I am committed to working with them to remediate 
flaws, and I do not expect to be able to fix them independently, nor do 
I think it would be a good course of action to do so if I could.

   So, a valid argument would be to say you don't think the Boost 
developers would be responsible about trying to address problems you 
exposed. If you believe that to be true, then that is a major problem 
with Boost that we really do need to address.

                        John