From: Robert Ramey (ramey_at_[hidden])
Date: 2005-09-26 16:43:59
David Abrahams wrote:
> "Robert Ramey" <ramey_at_[hidden]> writes:
>> So if its not the same discussion but it is related. Of course we
>> can test less. But the root of the problem is that probably only a
>> small percentage of the effort invested in testing is actually
>> testing anything. I know I've brought this up before but made no
>> headway so I won't harp on it anymore.
> It's hard to understand your objection to retesting library A in
> combination with a changed library B on which it depends, since that
> is essentially what you're doing with the various parts of the
> serialization with your N x M x K testing.
LOK, I could also say its hard to understand why you don't object to
re-testing A in combinarion with B but do object to doing the same thing
with NxMxK serialization library.
But there are some differences.
The serialization library is still one library with one person taking
responsability for reviewing the results and reconciling any issues. any
failure in any combination will be addressed.
The serialization libary is change only once every few weeks whereas
something in the whole of boost changed at least once / day. So the impact
on testing time is much greater from retesting every AxB combination.
Libraries A and B expect to change their public API only very occasionally.
If the API is being tested independently, then re-testing by other libraries
should be redundant. This doesn't apply to the internal behavior of any
particular library. Regarding the serialization library in particular, I
have strived to make the different aspects indepent and interact with each
other only through the most narrow of API. This is the basic motivation for
splitting the view of things in to archive and serialization. Similarly I
strived to make the DLL versions identical in usage to the static library
usage and all the archives implement the same API. The reason that we have
the MxNxKxL(don't forget deug and release !) is that I was successful in
doing this. Never-the-less, it has been a struggle and now the test
results show pretty good othogonality except for compiler quirks which show
some combinations trip compilers ICE's. I don't know that I could have
arrived at this point without testing all the combinations from time to
time. Libraries A & B don't have this situation.
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk