From: David Abrahams (dave_at_[hidden])
Date: 2005-09-26 21:47:38
"Robert Ramey" <ramey_at_[hidden]> writes:
> David Abrahams wrote:
>> "Robert Ramey" <ramey_at_[hidden]> writes:
>>> So if its not the same discussion but it is related. Of course we
>>> can test less. But the root of the problem is that probably only a
>>> small percentage of the effort invested in testing is actually
>>> testing anything. I know I've brought this up before but made no
>>> headway so I won't harp on it anymore.
>> It's hard to understand your objection to retesting library A in
>> combination with a changed library B on which it depends, since that
>> is essentially what you're doing with the various parts of the
>> serialization with your N x M x K testing.
> LOK, I could also say its hard to understand why you don't object to
> re-testing A in combinarion with B but do object to doing the same thing
> with NxMxK serialization library.
The former is hard to change, and it's not clear that the change is
> But there are some differences.
> The serialization library is still one library with one person taking
> responsability for reviewing the results and reconciling any issues. any
> failure in any combination will be addressed.
> The serialization libary is change only once every few weeks whereas
> something in the whole of boost changed at least once / day. So the impact
> on testing time is much greater from retesting every AxB combination.
Not for those who only do clean test runs.
> Libraries A and B expect to change their public API only very occasionally.
> If the API is being tested independently, then re-testing by other libraries
> should be redundant. This doesn't apply to the internal behavior of any
> particular library. Regarding the serialization library in particular, I
> have strived to make the different aspects indepent and interact with each
> other only through the most narrow of API. This is the basic motivation for
> splitting the view of things in to archive and serialization. Similarly I
> strived to make the DLL versions identical in usage to the static library
> usage and all the archives implement the same API. The reason that we have
> the MxNxKxL(don't forget deug and release !) is that I was successful in
> doing this. Never-the-less, it has been a struggle and now the test
> results show pretty good othogonality except for compiler quirks which show
> some combinations trip compilers ICE's. I don't know that I could have
> arrived at this point without testing all the combinations from time to
> time. Libraries A & B don't have this situation.
It's possible to set things up so you run all the combinations but the
regular Boost test suite runs fewer.
-- Dave Abrahams Boost Consulting www.boost-consulting.com
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk