Boost logo

Boost :

From: David Abrahams (dave_at_[hidden])
Date: 2005-09-26 21:47:38

"Robert Ramey" <ramey_at_[hidden]> writes:

> David Abrahams wrote:
>> "Robert Ramey" <ramey_at_[hidden]> writes:
>>> So if its not the same discussion but it is related. Of course we
>>> can test less. But the root of the problem is that probably only a
>>> small percentage of the effort invested in testing is actually
>>> testing anything. I know I've brought this up before but made no
>>> headway so I won't harp on it anymore.
>> It's hard to understand your objection to retesting library A in
>> combination with a changed library B on which it depends, since that
>> is essentially what you're doing with the various parts of the
>> serialization with your N x M x K testing.
> LOK, I could also say its hard to understand why you don't object to
> re-testing A in combinarion with B but do object to doing the same thing
> with NxMxK serialization library.

The former is hard to change, and it's not clear that the change is

> But there are some differences.
> The serialization library is still one library with one person taking
> responsability for reviewing the results and reconciling any issues. any
> failure in any combination will be addressed.
> The serialization libary is change only once every few weeks whereas
> something in the whole of boost changed at least once / day. So the impact
> on testing time is much greater from retesting every AxB combination.

Not for those who only do clean test runs.

> Libraries A and B expect to change their public API only very occasionally.
> If the API is being tested independently, then re-testing by other libraries
> should be redundant. This doesn't apply to the internal behavior of any
> particular library. Regarding the serialization library in particular, I
> have strived to make the different aspects indepent and interact with each
> other only through the most narrow of API. This is the basic motivation for
> splitting the view of things in to archive and serialization. Similarly I
> strived to make the DLL versions identical in usage to the static library
> usage and all the archives implement the same API. The reason that we have
> the MxNxKxL(don't forget deug and release !) is that I was successful in
> doing this. Never-the-less, it has been a struggle and now the test
> results show pretty good othogonality except for compiler quirks which show
> some combinations trip compilers ICE's. I don't know that I could have
> arrived at this point without testing all the combinations from time to
> time. Libraries A & B don't have this situation.

It's possible to set things up so you run all the combinations but the
regular Boost test suite runs fewer.

Dave Abrahams
Boost Consulting

Boost list run by bdawes at, gregod at, cpdaniel at, john at