Re: [boost] [testing] Refactoring serialization tests, and possibly others.

From: David Abrahams (dave_at_[hidden])
Date: 2005-09-26 14:39:08

"Robert Ramey" <ramey_at_[hidden]> writes:

> David Abrahams wrote:
>> "Robert Ramey" <ramey_at_[hidden]> writes:
>>
>>> David Abrahams wrote:
>>>> "Robert Ramey" <ramey_at_[hidden]> writes:
>>>>
>>>> Please do what's required to bring the overall Boost testing time
>>>> back down to something reasonable.
>>>
>>> I can do this.  Should this be RC_1_33_0 or head?
>>
>> Both.  We're testing them both pending the release of 1.33.1
>
> If I recall correctly, the idea was that there would be a release of 1.33.1 
> 30 September 2005. I will be leaving town this sunday 2 October.  Do we 
> really want to mess with something that's been this way for a year and will 
> only go on for couple more days at this time?  That doesn't seem
> wise to me.

Maybe not, if 1.33.1 is actually on target for release 30 Sept.  I
don't know if that's the case.

>>> Here is the scenario.  Library X uses something from library Y.
>>> Library Y is changed.  This triggers a rebuild on Library X.  This
>>> in turn triggers a re-build and re-test on Library X.  At least
>>> that's way it looks like it works to me.
>>
>> That is correct, but what you said made it sound like X would be
>> retested unconditionally.
>
>> The idea that we should not be re-testing libraries when their
>> dependencies change is debatable, but that's a different discussion.
>
> Well, if we weren't doing that we wouldn't have a problem.  

Wrong.  Many testers are doing "clean run" testing that forces
everything to be retested unconditionally, specifically to avoid the
sorts of inaccuracies that we'd have if we ignore dependencies.

> So if its not the same discussion but it is related.  Of course we
> can test less.  But the root of the problem is that probably only a
> small percentage of the effort invested in testing is actually
> testing anything.  I know I've brought this up before but made no
> headway so I won't harp on it anymore.

It's hard to understand your objection to retesting library A in
combination with a changed library B on which it depends, since that
is essentially what you're doing with the various parts of the
serialization with your N x M x K testing.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com