Re: [boost] The Release Manager's Point of View

From: Gennadiy Rozental (gennadiy.rozental_at_[hidden])
Date: 2007-06-07 00:46:21

"Beman Dawes" <bdawes_at_[hidden]> wrote in message 
news:f47qsj$m3j$1_at_sea.gmane.org...
> Thomas Witt wrote:
>
>>... The proposal seems to assume infinite resources in testing.
>
> I must not have been clear. The proposal would do much less testing than
> is currently done. Under the current scheme, testing is done even when
> no one requests a test for a particular library and compiler. Under the
> proposal, tests only get run when someone requests a test. So for
> example, a Windows developer would only likely request tests on
> non-Windows platforms.

1. I don't think it's correct expectation. Even if I run the test locally on 
windows box, I still
  a) don't have access to all configurations
  b) want to run the tests in non-development environment. My local setup 
can be somehow different. I always prefer to run the test though regression 
test suite even if it passes locally.

2. The procedure for on-demand test request doesn't seem to be easy to 
implement and will require significant investment in both development and 
maintenance of the tools.

3. In many (most?) cases I don't have an idea what configurations exists. We 
can't force developers to send dozens of test requests for system they know 
nothing about. IMO regular "shotgun" testing is still going to be in big 
demand and we can't afford not to support it.

In general I am not sure that investment in on-demand testing support will 
give us visible improvement.

Gennadiy