|
|
Boost Testing : |
From: Rene Rivera (grafikrobot_at_[hidden])
Date: 2007-10-05 16:02:07
Beman Dawes wrote:
> The release branch has been created, using Version_1_34_1 as the
> starting point. The URL is //svn.boost.org/svn/boost/branches/release.
>
> For now, no one except the release management team should commit to the
> release branch.
>
> We need to set up the infrastructure and start testing on this branch.
>
> Rene has agreed to be testing manager; this is really just precognition
> of the role he has already been playing. So this post is really a
> question to Rene: who needs to do what to get us started?
First task is to test that regression.py can handle testing the release
branch. I.e. someone takes a shot at running one set of tests. This of
course immediately means we have to merge into the release branch the
testing infrastructure changes: bjam, BB, and the regression tools.
Which raises the question of how we should manage changes to the tools
for the release? Bjam is easy as I made the recent release so it could
be frozen for the Boost release. For others how should we approach the
process? Develop on the trunk and merge to release? Or the other way
around? Or should we move the tools out of the release tree, as I
suggested some time ago?
> My preference is that testers currently testing on trunk don't have to
> do anything unless they want to. We still need good test coverage on the
> trunk. Any current tester who has the resources is welcome to also test
> on the release branch, or switch to release branch testing.
Hm, we are going to have to be a bit more formal than that. We need some
real criteria for deciding which testers and platforms can go from trunk
to release. In the main list you seem to be suggesting that release
testers need to have consistent frequent and hence stable test setups.
Some possible criteria:
* Can run tests at least X times a day. Definitely once a day is
minimal. But we could require twice a day to have results available
roughly to cover US and non-US timezones, so people have fresh results
during regular working hours.
* The tester can guarantee they can be active (meaning they respond to
emails within a few hours) on this list so we can resolve testing
problems quickly.
Aside from testers themselves, I would feel much better about testing if
we had redundancy on the result processing. Even though it looks like
meta-comm is running the processing again, they seem to be totally
missing from this list.
-- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo