|
|
Boost Testing : |
From: Robert Ramey (ramey_at_[hidden])
Date: 2007-05-21 23:03:11
Gennadiy Rozental wrote:
> Let's say you made two patches to your library after last
> release. If I want to run regression tests against your first patch I
> can't do it. The HEAD contains second patch and last release has
> unpatched version. I need to be able to somehow refer to your first
> patch in my Jamfile, so that test runners can test against it.
Hmm - that seems no different than the current situation if
I load two separate batches of patches to the HEAD.
>> If they want to wait for the next "official" release its fine. My
>> procedure
>> won't effect the official release date/timetable in any way.
>
> How can you be sure? If you make changes and no one is testing, there
> is no information. But If all continue to test against HEAD what is
> the difference with what we had before?
I'll continue to upload changes to the HEAD as I always have. Those
who find the current testing useful will find it as they always have.
>But at some point we need to combine it all together
> to make another boost release.
I never said "instead". Truth is testing against the HEAD is not
really useful to me. If others find it useful for their purposes
great.
> And if all test against last release instead of HEAD we may have problems.
But now you're getting to the really interesting aspect of my proposal.
Suppose all the developers did this. What would
boost look like.
a) Suppose we started with a well validated release version like 1.34.
b) a month after release the author of some new library such
as Boost Units announces that he's made all his changes, tested
against the last release and checked into the HEAD. Now some
users are just not going to wait a year for the next official
boost release. They say - well, its tested against the release
I'm using, I'll ask the author for the patches and put it in
my tree. Which the author will likely do. Aftar a few
occasions of this, someone might say - well what the hell.
let's just add this on to 1.34 and call it 1.34.2 and it would
be a good idea. Everyone would be better off.
c) A month later another announcement is made and the same
thing happens.
d) Now the testing on the HEAD is proceeding as usual but
its less relevant. In fact, the only testing that is really interesting
now is that which occurs on a release candidate which
validates interface breaking changes and compiler bugs.
e) So finally everyone concludes on a simple new policy.
i) each library proceeds at its own pace tested on its own branch
and / or machine.
ii) and added to the the release candidate one at a time
iii) the release candidate is re-validated each time a libary
added. When if all of boost passes, the release taged
as official and the cycle starts anew.
This results in:
a) more frequent releases.
b) in smaller increments
c) library schedules are decoupled
d) libraries and boost are much more up to date
d) testing ALL of boost happens far less frequently.
The only boost components which don't fit here are
those of boost infrastructure - boost test and bjam.
Robert Ramey