|
|
Boost Testing : |
Subject: Re: [Boost-testing] [test] trunk breakage
From: Eric Niebler (eric_at_[hidden])
Date: 2010-01-06 01:56:38
(Cross-posting to boost-testing and boost-devel...)
On 1/6/2010 7:32 AM, Robert Ramey wrote:
> I realise that I've brought this up before, but it seems that there is
> an opportunity to get another hearing on it.
Thanks for (re-)raising this issue.
> I would like to see trunk testing altered so that for each library
> it is tested against the release branch of all the other libraries
> rather than the trunk versions of all the other libraries as happens now.
>
> Boost.Test is a perfect example. The author of boost test tests his
> stuff and checks it into the trunk. This is what he has to do in order
> to see his stuff tested on all the other platforms he doesn't have.
> From time to time this is going to reveal a bug that he didn't know
> about. So far so good. Except, since all the other libraries are
> using this component, they show up with bugs also. Now the
> author of Boost Test is on the hot seat and can't work on his
> own schedule. The whole boost release is now waiting on him.
> It's unfair to the author and breaks/suspends trunk testing
> for all the other libraries which might depend upon Boost Test.
>
> Of course this can occur not only with boost test but any
> library which is depended upon by others.
Right. Boost.Test and Boost.Build are particularly critical.
> Also, the fact that a library passes test against the trunk is
> no guarentee that it will pass when merged into the release.
> A library might be dependent upon a change which has yet
> to be merged into release branch.
I've been bitten by this before.
> This situation wouldn't occur if each library in the trunk was
> tested against release.
I agree that the current situation, where we have several single points
of failure -- a bug introduced by one component on trunk gumming up the
whole works for everyone -- is unsatisfying. Your suggestion sounds
reasonable, but I don't know enough about our test infrastructure to see
the possible implications for our test runners and for our developers.
One concern I have is that most developers will not be running their own
tests in this configuration before committing to trunk. I suppose that's
not so bad because at worst, their component ends up broken on trunk,
but the rest of the world is unaffected because presumably they haven't
merged their broken change to release yet (and everybody else is using
the last-known-good version of that component from release).
Can someone who knows more about our test infrastructure say a bit more
about Robert's suggestion? As a half-measure, we could treat Boost.Build
and Boost.Test as special and have the test runners use the versions on
the release branch.
-- Eric Niebler BoostPro Computing http://www.boostpro.com