|
|
Boost Testing : |
From: Victor A. Wagner Jr. (vawjr_at_[hidden])
Date: 2006-02-23 01:15:39
At 08:11 2006-02-21, Vladimir Prus wrote:
>Hello!
>
>As many of you know, the Boost.Build system version 2 is in development for
>quite some time, and it becomes desirable to switch C++ Boost to V2. I
>would like to explain where we stand with this project.
>
>I've spend last few weeks making sure that there are no technical issues in
>V2 preventing tests to work, and that tests for all libraries are
>up-to-date. Specifically, I wrote a script that runs regressions with V1
>and V2, and compares the results. At the moment, on gcc/Linux the results
>are identical.
>
>Two volunteers on boost-build mailing list were running tests on
>msvc/Windows. We're also getting close. Except for Python tests, that Dave
>will hopefully look into, we've just two unresearched differences.
>
>However, I think it becomes impractical to continue this process inside
>boost-build mailing list. While it would be good to verify that V2 produces
>the same results as V1 on *all* compilers currently in use, folks on
>boost-build don't have that many versions at hand. And obviously, only the
>results produced by regular regression runners matter when it comes to
>working on release.
>
>I think that the best thing now is to gradually move regular regression
>tests to V2. In my opinion, the ideal process will looks like this:
>
> - One regression runner switches to V2
> - Next day I look at all test failures and classify them as "V2 bug",
> "library bug", or "unclear". I'll work with library authors to resolve
> bugs from the third category and will fix the V2 bugs.
> - Another regression runner switches to V2
>
>With such a process, any given regression runner will only have to set up
>Boost.Build V2 once, which will not take much time, and after that I'll be
>mostly working with library authors. The plan that when one person switches
>to V2 he won't be running V1 tests anymore, so the amount of required disk
>space won't double.
>
>
>I've talked with Thomas Witt, release manager for 1.34, and he does not mind
>the switch for 1.34.
>
>So, what do regression runners think of this idea?
sign me up
>- Volodya
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>The comparison work we've done before guarantees that there won't be big
>issues on gcc and msvc. We might get problems on other toolsets, but I'd
>expect them to be fixed quickly, because:
>
> - The most time-consuming task was syncing V1 and V2 Jamfiles. It's
>already
> done.
> - Global configuration problems can be fixed soon. As example, the HP-CXX
> (formely Tru64) toolset for V2 went from non-existance to fully passing
>V2
> internal regression tests in less than a week.
> - Failures on specific tests can be solved as part of regular bug triage
> that's done before release. As example, the first run of msvc/Windows
> tests was done on Feb 13, and one week again we're mostly in shape.
> I would expect fixing failures for any other toolset to take
>considerably
> less time.
>
>Because of greadual switch, at any given time only one regression runner
>will
>be in "V2-switch" mode. All other regression runners can be used to resolve
>bugs in libraries.
>
>
>It means that at the works case, any given regression runner will be
>unavaiable for regular use for one week. And we have just one toolset that
>never seen any user -- qcc. For all others the switch to V2 probably won't
>add noticable time to the release process. Typically, release process takes
>longer than expected, because library developers don't have the time to fix
>bugs in their libraries. This gives us extra time to fix V2-related issues.
>
>
>There are some good reasons to switch to V2 for this release. The biggest
>one
>is that separation between V1 and V2 fractures developement effort and user
>base.
>
>Another reason is that getting V1 and V2 Jamfiles in sync is very time
>consuming task. If we decide to make the switch after release, or for the
>next release, it will mean this task will have to be repeated.
>
>So, what is your opinion about switch to V2?
>
>_______________________________________________
>Boost-Testing mailing list
>Boost-Testing_at_[hidden]
>http://lists.boost.org/mailman/listinfo.cgi/boost-testing
Victor A. Wagner Jr. http://rudbek.com
The five most dangerous words in the English language:
"There oughta be a law"