Re: [boost] Google SOC 2007, Boost.Test Frontend

From: Stefan Seefeld (seefeld_at_[hidden])
Date: 2007-03-22 08:22:43

Gennadiy Rozental wrote:
> "Stefan Seefeld" <seefeld_at_[hidden]> wrote in message 

>> * Enhanced test run annotations, to allow report generation to build a 
>> more
>>  meaningful test report (e.g. fold multiple equivalent test runs into one,
>>  only consider test runs from a given time window, or associated with a 
>> given
>>  revision, etc.)
> 
> Ummm, I am not clear I undestood these

Right now test reports have multiple runs for the same toolchain. Thus
the absolute count of failures doesn't have a well established meaning.
What I propose is a way to either a) eliminate all but one results for
'equivalent' tests or b) enhance test run annotation to make clear that
these test runs in fact differ (say, because some now undocumented
environment parameter differs).

>> * Support for running tests in parallel, or even on distributed hardware, 
>> if
>>  available.
> 
> I don't see this as a part of Boost.Test GUI runner. It maybe different 
> project. But this is more in Boost.Build domain.

Possibly, yes.

>> * Cross-testing support (i.e. compiling with a cross-compiler toolchain on 
>> host A
>>  for target B, then uploading the test executables to B and run it there)
> 
> The sameas above.
> 
> We need to strive for clearcut problem domain. And everything that feels 
> like part of Boost.Build should not be considerred here IMO (we could, but 
> this will be a different project)

Agreed. The reason I bring these points up are

* to point out more things that should be enhanced in the testing harness
* to point out that a GUI frontend really is a frontend, and shouldn't include
  any other logic (i.e. the whole becomes a multi-tier system)

Regards,
                Stefan

-- 
      ...ich hab' noch einen Koffer in Berlin...