Re: [boost] Release Tools Analysis Part II: Developer Feedback Systems

From: Stefan Seefeld (seefeld_at_[hidden])
Date: 2007-08-17 15:07:27

David Abrahams wrote:

> So QMTest doesn't schedule build/test processes?  It's just a database
> manager?

Yes it does 'schedule' them, but according to rather specific requirements.
You wouldn't tell QMTest to run a test suite nightly, for example, or
hook up to some other external triggers (checkins, say). Rather, you
specify what (sub-) testsuite to run, and QMTest will work out the order
etc.

> 
>>>>>> How could this be useful for boost ?
>>>>> A good question, but I'm more interested in "how Boost might use it."
>>>>> That is, something like, "We'd set up a server with a test database.
>>>>> QMTest would run on the server and drive testing on each testers'
>>>>> machines, ..." etc.
>>> Still looking for that.
>> Yes, I realize that. But as I indicated earlier, I'm not convinced
>> QMTest is a good tool to schedule / drive that. 
> 
> I meant I want some kind of analogous statement about a way we could
> use it that you *are* convinced of.

* handle all test suite runs through QMTest
* aggregate test results in a central place using QMTest, and manage
  interpretation (including expectations) to generate test reports.

>>> There's no a priori reason that Boost.Build needs to maintain the test
>>> database, is there?
>> No.
> 
> So what are the alternatives to that arrangement?

A boost-specific test database implementation could work like this:

* by default, map each source file under libs/*/test/ to a test id,
  and provide a default test type which
    1) compiles,
    2) runs the result,
    3) checks exit status and output.
  Default compile and link options could be generated per component
  (boost library).

* For cases where the above doesn't work, some special test implementation
  can be used (that incorporate the special rules now part of the various
  Jamfiles).

Regards,
                Stefan

-- 
      ...ich hab' noch einen Koffer in Berlin...