Re: [boost] Release Tools Analysis Part II: Developer Feedback Systems

From: David Abrahams (dave_at_[hidden])
Date: 2007-08-09 13:20:13

on Wed Aug 08 2007, "John Maddock" <john-AT-johnmaddock.co.uk> wrote:

> Just thinking out loud here, but I've always thought that our test results 
> should be collected in a database: something like each test gets an XML 
> result file describing the test result, which then gets logged in the 
> database.  The display application would query the database, maybe in 
> real-time for specific queries and present the results etc.  

Sure, that's what the Trac plugin would do.

> Thinking somewhat outside the box here ... but could SVN be
> conscripted for this purpose,

The biggest downside of that idea is that it would be very expensive
in SVN resources to ever give real-time feedback from testing, because
you'd need a separate checkin for each build step.

> yes, OK I know it's an abuse of SVN, but basically our needs are 
> quite simple:
>
> * Log the output from each build step and store it somewhere, with 
> incremental builds much of this information would rairly change. In fact 
> even if the test is rebuilt/rerun the chances are that logged data won't 
> actually change.
> * Log the status of each test: pass or fail.
> * Log the date and time of the last test.
>
> So what would happen if build logs for each test were stored in an
> SVN tree set aside for the purpose, with pass/fail and date/time
> status stored as SVN properties?  Could this be automated, from
> within bjam or CMake or whatever?

Sure it could.

So you *are* actually advocating a separate file in SVN for each test?
I guess I also worry about the performance cost of doing a checkin for
each test.

> Of course we're in serious danger of getting into the tool writing
> business again here ....

Unless we entirely drop our display distinctions and markup, or we
stick with the same fragile/unreliable display tools we have,
*someone* has to write new display tools.  There just aren't any
existing tools out there that do what we want.

Unless Kitware is prepared to accomodate our display requirements, I
don't know where those tools are going to come from other than from
within Boost.

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com
The Astoria Seminar ==> http://www.astoriaseminar.com