[Boost-testing] Test processing changes...

From: Rene Rivera (grafikrobot_at_[hidden])
Date: 2008-05-04 19:28:57

In another step toward making the testing tools more reliable I've been 
working on rewriting process_jam_log.cpp into an equivalent 
process_jam_log.py. It is now mostly working and I'd like for some 
volunteers in switching their testing over to it. I'm looking for:

* Testers that do *full* runs only. The "mostly working" I mention above 
is that the new processing does not account for incremental testing yet.

* Testers in each of the main platforms, but only if they aren't the 
only ones on that platform.

* At least one tester that runs more than one toolset. To minimize 
potential impact to testing result the ideal would be someone who tests 
only two toolsets.

If you fit the above reply on the list which tester IDs you will be 
switching to the new PJL, so that people don't collide. To switch two 
things are needed:

1. Get the latest version of "run.py" if you don't update it as part of 
your testing 
<http://svn.boost.org/svn/boost/trunk/tools/regression/src/run.py>.

2. Add, or replace, "--pjl-toolset=python" when running the tests.

Now about the new PJL.py... The main reason I've been working on it was 
because Hartmut pointed out a bug with the existing PJL.cpp that was 
preventing the Spirit classic results from showing up even though they 
where being run. So instead of trying to hack some more at the C++ 
program I rewrote it Python. Briefly the rewrite:

* Instead of relying on post-processing the bjam output it uses the XML 
dump generated directly by bjam. This eliminates the multitude of 
problems that parsing the bjam output we've had in the past.

* Because all the output of the bjam run is in the bjam XML there's no 
need to read the captured output files and hence it doesn't suffer from 
stale results. Or at least it wont once I get the incremental part of it 
written.

* It includes the *all* output from building/running a single test or 
library in the results. In particular for building dependent libraries 
it includes all the compile commands run, not just the last one.

* It adds some extra information as part of the command output. 
Currently this includes timming information for each command and for 
compiles it includes any active defines (from the Boost.Build point of 
view).

* Because it's Python, it removes the requirement of having a capable 
C++ compiler. We are now down to a C compiler (for bjam) and Python 2.3. 
This should help in some of the more limited environments where 
installing a C++ compiler, other than the one being tested, is problematic.

Enjoy.

-- 
-- Grafik - Don't Assume Anything
-- Redshift Software, Inc. - http://redshift-software.com
-- rrivera/acm.org (msn) - grafik/redshift-software.com
-- 102708583/icq - grafikrobot/aim,yahoo,skype,efnet,gmail