Boost Testing :
From: Paul Baxter (pauljbaxter_at_[hidden])
Date: 2005-05-10 14:50:29
>> I made the suggestion before that if you want the variety of testing on
>> many setups, perhaps you could provide a user set of results separate
>> from the core largely automatic set of results. I certainly don't want to
>> confuse other users.
> If you can update the comment.html file then the confusion would
> disappear, and more testers are *always* welcome IMO.
> One other thing that would help, would be to run the tests using an
> appropriately named toolset, something like "intel-win32-8.1-release"
> would be good.
My last set of results (in which regex still fails, sorry) posted last night
has an updated comment.html though from discussions today I've also now
added contact details for next time.
In looking at existing documentation, the link to 'regression tests' on
http://www.boost.org/status/compiler_status.html (boost front page:
Regression tests-> General info) takes you to
http://www.boost.org/more/regression.html which doesn't describe the
python/regression.py method for regression testing. My grey matter is a bit
strained, but I thought until recently it took you to a different page
explaining the python 2.3 method. (I have a locally cached copy of the
'Running Boost Regression Tests.htm' page which explains this.)
Maybe this link never did take you there, but it probably should if still
available. Perhaps post 1.33 we could take a look at the docs. I'm not the
greatest writer but would be happy to put some of my painful experience to
good use by updating/supplementing some of what we have.
For the Regression testing FAQ/wiki??:
Q) If I want to submit test results what information do I need to supply?
A) Look at the basic pyrhon regression testing information on the boost web
site, and consider the following refinements to make it easier for others to
use the information you are supplying:
As well as a basic way of identifying you as a regression tester via a
unique setting for the 'runner' command line option, it is helpful for other
regression testers/maintainers to understand a little about your platform
The file 'comment.html' in your regression testing root directory provides a
means to incorporate some descriptive text about your test environment into
the final test output that is returned to the boost regression test archive.
Use this to list your platform hardware, OS, compiler(s) and other pertinent
If you are willing to help out further, it would also be useful to attach
contact name and email details for cases where the maintainers of particular
tests have further questions.
You can also make use of a feature of bjam's variable parsing to annotate
each set of tests with additional version information regarding your
compiler. After the toolset abbreviation or full bjam tool name for your
compiler, you may add additional descriptive text to tag compiler version
For example to invoke the tests with the 'intel-win32' bjam compiler
configuration, one may run regression.py with the '--toolsets=iw'
abbreviation, '--toolsets=intel_win32' full tool name or add some extra
descriptive text such as '--toolsets=iw-8_1-release'.
For example on WinXP bring up a CMD window and set compiler path, library
and include etc. and then use:
python regression.py --runner=pjb --toolsets=intel-win32-8_1-release
My comment.html file looks like:
<p>Tests are run on Win32 platform. Intel-win32 release build v8.1.027 on
AMD XP 2000+ running latest Win XP 32 bit. Contact: myname at myhost dot