[Boost-testing] Re: Re: pjb intel-win32 tests

From: Paul Baxter (pauljbaxter_at_[hidden])
Date: 2005-05-10 14:50:29

>> I made the suggestion before that if you want the variety of testing on
>> many setups, perhaps you could provide a user set of results separate
>> from the core largely automatic set of results. I certainly don't want to
>> confuse other users.
>
> If you can update the comment.html file then the confusion would
> disappear, and more testers are *always* welcome IMO.
>
> One other thing that would help, would be to run the tests using an
> appropriately named toolset, something like "intel-win32-8.1-release"
> would be good.

My last set of results (in which regex still fails, sorry) posted last night 
has an updated comment.html though from discussions today I've also now 
added contact details for next time.

In looking at existing documentation, the link to 'regression tests' on 
http://www.boost.org/status/compiler_status.html (boost front page: 
Regression tests-> General info) takes you to 
http://www.boost.org/more/regression.html which doesn't describe the 
python/regression.py method for regression testing. My grey matter is a bit 
strained, but I thought until recently it took you to a different page 
explaining the python 2.3 method. (I have a locally cached copy of the 
'Running Boost Regression Tests.htm' page which explains this.)

Maybe this link never did take you there, but it probably should if still 
available. Perhaps post 1.33 we could take a look at the docs. I'm not the 
greatest writer but would be happy to put some of my painful experience to 
good use by updating/supplementing some of what we have.

For the Regression testing FAQ/wiki??:

Q) If I want to submit test results what information do I need to supply?
A) Look at the basic pyrhon regression testing information on the boost web 
site, and consider the following refinements to make it easier for others to 
use the information you are supplying:

As well as a basic way of identifying you as a regression tester via a 
unique setting for the 'runner' command line option, it is helpful for other 
regression testers/maintainers to understand a little about your platform 
and compiler.

The file 'comment.html' in your regression testing root directory provides a 
means to incorporate some descriptive text about your test environment into 
the final test output that is returned to the boost regression test archive.

Use this to list your platform hardware, OS, compiler(s) and other pertinent 
setup information.

If you are willing to help out further, it would also be useful to attach 
contact name and email details for cases where the maintainers of particular 
tests have further questions.

You can also make use of a feature of bjam's variable parsing to annotate 
each set of tests with additional version information regarding your 
compiler. After the toolset abbreviation or full bjam tool name for your 
compiler, you may add additional descriptive text to tag compiler version 
and build.

For example to invoke the tests with the 'intel-win32' bjam compiler 
configuration, one may run regression.py with the '--toolsets=iw' 
abbreviation, '--toolsets=intel_win32' full tool name or add some extra 
descriptive text such as '--toolsets=iw-8_1-release'.

For example on WinXP bring up a CMD window and set compiler path, library 
and include etc. and then use:

-----------------

set BUILD=release
python regression.py --runner=pjb --toolsets=intel-win32-8_1-release

-----------------

My comment.html file looks like:

<p>Tests are run on Win32 platform. Intel-win32 release build v8.1.027 on 
AMD XP 2000+ running latest Win XP 32 bit. Contact: myname at myhost dot 
com</p>