|
|
Boost Testing : |
From: Robert Ramey (ramey_at_[hidden])
Date: 2007-07-22 12:17:34
Questions regarding testing of a boost installation.
The current scripts I see in the boost/tools directory
do (approximately) the following:
a) build bjam and process_jam_log
b) sync the local system with the current boost CVS
c) runs bjam on all the jamfile.v2 files
d) may (runtest.sh) or may not (regression.py) produce
a local table.
e) may (regression.py) or may not upload data to
some other place.
Suppose one wants to validate the current installation
along with any changes that have been made locally.
The above procedure isn't helpful. In the past I've
done with with a short script which
runs bjam with --dump-tests (hmm was dump-test) ...
runs process-jam-log
runs compiler_status or a variation thereof.
With v2 this stopped working. I traced one problem
down to relace --dump-test with --dump-tests (note the s)
But still I have a couple of issues:
a) if a library fails to build, I get fail for all the tests. But
there is no link to the log neither the test nor the library
build logs. Before I got a link into the links.html file
where some explanation could be found. I do find
the corresponding test_log.xml in the test run directory.
It shows compilation of the test passing - then nothing.
The bjam log does in fact show test test compiling but
failing to link because of lack of the library. I think
Also, the bjam.log does show the library build failing.
b) in some cases I would like to always show the output
and have the link generated even if the test is marked
"Pass". I'm sure this is possible because the the tables
generated from config to it. It seems that depends
on some cooperation from bjam v2 but I don't know
and haven't been able to find the magic.
So I can get what I need - sort of. But this raises the real
issue that concerns me.
a) Suppose a new user want's to load boost on his new
platform and want's to validate it. How is he expected to
do it? As noted above, I can do this though I don't get
much info if a library fails to build.
b) Suppose a user needs to make a slight tweak in his
local installation. For example, msvc uses a typedef
for wchar_t as default rather than the built-in type. Users
might want to use one or the other (who knows what the
boost default is - another issue). So he might want to
make the change, rebuild all the library and now run
test on his installation. I see no obvious way to do that
now.
c) Suppose a library developer wants to test his local
changes. There is no current obvious way to do this.
I asked another boost developer how he does this. He
told me that he just inspects the results by hand (eye?).
This isn't convenient for me.
Given the abi variations and combinations that are possible
there is no way that any sort of centralized testing can
address this issue.
Basically, boost installation should be followed by a
validation phase which tests the currently loaded/configured
on the local machine and produces a handy table. And
this should be part of the normal installation procedure.
Robert Ramey