Subject: Re: [boost] Trunk tests broken
From: Robert Ramey (ramey_at_[hidden])
Date: 2010-06-02 16:56:27
Beman Dawes wrote:
> On Wed, Jun 2, 2010 at 1:41 PM, Belcourt, Kenneth <kbelco_at_[hidden]>
First of all, let me apologize for any inconvenience this may have caused.
> bjam shouldn't stop entirely because a file isn't found. At worst,
> that library's tests should fail.
> Can Boost.Build or bjam be modified to be more tolerant of errors
> during regression testing?
This is the perfect example of why I think trunk testing should be changed.
Here is what I propose.
For each library: - e.g. serialization
a) Start with a machine which two separate release and trunk directory
Assume that these are named /boost_release and boost_trunk respectively.
b) invoke the following script:
mv boost_release/serialization boost_x/serialization
mv boost_trunk/serialization boost_release/serialization
mv boost_release/archive boost_x/archive
mv boost_trunk/archive boost_release/archive
mv boost_release/libs/serialization boost_x/libs/serialization
mv boost_trunk//libs/serialization boost_release/libs/serialization
We now have a boost_root tree which is the latest release but
has the trunk for the one library being tested.
c) run the test
# I run ../../../tools/regression/src/library_test.sh ...which is equivalate
to the above
# but creates a report in the test directory
c) run a script which reversed the changes made in a) above.
The testing of all of boost would be defined as nothing more than
the same procedure applied to each library.
What would this give is?
a) Problems such as the current one would be be limited only to the
b) bugs in one library wouldn't show up as test failures in another library.
c) Testing would be against the "next release". This would dimish surprises
that might occur when libraries are merged into the release branch.
d) Testing would be MUCH faster. Currently, if a header in library A
in the trunk - then ALL the libraries which depend on A are rebuilt and
This costs a huge amount of time for almost no benefit. Under the new
only the tests for library A would be re-run if library A changes. Tests
other libraries which depend upon A would occur when library A is merged
into the release branch.
e) It takes a baby step in the direction of making boost libraries
f) Certain libraries and tools such as bjam and boost.test are used in the
testing process. When the developer of one of these tools checks in
something that turns out to be an error - then it creates bogus test results
when can hang around for weeks until they are cleared up. How can
these developers work in such an environment?
Finally, what would it take to implement this? Only a change in the test
All current boost tools - bjam, boost test, etc would remain identical. I
know how the test matrix generation works - but I suspect that it would
change little if at all (I'm assuming it reviews the bin.v2 directory which
be the same under this proposal).
Who sees anything wrong with this? Better yet, who can see any
advantage in the current setup?
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk