Re: [jamboost] Increasing reliability of main regression test

From: David Abrahams (dave_at_[hidden])
Date: 2003-01-04 12:16:26

Beman Dawes <bdawes_at_[hidden]> writes:

> The following (or similar) scenario breaks the main regression tests fairly 
> often. Twice this week, for example.
>
> * Developer has small Jamfile in boost-root/libs/foo/build and another in 
> boost-root/libs/foo/test. There is also a section testing foo in 
> boost-root/status/Jamfile, often a cut-and-paste from 
> boost-root/libs/foo/test/Jamfile.
>
> * Developer makes some change in boost-root/libs/foo/build/Jamfile (or less 
> often in other code) that requires a change in 
> boost-root/libs/foo/test/Jamfile. Makes that change and tests, but 
> completely forgets to change boost-root/status/Jamfile. Main regression 
> test then breaks.
>
> For the cases I'm familiar with, a fix would be for 
> boost-root/status/Jamfile to be able to include the 
> boost-root/libs/foo/test/Jamfile. 

That's a much better way to go about testing anyway. The curent setup
is just a vestigal artifact of Jens' old testing system.

> I think several people (Gennadiy, Doug G.) have experimented with
> this.
>
> We need to settle on an official way to do this.

What's to settle?

subinclude $(BOOST_ROOT)/libs/whatever/test ;

> Currently, such schemes break the status table generation because
> compiler_status.cpp actually scans status/Jamfile looking for
> test-type and some other more minor information. 

I really don't like that setup. Jamfiles written in a general
programming language and any number of perfectly legal constructs
could obscure the information you want. 

> If we could figure a way to echo that information into the output
> (either to std::cout or in one of the residue files), it would make
> include schemes and compiler_status.cpp less fragile.

Tell me precisely what you need dumped and I'll add it to the testing
rules when --dump-tests is used.

-- 
David Abrahams
dave_at_[hidden] * http://www.boost-consulting.com
Boost support, enhancements, training, and commercial distribution