Re: [boost] Enhancing regression tests summary

Subject: Re: [boost] Enhancing regression tests summary
From: Robert Ramey (ramey_at_[hidden])
Date: 2014-10-31 12:05:34

Beman Dawes wrote
> The big question in my mind is whether we should enhance the current
> regression testing system or start fresh using one of the continuous
> integration frameworks?

I think we should think bigger.  I'd like to see us encourage each user
to run tests on the libraries he actually uses and post these results to a 
common area.  This would mean that 

a) we would have test results for (all?) the actual configurations that
users are using.  
b) The testing load be automatically be distributed to the users which
actually use a library
c) Libraries actually being used would be tested.
d) Resources wouldn't be expended on libraries not being used.

In short, distributing testing this way would scale to any number of
libraries.

I implemented a first cut of such a system as part of the Boost Incubator. 
This
system uses CMake/CDash as described here:
http://rrsd.com/blincubator.com/tools_cmak/
It permits users of libraries in the incubator to check out test results of
other users. 
You can see how this works by going to
http://rrsd.com/blincubator.com/bi_library/safe-numerics/?gform_post_id=426
and clicking the field in "Test Results Dashboard"

In my view, 

a) Boost has to continue to grow or die.
b) To continue to grow, it has to be scalable
c) the best way to do that is to distribute testing to the users

I see this a one of the ultimate consequences of the Boost modularization
effort.

Robert Ramey

--
View this message in context: http://boost.2283326.n4.nabble.com/Enhancing-regression-tests-summary-tp4668362p4668610.html
Sent from the Boost - Dev mailing list archive at Nabble.com.