Re: [boost] [testing] Refactoring serialization tests, and possibly others.

From: troy d. straszheim (troy_at_[hidden])
Date: 2005-09-20 10:35:57

Rene Rivera wrote:
> 
> One thing I should make clear about my suggestion to reduce the 
> combinations... Is that it doesn't mean reducing the number of types 
> that get serialized, rather increasing those and having more complex and 
> comprehensive test.

I need to think about it more... for now, I carpet bomb.  But it clearly 
would get out of control as types, archives, platforms and versions 
proliferate.   No question.

It would be interesting to see exactly where all the time is going, I 
have the feeling it is mostly in the build, and that if one simply 
clumped many of these tests together, you could eliminate a lot of 
duplicated effort (template instantiations and linking).  This is 
another motivation for switching to autoregistering tests:  an 
all-in-one-binary (fast but fragile) or one-binary-per-test (slow but 
durable) scenario could be fairly easily switched by bjam, since these 
autoregistering tests look like

BOOST_AUTO_UNIT_TEST(whatever) { ... }

and you can combine as many of these into one binary as you like, which 
isn't the case with the current

int test_main(int, char**) { ... }

where you'd get link errors.  So conceivably the all-in-one version for 
--test-level=basic, where you are just looking for a sanity check on 
your platform and aren't actually doing development on serialization, 
and the one-binary-per-unit-test version for the --test-level=complete 
scenario, where you're doing development on serialization and want to 
zoom in on test failures as closely as possible.  Or maybe I'm out of my 
mind.  Hmm.  This is looking like a lot of work.

 > ... don't want to use environment variables any more. Try making
> it based on a command line switch. For example we might want to 
> standardize on some set of testing level options, and incorporate them 
> into the build system. Perhaps:
> 
>      --test-level=basic
>      --test-level=regression
>      --test-level=complete
> 
> You can easily test for the options with:
> 
>      if --test-level=basic in $(ARGV)
> 

Thanks for the heads-up.  I didn't catch the discussions about getting 
rid of environment variables.  --cmdline-arguments=better.

-t