Re: [boost] Boost.Test updates in trunk: need for (mini) review?

Subject: Re: [boost] Boost.Test updates in trunk: need for (mini) review?
From: Gennadiy Rozenal (rogeeff_at_[hidden])
Date: 2012-10-21 20:55:07

Vicente J. Botet Escriba <vicente.botet <at> wanadoo.fr> writes:

> Hi,
> 
> thanks for taking the time to make this ready for release.
> 
> I would appreciate if all the new Boost.Test macros start by BOOST_TEST_ 

I think the train already left on this one. Plus unlike many other macros which 
you are using here and there testing tools are used quite extensively as part of 
any test module. Making this longer would by an unwelcome change. I do try to 
make BOOST_TEST as a prefix whenever it makes sense. In this case I'd rather 
strive for consise name.

> and you provide the equivalent macros for existing ones that don't 
> follows this pattern.
> I understand that you want short names, but avoid possible collisions is 
> better than having short names. I could accept also BOOST_TF_ as Boost 
> Test Framework or BOOST_TL_ as Boost Test Library.

All these abbreviation heart my head a bit. There are bunch of new libraries, 
which even referred to by appbreviation. Sometimes they are fine and if there is 
a concesus that this is the best option, I can live with it, but my preference 
would be something less cryptic.

> > So here we go:
> >
> > I. New testing tool BOOST_CHECKA
> I like this a lot. How it works?

A bit of expression template magic, plus some c++11 for advanced type deduction.

> What about BOOST_TEST_CHECK (see above)?

Again this is 2 verbs stuck together. I can live with this, but this is not as 
good as BOOST_CHECK ;o)

One option I thought about is to introduce a mode where this new tool is named 
BOOST_CHECK and old BOOST_CHECK is renamed into BOOST_CHECK_...something.

> > II. New "data driven test case" subsystem
> > BOOST_DATA_TEST_CASE( t1, data::xrange(4) ^ strs ^ data::random(),
> >                        intval, str, dblval )
> > {
> >     MyObj obj( dblval, str );
> >
> >     BOOST_CHECKA( obj.goo() == intval );
> > }
> >
> > Above test case will be executed 4 times with different values of parameters
> > intval, str, and dblval.
> Yes, this will be very useful.
> 
> Which will be the values for intval? is the index of the tuple?

The sample tuple is hidden and what you get an an actual argument with specified 
name. So intval is going to be an actual value of type int const&
 
> What about adding something like a product test case that will execute 
> the test with the Cartesian product? Is this already available.

Yes. This is what grid for.

> > III. Auto-registered test unit decorators.
> >
> > Previously is was not possible and/or convenient to assign attributes to the
> > automatically registered test units. To alleviate this I've introduced a
> > notion of test unit decorator. These can be "attached" to any test unit
> > similarly to like it is done in other languages
> Could you give some references?

Python for example.

> >      expected_failures - set expected failures for a test unit
> Could you give an example showing the utility of this decorator?

This is the same as an existing interface, but applied to auto test case. Yu 
just telling the framework an amount of assertion failures to expect in test 
unit, so these can be "ignored"

> >      description - sets a test unit description
> >      depends_on - sets a test unit dependency
> What do you mean by depends_on? Is the execution of the test subject to 
> the success of its dependencies or whether the test is enabled/disabled?

One test unit can depend on another one. For example it makes no sense to test 
access methods if construction test case failed. Thus you might want to have 
introduce dependency of former on to later and if constructio test failed, the 
second test is going to be skipped. 

> > Attachment of decorator is facilitated by using of 
> > BOOST_TEST_DECORATOR. Note
> > that you can use any of '+', '-', '*' symbols to attach decorator 
> > (and any number of '*'):
> Is there a difference between them? and if not why do you provide all of 
> them?

Only esthetic. I did not know what users would prefer and there were no problems 
maintaining all of them.

> Could the following be provided as well?
> 
> BOOST_CHECKA( foo(a) == 0, "a=" << a  );

BOOST_CHECK_MESSAGE does that already, but you might have a point. I can use 
variadic  interface to combine them together. The only thing is that Paul 
insists on treating empty list as non empty one and I'll need to jump through 
some hoops to implement it.
 
> Wow, you have added a lot of new interesting features. I will check the 
> documentation and start using it as soon as possible.

I am only starting working on docs for these. Feel free to try and ask any 
questions here meanwhile.
 
Gennadiy