Re: [boost] Unittest capability for meta-programs feedback request

Subject: Re: [boost] Unittest capability for meta-programs feedback request
From: Gennadiy Rozental (rogeeff_at_[hidden])
Date: 2011-09-27 13:29:26

Dave Abrahams <dave <at> boostpro.com> writes:

> 
> 
> on Tue Sep 27 2011, Ben Robinson <icaretaker-AT-gmail.com> wrote:
> 
> > This submission would make it possible to write a complete set of unit
> > tests for meta-programs, to test both the positive, compiling statements,
> > and the negative, non-compiling statements.  These tests will all compile,
> > and the negative tests can throw an exception instead of failing to
> > compile.
> 
> I think you're over-promising here.  You can't turn all non-compiling
> code into compiling code by wrapping it in something.  Some errors are

He can. He just ifdef it out essentially. If I understand correctly, in a few
words an idea comes to this:

Imagine you develop template component:

template<...>
class MyComponent {
  BOOST_MPL_ASSERT((condition))
  ...

};

Here "condition" is supposed to check at compile time some compile time
condition template arguments are expected to satisfy. In other words - concept.

Imagine now that you would have written this class like this:

template<...>
class MyComponent {
  typedef condition concept;
  ...
};

Now this version does not check anything at compile time, BUT somewhere else in
the code you can write SOME_ASSERT((MyComponent::concept)). Now this assert can
be compile time OR runtime - this is not really important, but in test
environment we usually want runtime error. We can even wrap it using macro like
this:

#ifdef UNITEST
#define TESTABLE_ASSERT( cond ) typedef cond concept;
#else
#define TESTABLE_ASSERT( cond ) BOOST_MPL_ASSERT((cond))
#endif

and use it in your class :

template<...>
class MyComponent {
  TESTABLE_ASSERT((condition))
  ...

};

In regular builds this is identical to MPL_ASSERT. In test environment check is
delayed. Why we want to delay? See below.
 
> Personally I am very uncomfortable with the use of exceptions to deal
> with failed assertions, and in a test that's just a bunch of
> compile-time assertions, I don't see *any* advantage whatsoever in using
> exceptions.

This is usual deal with unit tests: you want to test all the expectations.
Imagine you expect that your component does not work with int. Meaning
MyComponent<int> should fail to compile. How can you record and test this
expectation? In original version - no way to do this. Your only option is to put
into test module some test statements and comment them out.

Now imagine that you or someone else changes implementation of the component and
suddenly MyComponent<int> compiles. Your original expectation is broken. And yet
your test module does not notify about it. It goes unnoticed until very late in
a project some other third party mistakenly starting to use MyComponent<int> and
get's incorrect runtime numbers, only because they were suppose to use
MyComponent<double> instead.

With the approach above these expectations are testable. You define macro
UNITEST, use TESTABLE_ASSERT in your development and that's it.

Now if you can come up with another approach to test these expectations I'd be
happy to listen.

That's said I am not sure this whole deal deserves separate set of macros (it
does deserve maybe a page in docs explaining the approach), but I frankly did
not look what OP proposed in details - there might be something more there.

Regards,
Gennadiy