Re: [boost] Unittest capability for meta-programs feedback request

Subject: Re: [boost] Unittest capability for meta-programs feedback request
From: Dave Abrahams (dave_at_[hidden])
Date: 2011-09-27 14:09:15

on Tue Sep 27 2011, Gennadiy Rozental <rogeeff-AT-gmail.com> wrote:

> Dave Abrahams <dave <at> boostpro.com> writes:
>
>> 
>> 
>> on Tue Sep 27 2011, Ben Robinson <icaretaker-AT-gmail.com> wrote:
>> 
>> > This submission would make it possible to write a complete set of unit
>> > tests for meta-programs, to test both the positive, compiling statements,
>> > and the negative, non-compiling statements.  These tests will all compile,
>> > and the negative tests can throw an exception instead of failing to
>> > compile.
>> 
>> I think you're over-promising here.  You can't turn all non-compiling
>> code into compiling code by wrapping it in something.  Some errors are
>
> He can. He just ifdef it out essentially. 

Well, then it can't test anything, can it?

> If I understand correctly, in a few
> words an idea comes to this:
>
> Imagine you develop template component:
>
> template<...>
> class MyComponent {
>   BOOST_MPL_ASSERT((condition))
>   ...
>
> };

These cases are easy to deal with, and are not what I'm talking about
If there's a compile-time bool to work with, one can turn
that into anything one wants (compile-time error, runtime assert, etc.)
Sometimes, however, there's no compile-time bool to work with.

If I am the author of std::pair and I write:

  // test that pair doesn't somehow convert unrelated types
  // into values that can be used for construction
  std::pair<int,int> x("foo", "bar");

I expect that test to fail compilation.  There's no useful assertion you
can do that will turn it into a runtime error.

>> Personally I am very uncomfortable with the use of exceptions to deal
>> with failed assertions, and in a test that's just a bunch of
>> compile-time assertions, I don't see *any* advantage whatsoever in using
>> exceptions.
>
> This is usual deal with unit tests: you want to test all the
> expectations.

Yes

> Imagine you expect that your component does not work with int. Meaning
> MyComponent<int> should fail to compile. 

Yes.  This is the kind of case I'm talking about.

> How can you record and test this expectation? In original version - no
> way to do this. 

Hm?  Original versoin of what?

> Your only option is to put into test module some test statements and
> comment them out.

We do it today by having "expected compilation failure" (compile-fail)
tests (a more robust system would test the contents of the error
message, but that's another thing).

> Now imagine that you or someone else changes implementation of the
> component and suddenly MyComponent<int> compiles. Your original
> expectation is broken. And yet your test module does not notify about
> it. 

Not mine; I build a compile-fail test.

> With the approach above these expectations are testable. You define macro
> UNITEST, use TESTABLE_ASSERT in your development and that's it.

I'm sorry, I read what you wrote above but don't see anything in there
that would make this work.

> Now if you can come up with another approach to test these
> expectations I'd be happy to listen.

We already have an approach; it requires integration with the test
system.  Yes, it's imperfect, but it does do the kind of testing needed
to see that MyComponent<int> is prohibited.

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com