Re: [boost] Unittest capability for meta-programs feedback request

Subject: Re: [boost] Unittest capability for meta-programs feedback request
From: Gennadiy Rozental (rogeeff_at_[hidden])
Date: 2011-09-27 15:18:29

Dave Abrahams <dave <at> boostpro.com> writes:

> Sometimes, however, there's no compile-time bool to work with.
> 
> If I am the author of std::pair and I write:
> 
>   // test that pair doesn't somehow convert unrelated types
>   // into values that can be used for construction
>   std::pair<int,int> x("foo", "bar");
> 
> I expect that test to fail compilation.  There's no useful assertion you
> can do that will turn it into a runtime error.

Yes. There are always some implied/implicit expectations. And I also do not see
any way to make it into a testable concepts since it belongs to a member
function, unless we can come up with a means to attach concept to the member
function. In that case we would have checked

std::pair<int,int>::pair<std::string,std::string>::concept

 
> >> Personally I am very uncomfortable with the use of exceptions to deal
> >> with failed assertions, and in a test that's just a bunch of
> >> compile-time assertions, I don't see *any* advantage whatsoever in using
> >> exceptions.
> >
> > This is usual deal with unit tests: you want to test all the
> > expectations.
> 
> Yes
> 
> > Imagine you expect that your component does not work with int. Meaning
> > MyComponent<int> should fail to compile. 
> 
> Yes.  This is the kind of case I'm talking about.
> 
> > How can you record and test this expectation? In original version - no
> > way to do this. 
> 
> Hm?  Original versoin of what?

of MyComponent implementation in my first reply.
 
> > Your only option is to put into test module some test statements and
> > comment them out.
> 
> We do it today by having "expected compilation failure" (compile-fail)
> tests (a more robust system would test the contents of the error
> message, but that's another thing).

Yes. There is this option, but it's hardly robust. You can't be sure it fails to
compile for a reason you expect it to and checking against compiler output is
frankly madness. Not only it's different for different compilers, but it also
tend to change with every modification of component implementation.
 
> > Now imagine that you or someone else changes implementation of the
> > component and suddenly MyComponent<int> compiles. Your original
> > expectation is broken. And yet your test module does not notify about
> > it. 
> 
> Not mine; I build a compile-fail test.

In practice there are few people who rely on these. Having compilable, runtime
reportable and robust alternative would be of use for everyone else.
 
> > With the approach above these expectations are testable. You define macro
> > UNITEST, use TESTABLE_ASSERT in your development and that's it.
> 
> I'm sorry, I read what you wrote above but don't see anything in there
> that would make this work.

In a test code you'd define UNITTEST on top and write something like:

BOOST_CHECK( !MyComponent<int>::concept::value )

This failure will be reported at runtime.
 
> > Now if you can come up with another approach to test these
> > expectations I'd be happy to listen.
> 
> We already have an approach; it requires integration with the test
> system.  Yes, it's imperfect, but it does do the kind of testing needed
> to see that MyComponent<int> is prohibited.

Again very few users have testing system smart enough even to recognize
"expected compile failure" tests. And I personally would not use it if I can
help it.

Gennadiy