Re: [boost] What Should we do About Boost.Test?

Subject: Re: [boost] What Should we do About Boost.Test?
From: Jamie Allsop (ja11sop_at_[hidden])
Date: 2012-09-18 06:03:54

On 17/09/12 15:40, Dave Abrahams wrote:
>
> Hi All,
>
> I was just going through Boost.Test to try to figure out how to teach
> it, and while it looks to have substantial value, it is also in quite a
> mess.  It contains loads of features that are exercised in the examples/
> directory but neither included in any of the tests nor documented.
> There are facilities for command-line argument parsing!  There are
> "decorators" that turn on/off features for test cases.  There is support
> for mock objects! These are cool and sometimes necessary features, but
> who knew?  The third tutorial page
> (http://www.boost.org/doc/libs/1_51_0/libs/test/doc/html/tutorials/new-year-resolution.html)
> has a glaring typo in the code examples: "BOOST_AUTO_EST_CASE".  There's
> no reference manual at all.  There are nearly-identical files in the
> examples/ directory called "est_example1.cpp" and "test_example1.cpp"
> (Did the "t" key on someone's keyboard break?)  I could go on, but where
> would I stop?

Yes I agree there are some documentation issues but addressing these is 
I am sure a much lesser effort than the following suggestions made.

>
> I don't know what to do about this.  Because of the lack of redundancy
> (i.e. tests and documentation), it's hard to tell whether this library
> is correct or even to define what "correct" should mean.  It seems like,
> as long as the code is incompletely / incorrectly documented and tested,
> it's just someone's personal coding project that we happen to keep
> shipping with Boost, and not really a library for general use.  This
> situation reflects poorly on Boost as a whole and the fact that it
> centers around a _testing_ library, which is concerned with
> robustness...  well, let's just say that the irony isn't lost on me.

FWIW - we've been using Boost.Test for years in for testing production 
code in an unforgiving environment. I don't see a better option out 
there (and yes I am aware of the alternatives). We have recently created 
a few local patches that we'll push back out shortly for hopeful 
inclusion in Boost.Test that facilitate better test output generation.

>
> I don't mean this posting as an attack on Gennadiy in any way, but I
> think the situation is unacceptable and therefore am opening a
> discussion about what should happen.
>
> As a straw man, I'll make this suggestion:
>
> - Boost.Test is officially deprecated in the next release

I cannot agree this is the correct approach - I'd rather we all put 
effort into tidying up the docs and helping Gennadiy out. I'm sure he'd 
be receptive to anything that helps make this important library better.

> - Its documentation, such as it is, is removed from the release after
>    that

-1

> - Meanwhile, other tests in Boost that use this library are rewritten to
>    use a different mechanism

Now that's a huge effort, much greater than simply addressing 
documentation or feature issues.

> - The code is removed from Boost thereafter

-1

>
> I am not at all attached to removing Boost.Test from Boost, but IMO
> rescuing it would require a significant new investment of time and
> energy from people who are committed to bringing the library up to par
> with the rest of what we do.  (I seriously thought about volunteering
> for this myself, but realistically speaking, I don't have the time, and
> volunteering for something you can't actually do is worse than not
> volunteering at all.)  Even if volunteers show up, I'd suggest
> proceeding with the plan above, subject to reversal at any time the work
> actually gets done.

Let me put it this way. Boost.Test works and it is very good at what it 
does. It isn't perfect but I don't know of a test library that is. We 
are lucky to have an active and dedicated maintainer of Boost.Test. I'd 
much rather effort was invested in trying to address issues that may 
exist with how Boost.Test is maintained and updated to minimise 
breakages given the special importance of this library.

I mean - the source code is there - it's not the prettiest perhaps but 
that can be said for quite a few libraries in Boost. If someone really, 
really thinks they can do a better job then please, by all means write a 
test library for boost, put it up for review and try to get it accepted. 
Boost needs a test library now and it has one. If we end up with another 
in the future and people prefer it, sure we could look at migrating to 
that but until then I think we should do our best to improve what is there.

That's a two way street of course and Gennadiy I'm sure will be 
responsive to any efforts to help him with that, though I can't of 
course speak for him, I'm hoping he will. Given his responses to this 
thread so far I'm positive.

Jamie

>
> Thoughts?
>