Re: [boost] What Should we do About Boost.Test?

Subject: Re: [boost] What Should we do About Boost.Test?
From: Gennadiy Rozenal (rogeeff_at_[hidden])
Date: 2012-10-05 02:34:11

Dave Abrahams <dave <at> boostpro.com> writes:
> The question remains: "how do I learn/teach this library?"  If I can't
> answer those questions, I also can't answer the question

What do you need to be able to teach/learn? 
 
>        How do I use this library?

While I never claim that docs are excellent, they do answer in length to this 
question. Did you read user guide sections (test organization for example)?
 
> >> There are facilities for command-line argument parsing!
> >
> > Yes, indeed and while I like these much better than official boost
> > one, I do not insist it to be a public interface, thus it does not
> > need to be documented.
> 
> That's fine, but in the presence of so many other problems and of a
> large suite of examples/ directed at this feature, it contributes to the
> sense of uncertainty about what this library *is*.

Examples left from the time when I thought I might consider submitting this 
component as an alternative to program_options. These days I use them sometimes 
when I need to develop some improvements in this area. Would you rather have 
them removed? 

> BTW, also, it's
> completely unclear how CLA processing relates to the mission of the
> library.

Boost.Test needs to parse CLAs. No more, nor less. It hs nothing to do with 
mission of the library.

> Does that "improved documentation" apply to what's on the release
> branch, or only to what's available in trunk?

Improved documentation (not sure why you put it in quotes) applies to what is 
already released. There is no documentation yet for newly developed features.
 
> > 2. There is no *formal* reference documentation, but I am not convinced 
there 
> > is a huge need for one. 
> 
> There most certainly is, *especially* in the presence of so much other
> uncertainty.

What uncertainty?

> > In majority of the cases Boost.Test unlike other boost libraries is
> > not extended or accessed through class interface.
> 
> I don't see how that's relevant.
> 
> > There are few interfaces which are indeed used and they are
> > documented.
> 
> ??? I can't even begin to understand how you can say that.  Everything
> one does with a library, one does through an interface.  Every interface
> needs to be documented so that users know how to use it correctly.
> Otherwise, it's just your private code.

This comment just shows that you did not try to read the docs (I know they are 
not perfect, but the answer to above is clear even from skimming them) 

Public interface of unit test framework is predominantly macro based. This 
includes test tree management and test tools (Well, new interfaces deviate from 
this, but they are beyond the point). And almost every public interface (with 
few exceptions which I already pointed out) ARE documented in details. There are 
few non-macro public interfaces, which are documented as well. There are couple 
interfaces one could use to extend library functionality, which we can add to 
documentation, but these are:
  * well above basic usage
  * very rarely needed

So should we document these? Probably yes, but its absence does not affect 
majority of Boost.Test users and clearly does not preclude anyone from using it.

So in summary, "every public interface needs to be documented" is true for the 
most part in Boost.Test case. Or can you show any glaring gap?

> >> I don't know what to do about this.  Because of the lack of redundancy
> >> (i.e. tests and documentation), it's hard to tell whether this library
> >> is correct or even to define what "correct" should mean.  It seems like,
> >
> > I am not sure what you mean. There is an extensive self test modules.
> 
> I mean *redundancy* between the tests and the documentation.  The tests
> should check that the library does what the documentation says it does.

They do. Self tests are comprehensive and cover all public interfaces described 
in documentation (unless I missed something). Do you have example of clear gaps?

> >From the tests alone I can't even draw a conclusion about what you
> intend as a stable, supported, public interface.

Well, it might be difficult to read unit test modules (they are not really 
developed as a replacement for documentation), but they all test some part of 
public interface (for example test_tools_test tests all public Test Tools)

> Look, I teach classes on Boost.  If Boost.Test is not learnable and
> teachable, I have to tell my students to stay away from it.  That's
> embarrassing for me, and bad for Boost.

I'll be happy to help you prepare for this class (BTW, there are couple 
presentations I gave for Boost.Test, which can be used as basis for class
curriculum). For now all I see is just some misunderstanding of terms and what 
constitutes public interface of the library.
 
Regards,
Gennadiy