Re: [boost] What Should we do About Boost.Test?

Subject: Re: [boost] What Should we do About Boost.Test?
From: Dave Abrahams (dave_at_[hidden])
Date: 2012-10-05 15:29:09

on Fri Oct 05 2012, Gennadiy Rozenal <rogeeff-AT-gmail.com> wrote:

> Dave Abrahams <dave <at> boostpro.com> writes:
>> The question remains: "how do I learn/teach this library?"  If I can't
>> answer those questions, I also can't answer the question
>
> What do you need to be able to teach/learn? 
>
>>        How do I use this library?
>
> While I never claim that docs are excellent, they do answer in length to this 
> question. Did you read user guide sections (test organization for
> example)?

I did read through the user guide.

>
>> >> There are facilities for command-line argument parsing!
>> >
>> > Yes, indeed and while I like these much better than official boost
>> > one, I do not insist it to be a public interface, thus it does not
>> > need to be documented.
>> 
>> That's fine, but in the presence of so many other problems and of a
>> large suite of examples/ directed at this feature, it contributes to the
>> sense of uncertainty about what this library *is*.
>
> Examples left from the time when I thought I might consider submitting
> this component as an alternative to program_options. These days I use
> them sometimes when I need to develop some improvements in this
> area. Would you rather have them removed?

Not necessarily, although it would be good to explain in the docs what
they're doing there.

>> BTW, also, it's completely unclear how CLA processing relates to the
>> mission of the library.
>
> Boost.Test needs to parse CLAs. No more, nor less. It hs nothing to do with 
> mission of the library.
>
>> Does that "improved documentation" apply to what's on the release
>> branch, or only to what's available in trunk?
>
> Improved documentation (not sure why you put it in quotes) 

Just so you know I mean exactly what you were referring to.  No more, no less.

> applies to what is already released. There is no documentation yet for
> newly developed features.

OK, this makes no sense at all, IIUC.  You have improved documentation
in the trunk that applies to what's in the release branch, and no
documentation for what's in the release branch?  I can't
browse the trunk docs on the web, AFAIK... oh, you've checked in HTML...

>> > 2. There is no *formal* reference documentation, but I am not
>> > convinced there is a huge need for one.
>> 
>> There most certainly is, *especially* in the presence of so much other
>> uncertainty.
>
> What uncertainty?

For example: how to use the library, what's actually in it, what's part
of the official interface, whether it's being maintained... none of
these things feel to me like they have solid answers.  The uncertainty
is my personal experience, but I sense from replies here that I am not
alone.

>> > In majority of the cases Boost.Test unlike other boost libraries is
>> > not extended or accessed through class interface.
>> 
>> I don't see how that's relevant.
>> 
>> > There are few interfaces which are indeed used and they are
>> > documented.
>> 
>> ??? I can't even begin to understand how you can say that.  Everything
>> one does with a library, one does through an interface.  Every interface
>> needs to be documented so that users know how to use it correctly.
>> Otherwise, it's just your private code.
>
> This comment just shows that you did not try to read the docs

Uh...  No, it indicates that I misunderstood your statement to mean,
"most of how you interact with Boost.Test is not through interfaces and
therefore doesn't need to be documented."  I now understand that you
weren't saying that, sorry.

> (I know they are not perfect, but the answer to above is clear even
> from skimming them)
>
> Public interface of unit test framework is predominantly macro based. This 
> includes test tree management and test tools (Well, new interfaces deviate from 
> this, but they are beyond the point). And almost every public interface (with 
> few exceptions which I already pointed out) ARE documented in details. There are 
> few non-macro public interfaces, which are documented as well. There are couple 
> interfaces one could use to extend library functionality, which we can add to 
> documentation, but these are:
>   * well above basic usage
>   * very rarely needed
>
> So should we document these? Probably yes, but its absence does not affect 
> majority of Boost.Test users and clearly does not preclude anyone from using it.
>
> So in summary, "every public interface needs to be documented" is true for the 
> most part in Boost.Test case. Or can you show any glaring gap?

Let me go back, now that we're having this discussion, and try again to
read the docs.  I'll try to point at specific things that leave me
bewildered.  At this point in time, I only really remember the
experience of bewilderment but not the details.  Also, I'll look at
the docs on the trunk, but please, move those docs to release immediately.

>> >> I don't know what to do about this.  Because of the lack of redundancy
>> >> (i.e. tests and documentation), it's hard to tell whether this library
>> >> is correct or even to define what "correct" should mean.  It seems like,
>> >
>> > I am not sure what you mean. There is an extensive self test modules.
>> 
>> I mean *redundancy* between the tests and the documentation.  The tests
>> should check that the library does what the documentation says it does.
>
> They do. Self tests are comprehensive and cover all public interfaces described 
> in documentation (unless I missed something). Do you have example of clear gaps?

Again, at this point the details have slipped from my brain.  I'll try
to get back to you.

-- 
Dave Abrahams
BoostPro Computing                  Software Development        Training
http://www.boostpro.com             Clang/LLVM/EDG Compilers  C++  Boost