Re: [boost] Call for Review: Boost.Test documentation rewrite

Subject: Re: [boost] Call for Review: Boost.Test documentation rewrite
From: Richard (legalize+jeeves_at_[hidden])
Date: 2014-01-09 17:41:03

[Please do not mail me a copy of your followup]

boost_at_[hidden] spake the secret code
<loom.20140109T090205-364_at_[hidden]> thusly:

>Richard <legalize+jeeves <at> mail.xmission.com> writes:
>
>1. First and foremost: it is pretty much completely irrelevant
>Trunk version of boost.test will require us to rewrite 90% of what you did. 

I disagree.  I wrote 99% of everything here based on trunk which still
has all the stuff I documented -- and must have all of that in order
to be backwards compatible with release.

What you have done in trunk is add an unknown quantity of new,
untested features -- untested in the sense that noone is using them so
they have no user feedback as to the quality or usefulness of their
implementation.  You haven't documented them and you haven't solicited
anyone to perform any pre-release evaluation of those features.

I don't feel compelled to document something for which there appears
to be no design documentation, no statement of completeness, no
rationale for existing and has unknown amounts of bugs in it because
noone is really using it.

>I keep asking you to collaborate on something which we can actually release. 
>I am not sure why you keep ignoring me.

Funny, I could say the same thing.  I wrote my tutorial blog posts 4
years ago:

<http://article.gmane.org/gmane.comp.lib.boost.user/49092>
<http://article.gmane.org/gmane.comp.lib.boost.user/49110>
<http://article.gmane.org/gmane.comp.lib.boost.user/49111>
<http://article.gmane.org/gmane.comp.lib.boost.user/49113>
<http://article.gmane.org/gmane.comp.lib.boost.user/49114>

You responded saying that you would include a link to those tutorials
in the documentation:
<http://article.gmane.org/gmane.comp.lib.boost.user/49151>

That never happened.

On May 1st, 2013 I asked for early reviewers of this rewrite:
<http://article.gmane.org/gmane.comp.lib.boost.devel/240855>

You never replied to that, either in the list/newsgroup or in email.
All the people that replied were added to my list of reviewers and
have been getting updated snapshots along the way: Tom Kent, Alexander
Lamaison, Paul A. Bristow and Julian Gonggrijp (apologies if I
misspelled any of your names).  I went through at least 7 iterations
with those early reviewers, incorporating their feedback along the
way.

There was ample time for you to engage me at any step along the way.

In general, I have found you unresponsive to either questions from
myself or from others on the mailing list.  I would post questions
to the mailing list via the newsgroup on gmane.org and not get any
answer from you.  I have also sent you email to which you never
responded.

I found documentation bugs in the bug tracker that had been sitting
there for 4 years with no movement.  (I fixed them all in my rewrite.)

A good rule of thumb when you attempt to explain a technology is to
make sure that you can answer any questions that other people have
about that technology.  This is the approach I took for multiple years
with regards to Direct3D while I was writing my (free) book on the
subject.  I have experience in explaining APIs to a programmer audience.
<http://legalizeadulthood.wordpress.com/the-direct3d-graphics-pipeline/>

Questions get posted about Boost.Test on the user or developer mailing
lists and go unanswered by you and given the current state of affairs,
you were pretty much the only one that could answer them.

Lately, I have been answering questions about Boost.Test on the
mailing list, which has helped me to make sure that I have the
appropriate information in the documentation.  Ideally the
documentation should be good enough that people don't need to ask a
quesiton on the mailing list because they found the answer in the
documentation.

Examples:
<http://article.gmane.org/gmane.comp.lib.boost.devel/247897>
<http://article.gmane.org/gmane.comp.lib.boost.user/80578>
<http://article.gmane.org/gmane.comp.lib.boost.user/80577>
<http://article.gmane.org/gmane.comp.lib.boost.user/80575>
<http://article.gmane.org/gmane.comp.lib.boost.user/80574>
<http://article.gmane.org/gmane.comp.lib.boost.user/80341>
<http://article.gmane.org/gmane.comp.lib.boost.user/79220>
<http://thread.gmane.org/gmane.comp.lib.boost.user/78884/focus=78927>
<http://article.gmane.org/gmane.comp.lib.boost.user/78791>
...and more if you keep searching.

>2. This is not a Boost.Test documentation - this is is boost unit test 
>framework documentation

I find this to be a distinction that is both confusing and unnecessary
to users of Boost.Test.  Similarly, I removed all reference to "test
tools" because this terminology is non-informative (tool is too
generic) and not consistent with the terminology generally in use --
the industry calls these programming constructs assertions and I see no
reason why Boost.Test should deviate from industry standard terminology.

>You are missing the whole original point of layering in boost.test design. 

In 4+ years of using Boost.Test, I haven't found that I ever cared
about this.  When I talk to other people about the existing
documentation, they found all this discussion of these layers to be
distracting and irrelevant.  So I treated them as an internal
implementation detail that users didn't need to know about.

>You are missing description of all the other independent layers. In general 
>I think you are missing quite a few other things as well.

<shrug>  In my opinion, what remains is what is important and
necessary in order to use this library for unit testing.

>3. You eliminated the examples. I liked original examples much better with 
>their output etc.

I didn't eliminate examples, I replaced them.  Everything that you
need to do as a user of this library has an example.  For cases where
the output is important, the output is present in the documentation.
I don't see much utility in demonstrating the kind of message you get
from every single assertion when it fails.  I could be convinced
otherwise, but as a user of this library for 4+ years I haven't needed
it.  You simply look at the failed message and you know what to do
without reading documentation.

>4. You made quite a lot of description more terse

This was intentional.  The existing documentation used pages to
describe what could be explained in a paragraph or two, particularly
for the reference section.

>Take runtime parameters for example. Where I had few pages you left only few 
>paragraphs. I find new description very lacking.

Yet other reviewers have not commented on this at all.  That tells me
that the extra detail simply isn't needed.

>5. I understand you have your own favorite mocking library, but it is not 
>part of Boost.Test (not yet at least).

Correct, but once you start practicing TDD in C++ you quickly get to
the point where you need a mocking library.  Turtle is designed to
work seamlessly with Boost.Test and is proceeding towards being
submitted to Boost.  There are other mocking libraries out there and
I'm happy to link to them as well from the documentation, but the ones
that I've used in the past required more boiler-plate than Turtle.
gmock can no longer be used with Boost.Test because using gmock now
requires the use of gtest.  MockCpp requires more boilerplate and
hasn't been updated since Oct 2011.  There are probably others; while
I have a personal preference for Turtle that doesn't mean I'm adverse
to linking to any other mock libraries.

>6. I think quickbook navigation is seriously lacking.

Specific suggestions are welcome.
-- 
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
     The Computer Graphics Museum <http://computergraphicsmuseum.org>
         The Terminals Wiki <http://terminals.classiccmp.org>
  Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>