Re: [boost] Call for Review: Boost.Test documentation rewrite

Subject: Re: [boost] Call for Review: Boost.Test documentation rewrite
From: Gennadiy Rozental (rogeeff_at_[hidden])
Date: 2014-01-10 11:30:41

Richard <legalize+jeeves <at> mail.xmission.com> writes:

> 
> [Please do not mail me a copy of your followup]
> 
> boost <at> lists.boost.org spake the secret code
> <loom.20140109T090205-364 <at> post.gmane.org> thusly:
> 
> >Richard <legalize+jeeves <at> mail.xmission.com> writes:
> >
> >1. First and foremost: it is pretty much completely irrelevant
> >Trunk version of boost.test will require us to rewrite 90% of what you 
did. 
> 
> I disagree.  I wrote 99% of everything here based on trunk

No. It is not.

>  which still has all the stuff I documented -- and must have all of that
> in order to be backwards compatible with release.

Sure it will. But this is not what we want users to use. New version has 
much better tools and interfaces, more runtime parameters, new subsystems 
etc.
 
> What you have done in trunk is add an unknown quantity of new,
> untested features -- untested in the sense that noone is using them so
> they have no user feedback as to the quality or usefulness of their
> implementation.  You haven't documented them and you haven't solicited
> anyone to perform any pre-release evaluation of those features.

The plan was to write docs and ask for review. We can't ask anyone to review 
new library features without some form of documentation. Otherwise all the 
features are tested - I have unit tests for them.

> I don't feel compelled to document something for which there appears
> to be no design documentation, no statement of completeness, no
> rationale for existing and has unknown amounts of bugs in it because
> noone is really using it.

You did not feel compelled to ask the question. Would you care to point me 
to a single boost library developer, who wrote design document before 
implementing library changes? And do you think if I would be one like this, 
I would not finish whole docs by now?

The procedure is to make the changes, test them, write some docs. Ask for 
review if changes are significant enough. I hoped we can collaborate on last 
part parts. Before library is out there, there obviously will be no one 
using it.

> >I keep asking you to collaborate on something which we can actually 
release. 
> >I am not sure why you keep ignoring me.
> 
> Funny, I could say the same thing.  I wrote my tutorial blog posts 4
> You responded saying that you would include a link to those tutorials
> in the documentation:
> <http://article.gmane.org/gmane.comp.lib.boost.user/49151>
> 
> That never happened.

You are not the only one. I have a long list of links with references to 
various blogs, web pages etc. I still plan to have a separate page in docs, 
with all the "web wisdom" mentioned. That said, documenting new features is 
still a priority. 

> On May 1st, 2013 I asked for early reviewers of this rewrite:
> <http://article.gmane.org/gmane.comp.lib.boost.devel/240855>
> 
> You never replied to that, either in the list/newsgroup or in email.
> All the people that replied were added to my list of reviewers and
> have been getting updated snapshots along the way: Tom Kent, Alexander
> Lamaison, Paul A. Bristow and Julian Gonggrijp (apologies if I
> misspelled any of your names).  I went through at least 7 iterations
> with those early reviewers, incorporating their feedback along the
> way.

I did not reply because in our previous conversion (few month before that) I 
ask for you to work on a new version instead of spending time on old one and 
you ignored me. I did not see a reason to waste time reviewing something 
which is no help for new release.
 
> I have also sent you email to which you never responded.

Where did you send it? and when? I obviously expressed interest in talking 
with you. All people who are interested, can reach me easily.
 
> I found documentation bugs in the bug tracker that had been sitting
> there for 4 years with no movement.  (I fixed them all in my rewrite.)

Most of them are already fixed in a trunk version of Boost.Test docs.

> Questions get posted about Boost.Test on the user or developer mailing
> lists and go unanswered by you and given the current state of affairs,
> you were pretty much the only one that could answer them.

There are few threads from last several month in my "to answer" bin indeed. 
I am very busy and not always can get to it right away. If question is 
answered fine by someone else, I am not going to chip in second time. 

> >2. This is not a Boost.Test documentation - this is is boost unit test 
> >framework documentation
> 
> I find this to be a distinction that is both confusing and unnecessary
> to users of Boost.Test.  Similarly, I removed all reference to "test

I do not think you can make this decision.

> tools" because this terminology is non-informative (tool is too
> generic) and not consistent with the terminology generally in use --
> the industry calls these programming constructs assertions and I see no
> reason why Boost.Test should deviate from industry standard terminology.

This is fine. Especially since in a new version most of it will come to a 
single type of assertion.

> >You are missing the whole original point of layering in boost.test 
design. 
> 
> In 4+ years of using Boost.Test, I haven't found that I ever cared
> about this.

Well, you are not the only user of the library. There are lot more people 
using these features than you think. UTF is obviously the most used part of 
Boost.Test, but not the only one.

> When I talk to other people about the existing
> documentation, they found all this discussion of these layers to be
> distracting and irrelevant.

Those 5 people you were talking to are not the only users of the library. 

> So I treated them as an internal
> implementation detail that users didn't need to know about.

It is not. It is part of public interface and needs to be properly 
documented.
 
> >You are missing description of all the other independent layers. In 
general 
> >I think you are missing quite a few other things as well.
> 
> <shrug>  In my opinion, what remains is what is important and
> necessary in order to use this library for unit testing.

In my opinion - it's not.

> >3. You eliminated the examples. I liked original examples much better 
with 
> >their output etc.
> 
> I didn't eliminate examples, I replaced them.  Everything that you
> need to do as a user of this library has an example.  For cases where
> the output is important, the output is present in the documentation.
> I don't see much utility in demonstrating the kind of message you get
> from every single assertion when it fails.  I could be convinced
> otherwise, but as a user of this library for 4+ years I haven't needed
> it.  You simply look at the failed message and you know what to do
> without reading documentation.

In my experience of supporting this library (which is much longer)expected  
output is always a good thing. I would rather keep all of these.

> >4. You made quite a lot of description more terse
> 
> This was intentional.  The existing documentation used pages to
> describe what could be explained in a paragraph or two, particularly
> for the reference section.

Well, this is a matter of opinion. I am willing to discuss this.
 
> >Take runtime parameters for example. Where I had few pages you left only 
few 
> >paragraphs. I find new description very lacking.
> 
> Yet other reviewers have not commented on this at all.  That tells me
> that the extra detail simply isn't needed.

You reviewers are already experienced users of the library. They may not 
have an unbiased view on this.
 
> >5. I understand you have your own favorite mocking library, but it is not 
> >part of Boost.Test (not yet at least).
> 
> Correct, but once you start practicing TDD in C++ you quickly get to
> the point where you need a mocking library.  Turtle is designed to
> work seamlessly with Boost.Test and is proceeding towards being
> submitted to Boost.  There are other mocking libraries out there and
> I'm happy to link to them as well from the documentation, but the ones
> that I've used in the past required more boiler-plate than Turtle.
> gmock can no longer be used with Boost.Test because using gmock now
> requires the use of gtest.  MockCpp requires more boilerplate and
> hasn't been updated since Oct 2011.  There are probably others; while
> I have a personal preference for Turtle that doesn't mean I'm adverse
> to linking to any other mock libraries.

Let's review this separately. I have a preference to what I already have on 
Boost.Test obviously. In any case We should mention all possible candidates 
(including older version of gmock if if is applicable).

> >6. I think quickbook navigation is seriously lacking.
> 
> Specific suggestions are welcome.

Didn't you just delete one: let's consider something along the lines of 
Boost.Preprocessor.

Regards,
Gennadiy