Re: [boost] [test] Looking for co-developer/maintainer

Subject: Re: [boost] [test] Looking for co-developer/maintainer
From: Darryl Green (darryl.green_at_[hidden])
Date: 2014-05-25 00:16:59

On 24 May 2014 11:16, Richard <legalize+jeeves_at_[hidden]> wrote:
> [Please do not mail me a copy of your followup]
>
> boost_at_[hidden] spake the secret code
> <CADJ=SEPWyZosZh=V1u95Upu_ERyHRaTeJ53Ku5CSRu-dPuY7gw_at_[hidden]> thusly:
>
>>Agreed there is no explanation. But clearly the list of section headings
>>includes the heading *Custom report format support* as well headings
>>for  several other ways of controlling report output.
>>To that extent it documents that extensibility exists.
>>This is better than not mentioning the capability at all.
>
> I think reasonable people can disagree on whether this is better or
> not :-).
>
>>Your doc would be no worse then the original if it simply stated that
>>these features existed. Anything else is an enhancement request...
>
> I agree with this.
>
> If I were going to need custom reports, I'd take the XML report output
> and run it through a XSLT file before I spent time writing code, but
> maybe writing C++ is easier for people than XSLT.

Yes. Personally XSLT is a language I largely re-learn every time I
need to use it. I suspect many users of C++ unit test libs find it
easier to write C++ than XSLT. In any case documenting the isomorphism
between the C++ rep and the XML is worthwhile imho.

>
> In other words, except for the missing hint, these aren't deficiencies
> of my docs compared to the existing docs.

Correct.

>
>>Regardless of the question, the answer to the question requires use of the API
>>(or a commandline option that doesn't exist). The documentation should provide
>>answers, not questions.
>
> AFAIK, the existing documentation does not explain how to use the
> implementation classes to traverse the test tree.  So this is an
> enhancement.

Yes.
>
>>>>custom log formats:
>>>>http://www.eld.leidenuniv.nl/~moene/Home/projects/testdox/boosttest/
>>>
>
> Again, this is something that
> the existing documentation does not explain at all, other than perhaps
> hinting that you can have a custom report.
>
> ok, so you want:
>
> 1) custom report format support
> 2) (and I assume, it's cousin) custom log format support
> 3) traversing the test tree
>
> AFAIK, other than hints, these are all additions to the existing
> documentation.  (In fact what I wrote about the report format is
> already an addition to the existing docs.)
>
> So achieving better than parity with existing docs would be to simply
> say "You can customize the report output; see <this example>" and link
> to that url.

Almost. As accidentally demonstrated already URL stability is a
problem and the docs should be self contained/usable offline. It
appears use of that example including copying and editing is allowed
with attribution anyway - see
http://www.eld.leidenuniv.nl/~moene/Home/about/copyright/ with a
prohibition only on wholesale page copying. Maybe the author would
give explicit permission to include in boost?

>
>>You have twice asserted that uses of the API can be replaced by use of a
>>commandline argument. This is not really true (it doesn't solve my use case)
>
> Not sure what your use case is, but definitely listing out the tests
> in your project and getting custom report or log output can be handled
> without writing C++ code, albeit you might need to write some XSLT
> code.

A mix of test run control and output collection on a system that has
resource constraints fwiw. I agree this is unusual, but I also feel
that requiring yet another tool in a chain just to get the output
format you want etc is not a great solution, in general.

>>I agree it is unclear. I'm only trying to provide some clarity (with sources)
>>as to what is demonstrably already intended to be and in use as part of the
>>interface, so that the docs can make that clear.
>
> I think you raise some reasonable immediate points that can be
> addressed by adding a few sentences.
>
> Longer-term you're asking for new documentation on classes in the
> library.  I think that is also a reasonable request, but it's beyond
> the scope of what I'm trying to obtain with this change.
>

Agree with all of that.

>>>[I left out stuff about the Program Exeuction Monitor (PEM)]
>>
>>I agree it isn't used by itself. However knowing what it does (and some level
>>of how at the level of identifying what platform specific features it
>>uses) is useful.
>
> Is it really?  To me, it's an implementation detail.  The main thing
> you need to know about the PEM is that it attempts to intercept all
> badness and capture that as a test failure: unix signal(2), Windows
> SEH style exception, and C++ uncaught exception.
>

I found a brief mention of this in your doc of unit_test_main at
http://user.xmission.com/~legalize/tmp/boost.test/libs/test/doc/html/test/reference/test_classes/unit_test_main.html

There is also some reference to the behavior of the PEM in exception
handling tutorial.

While I don't think much depth is needed a clear description of what
the test main interface guarantees to catch/monitor and what happens
when it does should be part of the docs. I'd be inclined to separate
that from an individual function (unit_test_main) so in that sense it
is documenting the PEM, but only to describe what it does *in* the UTF
and not how to use it outside of it. Belatedly realised that this
makes it essential to document the role of the exception translator
interface...

For my use, the above constitutes is complete/sufficient docs to use
the library. I also think it documents sufficient of the library to
define/scope what is part of the interface (and needs to be stable for
that reason) without unduly constraining implementation. I don't have
any more objections/requests for yet more docs up my sleeve to hit you
with later :)