|
|
Boost : |
From: rogeeff (rogeeff_at_[hidden])
Date: 2001-12-20 15:20:34
--- In boost_at_y..., Jeremy Siek <jsiek_at_c...> wrote:
>
> The new Boost.Test library is not currently ready for inclusion in
> Boost mainly due to the documentation. However the new Boost.Test
is
> a good library and vitally needed by the Boost community. It will be
> given a "delayed acceptance", which means that it will be checked
into
> a CVS branch where Gennadiy and others can collaborate on improving
> the documentation and resolving the remaining issues. Once that
> process is complete, we will have another mini-review, and if that
> goes well Boost.Test will be added to the Boost library collection.
>
> Below are my comments on the library, followed by a summary/overview
> of other boost members' comments.
>
> My comments:
>
> index.html:
> The descriptions of the components are too vague. The words
> "monitor" and "control" are used, but the reader is left wondering
> what kind of monitoring, and what kind of control. Also, need
> to explain in which situations one would use one component, and
> not another component.
You are a bit generic here. For example:
The Program Exectuin Monitor - a simple helper facility to be used to
monitor a program execution. The Program Execution Monitor provides
the main() function and uses the Execution Monitor to control a
program execution.
What else do you want me to say here in the general introduction? And
what are "kinds" of monitoring or control exist?
I have tried to mark intended audiance for each component (reporting
facility,program,test program, test case). But I could try to add
some words here.
>
> execution_monitor.htm
> Would be good to start with an example to motivate the need for an
> execution monitor, and to show how to use it.
The execution monitor is not intended to be used in most cases. It
used internally to implement other components. But I though that this
class is pritty generic to be available to public reuse. For example,
by some advanced user to create some advanced testing or even in
production code to in a risky places to make sure that all errors are
caught. I do not know how to provide a simple real-life example and
is it worth do it. But if boosters think it should be included I can
add something.
> Also, the introduction
> does not flow very well. The train of thought jumps around.
> Also it should be longer.
You want me to reword everything?
> Also, the docs should compare the
> execution monitor with the program execution monitor to explain
> what the difference is, and when to use which.
You can't compare them. They are on different layers.
>
> prg_exec_monitor.htm
> Good example.
> Which exceptions are caught by the provided main, and how are
> they reported.
Original example does not throw any exceptions. And then I show an
examples what ahppend if exceptions do occured (I mostly inherit this
text from current test library)
> Should compare and contrast with the test execution monitor,
giving
> concrete examples.
I can add more to the benefits section.
>
> test_tools.htm
> Make sure the document the backwards compatibility macros.
I did not want t omention them cause I did not what users to use
them. But I can do it in special compartibility section. What would
be the reasons for that?
>
> BOOST_CHECK_EQUAL
> What are the requirements on the types of the parameters left
and
> right?
> I suppose there must be an operator<<(ostream, left_t)
> and operator<<(ostream, right_t) defined. Are there any other
> requirements also missing?
I argee it should be added. propable also operator==( left_t,
right_t ) should exist.
>
> test_exec_monitor.htm
> The docs for this component are quite good.
May be that is because it inherited from current Test Library ;-))
>
> unit_test_framework.htm
>
> How does the unit test library handle testing class templates and
> function templates? Typically one want to test with several
> variations on the template paremeters, in a systematic way.
> Would be good to have an example of this.
In to do list I sad that in plans is to support testing class and
function templates for the supplied typelist. But I will probably
wait till mpl to mature.
>
> Explain the situations when a user would choose function_test_case
> versus the class_test_case.
I did.
>
> Need to separate out what is reference material and what is
> tutorial material.
I need more explanations here. For example ...?
> Also, need to make sure there is tutorial
> and/or examples for every macro/function in the framework.
Did I missed something?
> Also, be careful to only document the public parts of the
> library, or perhaps separate parts of the interface into
> layers.
I have tryed. Even only public parts of some classes. Did I add any
implementation details?
> For example, to use the macros, does the casual user
> need to know about the underlying class that gets created?
Is there other way to discribe different kinds of test cases that you
could create. BOOST_TEST_CASE is only a mean to shorten typing for
object creation,
>
> Are the BOOST_TEST_CASE, BOOST_PARAM_TEST_CASE, and
BOOST_TEST_SUITE
> macro's really necessary? Then only save a little typing.
Compare:
BOOST_TEST_CASE( &test_constructors )
vs.
boost::unit_test_framework::create_test_case(
&test_constructors, "test_constructors" )
>
> I am very concerned with how BOOST_PARAM_TEST_CASE works. When
> trying to use this for the first time, I didn't realize that
> the parameter containers have to be global variables. I made
> them local, and then ran into memory problems. Even if this
> were documented well, I think it still poses a serious usability
> problem, especially when dealing with template classes, where the
> parameters may need to vary. One solution would be to have the
> parameter range *copied* into the test object. Perhaps there
> are better solutions...
I did describe all issues with parameters lifetime in documentation
plus the way how to solve it. But we may consider an alternatives.
>
> The extension to use bind looks good, that should definitely
> be included.
And documented ;-)
>
> Need examples of using the test_suite, unit_test_log,
unit_test_result
There are. unit_test_log, unit_test_result aren't intended to be
widely used by end-user. But all reasonable "usages" of them are in
tests.
>
> Perhaps should break this page up into several pages, each one
> focusing on a single function/class, which more examples and
> explanation of each one.
Perhaps. Why? Is it difficult to read?
>
> getting_started.htm
> * perhaps should list the invariants of the class, and discuss
> which tests cover which invariants
Ok.
> * forgot to test the case when length > strlen(s) for the
> constructor const_string(char const* s, size_t length). What
> should the specified behaviour of the constructor be?
Well I was not inteded to discuss const_string design here. But once
you mention that: It's not true. const_string in general is "big
beliver" ;). It's intentionally does not check length, cause it's
rationale to be efficient. So nothing to check by test case. (This is
partly discussed after the first version of test program)
> Would be good to have more full examples for class_test_case and
> for parameterized_class_test_case
I intentionally did not give any examples for class_test_case and
for parameterized_class_test_case, considering them an advanced
features, that should not be discussed in getting started page.
>
> Break up the unit_test_example.cpp into peices showing
> complete individual tests.
> Create an example file for the const_string.
Ok.
>
> Framework compilation
> Need jamfiles for everything. Should build libraries (.a's).
Agree. Plus jamfiles for self tests.
>
> Miscellaneous Comments:
> - The blue frame with the white border is not so great. It wastes
space.
It's not that I invent margins to the left and right. Look for
examples: cnn.com, yahoo.com, foxnews.com and tons of other places.
Colors are questionable. Look into
http://groups.yahoo.com/group/boost/files/unit%20test%
20framework/index.htm for an alternative.
> - The "Benefits" sections tend not to be very informative. Should
either
> drop them, or put more into them.
For example? They were originally.
>
> Testing:
> Would be nice to have documentation describing how the Boost.Test
> library was tested, describing which components are tested by
which
> tests. The current docs have a few pointers to files, but this
needs
> to be organized better, collected in one place, and made sure that
> it is complete.
What would be an advantages. I did not see that in any boost library.
Each component has a reference on appropriate test programs.
>
> Collected comments from others:
>
> From Dave
> documentation wording improvements
> was the proposal for the BOOST_THROW macro resolved?
No. Dave, could you again clarify what you meant?
>
> From Fernando, et. all
> the floating point equality checking issue, refactoring of the
> messages and the equality testing functions. Gennadiy has made
> changes to accomodate this, and we will need to look at the
> changes he made.
There still couple issues outstanding. See:
http://groups.yahoo.com/group/boost/message/21829
http://groups.yahoo.com/group/boost/message/21827
http://groups.yahoo.com/group/boost/message/21825
>
> From Bill:
> - Documentation needs improving.
> - BTW, I strongly disagree with the statement "(though obviously
this
> is not crucial to the acceptance or inclusion of this
library)".
> Good documentation *is* a *requirement* for boost libraries.
> - timeout implementation portability issues
It's in progress of discussion. But I did not expect it to be in the
first accepted version.
>
> From Ron:
> documentation improvements
> want clean separation between "reference" and "tutorial"
> documentation.
Specifically?
> (Ron was not talking about the separation between user and
> implementation
> documentation)
> separate binary library: this needs to be documented more clearly.
Specifically?
>
> From Beman:
> documentation
> jamfiles
> portability
partly done in last update.
> cleaner separation between "public" versus "detail" parts of the
> library
done in last update.
> details of test outputs
>
done in last update.
>
> --------------------------------------------------------------------
-- > Jeremy Siek http://www.osl.iu.edu/~jsiek/ > Ph.D. Student, Indiana Univ. B'ton email: jsiek_at_o... > C++ Booster (http://www.boost.org) office phone: (812) 855-3608 > -------------------------------------------------------------------- -- Regards, Gennadiy.
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk