Re: Review: Boost.Test library

From: rogeeff (rogeeff_at_[hidden])
Date: 2001-12-12 20:55:22

--- In boost_at_y..., Ron Garcia <garcia_at_c...> wrote:
> Review of the Boost Test Library:
> 
> I vote to NOT accept the Boost Test Library.  My primary reason is 
the
> state of the documentation.  While I don't think that the library is
> particularly unacceptable, I think it's necessary to emphasize the
> importance of good, clear, complete documentation as a component of 
a
> good library.  I understand that improving the documentation has
> been discussed, but I would like to see at least a first round of 
these 
> improvements before accepting
> the library. 

See my comments below. A general comment: could you give an examples 
of the things you peopose to change.

> 
> DOCUMENTATION:
> 
> I would like to see more of a distinction between user documentation
> and reference documentation.  The reference documentation is crucial
> for expositing the interfaces in the library, and the interfaces are
> one of the most important components of a library.  The user
> documentation is also very important, especially in this case. 

How do you separate on user and reference documentation? From my 
standpoint right now presented ONLY user docmentation. Cause the 
users of the library are programmers, the documentation discribe in 
detail how to USE the facilities provided in it. The implentation 
detail are going to be in the separate design document.

> Having
> a test library that is easy to use and easy to get started with is
> important as incentive to begin developing testcases early and
> developing them as often as possible.

I agree with this. I wrote a "getting started" page for novice user. 
What else do you think I need to clarify and simplify in 
documentation to speed up learning curve?

> 
> A less important nit on the documentation:  I really liked the 
layout
> and design of the current test library's documentation much 
better.  Sure it 
> wasn't much in the way of eye candy, but the simplicity actually 
appeals to me
> much more.
> 

It a bit a taste metter. I did it that way cause I though it would be 
easeir to read. I would like to here other opinions.

> I would really like to see the Test Library Design Page completed.

I am working on it.
> 
> 
> SEPARATE BINARY LIBRARY:
> 
> It appears that in all cases one must link tto he new test 
framework as
> an external library.  Is that true, and if so, is this necessary?

It is not nessesary, but recomended. online_test is presented to show 
that you can use library included rather then linked.

> I did notice a mention of compilation speed as an issue, but
> I'm curious as to how great of an extent this is an issue?  

Try to compile avaradge test with framework inline and offline.  It's 
big enough difference to me.

> In particular, I am significantly less likely to use the simple 
execution
> framework for uniform reporting of exceptional conditions  if I 
need to link 
> yet another
> library with any application with which I wish to use it. 
> The simplicity of using the current execution tools -- add an
> include file, add a #define (for test tools), and change the name
> of main -- is worth it to me in some situations at the possible
> expense of some compile time. 

It's a metter of taste and habit. While deveping a component I can 
compile it fifty times a day. Every addional minute spend in 
compilation will cause me to lose ah hour.

> I also wonder if a different factoring could make some features 
> (especially at the execution tools level) available as a simple 
include while 
> a separate library is still mandated for the full blown tool?

What exactly do you mean?

> 
> In addition, I am wondering how soon Boost.Build will replace the
> regression.cpp test framework?  Does the current framework have 
hooks
> for linking in libraries with test files?  
> 
> Given that this library is implemented as .cpp files, there seems to
> be a lack of build materials, ie. boost.build, makefiles,
> etc. available. I'd like it to be as easy as possible to get started
> using the library.

As far as I know Boost.Build is powerfull enouph to accomodate 
Offline test framework.

> 
> 
> PREPROCESSOR MACROS:
> 
> I see that the library includes a great deal of macros for testing.
> Are all of them really necessary?  I realize that the macros for
> explicit tests are needed for embedding line number and file
> information, but what about the ones for creating test cases, test
> suites, etc.  Is this done for some consistency reason?  

Test Tools implemented as macros to incorporate __FILE__, __LINE__ 
and to automate message production.
BOOST_TEST_CASE is implemented as macro to automate test case name 
passing.
BOOST_TEST_SUITE is implemented as macro to be consistent. It does 
not bring any other value.

> 
> 
> EXIT_SUCCESS: 
> I would like to see boost::exit_success and boost::exit_failure
> (defined in boost/cstdlib.hpp) preferred over the C macros.  
Perhaps someday
> these values could find their way into Standard C++.

What do you mean?

> 
> 
> BOOST.FUNCTION SUPPORT:
> 
> As previously mentioned, I too would like to see Test Cases that can
> take Boost.Function objects.  In some situations it improves the 
test
> case code immensely.  If Boost.Function is tested without using that
> component of the test library, (or maybe not using the test library 
at
> all), then there is not too great a concern about cyclic
> dependencies. The gain for people
> writing unit tests would be worth it!

It'a already implemented. See an update.

> 
> 
> CONCLUSION:
> To reiterate what I previously said, the library is pretty good, 
but I
> think that the documentation should be more than an afterthought.  I
> would like to see a more final version of the documentation before 
> accepting the library.  I'd also like to see the other issues 
considered
> as well.  In my opinion this is a very important library.  
> 
> ron

Thank you for your comments,

Gennadiy.