Re: [boost] Boost.Test - misc observations

From: Gennadiy Rozental (gennadiy.rozental_at_[hidden])
Date: 2006-06-12 01:44:25

"Robert Ramey" <ramey_at_[hidden]> wrote in message 
news:e6etq0$7he$1_at_sea.gmane.org...
> Dear Gennadiy Rozental,

Hmm, no "sir" - well for the fellow booster plain Gennadiy should work ;o)

> I've converted my local copy of the serialization library tests to use 
> test/minimal.hpp in the hope that I can use it in the future as the 
> "lowest common denominator" to permit testing against a wider array of 
> platforms.

Hi, Robert

If you care for my opinion, it would not be a wise move, unless you don't 
case about the quality of your tests of course. Current state of the Unit 
Test Framework (UTF) allows me to state that it's a preferred solution in 
all possible use scenarios (simplest one-liner, simple single function test 
with several assertions or complex multilevel test case structure) from all 
possible angles:

* usability
* functionality
* compilation time

There was a thread some time ago where I gave detailed comparison. As for 
the "wider array of platforms" - Boost.Test doesn't delve in most part any 
gray areas of the language and so both components covers the same set of 
platforms: at least among those that we test on. I personally never use it. 
The only scenario I could even imagine it make sense to use is write a 
"quickie" - simple test I plan to throw immediately in some unfamiliar 
environment, where I do not have time/permission to build a library on. Even 
in this scenario I would rather use included version of UTF.

> Boost Test for support of msvc is scheduled to be remove in the near 
> future.

We discussed this couple time and I thought I was clear.
Ok. Lets imagine two years from now boost decides to drop msvc 6.5 from 
regression testing. Would you still insist on maintaining this 
configuration? Even though none of the boost components you depend on are 
going to be tested on it and accordingly will start to fail sooner or later? 
My hope is that your answer is no. In this case I could most solemnly swear 
that Boost.Test will work on all platforms boost perform regression testing 
on and you have nothing to worry about.

 > I've had problems testing the serialization library with comeau compilers
> because both the serialization library and boost test include instantiated 
> code for string output and the comeau "pre-linker" fails here.

I do remember some issues with this compiler. But  I don't think that we 
couldn't find some workaround.

> I've felt the need for a "lighter weight" version of boost test.  Recently 
> on a contract job of mine I used some headers from boost.  I didn't need 
> to compile any boost libraries in this case.  The existence of 
> test/minimal.hpp permitted me to use boost test without going building the 
> libraries which turned out to be very convenient for a number of reasons.
>
> So - I'm happy with test/minimal.hpp -
>
>
> But - in order to make it work with the serialization library I had to 
> make a couple of tweaks which I would like to submit in the hope that they 
> can be incorporated into the official boost test version.
>
> Change # 1
>
> A small change to permit commeau to be tested with the serialization 
> library.  This is obviously a crude hack but perhaps it can be fine tuned 
> to make it acceptable.
>
> RCS file: /cvsroot/boost/boost/boost/test/impl/unit_test_parameters.ipp,v

I don't believe minimal_test.hpp depends on unit_test_parameters.ipp. So I 
do not see why you brig it up here. But post release we could revaluate the 
best fix for commeau issue.

> Change #2
>
> I added some test tool macros for test/minimal.hpp .  I would like to see 
> these incorporated but of course could life without them if I have to.
>
> Index: minimal.hpp
> ===================================================================
> RCS file: /cvsroot/boost/boost/boost/test/minimal.hpp,v
> retrieving revision 1.19
> diff -r1.19 minimal.hpp
> 17a18,19
>> #define BOOST_TEST_INCLUDED
>>
> 22a25,29
>> #define BOOST_WARN(exp)        \
>>   ( (exp)                      \
>>       ? static_cast<void>(0)   \
>>       : boost::minimal_test::report_message(#exp,__FILE__,__LINE__, 
>> BOOST_CURRENT_FUNCTION, true) )
>>
> 27a35,62
>> #define BOOST_CHECK_MESSAGE(exp, msg_)       \
>>   ( (exp)                      \
>>       ? static_cast<void>(0)   \
>>       : boost::minimal_test::report_error(msg_,__FILE__,__LINE__, 
>> BOOST_CURRENT_FUNCTION) )
>>
>> #define BOOST_WARN_MESSAGE(exp, msg_)       \
>>   ( (exp)                      \
>>       ? static_cast<void>(0)   \
>>       : boost::minimal_test::report_message(msg_,__FILE__,__LINE__, 
>> BOOST_CURRENT_FUNCTION) )
>>
>> #define BOOST_REQUIRE_MESSAGE(exp, msg_)     \
>>   ( (exp)                      \
>>       ? static_cast<void>(0)   \
>>       : 
>> boost::minimal_test::report_critical_error(msg_,__FILE__,__LINE__,BOOST_CURRENT_FUNCTION))
>>
>> #define BOOST_WARN_EQUAL( left, right ) \
>>     BOOST_WARN((left) == (right))
>> #define BOOST_CHECK_EQUAL( left, right ) \
>>     BOOST_CHECK((left) == (right))
>> #define BOOST_REQUIRE_EQUAL( left, right ) \
>>     BOOST_require((left) == (right))

These do not bring any value. Why would you want something like this?

>> #define BOOST_TEST_CHECKPOINT(msg_) \
>>         boost::minimal_test::report_message( (msg_),__FILE__,__LINE__, 
>> BOOST_CURRENT_FUNCTION, true )
>>
>> #define BOOST_TEST_MESSAGE(msg_) \
>>         boost::minimal_test::report_message( (msg_),__FILE__,__LINE__, 
>> BOOST_CURRENT_FUNCTION, true )
>>
> 29a65
>>
> 32a69,70
>> #define BOOST_TEST_DONT_PRINT_LOG_VALUE( ArgumentType )
>>
> 36c74,75
> < #include <boost/test/detail/global_typedef.hpp>
> ---
>> #define BOOST_DISABLE_WIN32
>> //#include <boost/test/detail/global_typedef.hpp>
> 38c77
> < #include <boost/test/utils/class_properties.hpp>
> ---
>> //#include <boost/test/utils/class_properties.hpp>
> 42c81
> < #include <boost/cstdlib.hpp>            // for exit codes#include 
> <boost/cstdlib.hpp>            // for exit codes
> ---
>> //#include <boost/cstdlib.hpp>            // for exit codes#include 
>> <boost/cstdlib.hpp>            // for exit codes
> 47c86
> < #include <string>                       // std::string
> ---
>> //#include <string>                       // std::string
> 63c102
> < report_error( const char* msg, const char* file, int line, const_string 
> func_name, bool is_msg = false )
> ---
>> report_message( const char* msg, const char* file, int line, const_string 
>> func_name, bool is_msg = false )
> 65d103
> <     ++errors_counter();
> 79a118,124
>> report_error( const char* msg, const char* file, int line, const_string 
>> func_name, bool is_msg = false )
>> {
>>     ++errors_counter();
>>     report_message(msg, file, line, func_name, is_msg);
>> }
>>
>> inline void

All in all it's just repetition of some of the staff UTF provides. Soon you 
need some other missing feature and next thing you know we copy/paste the 
whole UTF implementation into single header file. I would really want to 
avoid this repetition. minimal.hpp exist for particular purpose and I don't 
see a need to extends it's boundaries. It's just not a message I would like 
to send to Boost.Test users.

> Thank you very much - I should say that one of your boost articles - maybe 
> its still in there about how testing can change your life had a great 
> effect on me.  It has been very helpful to me personally.  The "testing 
> philosophy" expounded there - along with the instant availability of 
> test/minimal.hpp - has made a night / day difference in the quality of 
> code produced at one
> of my customers.

I don't want to repeat myself, but I would recommend to revaluate some other 
components presented by Boost.Test. UTF would be give you and your customer 
much more whatever your testing needs are.

> Though off topic, I never really thanked you for your in depth review of 
> the my original serialization library submission.  Except for one tiny 
> slip ("... contains this gem"), it was an exhaustive point by point 
> critique of every aspect of it.  Not that I enjoyed being on the receiving 
> end of such criticism, it was clear to me that it was basically correct 
> and well thought out.. After I recovered from the rejection I went back 
> and re-did it using your critique as a checklist and was eventually 
> successful in getting the library accepted.  So, for better or worse, 
> this library is part of boost only because of your six hour investment.

It was (and still is) a pleasure collaborate with you either. Let's hope you 
will find my response to your critique respectively successful.

Regards,

Gennadiy