Re: [boost] [testing] Refactoring serialization tests, and possibly others.

From: David Abrahams (dave_at_[hidden])
Date: 2005-10-13 10:33:13

"Robert Ramey" <ramey_at_[hidden]> writes:

> I will note one think. For my tests here, I use the Beman's
> compiler_status.cpp program which I "upgraded" (and loaded into the
> vault) to show all the results for different build types
> (release/debug/static library/...).  The prepares a gigantic table
> which is very useful to see that some failures are related to a
> particular build.

Noted.  Is that just an FYI, or was there some particular response you
were looking for?

>>> I would like to see the "Getting Started" or installation or
>>> whatever it is included a "configuration/system validation phase"
>>> whereby running the tests is a normal part of the boost
>>> installation procedure.
>>
>> Yikes!  Installing already takes way too long, in my experience.
>> Why?
>>
>> How many such problems have we seen?  Is there any evidence that this
>> would do something other than make installation longer for users?
>
> a) OK - make it optional.  Well its already optional so it would be just a 
> question of enhancing the documentation to make it easier for new users to 
> "validate" the instalation.

Patches welcomed :)

> b) On a regular basis we have new users who have problems.  

"We" meaning Boost?  I'm just asking because in my libraries at least
most new user problems don't seem to be related to platform and build
configuration; they are usually related to usage.

> This may be due the fact they are using a new library (e.g. stlport
> 4.?) or compiler variation (gcc releases more frequently than we
> do).  So when they ask a question about the serialization library, I
> really need to know if its a boost or system configuration issue or
> its an issue with the library itself.

So isn't that when you should ask them to run the tests?

> c) It would spread the testing effort.

Seems to me it would be happening a little too late to be considered
part of the "testing effort."  Testing, as we do it, is for catching
these problems *before* a release.

> d) we would be informed of new anomolies right away.

Why would a user who wouldn't normally take the trouble to tell us
about new anomalies be likely to do so if they were first instructed
to run all the tests?

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com