Re: [boost] [serialization] Header Ordering Fixes

From: Robert Ramey (ramey_at_[hidden])
Date: 2006-05-28 00:43:02

David Abrahams wrote:
> "Robert Ramey" <ramey_at_[hidden]> writes:
>
>> As for not running the cross product of the various
>> combinations - the problem is that failures now occur
>> in just a couple of select situations that can't be
>> predicted in advance.  BCB failed on text archives
>> only in some cases.  STLPort 5.0 failed when
>> using mapped collections.  etc. Paring down the
>> test suite would likely have missed these failures.
>> The selected tests would all pass - thereby giving the
>> false impression that the library was bug free.

> Oh, there's a lot you can do to reduce it without losing any
> interesting data.  For example, there is no reason to believe you're
> going to uncover serialization library bugs by testing against both
> static and dynamic runtimes,

I am testing for both static and dynamic runtimes.When creating
the DLL versions and also auto-linking it was different.  It is A LOT more
than just recompiling.  Now that things are working this is not necessary
for debugging the serialization library itself.

> or especially by testing both
> singlethreaded and multithreaded builds, since you're not doing any
> threading in your tests.

I don't believe I'm doing this.  Currently, tests are built with
the default threading model.  That is, the Jamfile doesn't specify
anything in this regard.

>If you discover a test failure that appears
> only in one of these cases it is undoubtedly due to a compiler or
> standard library bug, not anything having to do with your code.

That was my original point.  If I'm not making any changes, which
I haven't been,  almost no failures have anything to do with the
serialization code.

> Also, you could cut the testing time by 2/3 instantly if you'd just
> test all the archive types in a single executable, rather than
> building a separate executable for each one.

Hmmm - that was what I did originally but it was objected to
in the first review.  Also, more compilers failed with due to
resource exhaustion.  The current system has the benefit of permitting
any user to test his own archive with all the tests.  The documentation
encourages users to do this with their own archives and I know they
are doing it.  I know this because when ever a user has a problem
with his archive - I just tell him to run the whole test suite on it.
The current system also lets me run just one archive by
adding a command line argument - extremely useful.

I actually didn't have any idea how compilation and execution
time varies by making the separation.  I had just presumed
that compilation was more or less proportional to the number
of archives - given that a lot of time is spent on serialization
code instantiation which would be proportional to the number
of archives.  Other code is already compiled in the library.
Truth is since I run while I'm not watching I don't know
how the time would vary.  How do you know this?

>And that would cover the
> case where multiple archives are in use, which I don't think you're
> covering today.

see test_mult_archive_types

In anycase, I have no problem with diminishing the the tests
by just commenting out any combination of tests.  Also the
dynamic library versions can easily be suppressed by tweaking
the Jamfile.  As I said before, given that I'm making very few
changes and I test those here, the boost test results are not
used by me - or anyone else as far as I know.

Robert Ramey