Re: [boost] [serialization] Header Ordering Fixes

From: David Abrahams (dave_at_[hidden])
Date: 2006-05-28 08:48:47

"Robert Ramey" <ramey_at_[hidden]> writes:

> David Abrahams wrote:
>> "Robert Ramey" <ramey_at_[hidden]> writes:
>>
>>> As for not running the cross product of the various
>>> combinations - the problem is that failures now occur
>>> in just a couple of select situations that can't be
>>> predicted in advance.  BCB failed on text archives
>>> only in some cases.  STLPort 5.0 failed when
>>> using mapped collections.  etc. Paring down the
>>> test suite would likely have missed these failures.
>>> The selected tests would all pass - thereby giving the
>>> false impression that the library was bug free.
>
>> Oh, there's a lot you can do to reduce it without losing any
>> interesting data.  For example, there is no reason to believe you're
>> going to uncover serialization library bugs by testing against both
>> static and dynamic runtimes,
>
> I am testing for both static and dynamic runtimes. 

I think you meant "not testing," which I see is true.  You're testing
against static and dynamic versions of your library.

> When creating the DLL versions and also auto-linking it was
> different.

What was different?

> It is A LOT more than just recompiling.

What is?

> Now that things are working this is not necessary for debugging the
> serialization library itself.
>
>> or especially by testing both singlethreaded and multithreaded
>> builds, since you're not doing any threading in your tests.
>
> I don't believe I'm doing this.  Currently, tests are built with
> the default threading model.  That is, the Jamfile doesn't specify
> anything in this regard.

OK, maybe in my shock I overestimated.  Sorry.

>>If you discover a test failure that appears only in one of these
>>cases it is undoubtedly due to a compiler or standard library bug,
>>not anything having to do with your code.
>
> That was my original point.  If I'm not making any changes, which
> I haven't been,  almost no failures have anything to do with the
> serialization code.

But we want to find out when changes to other Boost libraries break
it.  And we want to find out when other people break it.

>> Also, you could cut the testing time by 2/3 instantly if you'd just
>> test all the archive types in a single executable, rather than
>> building a separate executable for each one.
>
> Hmmm - that was what I did originally but it was objected to
> in the first review.  

On what grounds?

> Also, more compilers failed with due to resource exhaustion.

On some weaker compilers you might have to break a few tests into
multiple translation units.

> The current system has the benefit of permitting
> any user to test his own archive with all the tests.  The documentation
> encourages users to do this with their own archives and I know they
> are doing it.  I know this because when ever a user has a problem
> with his archive - I just tell him to run the whole test suite on it.
> The current system also lets me run just one archive by
> adding a command line argument - extremely useful.

I noticed.  It's not only useful, but necessary, to cut down on the
number of targets the Jamfile generates.

All of that should be possible even with the tests built into a single
executable.

> I actually didn't have any idea how compilation and execution
> time varies by making the separation.  I had just presumed
> that compilation was more or less proportional to the number
> of archives - given that a lot of time is spent on serialization
> code instantiation which would be proportional to the number
> of archives.  Other code is already compiled in the library.
> Truth is since I run while I'm not watching I don't know
> how the time would vary.  How do you know this?

I don't know for sure, but it's easy to see that every time you start
a new compilation command there is overhead that can be avoided.
Every time you include the same file in a new TU there's overhead that
can be avoided.  One eventually develops a pretty good instinct for
these things after thinking about them a while.

>>And that would cover the
>> case where multiple archives are in use, which I don't think you're
>> covering today.
>
> see test_mult_archive_types
>
> In anycase, I have no problem with diminishing the the tests
> by just commenting out any combination of tests.  Also the
> dynamic library versions can easily be suppressed by tweaking
> the Jamfile.  As I said before, given that I'm making very few
> changes and I test those here, the boost test results are not
> used by me - or anyone else as far as I know.

The importance, to Boost as a whole, of having continuous testing of
all libraries should not be underestimated.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com