Re: [boost] shared_ptr under OS X

From: Jeff Garland (jeff_at_[hidden])
Date: 2005-08-23 19:59:15

On Wed, 24 Aug 2005 00:26:17 +0300, Peter Dimov wrote
> Ben Artin wrote:
> 
> > Can someone explain exactly why this wasn't caught in unit tests? I
> > am not sure I understand why we missed this, but I am pretty sure
> > it's important that we try to avoid this happening in the future.
> 
> The Dirxion tests:
> 
>
http://engineering.meta-comm.com/boost-regression/CVS-HEAD/developer/smart_ptr.html
> 
> are being run with inlining turned off.
> 
> The bug only manifests itself when inlining is on.

For what it's worth, there is a recognition of the issue of running regression
tests under debug only (search the mail archive if you really want to read
about it).  And, in fact, we do now have some platforms running tests using
release configurations where inlining should be enabled.  I believe the 1.33
is the first Boost release to have achieved this level of testing -- so, in
fact, I believe the level of Boost testing has never been higher.

> > Sure, bugs happen. My main point here is that we should make sure we
> > learn something from them -- or, more importantly, that we should
> > learn something from them *before* our users learn that Mac OS X
> > releases of boost are not thoroughly tested.

What I'd learn is that Boost is getting ever bigger in size and more popular.
 So even if we are getting better at testing we might very well see 'more
bugs' after a release because there's more code to have bugs and more people
with more configurations to see them.  This dynamic is likely to continue.
 
> The lesson here is that we can't test everything and that we must 
> make a beta or an RC available earlier. We've had similar 
> configuration-specific bugs before, and they weren't caused by 
> platform-specific implementations. Alpha testing simply can't 
> obviate the need for beta tests. 

I agree.

Jeff