Re: [boost] Boost 1.36.0 release notice

From: Vladimir Prus (vladimir_at_[hidden])
Date: 2008-08-18 02:35:05

Robert Ramey wrote:

> David Abrahams wrote:
>> Beman Dawes wrote:
>>
>>> Probably, but I don't know how to attack some of the tool problems
>>> otherwise. For example, the problem of people changing the tool chain
>>> without realizing it has an impact on the automated release tools, and
>>> the problem of the automated release tools no longer producing some
>>> component (like docs) because of a tool change, and no one noticing.
>>
>> on Sun Aug 17 2008, "Robert Ramey" <ramey-AT-rrsd.com> wrote:
>>
>>> My real point is that there is no reason that any tools that
>>> are used by boost should be treated any differently than
>>> any libraries used by boost.
>>
>> Does that assertion have any practical implications for the issue
>> being discussed?
> 
> LOL - of course it does.

Is there any reason the above exchange makes you laughing out loud?

> If regression testing were setup for 
> boost tools, they would be demonstrated to be functioning as
> expected before they were used in the actual release process.
> Had this been procedure been in place, the issue raised
> above would not have occurred.

Last time I've checked, boost.serialization had some automatic tests. Does it
mean that no issue has ever occurred with boost.serialization, and no issue
will ever occur in future? I don't think so.

Quoting Beman:

        Case in point, the doc build tool chain broke for this 
        release, forcing a really messy workaround. Joel is trying to fix it 
        now, but it is slow going since the test cases that fail for others 
        works for him.

So, it appears like the problem is not that there's no *testing infrastructure*
is missing, it's that for some behaviours, there are no tests or those tests
are not sufficiently reliable. Therefore, it does not seem like generic advice
of "we need tests" is going to help much -- there needs to be actual work on
specific tests.

And even if you add a new test for each issue encountered during release process,
it does not automatically mean that the resulting release packages need not be 
examined. For example, here at work we do have very extensive automatic tests, 
but still, every user-visible package goes through manual QA -- and yes, it does
checks that documentation exists.

- Volodya