Re: [boost] Boost 1.36.0 release notice

From: Robert Ramey (ramey_at_[hidden])
Date: 2008-08-18 11:59:05

Vladimir Prus wrote:
> Robert Ramey wrote:

>> Taking the case of boost book.  As far as I know there is
>> no regression testing to verify that it works when something
>> it depends upon changes.
>
> I don't know, either. But large part of the doc toolchain is outside
> of Boost -- e.g. Docbook and FOP. Those are huge, and it's not
> likely somebody will be able to create tests for those. So, manual
> inspection of results remain the only solution.

I'm not advocating that tests be create for those things outside of
boost.  I'm advocating tests for those things in boost which
depend on things outside of boost as well as the thing itself.
We do the same thing with our libraries.  We write code
which depends up STL.  In the course of testing ,we discover
that we either made a mistake in using STL, or that various
verstions of STL have differing interpretations of
ambiguous points in some standard or have bugs in thier own.

>> My suggestion is a lot more than "we need tests".  My suggestion
>> is that boost tools should be subjected to the same procedures
>> that boost libaries are.  Using boost book as an example, I would
>> like to see boost/tools/boostbook/test/Jamfile ... and see that
>> boost book shows up in the test matrix just like any library does.

> I think that presence or lack of boost/tools/boostbook/test/Jamfile
> is an implementation detail.

That is the essential difference in out view point.

>> I would like to see boost book "packaged" with documentation,
>> examples and tests so that users would feel confident to use it for
>> their own
>> projects.  If the "boost process" is good for libraries, I think
>> it would be good for tools too.  I would love to use boost book
>> / quickbook for my own projects, but I don't feel its polished
>> enough to depend upon.
>
> I'm positively sure I saw documentation for boost.book. Quickbook is
> also documented. There's tools/quickbook/test/Jamfile.v2, and some
> test files there. For Boost.Build, there's both documentation and
> extensive  test suite.
>
> So, it appears that your proposal boils down to:
> 1. Boost.Book should have tests
> 2. Test for tools should be included in the main test matrix

> Have I left out something?

Not much.  The only thing I might add is that I would like to
see the directory structure code/tests/headers, etc reflect the boost 
pattern
for libraries.  This would make it easy to verify that everything
exists.

>> No one has suggested that testing proves the absense of bugs.
>
> I've understood your prior statement ("the issue raised above would
> not have occurred.") in exactly this way. I apologize if I've
> misunderstood what you was saying.

Well, maybe I didn't say it right.  The reason I chose this
particular instance as an example was that I believe that
the application of the "boost process" would have made
a difference in this particular example.  I didn't mean that
that this would solve ALL problems.  I believe that it
would result in a net improvement in quality and
productivity.

Robert Ramey