Re: [boost] What Should we do About Boost.Test?

Subject: Re: [boost] What Should we do About Boost.Test?
From: Robert Ramey (ramey_at_[hidden])
Date: 2012-09-19 02:08:06

Vicente J. Botet Escriba wrote:
> Le 19/09/12 01:35, Robert Ramey a écrit :
>> Mathias Gaunard wrote:
>>> Isn't Boost.Test's only problem the fact that other Boost libraries
>>> are tested against the trunk rather than the release version?
>>>
>>> Isn't that something the modularized Boost is supposed to fix?
>> There's no need to wait for "modularized Boost" to start doing this.
>> This could be done now with relatively modest (though of course
>> tricky) adjustments in the test script.
>>
>>
> Hi Robert,
>
> IIUC, your strategy to tests has some advantages, but it has some
> liabilities also:
>
> * every developer should use the same strategy,

nope: I've been testing against the release versions of other libraries
for years.  MUCH less problem than testing against the trunk/experimental
version.

> * the trunk will surely be broken as no one is testing with the trunk
> of  the others libraries,

> * it delays the day conflicts are evident to the day the developer
> merges to release, as no one has tested with the new changes.

This presumes that using other libraries to test one's own library
is a useful strategy. Discovering a bug this way puts the burden
of tracking it down on other library authors.  This encourages
them from using other libraries and sometimes makes them
want to "roll their own" (as I and others have for boost test).

> I think the authors of the Boost libraries should try to avoid the
> introduction of breaking changes. Breaking changes should be manage
> using versions and deprecated periods.

we're all agreed on that.

> If this were the case, new features could be added in trunk without
> any problem, and the authors could have enough time to move to the new
> breaking ones.

Nope.

a) Suppose I'm the author of Boost Test.
b) a user requests a new feature or bug fix.
c) I make the change on my machine and test it. - looks good.
d) I check into the trunk.
e) turns out that the changes have subtle error which shows up
in half the compilers/os tested.
f) Now all the libraries using boost test start to fail their tests
on half their platforms.
g) fifty developers now have to check their code for errors and
track it down to boost test.
h) finally it get's sorted out and the author of boost test is
on the hot seat for hanging up all of boost.

This is a common scenario when happened on more than one occasion.

Now the author of boost test has learned his lesson.  He doesn't make
any but the most minor changes. If he makes a minor change, he has
to go to extraordinary lengths to avoid the above. He can't de-couple
his work from the rest of boost and work at his own pace.

Of course the same applies to boost tools as well.

> Even when we will have a modularized Boost, we should manage with
> breaking changes in the same way (versions and deprecated periods).

no question.

> Of course we are all subject to making some errors, but at least we
> should have in mind the consequence of unexpected breaking changes.

of course.

I urge anyone who doubts my position on this to try a simple experiment.

a) Start with the release version of boost on your local machine.
b) switch your directories header and lib to the trunk via SVN
c) Run the bjam for your libraries

It works great!

And it's much faster since you only have to sync the release branch once
in a while rather than every day.

If anyone trys this out and doesn't see the value in it, I would like to
hear about it.

Robert Ramey