Re: [boost] What Should we do About Boost.Test?

Subject: Re: [boost] What Should we do About Boost.Test?
From: Vicente J. Botet Escriba (vicente.botet_at_[hidden])
Date: 2012-09-19 02:43:54

Le 19/09/12 08:08, Robert Ramey a écrit :
> Vicente J. Botet Escriba wrote:
>> Le 19/09/12 01:35, Robert Ramey a écrit :
>>> Mathias Gaunard wrote:
>>>> Isn't Boost.Test's only problem the fact that other Boost libraries
>>>> are tested against the trunk rather than the release version?
>>>>
>>>> Isn't that something the modularized Boost is supposed to fix?
>>> There's no need to wait for "modularized Boost" to start doing this.
>>> This could be done now with relatively modest (though of course
>>> tricky) adjustments in the test script.
>>>
>>>
>> Hi Robert,
>>
>> IIUC, your strategy to tests has some advantages, but it has some
>> liabilities also:
>>
>> * every developer should use the same strategy,
> nope: I've been testing against the release versions of other libraries
> for years.  MUCH less problem than testing against the trunk/experimental
> version.
See below my comment to your point.
>
> If this were the case, new features could be added in trunk without
> any problem, and the authors could have enough time to move to the new
> breaking ones.
> Nope.
>
> a) Suppose I'm the author of Boost Test.
> b) a user requests a new feature or bug fix.
> c) I make the change on my machine and test it. - looks good.
With which compilers? I would expect that authors are checking using 
several compilers that could be freely available.
> d) I check into the trunk.
> e) turns out that the changes have subtle error which shows up
> in half the compilers/os tested.
Yes, this could occur, but less if the author test with several compilers.
> f) Now all the libraries using boost test start to fail their tests
> on half their platforms.
> g) fifty developers now have to check their code for errors and
> track it down to boost test.
> h) finally it get's sorted out and the author of boost test is
> on the hot seat for hanging up all of boost.
The impact of such a situation could be minimized if new features are 
added subject to a new version compilation flag.
I'm not saying I'm doing it every time, but it reduces dependencies.

The impact could be minimized if the author do little commits as the 
rollback is easier.
>
> This is a common scenario when happened on more than one occasion.
>
> Now the author of boost test has learned his lesson.  He doesn't make
> any but the most minor changes. If he makes a minor change, he has
> to go to extraordinary lengths to avoid the above. He can't de-couple
> his work from the rest of boost and work at his own pace.
>
> Of course the same applies to boost tools as well.
>
>> Of course we are all subject to making some errors, but at least we
>> should have in mind the consequence of unexpected breaking changes.
> of course.
>
> I urge anyone who doubts my position on this to try a simple experiment.
Don't misunderstand me. I have no doubts.
>
> a) Start with the release version of boost on your local machine.
> b) switch your directories header and lib to the trunk via SVN
> c) Run the bjam for your libraries
d) do some modifications and test
e) switch to trunk
f) verify that everything is OK
g) commit

If you test using the release of the other libraries only, the trunk 
could be broken more easily. This is why I said that every author should 
use the same strategy.
>
> It works great!
>
> And it's much faster since you only have to sync the release branch once
> in a while rather than every day.
I guess you will need to sync at least every time you commit, isn't it?

Best,
Vicente