Re: [jamboost] Regression test: link-fail problem?

From: Beman Dawes (bdawes_at_[hidden])
Date: 2002-08-21 16:21:11

At 02:23 PM 8/21/2002, David Abrahams wrote:

>> In another thread, you said:
>>
>> > The unit-test rule is really not very sophisticated: it makes
>> > the .exe target dependent on a successful run. Of course that
>> > means that it gets removed if the run fails.
>>
>> Is there some reason you can't do this for the regression tests?
>
>When you try to rebuild a bunch of tests, do you want it to work on
>rebuilding tests which have no chance of succeeding because nothing has
>changed since the last failure? That's the reason unit-test was deemed
>inadequate in the first place.

At the present, only way to tell a test should be reported as failed, if 
the reason for the fail is that a dependency failed, is to delete all the 
bin directories before running bjam. In other words, rebuild every 
library, rebuild and rerun every test. That's what we have to do now.

So almost anything is better than that, as long as it doesn't cause any 
tests to report the wrong result.

But see below. Just deleting the .exe (or any other) file would do. 
(Something that gave a more positive indication of why the test was to be 
considered failed would be even more helpful, but I realize that is 
probably way beyond V1).

>> I'll
>> settle for any clear indication at all a, b, c, and d have failed when 
e
>> fails. It doesn't have to be the presence or absence of
>> .test/.success/.failure files.
>
>I could easily arrange for the .exe to be removed if any of its
>dependencies fails, but that won't cause any of the residue files to be
>touched. The scheme right now is that those files are not considered
>targets in the build (they can't be, or they won't be updated when 
building
>their dependencies fails).

Removing the .exe would be very helpful. We wouldn't have to rerun all 
builds and successful tests the way we do now.

>> In the case where a is an .exe target, don't people complain when e 
fails
>> and bjam leaves a there as if building it had succeeded?
>
>Not so far, but that's easy to change.
>
>> Or are you saying that a failure of e in a->b->c->d->e deletes a, b, c,
>and
>> d targets in some cases but not others, depending upon the exact target
>> type?
>
>The normal behavior is to remove a target when any of its direct build
>actions fails. If I invoke the RMOLD rule on a target it will be removed
>when any of its direct or indirect build actions fails. It wouldn't be a
>good idea to do that for archives (static libs), for example because that
>means all the work of archiving files would be thrown out as soon as a
>compilation error occurred.

It is having to be thrown out now! All bin directories are being deleted 
before every regression test run, as the only apparent way to get correct 
test results:-( Thus it takes something like an hour on my machine to run 
the tests. Faster runs that report the wrong results aren't of interest.

--Beman