Re: [jamboost] Regression test: link-fail problem?

From: David Abrahams (dave_at_[hidden])
Date: 2002-08-21 10:53:02

From: "Beman Dawes" <bdawes_at_[hidden]>

> At 09:28 PM 7/8/2002, David Abrahams wrote:
>
> >This is exactly the sort of thing I was afraid would arise. AFAICT
> there's
> >no simple way to put this in the build system without making core Jam
> >changes.
> >
> >The basic problem looks like this. Given a dependency chain:
> >
> >a->b->c->d->e
> >
> >when building "e" fails, you want something to happen with the
> >.test/.success/.failure files. These are associated with A. However,
the
> >build system just stops working on "a" the instant any of its
> dependencies
> >fails to build.
>
> Ah. That is a nice explanation, and gives me a better understanding of
what
> is happening.

This explains exactly the behavior you last reported in
http://groups.yahoo.com/group/jamboost/message/1661

> As long as a program scanning the output log and the residue left in the
> test directory can figure out where in the a->b->c->d->e chain a problem
> occurred, I may be able to deal with it.

Was your assumption that this limitation could be dealt with incorrect?

> I've got some ideas about how to make the output from Boost.Build v2
> regression tests easier to used. I don't know it is premature, but I'll
> mention a few now:
>
> * Having results spread over so many files (.run, .output, .success,
> .failure, .test, cout, and cerr ) is a major pain. If these could be
> consolidated into a single file, or at least a smaller number of files,
> that would make life much easier.

Please tell us exactly what you want to see in the file(s).

> * It would be easier if the residue file name (or names, if they can't
be
> combined) were always the same. For example, instead of it being
> abc.output for test abc, and then xyz.output for test xyz, call it
> test.output for all tests.

That's easy enough.

> * Even if the cout/cerr output for a given test can't be combined with
the
> other output and placed in the appropriate sub-directory, some format
> improvements would help a great deal. For example,
>
> * Any output captured from a compiler, linker, test run, or whatever
should
> be nested inside some easy to identify header and trailer. This is
> happening now for some C++ compile failures:
>
> metrowerks-C++-action blah
>
> compiler output here
>
> ...failed metrowerks-C++-action blah
>
> But in other cases, the ...failed line is missing. Makes parsing very
> difficult, sometimes impossible.

The "...failed" line is missing whenever the command succeeds. I suppose
you'd like it to print something in any case. I guess that's no problem for
v2.

> * The "blah" in messages like the above is actually something like:
>
..\libs\regex\build\bin\libboost_regex.lib\metrowerks\debug\runtime-link-dy
namic\instances.obj...

That's generated by the bjam core; it tells you which file is being built.

> While that may be needed for other purposes, for regression tests what is
> often needed is the name of the test which failed, and the name of the
> toolset involved. So a separate message may be needed.

What do you mean by "the name of the test which failed", specifically?
How would you like to see that?

> * Top level needs identification on lib build failure. In the above
> example, some test needed the regex library, and that kicked off the
regex
> build which then failed. What may be needed here is an explicit test
type
> "lib-build" to go with "compile", "run", etc. Then lib build failures
> would show up on the status table in a very understandable way.

Specifically, I presume you'd like to see lib and dll dependencies of test
targets show up as tests in their own right. However, I'm not sure what
causes your system to create a test entry in the table, so I'd need to know
what results you need from the build system in order to achieve that.

-----------------------------------------------------------
David Abrahams * Boost Consulting
dave_at_[hidden] * http://www.boost-consulting.com