Re: [jamboost] On testing

From: Steven Knight (knight_at_[hidden])
Date: 2002-03-26 14:51:18

> > But you *can* process the content and look for a specific string inside
> > that suggests it was built correctly. Or, if it's a program you built,
> > you can execute it to see that it worked correctly.
> 
> In general I hope that our build system /framework/ tests will not be
> dependent on doing much of anything with binary files. I would like to
> localize all of that in tests for individual toolsets and target
> platforms. Any individual tester probably doesn't have all the toolsets
> and platforms at his disposal; especially not within a single test run.
> It should be possible to pass "all" tests anyway. That's why I favor
> doing most of the testing with dummy toolset/platform specifications
> that operate portably in terms of simple text files.

We've been drifting towards this model. For example, our LEX and YACC
tests use dummy Python scripts (again, created from in-line strings) as
place-holders for the real lex and yacc utilities. The dummy scripts
perform some simple processing on text-file input to mark that they've
been executed.

However, we currently also look for a 'lex' or 'yacc' executable and
run a test with "real" input if one exists. But we've already run into
tool incompatibilities there, when we used the GNU-specific -o flag for
lex and had to back off to the POSIX-standard -t option...

Your architectural separation of toolset / platform tests from build
tool tests makes a lot of sense here. We should take a look at
reorganizing the SCons tests along similar lines.

--SK