Re: [jamboost] On testing

From: David Abrahams (david.abrahams_at_[hidden])
Date: 2002-03-26 08:23:55

----- Original Message -----
From: "Vladimir Prus" <ghost_at_[hidden]>

> Since I feel a bit unsure about committing untested code, I'd like to
discuss
> how the testing will be performed.
>
> First there's code that is capable of running build system and finding
which
> files are added/removed &c. It also allows to change file content and
> rebuild, and similar operations. It's written in Python and it was
agreed
> that translating it in Jam is hard and unneeded. Does everybody agree
with it.

I agree in principle, but at the same time I'm unfamiliar with the test
code and can't find any documentation... I know you wrote something once
about how to use it, but I can't find it. A quick test for the test
system wouldn't hurt ;-)

> Second, the code to run individual tests. Now I have a naive code
based on
> Python unit tests and it even does not allow to run any individual
test,
> IIRC. I really don't want to do anything about it -- it was a
temporary
> solution.
>
> The questions is:
>
> What to do with the second component? We most likely will need
something more
> usable. If I was to choose, I'll simple use QMTest
(http://www.qmtest.com)
> for this. It is quite functional, quite easy to use, and is written in
Python
> -- i.e. test code for the build system will be easy to integrate with
it. On
> the other hand, it's more than a couple of modules -- i.e. if somebody
wants
> to test build system, he'll need to get QMTest. I actually don't think
this
> is big problem. So, what other's opinions on this?

The Scons project has a mature python-based system for testing that has
worked well for them. Should we reinvent the wheel, or would it be
better to just hijack the work they've done?

> For those who never seen the my test code I'd outline my vision of how
the
> test suite might look. It will be a directory hierarchy (corresponding
to
> tests hierarchy) and at some level we might have this:
>
> <some directory>
> test1.qmt
> test2.qmt
> test1-tree
> test2-tree
>
> *.qmt are files that descibe tests and test{1,2}-tree are just
directories.
> They will contain initial state of source tree for tests. Inside *.qmt
files,
> python code will be used to drive tests, something like
> touch_file("src/source.cpp")
> run_build_system()
> expect_touch("build/main/release/runtime-link-dynamic/source.o")

I'm inclined to go with whatever seems most natural to you, *as long as*
there's an easy way for the rest of us to learn how to use it. It would
be worth hearing what Steven Knight has to say about the Scons stuff,
though.

-Dave