Re: [jamboost] Test system

From: David Abrahams (david.abrahams_at_[hidden])
Date: 2002-04-04 09:13:48

Here's the missing feedback.

----- Original Message -----
From: "Vladimir Prus" <ghost_at_[hidden]>

> > > t.copy_tree('test1')
> >
> > What does the above line do?

I think we still don't have clarity about what this means... the
comments in the code seem fine, if it does what I think it does...

> > > t.run_build_system("-sTOOLS=borland")
> >
> > Whaat does the above line do? (I know what it does, but a short
> > comment would help)

You wrote "invokes the build system...", but something like, "invokes
'bjam -sTOOLS=borland'" would be superior.

> # First, create a list of three filenames with t.mul

s/filenames/pathnames/

> # Second, check if those files were added as result of the last build
system invocation.

s/check if/assert that/
or
s/check if/test fails unless/

> The new wording and method name is:
> set_tree -- sets the content of the working directory to be equal to
the
> content of the specified directory.

The example still uses copy_tree (however, I like that name better -- I
think).

>
> > > In addition, difference between the two states -- i.e. lists of
> > > files that were added, removed, modified or touched -- is stored
in
> > > two member variables, tree_difference and unexpected_difference.
The
> > > meaning of latter is to contain differences that are not yet
> > > accounted for.
> >
> > What does "accounted for" mean?
>
> Hopefully clarified.
>
> > > Writer of tests then removes from
> > > unexpected_difference elements which are OK, for example, if file
> > > foo was added, a call to expect_addition("foo") will remove that
> > > file from the list of unexpected additions.
> >
> > Oh, I think I see, after many re-readings. I think you described
this
> > in the wrong order. I think you should say,
> >
> > "...and after running is recorded.
> >
> > The test writer then calls methods such as
expect_addition("foo.o")
> > to denote expected changes in the tree. When all expected changes
> > have been registered, the "unexpected_difference" attribute will
> > contain a list of all changes not yet accounted for."
>
> Does the new version make it more clear?

Much more, thank you!

> > BTW, how is a difference recorded? It can't just be a list of names?
> > Is it three lists of names?

This is not answered anywhere AFAICT. Either you should describe the
data structure used to represent the diffs, or you should remove mention
of the attribute names (e.g. unexpected_difference) from the doc and
just mention that the class stores these sets of diffs internally
somehow.

> > Considering that there is no TestCmd documentation describing what
> > these parameters do, I don't think this description is adequate.
>
> Neither do I. I have removed any mentioning about various values,
becase
> using them is hardly ever needed.

There's still way too much reliance on the idea that the user knows how
TestCmd works.

> > > Method run_build_system(self, extra_args='', stdout=None,
stderr='',
> >
> > status=0, **kw)
> >
> > > Effects:
> > >
> > > Stores the state of the working directory in self.previous_tree.
> > >
> > > Calls TestCmd.run, passing it the right program name, and the
argument
> >
> > What is the "right" program name? Again, TestCmd docs are missing.
>
> > > Checks the stdout, stderr and exit status, if appropriate
arguments
> > > are not None
> >
> > How does it respond to these (IOW, what does it mean to "check"?)
>
> I've clarified this.

OK, better. It would be nice if we could pass a callable object for any
of these parameters which does some checking. This would allow us to do
regexp matching, for example. However, I think I'm generalizing
prematurely, here, so just put this idea on the shelf OK?

> > > Stores the new state of the working directory in self.tree.
Computes
> > > the difference between previous and current trees and store them
in
> > > variables self.tree_difference and self.unexpected_difference.
> >
> > Details, please? How are they stored?
>
> Wrote.

Better. You should describe the interface to tree.Trees_different
somewheree earlier in the doc.

> > > Methods for declaring expectations
> > >
> > > Accordingly to the number of changes kinds that are detected,
there
> > > are four methods that specify that test author expects a specific
> > > change to occur. They check self.unexpected_difference, and if the
> > > change is present there, it is removed. Otherwise, test fails.
> > >
> > > Each method accepts a list of names. Those names are always
> >
> > unix-style, even on other systems.

what is "unix-style"? If I'm a windows programmer, how do I know what
that means?

-Dave