Boost logo

Boost :

From: troy d. straszheim (troy_at_[hidden])
Date: 2005-09-18 09:50:05

Robert Ramey wrote:
> This is something that my test suite should test but doesn't. Regardless of
> what happens with your portable_binary archive, It would be very helpful to
> have the test suite enhanced to explicitly test archive portability.
> Since we're just talking, So I would like to see the following changes in
> the test suite:
> a) That output of the tests go to a particular directory. There would be a
> directory for each column in the test matrix.
> b) There would be a set of directories for each boost release 1.32, 1.33,
> ...
> c) Tests would be run on the four combinations which result from:
> i) set of archives created by the previous version
> ii) set of archives created by other compilers (portable archives only)
> This would result in a HUGE number of tests so it couldn't be run any
> more than on an occasional basis. But it would be helpful to run very
> occasionally.
> This would require more investigation into boost test in order to find
> out what facilities are already there for helping out with ths.

Yeah, hmm. It's a lot of work, more tedious than difficult. Since
we're just talking, here's some thinking out loud on the subject, for
sanity-check purposes:

There are a few different classes of tests. The compile-fail ones don't
need modification of course. A great many contain the following or
similar, one more more times:

int test_main( int /* argc */, char* /* argv */[] )
     const char * testfile = boost::archive::tmpnam(NULL);
     BOOST_REQUIRE(NULL != testfile);


     S s_out = create_some_serializable_data_structure();
         test_ostream os(testfile, TEST_STREAM_FLAGS);
         test_oarchive oa(os);
         oa << boost::serialization::make_nvp("S", s_out);
     S s_in;
         test_istream is(testfile, TEST_STREAM_FLAGS);
         test_iarchive ia(is);
         ia >> boost::serialization::make_nvp("S", s_in);
     BOOST_CHECK(s_out == s_in);

and one would concievably want to factor these out so that the
read-in-and-compare section would happen multiple times, once per boost
version and per compiler. For portable archive types, it would happen
for all boost versions, compilers *and* platforms. An additional check
is that at the end, all portable archive types have the same checksum.
tmpnam() would change to, say, persistent_testfile(). Many of the test
modules create multiple archives. You could deal with this by keeping
track of some index and appending this to the file name.

So you would end up with files like


for instance


which would be the third file written by tests inside test_something.cpp.

> d) An alternative to the above might be just to make some specific tests
> for backward compatibility and for cross compiler compatibility. This is
> probably a better idea as it would result in tests that could be run more
> frequently.

I would think that you would in fact want to run the full battery of
tests periodically. Or maybe constantly, on a couple machines dedicated
to the purpose, but I agree that the average Joe who checks something
out of CVS and runs the tests shouldn't be expected to sit through the
whole lot.

All that looks doable, to me. I wonder about storage. If you're really
going to keep all that data around, the question is where, and how are
you going to get at it. One could easily put together a little web
interface, where the test suites use a little python script to post and
retrieve test data.

Let me see what I can come up with.


Boost list run by bdawes at, gregod at, cpdaniel at, john at