From: Beman Dawes (bdawes_at_[hidden])
Date: 2002-07-11 21:40:37
At 03:51 PM 7/11/2002, William E. Kempf wrote:
>> But step back a bit. I quickly scanned back over this thread, and
>> see anyone explaining the benefits from generating XML as they would
>> to regression. In other words, if the sole use is to look at the
>> with a web browser, why bother?
>I gave one example, to view the data in a web browser in a different
>To make this a little more concrete, let's think about a compiler vendor,
>say Comeau C++. For marketing purposes it would be great if they could
>their own regression tests (possibly using more platforms and backends
>what we are using) and publish reports showing how well they can handle
>Boost code (or even some other library). With the hard coded HTML output
>they'll have to step through several hoops to get an accurate report that
>shows only the data they find pertinent and in a format appropriate for
>Or, they could use the XML generated by Boost, but combine the output
>the various platforms into one report about their compiler, something our
>reports don't do for compilers that work on multiple platforms. (That's
>format that might be useful even for us.)
>Or, imagine a compiler vendor that's working on their C++ conformance and
>using Boost as a test bed. Instead of an HTML report they could use the
>to generate bug reports with what ever bug tracking software they happen
>use in an automated build process.
>Theoretically (not really suggesting this, just brain storming) the Boost
>release process could even make use of this by e-mailing the owners of
>specific libraries about which tests are failing on what
>compilers/platforms. An automated script could even compare the current
>results against previous results (pulled from CVS by the tag used for the
>previous release) to pinpoint things that are breaking that never used
>Once the work of creating the scripts was completed this automation could
>simplify your job as the administrator of Boost releases.
>I'm sure there are other things that people could think of. I don't
>that Boost themselves will have a need beyond generating an HTML page for
>the web (well, again, maybe some automation in releases), but Boost users
>would benefit in many ways from a regression test system that created XML
>files instead... especially if that system can be reused in their own
Those sound like reasonable possibilities, as do the suggestions from Rob
Stewart and Peter Dimov.
We've been experimenting with generated diffs, and with automatic emails to
developers when some test event happens, but have just been whacking
together little scripts. Not very portable or maintainable.
Meanwhile, for the regression test, I've decided to pull together
information now in several different files and formats into a single new
file, and need a format for that. Since I'm starting from scratch as far
as the format goes, it shouldn't be much more difficult to use an XML
format. Usually I would describe such a format in EBNF, so I'll have to
figure out how to describe it in the XML equivalent.
Also, Dave Abrahams has been pushing me to generate a specific set of
requirements for regression testing under Boost.Build V2, and the XML specs
would serve to communicate what is desired in an unambiguous fashion.
Thanks for comments on this; it really helps to hear others perspectives
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk