RE: [boost] First try at regression test DTD

From: Boris Skidan (bskidan_at_[hidden])
Date: 2002-07-12 17:53:24

>From my experience it is often required to support multiple versions/branches of the same project simultaneously. For example the may be a need to release 1.28.1 while 1.29.0 being developed and both versions should be tested. Would it make sense to introduce something like version/branch/build number in this DTD?

Boris Skidan.

-----Original Message-----
From: William E. Kempf [mailto:williamkempf_at_[hidden]]
Sent: Friday, July 12, 2002 4:36 PM
To: boost_at_[hidden]
Subject: Re: [boost] First try at regression test DTD

----- Original Message -----
From: "Jeff Garland" <jeff_at_[hidden]>
To: <boost_at_[hidden]>
Sent: Friday, July 12, 2002 2:03 PM
Subject: RE: [boost] First try at regression test DTD

>
> I recommend using ISO 8601 formats for timestamps instead of an adhoc
format.  So instead of:
>
> timestamp="7 June 2002 14:52:03 UTM"
>
> Try:
>
> timestamp="2002-06-07 14:52:03 UTM"
>
> Besides being a standard and 'internationalized' it is much easier to
parse, compare etc.  Also, I assume by UTM you mean UTC
> (Universal Time Coordinated)?

I like this format as well.

> Also don't you want the results as sub-elements of the toolset?  Like
this:
>
> <toolset>Metrowerks
>   <test-directory-path>...</test-directory-path>
> - <compile-action source-path="../libs/config/config_info.cpp"
result="success" timestamp="7 June 2002 14:52:03 UTM">
>   <command>mwcc blah blah blah ../libs/config/config_info.cpp</command>
> - <output>
> - <!--   present even if no output text
>   -->
> - <!--  including warning messages, even if compile succeeds
>   -->
>   </output>
>   </compile-action>
> </toolset>

I think the results should be sub-elements as well.  Forgetting a DTD
specification (not really required for XML any way, so let's not complicate
the discussion by including formal DTD specifications), here's how I'd
rework the XML:

<regression-tests>
    <test type="run" name="config_info">
        <toolset name="metrowerks">

<test-directory-path>status/bin/config_info.test/metrowerks/debug/runtime-li
nk-dynamic</test-directory-path>
            <action type="compile" result="success" timestamp="2002-06-07
14:52:03 UTC">
                <command>blah blah blah</command>
                <output><!-- present even if no output text --></output>
            </action>
            <action type="link" result="success" timestamp="2002-06-07
14:52:03 UTC">
                <command>blah blah blah</command>
                <output><!-- present even if no output text --></output>
            </action>
            <action type="run" result="success" timestamp="2002-06-07
14:52:03 UTC">
                <command>blah blah blah</command>
                <output><!-- present even if no output text --></output>
            </action>
        </toolset>
    </test>
</regression-tests>

Rationale:

*  The root element should be named something more meaningful than
"example".  "regression-tests" seems logical.
*  The name "test" is more in line with the name "regression-tests" than is
"test-log", at least IMO.
*  The "test-info" element just specifies information about the test, so
it's a better fit as attributes then a full element.
*  "test-directory-path" might fit better as an attribute of "toolset", but
since this actually describes something about the test instead of the
toolset I like it better as an element on its own.
*  For the most part, the various "*-action" elements are identical, so I
think it might work out nicer to just make them a single element type, aptly
named "action".  (Note that this is one of the more controversial
suggestions for various reasons.)
*  The one exception to the last comment is the "source-path" attribute of
the "compile-action".  I'm not sure what this is.  Some tests will rely on
more then a single source file, so the example doesn't make much sense to
me.  In any event, this appears to be more an attribute of the test then of
the toolset (or of the compile action for the same reasoning) and thus
should be moved up in the hierarchy somewhere to avoid redundant duplication
of data.
*  I'm not sure why the "output" elements must be present even if there was
no output.  This too can increase the file size with no real need.  Along
these same lines, if a compile fails I see no reason the link or run actions
must be present since they won't even be executed.  (What does success or
failure mean for an action that's not performed?)

Bill Kempf
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost