Subject: [boost] [all][testing] Regression summary upgrades
From: Adam Wulkiewicz (adam.wulkiewicz_at_[hidden])
Date: 2015-05-11 11:46:46
Recently the regression summary was upgraded. The cells corresponding to
the failing tests can have names comp, link, run or fail.
This way we can see what's the nature of the failure without going into
See, e.g. the tests for Math:
The last category of failures (generic "fail") is for the others or unknown.
Currently the "Lib" errors are falling into this category. There are
some in Math, e.g.:
So we could mark them explicitly as "lib" failures and then the
other/unknown failures could be marked as "unkn" or left as it is now -
This has sense if there can be other "types" of failures, e.g. caused by
some error in the testing scripts, Build or Regression, for which the
reason is somehow unknown. Though I haven't seen them.
I propose to go further with this and to mark the compilation failures
for which we know the reason of the failure in a special way:
- "file" - file too big (reported on Windows, e.g.
- "time" - time limit exceeded (e.g.
I saw them in many libraries and I think it would be very useful for
libraries having many tests like Math or Geometry.
The other types of failures I could think of aren't that important for
me personally but maybe someone could appreciate them, e.g. the
- "ierr" or "cerr" - internal compiler error or segmentation fault
during compilation (e.g.
- "segf" - segmentation fault during run
Furthermore I propose to use various colors for different failure reasons.
You can see the examples here:
Of course the colors could be changed. We should be able to modify them
in the master.css file. In the examples above they are varying much to
make a clear distinction between them. But if you like the yelow collor
then all of the new failures could be yelowish with a slight accent of
another color, e.g. compilation errors could be yellow with orange
accent and "time"/"file" errors with green accent.
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk