Re: [boost] [EXTERNAL] [testing] Proposal - regression tests results display upgrade

Subject: Re: [boost] [EXTERNAL] [testing] Proposal - regression tests results display upgrade
From: Rene Rivera (grafikrobot_at_[hidden])
Date: 2014-08-14 16:00:17

OK, I only now have time to read email.. Is there something I can help
answer? (Not sure what the question refers to).

Rene.

On Wed, Aug 13, 2014 at 5:53 PM, Belcourt, Kenneth <kbelco_at_[hidden]>
wrote:

>
> On Aug 13, 2014, at 1:51 PM, Beman Dawes <bdawes_at_[hidden]> wrote:
>
> > On Wed, Aug 13, 2014 at 10:42 AM, Adam Wulkiewicz <
> adam.wulkiewicz_at_[hidden]
> >> wrote:
> >
> >> Paul A. Bristow wrote:
> >>
> >>> -----Original Message-----
> >>>> From: Boost [mailto:boost-bounces_at_[hidden]] On Behalf Of Adam
> >>>>
> >>> Wulkiewicz
> >>>
> >>>> Would it make sense to use some more descriptive color/naming scheme?
> >>>>
> >>>> In particular it would be nice to distinguish between the actual
> >>>> failures and
> >>>>
> >>> the
> >>>
> >>>> situations when the compilation of a test took too much time or an
> >>>> output file
> >>>>
> >>> was
> >>>
> >>>> too big.
> >>>> Would it make sense to also distinguish between compilation, linking
> and
> >>>> run
> >>>>
> >>> failure?
> >>>
> >>> +1  definitely.
> >>>
> >>> This is particularly a problem for Boost.Math - the largest library in
> >>> Boost, by
> >>> far, in both code and tests, with several tests that often time out.
> >>>
> >>> Various summary counts of passes and fails would be good as well.
> >>>
> >>> It takes a while to scan the current (nice) display, eg
> >>>
> >>> http://www.boost.org/development/tests/develop/developer/math.html
> >>>
> >>
> >> Ok, I managed to find some time to play with it.
> >> AFAIU the reports are generated using the code from:
> >>
> https://github.com/boostorg/boost/tree/develop/tools/regression/src/report
> ,
> >> is that correct?
> >>
> >
> > Yes, I believe so.
> >
> >>
> >> The first change is simple since I'd like to see if I'm doing everything
> >> right. I changed the way how those specific fails are displayed. If the
> >> compilation fails and at the end of the compiler output (25 last
> >> characters) one of the following strings can be found:
> >> - "File too big"
> >> - "time limit exceeded"
> >> the test is considered as "unfinished compilation" and displayed on the
> >> library results page as a yellow cell with a link named "fail?". So it's
> >> distinguishable from the normal "fail".
> >>
> >> Here is the PR: https://github.com/boostorg/boost/pull/25
> >>
> >
> > I skimmed it and didn't see any red flags.
> >
> >
> >>
> >> I only tested it locally on a test done by 1 runner for Geometry and
> Math
> >> libraries.
> >> Is there a way I could test it on results sent by all of the runners?
> >> How is this program executed?
> >
> > IIRC, Noel Belcourt at sandia runs the tests. Thus he would be a good
> > person to merge your pull request since he will likely be the first to
> > notice any problems.
> >
> > Noel, are you reading this:-?
>
> A little delayed but yes.
>
> > Is there some script which e.g. checks some directory and passes all of
> >> the tests as command arguments?
> >>
> > He would be the best person to answer that.
>
> I’ll have to investigate, don’t know off the top of my head.
>
> — Noel
>
>
> _______________________________________________
> Unsubscribe & other changes:
> http://lists.boost.org/mailman/listinfo.cgi/boost
>

-- 
-- Rene Rivera
-- Grafik - Don't Assume Anything
-- Robot Dreams - http://robot-dreams.net
-- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail