Re: [boost] Enhancing regression tests summary

Subject: Re: [boost] Enhancing regression tests summary
From: Beman Dawes (bdawes_at_[hidden])
Date: 2014-10-31 10:58:48

On Thu, Oct 30, 2014 at 10:23 AM, Adam Wulkiewicz <adam.wulkiewicz_at_[hidden]
> wrote:

> Paul A. Bristow wrote:
>
>> So I developed a simple program downloading summaries, test logs and
>> saving
>> the
>>
>>> enhanced result.
>>> You may check it out here: https://github.com/awulkiew/summary-enhancer
>>>
>> Neat :-)
>>
>
+1

>
>> I'd certainly find this very helpful when checking the Boost.Math test
>> monster
>> matrix.
>>
>
> Yes, it's big and various kind of errors can be found there.
>
> FYI, it has some new features.
> It can keep a log about the tests previously detected as failing and
> generate a report about new failures.
> It's also possible to send an email containing this report (to many
> recipients). It uses SMTP without authentication and e.g. Google throws
> such messages into SPAM, but a filter can be set for them. Alternatively
> SPF record could be configured in the machine's domain but I haven't played
> with it. Some local SMTP server could also probably work. Currently the
> mailing configuration must be placed in the file mail.cfg containing
> per-line settings (see the example in the repo).
> So if run periodically the program should notify about new regressions.
>

That's also very helpful!

The big question in my mind is whether we should enhance the current
regression testing system or start fresh using one of the continuous
integration frameworks?

--Beman