Re: [boost] Cleanup of test results.

Subject: Re: [boost] Cleanup of test results.
From: Dmitry Moskalchuk (dm_at_[hidden])
Date: 2015-02-11 11:33:35

On 11/02/15 19:23, Rene Rivera wrote:
> Roughly it goes like this:
>
> 1. Run full and complete results on the develop branch for some time.
> 2. Prove that you can run tests on a consistent schedule.
> 3. Prove that you are responsive to platform questions and requests for the
> tests you run.
> 4. For new platforms (OS and or toolset) prove that you can support that
> new platform with patches and collaboration with library authors.
> 5. Ask to be added to the master results list.
> 6. Ask to be considered a release supported platform.
>
> Note..
>
> * As far as I can tell you are still working on step #1.
Yes

> * For #2 consistent means a predictable schedule, or continuous. The
> minimum frequency I look for a at least weekly results on develop. For
> master results I prefer daily but can live with 3 times a week.
We have dedicated servers for Boost testing so we definitely will run it
continuously; however, full run of all Boost tests on one API level/ABI
could take several days - this goes from the cross-build nature of
Android testing. We build them on host OS (Linux), upload to target
machine (Android) as well as all required libraries and then run tests
there. It obviously take more time than just build&run on the same
machine. I already optimized many things and probably could optimize
more but my rough estimate is still ~3 days for one run.

> * #3 means responding to emails on both the testing and developer lists
> reasonably promptly. And addressing any problems.. Like looking into
> stalled tests, providing full logs for specific tests, running specific
> tests for authors, etc.
No problem with this. I already subscribed to boost-testing list and
we'll monitor it too.

> * You are actively pursuing #4.. Which is awesome :-)
> * #5 just gets the results into the web site.
> * And #6 gets your platform, Android, listed as supported in the release
> notes.
>
OK, got it. Thank you for explanation. We'll definitely go further to
apply to these rules.

-- 
Dmitry Moskalchuk