Tom Kent wrote:
On Sat, Dec 12, 2015 at 8:35 PM, Rene Rivera <grafikrobot@gmail.com> wrote:
Not going to comment on the aspect of purchasing a machine. But will point out that the real benefit to having dedicated machines is that of having non-traditional setups (OS+toolset). I.e. dedicated machines give you coverage.

On Sat, Dec 12, 2015 at 8:08 PM, 'Tom Kent' via Boost Steering Committee <boost-steering@googlegroups.com> wrote:

I also think that, like Niall said, we should move towards CI style testing where every commit is tested, but that is going to be a *huge* transition.

I wouldn't say huge.. Maybe "big". 
 
I would love to see direction on this in general from the steering committee, and am encouraged that almost all new libraries already have this. 

I can't speak for the committee. But as testing manager I can say moving Boost to CI is certainly something I work on a fair amount.


FYI, Boost.Geometry is setup to use CircleCI and Coveralls. See the readme:
https://github.com/boostorg/geometry

We're using CircleCI instead of TravisCI because the latter fails due to the lack of memory needed to run the tests. The integration of CircleCI with Coveralls is not as straightforward as it is for TravisCI esspecially in the case of parallel testing. I was forced to manually gather coverage info from parallel runs into one VM/container, merge chunks manually and send with curl into Coveralls. See the script if you're interested, it's based on the Antony Polukhin's TravisCI script:
https://github.com/boostorg/geometry/blob/develop/circle.yml

Currently I have to manually push the changes into my fork of Boost.Geometry in order to run the tests. Obviously the tests for pull requests for the main repository aren't run automatically either.
So if you plan to enable the support for online CI services I'd suggest to allow the maintainers to choose the services they prefer, somehow.

Btw, I'm also playing with the performance regression testing on CircleCI:
https://circleci.com/gh/awulkiew/benchmark-geometry-trigger/80#artifacts
https://circle-artifacts.com/gh/awulkiew/benchmark-geometry-trigger/80/artifacts/0/tmp/circle-artifacts.Nv98VEW/index.html
The above charts were generated by scripts, benchmarks and report generator tool. It's not that this is natively supported by CircleCI.

Regards,
Adam