Re: [boost] ATTN: Boost GSoC mentors and students: Continuous Integration for GSoC thoughts

Subject: Re: [boost] ATTN: Boost GSoC mentors and students: Continuous Integration for GSoC thoughts
From: Daniel Pfeifer (daniel_at_[hidden])
Date: 2013-06-04 04:37:28

2013/6/3 Niall Douglas <ndouglas_at_[hidden]>

> Dear Boost,
>
> I spent the weekend experimenting with getting CI set up for the async file
> i/o library which will hopefully become Boost.AFIO through Paul's GSoC. I
> tried a number of free-for-open-source CI providers which integrate tightly
> with GitHub, and eventually settled with Travis CI. You can see the last
> commit CI results at https://travis-ci.org/ned14/TripleGit.
>
> As much as Travis CI has a primitive and clunky UI, unlike the other CIs
> it's based on snapshotting VMs with root privs which gives you unparalleled
> flexibility for testing config, so I was able to configure this to happen
> with each and every git push to github:
>
> 1. Test that code builds with GCC 4.6 and clang 3.2.
>
> 2. Test that unit and functional tests execute without error when built
> with
> GCC 4.6 and clang 3.2.
>
> 3. Run unit and functional tests with valgrind --tool=memcheck and test
> that
> no memory errors nor memory leaks occur with binaries built with GCC 4.6
> and
> clang 3.2.
>
> 4. Install GCC 4.8 and compile everything using -fsanitize=thread (i.e.
> instrument using the port of clang's ThreadSanitizer v2 to GCC), then
> execute a full set of unit and functional tests looking for race conditions
> and if any, fail. You'll note I've marked that test as allowed to fail,
> because Boost v1.53 has a number of known race conditions when used as
> C++11
> which are hopefully fixed in v1.54 :)
>
>
> Personally speaking, I think this depth and degree of CI is *invaluable* to
> modern software development practice.
>

I could not agree more.

One thing you may want to add is coverage analysis with
https://coveralls.io/:

If you build with gcc, pass "-fprofile-arcs -ftest-coverage" both to the
compiler and the linker. After running the tests, you can analyze the
coverage with gcov and send it to coveralls.
To collect and upload, I use the following script:
https://github.com/purpleKarrot/Karrot/blob/develop/test/coveralls.in
(Note that gcov 4.6 does not support all flags that I use in the script, I
use it wiht 4.8 only).

cheers, Daniel