Re: [boost] Status on Travis

Subject: Re: [boost] Status on Travis
From: Raffi Enficiaud (raffi.enficiaud_at_[hidden])
Date: 2016-08-18 11:57:59

Le 18/08/16 à 16:16, Rene Rivera a écrit :
> On Thu, Aug 18, 2016 at 7:32 AM, Raffi Enficiaud <
> raffi.enficiaud_at_[hidden]> wrote:
>
>> Hi dear list,
>> Hi Rene,
>>
>> I would like to know more about Travis and its status. From what I
>> understand, a lot of efforts have been made recently for making things
>> running per commit on Travis and have a quick feedback.
>>
>> * Naturally the first question is: is the status given by Travis the
>> "official" one? so we consider the status given by Travis an official
>> source of information, especially wrt. to its competing regression
>> dashboard.
>>
>
> The integrated test results are still the "official" results.
>
> * What are the development plans for Travis? Right now I see 3 types of
>> builds, and I do not fully understand what they cover (platform, compiler)
>> nor their differences.
>>
>
> My plans, which are official as it can get around here when it comes to
> testing, are to move as much of the "generic" testing and automation as
> possible to cloud CI (would include Travis, Circle, and Appveyor). I don't
> know what you mean by 3 types of builds though. Can you point out an
> example?

Sorry I was not clear. Here for instance:
https://travis-ci.org/boostorg/boost

I can see 4 job builds, 3 green and one red, the 3 green saying:
- SCRIPT=ci_boost_status (~4min)
- SCRIPT=ci_boost_status EOL=LF (~32min)
- SCRIPT=ci_boost_status EOL=CRLF (~32min)

I do not know what those jobs are doing and in what type of environment.

>> * For boost.test, I am testing each feature branch before merging to
>> develop, by
>>   1. cloning boost superproject on develop
>>   2. switching only boost.test to the feature branch
>>
>>   This is for preventing the breakages on develop that we observed in the
>> past since several libraries are depending on boost.test.
>>   Would it be possible to have that as well on Travis or any continuous
>> CI? That would be per library then.
>>
>
> Yes. That is part of my plan.

Wonderful!

>> Would it also be possible to have like a template .travis.yml coming from
>> the boost.superproject which can partly be overiden by a library? (if we
>> want to build things slightly differently)
>>
>
> Kinda :-) My plan is to have a generic .travis.yml that authors can use for
> their libraries. It would manage doing all the testing setups possible (and
> wanted). It would look roughly like the one I currently use for Boost
> Predef <https://github.com/boostorg/predef/blob/develop/.travis.yml>. And
> the same would apply for Circle and Appveyor. Note that the current
> implementation of those Travis scripts doesn't do the integrated testing
> yet. That's coming soonish.

Cool, many thanks for those efforts. Can we help?

>
> * The CI I am using is able to parse the junit files generated by the test
>> framework, and summarizes the information of each build nicely (also tracks
>> the tests history). Is there anything similar in Travis?
>
> I don't think there's anything like that for Travis.

True, apparently they are not doing. That means that uploading the tests 
results to a dashboard will also be on boost superproject hands, is that 
right?

>
>
>> Right now, apart from looking at the full logs (which is hard), is there
>> any intermediate information than red/green overall status?
>>
>
> Not on Travis.. But eventually the goal is to have the above common scripts
> upload the CI results to our common test results. That's in the not so near
> future though as it's not a trivial task.

I am not even sure this is technically allowed by Travis 
(https://blog.travis-ci.com/2012-12-18-travis-artifacts/).

>
> * I can see many project under the boostorg of Travis
>> https://travis-ci.org/boostorg/ . How do we do that?
>>
>
> Just ask me and I can turn on any particular library to be included in
> that. Note that the Travis testing resources, with the free OO license,
> limit the number of concurrent jobs per organization. So all libraries in
> that list share a smallish quota. But this is one area where we will
> consider funding to increase that quota when we have better scope
> information.
>

Cool. Right now I have my own, but maybe in the near future (especially 
mine is in a private net).

Thanks for all those information,
Raffi