Re: [boost] Status on Travis

Subject: Re: [boost] Status on Travis
From: Rene Rivera (grafikrobot_at_[hidden])
Date: 2016-08-18 12:22:02

On Thu, Aug 18, 2016 at 10:57 AM, Raffi Enficiaud <
raffi.enficiaud_at_[hidden]> wrote:

> Le 18/08/16 à 16:16, Rene Rivera a écrit :
>
>> On Thu, Aug 18, 2016 at 7:32 AM, Raffi Enficiaud <
>> raffi.enficiaud_at_[hidden]> wrote:
>>
>> Hi dear list,
>>> Hi Rene,
>>>
>>> I would like to know more about Travis and its status. From what I
>>> understand, a lot of efforts have been made recently for making things
>>> running per commit on Travis and have a quick feedback.
>>>
>>> * Naturally the first question is: is the status given by Travis the
>>> "official" one? so we consider the status given by Travis an official
>>> source of information, especially wrt. to its competing regression
>>> dashboard.
>>>
>>>
>> The integrated test results are still the "official" results.
>>
>> * What are the development plans for Travis? Right now I see 3 types of
>>
>>> builds, and I do not fully understand what they cover (platform,
>>> compiler)
>>> nor their differences.
>>>
>>>
>> My plans, which are official as it can get around here when it comes to
>> testing, are to move as much of the "generic" testing and automation as
>> possible to cloud CI (would include Travis, Circle, and Appveyor). I don't
>> know what you mean by 3 types of builds though. Can you point out an
>> example?
>>
>
> Sorry I was not clear. Here for instance:
> https://travis-ci.org/boostorg/boost
>
> I can see 4 job builds, 3 green and one red, the 3 green saying:
> - SCRIPT=ci_boost_status (~4min)
> - SCRIPT=ci_boost_status EOL=LF (~32min)
> - SCRIPT=ci_boost_status EOL=CRLF (~32min)
>
> I do not know what those jobs are doing and in what type of environment.

That should be (and what they do):

- SCRIPT=ci_boost_status

Runs a "quick" status check on the full integrated Boost repo. The tests
are dry runs of what the regular building and testing does. I.e. it runs b2
for the general building at the root (what an end user would do), and by in
the status subdir (hence the name, and this is what ultimately the full
testing does). And by dry run I mean (b2 -n). Hence it catches errors like
missing tests, missing sources, bad jamfiles, etc. And generally provides a
quick pass/fail status for the prospective release.

- SCRIPT=ci_boost_release EOL=LF
- SCRIPT=ci_boost_release EOL=CRLF

Those generate the release archives in the two EOL flavors we support.
Building one of those releases essentially corresponds to: build support
tools, build all the docs, copy & filter the final release tree, generate
compressed archive files, upload the archives to Bintray and SF. Any
problems here are in the stop-the-presses category. As these are what we
give users.

- SCRIPT=ci_boost_library_check

That's the newest build.. And it runs the library checks you now see in the
general result matrix as "__boost_check_library__". Except it runs them for
all libraries at once. It helps me to asses (and harass authors) the basic
library requirements.

Would it also be possible to have like a template .travis.yml coming from
>>> the boost.superproject which can partly be overiden by a library? (if we
>>> want to build things slightly differently)
>>>
>>>
>> Kinda :-) My plan is to have a generic .travis.yml that authors can use
>> for
>> their libraries. It would manage doing all the testing setups possible
>> (and
>> wanted). It would look roughly like the one I currently use for Boost
>> Predef <https://github.com/boostorg/predef/blob/develop/.travis.yml>. And
>> the same would apply for Circle and Appveyor. Note that the current
>> implementation of those Travis scripts doesn't do the integrated testing
>> yet. That's coming soonish.
>>
>
> Cool, many thanks for those efforts. Can we help?

Welcome.. And maybe you can help. I'll think about how :-)

* The CI I am using is able to parse the junit files generated by the test
>>
>>> framework, and summarizes the information of each build nicely (also
>>> tracks
>>> the tests history). Is there anything similar in Travis?
>>>
>>
>> I don't think there's anything like that for Travis.
>>
>
> True, apparently they are not doing. That means that uploading the tests
> results to a dashboard will also be on boost superproject hands, is that
> right?

Yes, indeed.

Right now, apart from looking at the full logs (which is hard), is there
>>> any intermediate information than red/green overall status?
>>>
>>>
>> Not on Travis.. But eventually the goal is to have the above common
>> scripts
>> upload the CI results to our common test results. That's in the not so
>> near
>> future though as it's not a trivial task.
>>
>
> I am not even sure this is technically allowed by Travis (
> https://blog.travis-ci.com/2012-12-18-travis-artifacts/).

This would happen as an entirely outside of their regular channels of
uploading. It would either be something to process and upload to the test
ftp location, or some form of custom REST upload.

-- 
-- Rene Rivera
-- Grafik - Don't Assume Anything
-- Robot Dreams - http://robot-dreams.net
-- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail