Re: [Boost-testing] BenPope x86_64 runners + runner requirements

Subject: Re: [Boost-testing] BenPope x86_64 runners + runner requirements
From: Rene Rivera (grafikrobot_at_[hidden])
Date: 2015-03-20 15:55:34

On Fri, Mar 20, 2015 at 2:44 PM, Raffi Enficiaud <
raffi.enficiaud_at_[hidden]> wrote:

> Le 20/03/15 18:20, Rene Rivera a écrit :
>
>> On Fri, Mar 20, 2015 at 11:58 AM, Beman Dawes <bdawes_at_[hidden]
>> <mailto:bdawes_at_[hidden]>> wrote:
>>
>>     On Fri, Mar 20, 2015 at 12:00 PM, Raffi Enficiaud
>>     >
>>     > Would it make sense to let the run.py submit something to the
>> dashboard if
>>     > and only if the runner is able to compile/build a very simple C++
>> program
>>     > (kind of hello world)?
>>
>>     +1
>>
>>
>> A nice though.. But useless in this case. The tester can obviously
>> compile something. Since it *has* to compile the test tools themselves
>> to even start testing.
>>
>>
> Thanks. The test tools is the first requirement, but it is in pure C as
> far as I know. In this case the installation does not have some required
> header for C++.

The test tools are a combination of Bash, Batch, C, C++, Jam, and Python.
And it builds the C++ tools before starting. So at least the *default*
system C++ compiler works just fine. And that's all we can test at this
level. It's the tested toolsets that are not OK in this case. Which we
can't determine easily without running the regression tests themselves. We
could possibly add a global test to the Boost tests and use that to check
for upload or not. But at that point you've already run all the tests. So
there's no savings for the tester as 95% of the work has been done. The
other 5% is processing the results and uploading. Arguably you could save
that 5% of work. Which leaves possibly only avoiding the upload so that
user don't see results from broken testers. But if we avoid users the
slight displeasure of seeing those broken results we would likely never
notice that the tester is broken. And hence never complain to the tester to
fix it. And hence end up wasting 100% of the testing resource.

So.. I'd rather see those results. For the basic reason that having some
information is better than having none and wasting work.

-- 
-- Rene Rivera
-- Grafik - Don't Assume Anything
-- Robot Dreams - http://robot-dreams.net
-- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail