|
|
Boost Testing : |
Subject: Re: [Boost-testing] [EXTERNAL] Regression zip failure
From: Belcourt, Kenneth (kbelco_at_[hidden])
Date: 2014-01-29 15:48:49
On Jan 29, 2014, at 1:15 PM, Belcourt, Kenneth wrote:
> On Jan 29, 2014, at 1:05 PM, Belcourt, Kenneth wrote:
>
>> On Jan 29, 2014, at 11:38 AM, K. Noel Belcourt wrote:
>>
>>> On Jan 29, 2014, at 12:00 AM, Tom Kent wrote:
>>>
>>>> On Wed, Jan 29, 2014 at 12:57 AM, Tom Kent <lists_at_[hidden]> wrote:
>>>>> On Tue, Jan 28, 2014 at 7:05 AM, Beman Dawes <bdawes_at_[hidden]> wrote:
>>>>>> On Tue, Jan 28, 2014 at 4:20 AM, Tom Kent <lists_at_[hidden]> wrote:
>>>>>>>
>>>>>>> On Sat, Jan 25, 2014 at 7:22 AM, Tom Kent <lists_at_[hidden]> wrote:
>>>>>>>> I'm seeing this on a few of my runners at the end of what looks like a
>>>>>>>> successful run...any ideas what to do about it?
>>>>>>>>
>>>>>>>> Tom
>>>>>>>>
>>>>>>>> # Could not find 'zip_cmd' module in the script directory
>>>>>>>> File
>>>>>>>> "/home/tomkent/teeks99-03/g/tools_regression_src/collect_and_upload_logs.py",
>>>>>>>> line 351, in collect_logs
>>>>>>>> , os.path.join( results_dir,'%s.zip' % runner_id )
>>>>>>>> File
>>>>>>>> "/home/tomkent/teeks99-03/g/tools_regression_src/collect_and_upload_logs.py",
>>>>>>>> line 278, in compress_file
>>>>>>>> raise Exception( 'Compressing failed!' )
>>>>>>>> Exception: Compressing failed!
>>>>>>>
>>>>>>> Without any apparent action on my part the previous error changed to this:
>>>>>>> ... loop in tree walk
>>>>>>
>>>>>>
>>>>>> I'm wondering if the previous crash poisoned that runner somehow? Several
>>>>>> runners cycled successfully last night after the fixes Noel applied, so it
>>>>>> is possible that the problem you are seeing is something very specific to
>>>>>> that particular runner. Perhaps delete the directory, and then recreate the
>>>>>> runner?
>>>>>>
>>>>> I just tried a fresh git repo, no change. It seems to only be
>>>>> happening with clang builds, gcc runs fine.
>>>>> Tom
>>>>
>>>> My results/teeks99-03f-Ubuntu12.04-64.xml file is 2.4GB, is this too
>>>> big for the tool to process?
>>>
>>> My gcc builds produce an xml file that is order 40 Mb so 2.4 Gb is on the large side. Let me rerun with clang and see what happens, this is with master right?
>>
>> Clang 3.4 on master on Linux yields a 20 Mb xml file. I'll try develop next.
>>
>> Whoa, clang-3.4 with develop has a problem, I too hit the massive output file (this is just the log but the xml will be similar in size).
>>
>> -rw------- 1 kbelco kbelco 2932328219 Jan 29 12:58 bjam.log
>>
>> I'll take a look at this.
>
> Here's a big part of the problem, clang on develop generate 2000+ lines of compiler output messages, there's 116 warnings that generate all this output for this one build target.
>
> clang-linux.compile.c++.without-pth /scratch/kbelco/boost-testing/results/boost/bin.v2/libs/test/build/clang-linux-clang++/debug/test_tools.o
>
> "clang++" -c -x c++ -O0 -g -fno-inline -Wall -pedantic -g -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_TEST_DYN_LINK=1 -I".." -o "/scratch/kbelco/boost-testing/results/boost/bin.v2/libs/test/build/clang-linux-clang++/debug/test_tools.o" "../libs/test/src/test_tools.cpp"
>
> A few years ago, I put a capability into bjam to restrict how much output is captured from target (the default is to capture all output), we clearly need to restrict the buffer size to 5k per target so it doesn't overwhelm process_jam_log. The bjam (b2) option to do that is this:
>
> -mx Maximum target output saved (kb), default is to save all output.
>
> So pass it to bjam via this option to run.py (--bjam-options="-j80 -m5") to save only 5kb per target.
>
> FYI, most of the problems seems to be in this develop library boost/preprocessor, various kinds of macro related warnings.
I reran clang on develop with -m2, the log and xml file are now reasonably sized, hopefully the results will appear on the web page shortly (Sandia-Linux).
-rw------- 1 kbelco kbelco 35687373 Jan 29 13:45 bjam.log
-rw------- 1 kbelco kbelco 23085286 Jan 29 13:47 Sandia-Linux.xml
-- Noel