[Boost-testing] Re: Whitespace (again)

From: Misha Bergal (mbergal_at_[hidden])
Date: 2005-03-08 22:04:24

"Victor A. Wagner Jr." <vawjr_at_[hidden]> writes:

> Ok, after the big hiatus in posting results, I deleted ALL of my .obj
> and .exe from the results directory and voila, only ONE block of
> whitespace!!!
> here we are just a day later, and three(3) new ones have cropped up:
> concept_check
> preprocessor
> timer
>
> There is obviously something wrong in either the test running scripts
> or the results analysis
> (perhaps it is only the incremental stuff that's broken, but I'm not
> using the incremental option)

Victor, I've briefly looked at your test script and I think I see the
problem there.

Here is the relavant part of you test script:

del results\*.log
...
del results\*.xml /s >garbage
rem del results\*.obj /s >garbage
rem del results\*.exe /s >garbage

>From this I see that you do not completely clean up the results
directory (for sure *.exe files and *.test files are not being
deleted). This means that actually you are doing an incremental run. 

This is undesirable but OK, the big problem is that you delete all
*.xml files as you were doing the full run.

Here is my attempt to provide a helpful summary of the process:

Full run: 

1. bjam builds/test all components, generating bjam.log

2. process_jam_log parses bjam.log and generates *.xml files, one xml
   file for every component (test/lib etc.)

3. regression.py scans for *.xml files in the results dir concatenates
   them and uploads them to the processing server.

Incremental run:

1. bjam rebuilds modified components, generating bjam.log

2. process_jam_log parses bjam.log and generates *.xml for every modified
   component.

3. regression.py scans for *.xml files in the results dir, concatenates
   them and uploads them to the processing server. 

Important: Processing server doesn't distinguish between full and
incremental runs. It needs the results for all components.

It seems to me that in your case, some xml files have already been
deleted, so the regression.py doesn't send the results which were in
them to processing server. Thus some results are reported as missing.

I would recommend you to completely clean up the results dir at the
start of each run.

If this doesn't fix the problem, it might be a process_jam_log problem
and will require a closer examination.

-- 
Misha Bergal
MetaCommunications Engineering