[Boost-users] "Re: [boost] Boost-1.62: How to get list of all dirs under 'run-tests libs:' target from status/Jamfile?"

Subject: [Boost-users] "Re: [boost] Boost-1.62: How to get list of all dirs under 'run-tests libs:' target from status/Jamfile?"
From: Sergey Sprogis (sergey.sprogis_at_[hidden])
Date: 2016-09-09 15:04:03

This is nice idea, but unfortunately does not work for my purpose of 
calculatingpass rates for every test suite of regression testing.
It would fit perfectly if instead of ---dumptest b2 will also have 
another option, something like: --dumptestdirs. I mean, the option which 
generates the whole list of dirs containing all jamfiles executed for 
full regression testing.

Actually, I spent some time trying to generate such list of dirs based 
on the output produced  by 'b2 -n -a -d0 --dump-tests', but was not able 
to find general algorithm
[ wout using specific library names and whout parsing the content of 
jamfiles ]  which transforms list of tests into the list of test dirs 
from which they are launched
[ assuming that each test should be executed only once ]. Here just 
couple of examples which demonstarte those difficulties:

1. In 'wave' its 'test' subdir does not contain jamfile   like many 
other libsdo, but rather wave's jamfile is located   inside internal 
'build' subdirectory, and it launches testslocated from 'testlexers'  
'testwave' subdirs which are located  on the same level as 'build' 
itself.  When my  script sees those  wave tests from --dumptest output, 
how should it figure out that  jamfile which launches them is located 
inside 'build' dir?

2. Inside 'unordered' library  there are two subdirs with jamfiles:   
test/exception and test/unordered. All tests for 'unordered'  may be run 
in two similar ways: either by executing both those jamfiles,  or by 
executing  jamfile which is located on higher level inside 'test'  
subdir.   But to run every test once the script should choose  one or 
another way. Now, in a similar situation for 'core' library,  in order 
to run all its tests one time my script should run both jamfiles:  
internal one located inside core/test, and also another one located in 
core/test/swap. How the script should resolve such ambiguity for 
'unordered' and core'  libraries?

So, I think that list of test dirs which was present in 
status/Jamfile.v2 in boost-1.61.0 was really useful to me. But, of 
course if it was created manually, and there is no easy way to do it 
automatically, then  probably, I should also create it manually for 
Boost-1.62 and all later versions.

Sergey

 > 2016-09-04 11:45:41 Rene Rivera (/grafikrobot_at_[hidden]/)

 > wrote:

 > You can get a list of all the tests that get, or will get, run from 
b2 with the following:

 > % cd boost-root/status
 > % b2 -n -a -d0 --dump-tests

 > That will produce, interspersed with other output, lines like the 
following:

 >  boost-test(RUN) "accumulators/count" : 
"libs/accumulators/test/count.cpp"

 > One of those for each test that is tested. The format is:

 >  boost-test(<TYPE>) "<LIBRARY>/<TEST>" : "<SOURCE>"

 > Where LIBRARY can be of the form "<library>" or 
"<library>/<sublibrary>".
 > That should be sufficient to produce the results you want. And if you 
use
 > just the above list of tests it's likely to be more accurate than your
 > previous method. As the tests runs from the "boost-root/status" dir 
are not
 > alway the same as those run from the corresponding library dirs.

> --
>-- Rene Rivera

 >>On Sat, Sep 3, 2016 at 5:09 PM, Sergey Sprogis 
<sergey.sprogis_at_[hidden]>
 >> wrote:

>> I use latest Boost versions to test  new C++ compiler in development stage.
>>
>> It's important for me to know what  would be "pass rate" for every library
>> which Boost.org reports on its  Regression Testing website.
>> To calculate all those pass rates I  created script which goes inside every
>> library's 'test', 'example' or  whatever subdirectory and launches bjam from
>> there.
>> It creates a separate logfile(s)  for every library which I then analyze to
>> calculate "pass rate".
>>
>> But before launching my script I  need to know the whole list of dirs which
>> Boost.org uses for its Regression  testing.
>> In the previous Boost-1.61.0  version I was able to easily produce such
>> list by extracting it from  'status/Jamfile.v2'. It looked like this:
>>
>> run-tests libs :
>> accumulators/test # test-suite  accumulators
>> algorithm/test # test-suite  algorithm
>> algorithm/minmax/test # test-suite  algorith/minmax
>> ...
>> concept_check # test-suite  concept_check
>> config/test # test-suite config
>> container/bench # test-suite  container benchmarks
>> container/example # test-suite  container_example
>> .....
>>
>> As you may see this list contains  not only 'test' subdirs, but also some
>> others: 'example', 'bench', etc.
>> But now the problem is that in  Boost-1.62.0 'status/Jamfile.v2' does not
>> have such full list anymore.
>>
>> Could anyone suggest me some hack  how to produce the full list of tested
>> libraries for Boost-1.62.0, similar  to one which Boost-1.61.0 has?
>>
>> It would be nice if bjam will  eventually have a special option which will
>> allow to produce such list, but for  now I'd appreciate just a hack which
>> will help me to solve this issue.
>>