Re: [boost] [testing] Need optimization help!!!

Subject: Re: [boost] [testing] Need optimization help!!!
From: Dmitry Moskalchuk (dm_at_[hidden])
Date: 2017-02-01 15:02:21

> If some brave soul could look into why it's so terribly slow the Boost
> community would be very grateful. As it would make it possible to move the
> processing to a CI service and improve results reliability and frequency.

As you may know, we are running our own Boost-on-Android tests results
site https://boost.crystax.net/, which displays content, generated by
Boost regression scripts (i.e. the same as official Boost testing report
page), but filtered by Android OS. This is done for our convenience, to
make it possible to re-generate reports manually if needed, and to be
able to set own schedule of reports generation, not bothering Boost team
with that. We use Boost regression tests set mainly for testing CrystaX
NDK itself rather than Boost, but in practice this leads to the same
effect as for Boost team - we use the same scripts for reports
generation and we get similar results on our web site.

So we've already noticed the fact that reports generation scripts works
slowly. Of course, not so slowly as mentioned by Rene (due to the fact
that we filter runners by name), but still far from ideal. I was
planning to look on that and try to figure out what's wrong there, since
just reporting the issue and not suggesting any solution for it was
looking useless for me. However, last months I just have no free time,
so it's still just a plans and no real work was done in this direction yet.

Also, for those reading it, I'd like to point to the fact it's not only
subject of optimization. I clearly see floating bug in reports
generation. Sometimes (I can't find any regularity yet) generated
reports miss big amount of data from test results. You can see that on
http://www.boost.org/development/tests/master/developer/summary.html,
looking at CrystaX-apilevel-21-armeabi-v7a-llvm-libc++ runner - a) there
is only gcc-4.9 listed, even though test results contains data for
gcc-5, gcc-6, clang-3.6, clang-3.7 and clang-3.8 and b) even for gcc-4.9
it shows results only for couple of libraries, completely missing data
for others. Such wrong report generation happens from time to time for
different set of API level/ABI/C++ Standard Library used, without any
visible regularity.

So, taking above into account, I'd be happy to say that I'd get this
task and fix it, but I really can't say anything regarding timing - it
may be sooner than I suppose, but most likely will be later - couple of
months from now as I assess it. If someone will take it and fix before I
could, that would be just great (in this case please take into account
floating bug described above). However, if no one would be able to take
this task and complete it to the moment when I will have a time, I'll
definitely try to fix it, just because it's something needed for our
project too, not only for Boost.

-- 
Dmitry Moskalchuk