On Fri, Jun 10, 2011 at 5:13 PM, Gennadiy Rozental
<rogeeff@gmail.com> wrote:
Ovanes Markarian <om_boost <at>
keywallet.com> writes:
> All in all it sound as a huge amount of work ;)
Either on a local host or in a cloud. Running in multiple threads have some
small advantages (like no need to serialize extras test case data and some
existing infrastructure), but IMO it breaks most important restriction of unit
testing - all test units needs to be run independently (unless your classes are
specially prepared for running in MT environment). Running in multi-process
scenario is much more close to running test cases serially. In addition we do
not need to worry about syncing access Boost.Test internals.
I might agree, but it heavily depends on the target platform of test framework. It is really more clean to have the process based context for TC execution, but on many embedded platforms you simply do not have processes, but a single image (which includes your app, operating system etc). Usually such image is started by the bootloader and executes multiple tasks. Tasks might be seen in this context as thread equivalents (because each task owns a separate stack, but they all run within the same address space). Therefore, it is still beneficial to have a test app which can use multi-core ability of the embedded target to execute much faster. Because in some complex systems (e.g. telecoms ) there might be some thousands of TCs. Running them parallel on a multi-core processor might greatly reduce CI times. On the other hand it might be simply impossible or very difficult to produce and run per core images (instead of per processor). I see these major embedded systems as a C++ niche. Therefore there is a need of some easily portable task layer for parallel TC execution from within one process (or let's say execution image).
2. Managing dependency should not be a problem IMO. Boost.Test has rather simple
model and running test units in parallel should not complicate it much.
But how do you see the dependencies in the tcs? Does it make sence to run a dependent tc if the main tc already failed? There must be some split, which splits tcs and considers their dependency graphs. What happens now if e.g. there is a cyclic dependency?
3. I do not see, why we need to parallelize on test suites level. IMO it should
be on test case level.
Isn't it so that TC and TestSuite have a common base class, which is additionally is responsible for dependency management? If so this class should be used as parallelization unit ;)
4. I do not see a problem with dependence on either Boost.Thread or Intel TBB
(or anything else for that matter). In my design the logic of distribution
resides in standalone runner application and we can have number of different
runners with different dependencies. There is only some generic "distribution
support logic which will reside in main library.
Actually, when I wrote my previous post I was thinking about, but than it is some kind of a spin off, which in case of dependency upon TBB will require additional lib to be installed and will not build with the whole boost?
And it does indeed non trivial piece of work. So if there are any volunteers...
I'll be happy to help with general design and some core infrastructure changes
We could try that ;) Sounds really interesting to me.
Gennadiy
Regards,
Ovanes