|
|
Boost Testing : |
From: Aleksey Gurtovoy (agurtovoy_at_[hidden])
Date: 2005-03-15 05:15:16
Carl Daniel writes:
> Now that I can finally post :)
Welcome!
> I'd like to get my regression-test-running automated. Currently, a
> number of tests fail with runtime dialogs or JIT debugger dialogs.
> Using --monitored on regression.py helps somewhat, but it's not
^^^^^^^^
> really a solution.
Meaning you've encountered a situation when it didn't help? Just
curious, because to our best knowledge, while not without serious
usability problems (which I pointed out offline earlier), it does its
job after all. We've been running unsupervised Win32 regressions with
it 24/7 for almost 2 years.
>
> So, what other complaints are there about build monitor now? What
> desired features are not supplied by the current implementation?
To my best knowledge, there are two problems with the current
implementation:
1) It's too greedy in killing processes; specifically, it doesn't
conform to the "ideal" spec, which is:
"Regression tests' monitoring tool should terminate any process
that:
- Is spawned by 'bjam.exe' executable (or is the executable
itself).
- It has been running for more than N minutes _or_ showed an
error dialog window.
"
2) It doesn't handle the case of missing DLL dependencies; on Win32
this results in a error dialog "DLL such and such is not found",
which stalls futher running of tests.
The use cases that led us to the above formulation of the desired
behavior were:
- So-called "access violations"
- In bjam
- During compilation, linking, and running of tests
- Error dialogs shown by C++ runtime support code
- Processes hanging/running too long:
- Tests
- Compilers/linkers
Issue #2, which we discovered later, obviously adds this one:
- Other error dialogs shown by operating system (e.g. missing DLL
dependencies).
>
> I'd like to help make a better solution - and at least on Windows,
> I'm confident that a much better solution is possible.
That's great to hear! We were hoping for someone to come and say that
right :).
-- Aleksey Gurtovoy MetaCommunications Engineering