|
|
Boost Testing : |
From: Aleksey Gurtovoy (agurtovoy_at_[hidden])
Date: 2005-03-16 06:35:25
Carl Daniel writes:
> Aleksey Gurtovoy wrote:
>> Carl Daniel writes:
>>> I'd like to get my regression-test-running automated. Currently, a
>>> number of tests fail with runtime dialogs or JIT debugger dialogs.
>>> Using --monitored on regression.py helps somewhat, but it's not
>> ^^^^^^^^
>>> really a solution.
>>
>> Meaning you've encountered a situation when it didn't help? Just
>> curious, because to our best knowledge, while not without serious
>> usability problems (which I pointed out offline earlier), it does its
>> job after all. We've been running unsupervised Win32 regressions with
>> it 24/7 for almost 2 years.
>
> When running tests interactively with --monitored, it successfully
> kills the errant processes, but the JIT debugger dialog is produced
> by a different process, so at the end of the run I have a screen
> full of "do you want to debug" dialogs that I have to close.
Understood. Our regression machine has JIT debugging disabled, so
we never came across this one.
[...]
>
>> The use cases that led us to the above formulation of the desired
>> behavior were:
>>
>> - So-called "access violations"
>> - In bjam
>> - During compilation, linking, and running of tests
>>
>> - Error dialogs shown by C++ runtime support code
>>
>> - Processes hanging/running too long:
>> - Tests
>> - Compilers/linkers
>>
>>
>> Issue #2, which we discovered later, obviously adds this one:
>>
>> - Other error dialogs shown by operating system (e.g. missing DLL
>> dependencies).
>
> A couple of things that will help (I haven't yet tried to find the
> appropriate sources to see what's in there already). The main() for
> tests (and maybe bjam itself) should be written something like:
>
> int main(/* params */)
> {
> // prevent CRT error dialogs
> _set_error_mode(_OUT_TO_STDERR);
>
> // prevent JIT debug dialogs
> __try
> {
> // previous contents of main()
> }
> __except(EXCEPTION_EXECUTE_HANDLER)
> {
> // Courtesy stop in the debugger, if present
> if (::IsDebuggerPresent())
> ::DebugBreak();
> // Paranoid - bail out of the process rather than attempting
> // any orderly shutdown
> ::TerminateProcess(::GetCurrentProcess(),255); // or whatever exit
> code
> }
> }
>
> with that code in place in the test programs' main(), there should
> be no more dialogs from the CRT/Debugger.
That's MSVC specific, though, right?
>
> To handle all cases - like missing DLLs, as well as handle stuck
> programs, the best solution may be to write a custom
> debugger/lancher that's used as a trampoline for launching all child
> processes of bjam (or at least the tests, compiler and linker).
This is our current understanding as well.
>
> so, rather than invoke cl.exe <arguments>, bjam would invoke monitor
> <timeout> cl.exe <arguments>, or similar.
Ideally, bjam itself should be monitored as well, so I'd say the
command line for running tests would be
monitor <timeout> bjam ....
Obviously, timeout would apply only to the child processes.
>
> A debugger/launcher like that is not a difficult program to write.
> Such a debugger could also hook any number of Windows API functions
> (CreateDialog, etc) to prevent the subject process from creating any
> kind of GUI. A debugger can intervene in the DLL loading process as
> well (I believe - I haven't verified that).
Sounds perfect.
>
> One downside to the debugger solution is that the test runner would
> be required to have SeDebugPrivilege. I would expect that nearly
> all (if not all) developers (and their regression test suites) are
> running with SeDebugPrivilege already, since you cannot debug
> without it.
Yes, I don't think that's going to be a problem.
>
> I'm willing to take on writing that debugger/launcher if it sounds
> like it could help.
It most definitely does!
-- Aleksey Gurtovoy MetaCommunications Engineering