[boost] Re: Regression testing goes horribly wrong! (+ patch)

From: Guillaume Melquiond (guillaume.melquiond_at_[hidden])
Date: 2004-07-11 00:40:21

Le dim 11/07/2004 à 04:46, Aleksey Gurtovoy a écrit :

[...]

> >> This was fixed in the CVS about two weeks ago (see
> >> http://tinyurl.com/36v6h#dont_redirect, for instance); simple
> >> rebuilding "process_jam_log" executable will take care of it (I would
> >> have posted a notice about this fix if I didn't forget that unlike us
> >> not everybody rebuilds "process_jam_log" on every run).
> >
> > That's strange. I do use the CVS version and I did rebuild the
> > process_jam_log. 
> 
> Did you do a clean run?

Yes. It's what my sentence was supposed to meant.

[...]

> Given that we remove all binaries and sources and rebuild everything
> from scratch on every run, I'm inclined to think that our fix works
> and that the problems you see are caused by a stale environment.

Seems like your fix never get commited if it does fix for you.

> If you still can reproduce the problem on a clean state, you can
> always step through the process_jam_log code in debugger and see how these
> empty test types appear there.

I did. The reason is quite simple. And it's what I have been explaining
in my previous mails: test_info is not created with the same name that
the name used to access it later. The test type is set at creation time,
and the test_log.xml is used at a later time. So they are no more
related since the test type cannot be retrieved during the second
access.

[...]

> > But process_jam_log can't actually deal with them, 
> 
> It definitely can, please see below.

Please see below why it definitely cannot.

> > because it does
> > around line 520:
> >
> > test2info.insert( std::make_pair( test_name, info ) );
> >
> > test2info is a map and test_name is the short name of test. 
> 
> Nope, it's not. It's the name extracted from one of the top lines of
> the bjam log, and the lines corresponding to the "new" test cases do
> contain the library name:
> 
>     boost-test(RUN) "utility/ref_test" : "libs\utility\ref_test.cpp"
>     boost-test(RUN) "utility/addressof_test" : "libs\utility\addressof_test.cpp"
>     ...

boost-test(RUN) "ref_test" : "libs/utility/ref_test.cpp"
boost-test(RUN) "addressof_test" : "libs/utility/addressof_test.cpp"

Maybe you did patch some Jamfiles or bjam? It would explain why your
tests work. And why the tests of all other regression testers fail in a
similar way.

So all I said is still valid.

[...]

> > So maybe we should replace it with something like:
> >
> > test2info.insert( std::make_pair( library_name + '/' + test_name, info ) );
> >
> > What do you think about it?
> 
> That's basically what is already there.

No it isn't unfortunately, since the library_name is not in the
regress.log file when using the CVS version. My second patch is *really*
required so that process_jam_log correctly works. So, unless you find
and commit your patch that puts library_name in regress.log, I will
commit my second patch.

Regards,

Guillaume