Fwd: Re: Boost.Threads and Regression Testing

From: bill_kempf (williamkempf_at_[hidden])
Date: 2002-02-27 14:43:50

--- In jamboost_at_y..., "David Abrahams" <david.abrahams_at_r...> wrote:
> 
> ----- Original Message -----
> From: "William Kempf" <williamkempf_at_h...>
> > > > I'd really like to include Boost.Threads in the regression 
testing.
> > >Have
> > > > you put any thought into how this could work? I've tried the
> following,
> > >but
> > > > receive an error.
> > > >
> > > > run libs/thread/test/test_thread.cpp
> <lib>libs/thread/build/boost_thread
> > > > <dll>libs/thread/build/boost_threadmon
> > > > : #args
> > > > : #input-files
> > > > : #requirements
> > > > <include>$(BOOST_ROOT)
> > > ^^^^^^^^^^^^^^^^^^^^^^
> > >Shouldn't need this part, it's added automatically
> >
> > This is nice to know. Is it true only for the status/Jamfile or 
is this
> now
> > a feature in all Jamfiles in Boost?
> 
> No, just in status, and only for the testing targets described 
there (IOW,
> "exe" still doesn't get <include>$(BOOST_ROOT)
> 
> > > > <runtime-link>static/dynamic
> > > ^^^^^^^^^^^^^^
> > >Can't use this form in requirements specification. What's your 
intention,
> > >to
> > >say that it can be built both ways? In that case, you can just 
leave it
> > >out.
> >
> > Then which variant does it choose to use?
> 
> I think you're confusing terms. Variants are collections of build
> properties. debug, release, and profile are all variants.

Sorry, subvariant is the term used, I guess.

> Anyway, if you leave it out, it uses whatever is specified by the 
build. If
> the build doesn't specify, it uses the default value for the 
feature (as
> long as the feature is relevant to the toolset). The default for 
the feature
> is its first value as declared in features.jam. In this 
case, "dynamic".

The build specifies <runtime-link>static/dynamic.

> > > > <threading>multi
> > > > ;
> > > >
> > > > The error is:
> > > >
> > > > No compatible subvariant found for:
> > > > <status!C:!home!boost!libs!thread!build>libb
> > > > oost_thread.lib
> > >
> > >I'm pretty sure that's because it's trying to find a subvariant 
whose
> > ><runtime-link> setting is static/dynamic. That shouldn't work ;-)
> >
> > But it does in the actual Jamfiles for the projects. Why should 
it not
> work
> > here?
> 
> It "worked" by accident. What did it actually do? Nobody wrote code
> intending to make that do anything sensible.

The documentation mentions this in a few different places. Rene also 
just spent several hours getting this to work correctly. It's 
supposed to build multiple subvariants for both static and dynamic 
linking to the runtime.

> > > > I assume it can't figure out whether to use debug or release?
> > >
> > >No, debug is the default BUILD setting.
> >
> > It might be important to regression test stuff with both BUILD 
settings.
> > For example, the VC++ compiler does some special "magic things" 
in debug
> > builds, such as initialize all pointers to special values. 
Occasionally
> > this can result in bugs that occur in release builds that don't 
occur in
> > debug builds. Of course, this is a regression test and not a 
unit test,
> so
> > maybe you should just ignore what I said... ;).
> 
> Almost none of the tests make sense in release mode, since 
assertions get
> compiled away.

Assertions get compiled away, but the Boost.Test library still 
reports errors in release builds. I unit test Boost.Threads in both 
release and debug on a regular basis (and I have encountered errors 
in only one in the past). Again, unit testing and regression testing 
are different, so I'm not sure if this applies here.

> > >I tried this, and found a bug in the status/Jamfile which I 
fixed. I also
> > >added your tests. It doesn't link, but I'm sure you can figure 
that one
> > >out.
> >
> > Actually, no, that's proving a bit vexing and I *think* it's 
still a build
> > system problem. When I run using jam -d+2 to see what's going on 
I see
> the
> > error as:
> >
> > link /nologo /DEBUG
> >
> /PDB:"..\status\bin\test_thread.test\msvc\debug\runtime-link-
dynamic\threadi
> ng-multi\test_thread.pdb"
> >
> >
> /out:"..\status\bin\test_thread.test\msvc\debug\runtime-link-
dynamic\threadi
> ng-multi\test_thread.exe"
> > /LIBPATH:C:\PROGRA~1\MICROS~4\VC98\lib
> >
> @"..\status\bin\test_thread.test\msvc\debug\runtime-link-
dynamic\threading-m
> ulti\test_thread.CMD"
> >
> > libboost_thread.lib(tss.obj) : error LNK2001: unresolved external 
symbol
> > __imp__on_thread_exit
> >
> ..\status\bin\test_thread.test\msvc\debug\runtime-link-
dynamic\threading-mul
> ti\test_thread.exe
> > : fatal error LNK1120: 1 unresolved externals
> >
> > on_thread_exit() is a method from the boost_threadmon.dll so it 
appears
> that
> > the build system isn't linking against the DLL's import library. 
Sure
> > enough, looking at the contents of test_thread.CMD shows the 
following:
> >
> >
> "..\status\bin\test_thread.test\msvc\debug\runtime-link-
dynamic\threading-mu
> lti\test_thread.obj"
> >
> "..\libs\thread\build\bin\libboost_thread.lib\msvc\debug\runtime-
link-dynami
> c\threading-multi\libboost_thread.lib"
> >
> > Despite the <dll> specification the boost_threadmon.dll's import 
library
> > isn't being linked. Curiously, there is output indicating that 
the DLL
> was
> > built because of the <dll> specification, however.
> 
> Hmm, I think I'm going to have to ask Rene to look at this one. He's
> responsible for the <dll> dependency handling.

OK.

> > >I killed the bogus <runtime-link>static/dynamic in your 
requirements
> > >section
> > >(perhaps you'd like that in the default-BUILD, but we don't have 
a
> > >default-BUILD section for test targets in status/Jamfile. Maybe 
we should
> > >fix this).
> >
> > I think it's important to regression test every variant of the 
library.
> > Debug/release variants are a little debatable, but other variants 
can mean
> > very drastic differences in test results. Which runtime we use is
> something
> > that may, indeed, change the behavior of tests.
> 
> You can always make two separate test targets:
> 
> test-suite threads
> : [
> run libs/thread/test/test_thread.cpp
> <lib>../libs/thread/build/boost_thread
> <dll>../libs/thread/build/boost_threadmon
> : : :
> <runtime-link>static
> <threading>multi
> : threads-static
> ]
> 
> [
> run libs/thread/test/test_thread.cpp
> <lib>../libs/thread/build/boost_thread
> <dll>../libs/thread/build/boost_threadmon
> : : :
> <runtime-link>dynamic
> <threading>multi
> : threads-dynamic
> ]
> ;

OK, but for builds the <runtime-link>static/dynamic *IS* currently 
documented as the correct way to do this, and it would be better if 
the two behaved in the same manner here.

Bill Kempf