Re: [boost] Testing and toolsets

Subject: Re: [boost] Testing and toolsets
From: Andrey Semashev (andrey.semashev_at_[hidden])
Date: 2015-01-19 09:47:01

On Monday 19 January 2015 15:14:21 Andrzej Krzemienski wrote:
> Hi Everyone,
> Could anyone give me some information on how regression test matrix is
> supposed to work? I can see that when testers run their tests they specify
> a toolset; this toolset is reported in test matrix, and based on this
> toolset name failures in tests can be labeled as expected. Did I get it
> right?
> 
> However, the fact that a tester turns C++11 support or not, is not
> necessarily reflected in the toolset name (some testers indicate it in the
> toolset name, others don't). On the other hand C++11 being turned on or off
> can affect the test results. In that case the toolset name alone is not
> sufficient to make a decision whether I expect a failure or not.
> 
> Is there some policy for addressing this problem? Am I the only person to
> be affected?

AFAIK, there is no formal property of a toolset that would differentiate 
testers between C++ versions. You basically rely on toolset names. 
Unfortunately, some testers reflect C++ version in the tester name rather than 
the toolset, and I don't think there's a way to use the tester name in the 
markup table. I would kindly ask those testers to change their settings.

If you just want to see the tester description you can read it through the 
link which is the tester name in the matrix. Some testers provide more 
detailed info there.