Re: [boost] Testing and toolsets

Subject: Re: [boost] Testing and toolsets
From: Ben Pope (benpope81_at_[hidden])
Date: 2015-01-19 10:03:37

On Monday, January 19, 2015 10:47 PM, Andrey Semashev wrote:
> On Monday 19 January 2015 15:14:21 Andrzej Krzemienski wrote:
>> Hi Everyone,
>> Could anyone give me some information on how regression test matrix is
>> supposed to work? I can see that when testers run their tests they specify
>> a toolset; this toolset is reported in test matrix, and based on this
>> toolset name failures in tests can be labeled as expected. Did I get it
>> right?
>>
>> However, the fact that a tester turns C++11 support or not, is not
>> necessarily reflected in the toolset name (some testers indicate it in the
>> toolset name, others don't). On the other hand C++11 being turned on or off
>> can affect the test results. In that case the toolset name alone is not
>> sufficient to make a decision whether I expect a failure or not.
>>
>> Is there some policy for addressing this problem? Am I the only person to
>> be affected?
>
> AFAIK, there is no formal property of a toolset that would differentiate
> testers between C++ versions. You basically rely on toolset names.
> Unfortunately, some testers reflect C++ version in the tester name rather than
> the toolset, and I don't think there's a way to use the tester name in the
> markup table. I would kindly ask those testers to change their settings.
>
> If you just want to see the tester description you can read it through the
> link which is the tester name in the matrix. Some testers provide more
> detailed info there.

Should there be a C++ version in the list of build features?  Would that 
help?

http://www.boost.org/build/doc/html/bbv2/overview/builtins/features.html

I'm more than happy to change anything with my configuration, although I 
think enough details are there, if any more are needed, or the 
formatting can be improved, feel free to make suggestions.

Ben

(BP x86_64 C++11) - the name is no longer correct, but changing it means 
somebody has to delete the old results file)