Boost logo

Boost :

Subject: Re: [boost] Testing and toolsets
From: Andrzej Krzemienski (akrzemi1_at_[hidden])
Date: 2015-01-19 14:35:25

2015-01-19 18:29 GMT+01:00 Rene Rivera <grafikrobot_at_[hidden]>:

> Consider this my view as testing manager..
> On Mon, Jan 19, 2015 at 8:14 AM, Andrzej Krzemienski <akrzemi1_at_[hidden]>
> wrote:
> > Hi Everyone,
> > Could anyone give me some information on how regression test matrix is
> > supposed to work? I can see that when testers run their tests they
> specify
> > a toolset; this toolset is reported in test matrix, and based on this
> > toolset name failures in tests can be labeled as expected. Did I get it
> > right?
> >
> Correct.
> However, the fact that a tester turns C++11 support or not, is not
> > necessarily reflected in the toolset name (some testers indicate it in
> the
> > toolset name, others don't). On the other hand C++11 being turned on or
> off
> > can affect the test results. In that case the toolset name alone is not
> > sufficient to make a decision whether I expect a failure or not.
> >
> Right.
> Is there some policy for addressing this problem? Am I the only person to
> > be affected?
> >
> My view is that you should not use the toolset name as a basis for markup.
> In an ideal world the status markup would entirely go away. As it's an
> inferior mechanism for dealing with expected test variance. Instead please
> use the config method as previously (and below) mentioned.

I am sorry, I may be thinking slowly, but what config method are you
referring to?


Boost list run by bdawes at, gregod at, cpdaniel at, john at