Re: [boost] Fwd: Boost.Python build error in 1.58 RC1, 2 and 3 (OK in Beta 1)

Subject: Re: [boost] Fwd: Boost.Python build error in 1.58 RC1, 2 and 3 (OK in Beta 1)
From: Jürgen Hunold (jhunold_at_[hidden])
Date: 2015-04-16 04:21:51

Hi Rene,

Am Mittwoch, 15. April 2015, 08:42:40 schrieb Rene Rivera:
> On Wed, Apr 15, 2015 at 8:24 AM, Jürgen Hunold <jhunold_at_gmx.eu> wrote:
> > And our current testing system is hardcoded to "master" and "develop"
> > branches. And can't test anything else, as it relies on the test runners
> > admins to set up which branch to test.
> 
> I wouldn't call it hardcoded.. But yes it relies on testers signifying the
> branch to test. And yes it could be more flexible.

More flexible and more responsive are the keys I think. Turnaround times are 
bound to be long if there is no central distribution of tasks. Or at least 
branches to test, which should be sufficient for Boost.

> > Note that when I say "branch" above I don't necessarily mean "git branch".
> > 
> > > In fact, it may be more convenient to use forks to isolate the code to
> > > be
> > > released from the main code base and use pull requests for patches that
> > > need to get into the release.
> > 
> > As soon as we get an up-to-date regression testing and reporting system,
> > we
> > can use any branching scheme you can imagine. And a lot more probably.
> 
> Not sure what you mean by "up-to-date". Could you be more specific? And I
> don't accept "use -insert-name- tool" as an answer. I want to know
> structure and process answers (as tools are just tools).

Well, I was referring to the tools below. 

> That would mean using either Jenkins or BuildBot as a centralized server and
> > some dashboard reporting tool for visualisation.
> 
> No it doesn't mean that. Although Jenkins or BuildBot (or a variety of
> other testing resource management) system could be used in a solution.

I'm using BuildBot at work myself and have first-hand reports on Jenkins.

> But
> I'm not aware of *any* current reporting system that meets Boost needs. I
> do spend a minuscule amount of time writing such a system.

Unfortunately, you are right. There is no out-of-the-box system. And I think 
it is hard to write one for Boost due to the diversity of testing methods 
used. From Boost.Test over flyweight test to somethings home grown. Not to 
mention that we only have volunteer testers with some real handicaps dedicated 
by some strange policies...

I dream of something like Gerrit (see https://codereview.qt-project.org/#/q/status:open,n,z for the Qt one) for Boost. But this requires a 
working CI system, a completely different workflow and whatever. 
But on the other hand I don't know if growing a boost centric regression 
testing/CI framework is feasible. Especially if looking at the experience with 
Boost.Build.

> > But I don't know anyone with enough spare time to set up such a system
> > without
> > being paid and working full time. Not to mention the hardware necessary
> > for
> > this...
> 
> But as you just said.. I volunteer my unpaid free time to maintain and
> improve the testing infrastructure.

Which I really appreciate. 

My main point was to highlight the fact that we can't change the release 
process without tackling the testing infrastructure. And even adding another 
long standing release branch or whatever is not easy at the moment.

Yours,

Jürgen
-- 
* Dipl.-Math. Jürgen Hunold  ! 
* voice: ++49 4257 300       ! Fährstraße 1
* fax  : ++49 4257 300       ! 31609 Balge/Sebbenhausen
* jhunold_at_gmx.eu             ! Germany