Re: [boost] [BoostCon07][testing] Session reminder.

From: David Abrahams (dave_at_[hidden])
Date: 2007-05-04 10:41:45

on Wed May 02 2007, Rene Rivera <grafikrobot-AT-gmail.com> wrote:

> Boost Community,
>
> The Boost Conference will be here in very short order! And I need to get 
> feedback for the "Testing Boost" sprint we'll be doing 
> <http://boostcon.com/program/sessions#rivera-testing-sprint>. If you 
> have any ideas, concerns, off the wall comments, etc. at *minimum* send 
> them directly to me at "rrivera/acm.org". Why that account? Well because 
> that will be the one account I can guarantee I will access to during the 
> conference, and hence people can send me stuff up to the last minute. 
> Not that I'm encouraging lateness, but I know how busy we all are ;-)
>
> If you are going to send it to the Boost dev list you will also want to 
> eventually send me a summary follow up, if needed, from the ensuing 
> discussions.

Okay, here are some issues I think ought to be solved, in no
particular order, some (much) more important than others:

* inaccurate header dependency tracing impedes useful incremental
  testing

* report usability is poor for most constituencies.  This point could
  be explored in much more detail.

* no mechanism to pinpoint the checkin that caused a regression

* no mechanism for communicating that tests are known not to work on
  some platform (toolset+os) to the build system, and so avoid
  building them

* generating XML by parsing the jam log is fragile and prevents the
  use of multiple build processes (-jN).  This one should be almost
  embarassingly easy to fix.

* The smallest granularity of test that any tester can contribute is
  the entire Boost suite for one toolset, which makes for longer
  turnaround.

* No way for a developer to request testing results for a specific
  branch

* No "nanny system" that bothers the developer who causes a regression
  until it is fixed.

I think I've thought of others from time-to-time; these are just the
ones off the top of my head this morning.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com
Don't Miss BoostCon 2007! ==> http://www.boostcon.com