[boost] Re: Regression report for Releases?

From: Aleksey Gurtovoy (agurtovoy_at_[hidden])
Date: 2005-03-22 09:46:07

Rene Rivera writes:
> Aleksey Gurtovoy wrote:
>> Jeff Garland writes:
>> 
>>>>  - Regressions from the previous release are nice to know but less
>>>>    important.  I realize we show both in one report, but this may
>>>>    help us adjust our emphasis or coloring (maybe it's already
>>>>    perfect in the user report; I don't know)
>>>
>>>In fact, I think from the user perspective the question really goes something
>>>like: "I'm using Intel 8.1 on Windows and Linux and I want to use Python,
>>>smart_ptr, and serialization -- can I expect these libraries to work with
>>>release xyz?"  And several variations on that theme. So in an "ideal world"
>>>scenario I would have a web form where the user could enter her needs and a
>>>script would filter the regression results down to the set of interest for the
>>> user. 
>> Same for a developer, actually, who usually wants to track only a
>> couple of libraries. The problem with such scheme is that it by
>> definition bumps up requirements on the reports' hosting site. Right
>> now the pages are nothing but plain HTML. Ideally, to handle the kind
>> of dynamic requests you describe in real time we need a database
>> backend, and that severely limits where the reports can be hosted. If
>> we as a group decide that the benefits of having something like this
>> outweigh the downside, this can be pulled off quite easily.
>
> I'm not sure having such a set of dynamic result pages would
> help. It's nice to be able to answer the question of "I'm using _x_
> compiler and _y_ and _z_ libraries, does it work?". But that can just
> as easily be answered with reports that are limited to the platform
> and compiler. I've been in the situation of checking the results to
> see if what I'm using (smart_ptr, regex, spirit, threads, etc.) work
> for Linux+GCC-3.2.3. The user report I would have rather seen would be
> one that lists all the results for that one platform. 

I'm not sure how typical it is, though. For instance, I know that, as
Boost users, here at Meta we care for more than one platform and for
more than one compiler on each platform, and I'm sure that we are not
the only mutli-platform folks out there.

> I think that the big result grid, and the results by library only
> really help the library developers.

One advantage of a "big grid" is that when things are well (and when
we release, they are :), it inspires significant confidence in
quality and portability of the libraries. For instance, as a user, I
find this one is very inspiring:
http://www.meta-comm.com/engineering/boost-regression/1_32_0/developer/summary_release.html

>
> So if we are wishing for things to happen, I'd say to change the
> user reports so that it presents single toolset results
> individually.

I'd hate to loose a user-oriented "big picture", but if we are
targeting primarily _release_ user reports, we can have both.

-- 
Aleksey Gurtovoy
MetaCommunications Engineering