Re: [boost] The Release Manager's Point of View

From: Gennadiy Rozental (gennadiy.rozental_at_[hidden])
Date: 2007-06-04 19:40:32

"Douglas Gregor" <doug.gregor_at_[hidden]> wrote in message 
news:FC0BF0AD-CC21-430D-98B4-3C4FE793440F_at_osl.iu.edu...
> On Jun 4, 2007, at 5:06 PM, Gennadiy Rozental wrote:
>> Every library is tested against particular set of
>> dependencies selected by developer. But only *one* per lib. It does
>> require
>> additional disk space for source tree copy. I don't believe it major
>> requirement these days.
>
> I thought that too, but you are wrong. One of the most common
> failures with regression testers is that they run out of hard drive
> space, because testing Boost... just a single tree... requires tens
> of gigabytes.

Umm.. so what? My 3 year old desktop got 300 gig. Server sytems should have 
an access to even larger resources.

Anyway. The disk usage issue is manageble on Make system level. During 
testing if library X depends on library A:N1 make system pulls branch N1 
from snv if one is not present.Once library X testing is completed all 
non-HEAD branches of libraries X depends on are removed. It will add some 
time to the testing, but it can be optional and turned on in test host 
configuration.

Another direction is minimize amount to space used by temporary files. If we 
move toward keeping test results in DB like external host. All files 
produced during testing can be removed.

>>> * We don't test the build and install process.
>>
>> What do you want to test? In any case it doesn't make release
>> "unstable"
>
> In an ideal world, we would:
> (1) Build all of Boost, as a user would
> (2) Install Boost
> (3) Build tests against the installed Boost, then
> (4) Run those tests

Do you know any tools that allows this kind of automated testing? This is 
"nice to have", but not showstopper IMO.

>>> * We don't test release versions, even though this is the most used
>>> variant by users.
>>
>> We shouldn't be doing this at all IMO. NO testing during release.
>
> I believe Rene means the "release" variant, i.e., with optimizations
> turned on. This also saves a *ton* of disk space. Also, testing with
> shared libraries rather than dynamic saves a lot of disk space.

Boost.Test now supports testing with shared libs on all platforms. I can't 
say about other libs. As for "release variant" testing, I don't see what 
stoppes each developer from adding appropriate test modules to the test 
suite.

Gennadiy