Re: [boost] 1.34 Release Plan - an alternative view

From: Robert Ramey (ramey_at_[hidden])
Date: 2006-01-26 16:54:57

David Abrahams wrote:

>>> Again, the best thing we can do to make sure that the latest release
>>> of Boost is "more up to date" with the latest enhancements from
>>> library authors is to shorten the release cycle.
>>
>> This presupposes a release cycle defined by target dates.
>
> ??  A release cycle is the time between release dates, by definition.

Hmm - this is a source of confusion then.  My view is that a release
is set of modules with some minimal level (zero ?) of bugs, quirks
or other anamolies that we are comfortable recommending that users
use.  Whether this "release" occurs on some prespecified target
date or when some new feature is added isn't relevant to my
usage of the word "release".  So the phrase "release date"
wouldn't come in my comments.  "Up to date" is different.  By
"Up to date" I mean that the "release" has all features that we
we have added in and feel comfortable encouraging users to use.
So my system is by definiition always "Up to Date"

>> I'm proposing
>> a whole different view point.  Release every time the library is
>> significantly enhanced and retested.  This would mean that very
>> little
>> time would pass ( a couple of days?) between the time that a branch
>> is merged and all tests pass, and the time that it is available to
>> users.
>
> Maybe you should try managing a few releases before you suggest that.
> There is currently a great deal of administrative overhead for the
> release manager
> (http://www.boost.org/more/release_mgr_checklist.html).  If something
> could be done to reduce that overhead to almost nothing, it would be
> great, but until then I don't think we can afford to do it.

I didn't know about this and just now gave it an cursory examination.
It is helpful in understanding what's involved.  However, there are a
couple of observations that occur to me.

a) It seems out of date - its dated 04 Feb 2004 - two years ago.
b) It seems to suggest that most of the "tools" are being moved
in the direction of C++ while in practice it seems that we're depending
more on other stuff like python, quickbook and xml, etc.
c) "Pre-release activities" is really what I'd like
to see replaced with an continuous maintaince of the trunk in
a "releasable" state.  Only occasional merge from branches to
trunk and associated tests on trunk of the whole library would
put the trunk in a temporarily non-releaseable state.
d) "CVS Branch for Release" I would like to see disappear.
e) "CVS Release" would stay the same.
f) "Distribution" would hopefully be more automated.  This looks
like a huge pain to do by hand.

> Now that you've explained, I see potential in your ideas, but they're
> not quite fully-baked yet.

I won't dispute that.  Its not been my intention to propose a detailed
replacement for the current procedure.  I really just want to point
out that the current difficulties are the result of the way we use
cvs and the way we do our testing. And that things won't get much
better until these things are re-examined.

>> I see his release plan and schedule as an attempt to address the
>> difficulties of the past by making the schedule more detailed and
>> carefully controlled.
>
> It's not a break with the past.  He's replicating the release plan
> used by Doug for the last two releases (and that plan is just a
> refinement of what we did before).  It said as much in his posting.
> So we're trying to achieve a little consistency.

I know its not a break with the past.  I see it as a refinement of the
traditional approach - that's why I think its on the wrong track.

> Getting the process
> to be more systematic and regular is crucial to reducing the pain,
> IMO.

>> It is my opinion that this does not address the source of the
>> difficulties and if fact will only make the process even more
>> difficult as more "asyncrounous events" from more developers appear.
>> We'll see.

> I don't see how it can make anything worse.  The past two releases
> have been an improvement over those that came before.

The "chaos" (substitute your own word here) is not in any way due
to the lack of systematic approach or to any lack of effort
on the part of the those who manage the release.  The "chaos"
occurs because of things outside the managers control.  New
bugs found at the last minute.  New bugs introduced at the last
minute.  New compilers dropped in etc.  etc.  The fact that
boost is getting bigger means there are more of these
asyncronous events and working harder will help only for
a little while longer.

>>> The release procedure takes a sensible approach from a Boost-wide
>>> point-of-view, by which I mean that it is specified at a level that
>>> can be managed by a release manager -- and it is the only approach
>>> that I know of that's been shown to work for large project releases
>>> with many independent but related development efforts.

>>> Before choosing a different one, I would want to know that it had
>>> been shown to work in large projects like Boost.

Well - there isn't anything quite like boost - that's why we're here.

I would guess the closest analogy that comes to mind is a large
website.  Every time a new "branch" is coded it is tested and
folded in without any formal "release".

>> I don't think the changes I propose are as radical as they seem.
>> develop
>> on branch, test on branch, merge to trunk, retest, release.

> You don't plan to have a point at which the trunk is frozen?  Maybe
> once we switch to SVN that will be practical.  It certainly isn't
> practical for CVS.

My plan is that the trunk would be maintained so as to be in one of
the following states.

a) "current release" with a tag saying the release number

    when a library is merged into the trunk move to state b

b) "newly merged branch - subject to test"

    when a test on the trunk fails move to b

c) "broken - one or more boost wide tests fail"

    when boost wide tests pass - the state moves from c to a.

I don't see anything in CVS which would prevent this. In fact it seems
to me that this is the way these systems are intended to be used.

>> To summarize, my proposal differs from current approach
>>
>> a) a branch is open for the next release for each library
>> b) tests are against lastest release platform
>
> As mentioned, doesn't work when you depend on new changes in another
> library.  You could, however, include that library's changes in your
> branch.  That will be lots easier once we have SVN.

In my scenario the problem doesn't come up.  my library X depends
on your library Y.  I'm on my branch and you are on yours.  So my
library X can't even be built until yours is checked into the trunk.
As soon as your library Y is merged to the trunk and trunk is
retested (ie "released") its now available.

Of course in some special case I might check out your branch
to my machine but that would be a pretty unusual case.  In any
case my library X wouldn't be merged into the trunk until your
library Y has passed adn the trunk moves back to "current release"
state.

If the libraries are being developed together then they should be
handled as one - ie merged into the trunk as a pair. But generally
I would expect this to be unusual.

By shortenting the time between when a library passes its tests and
is "released" (made available to users) much of these issues disappear.

>> c) tests are run on specific libraries when and only when requrested
>
> Nice, but increases test load unless we drop trunk testing.

trunk testing would still occur but only after merging of a new branch
and only while the trunk is in a "temporarily broken" state.  I would
anticipate much less consumption of resources than now.

>> d) releases occur after every significant enhancement.
>
> Too much overhead.  Need (probably massive) infrastructure upgrade to
> make that feasible.

If you're refering to the "Distribution" section of the release manager
checklist I would agree that a big upgrade would be required.

>> They are not tied to any specific target date.
>
> Again, probably not feasible until we're using SVN.

Again, I don't see anything in CVS that prevents is usage in this manner.
In fact, I believe that's the way it is intended to be used.

>> In the course of development, developers sometimes change API either
>> on purpose or inadvertenly.  This breaks code which depends upon the
>> library.  For some "top level" libraries (e.g. serialization) this
>> isn't a big problem as hardly anything else in boost depends upon
>> it.  For "low level" (e.g. mpl, preprocessor, etc) the rest of boost
>> constitutes the "defacto test suite".  Errors in one library show up
>> as test failures in dependent libraries.  So after merging - the
>> current test setup has to be run again and results for ALL the boost
>> libraries need to be checked by the developer.  Currently this is
>> impractical and the test results display isn't set up for this.
>
> ?? It seems to me that we have exactly this today.

Here is an example.  The other day a change was made in config.  As a
side effect of this change, a number of tests in the serialization library 
start
to fail.  The author other the original change looks at the test in the 
config section
and sees no problem - he isn't even aware of the failures that occur in
other libraries due to this change.  Meanwhile, I look at the test results 
and find
no changes in any code which could account for the failure.  So the person
that made the change know of no failures and the person who knows
about the failure knows nothing about the change.

Of course if I have been making changes to my own code simultaneaously
I'll presume it's due to one of my changes and I'll go crazy trying to find 
it.

So although this problem is addressed by testing the trunk its not a
perfect solution - Of course if the change to config had been made
on a branch and that branch tested, the problem in this example
wouldn't have come up.  So maybe its not the best example. In general
the problem will still occur but using the approach I've described
here would make the whole process much easier.

Robert Ramey