Re: [boost] 1.34 Release Plan - an alternative view

From: Robert Ramey (ramey_at_[hidden])
Date: 2006-01-26 13:28:19

David Abrahams wrote:
> "Robert Ramey" <ramey_at_[hidden]> writes:
>
>> I understand that as boost gets bigger it gets harder to make
>> new releases.  There is more to do, more to keep in sync,
>> and more ripple effect in changes.  I see this more structured
>> and formal release procedure as an attempt to deal with this.
>> It seems to aim for a closer coordination of developement
>> and release efforts to avoid difficulties associated with
>> development of software libraries which have varying
>> degrees of coupling.  I sympathize with with the effort.
>>
>> I would summarize the general idea as getting developers
>> more "in sync" through different stages of development
>> and release.  I see this as getting more difficult as boost
>> gets bigger and more diverse.  The development of different
>> libraries can't really be "scheduled".
>>
>> So I think this is the wrong approach and I think it will
>> make things harder rather than easier.
>>
>> I would propose a different way of going about this.  Here
>> it is.
>>
>> For the moment, exclude those packages related to
>> building and testing from consideration - I'll address
>> those at the end.
>>
>> a) developers build and test their libraries on their local machine
>> against the latest release.
>>
>> b) when they think they are ready, they open a branch on the
>> main trunk and check in their "next release".  So the cvs
>> system will have a branch for every developers "next release"
>
> I'm not opposed to the general direction you're going in (in fact I'll
> cautiously say I like it ;->), but there absolutely needs to be room
> in the plan for developers to use the CVS/SVN repository for checkins
> during the development of their "next release."

Hmmm - I thought that's what I said.

> Making large changes
> on a local machine without small intermediate checkins is a recipe for
> disaster.  You may be comfortable with it, but it's generally not
> considered to be good practice and I wouldn't want to consign the
> other Boost developers to working that way.  It has to be possible to
> make small changes, test, and check in repeatedly.

At what point a developer checks in his changes to the branch is
up to him.  The important thing is that they be checked into a branch
separate from the main trunk.

> One way to do it is to create a branch for the next release of a
> library, then create a branch of that branch for any intermediate
> checkins that are not intended to result in a "releaseable state."
> But then, what is the trunk for?  Instead of using a branch and a
> meta-branch, why not the trunk and a branch?

I think you're agreeing with me here - but its hard to tell.

>> c) when checked in - the developer will queue up a test request
>> on the new branch.
>
> What happens with that test request?  You don't expect the testers to
> test each branch separately against the last stable release of the
> rest of Boost, do you?

Ahhh - here is the key.  If fact that's exacely what I expect and
I think it will reduce resources required for testing considerably.
Let me explain.

When a test is queued only the tests for THAT library need be
run at this point.  In boost terms that means the Jamfile in the
test directory for THAT library is run.  And its only run when
the developer queues up a request.  The test is run on the branch
for library X - inheriting from the trunk the last "stable" release
for modules not in library X.

> This would be a lot simpler if developers would just keep the
> library's "next releaseable state" on the trunk.  Of course, that's
> what I try to do with my libraries, and I think most of the other
> developers do, too.  The only reasons the trunk has seemed to get
> destabilized in the past is that some people seemed to be allergic to
> using branches (so they'd do intermediate development on the trunk),
> and some others did not view failures on the trunk as something to be
> avoided (see "test-driven development").

Sounds like you're agreeing with me again - but I again - I can't be sure.

> Having thought about the way you're proposing to use branches, I don't
> see much difference from the situation we have today.

Configuration of cvs/svn wouldn't change at all. I don't think its currently
customary to create a branch for each libraries next group of changes.

> I certainly
> don't see any difference in the (dis)incentives to "rush features to
> make the next release."  IMO the greatest thing we can do to make that
> less of an issue is to shorten the release cycle, so that if you miss
> this release you know there will be another a couple of months.

My proposal would shorten the release cycle to every time there is a
a significant upgrade to any library. It wouldn't be driven by a schedule
based on any target date.  When a developer is "done" - that is - when
all tests are passes - the release is tagged.  Occasionally, some screwup
will slip through - then that release is just retagged as a "turkey" and
not distributed. Hopefully that won't occur any more frequently than
it already does..

>> b) The latest release of Boost would more up to date.  That is, one
>> wouldn't be in the situation of needing the next ehancemet, knowing
>> its already in there but not being able to use it because the next
>> release isn't ready yet.
>
> I don't see how your suggestion would make any difference in this
> respect either.  The fact that, under your plan, a library author
> doesn't put his work *anywhere* in the repository until he's sure it's
> ready for release isn't going to keep the release any closer to the
> latest state of development.  It just keeps the latest state of
> development on private machines.  Of course, Boost is a community, so
> this state will migrate to wikis, the vault, and personal websites as
> library authors collaborate and try to get users to exercise some of
> their new ideas.  If you check your latest releaseable state into the
> trunk then at least it gets tested and you can find out what the
> problems are.

Perhaps my mention of private machines was misleading. The thrust
of the proposal is using a separate branch for development of each library
and periodically mergeing them.  From this standpoint, whether its
on a private machine or  a branch isn't relevent.  It has to be checked
in (to its own branch) before it can be tested on all platforms.  The
point at which a developer decides to do this is in hands - just as
it is now.

> Again, the best thing we can do to make sure that the latest release
> of Boost is "more up to date" with the latest enhancements from
> library authors is to shorten the release cycle.

This presupposes a release cycle defined by target dates. I'm proposing
a whole different view point.  Release every time the library is
significantly enhanced and retested.  This would mean that very little
time would pass ( a couple of days?) between the time that a branch
is merged and all tests pass, and the time that it is available to users.

>>  c) Testing resources would be used much more efficiently.  Basically
>> more testing would be focused on areas that are likely to have
>> problems and less testing is focused on things that "almost for
>> sure" should pass. Testing would be done only where and when changes
>> have been made.
>
> How so?  What about this plan would cause this effect?  I don't see
> it.

a) tests would only be run when requested
b) only on libraries which have made changes

>> Basically, I don't believe the current Boost developement model is
>> scalable and I think the procedure has to change to recognise this.

> That has been recognized, and we are changing it.  Boost is getting to
> be a big ship; changing its direction has significant cost, so there
> needs to be an obvious and plausible cause-and-effect relationship
> between any changes to procedure and the results we want to achieve.

Agreed.

>but I don't see any way in which the
> specific suggestions can improve things.

OK

>> So in my view the current proposal goes in exactly the wrong
>> direction.
>
> What proposal?
>
> Thomas is the release manager for 1.34.  What he posted was his
> release plan and schedule; it's his perogative.

Not arguement here - I'm just making a suggestion.

I see his release plan and schedule as an attempt to address the 
difficulties
of the past by making the schedule more detailed and carefully controlled.
It is my opinion that this does not address the source of the difficulties
and if fact will only make the process even more difficult as more
"asyncrounous events" from more developers appear.  We'll see.

> The release procedure takes a sensible approach from a Boost-wide
> point-of-view, by which I mean that it is specified at a level that
> can be managed by a release manager -- and it is the only approach
> that I know of that's been shown to work for large project releases
> with many independent but related development efforts.

> Before
> choosing a different one, I would want to know that it had been shown
> to work in large projects like Boost.

I don't think the changes I propose are as radical as they seem. develop
on branch, test on branch, merge to trunk, retest, release.

>> Note that this is starting to occur by necesity.  Multi-index has a
>> "beta" version compatible with 1.33 that one can download.

>> I've been
>> testing changes to serialization on my machine against 1.33.  I
>> haven't checked them into the HEAD.  So now I know what problems are
>> mine and what problems are associated with changes in compiler
>> versions, stlport versions etc.  My next step is to make a few more
>> changes, run some more tests locally (basically later version of
>> stlport) and upload a package similar to Joaquin's.  This would make
>> the  changes (mostly bug fixes and documentation upates) available
>> to those who need them now and also give those who want to help
>> me out a way to test my changes without waiting for the next release
>> when it will be too late to fix anything.

> You could have done this much more reliably (and, IMO, more easily) by
> doing the development on a branch off latest release state and merging
> each one into the head as you become satisfied with it.

Whether its on cvs branch or code on a private machine isn't really the
issue.  I could check it into a separate branch but it wouldn't change
anything here.  The issue is testing against the last stable release branch.
If I check it into a branch - it won't get tested on other platforms - so
there isn't much benefit to checking it in until I've got it all tested 
here.

To summarize, my proposal differs from current approach

a) a branch is open for the next release for each library
b) tests are against lastest release platform
c) tests are run on specific libraries when and only when requrested
d) releases occur after every significant enhancement.
They are not tied to any specific target date.

There is only one problem that I've not really addressed here.  Its a
big problem and its fundemental to boost and other large systems.  Were
it not for this I don't think the above would be contraversial.  Here it is.

In the course of development, developers sometimes change API either
on purpose or inadvertenly.  This breaks code which depends upon
the library.  For some "top level" libraries (e.g. serialization) this isn't 
a
big problem as hardly anything else in boost depends upon it.  For "low 
level"
(e.g. mpl, preprocessor, etc) the rest of boost constitutes the
"defacto test suite".  Errors in one library show up as test failures
in dependent libraries.  So after merging - the current test setup has to be 
run again
and results for ALL the boost libraries need to be checked by the
developer.  Currently this is impractical and the test results display
isn't set up for this.

(note this just happened to me today.  A change in config showed up as
a test failure in the serialization library.  I'm sure that the author of 
the
change in config never found out about this.)

In theory this would suggest that the low-level libraries need more tests.
In practice its always going to occur in varying degrees.  If it occurs a 
lot
it suggests that there is too much coupling between libraries or that
some libraries are changing "too fast" to be relied upon.

So as a practical matter, a few "complete boost library test" similar to the
current one would have to be performed before the latest merge
is tagged for release.  But this would be a lot less frequenly than its done 
now.

Robert Ramey