Re: [boost] 1.34 Release Plan - an alternative view

From: David Abrahams (dave_at_[hidden])
Date: 2006-01-26 14:57:38

"Robert Ramey" <ramey_at_[hidden]> writes:

> David Abrahams wrote:
>> "Robert Ramey" <ramey_at_[hidden]> writes:
>>
>
>> One way to do it is to create a branch for the next release of a
>> library, then create a branch of that branch for any intermediate
>> checkins that are not intended to result in a "releaseable state."
>> But then, what is the trunk for?  Instead of using a branch and a
>> meta-branch, why not the trunk and a branch?
>
> I think you're agreeing with me here - but its hard to tell.

I didn't think I was agreeing with you, but maybe I misunderstood you.

>>> c) when checked in - the developer will queue up a test request
>>> on the new branch.
>>
>> What happens with that test request?  You don't expect the testers to
>> test each branch separately against the last stable release of the
>> rest of Boost, do you?
>
> Ahhh - here is the key.  If fact that's exacely what I expect and
> I think it will reduce resources required for testing considerably.
> Let me explain.
>
> When a test is queued only the tests for THAT library need be
> run at this point.  In boost terms that means the Jamfile in the
> test directory for THAT library is run.  And its only run when
> the developer queues up a request.  The test is run on the branch
> for library X - inheriting from the trunk the last "stable" release
> for modules not in library X.

That would be great to have.  That said, it can only reduce the amount
of testing if we stop running regular tests against the trunk.  I
don't think that would be wise.

Also there's the issue that some authors may develop features that
depend on unreleased features of other libraries that are slated for
the next release.  This gets a little complicated. 

>> This would be a lot simpler if developers would just keep the
>> library's "next releaseable state" on the trunk.  Of course, that's
>> what I try to do with my libraries, and I think most of the other
>> developers do, too.  The only reasons the trunk has seemed to get
>> destabilized in the past is that some people seemed to be allergic to
>> using branches (so they'd do intermediate development on the trunk),
>> and some others did not view failures on the trunk as something to be
>> avoided (see "test-driven development").
>
> Sounds like you're agreeing with me again - but I again - I can't be sure.

Me neither.  I thought you were suggesting that developers only ever
check in new code on a branch when it was in a "next release" state,
but it sounds like I was wrong about that.

>> Having thought about the way you're proposing to use branches, I don't
>> see much difference from the situation we have today.
>
> Configuration of cvs/svn wouldn't change at all. I don't think its currently
> customary to create a branch for each libraries next group of changes.

I think that's because many developers find branching to be
burdensome.  I don't do it when my changes are small and can be tested
thoroughly on my local machine, but I probably should, because it
would give me a chance to test on a few platforms before committing to
the trunk.

>> I certainly
>> don't see any difference in the (dis)incentives to "rush features to
>> make the next release."  IMO the greatest thing we can do to make that
>> less of an issue is to shorten the release cycle, so that if you miss
>> this release you know there will be another a couple of months.
>
> My proposal would shorten the release cycle to every time there is a
> a significant upgrade to any library. 

...whoa, no way.  That would put release managers completely at the
mercy of the library authors.

> It wouldn't be driven by a schedule
> based on any target date.  When a developer is "done" - that is - when
> all tests

Which tests?  Of the trunk?

> are passes - the release 

Of his particular library?

> is tagged.  

And then what?

> Occasionally, some screwup
> will slip through - then that release is just retagged as a "turkey" and
> not distributed. Hopefully that won't occur any more frequently than
> it already does..
>
>>> b) The latest release of Boost would more up to date.  That is, one
>>> wouldn't be in the situation of needing the next ehancemet, knowing
>>> its already in there but not being able to use it because the next
>>> release isn't ready yet.
>>
>> I don't see how your suggestion would make any difference in this
>> respect either.  The fact that, under your plan, a library author
>> doesn't put his work *anywhere* in the repository until he's sure it's
>> ready for release isn't going to keep the release any closer to the
>> latest state of development.  It just keeps the latest state of
>> development on private machines.  Of course, Boost is a community, so
>> this state will migrate to wikis, the vault, and personal websites as
>> library authors collaborate and try to get users to exercise some of
>> their new ideas.  If you check your latest releaseable state into the
>> trunk then at least it gets tested and you can find out what the
>> problems are.
>
> Perhaps my mention of private machines was misleading. 

Perhaps.

> The thrust of the proposal is using a separate branch for
> development of each library and periodically mergeing them.  

Generally a good idea; that's what we did with the parameter library
recently, for example.  I don't know that it's a good idea to do
everything on a branch, though.  If I find a workaround for a Borland
bug doesn't it make sense to check it into the trunk right away?  I
think it would be a waste of resources to add a branch testing request
for something like that.
 
> From this standpoint, whether its on a private machine or a branch
> isn't relevent.  It has to be checked in (to its own branch) before
> it can be tested on all platforms.  The point at which a developer
> decides to do this is in hands - just as it is now.
>
>> Again, the best thing we can do to make sure that the latest release
>> of Boost is "more up to date" with the latest enhancements from
>> library authors is to shorten the release cycle.
>
> This presupposes a release cycle defined by target dates. 

??  A release cycle is the time between release dates, by definition.

> I'm proposing
> a whole different view point.  Release every time the library is
> significantly enhanced and retested.  This would mean that very little
> time would pass ( a couple of days?) between the time that a branch
> is merged and all tests pass, and the time that it is available to users.

Maybe you should try managing a few releases before you suggest that.
There is currently a great deal of administrative overhead for the
release manager
(http://www.boost.org/more/release_mgr_checklist.html).  If something
could be done to reduce that overhead to almost nothing, it would be
great, but until then I don't think we can afford to do it.

>>>  c) Testing resources would be used much more efficiently.  Basically
>>> more testing would be focused on areas that are likely to have
>>> problems and less testing is focused on things that "almost for
>>> sure" should pass. Testing would be done only where and when changes
>>> have been made.
>>
>> How so?  What about this plan would cause this effect?  I don't see
>> it.
>
> a) tests would only be run when requested
> b) only on libraries which have made changes

What about the trunk?

>>> Basically, I don't believe the current Boost developement model is
>>> scalable and I think the procedure has to change to recognise this.
>
>> That has been recognized, and we are changing it.  Boost is getting to
>> be a big ship; changing its direction has significant cost, so there
>> needs to be an obvious and plausible cause-and-effect relationship
>> between any changes to procedure and the results we want to achieve.
>
> Agreed.
>
>>but I don't see any way in which the
>> specific suggestions can improve things.
>
> OK

Now that you've explained, I see potential in your ideas, but they're
not quite fully-baked yet.

> I see his release plan and schedule as an attempt to address the
> difficulties of the past by making the schedule more detailed and
> carefully controlled.  

It's not a break with the past.  He's replicating the release plan
used by Doug for the last two releases (and that plan is just a
refinement of what we did before).  It said as much in his posting.
So we're trying to achieve a little consistency.  Getting the process
to be more systematic and regular is crucial to reducing the pain,
IMO.

> It is my opinion that this does not address the source of the
> difficulties and if fact will only make the process even more
> difficult as more "asyncrounous events" from more developers appear.
> We'll see.

I don't see how it can make anything worse.  The past two releases
have been an improvement over those that came before.

>> The release procedure takes a sensible approach from a Boost-wide
>> point-of-view, by which I mean that it is specified at a level that
>> can be managed by a release manager -- and it is the only approach
>> that I know of that's been shown to work for large project releases
>> with many independent but related development efforts.
>
>> Before choosing a different one, I would want to know that it had
>> been shown to work in large projects like Boost.
>
> I don't think the changes I propose are as radical as they seem. develop
> on branch, test on branch, merge to trunk, retest, release.

You don't plan to have a point at which the trunk is frozen?  Maybe
once we switch to SVN that will be practical.  It certainly isn't
practical for CVS.

>> You could have done this much more reliably (and, IMO, more easily) by
>> doing the development on a branch off latest release state and merging
>> each one into the head as you become satisfied with it.
>
> Whether its on cvs branch or code on a private machine isn't really the
> issue.  I could check it into a separate branch but it wouldn't change
> anything here.  The issue is testing against the last stable release branch.
> If I check it into a branch - it won't get tested on other platforms

It can be so tested, by request.

> so there isn't much benefit to checking it in until I've got it all
> tested here.
>
> To summarize, my proposal differs from current approach
>
> a) a branch is open for the next release for each library
> b) tests are against lastest release platform

As mentioned, doesn't work when you depend on new changes in another
library.  You could, however, include that library's changes in your
branch.  That will be lots easier once we have SVN.

> c) tests are run on specific libraries when and only when requrested

Nice, but increases test load unless we drop trunk testing.

> d) releases occur after every significant enhancement.

Too much overhead.  Need (probably massive) infrastructure upgrade to
make that feasible.

> They are not tied to any specific target date.

Again, probably not feasible until we're using SVN.

> There is only one problem that I've not really addressed here.  Its
> a big problem and its fundemental to boost and other large systems.
> Were it not for this I don't think the above would be contraversial.
> Here it is.
>
> In the course of development, developers sometimes change API either
> on purpose or inadvertenly.  This breaks code which depends upon the
> library.  For some "top level" libraries (e.g. serialization) this
> isn't a big problem as hardly anything else in boost depends upon
> it.  For "low level" (e.g. mpl, preprocessor, etc) the rest of boost
> constitutes the "defacto test suite".  Errors in one library show up
> as test failures in dependent libraries.  So after merging - the
> current test setup has to be run again and results for ALL the boost
> libraries need to be checked by the developer.  Currently this is
> impractical and the test results display isn't set up for this.

?? It seems to me that we have exactly this today.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com