Re: [boost] Development and Release Practices

From: Beman Dawes (bdawes_at_[hidden])
Date: 2007-08-07 21:12:16

Peter Dimov wrote:
> Beman Dawes wrote:
>> A draft "Development and Release Practices" document is up on the
>> Wiki:
>> http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl?Development_And_Release_Practices
>>
>> Comments welcomed!
> 
> I think that the Trac wiki is a better place for this document.

OK. Time to learn yet another set of wiki markup rules.

Time passes...

Done. See http://svn.boost.org/trac/boost/wiki/ImprovingPractices

The links within the page don't work because I couldn't figure out how 
to do anchors like <a name="#foo">foo</a>. If anyone knows how, please 
let me know:-) The docs seem to keep it a secret.

> Weak points:
> 
> 1. Merging from trunk to release. This is critical for the success of the 
> whole scheme, but the details are still to be worked out. It's not clear 
> whether tools can help much since we're talking about merges across two 
> independent parallel branches without a close common ancestor, but I can be 
> wrong.

I'm also concerned about merging from trunk to release, and have been 
giving it more thought. It really isn't a merge; it is more like 
updating a "Release_Candidate" tag. In Subversion, it would be done via 
the svn copy command.

I've updated the wiki page to use the correct Subversion terminology.

Before we actually try to do such a copy, we can experiment with a test 
repository to make sure we've got the SVN command sequence right.

> 2. Testing core libraries on branches. Not clear how or who, or even whether 
> at all.

Testing will certainly be needed, but I'm not sure whether that will be 
on a regular or as-needed basis.

> 3. Trunk. Unclear what would prevent the current wild west syndrome, except 
> for the continuous testing; but if this is all that it takes, why didn't we 
> institute this policy earlier?

Nobody thought of it? Lack of clear policies?

> Some comments on core libraries:
> 
> "Note that the requirements for being release-ready are transitive; if a 
> library has dependent libraries, it isn't considered release-ready if it 
> breaks any of those dependent libraries."
> 
> This is sensible but it can prevent progress if these failures are caused by 
> the dependencies rather than the core library. Two arbitrary shared_ptr 
> examples: (1) the serialization library in 1.32 relying on implementation 
> details, (2) Spirit using the long deprecated make_shared spelling of 
> weak_ptr::lock.

Yes, a definite problem. It's similar if not the same as the "breaking 
change" problem.

I'm not sure we can work out solutions to all problems ahead of time. 
But as problems come up and get solved, we will need to update our 
procedures and development practices docs to deal with various situations.

> A failure downstream is caused by either (1) insufficient test coverage in 
> the core library, (2) a "bug" (broadly speaking) in the dependency. We still 
> don't have a proper policy to address (1). We don't ask people to contribute 
> tests, and we don't (in general) add tests when we fix bugs.

My sense is that lots of Boost developers feel strongly that we need to 
change, and are willing to endure a period of experimentation while we 
come up with better policies, better tools, and whatever else needs to 
be done to make the Boost development process more fun and less pain.

> Looking at the big picture, all Boost libraries are core, because the World 
> depends on them. The World however doesn't get to receive the preferential 
> treatment that non-core Boost libraries enjoy; it relies on bug reports. I'm 
> not saying that this is a good or bad thing, just a food for thought.

Yep.

> Finally, a philosophical point.
> 
> I'm slowly coming to the realization that the problem we're having is not 
> caused by lack of process or lack of tools. It's caused by the fact that 
> release management (at this point) is a full time job.

It's certainly a full-time job with the old processes. It might still 
be, even with new and better processes and tools.

There will be an announcement in a few days that Boost has formally (and 
legally) joined the Software Freedom Conservancy. That opens up funding 
options we haven't had in the past. It has seemed to me for a long time 
that when Boost gets big enough to pay someone, the first priority would 
be for a release manager.

Thanks,

--Beman