Re: [boost] RE process (prospective from a retired FreeBSD committer)...

Subject: Re: [boost] RE process (prospective from a retired FreeBSD committer)...
From: Dave Abrahams (dave_at_[hidden])
Date: 2011-01-30 09:19:54

At Sun, 30 Jan 2011 16:43:17 +0300,
Vladimir Prus wrote:
> 
> Dave Abrahams wrote:
> 
> > First of all, I don't think there's general agreement that trunk
> > should hold code "supposedly not worse in test results."  That's one
> > reason our trunk test results are not all that useful.  Some people
> > like to use trunk for "TDD," which apparently means checking in tests
> > that are known to be broken and fixing them incrementally.
> 
> That seems like violating basic principles of software development.

It depends whose "basic principles" you subscribe to, apparently.

> Maybe, stricter published guidelines, together with automatic
> integration testing and nagging and reverting, is what we need here.
> 
> > Secondly, I don't see having "a single trunk" with these
> > properties as bringing with it any particular "goodness."  If
> > you're looking for a fix you still need to know which code to look
> > at.  So just go to that library's GitHub repo.
> 
> What if you don't know which library is that? 

A big monolithic repo has the same problem.

> >> Therefore, if I wish to make sure my code works with new release
> >> of Boost, of if I wish to check if some bug is fixed, that is the
> >> place to go.
> > 
> > But in reality, it doesn't tell you anything about the next
> > release of Boost, because things don't always get moved from trunk
> > to release.
> 
> That's the consequence of our non-standard release process. All
> other software projects create release branches off trunk, and Boost
> is the only one I know who has long-lived release branch. I though
> it was a conscious decision, but all the recent discussions makes me
> doubt that, so maybe we should briefly discuss relative merits.
> 
> 1. The 'make release branch off trunk' process is good for all
> purposes, really, except it requires that at some point in time,
> trunk is in good shape. Historically, for lack of tools and release
> management resources, we often could not get trunk in good shape
> quick.

And that requires far too much coordination to be practical on a project
of this size.

> 2. Ours 'merge to release branch' process has the advantage that
> even if somebody breaks library X in trunk completely, he might
> just forget to merge it to release. So, we get improved stability,
> at the cost of slow progress. It's often the case that somebody
> forgets to merge changes to release branch, especially for 
> patches and fixes applied outside of one's official libraries.
> 
> So, in comparison, to achieve the same rate of changes and quality,
> (1) requires discipline with commits and checking test results, 
> while (2) requires the same, and additional dancing and coordination
> with merges. So, (2) is strictly more complex and error prone, and
> should only be done if specific maintainers want separate branches
> for their components.

Anyone can make a separate branch at any time no matter which
procedure is used.

> >> What you propose breaks this useful thing, because:
> >> 
> >> - Only release managers "pull" into the unified thing
> > 
> > Not really.  The individual library developers determined which
> > versions will be *automatically* pulled into the unified thing by
> > tagging their individual releases as STABLE.
> 
> There are two problems here:
> 
> 1. Just like library developers forget to merge to release, they will
> forget to tag those 'releases'

That's totally up to the library developer.  Library developers
generally care Boost contains an up-to-date version of their
libraries, so they will be motivated to remember.

> 2. Because 'git cherry-pick' is fundamentally unable to record any
> mergeinfo, this means that any time you want to merge a single 
> specific fix to release branch, you would have problems.

What release branch?  Who said anything about cherry-pick?  What
problems?  

And regardless, SVN has all the same issues w.r.t. picking individual
changes, doesn't it?

> >> - Release managers only pull "releases",
> > 
> > Which can be much more frequent than what we do at Boost.
> 
> Oh. I think everybody thinks that our current rate of releases is
> almost-too-often already.

Now it seems like you're just looking for a snappy refutation without
really thinking.  Surely you realize that the release rate for
individual libraries does not have to affect the release rate for
Boost?

> >> which, together with delays added by release managers, means that
> >> a fix to a component will be added to the unified thing after
> >> considerable delay
> > 
> > I don't see where the delay comes from.
> 
> Either release managers look at the thing they merge (which adds
> considerable delay), or they don't, in which case direct push to
> release branch is more efficient.

They don't look, unless there are test failures or other problems.
And they don't merge.  At most we're talking about updating submodule
references.

> >> therefore the unified thing we have now will no longer exists, and
> >> two weeks before the release we'll get a frankenstein creature that
> >> might technically pass all tests (though I doubt that), but won't be
> >> tested by any users in real projects.
> > 
> > Welcome to the real world.  That's how every OS distribution and every
> > other collection of semi-independent modules works, and works well.
> > No sane user of a real project is going to slow down his development
> > by regularly testing against an unSTABLE Boost trunk, and that's
> > increasingly true as Boost grows.
> 
> Strangely, I know of other examples. Say, KDE, which is zillion
> times bigger than Boost, has relatively small number of separate git
> repositories, 

They're trying to fix that, IIUC.

> whereas most of the core (which is half-zillion times bigger than
> Boost), lives in two modules, which contain wildly different
> functionality, are updated by multiple people, and then built and
> used and tested by users all the time. So, this proves that Boost
> has a lot of grows ahead before having all code in place place will
> becomes, even in theory, a problem.

The problem isn't where the code is located, it's:

a. how much coordination is required to get things done

b. the mental drag of having to deal with all the parts of Boost that
   simply aren't relevant to you.

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com