Boost logo

Boost :

Subject: Re: [boost] RE process (prospective from a retired FreeBSD committer)...
From: Dave Abrahams (dave_at_[hidden])
Date: 2011-01-30 17:16:37

At Sun, 30 Jan 2011 18:06:28 +0300,
Vladimir Prus wrote:
> Dave Abrahams wrote:
> > At Sun, 30 Jan 2011 16:43:17 +0300,
> > Vladimir Prus wrote:
> >>
> >> Maybe, stricter published guidelines, together with automatic
> >> integration testing and nagging and reverting, is what we need here.
> >>
> >> > Secondly, I don't see having "a single trunk" with these
> >> > properties as bringing with it any particular "goodness." If
> >> > you're looking for a fix you still need to know which code to look
> >> > at. So just go to that library's GitHub repo.
> >>
> >> What if you don't know which library is that?
> >
> > A big monolithic repo has the same problem.
> Not really -- you need a single checkout operation to find whether
> the problem is fixed or not with 99% probability. And only then, to
> investigate further.

You mean, via a "try it and see" approach?

Making it a single user operation to update to the latest released
version of any or all libraries is under my proposal so trivial as to
be unworthy of much discussion IMO.

> >> > But in reality, it doesn't tell you anything about the next
> >> > release of Boost, because things don't always get moved from trunk
> >> > to release.
> >>
> >> That's the consequence of our non-standard release process. All
> >> other software projects create release branches off trunk, and Boost
> >> is the only one I know who has long-lived release branch. I though
> >> it was a conscious decision, but all the recent discussions makes me
> >> doubt that, so maybe we should briefly discuss relative merits.
> >>
> >> 1. The 'make release branch off trunk' process is good for all
> >> purposes, really, except it requires that at some point in time,
> >> trunk is in good shape. Historically, for lack of tools and release
> >> management resources, we often could not get trunk in good shape
> >> quick.
> >
> > And that requires far too much coordination to be practical on a
> > project of this size.
> I don't think this statement has been proven.

Nothing in this discussion is going to be "proven."

> >> 2. Ours 'merge to release branch' process has the advantage that
> >> even if somebody breaks library X in trunk completely, he might
> >> just forget to merge it to release. So, we get improved stability,
> >> at the cost of slow progress. It's often the case that somebody
> >> forgets to merge changes to release branch, especially for
> >> patches and fixes applied outside of one's official libraries.
> >>
> >> So, in comparison, to achieve the same rate of changes and quality,
> >> (1) requires discipline with commits and checking test results,
> >> while (2) requires the same, and additional dancing and coordination
> >> with merges. So, (2) is strictly more complex and error prone, and
> >> should only be done if specific maintainers want separate branches
> >> for their components.
> >
> > Anyone can make a separate branch at any time no matter which
> > procedure is used.
> And? (2) still requires more effort.

No argument. I am just saying that the one condition under which you
say 2 "should only be done" isn't even really valid.

> >> >> What you propose breaks this useful thing, because:
> >> >>
> >> >> - Only release managers "pull" into the unified thing
> >> >
> >> > Not really. The individual library developers determined which
> >> > versions will be automatically pulled into the unified thing by
> >> > tagging their individual releases as STABLE.
> >>
> >> There are two problems here:
> >>
> >> 1. Just like library developers forget to merge to release, they
> >> will forget to tag those 'releases'
> >
> > That's totally up to the library developer. Library developers
> > generally care Boost contains an up-to-date version of their
> > libraries, so they will be motivated to remember.
> "will"? In practice, right now, developers do forget.

This line of argument eventually devolves into "library maintainers
will forget to check their changes in at all, so they should work on a
network share that we snapshot for release." I think marking a change
as ready for release should be a fairly conscious decision, and one
that's separate from saying "let's test this code out." As long as
that's the case, someone can forget. IMO the trade-off is well worth

> And again, there are often commits made by those who are not
> official maintaners of library X (because X might not even have
> maintainers). The chances of such changes of falling through are
> even higher.

I don't see how. Right now someone has to explicitly decide which
changes are moved to the release branch, and something equivalent
will happen

> >> 2. Because 'git cherry-pick' is fundamentally unable to record
> >> any mergeinfo, this means that any time you want to merge a
> >> single specific fix to release branch, you would have problems.
> >
> > What release branch? Who said anything about cherry-pick? What
> > problems?
> Suppose you have library X with 200 new changes. For next release, it is
> necessary to include one of those changes.

You cherry-pick that commit onto the branch in X's repo that currently
points at X's last release.

> How do you do that? With current release process, it's a single
> command (obviously, run by a single person).
> > And regardless, SVN has all the same issues w.r.t. picking
> > individual changes, doesn't it?
> No. When you merge a single revision from branch A to branch B, SVN
> records accurate information about this merge. When you do 'git
> cherry-pick', git does not record any information whatsoever and is
> fundementally unable to record it.

Ah. I was just complaining to a friend about the fact that Git drops
that information. Mostly I don't miss it for any practical purpose,
but conceptually I'd like it to be there.

> >> >> - Release managers only pull "releases",
> >> >
> >> > Which can be much more frequent than what we do at Boost.
> >>
> >> Oh. I think everybody thinks that our current rate of releases is
> >> almost-too-often already.
> >
> > Surely you realize that the release rate for
> > individual libraries does not have to affect the release rate for
> > Boost?
> First, in "release managers only pull releases", above, we are
> surely talking about Boost release managers?

Yes. We are talking about them "pulling" the releases of individual
libraries, not of Boost.

> Second, could you name, exactly, all libraries whose developers
> expressed a need for individual releases?

No, of course I can't name all of them exactly. Can you? I doubt it.

> I know of one.

I know I have expressed that need, and I've heard others express it.

> >> >> which, together with delays added by release managers, means that
> >> >> a fix to a component will be added to the unified thing after
> >> >> considerable delay
> >> >
> >> > I don't see where the delay comes from.
> >>
> >> Either release managers look at the thing they merge (which adds
> >> considerable delay), or they don't, in which case direct push to
> >> release branch is more efficient.
> >
> > They don't look, unless there are test failures or other problems.
> > And they don't merge. At most we're talking about updating submodule
> > references.
> Which still has to be done manually?

No. From above: "The individual library developers determined which
versions will be automatically pulled into the unified thing" by which
I mean there's a "button" release managers can push to assemble a
collection of the latest release versions of all the libraries in the

> >> Strangely, I know of other examples. Say, KDE, which is zillion
> >> times bigger than Boost, has relatively small number of separate git
> >> repositories,
> >
> > They're trying to fix that, IIUC.
> Oh? It does not seem like any split of kdelibs is immediately
> forthcoming, or even discussed with such activity as we're having
> here.

IIUC there is strong interest within at least part of the KDE
community in following a Ryppl-like modularized approach. They have
been looking into various kinds of package management infrastructure
for this job. I'll be happy to try to find out more if you want.

> Maybe, that suggest that we're actually solving wrong problem?

Exactly the opposite. KDE's moving in that direction IIUC; FreeBSD is
moving in that direction. I'm confident we could quickly find 10
other projects that are trying to modularize and zero modularized
projects that are trying to switch to a monolithic organization. When
have you ever heard of that happening?

> > The problem isn't where the code is located, it's:
> >
> > a. how much coordination is required to get things done
> Yes. And given that right now, coordination basically boils down to
> library authors merging their things before deadline (or not merging),
> and given that merge is a single command,

svn switch, svn merge, svn switch == a single command? But anyway,
that's irrelevant because...

> it's not clear to me how you are improving on that.'s not about the number of commands, it's about the sense of
speed and freedom with which library developers can operate.

> > b. the mental drag of having to deal with all the parts of Boost
> > that simply aren't relevant to you.
> What do you mean by mental drag? Having extra directories in your
> checkout?

Yes, among other things. For me personally, Boost's monolithic
structure takes much of the fun out of working on it. Totally
subjective and un-provable, I know.

so-sue-me-ly y'rs,

Dave Abrahams
BoostPro Computing

Boost list run by bdawes at, gregod at, cpdaniel at, john at