Re: [boost] RE process (prospective from a retired FreeBSD committer)...

Subject: Re: [boost] RE process (prospective from a retired FreeBSD committer)...
From: Vladimir Prus (vladimir_at_[hidden])
Date: 2011-01-30 08:43:17

Dave Abrahams wrote:

> At Sat, 29 Jan 2011 10:55:21 +0300,
> Vladimir Prus wrote:
>> 
>> Dave Abrahams wrote:
>> 
>> > However, my vision for Boost is a bit different:
>> > 
>> > * each library has its own Git repo
>> > 
>> > * each library maintainer tests and produces versioned releases of
>> > that library on his/her own timetable
>> > 
>> > * the latest release of a library is tagged "STABLE"
>> > 
>> > * When assembling a new Boost release, the release manager marks the
>> > "STABLE" revision from each library and begins boost-wide
>> > integration/interoperability testing on those revisions
>> 
>> I am rather sceptical about this approach. The goodness of what we have
>> now is that there's a single trunk that holds code that is:
>> 
>> - supposedly not worse in test results in all other code,
>> - is the newest one satisfying the above constraint
> 
> First of all, I don't think there's general agreement that trunk
> should hold code "supposedly not worse in test results."  That's one
> reason our trunk test results are not all that useful.  Some people
> like to use trunk for "TDD," which apparently means checking in tests
> that are known to be broken and fixing them incrementally.

That seems like violating basic principles of software development. Maybe,
stricter published guidelines, together with automatic integration testing
and nagging and reverting, is what we need here.

> Secondly, I don't see having "a single trunk" with these properties as
> bringing with it any particular "goodness."  If you're looking for a
> fix you still need to know which code to look at.  So just go to that
> library's GitHub repo.

What if you don't know which library is that? Say, if you use multiindex,
serialization autolink kicks in. As a developer who could not care less
about all this "Boost C++ Libraries" thing, you probably want to just
get in-development version to see if either library got fixed, before
looking at the code for real.
 
>> Therefore, if I wish to make sure my code works with new release of
>> Boost, of if I wish to check if some bug is fixed, that is the place
>> to go.
> 
> But in reality, it doesn't tell you anything about the next release of
> Boost, because things don't always get moved from trunk to release.

That's the consequence of our non-standard release process. All other
software projects create release branches off trunk, and Boost is the
only one I know who has long-lived release branch. I though it was
a conscious decision, but all the recent discussions makes me doubt
that, so maybe we should briefly discuss relative merits.

1. The 'make release branch off trunk' process is good for all
purposes, really, except it requires that at some point in time,
trunk is in good shape. Historically, for lack of tools and release
management resources, we often could not get trunk in good shape
quick.

2. Ours 'merge to release branch' process has the advantage that
even if somebody breaks library X in trunk completely, he might
just forget to merge it to release. So, we get improved stability,
at the cost of slow progress. It's often the case that somebody
forgets to merge changes to release branch, especially for 
patches and fixes applied outside of one's official libraries.

So, in comparison, to achieve the same rate of changes and quality,
(1) requires discipline with commits and checking test results, 
while (2) requires the same, and additional dancing and coordination
with merges. So, (2) is strictly more complex and error prone, and
should only be done if specific maintainers want separate branches
for their components.

>> Further, this place is jointly updated by every boost developer.
> 
> That's a problem right there, not an advantage.

Ok, we disagree here.

> 
>> What you propose breaks this useful thing, because:
>> 
>> - Only release managers "pull" into the unified thing
> 
> Not really.  The individual library developers determined which
> versions will be *automatically* pulled into the unified thing by
> tagging their individual releases as STABLE.

There are two problems here:

1. Just like library developers forget to merge to release, they will
forget to tag those 'releases'
2. Because 'git cherry-pick' is fundamentally unable to record any
mergeinfo, this means that any time you want to merge a single 
specific fix to release branch, you would have problems.

> 
>> - Release managers only pull "releases",
> 
> Which can be much more frequent than what we do at Boost.

Oh. I think everybody thinks that our current rate of releases is
almost-too-often already.

>> which, together with delays added by release managers, means that a
>> fix to a component will be added to the unified thing after
>> considerable delay
> 
> I don't see where the delay comes from.

Either release managers look at the thing they merge (which adds
considerable delay), or they don't, in which case direct push
to release branch is more efficient.

>> therefore the unified thing we have now will no longer exists, and
>> two weeks before the release we'll get a frankenstein creature that
>> might technically pass all tests (though I doubt that), but won't be
>> tested by any users in real projects.
> 
> Welcome to the real world.  That's how every OS distribution and every
> other collection of semi-independent modules works, and works well.
> No sane user of a real project is going to slow down his development
> by regularly testing against an unSTABLE Boost trunk, and that's
> increasingly true as Boost grows.

Strangely, I know of other examples. Say, KDE, which is zillion times
bigger than Boost, has relatively small number of separate git repositories,
whereas most of the core (which is half-zillion times bigger than Boost),
lives in two modules, which contain wildly different functionality, are
updated by multiple people, and then built and used and tested by users
all the time. So, this proves that Boost has a lot of grows ahead before
having all code in place place will becomes, even in theory, a problem.

- Volodya

-- 
Vladimir Prus
Mentor Graphics
[phone]