Re: [boost] The problems with Boost development

Subject: Re: [boost] The problems with Boost development
From: Andrey Semashev (andrey.semashev_at_[hidden])
Date: 2010-03-20 16:04:10

On 03/20/2010 02:10 AM, Andrey Semashev wrote:
>
> First, here are my top 3:
>
> 1. The review procedure is failing to deliver new libraries to the users
> in a reasonable time frame. Some very important libraries stay in the
> queue for too long without even having a review manager assigned.
>
> 2. The lack of maintenance releases. In production environment it is
> often a rule of thumb that the first release is unstable, and the second
> (third?) security update is suitable for use. Not having such updates at
> all leaves Boost in a bad situation.
>
> 3. Monolithic design limits development and adoption of Boost. A more
> modular approach is needed.

2. The lack of maintenance releases.
====================================

Personally, this part gave me the most frustration as a user. It was 
really hard to convince my coworkers to make an upgrade to a new Boost 
release recently, and the main point of argument was the potential 
instability this upgrade could bring. This is not limited to API changes 
(which usually don't give much trouble, if they appear), but is about 
the potential bugs and performance degradation (and these are the real 
problems that took most of the porting time). I think, I'm not alone in 
this regard.

I understand that Boost is about pulling the C++ development forward. 
But the users should not be forgotten in the process. There are a lot of 
tickets hanging in the Trac, some are there for years, some have patches 
and test cases. There are attempts to reduce their number during the 
"bug fixing runs", but that doesn't radically change the situation.

I think, the release scheme should be changed slightly so that a feature 
release is always followed with at least one bugfix release. The release 
schedule may stay the same, the only thing that changes is that every 
odd release is focused on fixing tickets. No new libraries, no new 
features, no major rewrites. That policy could be maintained by the 
release manager, and it doesn't require additional testing resources.

Another thing that would improve the Boost support is an easy way to 
see, which critical problems have been identified with a particular 
Boost release, and possible solutions (with patches, if present). This 
information should stay available for all Boost releases that are 
available for download (e.g., on the release notes page). AFAIK, Trac 
fails to deliver such information currently.

The decision on whether a particular problem is critical enough to be 
added to this list, and backporting the fix into previous Boost releases 
should be after the library maintainer. But there should be a rule that 
at the next maintenance release all (or at least, most) such critical 
problems from the previous release are fixed. When the maintainer has no 
time to do it, the release manager and the maintainer can grant 
authority to do that to other developers. The terms on which this help 
is accepted can be established individually (e.g. only commit after a 
review, or commit without any review).

There will be a problem with the libraries that are currently 
unmaintained. This is a very disturbing problem, indeed. I can't suggest 
a solution off hand, but at least it would seem reasonable to mark such 
libraries in the release notes or documentation, so that users would be 
aware that if problems arise with them, they should not expect a quick 
response. There could also be a call for maintainers for those libraries 
somewhere on the web site. The donation system I mentioned in my 
previous email could also build more interest for one to become a 
maintainer.

3. Monolithic design.
=====================

This matter has already been discussed, and others have made the 
suggestions, so I'll just express my thoughts of improving things.

First, I agree with those proposing to separate the core libraries from 
the more specialized and less stable ones. At first the core can be 
bundled in a single package, but later, if needed, it could be divided 
further. The main point is that the core does not depend on any other 
Boost libraries and includes the components that are most commonly used 
and are very stable.

The other libraries that depend on the core are bundled separately. They 
can have their own versioning and the release cycle. Dependencies on 
other libraries, including, but not limited to the core, are allowed, 
but it should be stated explicitly in the docs and the release notes, 
which versions of the dependent libraries are required. Ideally, it 
would be good to have a web service of some kind, where users could 
select the desired libraries, and the system would suggest to also 
download the dependent libraries of the appropriate versions.

At some period a complete Boost release should be prepared. It may not 
happen as often as it happens now, and it may be aligned with whatever 
time frame is best for the packagers. This complete release should 
include the latest compatible releases of the libraries, and be bundled 
in a single package.

The web site will not require dramatic changes except for adding 
convenience shortcuts for the non-core libraries, such as lib.boost.org 
or www.boost.org/lib.

Second, the entering procedure for a library under the Boost umbrella 
should be simpler than for becoming a core library. There is a simple 
reason for that: the new library is less stable, it has less users and 
may still contain rough edges, and this is normal. The library should 
show that it offers a good potential for improving, while offering good 
functionality in order to be useful. But it doesn't have to be perfect. 
This may sound like a lesser level of requirement than what Boost 
currently has, but I think it's adequate.

On the other hand, core libraries are the beacon that all other 
developers aim at. The requirements to enter that set of libraries can 
be much higher, and also include a certain period of library use in the 
real world to prove its usefulness.

I admit that the more I think of this part, the more it looks connected 
to the review system. I even think that dividing libraries into several 
layers (e.g. gold, silver and bronze), with each layer having the 
different requirements for entering, could help both the development and 
the users. It would be easier for the developers and review managers to 
bring new libraries into the bronze layer, while the users will know 
which libraries are the most stable and polished ones (specifically, the 
ones from the gold layer).

Last, but not the least, I'd like to admit that I purposefully did not 
touch the instrumental side of the issue. I hope that Dave's proposal 
will address that, or at least will make a step towards to it.