Re: [boost] Breaking existing libraries

Subject: Re: [boost] Breaking existing libraries
From: Dave Handley (dave_at_[hidden])
Date: 2008-11-21 08:15:46

Comments inlined:

Markus Werle writes:

<snip>
>
> But let us go back on the we-do-not-pay-for-it path:
> IMHO it is a severe error of your development process that you do not
> check your code with new versions of all compilers and all libraries on a
> *regular* basis. In management parlance: your risk management is 
> defective.
> It always is a good idea to use the svn repo of boost and the beta version
> of the next compiler just to see the breaking changes early enough, which 
> gives
> everyone plenty of time to react.

I'm going to have to very strongly disagree here.  Depending on the usage of 
boost, checking a new version is often not a trivial process, and the vast 
majority of companies have other priorities for their software developers. 
My last company worked in shrink-wrapped software, and we had a policy that 
we would tag the new versions of all libraries just after our major annual 
release to give plenty of bed-in time before our next major release.  In my 
current job though, using a new version of boost involves rebuilding an 
entire library stack, some of which is outside our direct control.  Also 
stability is highly important, much more so that working on the latest and 
greatest.  Therefore, we have a tendency to move to the latest version of 
boost when we actually need a new feature.  As such we've skipped 1.30, 
1.31, 1.32, 1.34, 1.35, 1.36.

>
> Version 1.33.1 is from the stone age (remember boost is moving faster than
> light[ning]) and I always expect things to be broken the next time (which 
> is
> not nice, but compared to the problems solved by boost this still can be
> neglected).

This is actually a flaw with the current release process within boost.  Ever 
since boost moved to the accelerated release path that it is now on I have 
been worried that things like this would happen.  As far as I am aware (and 
I'm willing to be corrected here) the accelerated release process that 
probably started around 1.35 is primarily to get new libraries out into the 
wild faster.  In the process though, core libraries that have been around 
for years, or on which everything else is built, can become more unstable. 
What I would personally like to see is boost split into 2 parts, something 
like boost core, and boost general.  Boost core would contain all the stable 
libraries and the core libraries that other people build libraries off. 
Shared_ptr, bind, lambda, iterator, operators, range, ublas, etc.  Boost 
general would contain all the new libraries, and anything that is still very 
actively being developed.  The idea would be that if you download the latest 
version of both libraries, you could unpack them to the same area and you 
would get the equivalent of current boost (even down to identical directory 
structure); but the crucial difference would be the release process for each 
library.  Boost general would have the accelerated release process that 
boost currently has.  2 week beta trials, and new releases multiple times 
each year.  Boost core on the other hand would have much longer beta 
periods, with lots of users involved as Joaquin recommends later in this 
thread.  Beta periods could easily last 6 months for major new releases, and 
major releases could be restricted to being no more than annual.  Obviously 
dot releases are available for urgent bug-fixes.  You could even make the 
process relatively transparent by providing an already merged "boost 
complete" that took the latest core and general libraries.  The key thing 
though, core libraries would be way more stable.  In time, new 
infrastructure type libraries would get moved from general to core once they 
had stabilised.

I suspect that if you spoke to a lot of corporate users of boost they would 
really like a change of this type.  I remember from the time I supported 
boost for a very large company that there was always competing pressures for 
upgrading versions of boost.  On the one hand you always had developers 
shouting for the latest cool library; on the other hand, knowing that your 
entire internal library stack would need to be rebuilt, and that users would 
have to go through lengthy testing cycles, usually provided a strong 
pressure not to upgrade.  As such upgrades would usually come in every few 
years, and at least 2 different versions of boost are always in use at any 
time - with the application interoperability issues that you get with using 
2 different library stacks simultaneously.

I really like boost, and have used it consistently since the very early 
2000s.  Historically, it has been pretty good at avoiding major interface 
issues (with a few notable exceptions); but the rate at which it is growing 
now means that more controls are necessary IMHO to prevent major interface 
issues in the future.  If the community doesn't address this, my biggest 
fear is that our big corporate users will end up freezing on old versions, 
and a lot of the new and cool libraries that are coming out in new releases 
won't get the audience they deserve.

I've got one final point to make here.  I went through and reviewed the code 
changes in boost.range that Tom is referring to.  I wouldn't be happy 
putting this type of code change through in my own libraries.  Changing the 
functionality of a default constructed object just to get rid of a single 
test against bool in a couple of functions (ostensibly for performance) 
seems to me misguided.  Furthermore, having different functionality in debug 
and release, for most major users of a library, is just dangerous.  The size 
and empty functions for example assert in debug, but just silent fail in 
release.  The issingular function is even worse, it tells you whether a 
range is singular in debug, but always returns false in release.  Don't get 
me wrong, I think the boost.range library is an excellent library, but I 
just think that the implications of the changes in 1.35.0 weren't properly 
considered.

Dave