Boost logo

Boost :

Subject: Re: [boost] Release sponsorship (was: Re: Boost 1.58 schedule available?)
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2015-01-22 19:45:41


On 22 Jan 2015 at 9:58, Robert Ramey wrote:

> >> Certainly if having the newest integrated release isn't worth $1000 to
> >> any company on the whole planet, it's not worth a release manager's time
> >> to deal with all the hassle making and distributing a new release.
> >
> > You probably were being tongue in cheek Robert, but if not then this
> > might be the second time we are in complete agreement this week! Next
> > thing might be pigs flying!
>
> I'm very serious about addressing this particular suggestion and about
> evolving the boost revenue (or lack there of) model. I'm aware that
> this is a very difficult problem that won't be solved with just an idea.

Wow. We were in agreement then.

I always felt the Community Maintenance Team should have been
configured as a bounty based system. If someone needs an unmaintained
library to be maintained in some way, then they stump up a bounty for
it. Note by unmaintained I would cast the net to include libraries
with maintainers, though in that case the maintainer gets a veto. I
also think the maintainer should get first dibs to the bounty too,
they already have thankless jobs.

This could be seen as creating incentives to not maintain libraries.
I'm happy with that - software is expensive. I wouldn't go so far as
holding users to ransom for fixes on payment though.

> > Regarding releases, they are highly disruptive to the develop branch
> > as basically one has to down tools on feature work and spend an age
> > iterating fixing up master branch such that all the other master
> > branches in all the other submodules play well together on all
> > supported platforms.
> >
> > For quickly moving libraries such as Thread, I
> > can see four weeks per release being lost. Increasing release
> > frequency increases that cost substantially.
>
> If this is the case, then something is not right and needs to be
> addressed. It's hard for me to comment on your particular case without
> spending a lot of time looking at the details, but I can speak from
> the experience of maintaining the serialization library over many years
> through several different Source Control systems and several different
> workflows. My current view is that our issues with with these things
> are now largely solved and we are in a good position to take boost
> to the next level - whatever that might.

With respect Robert, Serialization has nothing like the amount of
code intricately dependent on it behaving exactly in some way,
including in ways it was broken in the past. Thread offers no less
than four versions of a threading API with often individual switches
for mashing up API version features. Some of those API versions are
intentionally broken to retain backwards compatibility with code
written to assume that brokeness. It gets even better: some users
want the old broken APIs to be C++ 11 aware, even C++ 11 capable. I
remember a bug last year where the wrong constructor in a legacy API
was being called under C++ 11 because the 11 rules are subtly
different from 03 due to rvalue refs being around.

One fun part is that parts of Boost were written for a different
Thread API version than other bits, so when you compile all of Boost
Thread has to be all things to all comers. This is a very brittle
situation to be in. Hence the costs of doing a release. Right now
Thread defaults to v3 of its API because v4 of its API, the current
one, breaks parts of Boost.

As I have often mentioned here, the ability to mark non-upgraded
libraries for official deprecation and removal would be an enormous
time saver for everybody else forced to work around them. It
shouldn't be the case that the maintainers of Thread must go fix
unmaintained Boost code nothing to do with Thread.

> I work on my own schedule. I work on my development branch and
> test there. When it's passing ALL tests, I merge into master. the
> master branch of the serialization library is always ready for release
> as far as I know. That it is a strict improvement over the previous
> version of the master branch and has no known regressions. The
> timing of the "official boost release" is irrelevant to me. I think that
> anyone who has to know the release date is not doing things the
> way I'm doing and that they should start doing it that way.

I do the same, but my codebase like yours was designed around making
maintenance easy for me. Thread is very old, some of it is quite
creaky, and almost all of it was designed around a Boost which hasn't
existed in a decade. Even its unit test suite reflects at least three
different testing paradigms. Historically the test coverage was very
poor, but this past year especially has seen enormous contributions
of additional tests. The problem more recently is we need to disable
quite a few of them simply to run the testing in a feasible timespan.
I recently looked into running a regular CI unit test run for Thread,
and I estimate it would take most of a day, enough to mean I couldn't
run it nightly.

For example recently Boost.Test made an update and broke a ton load
of Thread. It turns out that Thread hadn't been configuring Test
correctly for years, but the point remains that Thread is very
sensitive to changes anywhere else in Boost.

> > Hence I do wish there was a facility for sponsors to sponsor a merge
> > of develop to master. If this had been the case for Boost.Test, would
> > its develop branch have remained unmerged for so many years? No, I
> > think someone would have done the merge if $5000 were on the table
> > for it.
>
> Again, if merging from develop to release is a big problem, someone is
> doing something wrong. It should be almost trivial. As such
> it doesn't need any financial incentive. In fact, financial incentive would
> be the worst thing - it would be encouraging the continuance of a
> some sort of broken workflow.

master branch is special with Boost though. It's not just the "unit
tests pass here" branch, but also the branch used to generate Boost
release.

That means writing code and merging is 50% of the effort of merging
develop to master. I finished AFIO v1.3 some weeks ago, but I haven't
merged from develop to master yet despite all unit tests being green
apart from Mingw32, which I am going to drop support for (Mingw-w64
works, Mingw32 is just horribly broken). That's because for a merge
to release branch I have yet to do:

1. Update documentation to reflect the new facilities, and indeed
your Incubator review where I believe I have addressed the majority
of your concerns at least partially. I've been at this for nearly
three weeks now. You only get so much time outside of work.

2. Rewrite the design rationale as it is now quite stale given my
recent empirical testing findings.

3. Add unit test targets for Android 5.0 which now supports C++ 11.
Fix any failures.

4. Add more tests to improve line coverage as I know it's dipped
below 90%.

5. Run compilation time benchmarks on all platforms. Fix any
regressions. You might think this easy, but AFIO has about twenty
different build configurations now.

6. Run performance benchmarks on all platforms. Fix any regressions.

I had been hoping to merge to master for end of January, it's
currently looking like mid-February.

> Personally I think the problems of "maintenance" are over stated. If
> maintaining (as opposed to adding features) is a lot of work, one needs
> to step back and look at what he's doing wrong and think about how
> to change it.

I think a huge amount depends on how legacy your codebase is and how
tightly other people's code depends on yours being exactly as before.

I'll put it another way: QNX and Microsoft expend enormous resources
making sure all future versions of their software have identical
behaviour to before when in some form of compatibility mode. There is
a big customer demand for that sort of "reliability", and people
willing to pay serious bucks for it.

> One final thought. How about using the "surprise release" © model?
> This one is someone looks at all the test matrices for all the libraries.
> When they (almost) all look good. The release is announced and the
> release branch is closed to updates except for any pending errors. No
> new features, etc. The the release is no work for developers at all!!!!
> I know this sounds like I'm trying to be funny, but I consider this a
> serious proposal.

I am not a fan of this model. The Web 2.0 culture does this a lot,
but they provide services which can and are remotely updated if
showstoppers turn up. When you ship a library on the other hand, that
release is probably going to turn into stone in someone else's code.

Niall

-- 
ned Productions Limited Consulting
http://www.nedproductions.biz/ 
http://ie.linkedin.com/in/nialldouglas/



Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk