Re: [boost] Breaking existing libraries

Subject: Re: [boost] Breaking existing libraries
From: David Abrahams (dave_at_[hidden])
Date: 2008-11-21 14:44:31

on Fri Nov 21 2008, Tomas Puverle <Tomas.Puverle-AT-morganstanley.com> wrote:

> Hello Dave,
>
>> Naturally, nobody likes to be asked to fix the problem they're
>> complaining about.  Such is the open-source world, I'm afraid.
>
> The question is whether "fix the problem" means the Boost.Range issue or the
> overarching problem of breaking changes.  While the latter is something we could
> fix going forward, my current problem is with Boost.Range.  So my post has two
> reasons: 
> 1) Try to get a consensus whether the Boost.Range problem is actually a defect
> and should be rolled back
> 2) How to avoid such things in the future
>
> I am sure you understand I can't just waltz into the SVN repository
> and change Thorsten's code.  

Nobody can.  Such a thing has to be done with great deliberation and
after giving Thorsten a reasonable amount of time in which to respond.

> However, I feel that if we could participate in a reasoned discussion
> and decide on the pros and cons of the change we may be able to come
> up with something that would resolve the problem going forward and
> perhaps find a transition path. Finding the best way to fix that
> problem is my priority, as this is currently a complete blocker on my
> project.

That's gonna be difficult unless we can engage the library maintainer.
Not necessarily impossible, but difficult at best.

> My original post has also shown that there are quite a few people who
> feel similarly to me about such changes.  Perhaps at this point it
> would be wise to break this thread into one about Boost.Range and
> another about boost development, even though I feel that they are very
> closely tied together, as the outcome of the Range discussion may or
> may not set a precedent for the latter.

OK

>> > It's a politician's answer, in that it doesn't really answer my
>> > original query.
>> 
>> Umm.  Trying not to be insulted, here.  
>
> My apologies - I only used this as a figure of speech, with no intention to
> question your integrity. :)
>
>> Your original query was not a "query" at all.  You asked no questions.
>> There was a list of (valid) complaints.  
>
> My intention was to start a discussion.
>
>> The only actionable item I saw there (which you snipped out of the
>> context of my quote)
>
> No intention to misrepresent.
>
>> was the
>> expression of a laudable desire "to try to make sure this kind of silent
>> breakage doesn’t happen again," and *that* is what I responded to.
>
> And that was a speakers way of beginning a discussion.  I was only
> (respectfully) addressing an audience.  

I have no problem with any of that.  However, I would like you to
acknowledge that I *did* in fact address what you wrote, whether or not
it was what you wanted to hear.

>> Of course, you also have some power to control the quality of Boost
>> developers, by participating in the review process.
>
> No, we would purely have input into the initial quality of the reviewed library.
>  While this will have a high correlation with the submitters knowledge of C++
> and their problem domain, it doesn't allow us to evaluate whether or not they
> have the experience in large scale software development and the foresight not to
> make changes which will break existing code.

I don't know why you started out by saying "No" there, because it sounds
like we're in agreement.  You would have some (not total) power to
control the quality of Boost developers.

> However, I also accept your point about more participation.  My
> colleagues and I have at points been involved with library reviews and
> discussions.  One of the reasons for the lack of participation (beyond
> shifting job responsibilities) is how far the head of boost is removed
> from the "stable" versions we use.  It is hard to get excited or find
> the time get involved with something that has no immediate benefit and
> may not be available to us for another few years.  In this respect, I
> think the idea of "stable" and "new" boost libraries would help in
> this respect.

I'm not sure how.  Would the "new" code that results from reviews have
more immediate relevance to you than it does now, in such a scheme?

>> > However, it seems that once a library is accepted, the maintainer is
>> > pretty much free to do what they please.  
>> 
>> Yes.
>
> I hope behind this simple "Yes" there are alarm bells going off.

Not really.  I understand the risks and tradeoffs of that arrangement,
and I think Boost made the right choices there.  IMO what's needed is
more structure to support the maintainer in making good decisions.

>> > I think it may be wise to apply the same level of rigour to the
>> > continued development.  Do you think that a boost library that was
>> > broken should be rolled back?
>> 
>> Quite possibly.  Depends what you mean by "broken" and "rolled back."
>
> For the sake of example, let's stay with the Boost.Range library.  By
> "broken" I mean that the behaviour has changed drastically from one
> release to the next (not to mention the debug/release issue).  IMHO,
> it has also "broken" the notion of what an empty range is.  Steven
> Watanabe tried suggesting a workaround, which was broken in an
> insidious way.  This in no way is a reflection on him, but on the fact
> that the behaviour of the library is unintuitive and leads to code
> with undefined behaviour.  That's what I mean by "broken".
>
> By "rolled back" I mean that the original behaviour should be
> reinstated either in a patch or the next release, with either an
> alternative name for the new concept, a migration path, or both.

It's certainly conceivable that we could form a consensus on something
like that here.

>> I've got some bad news for you, Tom: you're "within."  
>
> This isn't bad news - I like using boost and would like contribute
> more if my employer allowed it.  (Having said that, I've made a few
> very minor additions)

But the downside (again, referring to missing context) is that it puts
the onus on you to do some of the work.

>> Boost library authors generally highly value the input of their
>> users, and are eager to learn.  
>
> Yes, this in general is the case.  However, you can't deny that that
> people who critique your creation will frequently put you on the
> defensive.  I am just as guilty of this behaviour as the developer
> next to me.

Sure.  Not to be too blunt (as I often am), but what's your point?

>> You may think you're being asked to set Boost policy on your own, but
>> that's not how it works.  Boost is driven largely by consensus.  We'll
>> discuss and tweak the document together in this forum until we can
>> arrive at a consensus that it should be adopted.
>
> In which case it's a shame that the Boost.Range changes just happened
> and were never discussed here.  

Are you certain they were not discussed here?

I know at least one instance where breaking changes were discussed
(handling of NTBS's):
http://www.nabble.com/-Range--Expected-complexity-constraints-tt4279749.html#a4279749

> Again, please take this for what it is: A point made by an external
> observer.

Please don't imagine that there's any need to soften such a sentiment
for my sake.  It doesn't bother me at all to hear things like that.

>> > The difficulty of rebuilding all of the internal systems just makes
>> > this impractical.  As you are aware, a lot of people avoid the .0
>> > releases.  What does this say to you?  
>> 
>> It says they are cautious.  Cautious people often have that as a blanket
>> policy; not just with Boost. What does it say to you?
>
> It says to me that they've been burnt a few times and are not going to
> risk their neck again.

Yes.  And as as I said, such policies often are used for *all*
dependencies because people have been burnt by *something* that may or
may not have been Boost in the past.

> At the end of the day, I think most people in this industry love what
> they do and love new toys: Boost is one of them.  If we could come up
> with a way to prevent the toy from occasionally spontaneously bursting
> into flames, it would be much easier to sell to the parents (forgive
> the metaphor).

Agreed.

>> I think you mean the opposite.
>
> Sorry, fat fingers.
>
>> I know; it's a problem.  If you still want to try to prevent this sort
>> of thing in the future, I suggest you take up my earlier encouragement
>> to write up a policy page.  
>
> Before writing such a recommendation, let's have a discussion first
> about the proposal for the altered release schedule, 

<snaps head around> What proposal?

> which would have dramatic impact on the contents of the document.

My advice: try to decouple these things as much as possible.  When
progress in one area is postponed in anticipation of progress in
another, often you get no progress.

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com