Re: [boost] Merge procedure to master

Subject: Re: [boost] Merge procedure to master
From: Barend Gehrels (barend_at_[hidden])
Date: 2015-01-26 16:44:17

Hi Robert, Vicente, Olaf,

Thanks for your answers!

Robert Ramey schreef op 23-1-2015 om 23:37:
> Barend Gehrels wrote
>> Hi,
>>
>> Robert Ramey schreef op 23-1-2015 om 21:05:
>>> Peter Dimov-2 wrote
>>>> Robert's perspective here, FWIW, is that he develops and tests (quite
>>>> extensively) in a configuration in which all libraries are on their
>>>> master
>>>> branch and Serialization is on develop.
>>> Thanks for pointing this out. This is true and of course a big reason
>>> that
>>> for me the merge from develop into master IS trivial.  Sorry I forgot to
>>> (re)mention that.
>> You forget to point out all the other actions I mentioned in my list.
>>
>>>> On this configuration, once everything is green, merge is as easy as
>>>> "merge --no-ff develop". You've already tested it, so there's no way
>>>> anything can fail.
>>> correct, that's why I do it this way.
>> And it is dangerous.
> why?  (btw - I don't use the --no-ff")
>
>> Unless you're not dependant on any other Boost library.
>>
>> If you (with e.g. library Y) always work on develop and test with other
>> librarys on master, you don't notice any other development or possible
>> problem.
> That's a good thing!!!.  I've the are libraries are in develop - they are
> not
> certified correct.  If I test against the I get my failures along with
> everyone
> else's - and I have to sort them out by hand - which takes a lot of time
> as you've noticed.  My way of doing things is more of a controlled
> experiment - I'm testing  one thing (the serialization) library at a
> time while holding all the other variables (all the other libraries)
> stable.  It's more accurate view of what I want to test.
>
>> Suppose you are dependant on library X. As soon as the schedule
>> comes out, moment S, you merge again because it is trivial (moment T) ,
>> and you go on holiday.
> when library X comes out (is merged to master) - I don't merge - I update.
> If I then run my tests and there is a new error - I know it's the other guy
> and I can tell him.  Note that my tests are not setup to test everyone
> else's stuff - they assume that the pre-requisite libraries are working.  If
> I don't know that - then my tests are pretty useless.
>
>> Library X decides (at moment M, long before S), to change a return-type
>> into something much better. You are using that feature. It happens.
> If he does this in the master, he's made a mistake in a public interface.
> I'll have to give him a hard time if I detect it.  But I can't be expected
> to
> be able to debug everyone else's stuff.
>
> As soon as the release schedule comes out, they stop developing, monitor
> regression matrix (for library X), and after a couple of days, just
> before the closing of master, they merge (moment U, after T).
>
>> Now your library is broken. It does not even compile anymore. You don't
>> see it, because you are on holiday and have done the job. Even if you
>> are not on holiday, you might not notice it immediately, depending on
>> how often you compile. And the masterbranch closes... They don't see it
>> because they run library X unit tests and not library Y unit tests. It
>> will be found by the users of the RC though.
>> Of course it is not your fault! Why the xxx did they change the
>> return-type? But it is better... even you have to agree with that. And
>> they did it already at moment M and they did send a notification to the
>> Boost mailing list (which by chance you did not read).
>>
>> If you had used develop for your developments, you would have noticed
>> the breaking shortly after moment M already.
> all correct - but at least the error is obvious.  The boost release manager
> will have to decide to ask to role back a breaking change or the
> rest of to adjust to it.  The benefits of detecting this earlier are no
> where no near the costs of testing against all other libraries which
> are not known to be in a bug free state.
> So it can be dangerous. Chances are low, but it happens.
>
>> Another scenario, but I will not make it too long: library Z has a new
>> feature you really need! Do you wait until they merge to master? Maybe,
>> indeed, that is good practice. But it happens that another library uses
>> it already earlier than that... both on develop.
> possible - but unusual.  Again not worth the pain of chasing down everyone
> elses mistakes - which they likely know about anyway.
>
>>>> Of course this has its own drawback - the boost.org tests do not work in
>>>> such a manner.
>>> True again! But I don't think this is a reason why I shouldn't do it for
>>> my
>>> own tests. My life is so much easier since I started doing things this
>>> way.
>>> Much of the back and forth cited above just goes away.
>>>
>>> Barend - Why don't you try my approach as an experiment.  It's super easy
>>> to set up.  And if you don't agree that it makes your life a lot easier,
>>> it's just
>>> as easy to switch back. Try this an let us know how it works out for you.
>> The approach is not so much different. You work always on master,
> I work on develop for the serialization library and the master for
> everything
> I need but don't have responsability  for.
>
>> I always on develop. And sometimes (indeed) I check on master too, by the
>> way, without immediately merging afterwards.
> I can tell you from personal experience that my way of doing things saves
> a huge amount of time.  I used to do it your way - but never again!
>
>> But please react on the other points on the list too. Do you monitor the
>> regression matrix? Wait for a couple of days? Or maybe it is not
>> necessary for you?
> I need to check develop to look for failures
> in compilers I don't have - but that's about it.  If things are stable for
> a couple of days, I then merge to master.
>
>> As I earlier point out, we have multiple people
>> working on the library daily. You state that that makes no difference -
>> is that really true?
> Git modularization supports the "feature branch" very, very well and
> very conveniently (especially with a GUI like SourceTree).   I use
> it even though I'm working by my self.  Each of your five people
> should be using a different branch.  The should test against all
> the libraries in master and your library in develop.  But they
> should only update their version of develop between times
> they're working on their feature. (subject to coupling problems
> of course).  Anyone who has his feature working can merge
> into develop and let then let the rest of the team test it.  The
> team leader - whom I presume is you - can decide when to
> merge develop in to master.
>
> I agree this situation doesn't dovetail that well with boost.org testing.
>
>> Do you neatly update the docs every time you merge with master?
> No, I update the docs BEFORE I merge to the master.  That is
> after it passes all the develop tests and before the merge so he
> docs are always in sync the code.  And I don't even use DOxygen!
>
> Theres another think I do - I increment the serialization library one
> feature at at time. when it works I merge it all the way through
> to the master.  I may have a couple of features going at once
> because some urgent thing comes up while I'm working on
> something else.  The upshot is that I'm basically always working
> on one thing at at time and pushing all they though to the master
> before I add the next thing.  This diminishes wasted time by a
> huge factor.  BTW - i've got almost 300 tests to watch.  But really
> compared to what I was doing before - this is so much easier,
> and effective.  I urge everyone to consider it.  And remember -
> it's not a commitment - it's just a workflow which can be easily
> changed if it doesn't work well for you.
>
> I really don't dispute the other points.  its just that my of doing things
> results fewer iterations so the other issues are less problematic.
> For example I do wait a few days looking at the develop tests - if there's a
> bug I know its not mine but I keep an eye on it anyway.  Of course
> I would prefer that the develop test matrix worked they way I work
> but it doesn't so I just live with it.  My way is still better so I use it.
> No one seems to care - or even notice.  No one has complained to me
> that I'm not testing their stuff.
>
>>> I've talked to Rene about changing the boost.org tests to work in the way
>>> I do it.  He appreciated the idea - but was concerned about the
>>> difficulties
>>> or implementing it on all the libraries one by one.  I'm still convinced
>>> that testing each library in develop against all the others in master is
>>> the way
>>> to go - but since Rene does the work, he get's to decide.  That is the
>>> boost
>>> way - as it must be.
>> I don't know the details of that. Merging multiple dependant libraries
>> is a complex thing, which can go wrong in many scenarios.
> Altering your workflow would make it much less burdensome.
>
>> That is why I
>> thought that it was good to react on your statements that it is trivial.
> No problem, I appreciate another opportunity to make my case!
>
>> Things can be solved, and we can do our best efforts to do it as good as
>> possible, but it is not trivial.
> OK maybe trivial is too strong - but not by much.  Honestly, taking
> best advantage of our newer tools has made the job of maintaining
> the serialization library much, much easier than it used to be.  But
> one has to change his practices.

It is clear, I will not go into detail on all the points again, most 
things have probably already been said. The message is test-with-master, 
merge-with-master, and we will do that more early and more often.

Thanks again, Barend