Re: [boost] Feedback requested: Demo build server

From: Jason Sankey (jason_at_[hidden])
Date: 2007-10-02 10:07:03

David Abrahams wrote:
> on Sat Sep 29 2007, Jason Sankey <jason-AT-zutubi.com> wrote:
>>>> The default Pulse behaviour is to do a full clean checkout and build. 
>>>> However, there is an option to switch to incremental builds, where the 
>>>> same working copy is used for every build after an svn update to the 
>>>> desired revision.  The reason I steered clear is that I noticed a 
>>>> mention somewhere in the regression testing documentation that 
>>>> incremental builds were not 100% reliable.
>>> It has the same unreliability that most projects' builds do: the
>>> #include dependency checkers can be fooled by directives of the form
>>>
>>>   #include MACRO_INVOCATION()
>>>
>>> It's still very useful to do incremental builds, but it makes sense to
>>> build from scratch once a day.
>> I see.  I guess the problem is if incremental builds are known to have 
>> issues, will the attitude to them be different?  If people are used to 
>> builds failing due to incremental problems they may begin to ignore 
>> failures.  
> 
> I think such failures are relatively rare.  It's more likely to mask a
> failure than to report a false positive, because a test that should be
> re-run will be skipped when its dependency on a changed header is
> missed.

OK.  I will just flick the incremental switch and we can worry about 
problems if they happen :).

>>>> As suggested elsewhere, breaking things down library by library would 
>>>> also help.  I have noticed a bit of discussion going around about this 
>>>> lately, and have to say that I think it would be very helpful for 
>>>> integration with Pulse.  
>>> That's good to know.
>>>
>>>> Apart from faster builds, it would also make it easier to see the
>>>> status of each library if it were a top-level Pulse project, and
>>>> developers could then subscribed to email/jabber/RSS notifications
>>>> for just the libraries they are interested in.
>>> Interesting.  So what, exactly, does Pulse need in order to achieve
>>> these benefits?  Reorganization of SVN?  Separate build commands for
>>> each library?
>> The most important thing would be the ability to build and test a single 
>> library.  
> 
> We already have that for nearly all libraries -- just go into
> libs/<libraryname>/test and run bjam there.  Getting it for the rest
> is a fairly trivial change.

Ah - OK.  Simple.  I guess I should really just get on and try this!

>> In the simplest case this could involve checking out all of 
>> boost and having all dependent libraries built on demand when building 
>> the library of interest.  
> 
> That happens automatically when you run bjam.
> 
>> Then the tests for the library of interest could be executed and the
>> results output in some readable format (like the current
>> test_log.xml files).  This wouldn't necessarily require any
>> reorganisation of Boost: I guess that building a library
>> independently is already possible, I'm just not sure about running
>> the tests.
> 
> It's possible as described above.
> 
>> Further on, further optimisations could be done.  Reorganising
>> Subversion to allow just the library of interest to be checked could
>> help a little (although this won't save much real time).  More
>> important would be allowing pre-built versions of the dependencies
>> of the library to be picked up so that the build time is reduced.
> 
> If you use the same build directory for all the libraries (without
> cleaning it out between tests), you'll get that automatically.

True.  Using the same build directory for different Pulse projects is 
not possible "out-of-the-box" though.  I think I should be able to find 
a way of achieving a similar effect, though.

>>>>>> 3) How some of the errors/failing tests can be resolved.
>>>>> Not connected to the 'net as I write this; I might be able to look
>>>>> later.
>>>> OK, thanks.  Getting to a state where a normal build is green will make 
>>>> things a whole lot more useful.
>>> If you're testing trunk, you may never get there because IIUC it isn't
>>> very stable.  I suggest you run your tests on the 1.34.1 release tag
>>> at least until you see all green.
>> OK.  This is interesting, because in my experience this will greatly 
>> reduce the value of automated builds of the trunk.  
> 
> I know.
> 
>> The problem is basically broken window syndrome: if it is normal for
>> the build to be broken people care less about breaking it even
>> further.  
> 
> Yep.  
> 
>> Perhaps it is expected that the trunk is unstable and
>> other branches are used to stabilise releases.  Even then, though,
>> if people are not taking care then the trunk can drift further and
>> further from stable making it a real pain to bring everything up to
>> scratch for a release.  For this reason my personal preference is to
>> have the trunk (as the main development branch) be stable and green
>> at all times and for any unstable work to happen on isolated
>> branches.  Of course all of this is just my opinion so feel free to
>> ignore me :).
> 
> Some of us are very motivated to fix that for Boost.  But until it's
> fixed, you'll be better off testing your work against a known-good
> release branch.

Ah - I see.  I can get down off my soapbox then, as obviously I'm not 
saying anything you don't already know.  I will start playing with 
individual libraries off the 1.34.1 branch as suggested.

>> This may also be another argument for splitting things up by library. 
>> At least that way libraries that do keep a green trunk can get the 
>> benefits without the noise of failures in other libraries.
> 
> Unfortunately, such dependencies as exist in Boost cause that noise
> anyway.

This is true if the dependent libraries remain in a broken state. 
However, if a point was reached where green builds happened reasonably 
frequently then it would be good to be able to build against the "last 
known good" version of the dependent libraries.  That way a breakage in 
one library does not cascade down the dependency chain: depending 
libraries can continue to build and test against the last stable 
version.  Something to keep in mind as a possibility down the track anyhow.

Cheers,
Jason