Re: [boost] Feedback requested: Demo build server

From: Jason Sankey (jason_at_[hidden])
Date: 2007-09-14 10:56:15

Jeff Garland wrote:
> Jason Sankey wrote:
>> John Maddock wrote:
>>> Jason Sankey wrote:
>>>> Hi all,
>>>>
>>>> You may have noticed the discussion going on re: developer feedback
>>>> systems.  As part of this I have just started setting up a demo build
>>>> server at:
>> <snip>
>>
>>>> 1) If you think this is useful and worth continuing with.
>>> Definitely.
>> Thanks for the vote of confidence :).
> 
> +1 -- this looks very cool.

Excellent!

>>>> 2) What important features you think are currently missing.
>>> I found navigation a tad tricky at first: it would be useful if I could get 
>>> to "all the failures for library X" in the last build directly and easily. 
>>> As it is I had to drill down into the detail report and then scroll through 
>>> until I found what I was after.  Or maybe I'm missing something obvious? :-)
>> You are not missing anything, the drilling is necessary at the moment. 
>> This is largely because of the structure of boost, where all libraries 
>> are built in one hit.  Hence they are all currently in one Pulse 
>> project, and the result set is large.  One possible solution is to have 
>> a view of tests organised per suite (in Pulse terminology: each library 
>> is grouped into one suite) rather than the current view which is 
>> organised by stage (another Pulse term which refers to a single run on a 
>> specific agent).  It is also possible to capture HTML reports and serve 
>> them up via Pulse, so the build process itself could generate different 
>> types of reports as necessary.  Pulse supports permalinks to such 
>> artifacts so you could always find the latest report at a known URL.
> 
> Maybe there's a way the tests could be broken down by library by library?  In 
> case you don't already know, you can test a single library by going to the 
> subdirectory (eg: libs/date_time/test) and running bjam.  I expect it might 
> take some scripting, but you're obviously doing some to set all this up 
> anyway.  Anyway, that way each library could show up as a 'success/fail' line 
> in the table.  And we'd also get a measurement of how long it took to 
> build/run the test for each library.  This would be huge...

It had occurred to me that this may be a better approach, although I 
have not experimented with it yet.  One thing I am uncertain of is the 
dependencies between libraries and what it would mean for building them 
separately.  Overall I think breaking it down this way would be much 
better if dependencies are manageable.

>>> Can you explain more about the "My Builds" page and what it offers, as it 
>>> looks interesting?
>> This page shows you the results of your "personal" builds.  Personal 
>> builds are a way for you to test outstanding changes before you commit 
>> them to subversion.  Basically, you install a Pulse command line client 
>> ...snip..
> 
>> This is a feature that really comes into its own when you need to test 
>> on multiple environments.  Developers usually don't have an easy way to 
>> test all environments, so would usually just commit and find they have 
>> broken some other platform later.  IMO it would be a great tool for 
>> boost developers, but there would be challenges:
>>
>> 1) Having enough machines available to cope with the load of both 
>> regular and personal builds.
>> 2) The long build time.  This could be mitigated if just the library of 
>> interest was built, which is possible if we configure recipes in Pulse 
>> for each library.
> 
> We've been in need of such a mechanism for a very long time. We're trying to 
> move to a place where developers only check-in 'release-ready' code -- even 
> though I think most of us have been trying to do that I think the reality is 
> that with so many platforms it's really easy to break something.  We're also 
> getting more and more libs that provide a layer on OS services -- so we need 
> more platforms to support developers trying to port these libs.  Bottom line 
> is that this would be major breakthru for boost to have an 'on-demand' ability 
> to run a test on a particular platform with developer code before it gets 
> checked in.  Ideally we'd be able to do this on a single library -- that might 
> also help on the resources.

Right, this is exactly what I am talking about.  In my experience 
without this facility the less popular platforms end up being constantly 
broken which is very frustrating for the people working on those platforms.

Cheers,
Jason