|
|
Boost Interest : |
From: David Abrahams (dave_at_[hidden])
Date: 2008-05-26 17:41:59
> Quick-ish report on this:
>
> I spent several days hacking on bitten. There are a few things that
> Dave and I discussed that need fixing.
I would put it less strongly. Bitten is usable for us today, but there
are definitely a few things that could be improved.
> It looks like the transformation of bitten from Build Controller to
> Build Result Displayer isn't trivial.
Huh; I've already hacked it a bit and it didn't look like we'd have to
make any deep, fundamental changes.
> Bitten is built with one way of doing things in mind: the server
> controls everything.
Well, sorta. Slaves periodically ask "is there anything for me to do?"
and the server answers.
> The central (trac) server has a bunch of 'build
> configurations', which consist of an SVN url and a 'recipe' for
> building, and several associated 'target platforms', which are
> basically a tag and a few regexes to match things like OS and compiler
> version. Target platforms are associated with one 'build
> configuration' and one build configuration only.
>
> This is nice (I guess) if you're adminstering your own group of build
> slaves, but not so good for us, as the
>
> - build recipe
> - svn URL
> - list of taret platforms
>
> are all tightly coupled. One cannot, for instance, allow somebody to
> run and post serialization tests only on some obscure platform. If I
> decided I wanted to continuously run a set of tests on the proto
> branch, I'd need a bitten admin to set up all kinds stuff... if
> somebody else wanted to join in and test with an obscure compiler, the
> bitten admin would be involed again. If you add a codebase to the
> rotation, you need to copy the target platforms and hack together a
> build recipe. If somebody brings up a new platform, you need to add
> the description to all recipes that it will build. And so on.
>
> This coupling touches just about every part of the code. The build
> progress display (i.e. 7 of 16 steps completed with a little progress
> bar) depends on the server knowing how many total steps are in the
> recipe that the client is building. In the Bitten Way, the server does
> know because it sent the recipe to the client in the first place.
I think you're making it sound much further from usable for us than it
actually is. The _main_ thing that needs to change is the way Bitten
identifies slaves with target platforms. I'm not sure how to make this
very clear to everyone, but I'll try.
My current Bitten setup has a single build recipe whose steps consist
essentially of:
1. SVN update boost source to the version being tested
2. SVN update boost tools source from http://svn.boost.org/svn/boost/branches/bitten/tools
2. rebuild bjam if necessary
3. tools/regression/run.py --library=boost/algorithm/minmax/test
4. tools/regression/run.py --library=boost/algorithm/string/test
5. tools/regression/run.py --library=boost/any/test
...
etc.
Each library is a separate build step in because that's the best way to
get an appropriate level of granularity in the visual display of output.
Recipes are not a big flexibility problem for us. It's easy to modify
run.py to selectively skip libraries on a given tester's machine based
on some configuration file they set up. The guy who only wants to test
serialization could simply turn off testing of all the other libraries.
If you want to make the progress display more sensible, only a *small*
change to Bitten would be required for slaves to report back that they
will perform 4 out of the full 92 build steps (or whatever). Yes, it's
not as elegant as it could be, but it would work. An incremental way to
make it more elegant would be that the slave sends a recipe back to the
server describing what it intends to do.
The real flexibility issue lies in the list of target platforms. Mine
currently looks like this:
Darwin
* os ~= Darwin
Linux 64-bit GCC 4.1.2
* os ~= [lL]inux
* boost.toolsets ~= gcc-4.1.2
Linux GCC 4.3.0
* os ~= ^[Ll]inux
* boost.toolsets ~= ^gcc-4.3.0
Solaris x86 SunPro 5.9 32-bit
* boost.toolsets ~= sun-5.9
* boost.bjam_options ~= ^.*(?!address-model=64)
Windows 64-bit MSVC 8.0
* version ~= ^[Ww]indows
* boost.toolsets ~= msvc-8.0
Windows MSVC-9.0
* version ~= ^[Ww]indows
* boost.toolsets ~= msvc-9.0
When a slave asks the server for work to do, it identifies itself via a
few built-in configuration variables,
os:
The name of the operating system (for example "Darwin")
family:
The type of operating system (for example "posix" or "nt")
version:
The operating system version (for example "8.10.1)
machine:
The hardware architecture (for example "i386")
processor:
The CPU model (for example "i386", this may be empty or the same as for machine)
name:
The name of the slave
ipnr:
The IP address of the slave
and any other optional variables that the administrators of the system
want to use in their build recipes (boost.toolsets and
boost.bjam_options are examples of the latter in my list above).
** Here is the key thing that needs to change **
The server then matches the slave against the list of target platforms
to identify which platform it is. If it matches a target platform X
(e.g. "Windows MSVC-9.0" above), it asks whether there are any SVN
repo versions containing changes in the recipe's area of the repo that
haven't been tested on X.*
If so, it begins sending recipe steps to the slave and will use the
slave's results as the official results for that target platform. If
not, the server says "nothing to do."
* (actually, whether it tests every revision in a range or only the
latest revision is configurable, but it always starts with the most
recent changes and works its way back in time)
The upshot is that, without changes to the target platform list from
someone with trac webadmin privileges (e.g. a moderator), a new tester
coming on-line can only contribute to the pool of testing power for
existing platforms (and in the same area of the repo). IMO fixing this
**would also be really easy** if we knew what we wanted. So that's the
discussion I think we need to have: what happens when a new slave comes
on line and how is it reported.
> What Dave and I discussed was bitten with less Control and more
> Display. If clients are to be allowed to formulate their own recipes,
> the protocol and database need changing (clients need to report the
> recipe, at some point, along with "Here are the results for step 7 of
> that recipe that I said I was working on".)
But clients don't need to be able to formulate their own recipes, at
least not yet. That would be more elegant, but I'm
> I got this much of it
> done as part of just learning to hack on trac plugins, then noticed
> that the queueing logic will need modification,
I think the queueing may be somewhat broken ATM, but I'd be interested
in knowing what changes you see as needed.
> as well as the association between platforms and urls/recipes... it
> looks like basically a rewrite.
Are you saying there isn't much more to Bitten than that?
> I haven't yet thought about CTest, transforming CTest XML to
> bitten-palatable format, automatic generation of build/test recipes
> from cmake, or the possibility of some entirely different test-running
> mechanism than ctest. For what I have been doing, dummy builds
> suffice. I'm willing to switch to this though.... We could decide for
> now to bear with the extra administrative overhead of the current
> bitten and focus on this stuff instead. Comments?
I'd be more than happy to work on Bitten mods with you.
-- Dave Abrahams BoostPro Computing http://www.boostpro.com