Boost logo

Boost :

Subject: Re: [boost] testing model
From: Julian Gonggrijp (j.gonggrijp_at_[hidden])
Date: 2012-02-13 11:19:07

Thomas Heller wrote:

> On 02/12/2012 12:34 PM, Julian Gonggrijp wrote:
>> [...] My guess is that,
>> given a branching model like gitflow, you envision something along the
>> following lines:

>> There is a "testing image" which the testing volunteers run their
>> periodic tests on. By default, the image mirrors the latest commit on
>> the develop branch. When working on a feature branch, library authors
>> can reversibly and transparently substitute the version of their
>> library from the feature branch for the default version in the image.
>> All of this is facilitated by some kind of automated tool which has
>> yet to be found.
>> The result is that most libraries in the image are from develop, but
>> some are from feature branches, without the testing volunteers needing
>> to worry about it. They can run a single testing cycle on the
>> heterogeneous image. For library authors, this brings the advantage
>> that they can already have their changes tested on all platforms
>> before they feel confident that it should be possible to make them
>> work. (Or while they emply a long-term feature branch specifically in
>> order to support a platform, per Daniel James.)

> *nod* looks like you absorbed it and summarized my ideas, thanks ;)

That's good to know. I think I figured out a way to do this. It's a
more elegant variant of the outline provided by Daniel in [1], quoted

Daniel James wrote:

> Or maybe, rather than being a traditional branch, the test repo could
> be created by a script which merges in active feature branches. It'd
> have to deal with conflicts though, maybe just keeping track of
> commits that are known to be good and sticking with them when a
> conflict is found. But something like that could be developed later -
> especially after having some experience of how the process works.
> This testing mechanism would be fairly separate from the main flow
> structure so it could be provided by an additional tool to gitflow.
> Hopefully wouldn't be too hard to develop. Does that make any sense? I
> know it isn't ideal, but it seems plausible given our current
> infrastructure. And we could work on improving it as we go.

I think it would be both wise and feasible to require (1) that authors
edit a given library either on the develop branch or on any number of
feature branches, but not on both in parallel, and (2) that a public
feature branch always affects only one library. Apart from avoiding
confusion, this offers a strong guarantee that a feature branch can
always be merged back into develop without conflicts.

Given such a convention, I think the test imaging tool is almost
trivial to implement. The core implementation takes only two
additional small files in the /status directory: a list of feature
branches that authors want to be substituted for the default in the
develop branch, and a small script that will be run by the volunteers.
Assuming that the list is a newline-separated text file like this
(just an example, take it with some grains of salt),

substitute.txt (must be edited on the develop branch only):
> newlibs/application
> math/octonion/hardwareaccel
> data/heap

the core of the script would be something like this,
> # must be run from $BOOST_ROOT/status
> git checkout develop
> git pull
> awk 'begin { print "git merge --no-commit --no-rerere-autoupdate" } ; { printf "remote/%s" , $0 }' substitute.txt >
> sh
> rm
> bjam
> # an additional command to report the results?
> bjam --clean
> git checkout develop

Note that the last line undoes the custom merge, so the test image is
only a temporary state of the volunteer's working directory.

The script I outlined and the existing testing tools will need some
adjustments to make the summary correctly display which commits have
been tested. I believe this shouldn't be an obstacle.
It should be easy to port to most platforms, as it's a plain command
list and the only external tools used are portable themselves, i.e.
git, awk and bjam.

I suspect it would be possible to realize this with a svn-based
implementation of the branching model as well. That would however have
some drawbacks with regard to the merge operation, in terms of
robustness and speed:

- substitute.txt and the awk command would be more complex, since svn
  merge requires authors to not only list the name of the branch but
  also the path to the library and the revision number;
- it would require multiple calls to svn merge, because svn can merge
  only two branches at a time;
- it would involve more server interaction because the volunteer has
  no local copy of the history.

Putting the VCS considerations aside, it seems like we have an
exciting opportunity here. I'm looking forward to all feedback.



Boost list run by bdawes at, gregod at, cpdaniel at, john at