Re: [boost] Testing Farm

From: Martin Wille (mw8329_at_[hidden])
Date: 2005-03-21 14:03:16

Jonathan Wakely wrote:

>>In my experience, they make a difference. And there's not only one 
>>glibc, since there are also many versions of it. Tests also depend on 
>>the kernel version, on the CPU(s), on the version of the system 
>>compiler, on the Python version et cetera.
> 
> 
> Then I stand corrected.
> 
> (The system compiler surely only matters if you're using the system
> compiler?  If you do parallel installs of five versions of GCC and use
> them for the tests, does the system compiler matter?)

It matters for icc. It also matters if you use Python compiled with the 
system compiler and with C++ support enabled (like the distro's Python).

> I may well be wrong again, but IMHO aiming for wider coverage (by
> testing on a couple of versions of Linux and some version of FreeBSD,
> for example) would pick up more portability problems than testing on a
> number of subtley different flavours of Linux.

Yes, I'm all for testing on a larger variety of systems. If I had a 
spare box then I'd set up BSD tests here myself.

> I know I've found several problems when I run the tests on FreeBSD that
> have clearly never showed up on any of the versions of Linux that are
> tested by other people.

Yes, and it is a pity we don't have enough BSD testers.

> Even if it does make quite a difference, it seems obvious to choose from
> the most popular distros, since that ensures compatibility with a higher
> number of users.  Choosing to run tests on the more obscure distros
> might find quirks of those distros but it might be that no Boost user
> ever uses that distro.

Well, that depends. If Boost doesn't work on certain distributions then 
it could be the distro's fault. However, more likely, it's lack of 
compatibility between Boost and the software versions the distro happens 
to use. I don't think we can blame a distro for that. (this is different 
if a distro uses broken software like gcc "2.96")

>>In fact, support for Boost packages on various Linux distributions isn't 
>>too great. There's no reason to expect distributors to run the tests.
>>Also, it actually doesn't matter whether a distribution tested the Boost 
>>version it has a package for. It's an already released version which 
>>might even contain patches from the distributor. We're talking about 
>>testing from the CVS, though.
> 
> 
> If *I* was the maintainer of a distro's Boost package I'd test against
> CVS, to know what problems might be coming up soon, but I'm not, so my
> opinion counts for nowt!

Please, become a maintainer ;-)

Seriously, the problem likely is that a distro's maintainer for Boost 
also has a bunch of other stuff to maintain. Chances are a maintainer 
doesn't have the time to properly test a package like Boost from the CVS 
regularly or even only at (Boost) release preparation times.

Regards,
m