Re: [boost] Breaking existing libraries

Subject: Re: [boost] Breaking existing libraries
From: Vladimir Prus (vladimir_at_[hidden])
Date: 2008-11-21 10:39:54

Johan Nilsson wrote:

> Sure, there are many potential problems with such an approach:
> 
> - Some library tests are likely to also excercise code at the
> implementation/detail level.
> - Already limited testing resources (on the other hand, interface changes
> are likely to be detected by running such tests for a single, reasonably
> compliant, compiler).
> - ...
> 
> Anyway, this is mostly me thinking out loud; perhaps someone smarter than me
> could expand on the feasibility of such an idea.

I hope somebody not claiming to be smarter than you is also allowed to comment ;-)

An approach similar to this is used by the GDB project -- there are several versions
of the interface from GDB to GUI frontends, and there are presently basically two sets
of tests -- one is fixed and tests specific version of the interface. The second set
tests all the current behaviour, when it's considered good to go, tests for the
current behaviour will be copied into a new set, and won't be touched any more.

The primary problem with that approach, in case of GDB, is that it's just not practical
to conditionalize every bit of output on used interface version, so the testcases are
written to tolerate some "good" additional fields of output. And the testcases have to
be modified when new "good" additional fields appear, so the maintenance burden increases.

Probably, for C++ Boost this is not an issue, and it's possible to freeze the tests of
earlier published interface, and don't touch them. But for C++ Boost, the performance
is a real problem. E.g, I suspect that two complete Spirit testsuites, for different versions,
will be very hard on test resources.

- Volodya