Re: [boost] [BoostCon07][testing] Session reminder.

From: Gennadiy Rozental (gennadiy.rozental_at_[hidden])
Date: 2007-05-03 12:56:20

Hi,

I do plan to attend this session. I've got some ideas on the subject. I 
pitched them while ago once. And I still believe that the synchronization is 
the root of all evils. The only real solution to break this deadloop is 
independent libraries versioning. This should resolve both out release and 
testing issues (which are closely connected IMO)

INDEPENDENT LIBRARY VERTIONING

Idea is to "bite the bullet" and completely separate boost into multiple 
independent components and use independent versioning/testing for each one.

I. Split

Physically it can be done either as straightforward directory restructuring 
or using some kind of smart tagging. Directory restructuring may look like 
change from

boost/
   lib1/
   lib2/
   lib3/
....
libs/
   lib1/
   lib2/
   lib3/
....

to

lib1/
   boost/
   libs/
lib2/
   boost/
   libs/
...

II Independent versioning

Each library/component will be independently versioned and released (see 
below how). The boost release is just a thin layer on top (see below)

III. Regression tester setup

Instead of single boost tree regression testers will have maintain something 
like this:

lib1/
   <version1>/
      boost/
      libs/
   <version2>/
      boost/
      libs/
lib2/
   <version1>/
      boost/
      libs/
   <version2>/
      boost/
      libs/
...

Why? See below.

IV. Makefile/Jamfile changes

Instead of -I (BOOST_HOME) (in general terms - I know bjam works a bit 
differently). We introduce the notion of versioned dependency. For every 
library A we will have to determine which library it depends on (lets say 
B,C and D) and somewhere in makefile/Jamfile we add the statement:

A.dependant_libs = B:<version of B> C:<version of C> D:<version of D>

For example:

A.dependant_libs = A:1.2.1 B:1.5.0 D:1.4.3

Above statements works two ways:

1. During compilation above statement causes -I A/<version of A> ... added 
to the compilation options.
2. For unit tests it should add appropriate library object to link with.

Now every library is developed and tested independently of other libraries 
development. I may even stick to older version of component D even though 
new one is available. "version of X" is the version of component X and it 
may be one of those:

1. <major>.<minor>.<patch>

Points to the specific version of component X.

2. <major>.<minor>

Points to the latest patch of component X in <major>.<minor> version

3. LATEST

Points to the LATEST released version

4. Boost:<version of boost>

Points to the version of component X that belong to the appropriate version 
of Boost (see below for boost releasing procedure)

5. DEV

Points to the development version of component X (cvs/svn HEAD). To be used 
to test against development version of another component. Is not recommended 
and would prevent releasing of your library.

V Single component/library release.

As soon as regression test for your library is green you are free to make 
release. You don't have to synchronize with/wait for anyone. It's done by 
single script that does following tasks:

1. Checks that all unit tests are green
2. Checks that you don't depend on development version of other components
3. Tag your library appropriately
4. Save version of all dependant component. Even it you point on the LATEST, 
this will save actual version at the time of release.
5. Post announcements "Library A version V is released" on dev list.

No binaries is released unless we invent another procedure for independent 
component packaging.

VI Boost release

The boost release is done automatically every predefined period of time 
(like three month). It requires NO testing, NO branching and NO 
merging/reverting. It's done by single script (a bit oversimplification, but 
close) that does these tasks:

1. It iterates through all components and check any component that was 
released since previous boost release.
Very important: The component should NOT depend on older version of 
component that is being released. For example if  library A depends on 
version 1.23 of library B and version 1.24 of library B was released, 
library A won't become part of the next boost release. If your library 
depend on older version component that is being release also, it's your 
responsibility to test it with new release on dependant component and 
release new version. This may become a problem if core library - library 
many others depends on - starts frequent releases. There are couple ways to 
deal with this:

a) Core library author shouldn't do this ;)
b) Do not depend on fixed version of the component. Use LATEST instead. This 
way you are always testing against latest released version of the dependant 
component. This should minimize the chance that your library released with 
dependency to older component. Though it still may happened.
c) Library can be released as "not latest" IOW without changing it's LATEST 
version. It can be done if library author wish to release new version for 
some users, but do not want to disturb upcoming release.

2. Once list of updates components created it iterates through it and if any 
library was released with <minor> version update this will be <minor> 
version update of the Boost. If all libraries release with <patch> version 
updates, it's going to be <patch> release of boost.

3. All the boost components are merged into single tree:

boost/
   lib1/
   lib2/
   lib3/
....
libs/
   lib1/
   lib2/
   lib3/
....

At this stage we should check that no components are using the same header.

4. The rest should be as usual

VII. Testing optimization.

IMO There are some testing procedure optimizations that can be done:

1. If space is any concern instead of keeping multiple copies of each 
component make system should support retrieving it by request. IOW if during 
library A compilation we determine that we need version V of component X and 
it's not present, we get it automatically from source control system. Once 
testing is completed the dependant library could be removed.

2. Development only testing.

We don't really need to test libraries that do not have anything new past 
previous release. IOW if here were no changes in a library since it's last 
release and none of the dependant libraries that are referred with LATEST 
version are changed, no need to run tests.

3. Request only testing

We may introduce request only test system. If library author wish it to be 
tested it indicate it somehow. For example by tagging it's development 
version with tag for_test. Until the request is not reverted testing is 
performed.

I must admit this proposal requires some changes in Boost.Build and other 
Boost procedures. It also might require more disc space on regression 
tester's site. But I believe this is the only direction that can really lead 
us to the salvation.

Regards,

Gennadiy