Boost logo

Boost :

From: Gennadiy Rozental (gennadiy.rozental_at_[hidden])
Date: 2007-05-03 12:56:20


Hi,

I do plan to attend this session. I've got some ideas on the subject. I
pitched them while ago once. And I still believe that the synchronization is
the root of all evils. The only real solution to break this deadloop is
independent libraries versioning. This should resolve both out release and
testing issues (which are closely connected IMO)

INDEPENDENT LIBRARY VERTIONING

Idea is to "bite the bullet" and completely separate boost into multiple
independent components and use independent versioning/testing for each one.

I. Split

Physically it can be done either as straightforward directory restructuring
or using some kind of smart tagging. Directory restructuring may look like
change from

boost/
   lib1/
   lib2/
   lib3/
....
libs/
   lib1/
   lib2/
   lib3/
....

to

lib1/
   boost/
   libs/
lib2/
   boost/
   libs/
...

II Independent versioning

Each library/component will be independently versioned and released (see
below how). The boost release is just a thin layer on top (see below)

III. Regression tester setup

Instead of single boost tree regression testers will have maintain something
like this:

lib1/
   <version1>/
      boost/
      libs/
   <version2>/
      boost/
      libs/
lib2/
   <version1>/
      boost/
      libs/
   <version2>/
      boost/
      libs/
...

Why? See below.

IV. Makefile/Jamfile changes

Instead of -I (BOOST_HOME) (in general terms - I know bjam works a bit
differently). We introduce the notion of versioned dependency. For every
library A we will have to determine which library it depends on (lets say
B,C and D) and somewhere in makefile/Jamfile we add the statement:

A.dependant_libs = B:<version of B> C:<version of C> D:<version of D>

For example:

A.dependant_libs = A:1.2.1 B:1.5.0 D:1.4.3

Above statements works two ways:

1. During compilation above statement causes -I A/<version of A> ... added
to the compilation options.
2. For unit tests it should add appropriate library object to link with.

Now every library is developed and tested independently of other libraries
development. I may even stick to older version of component D even though
new one is available. "version of X" is the version of component X and it
may be one of those:

1. <major>.<minor>.<patch>

Points to the specific version of component X.

2. <major>.<minor>

Points to the latest patch of component X in <major>.<minor> version

3. LATEST

Points to the LATEST released version

4. Boost:<version of boost>

Points to the version of component X that belong to the appropriate version
of Boost (see below for boost releasing procedure)

5. DEV

Points to the development version of component X (cvs/svn HEAD). To be used
to test against development version of another component. Is not recommended
and would prevent releasing of your library.

V Single component/library release.

As soon as regression test for your library is green you are free to make
release. You don't have to synchronize with/wait for anyone. It's done by
single script that does following tasks:

1. Checks that all unit tests are green
2. Checks that you don't depend on development version of other components
3. Tag your library appropriately
4. Save version of all dependant component. Even it you point on the LATEST,
this will save actual version at the time of release.
5. Post announcements "Library A version V is released" on dev list.

No binaries is released unless we invent another procedure for independent
component packaging.

VI Boost release

The boost release is done automatically every predefined period of time
(like three month). It requires NO testing, NO branching and NO
merging/reverting. It's done by single script (a bit oversimplification, but
close) that does these tasks:

1. It iterates through all components and check any component that was
released since previous boost release.
Very important: The component should NOT depend on older version of
component that is being released. For example if library A depends on
version 1.23 of library B and version 1.24 of library B was released,
library A won't become part of the next boost release. If your library
depend on older version component that is being release also, it's your
responsibility to test it with new release on dependant component and
release new version. This may become a problem if core library - library
many others depends on - starts frequent releases. There are couple ways to
deal with this:

a) Core library author shouldn't do this ;)
b) Do not depend on fixed version of the component. Use LATEST instead. This
way you are always testing against latest released version of the dependant
component. This should minimize the chance that your library released with
dependency to older component. Though it still may happened.
c) Library can be released as "not latest" IOW without changing it's LATEST
version. It can be done if library author wish to release new version for
some users, but do not want to disturb upcoming release.

2. Once list of updates components created it iterates through it and if any
library was released with <minor> version update this will be <minor>
version update of the Boost. If all libraries release with <patch> version
updates, it's going to be <patch> release of boost.

3. All the boost components are merged into single tree:

boost/
   lib1/
   lib2/
   lib3/
....
libs/
   lib1/
   lib2/
   lib3/
....

At this stage we should check that no components are using the same header.

4. The rest should be as usual

VII. Testing optimization.

IMO There are some testing procedure optimizations that can be done:

1. If space is any concern instead of keeping multiple copies of each
component make system should support retrieving it by request. IOW if during
library A compilation we determine that we need version V of component X and
it's not present, we get it automatically from source control system. Once
testing is completed the dependant library could be removed.

2. Development only testing.

We don't really need to test libraries that do not have anything new past
previous release. IOW if here were no changes in a library since it's last
release and none of the dependant libraries that are referred with LATEST
version are changed, no need to run tests.

3. Request only testing

We may introduce request only test system. If library author wish it to be
tested it indicate it somehow. For example by tagging it's development
version with tag for_test. Until the request is not reverted testing is
performed.

I must admit this proposal requires some changes in Boost.Build and other
Boost procedures. It also might require more disc space on regression
tester's site. But I believe this is the only direction that can really lead
us to the salvation.

Regards,

Gennadiy


Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk