Subject: Re: [boost] [svn/git/hg] Support for modularization of Boost?
From: Bjørn Roald (bjorn_at_[hidden])
Date: 2012-04-04 04:56:52
On 04/04/2012 09:20 AM, Robert Ramey wrote:
> Beman Dawes wrote:
>> As far as I can see, scaling Boost up to a much larger number of
>> libraries implies decentralization and decoupling, probably in the
>> form of per-library modules or something similar.
> 100 % aggree with this.
>> Modularization seems to have been missed in the discussions of
>> Subversion, Git, and Mercurial. Do distributed version control systems
>> in general and Git in particular have any important
>> advantages/disadvantages over svn for highly modularized projects?
> Hmmm - if boost is "modularized" - wouldn't that mean that
> each library has it's own repository.
I think this is a good idea as a basic assumption. Further I think an
official repository should be available somewhere well known for each
module from where the integration process can pull the changes.
> Or that each developer
> can decide where he want's to store his version?
> I would think that the testing/deployment process could
> update/export each library from a different place without
> much problem.
Not very clear what testing/deployment process mean here, but I think
this can work fine.
However, I think there also should be simple ways of using subset of
modules from boost without fetching them from a release of the complete
thing. Also, it should be simple to patch with per module hot-fixes
that never makes it into a main boost dot release. This implies support
for per module dependency control with explicit declaration of versions
of the dependencies supported with each version of a module.
Stuff like git submodule may provide part of the solution to this, but
there are issues that need to be resolved such as the fact that
currently a boost library does not fall into a a single mount point in
the boost structure. Typically there is at least 3:
This is something git-submodule does not handle well out of the box, at
least I think so. We have been down this path before on the list, I
think possible appealing alternatives are something like:
* SCM tool support multiple/overlapping mount points 
* SCM tool stage header files to common -I area 
* Build tool stage header files to common -I area 
* Move lib/x/.. files to boost/x/lib/...
A less appealing option is:
* clients use multiple -I when building against boost modules that has
not been integrated
A related issue that need to be considered is the need for common
baselines for dependencies, or at least some scheme for resolving valid
combinations of module versions (e.g. git commits). In boost there are
likely to multiple dependency paths to the same modules. How shall the
tools and processes support this when modules not pipelined through a
release process and aligned there? Assuming there will be such
alignment process for boost releases, but not for assuring compatibility
of more ad-hoc combinations of module versions.
One thought that come to my mind is that Boost need to track which
combination of module versions are tested in the test and release
process. Maybe tool support for this tracking also can track
compatibility of other module combinations not provided by the official
boost release team. I guess in git this is simply a set of module names
v.s. SHA1 keys similar to what git submodule uses. If there is a way to
publish results of testing other module combinations and all these
results are combined somehow into a sort of compatibility graph, this
may be a good way of sharing test results.
It should be simple enough for "possibly modified" git-submodule to
support this in a naive solution. However this may open the door for an
evil twin like problem that require some management. Evil twin is a
term used IBM to denote when multiple elements (files) under SCM control
map to the same file system path and one element eclipse (hides) the
other, see http://www-01.ibm.com/support/docview.wss?uid=swg21125072
This requires that header module files for module in boost/... are moved
to lib/x/boost/... or something similar
Make sure the staged header files are links to the files under SCM
control. Copied files are a real annoyance during debug - fix - debug
cycles, believe me! So whatever tool that does such staging in a
development environment should use links whenever possible, preferably
hard links to files or directory links (symbolic). Only when this is not
supported on the file system should the tool use copy.
evil twins are still a problem, but now a build time evil overwrite issue.
For this to be viable, boost/x.hpp files need to be in some common
module. I guess this will not be bad as long as they are forwarding
header and dependency from module x to common module is explicit end