Re: [boost] Radical idea for repository structure

From: Beman Dawes (bdawes_at_[hidden])
Date: 2007-09-18 10:38:57

David Abrahams wrote:
> on Mon Sep 17 2007, Beman Dawes <bdawes-AT-acm.org> wrote:
> 
>> David Abrahams wrote:
>>> A few days ago I started thinking about the implications of giving
>>> each Boost library a separate subtree of the repository, and it led me
>>> to some interesting places.
>>>
>>> * Start with the assumption that each library has a boost/ directory
>>>   containing its subset of the headers it currently has.  So, for
>>>   example, Boost.Python would have
>>>
>>>            boost/
>>>               python.hpp
>>>               python/  
>>>                  ...
>>>
>>>   where "..." above is identical to the current contents of
>>>   $BOOST_ROOT/boost/python/
>>>
>>> * Our release process would merge the boost/ directories
>>>
>>> * To test a library from a source distribution, you'd need to get the
>>>   boost/ directories of any libraries it depends on into the #include
>>>   path.
>>>
>>> * This list of dependency libraries would be encoded in each dependent
>>>   library's Jamfile.
>>>
>>> * Presto, a way to explicitly declare and track library
>>>   inter-dependencies!  If you fail to declare a dependency in your
>>>   Jamfile, your tests won't compile.
>>>
>>> Thoughts?

I would guess that almost everyone believes that it would be best if the 
header files for each library actually lived in that library. Lots of 
people have suggested we move to that organization. The showstopper has 
always been to avoid requiring end-users and library developers to have 
to many paths to their build system (bjam, IDE, makefile, whatever).

So if I understand you correctly, you are suggesting that as long as we 
can still provide end-users of releases with a single path (by copying 
to boost-root/boost during release preparation), it might be OK to force 
developers to do have to specify a path for each library their library 
depends on.

That strikes me as OK if they only have to specify direct dependencies, 
which is something they control. But it isn't OK if they have to specify 
indirect dependencies, which out of their control, much harder to 
identify, change all the time, and are sometimes subject exactly what 
macros are defined for a given compile.

>> Question 1: Does that mean different Jamfiles are needed for releases vs 
>> SVN working copies? That is too error prone - a way would have to be 
>> found so that the same Jamfile works regardless of whether it is run 
>> against a release or a SVN working copy.
> 
> I'm certain that's easy to arrange.

Good. That overcomes one hurdle.

> 
>> Question 2: Let's say lib A depends on lib B, and B in turn depends on 
>> lib C. Does that mean lib A's Jamfile must list both B and C? 
> 
> We get to decide.

See above. I don't see how it can work if a Jamfile has to be changed 
every time its indirect dependencies change.

> 
>> If such 
>> transitive dependencies have to be listed, that's a problem because if C 
>> gets changed to depend on lib D, then the Jamfile for A, B, and C all 
>> have to be changed, and that's a mess. OTOH, if only direct dependencies 
>> have to be listed in the Jamfile, it might possibly work.
> 
> Having to specify everything is more explicit, but more
> labor-intensive.  It's a tradeoff.  I'm not sure which one is right,
> but I think probably the implicit way is explicit enough.

I think that is the only practical approach, if the Jamfiles actually 
have to work against a raw svn checkout (which I think they should).

>> Question 3: What about developers who use an IDE or traditional make 
>> files? For example, using the VC++ IDE, the "Additional Include 
>> Directories" property is currently set to $(BOOST_ROOT) and you are 
>> done. On IDE's where this sort of property can be inherited, you only 
>> have to specify that in one place, and all Boost compiles work 
>> correctly. With your scheme it sounds like every library used will 
>> require an additional include path, and that path will be different for 
>> release compared to SVN working copies. That gets old fast. Or have you 
>> figured out a simpler way?
> 
> If you install the headers you can use them with a single directory
> in the #include path.

Hum... I've always rejected that approach as too cumbersome for 
developers. But nowadays disk space is so cheap I have separate boost 
working copies, one I update regularly and one I keep more stable to use 
for development. It wouldn't be such a problem if I was having to update 
the development working copy via a script that did the header install.

We also might want to brainstorm ways that avoid developers having to do 
a header install after every svn up. Possibilities:

* Hard links. Say the header install created hard links rather than 
copies. Then there is no need to do another header install after an 
update unless a header is added or deleted.

* Hard links, maintained by svn and thus totally transparent. Not sure 
if this is possible, but it sure is attractive. I'll research this one.

* Provide a property file for popular IDE's like Visual Studio. Not as 
attractive, but might be a fallback if other approaches fail.

> 
> It's also possible to use svn:externals to automate the collection of
> subdirectories so you can do this directly in an svn working copy.

David Greene's posting looks worrysome for externals.

--Beman