Re: [boost] Boost Modularization: did we get it right?

Subject: Re: [boost] Boost Modularization: did we get it right?
From: Bjørn Roald (bjorn_at_[hidden])
Date: 2012-05-10 23:48:50

On 05/11/2012 01:55 AM, Steven Watanabe wrote:
> AMDG
>
> On 05/09/2012 01:06 PM, Daniel James wrote:
>> On 9 May 2012 20:18, Daniel Pfeifer<daniel_at_[hidden]>  wrote:
>>> 2012/5/9 Steven Watanabe<watanabesj_at_[hidden]>:
>>>
>>>> Behavior:
>>>> If symlinks are supported, creates a symbolic
>>>>   link to the directory.
>>> This is fragile. Imagine Graph generates a link at boost/graph and
>>> GraphParallel generates a link at boost/graph/parallel.
>>> GraphParallel's link will end up in Graph's source directory!
>>> There is also the case that multiple libraries provide files in the
>>> same directory (eg. boost/pending). They cannot all link the
>>> directory.
>>>
>>> The script should always link individual files. That is dead slow,
>>> yes. But it is the only safe approach.
>> Have you seen GNU stow? It links directories, but if two packages
>> clash, replaces the link with a new directory, and fills that with
>> links.
>>
> I think I have this working now.  I've added
> a glob to Jamroot that finds all the include
> directories.  running 'bjam headers' should
> create all the links.  The only thing left
> to make it work seamlessly is to go through
> all the libs and add
> <implicit-dependency>/boost//headers
> to all the tests and compiled libraries.
>
> The algorithm now looks like:
>
> If symlinks are supported:
>    Sym link the directory
>    if there is a conflict, create
>      a subdirectory and symlink all
>      the members.
> Else if hardlinks are supported:
>    Hard-link all leaves
> Else
>    Copy all leaves
>
> Any thoughts on this?

Algorithm looks good to me as far as what to create when a missing link 
is detected.

Whether you have captured the rest of what should be there it is not 
clear to me. My ignorance of the details of bjam and Boost.Build, and 
thus my understanding of the effect of your patches does not allow me to 
understand in a glance.  So I ask instead.

Does your solution provide proper functioning build graph including the 
derived forward header links or files?

Symbolic and hard links are services of the file system and when set up 
correctly, they should work transparently to the build system as though 
it looks at the source files.

bjorn_at_frodo2:/tmp/linkjunk$ touch a
bjorn_at_frodo2:/tmp/linkjunk$ touch b
bjorn_at_frodo2:/tmp/linkjunk$ ls --full-time
total 0
-rw-rw-r-- 1 bjorn bjorn 0 2012-05-11 05:36:13.000000000 +0200 a
-rw-rw-r-- 1 bjorn bjorn 0 2012-05-11 05:36:17.000000000 +0200 b
bjorn_at_frodo2:/tmp/linkjunk$ ln a a_hard
bjorn_at_frodo2:/tmp/linkjunk$ ln -s a a_symb
bjorn_at_frodo2:/tmp/linkjunk$ ls --full-time
total 0
-rw-rw-r-- 2 bjorn bjorn 0 2012-05-11 05:36:13.000000000 +0200 a
-rw-rw-r-- 2 bjorn bjorn 0 2012-05-11 05:36:13.000000000 +0200 a_hard
lrwxrwxrwx 1 bjorn bjorn 1 2012-05-11 05:37:17.000000000 +0200 a_symb -> a
-rw-rw-r-- 1 bjorn bjorn 0 2012-05-11 05:36:17.000000000 +0200 b
bjorn_at_frodo2:/tmp/linkjunk$ touch a
bjorn_at_frodo2:/tmp/linkjunk$ ls --full-time
total 0
-rw-rw-r-- 2 bjorn bjorn 0 2012-05-11 05:37:34.000000000 +0200 a
-rw-rw-r-- 2 bjorn bjorn 0 2012-05-11 05:37:34.000000000 +0200 a_hard
lrwxrwxrwx 1 bjorn bjorn 1 2012-05-11 05:37:17.000000000 +0200 a_symb -> a
-rw-rw-r-- 1 bjorn bjorn 0 2012-05-11 05:36:17.000000000 +0200 b

note the time of a_symb is older than the file a after last touch of a.  
Horror.  Does the build graph analysis handle this correctly.  Note also 
that hard links are OK.

Cleanup of derived forwarding headers and links when source are moved or 
deleted is a bit more tricky than support to create them.  Any hope of 
support for this?

A side note: A really smart build system should detect changes in 
content of the source files, not just a forward motion of file time.  
But this is probably both costly and a very fundamental change.  I think 
lack of such a feature may bite us more with modularized boost as it is 
simpler to change a parts in your build graph to an earlier versions 
without realizing the need for clean builds.

--
Bjørn