Re: [Boost-build] [git] genrating forward headers

Subject: Re: [Boost-build] [git] genrating forward headers
From: Bjørn Roald (bjorn_at_[hidden])
Date: 2013-07-24 18:47:04

On 07/24/2013 04:44 PM, Rene Rivera wrote:
> On Wed, Jul 24, 2013 at 4:32 AM, Jürgen Hunold <jhunold_at_gmx.eu
> <mailto:jhunold_at_gmx.eu>> wrote:
>
>     Hi Bjørn,
>
>     On Monday, 22. July 2013 16:18:25 Bjørn Roald wrote:
>
>      > Ok, it seems to me that lacking b2 support for generating header
>      > symlinks is the primary blocker for progress with git conversion now.
>      > Someone with with better knowledge of b2 than me need to step up.  At
>      > least I need some directions as to what I may try.
>
>     Seconded.
>
>
> OK.. What kind of support are we talking about? Is it just having
> symlink targets? A native symlink command? Something else?
>
> Sorry if I haven't followed this symlink problem carefully before. Just
> have a lot on my plate this Summer :-\

Not to worry, I am just trying to keep some motion on the issue, and 
feel some help is needed to get it right and then blessed.

A little bit about how it works:
--------------------------------
The main support for this has been created by Steven and this is 
committed to trunk.

r84524 | steven_watanabe | 2013-05-27 02:02:43 +0200 (Mon, 27 May 2013)
Initial supprort for creating symlinks in the git layout.

The basic idea as I understand based on observations and previous 
discussions is that libs/<your-lib>/include/boost
is a folder in the git structure where all boost headers belonging to 
the respective library is moved.  The intention is to have b2 provide 
some effective and hopefully transparent forwarding of these in the 
boost folder to have the traditional boost header layout recreated on 
the fly as needed.  I feel like JÅ©rgen's patches on top of r84524 are 
very close to what will be a usable solution, at least for supporting 
the transition to git.

There has clearly been put some effort into the solution. E.g. assuming 
z is an arbitrary boost library, I manage to get b2 to:

1.
Create symlink to boost/z from folder libs/z/include if that is the only 
contributer to boost/z and symlinks are supported. Else create regular 
catalog boost/z

2.
for files in libs/a/include/boost/ or sub-folders if respective target 
subfolder in boost/ are not symlinks;
        if hardlinks are supported, use hardlinks,
        else if symlinks are supported, use symlinks,
        else if symlinks are not supported, copy.

There are not really any forwarding headers in place here with 
redirecting #include directives.  That is good as long as there are 
links instead, but copy is not a great solution, but copy should be rear 
these days except maybe WindowsXP which is likely soon to go away.
The main problem with copies is that it is so easy to edit the wrong 
file, maybe making the copies read only will be a good hint/reminder. 
So it could be considered, but not essential now.

What is needed:
---------------
What is needed now is for something as close to and simple as the standard:

./bootstrap.sh
export HOME=`pwd`:$HOME
cd libs/<your-lib>/test/
b2

and

cd status
b2

These general work-flows need to work after git conversion. JŨrgen made 
the headers alias in Jamroot explicit on your request I think. So we 
have a minor difference there as we need to call

b2 headers

explicit.  Suggested workflow for experimenting is given here:
https://svn.boost.org/trac/boost/wiki/TryModBoost

I tried to remove the "explicit headers" line in Jamroot and it actually 
works very nice on the git side, so that may be an alternative for some. 
  Certainly eventually we can remove the explicit line.

some observations:
------------------
When running with headers alias implicit, As the libs projects build, 
they seems to only create the forwarding headers they depend when 
needed,  Some breakage (very little) to this, but I think that may be do 
to missing dependency declaration - in that case, nice.

with
alias headers : $(all-headers)-headers
                 numeric-$(numeric-headers)-headers : : : <include>.  ;
#explicit headers ;

in Jamroot we can build libraries from top level and run individual 
tests, but we are not able to run b2 in the status folder for regress 
test.  Steven suggested to use dependency, and changing the above to

alias headers : : <dependency>$(all-headers)-headers 
<dependency>numeric-$(numeric-headers)-headers : : <include>.  ;
#explicit headers ;

That fixes the problem in the status folder, but that breaks the build 
in the top catalog.  So we are stuck and don't really know how best to 
proceed.

I have done a little bit of testing the last few days, an ugly 
work-around that kind of works for me was to have both aliases, where I 
just renamed one to headers-a.  I think this may be tweaked to niceness 
on the front end, but probably a lot better ways ;-)

I also made some changes to fix detection of hardlink support and use of 
hard links. My patches are attached, These are my tweaked versions of 
JÅ©rgens patches.  I have only tested on Ubuntu Linux, Beman has tested 
the patches on Windows, I am not sure what JÅ©rgen is testing on.  We all 
seem to concur on results.

--
Bjørn