Boost logo

Boost :

Subject: Re: [boost] Git hardlinks, and developing
From: Bjørn Roald (bjorn_at_[hidden])
Date: 2013-12-04 19:50:24

On 12/04/2013 10:13 PM, Beman Dawes wrote:
> On Wed, Dec 4, 2013 at 3:12 PM, Bjørn Roald <bjorn_at_[hidden]> wrote:
> Some thoughts:
>> Assuming we have changed b2 headers to prefer symlink for individual files:
> Before we change anything I'd prefer to see a set of requirements
> developed. What scenarios do we expect to work out-of-the-box on what
> platforms?
> I'd like to see highest priority given to recent versions of the most
> common platforms used by Boost developer. I.E. Linux, Mac OS X, Windows.
> And getting the scenarios to work on recent versions before worrying about
> versions already abandoned by their vendors. Just my opinion, of course.

OK, I give it a try. The list below is sort of in my priority order.
The words /shall/ is stronger than /should/, and /may/ is optional:

For simplicity I call file operated on when accessed from within the
BOOST_ROOT/boost folder a *proxy header* or simply *proxy* of the
*original* header, i.e.; all the "b2 headers" build target is doing is
staging a bunch of proxy headers.

Each requirement is followed by a compliance section with my
understanding of current and potential compliance. All potential
compliance require some work.

Please comment!

Requirement: BPH-01 Essential
Calling "b2 headers" explicitly shall ensure all proxy headers have
identical content of respective originals.
Compliance: b2 headers
                           current potential
        symlink folder: YES YES
        symlink proxy : YES YES
        hardlink proxy : YES/NO(1) YES(5)
        copy proxy : YES/NO(2) YES(6)

Requirement: BPH--02 Desired - high priority
Accidental (or casual) editing of proxy header files in boost folder
should have the immediate effect of updating the respective original

Compliance: b2 headers
                           current potential
        symlink folder: YES YES
        symlink file : YES YES
        hardlink file : YES/NO(1)(3) YES/NO(1)
        copy of file : NO NO

Requirement: BPH-03 Desirable
Changes to original header file should have the effect of updating the
respective proxy immediately.

                             current potential
        symlink folder: YES(4) YES
        symlink file : YES(4) YES
        hardlink file : YES(1)(4) YES/NO(3)
        copy of file : NO NO

Requirement: BPH-04 Desirable (short term) / Essential (longer term)
Any build with b2 that depend on a headers in boost shall ensure those
headers have identical content of respective originals in
libs/*/include/boost folder.

Note: longer term goal to get rid of need to do explicit b2 headers

Compliance: b2 headers
                               current potential
        symlink folder: NO(3) YES
        symlink proxy : NO(3) YES
        hardlink proxy : NO(1)(3) YES
        copy proxy : NO(2)(3) YES

Requirement: BPH-05 Desired
When git commands change content in working directory, hooks may be
used to call b2 headers.

Compliance: potentially YES, if b2 headers is called by
                git hook, then compliance would be as in BPH-01.

Requirement: BPH-06 Desired / Optional
Building with b2 may remove orphan headers in boost folder.

Note: Alternatives such as b2 clean-headers may be good enough and
       more feasible.

Compliance: b2 headers
                           current potential
                            NO YES

Requirement: BPH-06 Desirable
b2 builds may warn the user if the system does not support robust
features to manage header proxies and point to a web page with advise.
E.g.: if b2 headers have to use hard links or copies.

Compliance: current potential
                               NO YES

(1) Git and potentially some other editors and tools may change
     one end in hardlink so it will break automatically reflective
     update of the other end. E.g.: deleting the link, then create
     a new file in its place, thus causing the other end to become
     a regular copy of the original file.

(2) If file time of libs/*/include/boost header is older than
     proxy header, then current b2 header solution will fail.

(3) b2 headers does currently not have a complete dependency
     graph with productions of individual proxies that depend
     their respective originals.

(4) b2 headers must have been run once to set up symlink proxies

(5) b2 may always replace all hardlink proxies, thus fixing
     #(1) issues from the original side, e.g. git braking hardlinks.

(6) b2 may always copy original files to copy proxies if b2 detect file
     content mismatch.

I think changing b2 headers preference for proxy methods is the best
step 1, given also that symlink is more or less universally available on
newer developer host OS'es, given some teaks may be needed on Windows
for privileges.

I suggest changing current:

1. symlink dir
2. hardlink file
3. symlink file
4. copy

preference to:

1. symlink dir
2. symlink junktion dir
3. symlink file
4. hardlink file
5. copy

That would take care of BHP-01, BHP-02, BHP-03, for most users.
Then I would actually do BHP-06, as it is simple and would
hopefully warn those that are not as lucky, and potentially help them
fix it or understand and be cautious,


Boost list run by bdawes at, gregod at, cpdaniel at, john at