Re: [boost] Git hardlinks, and developing

Subject: Re: [boost] Git hardlinks, and developing
From: Bjørn Roald (bjorn_at_[hidden])
Date: 2013-12-04 19:50:24

On 12/04/2013 10:13 PM, Beman Dawes wrote:
> On Wed, Dec 4, 2013 at 3:12 PM, Bjørn Roald <bjorn_at_[hidden]> wrote:
>
> Some thoughts:
>>
>> Assuming we have changed b2 headers to prefer symlink for individual files:
>>
>
> Before we change anything I'd prefer to see a set of requirements
> developed. What scenarios do we expect to work out-of-the-box on what
> platforms?
>
> I'd like to see highest priority given to recent versions of the most
> common platforms used by Boost developer. I.E. Linux, Mac OS X, Windows.
> And getting the scenarios to work on recent versions before worrying about
> versions already abandoned by their vendors. Just my opinion, of course.

OK, I give it a try.  The list below is sort of in my priority order. 
The words /shall/ is stronger than /should/, and /may/ is optional:

For simplicity I call file operated on when accessed from within the 
BOOST_ROOT/boost folder a *proxy header* or simply *proxy* of the 
*original* header, i.e.; all the "b2 headers" build target is doing is 
staging a bunch of proxy headers.

Each requirement is followed by a compliance section with my 
understanding of current and potential compliance.  All potential 
compliance require some work.

Please comment!

Requirement: BPH-01 Essential
Calling "b2 headers" explicitly shall ensure all proxy headers have 
identical content of respective originals.
        
Compliance:                     b2 headers
                           current      potential
        symlink  folder:     YES          YES
        symlink  proxy :     YES          YES
        hardlink proxy :     YES/NO(1)    YES(5)
        copy proxy     :     YES/NO(2)    YES(6)

Requirement:  BPH--02 Desired - high priority
Accidental (or casual) editing of proxy header files in boost folder 
should have the immediate effect of updating the respective original
accordingly.

Compliance:                     b2 headers
                           current         potential
        symlink  folder:     YES            YES
        symlink  file  :     YES            YES
        hardlink file  :     YES/NO(1)(3)   YES/NO(1)
        copy of file   :     NO             NO

Requirement:  BPH-03  Desirable
Changes to original header file should have the effect of updating the 
respective proxy immediately.

Compliance:
                             current     potential
        symlink  folder:     YES(4)       YES
        symlink  file  :     YES(4)       YES
        hardlink file  :     YES(1)(4)    YES/NO(3)
        copy of file   :     NO           NO

Requirement:  BPH-04 Desirable (short term) / Essential (longer term)
Any build with b2 that depend on a headers in boost shall ensure those 
headers have identical content of respective originals in 
libs/*/include/boost folder.

Note: longer term goal to get rid of need to do explicit b2 headers
       build

Compliance:                         b2 headers
                               current         potential
        symlink  folder:       NO(3)            YES
        symlink proxy  :       NO(3)            YES
        hardlink proxy :       NO(1)(3)         YES
        copy proxy     :       NO(2)(3)         YES

Requirement:  BPH-05 Desired
When git commands change content in working directory, hooks may be
used to call b2 headers.

Compliance:    potentially YES, if b2 headers is called by
                git hook, then compliance would be as in BPH-01.
        

Requirement:  BPH-06 Desired / Optional
Building with b2 may remove orphan headers in boost folder.

Note: Alternatives such as b2 clean-headers may be good enough and
       more feasible.

Compliance:                 b2 headers
                           current      potential
                            NO            YES

Requirement:  BPH-06  Desirable
b2 builds may warn the user if the system does not support robust 
features to manage header proxies and point to a web page with advise. 
E.g.: if b2 headers have to use hard links or copies.

Compliance:                 current     potential
                               NO           YES

End-notes:
(1) Git and potentially some other editors and tools may change
     one end in hardlink so it will break automatically reflective
     update of the other end. E.g.: deleting the link, then create
     a new file in its place, thus causing the other end to become
     a regular copy of the original file.

(2) If file time of libs/*/include/boost header is older than
     proxy header, then current b2 header solution will fail.

(3) b2 headers does currently not have a complete dependency
     graph with productions of individual proxies that depend
     their respective originals.

(4) b2 headers must have been run once to set up symlink proxies

(5) b2 may always replace all hardlink proxies, thus fixing
     #(1) issues from the original side, e.g. git braking hardlinks.

(6) b2 may always copy original files to copy proxies if b2 detect file
     content mismatch.

I think changing b2 headers preference for proxy methods is the best 
step 1, given also that symlink is more or less universally available on 
newer developer host OS'es, given some teaks may be needed on Windows 
for privileges.

I suggest changing current:

1. symlink dir
2. hardlink file
3. symlink file
4. copy

preference to:

1. symlink dir
2. symlink junktion dir
3. symlink file
4. hardlink file
5. copy

That would take care of BHP-01, BHP-02, BHP-03, for most users.
Then I would actually do BHP-06, as it is simple and would
hopefully warn those that are not as lucky, and potentially help them 
fix it or understand and be cautious,

--
Bjørn
--
Bjørn