Re: [boost] [modular-boost] Development procedures

Subject: Re: [boost] [modular-boost] Development procedures
From: Bjørn Roald (bjorn_at_[hidden])
Date: 2014-02-03 19:35:10

On 02/03/2014 10:37 PM, Edward Diener wrote:
> On 2/3/2014 3:49 PM, Bjørn Roald wrote:
>> On 02/03/2014 05:29 PM, Edward Diener wrote:
>>> On 2/3/2014 11:00 AM, Vyacheslav Andrejev wrote:
>>>> Hello Gennadiy,
>>>>
>>>>>> can run "b2 headers" again. If we were using soft links we wouldn't
>>>>>> have this problem, I thought there was consensus to do that, but no
>>>>>> one seems to be doing it.
>>>>>>
>>>> GR> There is no soft links on Windows.
>>>>
>>>> 1. There are soft links in Windows. See documentation for mklink /D.
>>>
>>> mklink without any option creates a file symbolic link.
>>>
>>>> 2. This is exactly what b2 tries to do: create soft links. Only it
>>>> fails, since in Windows you need elevated privileges by default to do
>>>> this. Then b2 falls back to creating hard links to files, since it
>>>> cannot create soft links to directories.
>>>
>>> It does not test whether the user has administrator priveleges or not on
>>> Windows in order to create file symbolic links.
>>
>> What test are you suggesting and what should b2 do given each result of
>> that test?
>
> I do not know. I only know that as a user with administrator rights on
> Windows 7 that I can create symbolic links.

OK, but that does not help anything.  Given that b2 checked your 
privileges, it would not provide any information it does not already 
have by simply testing if it succeeds making symbolic and hard links.

> Yet when using 'b2 headers'
> the links to files in the modular-boost boost subdirectory are hard links.

Yes, and b2 does not need to do any new test to gain the needed 
information.  It already knows if symbolic links works or not.  T

>>> I have brought this issue many times before in various posts. The 'b2
>>> headers' command should always create symbolic links if possible, not
>>> hard links which are problematical in our modular-boos setup. Each time
>>> I have brought up this issue I have been told we are waiting for
>>> modular-boost issues itself to be fully resolved. We are evidently still
>>> waiting.
>>
>> This thread is more about credentials on windows than the more general
>> questions we have debated on the preference on use of hardlinks over
>> symlinks to files in current "b2 headers".  I think there are more or
>> less consensus that it will be better to use symlinks also for files,
>> but Beman asked for requirements and time to let the modularization dust
>> settle before we made changes.  I have not got any feedback on the
>> requirements I posted, and there have as far as I have noted been a few,
>> but not many reported real problems with hardlinks.
>
> The problem with hardlinks has already been reported and commented on.

I think I said pretty much the same.

> Do I or anyone else really need to explain this again vis-a-vis
> modular-boost ?

Not to me or anybody else as far as I am aware.  There are still some 
caution as there is a lot of changes for a lot of people, so risking 
breaking something the build which sort of works right to fix this is 
not a priority.

>>  So rushing this may
>> not be warranted.  Nevertheless, that should not prevent you from trying
>> preference for symlinks in "b2 headers" if you need to, and report
>> results.
>
> Why not have someone knowledgable with Boost Build actually make the
> changes in the 'develop' branch to 'b2 headers' so that symbolic links
> get created on Windows rather than hard links when the end-user's
> administrator priveleges allow it ?

Yes expect I don't think anybody can just have somebody else doing what 
they like around here.  That is not how it works.

> I would be glad to test this out. I
> assume that you may be knowledgeable enough with Boost build to do this,
> but I am not.

Well, I am an ignorant bjam hacker, but if you are brave and edit 
tools/build/src/tools/link.jam, you will find:

rule do-file-link
{
     local target = [ path.native [ path.relative-to [ path.pwd ] $(<) ] ] ;
     local source = [ path.native [ path.relative-to [ path.pwd ] $(>) ] ] ;
     LOCATE on $(target) = . ;
     DEPENDS $(.current-target) : $(target) ;
     if $(.can-hardlink) = true
     {
         DEPENDS $(target) : $(source) ;
         link.hardlink $(target) : $(source) ;
     }
     else if $(.can-symlink) = true
     {
         link.mklink $(target) : $(source) ;
     }
     else
     {
         DEPENDS $(target) : $(source) ;
         common.copy $(target) : $(source) ;
     }
}

so simply swap the order of the:

     if $(.can-hardlink) = true
     {
         DEPENDS $(target) : $(source) ;
         link.hardlink $(target) : $(source) ;
     }

and the:

     if $(.can-symlink) = true
     {
         link.mklink $(target) : $(source) ;
     }

blocks should do it -- I think.

>> End-note:  this thread started with a problem related to symlinks not
>> working as expected on Windows, so the changes you say we are waiting on
>> would likely not resolve anything here.
>
> It started  with Gennadiy Rozental asking about a number of issues with
> Modular Boost, one of which is:
>
> "1. Why does ./b2 headers on windows creates hardlink for each file instead
> of directories? The process takes forever."

The change above, which I think is what you have asked for all along 
will not change Gennadiy's slow "b2 headers" build. To speed that up he 
needs to change credentials to get symlinks to work.  And the key to 
improved performence is symlinks to directories, not to files.  I would 
be surprised if the change above will improve performance at all, I sort 
of expect it to be a tad slower, but I may be wrong.

-- 
Bjørn