Re: [Boost-build] Boost build failed if custom toolset path is not available

Subject: Re: [Boost-build] Boost build failed if custom toolset path is not available
From: Edward Diener (eldiener_at_[hidden])
Date: 2016-02-24 19:05:07

On 2/24/2016 5:45 PM, Steven Watanabe wrote:
> AMDG
>
> On 02/24/2016 02:57 PM, Edward Diener wrote:
>> On 2/24/2016 3:39 PM, Steven Watanabe wrote:
>>>
>>> - Delaying the actual configuration to some
>>>     unspecified point will make any problems
>>>     that much harder to debug.  As things stand,
>>>     at least anyone can figure out why it's
>>>     failing: user-config.jam runs exactly what
>>>     you tell it to.
>>
>> Granted, but I do not think you realize how hard it is for the actual
>> end-user of Boost build to debug anything now given the complexity of
>> how Boost build decides anything, so the change will hardly affect
>> anyone but Boost build developers and experts.
>>
>
>    The OP apparently was able to diagnose
> the problem.
>
>>> - The implementation of laziness needs to
>>>     be handled separately by each toolset, as
>>>     we still need to do enough initialization
>>>     to determine when a toolset is "used"
>>
>> Why does running the "using xxx" command determine when a toolset is
>> used ? I would think that an actual specification of a toolset in a b2
>> command line determines when a toolset is used.
>>
>
> a) The first toolset specified is the default,
>     which will be run if there is no toolset=xxx
>     specified.
> b) The using declaration itself doesn't provide enough
>     information to distinguish between C++ toolsets
>     like gcc that are controlled by <toolset> and
>     other tools like xsltproc.  Without actually
>     running xxx.init, we can only treat xxx as
>     an opaque string, which is basically useless.
>
>>> - Instead of having two states, initialized
>>>     or not initialized, each toolset would have
>>>     three states: ready to use, initialized but
>>>     not fully configured, and not initialized.
>>>     This increase in complexity is likely to
>>>     propagate everywhere, possibly even into
>>>     user Jamfiles.
>>
>> Could you not just cache "using xxx" statements in memory and then when
>> it is determined that the "xxx" toolset is being used, invoke the "using
>> xxx" command line which you presently invoke when Boost build sees a
>> "using xxx", remove the "using xxx" from the cache, and then do whatever
>> else using a toolset involves for its particular case.
>>
>
>    The problem is with "when it is determined that
> the "xxx" toolset is being used."  The mapping
> from command line arguments to the toolset is
> set up by the call to using xxx ;.
>
>>
>>>     With that being said, this change is on my
>>> long-term todo list, but it's a fairly major
>>> project and is unlikely to happen in the
>>> immediate future.
>>
>> OK, thanks !
>>
>>  From this end-user's point of view, where various versions of
>> gcc/mingw(-64) on Windows and clang on Windows ( which depends on
>> mingw(-64)/gcc ) are mutually exclusive as far as needing that
>> particular version first on the Windows PATH, it would be a great
>> improvement if "using xxx" were not executed until xxx is actually being
>> used.
>
>    This particular problem can (and should) be
> solved by prefixing the gcc command with
> a command to set the path.  Unfortunately you
> can't solve this manually with:
>
> using gcc : : set "PATH=C:\\MinGW\\bin;%PATH%" && C:/MinGW/bin/g++ ;
>
> because the && gets quoted.

There are other ways to solve this problem ( command as a batch file 
which eventually invokes g++ or multiple user-config.jams with the 
correct one being set for any particular toolset ), but the entire point 
is that none of this should be needed if the "using xxx" were delayed 
until a toolset was specified ( or defaulted ).

>
>> The internal manipulations I have to do in order to run multiple
>> versions of gcc and/or clang on Windows for Boost build is ridiculous
>> and would be totally unnecessary if Boost build just didn't have to
>> execute every "using xxx" it sees in my user-config.jam when I am just
>> using a single version of gcc or clang in my b2 command line.
>>
>
>    There used to be a guideline that using xxx ; should
> always succeed even if xxx doesn't exist, but this seems
> to have been dropped by the wayside.
>
>> It would also speed up the execution of Boost build as "using xxx"
>> statements in places like user-config.jam would never get invoked the
>> vast majority of times when running b2 against jam files. I am not
>> talking of only compiler toolsets here but of many other toolsets in a
>> user-config.jam such as xsltproc, boostbook, doxygen, fop, quickbook,
>> auto-index, python, and whatever else, most of which will never be used
>> during a single invocation of Boost build using the b2 command line.
>>
>
> using xsltproc ; does two things:
> a) It checks that xsltproc exists, (on windows
>     I believe that it also checks whether it's
>     cygwin or windows, in order to adjust the
>     input paths correctly)
> b) It sets up internal Boost.Build structures
>     that describe when xsltproc should be run.
>
> (b) is unavoidable, as without it we wouldn't
> even recognize when we need to initialize xsltproc.

Do you mean to say that you wouldn't know whether or not to use xsltproc 
during a b2 invocation until you process a "using xsltproc" statement ?

>
>    What makes this a hard problem is that for
> complex toolsets like gcc, (a) and (b) are
> heavily intertwined.  i.e. it runs gcc to
> find the version number and this deduced
> version number can be used by the toolset
> selection algorithm.

But you never are going to use gcc unless the b2 invocation specifically 
tells you to use it or you are searching for a default compiler to use. 
In either case why could not processing of any "using gcc" statements be 
put off until you encounter either situation. Clearly in cases where the 
end-user is specifying a toolset that is not gcc something there would 
be no need to process a "using gcc" statement.

My objection to the way that Boost build currently works is that I can 
be passing a b2 command line like:

b2 toolset=msvc-12.0 ...etc.

and Boost build will still be processing a large list of "using xxx" 
statements in my user-config.jam which are completely irrelevant to my 
Boost build invocation, and stopping the invocation if any of these 
irrelevant "using xxx" statements fail.