Re: [Boost-build] Building Boost with multiple python versions

Subject: Re: [Boost-build] Building Boost with multiple python versions
From: Stefan Seefeld (stefan_at_[hidden])
Date: 2017-02-15 16:50:18

On 15.02.2017 11:39, Rene Rivera wrote:
> On Tue, Feb 14, 2017 at 5:15 PM, Stefan Seefeld via Boost-build
> <boost-build_at_[hidden] <mailto:boost-build_at_[hidden]>> wrote:
>
>     On 13.02.2017 23:53, Rene Rivera via Boost-build wrote:
>     > Warning: This is only a quick reply after some investigation. I
>     would
>     > have to delve deeper to figure out more of the why things are
>     the way
>     > they are (for better or worse)..
>     >
>     > On Mon, Feb 13, 2017 at 9:35 AM, Stefan Seefeld via Boost-build
>     > <boost-build_at_[hidden] <mailto:boost-build_at_[hidden]>
>     <mailto:boost-build_at_[hidden]
>     <mailto:boost-build_at_[hidden]>>> wrote:
>     >
>     >     Hello,
>     >
>     >     having learned a bit more about Boost.Build and its
>     internals, I'm
>     >     trying to apply my understanding.
>     >
>     >
>     >     My user-config.jam file contains
>     >
>     >       using python : 2.7 ;
>     >       using pyhon : 3.5 ;
>     >
>     >     Then I issue this command:
>     >
>     >       ./b2 --with-python python=2.7
>     >
>     >
>     >     Inside my Jamfile, I see that `feature.values python`
>     reports "2.7
>     >     3.5",
>     >     independently on what I specify on the command-line.
>     >
>     >
>     > The "python" feature is composed of *all* the configured version of
>     > python.
>
>     I find that quite counter-intuitive.
>
>
> It's how non-free features work. They are a set of values. 
>
>     >     And what does it mean to provide the `python=2.7` option on the
>     >     command-line ?
>     >
>     >
>     > It means to add the "<python>2.7" as a build request to match
>     against
>     > the targets to build.
>
>     OK, makes sense, too. So let me paraphrase, to make sure I understand
>     the idea:
>
>     Having the two "using python" lines above means that those two python
>     configurations are made available for future reference (in
>     Jamfiles, as
>     well as command-line). It does *not* imply that both are being
>     used (as
>     feature values) automatically (i.e. not without some other code
>     elsewhere to specify that "all" is indeed what the Jamfile requests.)
>
>
> Precisely correct. 
>
>     >     How can I specify that I want to
>     >
>     >     * build with Python 2.7 only
>     >
>     >
>     > You can't [1].
>     >
>     >
>     >     * build with Python 3.5 only
>     >
>     >
>     > You can't [1]
>     >
>     >
>     >     * build both variants at the same time
>     >
>     >
>     > [1] The way the BPL build file is written by default it builds
>     both a
>     > 2 and 3 version of the library if both a 2 and 3 python version are
>     > configured. Now.. Theoretically it would be possible that specifying
>     > the version on the command would limit it to just that version to
>     > build. But it seems that the way the python.jam module sets up
>     python
>     > libraries it fails to trigger the b2 target requirements selection
>     > mechanics. Or simply put.. It's broken. And I don't know
>     entirely why
>     > yet. I'll keep looking at it so see if I can resolve this better
>     > eventually.
>
>     OK, so can we agree that this is a bug (no matter where) ? After all I
>     have read about b2, it would seem logical that the above amounts to
>     declaring a "python" feature, with two possible values, such that a
>     command-line option of "python=2.7" would then pick one of the two.
>
>
> After more investigation.. Calling it a bug is an understatement! It's
> so broken I haven't figured out why it's broken. The current symptoms,
> even after some cleanup on my part of the python build file is as follows:
>
> (a) If a py2 interpreter is configured BPL builds py2 libraries that
> link against python2 runtime.
> (b) If a py2 and py3 interpreter is configured BPL builds (a) and py3
> libraries that link against **python2** runtime!

That's exactly the bug that was reported to me in a couple of places,
and which I had no idea how to solve.

> (c) If only a py3 interpreter is configured BPL builds py3 libraries
> that link against python3 runtime.
>
> Read that at least twice and contemplate the implications of how
> broken that makes everything be. I'm trying to fix it. But the results
> defy expectations of how b2 is supposed to work. So it's taking some
> effort to track down the root of the problems.

Thanks a lot for looking into this !

>
>     However, this example raises another question (already asked by
>     Tom): As
>     in the above the discriminating label is the version number, it isn't
>     clear how one can declare multiple python configurations that use the
>     same version. So if, as per Tom's example, I configure two Python 2.7
>     variants (one for 32-bit and one for 64-bit builds), how can I
>     refer to
>     them unambiguously ? The version string clearly isn't enough. Is that
>     also a bug / limitation in the python.jam file ?
>
>
> You can specify multiple features in the command like to indicate the
> collective requirement request of what you want to build. Hence if you
> configure the python interpreter with conditions that match that build
> request it should use that interpreter only. That's assuming it wasn't
> broken though :-(

OK. But all existing bugs in the implementation aside, I think this is
more of a UI concern: If I understand correctly, the "python" feature
expects values that express versions (so I can select a python config on
the command line with "<python>2.7".
my question is about what happens if the version alone isn't an
unambiguous discriminator, i.e. if I need to distinguish between a
64-bit python and a 32-bit python, both using the same version 2.7 ? How
can I express that, both on the command line as well as in my
user-config.jam configuration with the `using` rule ?

Thanks,
        Stefan

-- 
      ...ich hab' noch einen Koffer in Berlin...