Re: [boost] proposal - modularize Boost build system

Subject: Re: [boost] proposal - modularize Boost build system
From: Raffi Enficiaud (raffi.enficiaud_at_[hidden])
Date: 2017-06-18 19:30:53

Le 18.06.17 à 21:05, Stefan Seefeld via Boost a écrit :
> On 18.06.2017 14:48, Raffi Enficiaud via Boost wrote:
>> Le 18.06.17 à 18:09, Stefan Seefeld via Boost a écrit :
>>> One of Boost's bottlenecks is its monolithic build infrastructure. It
>>> would be beneficial for many different use cases (testing, packaging,
>>> etc.) to decouple the building of Boost libraries (well, those that
>>> require building, see the listing in
>>> http://www.boost.org/doc/libs/1_64_0/more/getting_started/unix-variants.html#header-only-libraries)
>>>
>>> such that they may optionally be built stand-alone.
>>>
>>> At present, building a Boost library requires the entire
>>> (super-)repository to be checked out, and the build logic itself
>>> involves traversing the entire source tree, looking for a "Jamroot".
>>>
>>> What would be beneficial to many would be a workflow like this:
>>>
>>> 1) Have a development environment with (some version of) Boost
>>> pre-installed (or at least the parts that the library being built
>>> depends on).
>>
>> If you want to easily distribute boost libraries independently, having
>> boost preinstalled (or already cloned or already downloaded from a
>> release) is not exactly what I would like to see as a user.
>
> You are right. As was pointed out before, "user" stands for two distinct
> roles:
>
> * "downstream boost developer" using an "upstream boost library"
> * developers of other software using any boost library
>
> My proposal is more targeted to benefit the former. While independent
> release cycles would obviously also impact the latter. But decoupling
> releases is an entirely separate discussion, which I'd rather not get
> into in the context of this proposal.
>
>> As a library maintainer, I can say that boost.test is having hard time
>> to compete with other unit testing libraries such as Catch or google
>> test, not because of the number of files in boost.test itself, because
>> you need to pull a full boost release to run it, which is another
>> level of magnitude.
>
> Precisely, this is exactly one of the things modularization may improve.
>
>> Packaging is out of the user story to my opinion, and there are nice
>> people very good at this. Also if installation is easy, packaging is
>> less important I would say (IMO).
>
> Not true in practice. Consider a developer (of the second category
> above) on Linux. Different distributions use different ways to split
> Boost libraries into packages, so to be portable, I need to check for
> them all. That's a huge maintenance burden which could be solved if some
> basic support for packaging would be built right into the Boost build
> system (be it Boost.Build or something else).

Maybe I misunderstood, but I thought that you were suggesting that we 
include the possibility to package boost or any of its component 
"easily" in boost itself.

First of all, my understanding of packaging is: I can create "packages" 
that indicate their dependencies and the way they can be built such that 
I can install a library on my OS and remove it afterwards (like .deb, 
pip install, and in some extent brew).

To me this goes to the distros/OS packager desk, like the Debian 
packagers or brew people. Also for some, whatever good packaging support 
you provide them with, they will just not use and use their own system 
instead, because it is stable and they are comfortable with. So whatever 
initiative we take, IMO, within boost concerning that is a bit useless.

I see packaging as hard, I do not want to do it, not even starting 
thinking of it :) Also for users, packaging is a function taking as 
first parameter the OS version, rather than the boost version. There are 
things like PPA but this is not really helping the end user I have to say.

> Of course you could argue that in many real-world cases even that
> use-case isn't very common, as lots of commercial software is built by
> cloning Boost and building that in-house. I think that's very
> unfortunate if unavoidable, given the current state of things. See my
> other proposal (about stability / compatibility) for how to improve
> things on that front.

Well, after almost two decades of boost, I think now (at least on linux) 
ppl know that there is no ABI compatibility among minor versions of 
boost, but only between patch version (and since there is barely patch 
release ...). Which means that the way boost make version number is just 
different from the "convention" (whatever convention means).

To sum up, I would rather limit the discussion at the "installation" 
level. As a user of boost.whatever, I want to install boost.whatever 
once and use it many times. I think this impacts the two notions of 
users you mentioned.

>> There is also something a bit awkward I wanted to indicate, but we
>> just forgot to mention one of the most important tools so far: the
>> **compiler**.
>>
>> We cannot achieve any modularity if there is no agreement on what
>> should be supported, and this definitely is the most important
>> variable to take into account in the dependency graph of the
>> boost.whatever libraries. If I consider only boost.test, a lot of
>> dependencies can just be removed just by considering C++11, which in
>> turn would lower dramatically the user overhead in order to use the
>> library.
>
> True. But again, let's not kill the discussion by widening the scope
> further. For the sake of the discussion let's assume that the compiler
> is a pre-defined constant, rather than an additional (almost free)
> parameter.

Right. Let's call the pruning of the dependency graph an optimization.