Re: Technical Excellence?

From: Andrey Semashev (andrey.semashev_at_[hidden])
Date: 2024-04-02 16:08:22

On 4/2/24 17:27, Daniele Lupo via Boost wrote:
> On 02/04/2024 13:20, Andrey Semashev via Boost wrote:
>> No, not ok.
>>
>> Incompatibility with newer Boost releases means that the code is no
>> longer compatible with other code that *requires* the newer Boost. For
>> example, a library that uses boost::shared_ptr will be incompatible with
>> an application or another library that requires Boost version where
>> boost::shared_ptr has been removed.
> 
> Sorry, but I don't see the point here. What you're saying it's true, but
> it's the normality. If I write a program that links to Qt4, cannot be
> linked to another program that uses Qt6. Nobody complains it. If you
> want to link to another library/interface, you need to hide the
> libraries that you uses, for example with Pimpl, or expose the depedency
> if you need it in your public API and you have to require that the other
> library will use the same version.

Pimpl is not the solution, at least not on Linux, where publicly visible
symbols are linked across all modules in the process. Even if you hide
Boost X components behind Pimpl in one module, you still can't use Boost
Y in other parts of the process, unless you explicitly hide all symbols
coming from Boost X and Y. This means no interaction involving Boost
types, including exceptions, is allowed between the two parts of the
program.

> That's the reason for which, for
> example, some commercial software for which you can write plugins, gives
> the SDK to you, with the libraries that the company uses for building
> the program, forcing you to use the same version. MAK does it with
> Vr-Forces (you can find the qt version that they use in their site so
> you can download and use it), Unreal Engine does the same including all
> its dependencies in a ThirdParty folder that you download when you clone
> its repository.

Right, that's the problem with binary SDKs. That's usually not the
problem with open source software, since you can usually just rebuild it
with the dependencies you want. But this becomes problematic if you have
two dependencies that require different Boost versions.

> Also, the incompatibilities can arise even when the
> library is not removed, but only updated, in API or in ABI. So this is a
> no-problem for me: if you want to use code that use a specific boost (or
> other libraries) version, you must use that version or anyway a
> compatible version.

If API of some Boost library is updated, usage of that API in other
Boost libraries is also updated. So when you take a new Boost release,
you're receiving a package that works. It may require updating
downstream code, but nonetheless, you get a single Boost version that
works, as opposed to having multiple Boost versions, each working in
some but not all downstream projects.

>> There is also an issue of shipping the code that requires an older Boost
>> version in a Linux distro (and probably other OS distros), because
>> typically distros only ship one Boost version system-wide. This means
>> that either distro maintainers now have to ship multiple Boost versions
>> (which is a maintenance and technical problem) or the code in question
>> needs to be removed from distro and manually built by users who need it
>> (which is, again, a maintenance and technical problem shifted
>> downstream).
> 
> I'm not an expert in this, so you probably have a point here, but at the
> moment I'm looking at Ubuntu distro, and I can see that in the
> repository there are two different versions of boost:
> 
> apt-cache search --names-only "libboost(.*?)-all-dev"
> 
> return me version 1.74 and version 1.71.

Not on my system (it only shows 1.74 for me on Kubuntu 22.04).

I vaguely remember there was a time when two Boost versions were
packaged, but you couldn't install both versions of -dev packages at the
same time, for the reasons I described. This allows you to build
software that depends on *one* Boost version at a time, but not both.
Meaning, you still cannot combine the two versions in one project.

>> Note that building old Boost on a newer system may be problematic by
>> itself due to updated dependencies. For example, if the old Boost was
>> only compatible with OpenSSL 1.0 and the newer system has migrated to
>> OpenSSL 3.0, you won't be able to build without extensive patching.
> 
> Again, that's not a problem in my opinion. This is the same dependency
> problem handled and solved in many ways. For example, with vcpkg, you
> should be able to download a specific version of a library, and build
> compatible version dependencies. I don't know if this is already done
> with boost, but it works with other libraries. If you need to work with
> specific versions of a library, especially an old one, you're not
> usually using the system-wide installed libraries, but old ones, so it's
> normal to use them together. If you use the old boost version, you
> should also use the old openssl version. I don't see the problem. I see
> more problems in trying to maintain this overhelming compatibilities
> between many library versions.

The problem is that the old OpenSSL version is no longer shipped by the
distro. Distro maintainers are generally not happy about having to drag
multiple versions of the same software, as it has an ongoing maintenance
cost.

>>> My idea, as I've said, is to deprecate old libraries at some point,
>>> remove them later in a future version, and maintain old boost version
>>> for bugfixing.
>> That's also problematic. The current Boost workflow is not well suited
>> for maintenance releases, let alone maintenance of older branches of
>> Boost. There are no older branches. Every Boost release is made from
>> master, and represents its current state, with all bug fixes, new
>> features and removals, should those happen.
> The branch can be unique, but we can have different tags. If we are at
> version 1.90, and there's a problem in 1.88, we can simply create a new
> branch starting from the 1.88 release commit, named for example
> "xxx-hotfix", where xxx is the issue, solve it, test it, and commit.

It's not only about a branch in the library, it requires changing the
logic how the superproject works, how testing is done across all
libraries, and probably how release packaging is done. There's
organizational part as well, e.g. how often do we make point releases
and who's going to manage them. It's not impossible to do, but it surely
doesn't look trivial either.