Re: Technical Excellence?

From: Daniele Lupo (jepessen_at_[hidden])
Date: 2024-04-02 14:27:58

On 02/04/2024 13:20, Andrey Semashev via Boost wrote:
> No, not ok.
>
> Incompatibility with newer Boost releases means that the code is no
> longer compatible with other code that *requires* the newer Boost. For
> example, a library that uses boost::shared_ptr will be incompatible with
> an application or another library that requires Boost version where
> boost::shared_ptr has been removed.

Sorry, but I don't see the point here. What you're saying it's true, but 
it's the normality. If I write a program that links to Qt4, cannot be 
linked to another program that uses Qt6. Nobody complains it. If you 
want to link to another library/interface, you need to hide the 
libraries that you uses, for example with Pimpl, or expose the depedency 
if you need it in your public API and you have to require that the other 
library will use the same version. That's the reason for which, for 
example, some commercial software for which you can write plugins, gives 
the SDK to you, with the libraries that the company uses for building 
the program, forcing you to use the same version. MAK does it with 
Vr-Forces (you can find the qt version that they use in their site so 
you can download and use it), Unreal Engine does the same including all 
its dependencies in a ThirdParty folder that you download when you clone 
its repository. Also, the incompatibilities can arise even when the 
library is not removed, but only updated, in API or in ABI. So this is a 
no-problem for me: if you want to use code that use a specific boost (or 
other libraries) version, you must use that version or anyway a 
compatible version.

> There is also an issue of shipping the code that requires an older Boost
> version in a Linux distro (and probably other OS distros), because
> typically distros only ship one Boost version system-wide. This means
> that either distro maintainers now have to ship multiple Boost versions
> (which is a maintenance and technical problem) or the code in question
> needs to be removed from distro and manually built by users who need it
> (which is, again, a maintenance and technical problem shifted downstream).

I'm not an expert in this, so you probably have a point here, but at the 
moment I'm looking at Ubuntu distro, and I can see that in the 
repository there are two different versions of boost:

apt-cache search --names-only "libboost(.*?)-all-dev"

return me version 1.74 and version 1.71. So it's possible (and I think 
mandatory) for a distro to maintain and use different versions of a 
library. If you want to install programs from repository that use 
different version of boost, they will install the corresponding 
dependencies.

> Note that building old Boost on a newer system may be problematic by
> itself due to updated dependencies. For example, if the old Boost was
> only compatible with OpenSSL 1.0 and the newer system has migrated to
> OpenSSL 3.0, you won't be able to build without extensive patching.

Again, that's not a problem in my opinion. This is the same dependency 
problem handled and solved in many ways. For example, with vcpkg, you 
should be able to download a specific version of a library, and build 
compatible version dependencies. I don't know if this is already done 
with boost, but it works with other libraries. If you need to work with 
specific versions of a library, especially an old one, you're not 
usually using the system-wide installed libraries, but old ones, so it's 
normal to use them together. If you use the old boost version, you 
should also use the old openssl version. I don't see the problem. I see 
more problems in trying to maintain this overhelming compatibilities 
between many library versions.

> (By "technical problem" above I mean that two different Boost versions
> may not be possible to install on the same system using the standard
> package manager. Yes, shared libraries can coexist via version suffixes
> in library names, but this doesn't work for headers and static
> libraries. Fixing this would require renaming libraries and header
> directories and would break builds of everything downstream, so
> basically won't happen.)

It's true, if I try to install both of them I've errors, like you say 
and this is correct, but someway it can be solved. Since there are 
breaking changes for me it's ok to update the major version of boost 
every time that a library is removed, from boost1 to boost2 for example, 
and then the maintainers should be able to handle this. It's a bit of 
work, you're right, but it's no different from the kind of work that 
they already do for other libraries.

If I want to develop a software with the system-wide libraries, I need 
to select one version and use it. If I want to use another version, I 
need to install the correct one, or I need to install the version that I 
need with a package manager. I can see some problem here, you're right, 
but nothing that cannot be solved with a bit of configuration.

>> My idea, as I've said, is to deprecate old libraries at some point,
>> remove them later in a future version, and maintain old boost version
>> for bugfixing.
> That's also problematic. The current Boost workflow is not well suited
> for maintenance releases, let alone maintenance of older branches of
> Boost. There are no older branches. Every Boost release is made from
> master, and represents its current state, with all bug fixes, new
> features and removals, should those happen.
The branch can be unique, but we can have different tags. If we are at 
version 1.90, and there's a problem in 1.88, we can simply create a new 
branch starting from the 1.88 release commit, named for example 
"xxx-hotfix", where xxx is the issue, solve it, test it, and commit. 
Then we create the new tag 1.88.1 in the fixed commit and, if necessary, 
we merge this commit in the main branch, or in later releases (creating 
also 1.89.1 in this case). It's something that can be easily done, in a 
way similar to what git workflow does. I don't think that this can break 
the existing development logic, it's simple a simple addition that must 
be used only if necessary.I don't expect that this will happen ofter, 
it's not a normal developmente workflow, we need to do it only if 
necessary, but in the few cases that it's necessary to manage, we know 
what to do.
> Then there is the added maintenance burden of the older branches, if we
> were to create them.
>
> So there's another problem to the "old code uses old Boost" approach: we
> don't do point releases. I'm going to leave the discussion on whether
> this is good or bad aside, but this means if there's a bug or security
> vulnerability in the older Boost version, the old code is either stuck
> with it or has to patch their Boost themselves.
>
> So my main point is that "old code uses old Boost" is a myth in practice.

I don't think that's a myth. It can be achieved easily by adapting a bit 
the development, putting tags in correct commits and creating hotfix 
branches like many other project already do by using this logic for 
example A successful Git branching model » nvie.co 
<https://nvie.com/posts/a-successful-git-branching-model/>. If people 
doesn't want to do it because it's not used to do it, fine by me, but 
it's not a technical issue.

If you decide that we don't need this, it's fine by me, but in my 
experience at some point every project needs a refactoring and a 
re-thinking, by removing old and unused code and try to modernize it a 
bit. The more we postpone this step, the more problems we will have.

Regards

Daniele Lupo