Re: Interest for C++20 modules support of boost officially

From: Ruben Perez (rubenperez038_at_[hidden])
Date: 2024-12-22 09:38:52

On Fri, 20 Dec 2024 at 03:15, Chuanqi Xu <chuanqi.xcq_at_[hidden]> wrote:
>
> Hi Ruben,
>
>     I think we’re in the same page except a few wording issues. e.g, I won’t call it as pre-compiled headers. But this doesn’t matter.
>
> > 2. Boost.Url in the module world would install:
> >   * libboost_url.a, containing the same function definitions as above.
> >   * A boost_url.cppm file, akin to what headers are today.
> >    CMake would build a BMI from this file when a user needs to
> import boost.url.
> >     In practice, this cppm file will likely be implemented in terms of the
> >     header files that we have today.
>
> The libboost_url.a with modules support may have the same entities with original libboost_url.a **plus** the module initializer.
>
> And also, as discussed in other mail, the headers will be installed too.

Yes, completely. I'm building a small prototype to show how things
would work, as discussing with concrete code is much more fruitful.

>
> > Unfortunately, CMake doesn't seem to have first-class support
> for this, meaning that the user would need to figure out dependencies
> manually.
>
> What do you mean by figuring out dependencies manually? Do you mean, in cmake, the dependencies will be calculated if the libraries are linked? If yes, I feel it might be fine. Or did I misunderstand it?

Consider Boost.variant2. It's a header-only library that depends on
Boost.assert, Boost.config and Boost.mp11, which are also header-only.
This is what the CMake code to build these as modules could look like
(simplified - I will post the entire prototype when ready):

add_library(boost_variant2)
target_sources(boost_variant2 PUBLIC FILE_SET CXX_MODULES FILES
modules/variant2.cxx)
target_include_directories(boost_variant2 PUBLIC include)
target_link_libraries(boost_variant2
    PUBLIC
      Boost::assert
      Boost::config
      Boost::mp11
)

Here, Boost::assert and Boost::mp11 have been created with similar
code as the one shown above. Boost::config is an interface library
with only headers (as it only exports macros).

The targets are installed by the Boost.CMake infrastructure, with a
call akin to:

install(TARGETS ${LIB} EXPORT ${LIB}-targets
  RUNTIME DESTINATION "${CMAKE_INSTALL_BINDIR}"
  LIBRARY DESTINATION "${CMAKE_INSTALL_LIBDIR}"
  ARCHIVE DESTINATION "${CMAKE_INSTALL_LIBDIR}"
  PRIVATE_HEADER DESTINATION "${CMAKE_INSTALL_INCLUDEDIR}"
  PUBLIC_HEADER DESTINATION "${CMAKE_INSTALL_INCLUDEDIR}"
  FILE_SET CXX_MODULES DESTINATION .
)

When building Boost with CMake, for each module (Boost::assert,
Boost::mp11 and Boost::variant2), a BMI and a static library with the
module initializer is built.
When installing Boost (with CMake), for each module (Boost::assert,
Boost::mp11 and Boost::variant2), the headers, module code and static
library is installed.
When consuming Boost from another project, e.g. using
find_package(Boost COMPONENTS Boost::variant2), the headers, module
code and static library for Boost::variant2 and its dependencies are
found and used. The BMIs are built in the consumer project. Right now,
the flags used originally to build Boost are used to re-build the
BMIs. This causes trouble, but is intended to change in the future,
with an unknown time frame [1].

Following this approach, dependencies are automatic. But this has a
number of downsides:
1. At the moment, BMIs are built with the settings used to build Boost
in the first place, which causes a lot of trouble.
2. The installation generates many more binaries than in the headers
world. It is uncertain whether these binaries will be compatible if
build flags change.
3. Some libraries support configuration macros. For instance,
Boost::throw_exception supports BOOST_NO_EXCEPTIONS. This is used by
other Boost libraries to support builds that disable exceptions. In
the current world, the end user defines BOOST_NO_EXCEPTIONS in their
code, includes the library, and everything works. Using a scheme like
the one I suggested requires the user to build all the Boost modules
using BOOST_NO_EXCEPTIONS, which is more involved than what we have
today.

For this reason, it makes sense to consider whether there is a way to
completely avoid the generation and installation of these binaries for
header-only libraries, and follow a model similar to standard modules.
In this ideal model, installing Boost with CMake would only install
the headers and module code for each header-only library, and no
binary artifacts. The consuming project would build both the BMIs and
the objects containing the module initializers with the adequate build
flags, as many times as required by the different targets.

Ideally, the following syntax would work and do what I'm suggesting:

# This does NOT work as of today
add_library(boost_variant2 INTERFACE) # Don't build the library in the
current project
target_sources(boost_variant2 INTERFACE FILE_SET CXX_MODULES FILES
modules/variant2.cxx)
target_include_directories(boost_variant2 INTERFACE include)
target_link_libraries(boost_variant2
    INTERFACE
      Boost::assert
      Boost::config
      Boost::mp11
)

In the ideal world, this wouldn't build any binary library or BMI per
se. When linking to the boost_variant2 target, a BMI and a static
library would be built with the flags required by the consuming
target. Note that this does NOT work as of today [2]. I've raised a
feature request to the CMake team with this suggestion [3], but I
don't think this is happening in the short-term.

Given that this does not work, the other option would be installing
the headers and the module code manually. For instance:

# This is what we have today. Note that we have removed the target_sources
add_library(boost_variant2 INTERFACE)
target_include_directories(boost_variant2 INTERFACE include)
target_link_libraries(boost_variant2
    INTERFACE
      Boost::assert
      Boost::config
      Boost::mp11
)

# Install the module code separately
install(DIRECTORY modules DESTINATION .)

With this setup, it is the user's responsibility to issue the
target_sources calls required to build the boost_variant2 module.
There is also nothing telling CMake that the boost_variant2 module
requires building boost_assert and boost_mp11, though. Thus, it
becomes the user's responsibility to know this dependency chain and
add the relevant target_sources calls. This is what I mean by
"figuring out dependencies manually". While it may not seem that hard
in this case, things get much worse as you study libraries with more
dependencies. I'm the author of Boost.MySQL, and I wouldn't be able to
spell you the current dependency tree without an automated tool to
help.

I hope this helps clarify what I meant.

If I have missed anything obvious, or anyone has ideas on how to make
this better, please let me know.

>
> Thanks,
> Chuanqi
>

Regards,
Ruben.

[1] https://discourse.cmake.org/t/advice-on-c-20-modules-boost/10641/13
[2] https://cmake.org/cmake/help/latest/command/target_sources.html#file-sets
[3] https://discourse.cmake.org/t/distributing-c-20-modules-as-source/13246