[config] First steps towards C++20 modules
Hi, C++20 modules seem to be a good tool to improve compile times and to hide the implementation details of the libraries. I've been looking throught the sources of {fmt} library. Looks like the fmt library approach to C++20 modularization seems to fit Boost quite well. How about adding the following macro to Boost: * BOOST_BEGIN_MODULE_EXPORT - starts the scope of things to export if Boost is build as a C++20 module * BOOST_END_MODULE_EXPORT - ends the scope of things to export if Boost is build as a C++20 module * BOOST_MODULE_EXPORT - for a single entity export if Boost is build as a C++20 module As a result, if Boost is build as a module the above macros are defined in the following way: #define BOOST_MODULE_EXPORT export #define BOOST_BEGIN_MODULE_EXPORT export { #define BOOST_END_MODULE_EXPORT } Helper macro BOOST_NO_CXX23_IMPORT_STD for detecting availability of `import std` also seems useful. Macro BOOST_NO_CXX20_MODULES - for notifying the libraries that compiler does not support modules or Boost modules are not compiled. With all the above macros the Boost libraries could look like: /// header #include <boost/library_that_is_known_to_not_support_modules.hpp> #ifdef BOOST_NO_CXX20_MODULES #include <boost/library_that_supports_modules.hpp> #else import Boost.LibraryThatSupportsModules; #endif #ifdef BOOST_NO_CXX23_IMPORT_STD #include <iostream> #else import std; #endif namespace boost { namespace detail { template <class T> bool try_do(const T&); } BOOST_BEGIN_MODULE_EXPORT template <class T, class From> bool try_lexiacal_convert(const From& f, T& to) { /*impl*/ } template <class T, class From> T lexiacal_cast(const From& f) { /*impl*/ } BOOST_END_MODULE_EXPORT } /// module src file export module Boost.MyLibraryName; #ifdef BOOST_NO_CXX20_MODULES #error ??? #endif #include <boost/my_library.hpp> -- Best regards, Antony Polukhin
Hi, what you're describing is the technique what I coined the term "dual mode support" for in my presentations. I've added it to {fmt} in 2021 to enable the use of the library as both #include and import; MS-STL adopted it for their implementation of the standard library Please take into account that I usually also add something along xxx_ATTACH_TO_GLOBAL_MODULE (in this case xxx = BOOST) to avoid potential ODR violations for projects that use a library either as module or through headers. Otherwise users may end up with conflicting, but apparently *identical* definitions that they cannot decipher the reason for: one definition is attached to the named module, the other to the global module. The compiler will be very upset. Linkers usually work. Boost libraries that intend to support modules *must* also *add* tests that check that the exported entities are * available on the consumer side * instantiation and name lookup works when called from the consumer side Module implementation units have two relevant interfaces, not one. Almost all libraries that claim module capability support, and that I've seen so far, fail to do that. In rare cases I have forks that add this missing piece. Lacking those gives users horrible experiences and modules a bad reputation. On the standard library: *all* C++ implementers of the standard library have agreed to support C++20 modules also for C++20 build modes. So you're not tied to C++23. And lastly, there is BMI compatibility and the story of compiler flags matching. But that's for a different day. Dani Am 06.07.2024 um 14:20 schrieb Antony Polukhin via Boost:
Hi,
C++20 modules seem to be a good tool to improve compile times and to hide the implementation details of the libraries.
I've been looking throught the sources of {fmt} library. Looks like the fmt library approach to C++20 modularization seems to fit Boost quite well.
How about adding the following macro to Boost: * BOOST_BEGIN_MODULE_EXPORT - starts the scope of things to export if Boost is build as a C++20 module * BOOST_END_MODULE_EXPORT - ends the scope of things to export if Boost is build as a C++20 module * BOOST_MODULE_EXPORT - for a single entity export if Boost is build as a C++20 module
As a result, if Boost is build as a module the above macros are defined in the following way:
#define BOOST_MODULE_EXPORT export #define BOOST_BEGIN_MODULE_EXPORT export { #define BOOST_END_MODULE_EXPORT }
Helper macro BOOST_NO_CXX23_IMPORT_STD for detecting availability of `import std` also seems useful.
Macro BOOST_NO_CXX20_MODULES - for notifying the libraries that compiler does not support modules or Boost modules are not compiled.
With all the above macros the Boost libraries could look like:
/// header
#include <boost/library_that_is_known_to_not_support_modules.hpp>
#ifdef BOOST_NO_CXX20_MODULES #include <boost/library_that_supports_modules.hpp> #else import Boost.LibraryThatSupportsModules; #endif
#ifdef BOOST_NO_CXX23_IMPORT_STD #include <iostream> #else import std; #endif
namespace boost { namespace detail { template <class T> bool try_do(const T&); }
BOOST_BEGIN_MODULE_EXPORT
template <class T, class From> bool try_lexiacal_convert(const From& f, T& to) { /*impl*/ }
template <class T, class From> T lexiacal_cast(const From& f) { /*impl*/ }
BOOST_END_MODULE_EXPORT }
/// module src file
export module Boost.MyLibraryName;
#ifdef BOOST_NO_CXX20_MODULES #error ??? #endif
#include <boost/my_library.hpp>
-- PGP/GPG: 2CCB 3ECB 0954 5CD3 B0DB 6AA0 BA03 56A1 2C4638C5
On 07/07/2024 07:56, Daniela Engert via Boost wrote:
Hi,
what you're describing is the technique what I coined the term "dual mode support" for in my presentations. I've added it to {fmt} in 2021 to enable the use of the library as both #include and import; MS-STL adopted it for their implementation of the standard library
Please take into account that I usually also add something along xxx_ATTACH_TO_GLOBAL_MODULE (in this case xxx = BOOST) to avoid potential ODR violations for projects that use a library either as module or through headers. Otherwise users may end up with conflicting, but apparently *identical* definitions that they cannot decipher the reason for: one definition is attached to the named module, the other to the global module. The compiler will be very upset. Linkers usually work. Sorry, don't quite follow, can you explain?
Boost libraries that intend to support modules *must* also *add* tests that check that the exported entities are * available on the consumer side * instantiation and name lookup works when called from the consumer side Absolutely. Complete lack of build/tooling support for modules is currently a showstopper here. When experimenting with a Regex module I basically had to resort to the command line.
Module implementation units have two relevant interfaces, not one. Also don't follow, can you explain? Almost all libraries that claim module capability support, and that I've seen so far, fail to do that. In rare cases I have forks that add this missing piece. Lacking those gives users horrible experiences and modules a bad reputation.
On the standard library: *all* C++ implementers of the standard library have agreed to support C++20 modules also for C++20 build modes. So you're not tied to C++23. Good. But if the module uses "import std;" and something somewhere else #includes <iostream> my experience is that everything breaks. It's not supposed to, but it does. I must test the latest MSVC release though.
And lastly, there is BMI compatibility and the story of compiler flags matching. But that's for a different day.
Right, there is no possibility of a build-and-install step, each project must build the modules they are using from source using options that exactly match their project. It's not hard, but it is a barrier to adoption. John.
Am 07.07.2024 um 12:25 schrieb John Maddock via Boost:
On 07/07/2024 07:56, Daniela Engert via Boost wrote:
Hi,
what you're describing is the technique what I coined the term "dual mode support" for in my presentations. I've added it to {fmt} in 2021 to enable the use of the library as both #include and import; MS-STL adopted it for their implementation of the standard library
Please take into account that I usually also add something along xxx_ATTACH_TO_GLOBAL_MODULE (in this case xxx = BOOST) to avoid potential ODR violations for projects that use a library either as module or through headers. Otherwise users may end up with conflicting, but apparently *identical* definitions that they cannot decipher the reason for: one definition is attached to the named module, the other to the global module. The compiler will be very upset. Linkers usually work. Sorry, don't quite follow, can you explain?
Ok, a simple example. Consider this definition in an internal header. This acts just as a stand-in for ex. definitions in a Boost library: // internal.h namespace lib { BOOST_MODULE_EXPORT struct S { int s; }; } Use it as a traditional include (the public interface): // lib.h #pragma once #ifndef BOOST_MODULE_EXPORT # define BOOST_MODULE_EXPORT #endif #include "internal.h" Use it as a C++20 module (the public interface): // module lib export module lib; #define BOOST_MODULE_EXPORT export #include "internal.h" 3rd-party TUs: // a.h #include <lib.h> class A { lib::S s; }; // b.h import lib; class B { lib::S s; }; An unsuspecting consumer brings both 3rd-party TUs into the same TU. The ordering is irrelevant for the purpose of reasoning: // c.cpp #include <a.h> #include <b.h> struct C { A a; B b; }; Now you end up with *two* definitions of the same entity S: 1. one attached to the global module (through lib.h) 2. one attached to the named module lib (through module lib) C++ programmers can't discern them by language means, compilers and linkers do. In the first case it is entity S, in the second it is entity S@lib. From the compiler perspective, they have the same name but different attachment - oops. From the linker perspective they have different symbols and/or other properties in the object files. The latter would be fine with it, the former will teach you in no uncertain language that this is a no-no. User will scratch their head about why they are told that an entity is the same but conflicting. In general, export entities *either* through a header interface *or* through a module interface, but never both. Compilers take advantage of different attachments and lazily load module interfaces only when really needed. If you as an author want to support mixing of both traditional #includes and module imports, then you'd better make sure that entity S is attached to the global module in both cases. Like along these lines: // module lib export module lib; #define BOOST_MODULE_EXPORT export #ifdef BOOST_xxx_ATTACHED_TO_GLOBAL_MODULE extern "C++" { #endif #include "internal.h" #ifdef BOOST_xxx_ATTACHED_TO_GLOBAL_MODULE } #endif This way, name attachment is a module compile-time option. I use this f.e. in my Asio module for a long time.
Boost libraries that intend to support modules *must* also *add* tests that check that the exported entities are * available on the consumer side * instantiation and name lookup works when called from the consumer side
Absolutely. Complete lack of build/tooling support for modules is currently a showstopper here. When experimenting with a Regex module I basically had to resort to the command line.
I'm surprised. I've given talks on modules since 2019 then they became kind of usable in MSVC, use them in production since the beginning of 2022, gave a CppCon keynote about a non-trivial application using only modules and the modularized standard library in 2022 (using MSBuild), and gave talks how to use them cross-platform with CMake in 2023. If you happen to have gcc in mind, then you're in a tough spot, though
Module implementation units have two relevant interfaces, not one.
Also don't follow, can you explain?
The implementer-facing interface is excercised while *compiling* the module interface unit. This means, the module can be compiled, name lookup of all non-dependent entities and 1st-phase name lookup of dependent entities succeeds, and overload resolution takes place as much as name lookup allows. The user-facing interface is excercised while *using* a module in a different TU. The exported entities must be found by name lookup, ADL and 2nd-phase name lookup of module-internal entities must succeed for instantiations of exported templates. This includes lookup in non-exported namespaces buried within the module performed in a non-module TU! Therefore, the 2nd (mostly forgotten) interface is the crucial one!
Almost all libraries that claim module capability support, and that I've seen so far, fail to do that. In rare cases I have forks that add this missing piece. Lacking those gives users horrible experiences and modules a bad reputation.
On the standard library: *all* C++ implementers of the standard library have agreed to support C++20 modules also for C++20 build modes. So you're not tied to C++23. Good. But if the module uses "import std;" and something somewhere else #includes <iostream> my experience is that everything breaks. It's not supposed to, but it does. I must test the latest MSVC release though.
And lastly, there is BMI compatibility and the story of compiler flags matching. But that's for a different day.
Right, there is no possibility of a build-and-install step, each project must build the modules they are using from source using options that exactly match their project. It's not hard, but it is a barrier to adoption.
John.
Thanks, Dani -- PGP/GPG: 2CCB 3ECB 0954 5CD3 B0DB 6AA0 BA03 56A1 2C4638C5
воскресенье, 7 июля 2024 г. пользователь Daniela Engert via Boost < boost@lists.boost.org> написал: <...>
If you as an author want to support mixing of both traditional #includes and module imports, then you'd better make sure that entity S is attached to the global module in both cases. Like along these lines:
// module lib export module lib; #define BOOST_MODULE_EXPORT export #ifdef BOOST_xxx_ATTACHED_TO_GLOBAL_MODULE extern "C++" { #endif #include "internal.h" #ifdef BOOST_xxx_ATTACHED_TO_GLOBAL_MODULE } #endif
This way, name attachment is a module compile-time option.
I use this f.e. in my Asio module for a long time.
Does it make sense to go another way and make the header auto-import self as a module if module exists: // lib.h #pragma once #ifndef BOOST_NO_CXX20_MODULES import lib; #else # ifndef BOOST_MODULE_EXPORT # define BOOST_MODULE_EXPORT # endif # include "internal.h" #endif ? -- Best regards, Antony Polukhin
Am 08.07.2024 um 08:32 schrieb Antony Polukhin:
воскресенье, 7 июля 2024 г. пользователь Daniela Engert via Boost <boost@lists.boost.org> написал: <...>
If you as an author want to support mixing of both traditional #includes and module imports, then you'd better make sure that entity S is attached to the global module in both cases. Like along these lines:
// module lib export module lib; #define BOOST_MODULE_EXPORT export #ifdef BOOST_xxx_ATTACHED_TO_GLOBAL_MODULE extern "C++" { #endif #include "internal.h" #ifdef BOOST_xxx_ATTACHED_TO_GLOBAL_MODULE } #endif
This way, name attachment is a module compile-time option.
I use this f.e. in my Asio module for a long time.
Does it make sense to go another way and make the header auto-import self as a module if module exists:
// lib.h #pragma once #ifndef BOOST_NO_CXX20_MODULES import lib; #else # ifndef BOOST_MODULE_EXPORT # define BOOST_MODULE_EXPORT # endif # include "internal.h" #endif
I might be dense, but this automatically switches between #include and import on the consumer side of a library. What you cite is one possible approach to selectable name attachment on the producer side. Dani
Daniela Engert via Boost <boost@lists.boost.org> writes:
// module lib export module lib; #define BOOST_MODULE_EXPORT export #ifdef BOOST_xxx_ATTACHED_TO_GLOBAL_MODULE extern "C++" { #endif #include "internal.h" #ifdef BOOST_xxx_ATTACHED_TO_GLOBAL_MODULE } #endif
Are you aware of any substantial codebases that successfully use this approach? Specifically, this idea of having a header that implements both the header and module interfaces with the same source code and which is then included in the module purview wrapped in extern "C++"? Feels like there bound to be gotchas. For example, that header likely includes other headers (is it ok to wrap a standard library headers in extern "C++"?) and maybe even imports some modules (potentially without even knowing that, for instance, via a third-party header). BTW, we are in the process of considering what mechanisms we should have in build2 for mixing modules/headers packages and I wrote some thoughts on the possible approaches to implement such dual header/modules support, in case anyone is interested: https://github.com/build2/build2/issues/413#issuecomment-2272932212
Am 07.08.2024 um 11:29 schrieb Boris Kolpackov:
Daniela Engert via Boost<boost@lists.boost.org> writes:
// module lib export module lib; #define BOOST_MODULE_EXPORT export #ifdef BOOST_xxx_ATTACHED_TO_GLOBAL_MODULE extern "C++" { #endif #include "internal.h" #ifdef BOOST_xxx_ATTACHED_TO_GLOBAL_MODULE } #endif Are you aware of any substantial codebases that successfully use this approach?
Does Microsoft's STL implementation count as "substantial"? BTW: do you know of *any* substantial non-STL codebase that uses modules in any meaningful way? Besides Microsoft Office, of course.
Specifically, this idea of having a header that implements both the header and module interfaces with the same source code and which is then included in the module purview wrapped in extern "C++"? We do in our company. We use a mixture of headers, pure modules (made from scratch), modules with headers wrapped by 'extern "C++"' like you mention, modules with headers exporting (parts) of their interface by exported using-declarations, and combinations of that, in the codebase driving a huge machine that is developed since 2021 and has precursors (i.e. builds from code) starting from 2007. 3rd-party libraries (usually modules wrapping headers) are part of that. This codebase is in the 1M LOC range. Feels like there bound to be gotchas.
Right: Clang is sometimes failing to merge definitions in the global module coming from both headers and modules. (documented here: https://clang.llvm.org/docs/StandardCPlusPlusModules.html#known-issues)
For example, that header likely includes other headers (is it ok to wrap a standard library headers in extern "C++"?)
It is, see [std.modules] and in particular [std.modules]/4. Microsoft does. I did in my CppCon 2022 keynote and later when there were no working implementer-supplied modularized standard C++ library modules available yet (e.g. ms-stl and libstdc++).
and maybe even imports some modules (potentially without even knowing that, for instance, via a third-party header).
If you can't control that, you *need* to attach affected pieces of your code to the global module - with all its drawbacks like increased compile times. The biggest problem today in my opinion are alleged modules that are missing meaningful tests. By that I mean modules which don't dare to test their usability from the consumer perspective. Just a week ago, I gave the 'argparse' library a shot and reported by findings in this thread here: https://hachyderm.io/deck/@DanielaKEngert/112876206120670198. The code - in particular the tests - is here: https://github.com/DanielaE/argparse/tree/module It uses the 'extern "C++"' technique that I invented for the {fmt} library early 2021. Clang also (quite recently) documents it as one of two options to modularize existing libraries (see: https://clang.llvm.org/docs/StandardCPlusPlusModules.html#transitioning-to-m...). It is more powerful - but also more involved - than the 'export using' technique used by e.g. the modularized libc++ standard library.
BTW, we are in the process of considering what mechanisms we should have in build2 for mixing modules/headers packages and I wrote some thoughts on the possible approaches to implement such dual header/modules support, in case anyone is interested:
https://github.com/build2/build2/issues/413#issuecomment-2272932212
Thanks, Dani -- PGP/GPG: 2CCB 3ECB 0954 5CD3 B0DB 6AA0 BA03 56A1 2C4638C5
Thanks for the insight. Some comments/questions below. Daniela Engert <dani@ngrt.de> writes:
Does Microsoft's STL implementation count as "substantial"?
I assumed they use the export-using approach, but you are right, they include the standard library headers in the module purview. Though they don't wrap them in one extern "C++" block but rather modified each header to explicitly `export extern "C++"` each name (via the _EXPORT_STD macro). So it's not quite the same -- they have a lot more precision over what ends up with extern "C++" (as well as what gets exported).
BTW: do you know of *any* substantial non-STL codebase that uses modules in any meaningful way? Besides Microsoft Office, of course.
My go-to test is the async_simple project that was converted to modules by Chuanqi Xu: https://github.com/alibaba/async_simple/tree/CXX20Modules/ I added build2 support and successfully built it using the standard library modules with Clang/libc++. That work is here: https://github.com/boris-kolpackov/async_simple/tree/CXX20Modules-build2 Other than that, I am aware of a couple of build2 users that are using modules (including import std) in green-field projects, but those are not public.
We do in our company. We use a mixture of headers, pure modules (made from scratch), modules with headers wrapped by 'extern "C++"' like you mention, modules with headers exporting (parts) of their interface by exported using-declarations, and combinations of that, in the codebase driving a huge machine that is developed since 2021 and has precursors (i.e. builds from code) starting from 2007. 3rd-party libraries (usually modules wrapping headers) are part of that. This codebase is in the 1M LOC range.
That's encouraging, thanks for sharing. Is this MSVC-only or do you also build with Clang?
Feels like there bound to be gotchas.
Right: [...]
Another thing I am fuzzy about that I forgot to mention is the implementation unit. Feel like the most sensible way is to build it off the header (rather than the module). The module interface also produces an object file. Are there no issues combining these two object files in the same build (duplicate symbols, etc)?
Am 08.08.2024 um 08:11 schrieb Boris Kolpackov:
Daniela Engert<dani@ngrt.de> writes:
Does Microsoft's STL implementation count as "substantial"? I assumed they use the export-using approach, but you are right, they include the standard library headers in the module purview. Though they don't wrap them in one extern "C++" block but rather modified each header to explicitly `export extern "C++"` each name (via the _EXPORT_STD macro). So it's not quite the same -- they have a lot more precision over what ends up with extern "C++" (as well as what gets exported).
Language-wise either form is the same [dcl.link]/2 You're right. I remember when Stephan added the language linkage specification to exported entities. I think, he somewhere called it a "hack" to work around the compiler quirks that MSVC was experiencing when it came to definition merging of standard library entities. Fixing the (unfortunately still) outstanding bugs requires a decent amount of developer love (mostly by Cameron), more love than available these days.
BTW: do you know of *any* substantial non-STL codebase that uses modules in any meaningful way? Besides Microsoft Office, of course. My go-to test is the async_simple project that was converted to modules by Chuanqi Xu:
https://github.com/alibaba/async_simple/tree/CXX20Modules/
I added build2 support and successfully built it using the standard library modules with Clang/libc++. That work is here:
https://github.com/boris-kolpackov/async_simple/tree/CXX20Modules-build2
I'm aware of this repo. Giving it another shot, I tried it in multiple environments: * Windows MSYS2 (Clang 18.1.8 + libc++): configure ok, build fails quite early due to some platform-related issue * Ubuntu 23.04 (Clang 18.1.3 + libc++): configure ok, build fails, the Ubuntu upgrade has killed some Clang tools again, sigh * Windows (MSVC 17.10): configure fail * Windows MSYS2 (GCC 14.2 + libstdc++): configure ok, build fails due to Clang-isms I have no inclination to clean up this mess. The 3rd-party dependency 'Asio' is highly platform-malleable, I'm using it in all of my presentations since 2022 - as a module on *all* platforms above (and more). TBH: I don't see said repo as a good example for something remotely module-ish. I wouldn't go to the stage and present it as cross-platform at all. We can do much better.
Other than that, I am aware of a couple of build2 users that are using modules (including import std) in green-field projects, but those are not public. Very unfortunate, we could learn a lot. We do in our company. We use a mixture of headers, pure modules (made from scratch), modules with headers wrapped by 'extern "C++"' like you mention, modules with headers exporting (parts) of their interface by exported using-declarations, and combinations of that, in the codebase driving a huge machine that is developed since 2021 and has precursors (i.e. builds from code) starting from 2007. 3rd-party libraries (usually modules wrapping headers) are part of that. This codebase is in the 1M LOC range. That's encouraging, thanks for sharing. Is this MSVC-only or do you also build with Clang?
Our customers demand Windows, and Windows only. I use Clang in some of the tooling, but not for the core build process. MSVC was the only compiler that wouldn't choke on our code at around 2021/2022 and we stick with it 'til today.
Feels like there bound to be gotchas. Right: [...] Another thing I am fuzzy about that I forgot to mention is the implementation unit. Feel like the most sensible way is to build it off the header (rather than the module). The module interface also produces an object file. Are there no issues combining these two object files in the same build (duplicate symbols, etc)?
It shouldn't, unless there are still unresolved issues. I had a really hard time with Clang 16 in the demo code from from my 2022/2023 presentations. I could barely work around them and had to change the Asio module to fix the exception issue. Clang 17 is much better, but still needs workarounds. I didn't test Clang 18 so far because I'm occupied with other stuff and a different presentation. May be, I'll find the time to recheck again on all platforms and see if all the 3rd-party libraries didn't evolve too much. Regarding your specific question: the current Clang documentation still mentions a few implementation issues with object files that require workarounds. May be you're seeing that kind-of-bug in Asio where it is exposing TU-local entities: that's discouraged in general and outright ill-formed in modules (Clang hardly detects this!). This needs fixing! You can't just wrap existing Asio and call it a day, despite what some people may think. Thanks Dani -- PGP/GPG: 2CCB 3ECB 0954 5CD3 B0DB 6AA0 BA03 56A1 2C4638C5
On 06/07/2024 13:20, Antony Polukhin via Boost wrote:
Hi,
C++20 modules seem to be a good tool to improve compile times and to hide the implementation details of the libraries.
I've been looking throught the sources of {fmt} library. Looks like the fmt library approach to C++20 modularization seems to fit Boost quite well.
How about adding the following macro to Boost: * BOOST_BEGIN_MODULE_EXPORT - starts the scope of things to export if Boost is build as a C++20 module * BOOST_END_MODULE_EXPORT - ends the scope of things to export if Boost is build as a C++20 module * BOOST_MODULE_EXPORT - for a single entity export if Boost is build as a C++20 module
I can't see all of Boost as one module - honestly I think at present that's a terrible idea - even if it were actually possible with current compiler technology. IMO this is more likely to be done on a library by library basis, with the module source files defining BOOST_MYLIB_EXPORT (or whatever). Note that it's pretty trivial to define higher level groupings of modules: export module stuff_from_boost; export import boost.A; export import boost.B; // etc We do need to add BOOST_NO_CXX20_MODULES and BOOST_NO_CXX23_STD_MODULE though (the latter in particular is supported by msvc even though it does not define the relevant std feature test macro). And yes, Boost should experiment with some bleeding edge once more. John.
Hi,
C++20 modules seem to be a good tool to improve compile times and to hide the implementation details of the libraries.
I've been looking throught the sources of {fmt} library. Looks like the fmt library approach to C++20 modularization seems to fit Boost quite well.
I performed an analysis for Boost some months ago. Here is what I found out: - https://anarthal.github.io/cppblog/modules - https://anarthal.github.io/cppblog/modules2 And here's the discussion we had on this ML: - https://lists.boost.org/Archives/boost/2024/04/256460.php - https://lists.boost.org/Archives/boost/2024/04/256597.php TL;DR: - Modules help build times when parsing dominates. Clean builds don't gain as much as you'd expect, but re-builds are much faster. - Mixing #include <std_header> and import std causes errors on most compilers at the moment (although it should not). We were in contact with MSVC devs and the fix may take time. - Distributing modules to users so they can consume them with CMake isn't trivial. In this sense, Boost is different to fmtlib in that a) most libs are header-only and b) we have a deep dependency tree (where fmtlib is a single library). - Some core maintainers (including Peter Dimov and Rene Rivera) manifested that the amount of work required to enable modules for core libraries wasn't worth the effort. Together with point 2 above, this limits the number of libraries that can be modularized to only the ones without dependencies (e.g. Boost.Decimal). I think the biggest challenge in distributing Boost as a module is the CMake consumption part. Without solving this, we won't get anywhere. I'm happy to hear your opinion. Regards, Ruben.
понедельник, 8 июля 2024 г. пользователь Ruben Perez < rubenperez038@gmail.com> написал: <...>
I'm happy to hear your opinion.
My opinion is that I have not enough experience with modules yet :) To fix that I'll try to modularise Boost.PFR in a few months. Many thanks for the links, especially with CMake scripts for modules -- Best regards, Antony Polukhin
ср, 10 июл. 2024 г. в 00:01, Antony Polukhin <antoshkka@gmail.com>:
понедельник, 8 июля 2024 г. пользователь Ruben Perez <rubenperez038@gmail.com> написал: <...>
I'm happy to hear your opinion.
My opinion is that I have not enough experience with modules yet :)
To fix that I'll try to modularise Boost.PFR in a few months. Many thanks for the links, especially with CMake scripts for modules
Here's my first attempt https://github.com/boostorg/pfr/pull/177/files I've only tested it on clang-15. Looks like the compiler has issues with mixing global module fragment and includes (just as was noted above), so I've ended up with a toy sample that uses only std::puts. And it works! The compilation time change is qute impressive: $ time clang++-15 -std=c++20 -I /data/code/boost/ example/motivating_example0.cpp real 0m2,051s user 0m1,844s sys 0m0,144s $ time clang++-15 -std=c++20 -fmodule-file=pfr.pcm pfr.pcm example/module_usage.cpp real 0m0,375s user 0m0,249s sys 0m0,107s No idea how to deal with dependencies between diffrent Boost libraries, so I'd probably try to modularise Boost.Core + Boost.TypeIndex to get the idea. -- Best regards, Antony Polukhin
Am 12.08.2024 um 19:38 schrieb Antony Polukhin via Boost:
ср, 10 июл. 2024 г. в 00:01, Antony Polukhin<antoshkka@gmail.com>:
понедельник, 8 июля 2024 г. пользователь Ruben Perez<rubenperez038@gmail.com> написал: <...>
I'm happy to hear your opinion. My opinion is that I have not enough experience with modules yet :)
To fix that I'll try to modularise Boost.PFR in a few months. Many thanks for the links, especially with CMake scripts for modules Here's my first attempthttps://github.com/boostorg/pfr/pull/177/files
I've only tested it on clang-15. Looks like the compiler has issues with mixing global module fragment and includes (just as was noted above), so I've ended up with a toy sample that uses only std::puts. And it works!
The compilation time change is qute impressive:
$ time clang++-15 -std=c++20 -I /data/code/boost/ example/motivating_example0.cpp real 0m2,051s user 0m1,844s sys 0m0,144s
$ time clang++-15 -std=c++20 -fmodule-file=pfr.pcm pfr.pcm example/module_usage.cpp real 0m0,375s user 0m0,249s sys 0m0,107s
No idea how to deal with dependencies between diffrent Boost libraries, so I'd probably try to modularise Boost.Core + Boost.TypeIndex to get the idea.
Cool! Please be aware that Clang-15 is riddled with module-related bugs (according to one of the compiler devlopers I've been speaking to in Varna last year). With Clang-16, this is still true, Clang-17 is quite usable if you know how to deal with the ABI problems. And yet, Clang-18 still has troubles with merging definitions coming from multiple TUs. Dani -- PGP/GPG: 2CCB 3ECB 0954 5CD3 B0DB 6AA0 BA03 56A1 2C4638C5
participants (5)
-
Antony Polukhin -
Boris Kolpackov -
Daniela Engert -
John Maddock -
Ruben Perez